Make learning happen
CREATING A MIXED REALITY SOLUTION FOR POOR STUDENTS ENGAGEMENT USING THE DESIGN THINKING PROCESS
Empathise /
Learning about the end users of the solution involved having class discussions around the challenges as a student. In this rare occasion where we are both the designer and user, we were able to identify the difficulty to stay interested and remember class content. Students are not receiving educational stimulations that actively replicate the respective working environment within their intended industry. This portfolio will use the health care industry as an example.
Define /
We aim to resolve the problem of poor student engagement and memory retention in the classroom by making an application to facilitate immersive quizzes.
The goal is to develop a solution that provides a platform for students to complete quizzes in Mixed Reality. The solution should be an immersive and interactive learning environment that engages students' operational skills, in preparation for their working lives. This will be achieved through appropriate UX design so that the application is instinctual within a classroom setting. The transformation of theory application into a practical activity, increases student engagement and memory retention.
Ideate /
- Develop an app that provides a platform for students to complete quizzes in Mixed Reality.
- Create an application that can integrate with existing university infrastructure e.g. Canvas - student scores (data created from app) automatically populate as Canvas grades (data stored). Our scope includes:
Non-marked quizzes
Marked quizzes
Scoreboards/leaderboards
Reviewing answers.
Non-marked quizzes
Marked quizzes
Scoreboards/leaderboards
Reviewing answers.
- Create an immersive and interactive learning environment that engages students' operational skills, in preparation for their working lives. This will be achieved through appropriate UX design so that the application is easy to use and adopt within a classroom.
- Use 360 degree spacial sound to play the quiz questions aloud. The voice originate from a virtual instructor in the mixed reality world.
- Include a legend to track the question number, score, repeat question (in text) and time remaining.
Data needed /
Required data from University of Auckland:
- Student Data (e.g. student ID, enrolment information)
- Course Syllabus information
- Quiz structure for new platform
- Quiz entity projections (code foundations)
- Quiz answers (input)
- Quiz Questions (input)
- Scoreboard data
Aligning the solution with the SAMR model /
Redefinition: Further development of the quiz allows for more than one student to participate in a quiz. The group quiz function allows for a collaborative learning space whereby the students rely on each other to perfect the score – mimicking the work space dynamic of the real world. Previously inconceivable, the group quiz allows students to work collectively on the same quiz, leveraging off each other’s strengths and carrying out problem solving in a realistic situation. The technology enhances the involvement of sound in the quiz process and having the questions be read aloud with spacial sound design, further replicates a real world surgical room. This feature increases the immersion of the students and reproduces the noise and direction of voices and sounds in a practical working area.
Modification: The traditional quiz format of the existing Canvas infrastructure is significantly redesigned with the implementation of mixed reality. The 2D interaction is translated into a 3D model, changing significant interactive functions of the process. Instead of using a mouse as a curser and initiating an event with a click, the user will now use ‘gaze’ as well as ‘air tap’ to catalyse activity. Functions such as sample quiz questions will expose the students to the new technology and increase their adaptability to the new format. The sample quiz questions also allow the students to test the functional outcomes of their gestures and interactions e.g. what happens on screen when air tap is initiated.
Augmentation: Functional ability is improved by the interactive and immerse properties of virtual reality. The quiz is substituted with mixed reality advantages including the way of selecting something on-screen and the way information is delivered aurally, transforming the learning experience and memory retainment of the students. Interaction of the body, the mind and the quiz along with the immersion of spatial sound design and 3D projections, improves the quality of education and the preparation for a real world event. Aspects in the real world are accounted for, to teach students a practical instance of their degree.
Substitution: Actions such as checking scoreboards / leaderboards have no function change and at this stage are not the primary focus of the initiative. Therefore, the technology will substitute the function and will not improve it. Marking of the quizzes by the lectures also remains within the same functional scope of the original Canvas abilities. Effort is concentrated on improving the quiz experience of the students and less on the backend marking of the instructor at this stage.
Use cases /
Use case explanation /
Check Dashboard – Assuming that the student has already logged in into their profile, they will have the chance to check their dashboard which show a selection of modes that can be chosen. Additionally, the dashboard will also remind the student if there is a mandatory individual/team quiz due.
Includes: Choose Mode – Modes include practice quiz, quiz and explore.
Complete Practice Quiz - The practice quiz mode allows students to complete a quiz without a timer and without recorded results. This can be done individually or in teams.
Complete Quiz – The quiz mode allows students to complete a quiz in a timed environment where the results are recorded. This can be done individually or in teams.
Check Scoreboard – Students can check the scoreboard to observe where they rank in the class amongst fellow students. The scoreboard is ranked in terms of point whereby one point comes from one correct answer to a quiz question. This use case acts to drive student motivation for grade incentives given by the tutor.
Check Answer – Students can check the correct answers to their incorrectly answered questions. This use case acts to improve the students’ performance for future quizzes.
Complete Explore Mode – Explore mode allows students to freely educate themselves with 3D diagrams and accompanied voice descriptions on relevant syllabus material. There are no quiz questions, timer or score. The purpose of this use case is to create a mode where students can learn interactively with no time or score pressure.
Observe Score Board – Lecturers can observe the score board to see where the class average is. Thus he/she can determine if the content is at a suitable level. Lecturers can also see individual progress so that they can further give the student assistance if the student is struggling or give them grade incentives if the student is excelling. Team score boards are available so that the lecturer can identify if students within a team are not participating enough.
User stories /
User: Student
As a student, I want to check my dashboard so that I have the freedom of choice over which educational activities I interact with and also be reminded of any due quizzes that must be completed.
As a student, I want to complete quizzes so that I am constantly learning at a gradual and incremental pace. I can also make sure I understand each quiz topic thoroughly before I move onto the next quiz topic.
As a student, I want to check the score board so that I am aware of my standard of understanding and level of effort compared to the class. This will also motivate me to be at the top of the scoreboard as there is a grade incentive.
As a student, I want to check the answers so that I am aware of my mistakes and how to correct them in the future.
As a student, I want to complete explore mode so that I can freely interact with the learning materials in a stress-free environment and learn at my own pace, whilst getting familiar and enjoying the interactive technology.
As a student, I want to work in a team so that I can experience a real life world reflection of collaboration in the work space.
As a student, I want to work individually so that I can focus on content rather than context.
User: Lecturer
As a lecturer, I want to observe the score board so that I can determine the class average. According to this, I will be able to know if the class material is at a suitable level. I am also able to give students in the bottom half of the score board additional assistance and those at the top half, grade incentives. Looking at team scoreboards, I can identify if students within a team are not participating enough.
Product design and UX decisions /
Use case: Check Dashboard
Uncompleted quiz notifications will flash and pop out towards the individual wearing the HoloLens to catch their attention. This utilises the interactivity of the technology and breaks into the 3D functionality. This is an activity focused design as it emphasises a task to be done. Because it is assumed that the students are fairly new to the platform, a clear call-to-action must be design, instead of relying on their own experience.
Use case: Complete Individual Practice Quiz / Complete Individual Quiz
Individual quizzes consist of students interacting with 3D objects in front of them, whether it is a human body or an engineering system. The students are able to strip back layers of the projects e.g. see the bones / circulation in a human body or see the pipes / construction inside a building. This is a user-focused design as it looks into the goals, needs and the context of the users. The design of the projections aids the learning and stimulates the traditional educational process.
Use case: Explore mode
Explore mode allows the student to interact with projections from the HoloLens in a stress-free environment which promotes self-learning. This is reference based design following the innovations of Galaxy Explorer. The success following its release proved the creativity and effectiveness of their designs. It creates an immersive experience for the students and disguises learning with the excitement and detail of the subjects onscreen.
Storyboard of prototype solution in Unity /
Sample code of Azure connected Prototype in Unity /
Evidence of Toolkit, Cloud Service and Design Guideline evidence used in storyboard /
Evidence of Spatial Sound in Unity prototype /
Spacial Sound examples from storyboard /
Systems diagram /
Evidence of Design Thinking iteration /
Initially, the idea of a quiz prototype that surrounded the health science sector was to be executed. The quiz would be set in a surgical room whereby the students’ aim was to locate veins and arteries in the heart in order to boost the patient’s health, as shown by the score.
After further discussion with instructor and deeper investigation into the code of the quiz, we made the decision to prototype our quiz in the context of the spacemen case. This was because we already had the functioning code on hand. This allowed us to allocate more time on the design principles as well as further features of the quiz, such as sound design and the tag-a-long menu. Ultimately we are creating iterations on the prototype phase of the design thinking as have already demonstrated ‘empathise’, ‘define’ and ‘ideate’ with the research, problem definition and constant ideas. We aim to prototype the health science case in the future iterations.
We plan to provide the spacemen case for the first iteration and to then produce a surgical room case for the second and third iteration. We will implement the sound design with the spacemen case as a test of the feature for its effectiveness. We will not be making changes to the ‘Further Development’ section, as we firmly believe that the surgical room case will benefit from the ideas and additional features in future development.
After familiarizing ourselves with the code of the Spacemen and its respective functions, we have progressed to using the 3D model of a heart. This was the original idea of our project and since the first iteration, we have evolved our code and Unity to fit the health science industry.
We have implemented a 3D model instructor to be in the scene to give user a sense of scale as well as a surgical room background to imply an appropriate environment.
We divided the surgical room background into thirds to create walls, mimicking a 3D environment for the user when they are immersed into the virtual reality. Users will be able to look from left to right and still stay immersed within the surgical room. Rather than focusing purely on the spatial sound aspect of the project, we have improved the visual scenery of the project hierarchy to enhance immersion to mimic the working environment for a believable setting.
Upon further review, the walls we had created did not offer the sense of immersion we had aimed to build. We imported an environment from the Unity Store to amplify our project and provide a complete 3D experience.
ERD /
Further development ideas /
Further development lies in the implementation of group quizzes;
- Quiz to be completed in groups of three students
- Each student will wear a mixed reality device
- The mixed reality devices will be interconnected so that each student can see their team players (either as digital avatars or through augmented reality)
- The devices will ask the players individually to answer questions
> “player #1 please air tap the left forearm”
- The device will also ask questions that require the team to work together
> “player #1 please gaze commit the torso so that player #2 can select the chest. Player #3 please correctly identify the body part on the left of the chest.”
- All the players will have the same spacial mapping and tag-a-long legend
> The legend will show the time remaining, repeat question option, quiz progress and the score.
The group quiz simulation mimics the collaborative nature of a real life working procedure. The students must work together to increase the score. By expanding the scope of the quiz, it meets an extended problem of reproducing a multi-player environment and experience. This further development leverages the use of technology and provides a redefined solution, according to the SAMR Model alignment. This transformative task will increase students’ understanding and learning towards a more practical and previously inconceivable process of collaboration.
The collaborative model allows students to socialise in a classroom setting, tackle problem solving with peer support and feedback as well as have the power to define roles and responsibility within their team. The function additionally improves communication skills as well as numerous soft skills that are admirable in a real world working situation.
The culture of learning is enhanced within the classroom and students are more likely to feel comfortable in sharing their opinions. Creating a unified space in the classroom in a valuable asset as it provides an opportunity for students to better their performance through encouragement of peers. Collaborative learning fosters innovation as well as effort, proving a worthwhile development for the quiz.
