Findings from User Testing
Big Aha’s
- Explaining is challenging for people, no matter their age or background.
- Explaining is not an inherent skill; it is something one has to learn.
- Scaffolds need to be provided to facilitate learning to explain.
- Understanding the big picture is important in creating a clear explanation.
User Testing
User Test Round 1:
Grade: 2nd Grade
Number: 14
Prototype: Play Pad
Guiding Questions:
Results/Conclusions:
Number: 14
Prototype: Play Pad
Guiding Questions:
- How would students use sound dots to explain a math problem?
- When given complete freedom how did the students use the sound dots?
Results/Conclusions:
- User interface took time for students to understand how to use
- Students completed Math problem in head and had to be prompted to leave sound dots
- Students’ explanations were short, concrete and incomplete
- Students left sounds that were directly correlated to picture. (e.g. animal sounds, comic strips, etc...)
User Test Round 2:
User Test 2:
Grade: 3rd Grade
Number: 1
Prototype: Process Pad Version 2 (First set of activities)
Guiding Questions:
Results/Conclusions:
Grade: 3rd Grade
Number: 1
Prototype: Process Pad Version 2 (First set of activities)
Guiding Questions:
- Do the activities scaffold the use of Process Pad?
- Do the activities provide scaffolding to help the users’ explanations?
- When given complete freedom how did the students use the sound dots?
Results/Conclusions:
- Performance was better with more concrete tasks (e.g. labeling, putting a comic strip in order).
- Transitioning from concrete tasks to explaining a math problem was difficult.
- Difficulty in using the sound dots increased on more complex math problems.
User Test Round 3:
User Test 3:
Grade: Graduate Students
Number: 3
Prototype: Process Pad Version 3 (More intermediate scaffolded activities)
Guiding Questions:
Results/Conclusions:
UI Changes
Grade: Graduate Students
Number: 3
Prototype: Process Pad Version 3 (More intermediate scaffolded activities)
Guiding Questions:
- What is confusing about the user interface?
- What does an expert explanation look like?
Results/Conclusions:
- Graduate students’ explanation abilities varied drastically
- There is no single expert explanation.
- The graduate students had more difficulty explaining some of activities than the younger students.
UI Changes
- The close menu button should stop all recordings
- The picture and video “dots” were not used consistently
User Test Round 4:
User Test 4:
Grade: 5th Grade
Number: 1
Prototype: Process Pad Version 3 (More intermediate scaffolded activities)
Guiding Questions:
Results/Conclusions:
Grade: 5th Grade
Number: 1
Prototype: Process Pad Version 3 (More intermediate scaffolded activities)
Guiding Questions:
- Do the new activities help the user transition from explaining concrete tasks to more abstract tasks (e.g. math problems)?
- How does an older student perform on explanations?
- What is confusing about the user interface?
Results/Conclusions:
- There was effective use of sound “dots” and clear, detailed explanations of simple algebra problems.
- One user began stating each step he took as a question
- e.g. 5 + a = 12 What plus 5 equals 12?
- When explaining math problems only sound was used, and not picture or video.