Luna was an ongoing research project at CMU’s Human Computer Interaction Institute on how a teacher dashboard could assist middle school teachers’ use of an adaptive tutor. I worked with with graduate students Françeska Xhakaj and Kenneth Holstein while under the direction of Vincent Aleven and Bruce McLaren.
An adaptive tutor is unique because it moves students at their own pace, making it difficult for teachers to provide students feedback. Teachers we talked to supplemented their teaching twice a week with adaptive tutor sessions.
Final Concept: Interface
This interface was meant to be used by teachers the night after a computer session in order to adjust their lesson plan for the next day.
Through research and interviews I conducted, I learned that teachers have a fair amount of skepticism towards dashboards. Presenting actionable information without infringing on the agency of the teacher was one of the balances I needed to strike.
Class Level: Skills
From this dashboard, a teacher can make quick adjustments to their lecture plan while having an idea of which students may be struggling. The “skills” data has been more visually weighted compared to the “errors” data in order to reflect the varying strength behind their statistical models.
Class Level: Errors
Errors are less actionable for planning lectures and thus are displayed on a more problem by problem level. I made an effort to help teachers understand the tutor by including pop-ups with actual student problems (2x+3=10) next to the language used by the tutor (combine unlike terms to make variable).
Individual Level Dashboard
I primarily redesigned the bottom graphic titled, "How fast are students progressing?". Showing direct progress of students over a calendar helps teachers understand the tutor better and what the student experience is like. The filters included for the bottom graphic support either a requested feature by teachers or a current teacher practice.
Final Concept: Interviews
These interviews were for a different type of dashboard, one which a teacher would use in real time while students were working in the computer lab.
I designed almost all of the interview materials myself, as well as conducted three interviews. My objective was to understand what needs a dashboard must fulfill in order for a teacher to circulate effectively throughout the class.
"If you could have any super power to help you teach, what would it be and how would you use it?"
In beginning the interviews with a question about super powers, I intended to frame the interview as being exploratory and build rapport. Moving the teachers expectations towards participatory design with a simple, open-ended question encouraged participation and creativity.
"While drawing on the computer lab seating chart, how did you usually spend your time?"
Given contextual inquiry restrictions due to the time of year (summer), I resorted to directed storytelling. Some teachers had never taken their students to computer labs, so I improvised, asking them to describe how they supervised students working in activity stations.
"What are your gut reactions to the idea behind these hypothetical 'teacher dashboards'?"
Encouraging teachers to respond to the idea, versus focusing on details, was an essential part of the process. Another key was to have intentionally vague scenarios and use teacher's clarifying questions as an opportunity to ask what they would ideally want.
Notification Threshold Feature
Given the variability in student performance between classes, schools, and states, it is essential to be able to change the tutor’s threshold for notifications. This feature didn’t make it into the final interface, but the varying definition of whether a student is having trouble was interesting to think about.
Sketches of different graphics and how they were intended to be used was important to my process. My interfaces were designed with the intent of being directly connected to a specific teacher action.
Vincent Aleven designed these two graphics.
I read literature published on teaching, dashboards, and research methods to better understand how to design dashboards in educational environments. Information Dashboard Design by Stephen Few was especially helpful, I adapted many of his principles and a few of his graphics into the Next-Day Dashboard.