What are students thinking?

In this third blog post running up to the #micer16 meeting in May, Ross Galloway of University of Edinburgh introduces us to his topic – use of smart pens to gain an insight into student thinking while they work. As he describes below, Ross will pick up on this introduction in his talk, following on from the prompts at the end of this article. You can read the paper associated with the ideas introduced in this article on the APS website, which is free to access: Analyzing learning during Peer Instruction dialogues: A resource activation framework. Feel free to comment below as these comments can help us inform the discussion after Ross’ talk.

The inner working

A burning question for almost every educator is “What are my students thinking?”  Whether it’s trying to get to the bottom of why they don’t grasp some particular concept, or why they have difficulty solving problems, or why they struggle to interpret the problems in the first place, getting to grips with student thoughts is essential.  Of course, we can ask them what they think, but students can be reticent to open up about their processes.  A still greater obstacle is that students often struggle to articulate their thoughts, perhaps because they don’t themselves know what are the significant issues.  (After all, if they did, we probably wouldn’t have a problem…)  I’m sure we’ve all had that frustrating moment of “Oh, that’s  what you were thinking,” after twenty minutes of  bewildering discussion.

The issue is even more acute for education research: here, not only are we trying to understand individual student thoughts, but also to generalise and find wider connections, underlying themes, and common conceptions.  While individual conversations with students might be effective, they hardly scale well, and in any case aren’t suitable for all learning contexts.

Under these circumstances, we look for technology and techniques to assist.  I (and collaborators) have made use of devices called smart pens, allied to suitable investigative and analysis techniques, to try to make some headway in this area.

What is a smart pen?

Put simply, a smart pen is a pen that knows what you are writing, and usually also what you are saying.  The core of the device is a normal ink pen that writes on paper, so actually writing with one is a familiar process (although smart pens are still comparatively ‘chunky’ so can feel large in the hand).  The key to the technology is special paper with a grid of barely visible microdots printed on it: a tiny camera in the pen sees the dots and can tell where the pen is pointing.  A pressure sensor behind the nib knows when a pen stroke is being made.  In most smart pens, a microphone completes the trio of sensors, recording any sounds made near the pen.  The data captured by the pen is in the form of a movie, showing a page of penstrokes appearing in the order in which they were made, synchronised with the matching sound recording.

We have made use of smart pen technology* for two main use cases.

1. Individual ‘Think Aloud’

In this case, the student is given a problem to solve and is asked to solve it as they normally would, writing down any working with the smart pen and giving a running commentary as they go.  The basic idea of ‘think aloud’ is for them to vocalise their internal monologue [1].  The pen captures the commentary, and on re-watching the video the researcher can match up what the student says to the corresponding part of the solution captured on the paper.

This technique is most commonly used in a ‘lab’ setting, i.e. not in the normal classroom environment; students are specifically asked to attend research sessions.  However, the researcher does not have to be present in the room, which can otherwise be an inhibition or a distraction to some students.  Additionally, the researcher will not give any unintentional or unconscious cues to the student.  Thus, the smart pen technology allows a more authentic impression of the student’s thought processes to be obtained.

2. Natural environment capture

The other main use case we have explored is to ask students to use smart pens in everyday classroom settings, such as workshops or lectures.  This naturally requires consent from all nearby students, but does allow realistic discourse to be captured.  In this case, student thoughts are surfaced not by intentional monologue, as in the case of Think Aloud, but naturally in conversation as students discuss problems, assist each other or resolve disagreements.  We have used this technique to observe group problem solving dynamics in workshops, and to explore what students talk about [2] during Peer Instruction [3] sessions in interactive lectures.

The smart pen technology is useful here as it is easily portable, robust and unobtrusive.  The latter is a substantial advantage over conventional video-recording: many students dislike being video-recorded, but our experience is that the smart pens become normalised very quickly and the students seem not to be consciously aware of them.  What you get is a very natural and free-flowing conversation, that frequently provides eye-opening insights into student thinking that you just don’t get from physically eavesdropping.

Could you use this?

At the workshop session we will look at the practical details of implementation of these approaches, some of the findings we obtained, and demo both the technology and the techniques.  In the mean time, some things to consider:

  • What aspects of student thinking would you like to explore?
  • Would smart pen technology help?
  • What tasks would you set for the students?
  • What context would you use?

*We used LiveScribe smart pens (www.livescribe.com) but other technologies are available.


[1] K.A. Ericsson and H.A. Simon. Protocol Analysis. Verbal reports as data, Cambridge, Massachusets, 1984.

[2] Anna K. Wood, Ross K. Galloway, Judy Hardy, and Christine M. Sinclair, “Analyzing learning during Peer Instruction dialogues: A resource activation framework”, Phys. Rev. ST Phys. Educ. Res. 10, 020107, 2014

[3] E. Mazur. Peer Instruction: A User’s Manual. Series in Educational Innovation. Prentice Hall, 1997.


2 thoughts on “What are students thinking?”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s