Who put the 'ass' in assessment?

This week's eAssessment Scotland Conference was an interesting and thought provoking event. Hosted by the University of Dundee, the conference attracted almost 300 delegates from all over the UK, and farther afield and as could be expected, saw a number of papers presented on all aspects of technology enhanced learning and assessment. These included presentations on the use of blogs, peer collaboration, mobile assessment, serious games, Google forms, Mahara and other e-portfolio applications, audio feedback and personal learning environments.

I was very pleased to have been invited to give the opening keynote, which I entitled: Assessment in the Digital Age: Fair Measures? The slideshow is below:

Assessment in the Digital Age: Fair Measures?

View more presentations from Steve Wheeler

I started off with some horseplay on accents and language (I do an impressive Shetland accent, but my French accent sounds more like Inspector Clouseau gargling). Although we had a good laugh, there was a serious point to the funny accents. I made a remark that is still crystallising in my own mind, that accents tend to divide people - they are not only an indication of where we may have spent our time growing up, they are also a cultural marker and a statement of our identity. As such, there can be problems of comprehension and confusion if the accent is strong. On the subject of assessment, might it be fair to claim that the accent used by those who are assessing may be confusing or alienating to those who are being assessed? I can't recall how many times I sat down for an exam and turned the paper over, only to be confronted with what seemed to me like a foreign language. Throughout the day, both Donald Clark (another eAS11 keynote) and I showed some hilarious examples of misinterpreted exam answers. The responses given to the answers may have seemed funny, but in fact they were generally correct. The point we both made was that the students weren't wrong, the exam questions were wrong. They were either impenetrable, ambiguous, or simply poorly worded.

I have just reviewed a new book for the Times Higher Ed. It's called Now You See It, and is authored by a well known American academic and brain behaviour scientist called Cathy N. Davidson. In it, she recounts a story of a time she sat a multiple choice question paper. She got very low marks, because she spent most of the time on the reverse of the answer sheet correcting all the errors and ambiguities in the questions. She pointed out that some of the questions could not be answered because none of the options were correct. Surely she should have been given very high marks for demonstrating her creativity and intelligence? No, she didn't answer the questions, and therefore scored a low grade. There were no points for critical thinking or creative solutions. The only reward you can receive in this system is if you play by the rules and regurgitate the facts that were drilled into you.

One of the conclusions of the eAssessment Scotland conference, which very few people argued against, was that examination authorities really need to get their act together if they are to continue to administer exams that shape the future of young people. It's an absolute disgrace and entirely unforgivable when exam boards such as AQA, OCR and Edexcel cannot find the expertise within their organisations to create examination papers that are error free. Let's face it, that's all they are meant to do. Yes, we struggle with understanding people when they have strong accents, but it's more than a struggle when children are penalised because it's impossible to answer exam questions.


Who put the 'ass' in assessment? by Steve Wheeler is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Uncertainty principles

In the lead up to eAssessment Scotland where I will be speaking later this week, I offer some thoughts on assessment. There are some things you just can't assess. One is creativity. Another is character. Yet another is how tenacious or resilient a student is - do they persist in their learning despite the odds? Attempting to measure such things actually makes a nonsense of assessment. It's a bit like over-analysing a joke. What makes a joke funny? Is there a formula involved? Look at it too closely and it's no longer funny. It begins to disappear in front of you as you stare at it.

In 1927 Werner Heisenberg proposed a theory of quantum mechanics that became known as the Uncertainty Principle. In essence says Heisenberg, you can measure the position of a particle, or you can measure the future momentum of the particle. What you can't do is measure both at the same time. The more precisely one property is measured, (say the Wikipedia article) the less precisely the other can be controlled, determined, or known. Applying this outside of particle physics could be problematic, but let's try (because we're all made up of particles).

So you want to assess creativity? What are you actually trying to measure? A child's natural imagination? The creative outputs that are a result of that imagination? The value of their creativity in relation to that of the rest of the group? Against your own creativity? Against the standardised norms of the creative expectations of the entire society perhaps? Oh dear. You can identify that a child is talented in a particular area and their art is easy on the eye. They are good at painting. They have a propensity to be able to play a musical instrument pleasingly. Can you attach a value to it though? So what about Pablo Picasso? Or Karlheinz Stockhausen? Picasso wasn't particularly pleasing on the eye, Stockhausen was not easy to listen to. Although not everyone agrees with that last statement - even if they don't understand the art of the music - few would dare to suggest that Picasso and Stockhausen lacked creativity. Creativity is a very subjective thing, so should we attempt to assess it?

If you try to measure the current state - the effects of creativity on your emotions, the atmosphere, the ambiance of the experience, you will not discover where the creativity is leading - the message, the genre (sometimes) the theme. What are we doing with assessment of learning in our schools? Are we measuring the worth of the learning, or (as is inevitably the case) the worth of the individual? If we do the latter, we are betraying the trust of the child, because they will own that grade for the rest of their lives, citing it on every CV and job application form they complete. Is this fair? Is it fair that the grades they are awarded do not reflect their personality, their creativity, their tenacity, their resilience, their uniqueness? Standardised Testing and end of term examinations are absolutely unfit for this purpose. They are great for testing the recall skills of students, but useless in finding out more meaningful information about the knowledge and skills of the individual.  And yet we measure children's worth in exactly this way. Unfortunately, in this society that is what seems to count the most. Unless we value creativity, character and resilience (and resist measuring them), we will only create uncertainty in the minds of the young people who are in our charge.

"The person who scored well on an SAT will not necessarily be the best doctor or the best lawyer or the best businessman. These tests do not measure character, leadership, creativity, perseverance." - William J. Wilson


Uncertainty principles by Steve Wheeler is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.