Reasons to be optimistic about assessment

This blog post is taken from the first part of a presentation Stephen Tierney (@leadinglearner) gave at the Learning First Conference in Sheffield on the 5th November 2016, with an introduction by Rob Coe.

"I already knew about Stephen from his blogs and twitter posts, so I had high expectations before I first heard him speak. I was not disappointed. He was able to make something that was fundamentally quite complex and difficult seem really simple and obvious.

He has quoted me in this piece as saying that what we need is ‘teachers with greater wisdom.’ For me, he is a shining example of that wisdom: someone who has a deep, critical and connected understanding of theoretical issues in assessment, but is also working in schools to apply and connect that knowledge to things that real teachers are doing daily with real pupils, with all the constraints that make real life different from theory.

It is always easy to get depressed about things in education: government policy, funding cuts, workload, or any number of other triggers can reliably dampen the mood. But I can always get a boost of optimism from recalling that working in our schools we have people like Stephen, bringing energy, moral leadership, passion – and wisdom. We are delighted to be able to host his thoughts on the CEM blog."

(Rob Coe November 2016)

Principled assessment in practice – Stephen Tierney

@LeadingLearner

In talking about principled assessment in practice and why we should be optimistic, I want to focus on the central assessment concept of validity from three different perspectives.

1. Validity

If we are going to move assessment forward in our schools, we must understand the central concept of validation; “the process of establishing what conclusions are warranted and which are not” from the evidence we have (Wiliam, 2014).

Dylan Wiliam asserts:

This is important, because it means that a question like ‘is this test valid?’ is meaningless. Asking whether a test is valid is to commit what Gilbert Ryle described as a ‘category mistake’ (Ryle 1949) – ascribing something a property it cannot have, like asking whether a rock is happy.

Take for example the construct of “high quality teaching”. For well over a decade inspectors and school leaders have drawn substantial, unwarranted conclusions from the flimsiest of evidence. Observing twenty minutes or one hour of a lesson and then making conclusions on the quality of teaching, and by implication the quality of the teacher, now seems bonkers to many of us who have done it hundreds of times.

The latest red herring is the book review - remember, this only tells you so much.

The elusive search for a single, simple, quick assessment indicator of the quality of teaching will remain just that; elusive, since it doesn’t exist. Great teaching is so much more complex.

2. Being Informed

CEM Director and Professor of Education at Durham University, Rob Coe, asks:

What makes great teaching?...we don’t know as much about it as we’d like to. We get glimpses at the moon through a cloudy sky – snippets that give insight.

We try and bind them together, but we have a precarious grasp of something that is very complicated …what we really need …is teachers with greater wisdom.

Teachers who know the research evidence and debates…and can integrate this into their own experience, skill base and practical repertoire, so their teaching is more creative, individualised…the problem is this could be taken as a license to say whatever feels good, is good.

The dilemma is too much freedom vs too much constraint. Neither quite works. Somewhere in between is right.

Coe, 2016

Similar challenges are faced when holding schools accountable for the quality of education provided. In helping develop the Headteachers’ Roundtable Alternative Green Paper, Schools that Enable All to Thrive and Flourish, we called for an end to the use of attainment outcomes.

The most valid conclusion which may be drawn from attainment outcomes relates more to a school’s intake than its effectiveness. Our call for a three year contextual valid added measure would be a major step forward, if adopted.

The new Progress 8 measure makes this a possibility for secondary schools but we are years away from developing a reliable and valid measure for primary schools.

However, even if we can move towards a multi-year progress measure many would argue that this gives a very limited view of what a good school should be.

Both the process of lesson observations and inspection of schools fail to provide valid conclusions about good teaching or effective education, respectively, as the assessments are too small to measure the actual construct. In technical language, they suffer from construct under representation.

3. Moving beyond ethical to humane

As John Sentamu, the Archbishop of York (2016) states education is about both educare and educere. We are much better at assessing the first rather than the latter.

What do we mean by ‘education’? There are two latin roots for the word: educare meaning to ‘bring up, to train, and to teach’, and educere, meaning ‘to lead and draw out that which likes within’.

Together both meanings provide a helpful picture for what education should be. But I believe we now need to place greater emphasis on the educational qualities expressed in the word educere.

Reasons to be cheerful

The great possibility; the reason to be cheerful about Principle Assessment in Practice is we become empowered to ask better questions and better understand the limitations of the answers.

This is all part of developing a deep knowledge and understanding of assessment. We can then start exploring and understanding the world of trade-offs.

Using the example above, it is possible to look at other important outcomes of education that we would want to see in a good school; we’d need to agree what they are, determine suitable metrics and then provide the time and funding to enable it to happen.

The question then is whether this would be the best use of the time and money available in improving schools. What are the alternatives? Should we pursue these alternatives in preference?

We can take this thinking into the classroom. It is part of becoming an evidence based profession.

References

Wiliam, D. (2014) Redesigning Schools – 8: Principled Assessment Design, London, UK; SSAT

Coe, R. (2016) SSAT meets Professor Rob Coe: Part 1 – Proper research is what identifies great teaching. Available: https://www.ssatuk.co.uk/ssat-meets-rob-coe-1/ Last accessed 22nd May 2016

Archbishop of York (2016) Nurturing the heart, mind and soul: the spiritual context of education. In: Chambers, P. Schools for Human Flourishing. London: SSAT. 84-90.

Read Stephen Tierney’s book Liminal Leadership.

Read his @leadinglearner blog and follow him on Twitter @LeadingLearner.