On Assessment – part 2
The first post of this series has led to a response by Gregory Thompson (whom I also ‘met’ through Twitter on #edchat). I’ve read it and enjoyed all the points he made regarding what I wrote on the first post. I feel like replying to it. I believe the conversation will evolve if I continue with the series as I originally intended to, adding a couple of things his post made me think about.
I finished the post by mentioning rater-reliability. If you remember the scenario that I depicted, you’ll have to bear with me that it’s not possible for anyone to do what is expected of teachers in that condition. It’s not about giving up, or abandoning principles because it’s all a lost battle. Teachers are teachers because they believe they can make a difference. Teachers have been students and know students look up to their teachers. Good teachers know a lot is at stake when they walk into a classroom and this is why they do not give up.
Reliability is how we can assess whether a test is ‘consistent and dependable’ (Brown). Thus, a test is reliable when the results are similar if you give the test on two different occasions to the same student. However, there are some factors that come into play and may affect reliability of a test. In the scenario I mentioned, I believe the most influential factor are the mental and physical conditions of the teacher, i.e. rater-reliability. Ask anyone to read and analyse 50 texts on the same topic and provide feedback for each one of them. This is feasible, OK. However, tell this person that he or she will have 4 hours to do that. Even if the first texts are carefully corrected, some issues, such as fatigue, will heavily influence the results of the tests and the feedback given. When I mentioned I can’t blame teachers who have to assess 800 students for not doing it using an alternative to tests, this is what I meant. It’s not the teacher’s fault, it’s just not possible because of the way these schools, inserted in these educational systems, are organised.
Rater-reliability is not the only issue that may affect the results of a test. Student-related reliability also has to be taken into account, and so does test-administration reliability. However, I guess the second most important reliability issue is test-reliability. Tests which are too long, timed or with ambiguous items are likely to be unreliable. As we’ve discussed on #edchat, and as we can easily see daily, it’s not uncommon for students who know everything they should know to perform poorly on a test. This is why tests must be constantly revised and re-written. It is possible for teachers to design good tests (I’m not talking about assessment just yet) as long as they take enough time to prepare, grade, and then evaluate their own tests and each item individually. It’s hard work, but that is teaching.
Overall, standardised tests tend to score high on practicality and reliability, but they score low on authenticity (remember I’m an English Language teacher when you read the term “authenticity”) and washback. The point is not that we should make sure assessment is not reduced to testing. However, we’ve got to understand the context each one of us is inserted in order to come up with alternatives that truly work. Asking someone who teaches more than 500 students to analyse each student’s progress, provide effective and meaningful feedback and still be able to teach properly is just insane. It’s the same as saying to a CEO of a company that he should have teams as large as 500 people with only one supervisor per team. If you can keep track of 500 people and make sure you’re not lacking in rigour, please let me know how you do it.
Saying that we should change the way we assess our learners is a bit like preaching to the choir. The true challenge is to provide effective alternatives that can be implemented. This means we should understand the context in which we are inserted before we say something is right or wrong. We can’t simply point fingers at people because of their way of doing things until we’ve been in their shoes.
Assessment needs change? Yes. Assessment needs to be seen as not only testing. Assessment is broader than testing. However, teaching is more than assessing. Of all roles teachers play, the role of the assessor is just one of them. If we lose track of what our reality is, we might be tempted to see things from a simplistic point of view. What is true for A may not be true for B, and the only way we can help is by listening and understanding the seriousness of the situation, getting the big picture, prior to making suggestions.
The ultimate purpose of assessment is to enable for ongoing progress. This is the one thing we can’t forget when discussing assessment. And this can be accomplished through both formative and summative assessment. A reflective piece of writing which is marked only with “excellent” or “very good” will be just as useless as a end of unit test that just has an “A” or a “10” on it. It’s not exactly about changing it immediately, but learning how to make use of the tools we have at hand now more effectively. Once this is done, a change will take place smoothly.
Back to you!