Archive for the ‘Assessment’ Category

To test or not to test, what’s the benefit?

February 14, 2011 16 comments

Had I been asked the question above, I believe my answer would be sort of evasive. I’m not particularly in favour of using tests as the only way to assess learning, but I also have a hard time believing tests are all bad. What I believe in, though, is that there are well-written tests and poorly made tests – and I also believe that fewer and fewer people have actually learned how to write a test. Much on the contrary, it seems we, educators, are all onboard with the view that tests do very little to help learners go beyond the remembering stage of learning. What I get from most talks about tests is that we equate tests with standardised testing, and that as fewer teachers are brave enough to say that tests may be useful, we are witnessing a time in which tests are getting simpler and simpler to score, and, consequently, demanding more memorisation skills than learning from students.

Reality check, teacher. Abandoning tests alone won't change that kind of attitude. // Photo by Richard Phillip Rücker

However, as William Shakespeare said, “There is a tide in the affairs of men…”, and we have to take advantage of things as they happen, so I decided to take advantage of a recommended article by one of the members of my PLN I grew to admire due to her insights and always amazingly packed with important information yet straight to the point comments and blog posts, @ssuzip. And even though Susi hasn’t made clear what her opinion is regarding this matter, I’m going to put myself out in the open and grant any kind of criticism that may come so I can maybe change my mind on the topic, or at least have better arguments the next time I write or talk about it.

The article was published on The New York Times on January 20th, and it’s title is To Really Learn, Quit Studying and Take a Test. As you’ve got it by now, it is in favour of testing as a learning tool. What comes to mind is that most of our parents and grandparents were educated in a system that praised tests. Furthermore, and I don’t think this is only nostalgia, standards were higher in terms of actually learning what you were supposed to learn. There weren’t things such as ADHD, as Sir Ken Robinson pointed out well here, and the whole concept of an affective side to education wasn’t really that big a deal. Nowadays, with the decline of another very important institution – family – schools have to bend over backwards to try to cater for both the cognitive and affective growth. On the one hand, this has helped us see there are numerous benefits in taking the learner as a whole human being. On the other hand, well, it may at times drive us away from our main goal of teaching and making sure our students achieve what they can fully achieve. Now, if all teachers were just as the ones who are likely to actually read this blog, I’d be completely off my rocker to say such a thing. Nevertheless, no matter how hard our PLN is, we’re still far from being the majority.

Unfortunately, there are many teachers who claim to be against tests without having ever read anything from, say, Alfie Kohn to back them up. I myself can’t say I’m familiar with his work, but most people whose blogs I visit and who are firm in their opinion of abandoning tests tend to consistently citing Kohn. The problem is not saying tests are good or bad – the problem is not being able to reason on your own whether they are good or bad. Coming back to the article and my beliefs towards testing, I must say I saw myself arguing with the text many times while reading it. For instance,

“I think that learning is all about retrieving, all about reconstructing our knowledge,” said the lead author, Jeffrey Karpicke, an assistant professor of psychology at Purdue University. “I think that we’re tapping into something fundamental about how the mind works when we talk about retrieval.”

The very first thing to think about there regards the concept of learning. Had Mr. Karpicke stopped at retrieving, I’d probably stop reading the article right there. However, I agree with his view of learning as the ability to using what we’ve already seen and being able to adapt and make effective use of such memory. Shall we compare it to Bloom’s taxonomy of learning domains and focus on the cognitive domain? We need to remember what we have learned, but this has got to come into being with some sort of purpose, which will lead us to the following step – making use of what we’ve just remembered. In this regard, the very first thing (and, unfortunately, the only thing in some cases) tests do is make us retrieve information. Moving on with the article, we find out that students who are going to be tested have the feeling they don’t know as much as they should. In comparison, those students who are allowed to create diagrams and concept maps for the sake of learning, but who know they are not going to be tested, have the feeling they know pretty much everything they need to know. It turned out in research that the result was exactly the opposite when students were asked to remember the subject one week later.

Why retrieval testing helps is still unknown. Perhaps it is because by remembering information we are organizing it and creating cues and connections that our brains later recognize.

However, in order for such things to take place, we’ve got to be able to write good tests. It can’t all be about rote learning and memorising passages. Tests can be used in education; let’s not throw out the baby with the bathwater. I guess the very first thing is keeping an open mind. As Howard Gardner, and education professor at Harvard said,

the results “throw down the gauntlet to those progressive educators, myself included. Educators who embrace seemingly more active approaches, like concept mapping,” he continued, “are challenged to devise outcome measures that can demonstrate the superiority of such constructivist approaches.”

The final word on testing in education, in my humble opinion, is yet to be said. I am, however, not in favour of abandoning tests altogether. I don’t think tests should be the only source of assessment in the classroom, either. My final quote from the article comes from Dr. Marcia Linn, “Some tests are just not learning opportunities. We need a different kind of testing than we currently have.” I guess she’s just touched the crux of the matter. As a teacher, I have to say that I had to learn about writing tests on my own, and I did take the time to read about it and look for data on the matter. This is not what most people who started teaching with me have done, though. It seems to be a big gamble and, in this hit-or-miss game of ours, more misses are made than hits, which only proves to strengthen the view that tests aren’t beneficial for learning. Add to that the fact that tests have become a million dollar business, and you’ll see that most people who advocate for tests aren’t exactly worried about learning. This, obviously, make them an easy prey for educators who, once again, see others not only making money out of something they consider serious – children’s learning – but also see those people doing things that make us, educators, look bad. We want to help children learn, testing companies want to make children take as many tests as possible so they can profit from them.

As my final words, I add that I agree with Penny Ur when she says that tests are one of the many sources of extrinsic motivation that teachers have control over in class. Just as anything, it can’t be overused. If tests are well-written, they can certainly help in the learning process. To what extent? This seems to be uncertain at the moment, but if we don’t keep an open mind, we might end up doing more harm than good by abandoning tests just because we want to follow the trend.

Teaching for the test – based on a true story

March 28, 2010 12 comments

There is a school in Brasília that seems to be interested in approaching learning from a more ‘humanistic’ and ‘holistic’ perspective than what the current Brazilian educational system forces other schools to do so. They are concerned about arts, sports, music and creative thinking (as far as I’m concerned) than the other 99% of schools are. It’s almost as if Sir Ken Robinson’s idea of how schools should be like had come to life.

Wouldn’t it be wonderful if all schools could actually teach for life instead of teach for the test? I mean, it seems to me that it’s common knowledge among most educators that standardized tests don’t deliver what they claim to deliver – if you do well on the test, it means you have learned the subject. (You can read a lot of great posts on that here and here, to cite just a few people who will lead you to many others)

And my answer to that question is, yes, it would be wonderful. But I recently heard a story from a student from the school I mentioned in the first paragraph above. She had been studying in that school for quite a while and was really happy with her routine. It all changed on her second to last year of high school. As kids grow older, so do their social circles (hopefully), and no longer did this particular student live with people from her school circle. To make matters worse, she had already learned of the standardised test all Brazilian learners are submitted to and which defines whether you’re going to college or not.


Read the whole comic - it's much better explained there

Reality hit hard on this student when she had a chance to compare her “knowledge” on physics, for instance, against her friends’ who were also going to compete against her for a place at university. It suddenly hit her that, if she was to succeed, she’d have to change schools and go where they actually abide by the system – they teach for the test. And so she did. Needless to say, she’s feeling miserable, but one’s got to do what one’s got to do, right?

The aim of this story is to pose a question: how can teachers be responsible for changing the system when there are no teachers in charge of thinking education? I mean, even though we may want to change the way our students learn and value their personal characteristics, we seem to be in a catch-22 situation. What I mean is, education is, ultimately, between the teacher and the learner. However, these two elements are under pressure from many different angles: schools policies, state regulations, ministry of education, and society as a whole. If teachers start this change, students will fail the big test they are forced to take in order to have better chances in life. If teachers choose not to do anything to change this situation, they’ll end up frustrated as they know what they’re doing doesn’t really help much in real life.

I’m of the opinion that we should re-think schools, and education in general. But when we’re in such a sad situation as this, I think the easy way out is a top-down change. If the ministry of education don’t change the rules for university entrance, parents will insist on enrolling their kids on schools which are well-known for their high “pass” percentages. If schools and their teachers fight the system, no matters what parents say, they’ll end up with no students – all parents want to give their kids a good chance to succeed, and if by that they need to go to a good university, they’ll take their kids to the schools that’ll better prepare their kids to get to such universities.

The situation is even worse when you learn that state schools are left to their own devices by the government and no matter how hard principals and teachers try, they can only do as much. Besides, if the salary is significantly less than what private schools pay, there isn’t much to keep good teachers in state schools.

To sum it up, it seems that our little friend will have to put up with the fact that, in order to go to university, she’ll have to be taught for the test – reality check. It doesn’t seem to matter that the kind of education she was getting before was preparing her better for life. In order to be able to “get a life”, she needs to pass the test.

How would we solve this if change doesn’t come from the top? I mean, teachers can and should do their share – pressure the government for change. But if things remain the same, the good teachers, the ones who care about teaching for life, will actually be putting their kids at a disadvantageous position. And this will be true as long as the yardstick we measure our kids against is a test.

* I first learned about Sir Ken Robinson’s video here – another nice blog!
* Read the complete comic strip here. A lot of fun stuff for those who like reading about grad school.

On Assessment – part 2

December 17, 2009 2 comments

The first post of this series has led to a response by Gregory Thompson (whom I also ‘met’ through Twitter on #edchat). I’ve read it and enjoyed all the points he made regarding what I wrote on the first post. I feel like replying to it. I believe the conversation will evolve if I continue with the series as I originally intended to, adding a couple of things his post made me think about.

I finished the post by mentioning rater-reliability. If you remember the scenario that I depicted, you’ll have to bear with me that it’s not possible for anyone to do what is expected of teachers in that condition. It’s not about giving up, or abandoning principles because it’s all a lost battle. Teachers are teachers because they believe they can make a difference. Teachers have been students and know students look up to their teachers. Good teachers know a lot is at stake when they walk into a classroom and this is why they do not give up.

Reliability is how we can assess whether a test is ‘consistent and dependable’ (Brown). Thus, a test is reliable when the results are similar if you give the test on two different occasions to the same student. However, there are some factors that come into play and may affect reliability of a test. In the scenario I mentioned, I believe the most influential factor are the mental and physical conditions of the teacher, i.e. rater-reliability. Ask anyone to read and analyse 50 texts on the same topic and provide feedback for each one of them. This is feasible, OK. However, tell this person that he or she will have 4 hours to do that. Even if the first texts are carefully corrected, some issues, such as fatigue, will heavily influence the results of the tests and the feedback given. When I mentioned I can’t blame teachers who have to assess 800 students for not doing it using an alternative to tests, this is what I meant. It’s not the teacher’s fault, it’s just not possible because of the way these schools, inserted in these educational systems, are organised.

Rater-reliability is not the only issue that may affect the results of a test. Student-related reliability also has to be taken into account, and so does test-administration reliability. However, I guess the second most important reliability issue is test-reliability. Tests which are too long, timed or with ambiguous items are likely to be unreliable. As we’ve discussed on #edchat, and as we can easily see daily, it’s not uncommon for students who know everything they should know to perform poorly on a test. This is why tests must be constantly revised and re-written. It is possible for teachers to design good tests (I’m not talking about assessment just yet) as long as they take enough time to prepare, grade, and then evaluate their own tests and each item individually. It’s hard work, but that is teaching.

Overall, standardised tests tend to score high on practicality and reliability, but they score low on authenticity (remember I’m an English Language teacher when you read the term “authenticity”) and washback. The point is not that we should make sure assessment is not reduced to testing. However, we’ve got to understand the context each one of us is inserted in order to come up with alternatives that truly work. Asking someone who teaches more than 500 students to analyse each student’s progress, provide effective and meaningful feedback and still be able to teach properly is just insane. It’s the same as saying to a CEO of a company that he should have teams as large as 500 people with only one supervisor per team. If you can keep track of 500 people and make sure you’re not lacking in rigour, please let me know how you do it.

Saying that we should change the way we assess our learners is a bit like preaching to the choir. The true challenge is to provide effective alternatives that can be implemented. This means we should understand the context in which we are inserted before we say something is right or wrong. We can’t simply point fingers at people because of their way of doing things until we’ve been in their shoes.

Assessment needs change? Yes. Assessment needs to be seen as not only testing. Assessment is broader than testing. However, teaching is more than assessing. Of all roles teachers play, the role of the assessor is just one of them. If we lose track of what our reality is, we might be tempted to see things from a simplistic point of view. What is true for A may not be true for B, and the only way we can help is by listening and understanding the seriousness of the situation, getting the big picture, prior to making suggestions.

The ultimate purpose of assessment is to enable for ongoing progress. This is the one thing we can’t forget when discussing assessment. And this can be accomplished through both formative and summative assessment. A reflective piece of writing which is marked only with “excellent” or “very good” will be just as useless as a end of unit test that just has an “A” or a “10” on it. It’s not exactly about changing it immediately, but learning how to make use of the tools we have at hand now more effectively. Once this is done, a change will take place smoothly.

Back to you!

On Assessment – part 1

December 16, 2009 3 comments

Yesterday evening the discussion on #edchat was about assessment. As usual, a stream of thought-provoking tweets and a lively discussion took place. I had been thinking about writing a post on assessment and my thoughts on the matter, so I feel now is a good time to do it.

As I see it, there are certain principles which should be taken into account when we discuss assessment, and I borrow these from Brown (Language Assessment – Principles and Classroom Practices) and some other readings.

To begin with, assessment is part of teaching, just as tests are part of assessment. This means that there is the group TEACHING, the subgroup assessment, and, inside assessment, the subgroup of testing. Teaching is more than assessing, just as assessing is more than simply testing. I guess the first problem lies there. Many teachers tend to merely equate assessing with testing. I’m going to start this series talking about testing.

Tests are usually standardised and tend to measure discrete points of what has been taught. They’re tools teachers have to gauge how much students have learned. Tests are summative instead of formative, i.e. they aim at measuring and summarise what has been taught through a period of time, and usually come at the end of a unit or a course. Tests, just as any kind of assessment, may be good or bad, they aren’t necessarily the bad guys of education. The way most teachers have been treating tests is the main problem, I guess.

If we think about the principles Brown mentions in his book (practicality, authenticity, reliability, validity, and washback), standardised tests are high on practicality by nature. This means they’re usually easy to administer and grade. There’s nothing wrong with this aspect of standardised tests per se. However, any kind of assessment should provide learners with meaningful and effective feedback. Learners should be able to use their tests results to find out how to improve and what they need to work on. Unfortunately, most teachers don’t do anything else but giving students a grade, be it a number or a letter. Students need guidance to find out what they need to study. Now, it’s not that most teachers don’t want to give students useful feedback, but, depending on the context, it’s simply impossible.

There are classrooms around the world with 50 students, and some teachers have to teach 16 or 18 groups. This means some teachers have more than 800 students. Not only do these teachers have to plan their lessons, but they also need to design and grade all these tests, and they usually are forced to have reports on students’ progress every other month. Now if teachers have 16 to 18 groups of 50 minutes each, they’re in the classroom around 30 hours a week. Add to that all the time it takes to assess students outside class, planning lessons, and being an educator in the core meaning of the word (worrying about each student and his or her learning, and empowering your learners), then you tell me how such an educator would be able to radically change his way of assessing students, going from summative to formative, using portfolios (for instance) instead of standardised tests, or tests made by the teacher him or herself. This means keeping track of 800+ students’ writing. I can’t blame teachers for not doing that. Besides, if a teacher has to assess that many students, there’s the serious risk of rater-reliability issues. But this is something for another post.

Your turn.

%d bloggers like this: