Test Validity (or Justification) and Beneficial/Negative Consequences

Almost finished a submission on a major theme of my Lit Rev: current paradigms in Language Test Validity. I'm looking at what Lyle, (Bachman & Palmer, 2010) has to say about "unintended detrimental consequences". I've got to reframe that for the language test score = indefinite leave to remain discourse. A language test like this is unlike any other. 

For example, say a student needs IELTS 7.5 to get her place at a UK University. Let's also agree, just for now, that IELTS scores have been fully justified/validated, and that the University admissions department knows full well, using past experience, that anyone with less than 7.5 will be unable to keep up, submit essays of the required standard, and will not graduate. Maybe they know that because until a few years ago the bar was set at 7.0, and they had a whole load of mushkila situations with L2 students. 

So our student gets 7.0. This looks like a "detrimental consequence" for her. She really, really wanted that particular course at that particular University. Now she has to take shit from her parents, teachers and friends, and make do with a place at a less prestigious University. But it ISN'T a detrimental consequence for her in the framework we're using. If she'd gone to her 1st choice University, she would have struggled, failed, and wasted at least a year of her life. Whereas, she'll graduate from her second best option, maybe an L1 University. Beneficial consequences for all stakeholders. 

Whereas. With language tests whose consequences are leave to remain or no leave to remain. You can see the beneficial consequences for those who reach the cut-score. But those who fail? Does their lower language ability mean they would have failed as a new Scot? That's the crux of it. I need to dig into the policy and politics of it, to see what arguments were advanced for a B1 cut-score, and how someone who didn't meet that requirement, an A2 learner perhaps, would be seen as sure to fail as a new Scot. 



Bachman, L. F., & Palmer, A. S. (2010). Language assessment in practice. Oxford, UK: Oxford University Press.

Kane, M. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50, 1–73

Comments

Post a Comment

Popular posts from this blog

Leech or flatworm? Ants and Swiss Chard

Image
For the 2nd time in a week or so, (1st time regarding dwarf beans ), I'm following Alys Fowler's advice, this time with regard to Swiss chard .  Long story short, after midsummer plants which often bolt if planted in spring will resist the impulse to do so, probably because of shortening days; (I say shortening, but in Glasgow, in July, we're still getting 18 hours of daylight).   I got a packet of Bright Lights , at a good price, (another reason to wait until July before sowing, lots of bargains, I've noticed). But where to sow them? Most of the 2nd bed is under a tarpaulin, but it wasn't quite big enough to cover the whole bed, so the last 5ft or so went under a sheet of plastic, (in the background of the photo in this post, here .) Transparent plastic was a foolish idea, weeds were growing underneath it, so I covered most of it with sheets of metal from the old shed. I decided to leave the tarp, but use the area beneath the plastic for the chard. Who-ah there
Read more

LTPTP XXV: We Shall Overcome

Image
We shall indeed.  It's a lovely wee tune to learn.  I've almost memorized it this morning, the idea being, when I encounter a piano, I want something to sit down and play on the bugger.  I've a feeling that my keyboard's keys are lighter than a proper piano's, and there'll be a bit of effort transferring the skills I've gotten so far.  Probably, at some point this year, I'll be in workplace with a piano.  I'll print off some simple music and scales, and work at it, but it's likely to be in a public place so I want something to avoid making a twat of myself initially. It feels right that this should be the first tune I learn by heart.
Read more

"Chevy Chase. Fuck ever happen to him?"

Image
As Paulie Walnuts said in The Sopranos. It got me thinking. As it happens, he's been working steadily , but you can see what Paulie means. It's like that with film and television actors: you don't see them in anything for a year or two, and you wonder if they're dead or perhaps on a lengthy charlie and booze session. Or maybe doing proper acting on the stage? Only appearing on the telly gives them meaning to us. That video with Paul Simon was on one of the cable music channels the other day. It always makes me smile. As for Paulie and the rest of the crew, the clock's ticking. Of course, America already knows how things finish, but we don't. My mate Ian was telling me about a friend of his, visiting the States a few weeks ago, flicking through the channels in his hotel bedroom, he hit on a repeat of THE final episode, just beginning. What a quandry! He watched it, as it goes, but kept shtum when he came home.
Read more