Thursday, January 28, 2016

EdD: KoLL & Test Validity. Probably.

I turned in a slightly new direction on Monday. The "migration crisis" which dominates the news, (and the thoughts of many of us) has skewed my focus. What I'm looking at is the other end of the formal "migration" process, the claim for ILR, or application for citizenship, where the language testing is actually taking place. And it needs to be kept close to test validity.

Let's assume for the moment that I'm going with the consequential model of test validity, per Bachman & Palmer (2010) and Chalhoub-Deville (2015), (which is not to say that I've made any kind of binary choice between them and Kane, 2013).

So, it needs to be shown, (note use of passive), that the (currently) B1 CEFR Speaking & Listening test, being the language component of KoLL, has beneficial consequences. Now, it may be shown that achieving the cut score has benefits for the test taker. But what about a failure?  See, if someone fails to get their 6.5 in IELTS, say, and doesn't get their UK University place, then that's still a beneficial consequence for all of the stakeholders because time, money and emotional input won't be wasted. But what beneficial consequences flow from success or failure to reach the cut score for KoLL?

It might be necessary to widen this, and look at KoLL as a whole, that is include also the Life in the UK test. Alternatively, if it's looking unwieldy, and I want more manageable populations for the qualitative data, and a deeper language test focus, there are the B2 tests for Ministers of Religion. But, whatever, the heart of it must be test validity. 


Bachman, L. F., & Palmer, A. S. (2010). Language assessment in practice. Oxford, UK: Oxford University Press. 
Chalhoub-Deville, M. (2015). Validity theory: Reform policies, accountability testing, and consequences. Language Testing. 

Kane, M. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50, 1–73