Saturday, February 20, 2016

Reading on Test Validity and Integration (Inclusivity and Exclusivity) - 2

"Mastering the language of a country of immigration is a skill acquired over time through residence and social interaction and is therefore one of the most prevailing markers of an applicant’s personal level of integration," (Goodman, 2010). "Mastery" has a specific meaning in CEFR terms, referring to C2 level. Most EU states' language requirements are in the A2/B1 region. Many language learners will reach A2/B1 levels in the L1 of a country which they have never visited, or at any rate never resided in or had social interaction with L1 speakers, for example [get hard data] school learners of French in the UK, or English in China or the Netherlands. 

At p14 Goodman (2010) refers to the Austrian requirement that learners do 300 hours of instruction. Is this an alternative to the A2 referred to in that paper's Table 4, below?  So that 300 hours of instruction can be sufficient regardless of test result? That would be a rather honest acknowledgement of the fact that the Learner is jumping through hoops, demonstrating a willingness to "be on the team".  

Copied from Goodman 2010.

[NB check currency of this data, 2016]

At p16 it is suggested 300 hours of instruction would be enough to reach A2 or B1. I'm not sure. ESOL teachers' understanding of what an hour is can vary between 40 and 50 minutes, but it's rarely 60 minutes. A lot would depend on the quality of instruction and the materials. For example, spread over 12 months, with lots of good quality real-world homework, (where Learners are, for example, watching a TV programme in their own time and discussing it in class), this could be enough to get to B1. If the 300 hours were put into a 10 week intensive course, then maybe not. And we need to consider Learners' starting levels. There's a big difference between pre-A1 and A1, for example. And what about basic literacy in their L1? And those who know Roman script, and those who don't? And those with/out conventional European classroom study skills. 

It's worth benchmarking the A2/B1 clusters, and the oft-repeated 300 hours, for now, to reconsider going forward.  It might have an effect on Validity, if a test provider stakeholder is saying "You can do this after 300 hours instruction."

Good point on p15: it's not necessarily about integration per se. A more sophisticated language level requirement, (see Croatia and Denmark) will exclude many but include "expatriates and persons of co-ethnic ancestry". There might be an element of that in any test of Reading and Writing, favouring those whose L1 includes Roman script. 

[Which makes me think the UK Speaking & Listening B1 test might favour Arabic L1s over Chinese. ESOL teachers with deep experience of both know that Arabic L1s often have an almost reckless self-confidence in speaking, unlike Chinese L1s who can be tongue tied even if possessing sophisticated Reading and Writing skills. This might be the result of the difficulties faced by L1 tonal language speakers.] Khalas.
 

REFERENCES

Goodman, S. W. (2010). Naturalisation Policies in Europe: exploring patterns of inclusion and exclusion. EUDO Citizenship Observatory. Robert Schuman Centre for Advanced Studies.