Language Testing Journals [EDITED 22nd March 2016 to include journals publishing articles on Critical Language Testing, heading changed]

[This blogpost was formerly entitled "Testing Validity Paradigm(s) because that is what was being researched at the time. But it has taken on a life as its own as a database of Journals regularly publishing articles on Language Testing, (all of which need to be therefore regularly consulted). I have amended it today because I have expanded the scope of research beyond straightforward validity and into Critical Language Testing, and wish to include journals discovered in connected literature searches, e.g. (5) Educational Research and Evaluation. 22/03/2016.]

Today's the day when I stop flitting around the garden of academia like an unfussy butterfly, and get my oomska together.

1st order of business is a list of journals which regularly publish articles on language testing, so that I can be aware of the current paradigms, focusing today on Test Validity, (so as not to be distracted, - I've got a bloody deadline approaching).

1. American Journal of Speech-Language Pathology Interesting but specialized. Nothing theoretical on validity, this journal is almost entirely devoted to research. 
2. Annual review of applied linguistics Really need to be following this closely, (see for example Wodak & Boukala, 2015, which I wasn't aware of until today). However, nothing recently on Test Validity has been published, (though it did publish, for example Chalhoub-Deville, 2009, so it's a player in the test validity paradigm (TVP) discourse
3. Applied linguistics All good stuff but nothing on the validity paradigm since 1999 and an article by Fulcher which I'm not even going to reference. 
4. Assessment in Education: Principles, Policy and Practice 
5. Current Issues in Language Planning Publishes articles on Critical Language Testing
6. Educational Measurement: Issues and Practice Great for real research, the "Practice" in the title is there for a reason. See Sireci 2012 for example. But nothing on the TVP.
7. Educational Research and Evaluation Publishes articles on Critical Language Testing.
8. Educational Researcher Bingo! See Sept 2007 [Come back to this when there's more time - there are other journals published by AERA.] Larsen McClartey 2013 will be worth following up when I address cut-scores.
9. ELT Journal. See Hall, 2014, (relevant to research though not TVPs).
10. Harvard Educational Review NO RESULTS in SFX linking service.
11. Interchange. Publishes articles on Critical Language Testing. 
12. International Reading Association [not searched for, seems tangential with a deadline shining its headlights at me].
13. Investigating Language NO RESULTS in SFX linking service
14. Journal of Academic Language and Learning [not searched for, relates to EAP]
15. Journal of Literacy Research [not searched for, as I'm primarily concerned today with Speaking & Listening domains; come back, though]
16. Language Assessment Quarterly Significant journal. McNamara 2006 clearly part of the TVP, though not currently. Harsch & Hartig 2015 relevant to my research, though not TVP.
17. Language Learning Has lots of interesting but very specific research papers. A paper, No Author, 2006, addresses TVP of the time.
18. Language, Speech, and Hearing Services in Schools [Come back some other time]
19. Language Teaching Like Language Learning, devoted to actual research rather than TVP, but Cheng et al 2015 using Kane's ABV is intriguing. 
20. Language Testing Will never let me down. That's where I got Chalhoub-Deville 2015 - the latest word on TVP (but see NB3 below). And then just now I found Macqueen et al 2015, with the concept of assessment as a "boundary object", which is really relevant to what I'm doing.
21. Linguistics in Education Publishes articles on Critical Language Testing. 
22. Practical Assessment, Research & Evaluation Open Access Journal with some useful-looking research but no references to TVP.
23. Review of Educational Research Another journal from AERA, which seems to specialise mostly in actual research, but has addressed TVPs in the past, see Kieffer, 2009.
24. Review of research in education Another journal from AERA, but much more theoretical, and willing to take on TVP. See Sireci & Faulkner-Bond, 2015. Also involved in controversy on CCSS. Need to come back to this for theory.
25. Studies in immigrant English language assessment. Not found on SFX, but obviously I need to google this, as, from its title, it is highly relevant. Australian. 
26. Studies in Educational Evaluation Publishes articles on Critical Language Testing. 
27. System Not Searched for today. The article I saw from this earlier today was about rubrics in CAT.
28. TESL-EJ Good open-access journal for publication of research, no TVP except in some old book reviews.
29. TESOL Quarterly Contains some theory, but mostly actual research. It's as if an editorial decision has been made to keep away from assessment wherever possible.

I located these journals' titles by means of a generic search (Google Scholar) using the terms language assessment and then "test validity". Journals which were publishing articles on first language assessment were ignored, as were results in the second search which related to test validity unrelated to second languages. Two searches meant that the results were somewhat triangulated.

Language testing as a discipline is located in the overlap between Education, Applied Linguistics, Linguistics and Psychology. Which means the list of journals one needs to be aware of is quite long, (the above is likely not exhaustive), and much of what is there will be irrelevant to current research, though distractingly within my fields of experience and interest.

Today's big "take away": some journals (eg LAQ) focus almost exclusively on actual research. Others, eg Language Testing, will publish research, but their pages are where we'll find paradigm shifts: theoretical papers from the Big Cheeses like Chalhoub-Deville, Lyle, etc.

NB, It was concerning to see what appeared to be a recent work specifically on the subject of Test Validity which I had not before today encountered, (Wainer & Braun, 2013). However, it turns out that this was actually a re-issue of a collection of conference papers from 1988.

NB2, Came across the Linguist List. This link is to its journal index. The categories not discrete, to the point of eccentricity, but it's a good way to see if anything's happening off the beaten track.

NB3, for Lit Rev, need to cite Hill & McNamara, 2015; and Bennet et al, 2016, which are responses to Chalhoub-Deville, 2015.  Also see the journals they were published in, which aren't in list above.

Khalas. Tomorrow, edit above list to the things I really need for TVP. Maybe do another list of good research journals. See NB3 re 2 other titles. Start writing up.

REFERENCES

(2006), Validity and the Social Dimension of Language Testing. Language Learning, 56: 9–42. doi: 10.1111/j.1467-9922.2006.00379.x

Bennett, R. E., Deane, P., & van Rijn, P. W. (2016). From Cognitive-Domain Theory to Assessment Practice. Educational Psychologist, 1-26.

Chalhoub-Deville, M. (2009). The intersection of test impact, validation, and educational reform policy. Annual Review of Applied Linguistics, 29, 118-131.

Chalhoub-Deville, M. (2015). Validity theory: Reform policies, accountability testing, and consequences. Language Testing, 0265532215593312.

Cheng, L., Sun, Y., & Ma, J. (2015). Review of washback research literature within Kane's argument-based validation framework. Language Teaching, 48(04), 436-470.

Hall, C. J. (2014). Moving beyond accuracy: from tests of English to tests of ‘Englishing’. ELT Journal, 68(4), 376-385.

Harsch, C., & Hartig, J. (2015). What Are We Aligning Tests to When We Report Test Alignment to the CEFR?. Language 

Assessment Quarterly, 12(4), 333-362.

Hill, K., & McNamara, T. (2015). Validity Inferences Under High-Stakes Conditions: A Response From Language Testing. Measurement: Interdisciplinary Research & Perspectives, 13(1), 39-43.

Kieffer, M. J., Lesaux, N. K., Rivera, M., & Francis, D. J. (2009). Accommodations for English language learners taking large-scale assessments: A meta-analysis on effectiveness and validity. Review of Educational Research, 79(3), 1168-1201.

McClarty, K. L., Way, W. D., Porter, A. C., Beimers, J. N., & Miles, J. A. (2013). Evidence-Based Standard Setting Establishing a Validity Framework for Cut Scores. Educational Researcher, 42(2), 78-88.

McNamara, T. (2006). Validity in language testing: The challenge of Sam Messick's legacy. Language Assessment Quarterly: An International Journal, 3(1), 31-51.

Macqueen, S., Pill, J., & Knoch, U. (2015). Language test as boundary object: Perspectives from test users in the healthcare domain. Language Testing, 0265532215607401.

Sireci, S. G. and Forte, E. (2012), Informing in the Information Age: How to Communicate Measurement Concepts to Education Policy Makers. Educational Measurement: Issues and Practice, 31: 27–32. doi: 10.1111/j.1745-3992.2012.00232.x

Sireci, S. G., & Faulkner-Bond, M. (2015). Promoting validity in the assessment of English learners. Review of Research in Education, 39(1), 215-252.

Wainer, H., & Braun, H. I. (2013). Test validity. Routledge.

Wodak, R & Boukala, S (2015). (Supra)National Identity and Language: Rethinking National and European Migration Policies and the Linguistic Integration of Migrants. Annual Review of Applied Linguistics, 35, pp 253-273. doi:10.1017/S0267190514000294.