Elena Shohamy - The Power of Tests (Pt 1)

 Close reading of this text, as it appears to be a significant one in CLT. Google Scholar's current algorithm suggests it's been cited more than 700 times in total, 255 since 2012. I'll follow the author's chapters for note taking.The whole work is interesting to me, but I shall try to stay focused on my current area of interest, which is the interplay between current validity paradigms and CLT. The book is in 4 parts, and I'll publish a blog post for each part

1. USE ORIENTED TESTING is compared with 'Traditional testing' which had no interest in what uses tests were put to, the role of the test's designer ending when 'evidence of validity [is] obtained', (we must assume that the author is referring to the construct validity, the paradigm still in 2001). 'The task ends when psychometrically sound results are satisfactorily achieved.'  

Messick, 1994, and Gipps, 2012, (first published in 1994) are referred to as signalling a shift from purely technical to test use perspectives. Actually Messick, 1994 refers to '[t]he consequential aspect of construct validity'. Spolsky, 1998 brought this shift into language testing.

2. VOICES OF TEST TAKERS provides a sample of what test takers say about their feelings regarding tests, essentially seeing the tests as powerful, rendering the test taker as 'powerless'. Almost every one in the world could tell a test anecdote, few would be positive.

3. POWERFUL USES OF TESTS. It is argued that 'it is the manner in which tests are used' that causes test takers to have such negative views of them. 'Doing well on a test may mean that a person is given permission to migrate to a new country and start a new life, while doing poorly may force a person to stay somewhere he or she does not wish to be.'

It is the ability which tests have to make life-changing decisions for individual human beings which gives them their power. This is particularly so with regard to 'detrimental decisions', p17. 

In examining assessment's role as a disciplinary tool, it is opined that this role goes beyond individuals and can affect "whole societies. Tests can affect and manipulate educational or political systems, control curricula and redefine the knowledge of communities", p18. I can see that we could easily find examples of manipulation of educational systems, and curriculum control. Uncertain about tests' being responsible for political manipulation and redefining communities' knowledge. 

4. FEATURES OF POWER: 

• Administered by powerful organisation (eg an ancient and venerable educational institution). 
• They use scientific jargon. '[T]here is an unquestioned belief... that the evaluator is an expert who provides scientifically derived knowledge'. This reminds me of my time at EF, where no-one really understood the need to get even criterion validation through trialing; instead they would refer to my (!) 'knowledge and experience as a testing expert'. And then I was packaged to my intense but silent embarrassment as an 'assessment guru'. 
• Statistics used, giving a veneer of objectivity in a domain most people find dull or incomprehensible. 
• [written forms of communication; and documentation, not relevant to my research]
• objective language used

5. EMERGENCE OF POWER. In the early 20th century there was optimism about the use of tests to give opportunities to everyone regardless of social class and connexions. They would be useful, and would also be objective and based on scientific principles, (p26-7). This was exemplified in the introduction of the multiple choice format in the 1920s by the emerging test-industry.

'Centralized' and 'decentralized' educational systems are then compared. In the former, access to higher education is only given to the most deserving, and there will be a high stakes 16/18 test, the ESS, (End of Secondary School). In a decentralised system, it is assumed that everyone has the right to higher education, and testing, such as it is, consists of informal measurements, such as teacher grades. Scotland is clearly a centralized system, with access to University undergraduate courses being dependent on Highers' grades. No examples are given of a decentralized system, and I can't think of any.

6. TEMPTATIONS. Some other characteristics of tests which make them attractive to those in authority:

• public perceptions
• cut scores can be shifted to, eg, fulfill quotas
• the control and manipulation of knowledge
• parents find them appealing; so do politicians, [NB it might be interesting to go back, now, to the CCSS, with all of this in mind].  
• testing gives 'objective' measurement, letting teachers and others off the hook
• they're cost effective when compared to other methods of educational control, such as teacher training
• introducing a test or testing regime will suggest a politician is doing something

 These are all good ideas, but my feeling is that they need stress tested, kicked around, somewhat. For example, test takers do (I'm saying this subjectively as an experienced language teacher) have negative views of tests. But what about test takers who have succeeded in a test? Do they continue with their negative view after the fact of getting a certain score? I wonder if there's any research on that?

NB checking references I came across Stobart, 2008, which I'll read when I've finished this.

REFERENCES 

Gipps, C. (2012). Beyond testing: Towards a theory of educational assessment (Classic Edition). Abingdon: Routledge.

  

Hanson, F. A. (1994). Testing testing: Social consequences of the examined life. Univ of California Press.

Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational researcher, 23(2), 13-23.

Shohamy, E. (2001). The power of tests: A critical view of the uses of language tests. Harlow, UK: Pearson Longman.

Spolsky, B. (1998). What is the user's perspective in language testing. In Colloquium, The state of the art in language testing: The user's perspective. National Foreign Language Center, the Johns Hopkins University, Washington, DC.

 

Stobart, G. (2008). Testing times: The uses and abuses of assessment. Routledge.