Close reading of this text, as it
appears to be a significant one in CLT. Google Scholar's current
algorithm suggests it's been cited more than 700 times in total, 255
since 2012. I'll follow the author's chapters for note taking.The whole work is interesting to me, but I shall try to stay focused on my current area of interest, which is the interplay between current validity paradigms and CLT. The book is in 4 parts, and I'll publish a blog post for each part
1. USE ORIENTED TESTING is compared with 'Traditional testing' which had no interest in what uses tests were put to, the role of the test's designer ending when 'evidence of validity [is] obtained', (we must assume that the author is referring to the construct validity, the paradigm still in 2001). 'The task ends when psychometrically sound results are satisfactorily achieved.'
Messick, 1994, and Gipps, 2012, (first published in 1994) are referred to as signalling a shift from purely technical to test use perspectives. Actually Messick, 1994 refers to '[t]he consequential aspect of construct validity'. Spolsky, 1998 brought this shift into language testing.
2. VOICES OF TEST TAKERS provides a sample of what test takers say about their feelings regarding tests, essentially seeing the tests as powerful, rendering the test taker as 'powerless'. Almost every one in the world could tell a test anecdote, few would be positive.
3. POWERFUL USES OF TESTS. It is argued that 'it is the manner in which tests are used' that causes test takers to have such negative views of them. 'Doing well on a test may mean that a person is given permission to migrate to a new country and start a new life, while doing poorly may force a person to stay somewhere he or she does not wish to be.'
It is the ability which tests have to make life-changing decisions for individual human beings which gives them their power. This is particularly so with regard to 'detrimental decisions', p17.
In examining assessment's role as a disciplinary tool, it is opined that this role goes beyond individuals and can affect "whole societies. Tests can affect and manipulate educational or political systems, control curricula and redefine the knowledge of communities", p18. I can see that we could easily find examples of manipulation of educational systems, and curriculum control. Uncertain about tests' being responsible for political manipulation and redefining communities' knowledge.
4. FEATURES OF POWER:
- Administered by powerful organisation (eg an ancient and venerable educational institution).
- They use scientific jargon. '[T]here is an unquestioned belief... that the evaluator is an expert who provides scientifically derived knowledge'. This reminds me of my time at EF, where no-one really understood the need to get even criterion validation through trialing; instead they would refer to my (!) 'knowledge and experience as a testing expert'. And then I was packaged to my intense but silent embarrassment as an 'assessment guru'.
- Statistics used, giving a veneer of objectivity in a domain most people find dull or incomprehensible.
- [written forms of communication; and documentation, not relevant to my research]
- objective language used
'Centralized' and 'decentralized' educational systems are then compared. In the former, access to higher education is only given to the most deserving, and there will be a high stakes 16/18 test, the ESS, (End of Secondary School). In a decentralised system, it is assumed that everyone has the right to higher education, and testing, such as it is, consists of informal measurements, such as teacher grades. Scotland is clearly a centralized system, with access to University undergraduate courses being dependent on Highers' grades. No examples are given of a decentralized system, and I can't think of any.
6. TEMPTATIONS. Some other characteristics of tests which make them attractive to those in authority:
- public perceptions
- cut scores can be shifted to, eg, fulfill quotas
- the control and manipulation of knowledge
- parents find them appealing; so do politicians, [NB it might be interesting to go back, now, to the CCSS, with all of this in mind].
- testing gives 'objective' measurement, letting teachers and others off the hook
- they're cost effective when compared to other methods of educational control, such as teacher training
- introducing a test or testing regime will suggest a politician is doing something
NB checking references I came across Stobart, 2008, which I'll read when I've finished this.