The applied linguistics and testing community must take some of the blame for public and government ignorance about how fair assessment works
In 2008 Canada's immigration service announced that it was considering changing its language testing policy to require all prospective English-speaking immigrants to take the Ielts test. This prompted a storm of protest from immigration lawyers and migrant support groups which in turn was picked up on by the media. Largely absent from that debate were the voices of testing experts.
One prominent language-testing expert in Toronto was asked to comment at the height of the controversy, but he declined. He was working at the time as a consultant to the government. His was a challenge faced by many testing experts who are asked to "consult" on language tests. Once hired, the tester becomes an "insider", bound by confidentiality requirements and silenced from speaking publicly. Later, when the government was already withdrawing the suggested change, the first comment from a language-testing expert appeared in print.
Traditionally, language testers have focused on the practice of test making, sharing insights and concerns about testing in the corporate board rooms of the testing industry, in academic journals, at conferences, or among test development teams.
Language-testing experts have primarily focused on the technical properties of tests (items, tasks, formats, rubrics, rating consistency, etc), limiting their responsibility to considerations of test quality or "fairness" within tests themselves.
Considerations of how tests were used (or misused) were the responsibility of test users – governments, policy makers, institutions – theirs was a social responsibility; their concern was "justice".
The International Language Testing Association (ILTA) has ably represented traditional language testing interests. It has steered a course away from considerations of local "justice" in testing, opting instead to articulate codes of best practice, to support scholarship in testing, and to lead by ethical example. This apolitical stance, however, has left a gap at both national and local levels.
Last May, a group of 15 language-testing experts from across Canada met in Ottawa to form a Canadian
language testing association. All of the language testers at the Ottawa meeting were already active members of ILTA, and indeed several had occupied key executive positions within ILTA and included ILTA's current president. Why did these language testers feel there was a pressing need for a national, Canadian association?
Given how pervasive language tests have become in daily life, it is ironic that in general so little is understood about them. For example, many would argue that tests provide a "neutral" means of sanction, selection or decision-making – "leveling the playing field", by offering the same, systematic procedure to all. They would point out that tests are a practical, efficient, economic, and fairer means of selecting and sanctioning, and their outcomes can be rendered dispassionately. The problem with this view, however, is that tests are not always neutral. In defining what is measured and how it is measured, tests can also define what counts – who and what are valued.
There were excellent reasons for the Canadian government to withdraw the one-test option, but these were not clearly articulated in the debate sparked by the proposed changes. For example, it can be argued that requiring all prospective
Canadian immigrants to take Ielts would have favoured those from educational cultures where testing is prevalent and who have test-taking skills that other immigrants lack. Many applying from predominantly oral cultures or ones where tests are uncommon would have been disadvantaged by the single-test option.
Test-wise test takers, typically knowledge workers, would have been advantaged; skilled workers – masons, tile layers, plumbers – may have been disadvantaged. A single-test approach could undermine Citizenship and Immigration Canada's goal to attract more skilled workers to Canada – an unintended consequence of such a policy decision.
It would also have created an unnecessary barrier to mobility – one that immigrant groups within Canada were quick to recognise and react to. An independent group of experts in language testing could have made this and other issues clear to government policy makers.
In the past, language testers have tried to influence testing policy by modelling best practice in their international association, by working on testing projects as technical experts from within, and by engaging in dialogue or advocating for change with policy makers in public debates. But these approaches have had only limited impact.
Forming a national association of language testers is only a first step in giving voice to language testing expertise. Although next steps will differ by context, in Canada, there is a pressing need for an independent, arms-length, publicly funded board of review made up of testing experts with a mandate to evaluate and respond to testing practice.
We live in an age of accountability. It is common practice to hold education systems, transportation systems, pharmaceutical companies accountable through quality assurance agencies; to accredit practices within the professions through external accreditation boards. But no one is holding testing practices accountable. There is no arms-length "watchdog" or board of review of testing experts – issuing a "stamp of approval" or a directive on problems of quality or ethics in testing practice. This may be essential if language testers are to address both issues of fairness and justice in language testing practice.
• Janna Fox is Associate Professor, Applied Language Studies at Carleton University, Ontario, Canada