TEST USEFULNESS

1. RELIABILITY

1.1. IS THE TEST RELIABLE?

1.2. Scoring

1.2.1. Are the grading scales clear and detailed?

1.2.2. For subjective scoring: are the scorers trained properly?

1.2.3. Are the scores interpretable by everyone?

1.3. Instructions

1.3.1. Are the instructions clear?

1.3.2. Are the candidates familiar with test format and techniques?

1.4. Administration

1.4.1. Is the test adminstrator well trained?

1.4.2. Are there measures for preventing copying/cheating/dishonest acts?

1.4.3. Is the environment suitable and free of distractions?

1.5. Others

1.5.1. Are there any ambiguous items?

1.5.2. Is the test well laid-out and legible?

2. IMPACT/ WASHBACK

2.1. Is it a high-stakes test? If yes, is washback beneficial?

2.2. Are items in the test predictable?

2.3. Does it motivate and encourage candidates to study for authentic situations?

3. AUTHENTICITY

3.1. Are the characteristics of the test task authentic?

3.2. Does it use authentic source material?

3.3. Achievement test: Is the test based directly on a useful course objective?

4. CONSTRUCT VALIDITY

4.1. TO WHAT EXTENT CAN WE JUSTIFY INTERPRETATIONS ON LANGUAGE ABILITY BASED ON TEST SCORES?

4.2. Content Validity

4.2.1. Is the content suitable for the purpose of the test?

4.2.2. Is the level of difficulty appropriate for candidates?

4.2.3. Does it test all representative skills?

4.3. Face Validity

4.3.1. Do the candidates think it is a useful/reiable test?

5. PRACTICALITY

5.1. Is it worth the resources?

5.2. Is it possible to do in given time frame?

5.3. Are the graders able to grade it on time?

6. INTERACTIVENESS

6.1. Can the test be done or guessed without language knowledge?

6.2. Is the test affected by topical knowledge or affective schemata?