Validity and Reliability

1. Validity- Does the test measure what it is supposed to measure?

1.1. Content validity evidence- The content validity evidence for a test is established by inspecting test questions to see whether they correspond to what the user decides should be covered by the test.

1.2. criterion-related evidence- In establishing criterion- related validity evidence, scores from a test are correlated with an external criterion. There are two types of criterion-related validity evidence: concurrent and predictive.

1.3. Concurrent criterion- related validity evidence deals with measures that can be administered at the same time as the measure to be validated.

1.4. Predictive validity evidence- refers to how well the test predicts some future behavior of the examinees. This form of validity evidence is particularly useful and important for aptitude tests, which attempt to predict how well test-takers will do in some future setting.

1.5. A test has construct validity evidence if its relationship to other information corresponds well with some theory. A theory is simply a logical explanation or rationale that can account for the interrelationships among a set of variables. Many different kinds of theo- ries can be used to help determine the construct validity evidence of a test.

2. Reliability- refers to the consistency with which it yields the same rank for individuals who take the test more than once. In other words, a test (or any measuring instrument) is reliable if it consistently yields the same, or nearly the same, ranks over repeated administrations during which we would not expect the trait being measured to have changed.

2.1. Test–retest is a method of estimating reliability that is exactly what its name implies. The test is given twice, and the correlation between the first set of scores and the second set s determined.

2.2. If there are two equivalent forms of a test, these forms can be used to obtain an estimate of the reliability of the scores from the test. Both forms are administered to a group of students, and the correlation between the two sets of scores is determined. This estimate eliminates the problems of memory and practice involved in test–retest estimates.

2.3. If the test in question is designed to measure a single basic concept, it is reasonable to assume that people who get one item right will be more likely to get other, similar items right. In other words, items ought to be correlated with each other, and the test ought to be internally consistent