Validity and Reliability
by Katie Gilbert
1. Estimates of reliability are obtained by administeraing the same test twice to the same group of individuals.
2. A reliable test will yield stable scored over repeated administrations, assuming the trait being measured has not changed.
3. Reliability
3.1. The stability of a test score over repeated administration.
4. Test-retest or Stability
4.1. There is a small time interval between testing, and correlating the scores.
5. Alternate Forms or Equilvance
5.1. Giving two alternate forms of a test to the same group and correlating their scores.
5.2. Two equivalent forms of the test must be available, and they must be administered under conditions as nearly as equivalent as possible.
6. Internal Consistency
6.1. Split half or Odd/Even Estimates
6.1.1. Divide a test into halves and correlate the halves with one another.
6.2. Item total correlations
6.2.1. Determine the extent to which the entire test represents a single, fairly consistent measure of a concept.
7. Reliability Coefficients
7.1. Group variability affects test score reliability. As group variability increases, reliability goes up.
7.2. Scoring reliability limits test scores reliability. As scoring reliability goes down, so does the test's reliability.
7.3. Test length affects test scores reliability. As test length increases, the test's reliability tends to go up.
7.4. Item difficulty affects test score reliability. As items become very easy or very hard, the test's reliability goes down.
8. Construct Validity Evidence
8.1. Determined by finding whether test results correspond with scored on other variables as predicted by some rationale or theory
9. Validity
9.1. A valid test measures what its supposed to measure
10. Content Validity Evidence
10.1. Systematically comparing a test item with instructional objectives to see if they match.
10.2. Does not yield a numerical estimate of validity evidence.
10.3. Tests
11. Criterion-related Validity Evidence
11.1. Established by correlating test scored with an external standard or criterion to obtain a numerical estimate of validity evidence.
11.2. Two Types of criterion related validity evidence
11.2.1. Concurrent Validity Evidence
11.2.1.1. Established by correlating test scores with a criterion measure collected at the same time.
11.2.2. Predictice Validity Evidence
11.2.2.1. Determined by correlating test scores with a criterion measure after a period of time has passed