Validity & Reliability
by Taileigh Prickett
1. Content Validity
2. Second form of validity: Criterion-related evidence
2.1. In establishing criterion-related validity evidence, scores from a test are correlated with an external criterion; concurrent and predictive validity
3. Concurrent criterion-related validity evidence
3.1. Deals with measures that can be administered at the same time as the measure to be validated.
4. If a test is to be used in any kind of decision making, or indeed if the test information is to have any use at all, we should be able to identify the types of evidence that indicate that test’s validity for the purpose it is being used for.
5. A test has validity evidence if we can demonstrate that it measures what it says it measures.
6. Predictive validity evidence
6.1. Refers to how well the test predicts some future behavior for the examinees. This form of validity evidence is particularly useful and important for aptitude tests.
7. Construct Validity Evidence
7.1. A test has construct validity evidence if its relationship to other information corresponds well with some theory.
8. Why validity and reliability are important to learning and assessment: They both play a huge role in learning and assessment. Validity and reliability provide the techniques and instruments necessary to conduct various forms of measurements. These instruments promise the quality of the tests are measuring what they are claiming to measure, and the measurements are coherent. Validity and reliability are the honest measurements of learning and assessment.
9. Taileigh Prickett EDU 645 Instructor: Raney September 5, 2012
9.1. New node
10. Refers to the consistency with which it yields the same rank for individuals who take the test more than once.
11. Test-Retest or Stability
11.1. The test is given twice and the correlation between the first set of scores and the second set of scores is determined
12. Alternate Forms or Equivalence
12.1. If there are two equivalent forms of a test, these forms can be used to obtain an estimate of the reliability of the scores form the test.
13. Internal Consistency
13.1. If the test in question is designed to measure a single basic concept, it is reasonable to assume that people who get one item right will be more likely to get other, similar items right.
13.2. Internal consistency is a way of finding alternate-forms of reliability.
13.3. Internal consistency techniques are useful and popular measures of reliability because they involve only one test administration and are free from memory and practice effects
14. Split-half methods
14.1. To find the split-half (or odd-even) reliability, each item is assigned to one half or the other.
15. Problems with internal consistency estimates
15.1. Should only be used if the entire test consists of similar items, but not for a language test involving a spelling section, a reading comprehension section and a composition section.
15.2. It measures internal consistency yield inflated estimates of reliability when used with speeded tests. A speeded test consists entirely of easy or relatively easy item tasks with a strict time limit.
16. Test length affects test scores reliability. As test length increases, the test’s reliability tends to go up.
17. Item difficulty affects test score reliability. As items become very easy or very hard, the test’s reliability goes down.