1. VALIDITY
1.1. What is validity?
1.1.1. If a test is considered valid then it is measuring what it is supposed to measure.
1.2. Types of validity evidence
1.2.1. Content Validity Evidence
1.2.1.1. Established by inspecting the questions of the test to make sure they correspond to what the creator decides what the test should cover.
1.2.2. Criterion-Related Validity Evidence
1.2.2.1. Established by correlating the test scores with an external standard/criterion in order to obtain a numerical estimate. There are two types.
1.2.2.1.1. Concurrent
1.2.2.1.2. Predictive
1.2.3. Construct Validity Evidence
1.2.3.1. Determined by finding out whether the test results correspond with scores on other variables that are predicted by a rationale or a theory.
1.3. Principles to consider when interpreting validity evidence
1.3.1. The adequacy depends on both strength of validity coefficient as well as the purpose the test is used for.
1.3.2. Group variability will affect the strength of the validity coefficient.
1.3.3. Validity coefficients need to be considered in terms of the relevance and the reliability of the criterion or standard.
2. RELIABILITY
2.1. What is reliability?
2.1.1. A test is considered reliable if it yields the same or similar scores consistently when all factors are equal.
2.2. Methods of checking reliability
2.2.1. Test-Retest
2.2.1.1. Test is given twice and correlation between each set of scores is determined.
2.2.2. Alternative Form
2.2.2.1. When two equivalent forms of a test are available, both are given to the same group of students and the correlation between the two scores is determined.
2.2.3. Internal Consistency
2.2.3.1. Split-Half or odd-even estimates
2.2.3.1.1. Divide test into halves and correlate the halves with each other.
2.2.3.2. item-total correlations
2.2.3.2.1. Determines the extent an entire test represents a single and fairly consistent measure of a certain concept.
2.3. Principles to Consider when interpreting reliability coefficients
2.3.1. Group variability affect the test score reliability. If group variability increases so does the reliability.
2.3.2. Scoring reliability can limit the test score reliability. As scoring reliability goes down so does the test's reliability.
2.3.3. Length of test affects test score reliability. As the length increases, the test's reliability also increases.
2.3.4. Item difficulty affects test score reliability. As items become either very easy or very hard, the test's reliability goes down.