Assessment - a variety of procedures used to obtain information about student learning and the fo...
1. Measurement: The process of obtaining a numerical description of the degree to which an individual possesses a characteristic (ex. can be through testing). Measurement is a quantitative characteristic (Miller et Al, p. 28).
1.1. Describing what the test measures to AVOID misinterpretation WITH PARENTS (Miller et Al, p. 382).
1.1.1. Do no refer to aptitude or learning ability tests as intelligence tests as the term is emotionally charged and understood.
1.1.2. Aptitude or learning abilities tests measure learned abilities NOT fixed.
1.1.3. Do not say that the test results predict how a child will do in school as many factors determine success and these tests do not predict.
1.2. Letter Grades: Mist teacher will be responsible for assigning a letter grade to students. It should provide a measure of achievement and reflect student learned outcomes in the instructional goals and objectives. It should be based on valid measures of achievement and each component that is weighted in relevance of imrpotance.
1.2.1. PARENTS AND COMMUNICATION OF CLASSROOM ASSESSMENTS AND TESTS
1.2.1.1. Interpretation of test results should include an explanation of how the test results are to be used in the instructional program and what action could be taken by both teacher and parent to improve student learning and development.
2. The Measurement and Instructional Process (Miller et Al, p. 31).
2.1. Identify Instructional Goals: determining learning outcomes to be expected
2.2. Preassess learner needs: do the students possess the abilities and skills needed to proceed with instruction? Otherwise plan and modify instructional plans.
2.3. Providing Relevant Instruction: course content and teaching methods are integrated into planned instructional activities so that students can reach intended learning outcomes. (Monitor learning process and diagnose learning difficulties).
2.4. Assessing Learning Outcomes: Determine the extend to which learning objectives were achieved by students and make use of the information (improvement of instruction, reporting, results for administration).
3. Types of Assessment Procedures (Miller et Al, p. 35).
3.1. Depends on what we want to measure!
3.2. "Measurement of Maximum Performance: "What a person can do"
3.2.1. TESTS are defined by functional role in the classroom: Placement assessments. formative assessments, diagnostic assessment, summative assessment
3.3. Measurement of Typical Performance: "What a person will do"
3.4. Performance Based Assessment: such tests as short answer, essay, projects, labs, and presentations. Some advantages to these types of tests are that they allow students to use their critical thinking skills, demonstrate production of what they learned, and allows students to demonstrate complex, in-depth understanding.
3.5. Fixed choice tests are such tests as multiple choice tests, true/false tests, and matching tests. These tests have the students regurgitate the information that they have learned. Some advantages to them are they can be scored objectively, they can machine scored, and that they are highly reliable
4. Assessments should be valid, reliable, and usable (Miller et Al, p. 70).
4.1. Validity is concerned with the specific use of assessment results, the soundness, and the fairness of the interpreted results (Miller et Al, p. 71).
4.1.1. NATURE OF VALIDITY:
4.1.2. refers to the appropriateness of interpretation and the use of results. It contains a matter of degree and does not exist on an all or none basis. It is unitary concept and usually prepared by a professional organizations like APA, AERA and NCME (the Standards). Validity involves value judgement to which uses interpretations and uses of assessment results justified by supporting evidence. (Miller et Al, p. 73).
4.1.3. CONSIDERATIONS IN ASSESSMENT VALIDATION:
4.1.3.1. Content: How well the assessment represents the domain of tasks to be measured and covers important content, Construct: does the performance imply that the student understands the content , concepts and principles intended in responding to the tasks? Assessment Criterion Relationship: how well the performance on the assessment predicts future performance on the value measure. Consequences: How well the assessment results accomplishes intended purpose and avoids unintended effects. (Miller et Al, p. 24).
4.1.3.1.1. CLASSROOM INSTRUCTION and CONSIDERATIONS ABOUT ASSESSMENT & CLASSROOM ACHIEVEMENT (Miller et Al, p. 28).
4.1.4. FACTORS INFLUENCING VALIDITY (Miller et Al, p. 98)...to be considered in the classroom when delivering assessments!
4.2. Reliability refers to the consistency of assessment results (Miller et Al, p. 71).
4.2.1. TEACHERS TO CONSIDER!! unclear directions, vocabulary and sentence structure too difficult, inadequate time limits, construct under-representation, poorly constructed test items, test items inappropriate for the outcomes being measured, poorly constructed test items, test is too short, improper arrangement of items, identifiable pattern of answers.
4.2.1.1. TEACHING AND EVALUATING ASSESSMENTS
4.2.1.1.1. Questions to consider when teachers evaluate assessments (Miller et Al, p. 102)
5. Construct Validation
5.1. Construct validation is the process of determining the extent to which performance on assessment can be interpreted in terms of one or more constructs. (Miller et Al, p. 81).
5.2. -Does the assessment adequately represent the intended construct? - Is performance influenced by factors irrelevant factors that are irrelevant to the construct?
5.2.1. Construct irrelevant factors can lead to unfairness in interpretation of assessment results (Miller et Al, p. 81).
5.3. Predictive Validation Study ex: a test performance like Scholastic Aptitude achieves criterion performance or achievement test scores.
5.3.1. Can use correlation coefficients and scatter plots to show test criterion relationships (Miller et Al, p. 81) Example and information on Scatter Plots for tests. http://www.icoachmath.com/math_dictionary/scatter_plot.html
5.3.2. perfect positive correlation r=1, Moderate Positive correlation r=.60 and zero correlation r=0.00
5.3.2.1. INFLUENCING FACTORS ON CORRELATIONS
5.3.2.1.1. Characteristics measured, the spread of scores, the stability of scores and the time span between measures. (Miller et Al, p. 92)
6. Methods for Construct Validation
6.1. (1) Defining the domain to be measured, (2) analyzing the response process required by the assessment tasks, (3) comparing the scores for known groups (4) preparing scores before and after the learning experiment, (4) considering the scores with other measures (Miller et Al, p. 85).
7. Effects of Assessment and important questions educators should be considering when they give assessments (Miller et Al, p. 97)
7.1. Do the tasks match the important learning objectives, What you test is what you get mentality.
7.2. Is there a reason to believe that students study harder for the assessment preparation (students could be given understanding on what to expect on the assessment and how results will be used.
7.3. Does the assessment artificially constrain the focus of students' study?
7.4. Does the assessment encourage of discourage exploration creative modes of expression; it should avoid narrow and artificial constraints.
8. TEACHERS SHOULD PLAN VALIDATION STUDIES TO EXAMINE THE DATA TO GUIDE THE APPROPRIATE USES AND INTERPRETATIONS THAT CAN BE MADE FROM AN ASSESSMENT. SEE EXAMPLE ON Miller et Al, p. 104).
9. Availability of Comparable Forms: equivalent forms of the same test are desirable which can also make it possible to test students twice, if needed, in close succession with answers on the first test not influencing their answers on the second test.
10. Factors influencing Reliability of Assessments (Miller et Al., 128)
10.1. Number of assessment tasks (larger number of tasks = higher reliability).
10.2. Spread of scores: reliability coefficients are directly influenced by the spread of scores in the group being assessed.
10.3. Objectivity: the degree to which equally competent scorers obtain the same results and that there is no judgement involved.
11. Test (a form of assessment): A systematic procedure for measuring a sample behavior (Miller et Al, p. 28).
11.1. Norm Reformed Assessment: Test or other type of assessment designed to measure performance that is interpret-able in terms of individuals relative standing in a group.
11.2. Criterion Based Assessment: A teast or other tyoe of assessment that id designed ti provide measure of performance that is interprettable in terms of clearly defined donmain of learning tasks.
11.3. Non Referenced Test: selection of items of average difficulty and the elimination of of items that all students will likely answer correctly. Large domain of learning tasks, emphasis on discrimination, favors average difficulty.
11.4. Criterion Referenced Based Assessment: includes items that are directly relevant to learning outcomes to be measured without regard and without discriminating students (no alternations in regard to difficulty). Focuses on delimited domain of learning tasks, matches item difficulty to learning tasks.
12. Non measurement: Informal observation
13. Assessment Categories (Miller et Al, p. 42).
13.1. Informal vs. Standard Tests: Informal are those constructed by classroom teachers and standard are those constructed, scored by professional experts under standard conditions.
13.2. Individual vs. Group: Individual are administered on a one to one bases where as group is more than one.
13.3. Mastery vs. Survey: Mastery tests typically use criterion referenced interpretations where as survey emphasize norm referenced interpretations,
13.4. Supply vs. Fixed Response: Essay tests require examines to supply the answer and fixed response offers choices.
13.5. Speed vs. Power: Speed tests measure the number of items and examine can complete in a given time where as power has the order of items arranged in order of increasing difficulty.
13.6. Objective vs. Subjective: Objective have scoring terms exactly defined and subjective is where the scores are influenced by judgement.
14. Use in Classroom Instruction (Miller et Al, p. 37).
14.1. Placement assessment - measures entry behavior and is given prior to instruction.
14.2. Formative Assessment -monitors learning progress and is given during instruction.
14.3. Diagnostic -identifies causes of learning problems and is given during instruction.
14.4. Summative assessment - measures end of course achievement and given after instruction.
15. Reliability
15.1. Reliability is the measurement of how consistent various assessment scores are compared to one another. Reliability focuses on the notion that results of an assessment cannot be consistent; there are a myriad of factors to be considered when examining the results. For example, if the same exam is administered right after a previous one, individuals may not score as highly the second time around due to fatigue, temporary memory relapses, effort, emotional strain, and other intangible factors (Miller, et Al., p.109)
15.2. The basis of reliability in reference to assessments is dependent upon some key points: the results obtained by an assessment, but not the assessment instrument itself is what reliability refers to. Reliability always refers to a specific type of consistency, meaning that assessment results are not reliable in the sense of acquiring a measurement of a candidate's aptitude, but rather a generalization of how the individual performs. Reliability is required for validity, as an exam yielding inconclusive information cannot be used as an accurate measurement. Finally, reliability is measured primarily through statistical analysis, usually completed through the examination of two or more parties scoring the assessments (Miller et Al., 110 - 111).
15.2.1. Methods to estimate reliability and consistency (Miller et Al., 110 - 111:
15.2.1.1. Test Retest: give the same test twice to the same group with time interval difference then correlate the two scores.
15.2.1.2. Measure of Equivalence: give two forms of the test to the same closed group in close succession them correlate two sets of scores.
15.2.1.3. Split Half: give test once score two equivalent halves and correct correlation between halves by applying Spearman-Brown formula.
15.2.1.4. Measure of Internal Consistency: Give test once and apply the Kuder-Richardsn or Cronbach's formula.
15.2.1.5. Measure of consistency ratings/Inter-rater consistency: give a set if student responses requiring judgmental scoring to two or more raters and have them independently score and correlate the two scores.
15.3. Standard Error of Measurement
15.3.1. It assumes that each person has a true score on a test and if a person were tested repeatedly, the average of the scores would be approximately distributed around the true score. Although the true score can never be known, the standard error of measurement can be applied to a person's obtained score to set reasonable limits that give "confidence bands." Confidence bands help to interpret the score and give greater confidence that we have obtained the true score (Miller et Al., 121)
15.4. HOW HIGH SHOULD RELIABILITY BE? It depends on what we will use the assessment results to determine (Miller et Al., 132)
15.4.1. High reliability is demanded when the decision is important, final, irreversible and has lasting consequences for example select or reject college applicants.
15.4.2. Low reliability is tolerable when decisions are of minor importance, decision making is in early stages, reversible, confirm-able by other data and has temporary effects. Example: whether to review a classroom lesson.