1. CONSTRUCT VALIDITY (~ Scope and Specificity and Accuracy)
What precisely does this question test?
a. Can the competencies assessed be categorized (roughly or clearly) into LITERARY KNOWLEDGE, SKILLS, and DISPOSITIONS?
b. To what extent does the demonstration of "LITERARY KNOWLEDGE" require the thinking skills of RETRIEVAL/RECALL and/or UNDERSTANDING/COMPREHENSION?
c. To what extent does the demonstration of "LITERARY SKILLS" demand the higher-order thinking skills (or "critical thinking skills) of APPLICATION, ANALYSIS, SYNTHESIS, EVALUATION, and CREATION?
d. To what extent does the demonstration of "LITERARY DISPOSITIONS" require the sincere exercise of EMPATHY, PERSPECTIVE-TAKING, MORAL/ETHICAL REFLECTION, HONEST SELF-EVALUATION (through "personal response")?
e. Does the question deliberately lend itself to the demonstration of a RANGE of competencies (so as to differentiate the "stronger" from the "weaker" answers?
f. Does the wording of the question lend itself unintentionally to the demonstration of non-specific or non-specified competencies?
g. Does the question fairly assess what the teacher has taught for the test/exams?
2. CONSEQUENTIAL VALIDITY (~ Aims: Explicit & Implicit, Intended & Unintended)
What are the consequences of the test? Whose interests does the test serve?
a. Is the test used explicitly for FORMATIVE and/or SUMMATIVE purposes?
b. Will the test have an unintended or intended PUNITIVE effect on the test-taker?
c. Does the test unintentionally or intentionally discriminate against certain test-takers on the basis of CULTURAL IDENTITY and EXPERIENCES (related to language, ethnicity, class, age, gender, sexual orientation, religion)?
d. Does the ideological framework in which the test is embedded recognize, reward, and promote certain educational outcomes above others? (or simply: Does the test "objectively" privilege certain outcomes over others?)
e. To what extent will the examiner's (teacher's) feedback on the test enable or encourage further learning and improvement?
f. What might be the cognitive and affective effects of the examiner's feedback?
