A norming session is a group scoring of student work. Participating in a norming session allows faculty to come attain agreement on how standards of performance (in the form of rubrics) are applied to student work. It is also a great way to score student work for assessment data, because norming and having multiple raters for each piece of student work ensures the reliability of the assessment data.
Before the Norming Session
- Agree on a common rubric.
- Gather the student work and obscure any identifying information (for both student and teacher).
- Select representatives of exceptional, mid-range, and weak student work. Make a set of these three levels of student work for each scorer who will attend the Norming Session.
During the Norming Session
- Describe the purpose of the Norming Session (alignment of standards, assessment, etc.). Identify what the Norming Session is not for: faculty or individual student evaluation.
- Describe the student work that will be reviewed and the process for collecting it.
- Review each element of the rubric and the process for creating it.
- Distribute to each scorer a set of the representative levels of student work and ask them to score each independently.
- The facilitator the leads the scorers through a discussion of why assigned the scores that they did. The aim of this discussion is to reach consensus on the meaning of the rubric and how to apply it. A second set of representative work samples can be scored, if necessary, but it is usually not.
- After the discussion is complete and consensus is reached, unread assignments are distributed to the group and scored twice. If the two scores agree, the student work does not need to be scored again. If the two scores disagree, a third scorer will need to score the work.