Saturday, May 1, 2010

Subjective evaluations

Schools "mark" and "grade" students, as a way of "evaluating" their "grasp" of the material in the curriculum. And in subjects such as arithmetic, spelling, historical facts like dates and names of leaders, and in the geographic names of places and landforms....this makes sense.
It is a convenient way to both document and to encourage improvement, of some cognitive skills such as memory and language development, reading, listening, note-taking, as well as conforming to the political need to illustrate, in some manner, to the parents, the trustees and the society at large, that the student is "normal" and the school is also "normal" or at least demonstrating some level of effectiveness.
Many important decisions are then based by all participants on the "scores" of these tests, exams, homework assignments, classroom behaviour etc. For the student, "I seem to be doing well in this subject, but not so well in that one. So perhaps I might like to spend more time in subjects that I seem to be doing well in." For the parent, "S/He seems to score better in this than in that area. I think s/he ought to be doing more extra work in the subject where the scores are weaker." And for the school, "Perhaps, based on these scores, this student would be advised to move in "this" or "that" direction, academically, vocationally and eventually personally."
However, while there is some recognizable agreement in some subjects, at some levels, in the validity and in the predictability of academic scores, there is also a significant set of variables that can be termed "subjective" for both the student and the evaluator.
Such things as:
  • home conditions where the student must prepare for evaluations;
  • relationship between student and evaluator
  • degree of agreement between student's answers and evaluator's preferred answers
  • appearance of the test answers (writing neatness, letter formation, page crispness)
  • health of the student in preparing and performing the evalution
  • mental and emotional state of the evaluator when reviewing the evaluation instrument
  • previous scores both for this evaluator and for this student
  • capacity and ability of the evaluator to set clear expectations in the instrument
  • ethnicities of both evaluator and student, and their relative compatibility

This list, while not comprehensive, at least demonstrates the multiple factors that can, and often do, impact those "scores".

And yet, personal reputations rise and fall depending on those school scores. And the process continues into adult life. We have placed some individuals in positions from which they "evaluate" the performance of others: in the law, in the graduate schools, in the hiring offices, in the journalism profession, in public performances like athletics, politics, economics/finance where the scores determine salary size.

Recently, there has been a public outcry about the disconnect between CEO bonuses and corporate performance, especially in the financial sector of the economy. And yet, there is little discussion about the multiple contaminations of other scores relative to income in other fields.

Is Rush Limbaugh, for example, worth the reported $40m salary he receives. Hardly. And yet somebody, or some corporation, is prepared to pay, for the "ratings" he brings because those ratings are the foundation for advertising rates, also under serious scrutiny, for company revenues.

We have all bought in to a system which works modestly effectively in the education process, as our way of making serious judgements "objectively" while failing to acknowledge the high degree of subjectivity and the conflicts of interest of those making the judgements.

An alternative system is not easily accessible. Yet, we would all be helped if we could begin to acknowledge our own "biases" and "self-interest" and personal component in every evaluative statement we make of another, regardless of the arena of the performer.

In most cases, we are merely "scoring" our sentiments, feelings, and preferences while refusing to acknowledge our contamination of the process.

No comments:

Post a Comment