The Slippery Concept of Mastery

The concept of mastery is a slippery concept. It seems so simple when an educator asks, “Give me a list of all the objectives (standards) the student has mastered,” or, “Let me see which students have mastered Standard 3.6a (Objective 3.6a).” Such lists are easy to produce from any assessment where each test item (question) is assigned to measure a given standard and where we can agree to some simple definition of mastery, such as 3 correct out of 4 questions on a given standard or objective is “mastery”.

Here is a simple scenario: We have a 60-item test where each of 15 standards is assessed by 4 items. The items (questions) are scored correct/incorrect. We say that 75% correct (3 out of 4) is called “mastery”. Our probable choices here were: 2 out of 4, 3 out of 4, or 4 out of 4. Most would say 4 out of 4 is too stringent, 2 out of 4 is too lax, whereas 3 out of 4 is just right! So, 3 out of 4 is what we will use. The rule is simple: if the student gets 3 out of the 4 items said to measure a given standard correct, that students is a master of that standard. We can have the following data:

Test Data — Number of Items Correct   (for 15 standards, 4 items each)
Standard	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	Total
William	4	2	3	4	3	2	2	2	2	1	1	2	0	2	1	31
Carol	3	4	4	4	3	3	2	3	3	2	2	1	2	0	1	37
Theodore	4	3	4	2	4	2	3	3	4	2	2	2	3	3	2	43
Alice	4	3	3	3	4	2	4	4	3	4	3	4	3	2	3	49

These data produce the following report:

Mastery State Report   (M= Master, n=Nonmaster)

Standard

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Total

William

M

n

M

M

M

n

n

n

n

n

n

n

n

n

n

4

Carol

M

M

M

M

M

M

n

M

M

n

n

n

n

n

n

8

Theodore

M

M

M

n

M

n

M

M

M

n

n

n

M

M

n

9

Alice

M

M

M

M

M

n

M

M

M

M

M

M

M

n

M

13

From this second table we can derive two list reports:

List Report of Students Mastering each Standard –  

Students mastering Standard 1: William, Carol, Theodore, Alice
Students mastering Standard 2: Carol, Theodore, Alice
Students mastering Standard 3: William, Carol, Theodore, Alice
     •
     •
     •
Students mastering Standard 14: Theodore
Students mastering Standard 15: Alice 

List Report of Which Standards each Student Mastered –  

William mastered the following Standards: 1, 3, 4, 5
Carol mastered the following Standards: 1, 2, 3, 4, 5, 6, 8, 9
Theodore mastered the following Standards: 1, 2, 3, 5, 7, 8, 9, 13, 14
Alice mastered the following Standards: 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 15

Or we could derive two alternative lists of nonmastery students and nonmastered standards. In either case, a teacher should know when and how to instruct the students.  

 Except that there are problems with this simple, intuitive scheme, and the problems are around the very concepts of mastery and the notion of being a master, given the nature of testing/assessment. Let’s talk about the concept of mastery first.

The mastery concept orininally came out of a particular learning model. Mastery is the desired end-state of each student under the “Mastery Learning” paradigm:

Mastery Learning is an instructional method that presumes all children can learn if they are provided with the appropriate learning conditions. Specifically, mastery learning is a method whereby students are not advanced to a subsequent learning objective until they demonstrate proficiency with the current one.

Mastery learning curricula generally consist of discrete topics which all students begin together. Students who do not satisfactorily complete a topic are given additional instruction until they succeed. Students who master the topic early engage in enrichment activities until the entire class can progress together. Mastery learning includes many elements of successful tutoring and the independent functionality seen in high-end students. In a mastery learning environment, the teacher directs a variety of group-based instructional techniques, with frequent and specific feedback by using diagnostic, formative tests, as well as regularly correcting mistakes students make along their learning path. (Mastery Learning, 2010)

Using the above description from Wikipedia, mastery is treated under Mastery Learning as a binary state, either you have it or you don’t. For some learning this might be true, but for learning most significant material, there are many degrees between “clueless” and “total mastery”. Mastery is seldom all or nothing. Even at the level of whole number operations in mathematics, a skill such as long-division is composed of the application of a number of rules in succession. Sometimes all rules may be applied correctly and sometimes not. Some rules are not needed in some situations, so a student may get some items right and others wrong. This is not necessarily a matter of forgetting, so much as it is incomplete learning.

Another example is in reading comprehension. A student may well be able to get the gist of a story or an article without understanding all the subtleties and implications of what was read. To summarize this issue: we usually have to specify how much of this skill or how broadly exhibited it has to be to demonstrate mastery. This opens up the concept of partial mastery. That opens up … and it goes on.

A problem I have in thinking about mastery in the context of student school-based learning is the meaning of the word itself. To me, mastery implies such a high level of achievement that many adults can not attain it, much less it is likely for students. This is the mastery of the Zen master, or a consummate concert musician, of a great philosopher, a famed artist, an acclaimed scientist, a Shakespeare or a Goethe, Bach, a  powerful orator, or of a master carpenter or a gifted businessman, a surgeon, and so one. (But leaving out these connative meanings, the concept of Mastery is still slippery.)

Even when some type of student mastery has occurred, forgetting often follows. Then there is re-learning; and forgetting again. This is to say, such a mastery state is not permanent. If I do a follow-up test of mastery, some students who were masters earlier may well have forgotten now. A mastery report or a list, as above, shows each student’s status at only one point of time and assumes it does not change. If a student masters standards 1 to 4, in October, and standards 5 to 8, in November, and standards 9 to 12 in December, can we necessarily assume in December that the students are masters of standards 1 to 12? Probably not. A cumulative mastery report is nearly certain to be wrong in some degree, because of forgetting.  

There are problems in the assessment of mastery, too. The way in which mastery is measured is an issue. There are often a variety of ways to discover whether a student has mastered subject matter. There are various item types such as true/false, single-alternative multiple-choice, multiple-alternative multiple-choice, matching, short fill-in the-blank, longer fill-in types, essay, etc. All of these may lead to very different mastery decisions for a given student.

Even within each specific type of assessment measuring a standard, certain items can be easier or harder than others. There can be very wide ranges of difficulties. Some items (questions) can be answered correctly by as many as 90% of the students assessed and others by as few as 10%. Clearly, 3 out 4 correct on four of the easiest items or 3 out of 4 on four of the hardest would show different proficiencies. But with a simple mastery scheme, each 3/4 is still mastery! (The way around this problem is to know how hard each individual item is, so that its difficulty can be taken into account to produce a true positioning of a student’s proficiencies on an underlying measurement scale. That is a whole different topic and deserves an entire future post, or two).

Finally, we have the inherent limitations of the assessment process itself. All measurement has error. Students are alert or sleepy, calm or anxious. Certain questions may confuse certain students. Classroom conditions can vary — tomorrow the student might do better (or worse). A student may mark the wrong answer by accident. One class of students may take an assessment on a Monday and another class on a Friday. Every test only tests a small sample of all the possible questions that could be asked about the assessed standards. The statistical reliability of a test of 4 or five items is very low, even if the items themselves are of excellent quality. The sources of possible error are vast.

CONCLUSION

Mastery itself is a slippery and a fuzzy concept. The concept of cumulative mastery is even more slippery and fuzzy. When the errors in assessment are applied to mastery, the situation is yet less well-anchored. And then, remember that mastery depends on the levels of difficulty of the items. A 3 out of 4 may be a reasonable criterion for some sets of items. For hard items, 2 out 4 might be more reasonable. Even 4 out of 4 might not be enough for super-easy items; for them, 9 out of 10 might be best!

In the end, we can say that there is some value in looking at “mastery” states in standards-directed assessment, provided that we understand the vast limitations. Perhaps we can regard reports of mastery states as hints or inklings of the truth, rather than the truth itself. If we combine this information with what a teacher already knows about a student, reasonable instructional decisions may be made.

David E. W. Mott

 REFERENCE

Mastery learning. (2010, August 14). In Wikipedia, The Free Encyclopedia. Retrieved 20:54, August 23, 2010, from http://en.wikipedia.org/w/index.php?title=Mastery_learning&oldid=378863498