Reidar Mosvold's Library tagged article →  View Popular

Although numerous calls have been made in the literature for valid and reliable assessments of teacher knowledge (e.g., Barnes,
1985; Lanahan, Scotchmer, & McLaughlin, 2004), it was only at the beginning of the twenty-first century that direct tests of mathematics teachers’ knowledge were constructed
independently by several research groups.

Beside COACTIV, tests on the professional knowledge
of mathematics teachers have been developed by Deborah Ball and colleagues in Michigan (targeting elementary teachers in the
United States; for results, see, e.g., Hill, Schilling, & Ball, 2004), by the Educational Testing Service (the Praxis series testing candidate mathematics teachers in the United States; see
Educational Testing Service, 2006), and within the MT21 project (investigating candidate mathematics teachers and trainee teachers in Germany; see Blömeke,
Kaiser, & Lehmann, 2008).

Construct validity (i.e., the extent to which an operationalization measures the concept it purports to measure) is a crucial
issue in psychometric research (Messick, 1988). Other criteria indicating psychometric test quality (i.e., reliability or objectivity) cannot inform on the meaning of the constructs measured; in the worst case scenario, interpretations of and conclusions drawn from test results may therefore
be invalid. Evidence is thus needed to confirm that the tests of PCK and CK applied indeed measure “pedagogical content knowledge”
and “content knowledge” (and not, for example, pedagogical knowledge or general intelligence).

For the Michigan group, the validity concept is of such importance that a whole issue of Measurement (Vol. 5, No. 2–3, 2007) was devoted to examining the validity of the Michigan tests on the professional knowledge of elementary
teachers in mathematics (with prominent discussants such as Alan Schoenfeld).

The COACTIV project on Professional Competence of Teachers,
Co
gnitively
Activ
ating Instruction, and the Development of Students’ Mathematical Literacy aimed at conceptualizing and assessing a broad spectrum of teacher competencies, personality variables, and work-related
variables in the context of secondary mathematics instruction. The project was funded by the German research foundation (DFG)
from 2002 to 2006

close relationship between COACTIV and PISA

Whereas the achievements of students and personality variables were assessed in PISA (right column), their teachers were
surveyed in COACTIV (left column).

Besides knowledge tests, a broad battery of newly developed (or adapted) instruments tapped teachers’ biographical variables,
motivational orientations, professional beliefs, and self-regulation (for an overview of the COACTIV instruments, see, e.g.,
Krauss et al., 2004; Kunter et al., 2007).

Tasks play a central role in teaching mathematics; much of the time allocated to mathematics lessons is devoted to tasks and their
solution. When appropriately selected and implemented, mathematical tasks lay the foundations for students’ construction of
knowledge and represent powerful learning opportunities (e.g., Jordan et al., 2008). Because this potential can be exploited by having students consider multiple solutions to specific problems (e.g., Silver,
Ghousseini, Gosen, Charalambous, & Font Strawhun, 2005), we assessed teachers’ knowledge of tasks by testing their ability to produce multiple solutions. To this end, four items
in our PCK test required teachers to list as many different ways as possible for solving a given task.


(2) 
Teachers need to work with students’ existing conceptions and prior knowledge. Because mistakes can provide valuable insights
into the implicit knowledge of the problem solver (Matz, 1982), it is important for teachers to be aware of typical student misconceptions and difficulties. In our PCK test, this aspect was assessed by presenting teachers with seven scenarios and asking them to detect, analyze
(e.g., give cognitive reasons for a given problem), or predict a typical student error or comprehension difficulty.


(3) 
Students’ construction of knowledge is often only successful with instructional support and guidance; for example, in the
form of explanations or representations. In our PCK test, knowledge of subject-specific instructional strategies was assessed by 11 items that required teachers
to explain mathematical situations or to provide useful representations, analogies, illustrations, or examples to make mathematical
content accessible to students (see Kirsch, 2000).

Note that the Michigan group uses a somewhat different terminology (basically, the two knowledge categories CK and PCK
are subsumed under mathematical content knowledge needed for teaching
(MKT), which is distinguished from subject matter knowledge or “pure” content knowledge; see also Leinhardt & Greeno, 1986; Sherin, 1996).

Thus, the members of the Michigan group evidently assume teachers to require not only content
knowledge, but also knowledge of explanations and knowledge of student errors.

The Michigan group used a matrix of three content areas by three knowledge dimensions as a theoretical framework for developing
test items (Hill et al., 2004; Ball, Hill, & Bass, 2005): The three content areas identified were (a) numbers/operations, (b) patterns/functions, and (c) algebra. The three knowledge dimensions were (a) common knowledge of content (CKC), which is the mathematical everyday knowledge that all educated adults should have, (b) specialized knowledge of content (SKC), which is thought to be teacher specific and acquired only through professional training and classroom experience,
and (c) knowledge of students and content, a dimension that links mathematical content and student thinking, thus covering knowledge on typical errors or student strategies.

3.5  
Corresponding findings of the Michigan group and the MT21 project group

These findings confirm two issues raised in the context of COACTIV. First, it is difficult to develop items that tap PCK alone,
given that CK seems to be one route to PCK. Second, there is another, very teacher-specific route to PCK that is not strictly
mathematical.

Before we answer
these questions, the following issues should be noted:

All these aspects suggest that it would not be justified at all to simply hire subject matter specialists for schools in the
expectation that their content knowledge would automatically enable them to deliver high-quality teaching that would in turn
foster student learning.

Qualitatively,
we documented the process of selecting items, and then carefully recorded the adaptations made for Ireland.

Both qualitative and quantitative methods were used to describe, and analyze the success of, the adaptations to items.

When making the
adaptations, we conducted a focus-group interview with four practicing Irish teachers and with one mathematician.

In addition
we conducted follow-up interviews with five (different) teachers after they had completed the test, to learn about their responses
to the items.

Quantitatively, we studied how the measures performed psychometrically in a pilot study.

convenience sample of 100 Irish primary teachers

about 90 minutes to complete the test

Teachers were required to complete the survey in the presence of the researcher and no time limit was imposed

A convenience sample of teachers
who were willing to spend one extra hour answering questions about the test was selected.

About seven general questions were asked to elicit teachers’ general opinion on the items, the extent to which
they related to the work of primary teaching, the authenticity of the characters in the questions, the clarity of the language,
and the questions that respondents are likely to find difficult. The teachers were also asked if they could think of teaching
situations, not reflected in the chosen items, where they draw on their mathematical knowledge. In addition, 16 questions
from the test were selected and interviewees were asked to state what answer they gave and why they chose that answer and
not another answer. These interviews were recorded and transcribed.