Paper presented at the 5th Annual Congress of the Association for Mathematics Education of South Africa (AMESA), Port Elizabeth, 5-9 July 1999.

ASSESSMENT IN SUPPORT OF EQUITY

Liora Linchevski, The Hebrew University of Jerusalem & Malati
Bilha Kutscher, The David Yellin Teachers College, Israel
Alwyn Olivier, University of Stellenbosch & Malati

TAP (Together and APart) is an ongoing project where mathematics is taught in mixed-ability classes in Israeli junior high schools (Linchevski & Kutscher, 1998; Linchevski & Kutscher, 1996). The Malati (Mathematics learning and teaching initiative) project in South Africa bases its work on the same together-and-apart principles. One of the main goals of both projects is to support equity. In this paper we first discuss our view of equity, then we describe the TAP assessment model and its role in supporting equity, and finally we present some of our main findings regarding assessment in support of equity.

 

Our view of equity

Research literature indicates that the most commonly employed learning system, the system in which students are allocated to learning groups according to their prior knowledge and abilities (henceforth called ‘sorting systems’), produces non-equity in two main areas: Firstly, the gap in mathematical achievement between learners who are allocated to different levels widens far beyond what is to be expected on the basis of the initial differences between them (e.g. Cahan & Linchevski, 1996). We will call this the added gap. Secondly, it is suggested that this usually happens not because of the higher-track students’ gain, but because of the lower-track students’ loss (Gamoran, 1993).

In our view, equity is a learning environment in which these elements of non-equity are eliminated. This elimination is not achieved at the expense of the mathematical knowledge of any group of students, whether higher- or lower- achievers. We believe that all this can be achieved when the learners are members of a fruitful, diverse mathematical community. However, in our view such a community can be mathematically productive and endure to the satisfaction of all its members only if on one the hand its members have sufficient shared mathematical knowledge to make meaningful interaction possible, and on the other hand there is enough space for all members to express their mathematical diversity. In our view all this can be realized only if the learning environment is designed to both ‘acknowledge diversity’ and to ‘ignore diversity’.

By acknowledging diversity we mean allowing all students to fulfill their mathematical needs, abilities and preferences. Thus, acknowledging diversity should lead to the construction of an assessment model that accommodates differences in the ways learners think about and display mathematical understanding. It should lead to the design of assessments that capture the whole spectrum of students’ performance and level of understanding regardless of any "objective" criteria. (Mathematical Sciences Education Board, 1993).

By ignoring diversity we mean that sometimes we "do not accept" diversity in students’ exit points: certain essential mathematical knowledge (henceforth called ‘Indispensable Mathematical Knowledge’ or IMK) should be owned by all students notwithstanding the acceptable diversity in other parts of their mathematical knowledge. Thus ignoring diversity should lead to the design of assessments that capture the students’ IMK. What is Indispensable Mathematical Knowledge? If we want to give the learners a ‘fair’ chance to succeed in mathematics in the long term, and not only in the short term, we have to be able to discriminate between cases in which legitimizing a wide range of "different levels of mathematical knowledge and sophistication" (ibid, p. 92) is the right approach and cases in which it is, eventually, at the expense of a fair chance to cope with future activities in mathematics. The latter cases are those in which Indispensable Mathematical Knowledge may not have been adequately constructed, knowledge without which the learners will gradually be filtered out of higher levels of mathematics and eventually out of mathematics itself. IMK, to a great extent, is the mathematical knowledge that enables the heterogeneous mathematical community to conduct further fruitful mathematical discourse to the satisfaction of all its members. Equipping each student with IMK supports equity by keeping the gates open to a heterogeneous study group. Otherwise, as happens in many sorting systems, these students are gradually assigned to an impoverished learning environment. This process is the beginning of non-equity that leads to the added gap in the students’ mathematical knowledge. Thus, in our approach, there are points in the learning process where certain mathematical knowledge is expected of all students. This implies that we should incorporate components for following-up students’ IMK into our assessments. In our view, it is the teachers’ duty to identify IMK as well as to identify students whose IMK is insufficient, and to take responsibility for providing these students with repeated opportunities for acquiring it. (Naturally, IMK might vary among different educational systems.)

Acknowledging diversity while ignoring it, two ostensibly contradictory goals in our perception of equity, is achieved in our teaching model by alternating between two basic types of learning groups: heterogeneous groups and homogeneous groups. The various heterogeneous groups are generally engaged in the same activities (Together), while the homogeneous groups are generally engaged in different activities (Apart). The heterogeneous groups provide a shared rich learning environment. The learning plan is designed to optimize opportunities for heterogeneous interaction. In the homogeneous groups the students’ differential needs are addressed: The students are provided, for example, with additional opportunities for revisiting IMK, for orientation to a next topic, for enrichment, for involvement in unshared topics, for experiencing multivocal mathematical interaction and more (for more details see Linchevski & Kutscher, 1996). Homogeneous groups are set up only if the differential needs of the students cannot satisfactorily be addressed in the heterogeneous setting. Assessment is designed and its results analyzed to provide the teacher with the necessary information for structuring the various settings described above, and for planning appropriate interventions for these settings. Assessment is designed to, on the one hand, accommodate, evaluate and reward equally the diverse thinking processes and mathematical knowledge that different students display, as well as the diverse activities in which the different students are involved, and on the other hand to guarantee that IMK is followed up. In the following section we describe the TAP assessment model and its role in realizing our equity goal.

The TAP assessment model: Informative assessment

Traditionally assessment was viewed as a tool for determining what students know, its results were used to justify the allocation of students to different levels of mathematics learning and eventually to filter many of them out of learning mathematics. While it becomes more and more widely accepted that this filtering role of assessment must change, the search for ways to fulfil it is still in its infancy. Rethinking the role of assessment in mathematics education has recently led to the distinction between different forms of assessment: summative, formative and diagnostic assessment. We reject this distinction. We believe that distinguishing between ‘formative’, ‘diagnostic’ assessments and the like on the one hand and summative assessment on the other implies that these are two independent assessment categories. At best the diagnostic and formative assessments modify the teaching plan to prepare the students for "the moment of truth", the moment when they sit for the "final" summative assessment. We believe that every assessment regardless of ‘name’ can and should be a continuous communication process between teacher and learners, providing information for students’ mathematical needs, and thus serving as an integral part of the learning process and teachers' planing. For instance, if teachers judge some knowledge to be IMK, teachers should use every opportunity to provide their students with further opportunities to revisit this IMK. From this perspective there should be no final, "summative" test. Every test is first and foremost diagnostic. Of course it contains the summative element; however the summative role should be minor.

We claim that if the education system is oriented towards limiting students’ access to certain educational routes, all assessments will vindicate the filtering process. However, if the education system is oriented towards leaving access to all routes open for students, every assessment will be used as a constructive informative device (see also Ridgway & Passey, 1993). If, for example, a student’s performance on a test – even on the most traditional "summative" test – has indicated that he/she has inadequate mathematical knowledge, does this mean that for the rest of his/her "school mathematical life" he/she will not have another opportunity to revisit this topic? And what if this mathematical knowledge is IMK? For example, if this studeent showed inadequate knowledge in linear equations in one unknown and we consider this topic to be IMK and despite all this we did not offer the students an opportunity to consolidate this IMK, does this not violate the equity principle? We argue that we have thereby prevented this student from having a fair chance to cope with subsequent topics to be studied, for example system of equations.

We are therefore committed to seeing assessment first and foremost as a diagnostic tool. In our approach even the most traditional test serves as a first step in the diagnosis process. It enables, for example, teachers to identify students in need and through direct communication with them to try to analyze where their thinking processes have misled them and to modify their teaching plan by providing students with opportunities to consolidate their IMK. It also enables teachers to identify the students who have acquired a sufficient level of understanding in the topic just assessed and to offer them the opportunity to be engaged in other challenging mathematical tasks.

The teachers are not expected to be experts in developing sophisticated diagnostic tasks However, they are expected to be able to analyze students' responses to any task and to be willing and able to communicate directly with identified students. In the words of one of Malati’s co-ordinators:

I am finding with my TAP efforts … that the diagnostic assessment often results in a diagnosis for further diagnosis!…This morning the teacher and I worked with such pupils (identified during the analysis of his assessment test), while the rest of the class continued with an extra activity. I had suggested a list of possible questions to diagnose problems with the concept of a fraction ... I suppose this boils down to a second diagnostic 'test' but it was useful to be sitting WITH the group while they responded. In more than one case it was enlightening to ASK the pupil what they meant by their response.

The TAP assessment model is a combination of several components, both formal and informal. We will briefly describe the various components.

Evaluation after core material: At the end of a topic, or part of a topic (which is included in the core material), a uniform test is given to all students in the class. This test is generally composed of two parts:

  1. The basic component, which comprises about 70% of the test, consists of questions of knowledge and application on a level corresponding to the tasks that most of the students had successfully completed during the study of the topic. In this part we take great care to assess what we believe the students can realistically achieve under given conditions (which are, of course different in different systems) rather than assess what we would have ideally hoped that they might achieve (Johansson, 1993).
  2. The advanced component, which comprises about 30% of the test, consists of questions on a higher level. The two parts of the assessment can be presented consecutively or the questions of the two levels can be interspersed.

A similar idea is presented by Bodin (1993). He suggests differentiating between two separate kinds of tests: "required" tests aim "to cover closely the official list of required proficiencies" while "complementary" tests assess "deeper skills" (ibid, p. 128). Although TAP’s tests after core material present both types of tests in one test-paper, the rationale in both cases is the same: Abilities in mathematics are not necessarily hierarchical. This type of assessment enables us to identify students with high or low correlation between the two parts of the tests. Especially intriguing are those students who show higher competence in the advanced component than in the basic component. All data is analyzed, informs the teaching plan, and is expressed in appropriate activity design and class organization.

Evaluation during core material: Short tests and informal assessments, generally aimed at evaluating both processes and products, take place during the learning of core material according to the teachers’ discretion. The purpose of these evaluations is to give teachers an idea of their students’ competence in small segments of the core material, so that when necessary, the teachers can modify their teaching plan to address the needs of their students. Immediate response to the students’ needs frequently helps avoid the creation of added gaps and enables the students to be profiting learners in heterogeneous groups (e.g. Linchevski & Kutscher, 1996).

Informal assessment of topic to be studied: The model suggests that before studying a certain topic, teachers might determine students’ prior knowledge in the topic through informal means such as class discussions, and thereafter plan the class activities accordingly. For example, the teachers might consider offering alternative activities to students who already have substantial knowledge in the topic.

Evaluation after differential topics: Whenever different students, or groups of students, engaged in different topics are to be evaluated, different assessments are prepared according to the nature and role of the topic. For example some students may hand in a project, others might take a written test, while still others might be assessed informally.

Evaluation after consolidation: IMK is the kernel of the basic component of core material. As previously discussed, students are given repeated opportunities to revisit IMK. Thus, for students who have been identified as being deficient in IMK, special sessions (henceforth called ‘consolidation’) are organized to consolidate their IMK. When consolidation follows ‘evaluation after core material’ the students who participate in this consolidation are sometimes re-tested on the basic component. If their achievements improve, their new grades replace those they obtained on the basic component of the original test. This ‘grade replacement’ motivates students who may not have been so mathematically competent in a certain area, to attempt to improve their understanding and proficiency. Rather than perceiving tests as a summative evaluation of a specific topic, the students consider TAP assessments as stepping stones towards further improvement.

Constructing a profile

Each assessment generates new information about the individual student and about the class as a whole. This information is then analyzed and organized into a profile that maps the students’ responses on the items assessed. The profile may portray processes or attainments depending on how the assessment items are constructed and on what the teachers find more informative for their teaching plan at that moment. Every profile provides information that is diagnostic, formative and summative with each of these three aspects informing the teaching plan to support equity in its own way.

The diagnostic aspect allows teachers to trace students’ thinking processes, to identify students who seem to be experiencing some cognitive obstacles, to identify students whose IMK is still in need of consolidation, students with particularly high mathematical skills in the particular mathematical area, and the like.

This provides teachers with essential information for planning future activities for the class and as such serves as the formative aspect. With this information in hand the teachers can better decide which activities to present in the whole-class setting, which in the heterogeneous groups and what to plan for the homogeneous groups.

The summative aspect enables us to roughly identify the high, middle and low achievers and to get a general idea of the mathematical forces in the class. It enables us, if needed, to communicate results of assessment to "outsiders" (parents, other people in the school etc.).

Some TAP assessments are designed to include items tagged ‘basic’ or ‘advanced’ (as described above). Students’ responses to the two different groups of items are analyzed and recorded in the profile separately, thereby providing an additional angle for looking at the students’ and class’s mathematical profile. Armed with all this profile-generated information, the teachers continuously modify their teaching plans, the designing of activities and the appropriate settings for presenting them. As pointed out, we believe that, generally, heterogeneous settings benefit all students. However, we also believe that on certain occasions equity implies working in homogeneous groups. The information generated by the profiles offers teachers greater insight into their students’ thinking processes, which the teachers can use to construct these different settings more optimally. The profile informs the design of many learning events. For instance, in the case of IMK consolidation, the teachers might decide on homogeneous grouping for a short while, where groups of students who are not engaged in consolidation are involved in different activities. Or the teachers might decide that engaging all students in specially planned activities in heterogeneous groups, might be beneficial for all students and thus the preferred mode of IMK consolidation.

Thus the TAP assessment model provides us with tools both to acknowledge diversity and to ignore it. To acknowledge diversity and reward it by enabling us to analyze the specific class at hand and prepare appropriate learning events in which all expressions of mathematical knowledge are equally valued. To ignore diversity by guaranteeing that diversity is not at the expense of students’ right to achieve IMK and by believing that all students can indeed achieve IMK. In this way we are able to satisfy the conditions for enabling a diverse mathematical community to endure to the satisfaction of all members where there is both sufficient shared mathematical knowledge making possible meaningful interaction and also enough space for all members to express their mathematical diversity.

Research questions

Our main research questions are:

I. Are teachers able to create a learning environment according to principles underlying TAP and what is the role of TAP assessment in this implementation?

II Does the TAP learning environment support equity?

More specifically:

a) Is the added gap found in the sorting system prevented in the TAP environment?

b) Is the prevention of the added gap at the expense of the mathematical knowledge of any group of students?

c) Do lower-achievers, whose IMK was identified with the help of TAP assessment and who had the chance to consolidate there IMK while full members of the heterogeneous class, achieve better than lower-achievers in the sorting system?

We will now briefly address each of these questions.

 

I. The first research question - A tale of two countries

The research population is drawn from 50 Israeli junior high schools participating in the TAP project and from 7 South African schools participating in the MALATI project. In the Israeli case we have complete data; in the South African case we have initial observations.

Israel: The Israeli teachers who joined the TAP project underwent a process of change, slowly resolving their feelings of ambivalence: They believed that students’ ability is only one factor in determining their performance and that the human learning environment and teaching methods play a major role in students’ development. In fact, they resonated with most of TAP’s goals and principles. However, they were skeptical of TAP’s – or any other project’s or program’s – ability to answer all students’ differential needs. They were also unsure of their own ability to implement a teaching plan such as TAP. However, most of the teachers were willing to try, and they soon found themselves quite naturally backing-and-forthing between the various learning settings and activities designed to address the students’ differential needs. Thus, it seems to us that working differentially according to TAP did not pose a major psychological or conceptual obstacle for these TAP teachers. Within quite a short time-span (usually after a year or two) these teachers indeed believed that learning in a heterogeneous class was not only desirable but also possible (Linchevski & Kutscher, 1998).

However, when it came to the TAP assessment model, the teachers’ process of change was much slower: They struggled, and many of them are still struggling to accept and adopt it in all its facets. They could see the value of informative assessment, and indeed used it as a diagnostic tool for addressing the students’ needs, but they did not believe that it could also serve as summative assessment. They believed – and claimed their students intuitively sensed – that there is some '"innate order" of students’ abilities in their class and that this order should be reflected in the reported evaluation. They believed that the role of summative assessment is to generate this innate order. However, working through the TAP principles and beginning to implement them in their classes, they increasingly felt dissonance between their "new" practice and traditional summative assessment. If diversity is the rationale of the teaching plan should it not also be reflected in the assessment model? Teachers gradually came to seeing this point of view. They began to realize that if the essence of TAP is indeed the "together" and "apart" – the IMK and the differential topics – then on certain occasions different students should be assessed against different criteria and even on different topics. However, when it comes to IMK all students should be assessed against the same criteria.

Nonetheless, this does not mean that even now all TAP teachers completely agree with the TAP assessment model. The emergence of a new practice does not immediately replace its predecessor. The "old" and "new" practice in fact live together, side-by-side, during the process of change and the new TAP teacher has to find ways "how to coordinate the 'old' and 'new' into a totality" (Mellin-Olsen, 1993, p. 152).

Nevertheless, TAP has been successfully implemented in the project schools for two main reasons. Firstly because the TAP counselors and the TAP schools in which these teachers teach, give these teachers enough space and support, allowing them to progress in this process of change (beliefs, attitudes and practice) each at his/her own pace. Secondly, and most importantly, being aware that different teachers are at different stages in their process of change, TAP provides each teacher with tools he/she can handle for addressing diversity while teaching mathematics effectively.

South Africa: The basic TAP principles and assessment model are only into its second year of implementation in the MALATI project. It seems to us that the process of implementation is taking a very different route from the Israeli one (for more details, see Bennie, Olivier & Linchevski, 1999). For many teachers in the MALATI project schools equity in class meant that all students should be taught the same mathematics at the same time throughout the year. This perception of equity is, of course, at odds with the basic principles of TAP –sometimes "together" and sometimes "apart"; IMK for all and differential topics for some. Indeed, initially project teachers did not seriously implement the TAP principles – until after the first written tests. The first breakthrough came when the teachers, under the guidance of their MALATI co-ordinator, constructed an informative assessment and its profile. Studying the students’ responses recorded in the profile, they saw how it portrayed the diversity in their class as reflected in the assessment items. Analyzing their own class profile was the second major step. This analysis triggered discussions in which alternative interventions were considered. Questions such as what are IMK items, which students are at risk, which difficulties should be addressed immediately, later or completely ignored, which students need no review and should be engaged in alternate activities while others are involved in consolidation, were all discussed. These discussions were then translated into actual lesson-plans; learning together and apart became relevant.

Obviously this breakthrough did not make an immediate change in the teachers’ practice. Most teachers did not feel ready to implement such an assessment process and its consequent teaching implications independently. Moreover, many teachers felt that profiling is tedious and impractical, to be implemented only on ‘important’ occasions. Preparing an informative assessment seemed to them a task reserved for ‘professionals’, far above what can be expected from a regular schoolteacher. Our major challenge was (and still is) to help the teachers to realize that they are not expected to design sophisticated diagnostic assessments. Our challenge is to help them realize that the assessments they prepare can be used as a first step in the diagnostic process. For example, in one school the teachers administrated a test they had prepared. They did not specifically pre-design the test items to generate illuminating diagnostic information about students’ thinking processes. However, studying the test profile the teachers very quickly realized they were able to identify which items were posing difficulties and to identify students who performed poorly on these items. They did not feel that the profile provided them with enough information for better understanding their students’ difficulties, but they realised that the next step should be to address these students’ difficulties, and this should be done in a group apart from the others. So, encouraged by the project co-ordinator, the teachers sat with these students and simply began to talk to them. They soon found that this in itself was a rich source for diagnosis that they could now use when addressing these students’ needs. In the meantime the rest of the class was engaged in different activities (apart).

Teachers in another school had a different starting point: they ventured to plan and construct a more informative assessment so they could gain more insight into their students’ understanding directly from the assessment and its profile.

Evidently, TAP assessment was the trigger for these teachers’ change and continued to be the springboard for their professional development in general and for implementation of TAP principles in particular. Each additional assessment further enriched the teachers. It exposed them to their students’ thinking processes previously unnoticed by them, to the diversity in the mathematical knowledge of their students, and to the realization that in order to cater for all these diverse needs the mathematics class had to be organized very differently. We are inclined to believe that for these teachers TAP assessment was a beginning for their change: it gave TAP principles a chance to be implemented.

From both the Israeli and the South African case it is clear that teachers are indeed able to create a learning environment in line with TAP principles and that TAP assessment plays a major role in this process. The TAP assessment model and its teaching plan are intertwined. TAP’s goals can be achieved because there is consistency between its approach to learning and its approach to assessment. The TAP assessment model is designed to support our perception of equity. We do not believe that it is possible or appropriate to look at alternative modes of assessment without relating them to specific teaching approaches and to the rationales and assumptions underlying these teaching approaches (Clarke, 1988).

II. The second research question: Does the TAP learning environment support equity?

Our criteria for examining if equity was achieved is formulated in research questions II a, b & c. To answer these questions, a study that used a quasi-experimental and a random experimental design was conducted in TAP’s schools in Israel. A full report of this research is given in Linchevski and Kutscher (1998). The results of our research show that the added gap, created in the sorting system, was prevented in TAP schools and this prevention was not at the expense of the mathematical knowledge of any group of students. The results of this research also indicate that the average and weaker students’ achievements in TAP’s heterogeneous classes were significantly higher than those of the parallel students in the sorting system; while there was no significant difference between the achievements of the strongest students in the two systems. Since our findings give a positive answer to our criteria for achieving equity, we may conclude that the TAP principles and assessment model do support equity.

References

Bennie, K., Olivier, A. & Linchevski, L. (1999). Everyone can learn mathematics: Addressing the fundamental assumptions and goals of Curriculum 2005. This Proceedings.

Bodin, A. (1993). What does to assess mean? The case of assessing mathematical knowledge. In M. Niss (ed.), Investigations into Assessment in Mathematics Education, pp. 113-145. Kluwer Academic Publishers, The Netherlands.

Cahan, S. & Linchevski, L. (1996). The cumulative effect of ability grouping on mathematical achievement: A longitudinal perspective. Studies in Educational Evaluation, 22, 1, 29-40.

Clarke, D. (1988). Mathematics curriculum and teaching program: Professional development package: Assessment alternatives in mathematics. Curriculum Development Center, Canberra, Australia.

Gamoran, A. (1993). Alternative uses of ability grouping in secondary schools: Can we bring high-quality instruction to low-ability classes. American Journal of Education, 102, 2-22.

Johansson, B. (1993). Diagnostic assessment in arithmetic . In M. Niss (ed.), Investigations into Assessment in Mathematics Education, pp. 169-185. Kluwer Academic Publishers, The Netherlands.

Linchevski, L. & Kutcher, B. (1998). Tell me with whom you’re learning, and I'll tell you how much you’ve learned: Mixed-ability versus Same-ability grouping in mathematics. Journal for Research in Mathematics Education, 29(5), 533-554.

Linchevski, L. & Kutscher, B. (1996). The TAP Project - Teaching mathematics in heterogeneous classes. Paper presented at ICME8, Seville, Spain.

Matematical Sciences Education Board, National Research Council (1993). Measuring What Counts: A Conceptual Guide for Mathematics Asessment. National Academy Press, Washington. DC.

Mellin-Olsen, S. (1993). In M. Niss (ed.), Investigations into Assessment in Mathematics Education, pp. 57-73. Kluwer Academic Publishers, The Netherlands.

Ridgway, J. & Passey, D. (1993). An international view of mathematics assessment – through a class, darkly. In M. Niss (ed.), Investigations into Assessment in Mathematics Education, pp. 57-73. Kluwer Academic Publishers, The Netherlands.