International Bureau of Education
Tel.: +41.22.555.06.00
Fax: +41.22.555.06.46
Email

Content Section

Ten notes on learning assessment systems

Curriculum, learning and assessment

Details

In-Progress Reflection No. 5 on Current and Critical Issues in the Curriculum, Learning, and Assessment: by Juan Carlos Tedesco

Abstract: Juan Carlos Tedesco, a renowned and prestigious educator, shares his perspective on gauging and assessing learning. The author suggests that by taking a close look at the history of assessment and discussing the strengths and weaknesses often associated with it, steps forward can be taken in the field, understanding that a great many dimensions are involved, both outside and within the education system that are both material and cultural. This document furthers the understanding that by using a systemic approach to meet these ends, it then becomes possible to establish procedures which seek to promote higher levels of equality and social justice. On the other hand, an appropriate combination of external and internal assessment can ultimately improve the quality of education. We must stop considering evaluation as an end in itself and, instead, look upon it as a tool to serve the aims of social justice.

Keywords: Curriculum – assessment – systemic approach – social justice

Participate in the discussion via our In-Progress Reflections blog. Discover videos and expert comments.

Ten notes on systems for assessing learning processes
Juan Carlos Tedesco

Introduction
 
We are seeing a renewed debate on the mechanisms for gauging and assessing learning achievements. The range of stakeholders to be assessed and the dimensions of assessment widen, and discussion centres on the institutional designs in which their activities are defined. These notes seek to contribute to the debate from the perspective of a user of the products of assessment systems, on the basis both of my research activities and of my management and decision-making time as former Minister of Education of my country. Written for this purpose, these notes do not necessarily embrace all the features of an academic article. I, nevertheless, consider that the testimonial style of the text is more suitable for achieving the aim of promoting and sharing the debate.

The assessment and gauging of educational outcomes has a history deserving to be summarily remembered.
 
The origin of these mechanisms draws on the recognition that one of the major traits of the administrative culture of education systems is the low level of responsibility for the outcomes. This trait – usually underestimated in the criticisms levelled at assessment systems by the “progressive” academic world – has been functional in the “expulsion” of the most vulnerable social sectors. As a corollary of this cultural paradigm, failure in learning was blamed on the pupils and not the education system, which takes on special significance in Latin America since we are the region in the world with the highest repeating rates in basic education.
 
The low responsibility for the results was one of the dimensions on which rested the neoliberal discourse of the 1990s to promote the introduction of mechanisms for gauging outcomes of educational administration. The novelty of that approach was that it blamed the failure of pupils on the poor performance of the teachers. In this context, the assessment mechanisms appeared to be linked to a threatening message for teachers. In addition to threatening the teachers, however, the neoliberal approach was based on the assumption that information on the results would improve the quality of the demand for education and establish competitive relations between schools as the main mechanism of policies intended to improve the quality of education.
 
After more than 20 years of experience, some “painful” learning processes ascribable to the use of systems of assessment and gauging of results can be identified.
 
In the first place, we know that gauging results does not improve them and that competition between schools, far from improving the quality of the system as a whole, makes for more inequality, segmentation and inequity, particularly in compulsory education. In this respect, Chile’s experience is very illustrative.
 
Secondly, the measurements enabled us to ratify the existence of a powerful social determinism of the learning outcomes. But while the school is scarcely able to break the social determinism of learning outcomes, attention needs to be given to phenomena of countries that improved their conditions of social equity but failed to match this with more educational equity. In this respect, the two most interesting cases are those of Uruguay and Argentina. As we know, Uruguay is the country with the best social equity indicators of the region, and yet its results in education do not match these social advances. The high rates of failure at secondary school, where it has not proved possible to modify the traditional highly elitist design, is probably the most eloquent indicator of the difficulty that exists when it comes to reflecting social equity in educational equity. In Argentina, for its part, it is noteworthy that, despite the improved material living conditions of the population since the 2001 crisis, together with better material inputs for learning, no improvements have been recorded in learning outcomes.
 
Thirdly, there was an overestimation of the importance of the assessment tools for improving quality and equity. The results of the measurements, widely echoed by the media, have an enormous political impact not matched by the relative technical soundness of some of the tools used or by the capacity of the media to interpret the information correctly. The demoralizing effect of the dissemination of the results outweighs the mobilizing capacity to improve them.
 
Finally, we also learnt that we have to clearly identify the difference that exists between meeting demands and satisfying needs. The demand capacity is unevenly distributed and, while education policies are confined to meeting demands, there is a tendency to leave individuals in the place their capacity to demand can be practised. Transforming necessity into a demand is a complex and not automatic process.
 
Priority for teaching and learning strategies
 
The first general conclusion that we can draw from this summary historical analysis is that to improve the quality of education we have to assume that the focus must be on teaching and learning strategies and on their use by the stakeholders in the academic process (teachers, pupils, family). Obviously this does not mean that we have to stop measuring results and investing in better material inputs for learning. We need to continue with those lines of action, but for these lines to bring about better results, we have to enter that space dubbed – sometimes disparagingly – by the specialists in education policies as the “black box” of the education process.
 
What is happening with the academic process?
 
In this respect we have to avow that – although the word is overused – we are up against a “crisis” of academic knowledge vis-à-vis the learning problems advanced today by the pupils, particularly (but not exclusively) those from socially vulnerable sectors. Noteworthy to this point, is the dialogue of George Steiner with a literature teacher in schools of the Paris outskirts, in which she confesses to him that she never had so many materials, training courses and supports for her work and never had so many difficulties in overcoming the learning problems of her pupils. Steiner answers her ironically that Goethe in his day had said: one who knows how to do something proceeds to do it, and anyone not knowing how to do it teaches. And he adds: one who does not know how to teach writes academic manuals.
 
To this reflection showing the present poor standing of education can be added the evidence afforded by the Talis report concerning the disparity between what the teachers say and what they actually do. While the vast majority of them profess a constructivist approach to learning, their practices are traditional. This is not the place for an exhaustive analysis of the evolution of academic thinking, but it has to be recognized that there has been a considerable trivialization of the constructivist approach and its application in the classroom. Furthermore, the constructivist approach has itself been enhanced with new inputs, and we are today seeing a renewed debate between the Piaget conception of the development of intelligence, which is defined as natural and spontaneous, and the approaches of Vigotsky, attributing a very important role to the teacher and the school. In any case, to focus on the academic process means making the teacher and the school an institutional space central to reflection on better quality.

To analyse the role of the teacher and the school, it is very instructive to observe what happens in those places where people manage to break the social determinism of learning outcomes.
 
Research on the subject indicates that in those places and with their stakeholders, there exist at least the following factors: confidence in the learning capacity of the pupils; capacity to draw up a coherent account of why they do what they do, have a project and promote the capacity in pupils to draw up a project; relatively high levels of self-esteem and responsibility for the results. All these factors make it possible to begin to speak of the need for what we could call subjectivity policies, referring to the teachers and to the pupils and their families. But to move forward in the analysis of these dimensions, it would be very important for them to be incorporated in the assessment instruments. How much confidence do teachers have in the learning capacity of their pupils? What level of responsibility are they assuming for the results? How much self-esteem? Assessing these dimensions will require experimentation, pilot tests and complex learning processes. However, the complexity should not cause us to continue postponing the entry upon a public policy realm that is assuming ever greater importance for forsaking the present resistance to change.
 
The subjectivity policies should confront the problem of how to generate commitment to social justice as part of professional teaching culture.
 
The ensemble of subjective dimensions mentioned in the previous section fall within a more general category concerning adhesion to social justice as a trait that should define the culture of professional teaching performance. In this respect, it is appropriate to recall the analysis of Rosanvallon on the adhesion to justice or, as Dubet puts it, the preference for inequality nowadays characterizing the culture of the new capitalism. According to these analyses, there exists a cognitive and ethical dissonance between the general condemnation of injustice, characteristic of the population as a whole, and the particular adhesion to the mechanisms that bring it about. But this dissonance is of special significance in the exercise of teaching, where we can appreciate the existence of an approach in favour of educational equity associated with professional practices that make for segmentation. Other professions have faced this challenge with relative success, which suggests that, just as doctors have the Hippocratic oath as a fundamental component of their professional culture, teachers should have a deep-seated commitment to the learning results of their pupils. Achieving this commitment is a high-level requirement not only drawing upon all the teacher-training mechanisms, but also upon the institutional designs.
 
We are in need of a cultural change.
 
All this analysis faces education policies with the challenge of a cultural change. However, what variables are involved in the processes of cultural change? The assessment mechanisms are one of them, but we must associate the use of assessment mechanisms with the meaning of education. In this context, it is worth discussing the idea that what is at stake in our countries is the need to incorporate the culture of assessment. Raising the subject in such terms is part of the overrating that surrounds the administration of measurements. It seems more important and mobilizing from the cultural point of view to place this discussion in the ambit of the culture of social justice, with assessment serving greater responsibility for results. This assumes the importance of the institutional dimension, of teamwork and of self-assessment, since the responsibility for results is collective and not individual.
 
Institutional dimension of the assessment mechanisms. Tension between autonomy and articulation, between independence and coherence.
 
The discussions on the institutional design of the body responsible for measurement are known. On one side, we have those who maintain the need to be able to call on independent bodies guaranteeing the objectivity of the measurements. On the other are those who speak out for the need to articulate the measurement with policy decisions and, for this purpose, it is appropriate to associate the measurement bodies with the place where decisions are made. The experience over these years indicates that, in actuality, the problem of dissociation exists both in the cases where those who assess do not belong to the public body and in the cases where everything falls on them. Furthermore, such dissociation, occurs not only between measurement and policies, but also between those who define the contents of curricula and those who define the assessment tests. An indirect indicator of this dissociation is found in the analysis of the data of the PISA tests regarding the replies of school principals on what teachers should teach and pupils must learn. A study, in this respect, found that, except in the case of Colombia, in the Latin American countries taking part in the test there is great distance between the principals informing what has to be taught (following what is prescribed in the curricula) and those relating to what the pupils should learn (following the learning achievement standards). The number of principals informing about the first dimension is significantly higher than the second, which reveals the existence of significantly more commitment to the culture of what the curriculum prescribes than to the learning outcomes. This dissociation usually finds expression at the institutional level, where there are frequent examples of tensions, conflicts or simply lack of linkage between the ministerial departments responsible for designing curricula and those concerned with designing the tests of what it seeks to assess.

These phenomena open the discussion about the processes whereby the contents of curricula and of the tests are defined. The curriculum designs are usually the outcome of collective negotiations, which are to a great extent corporative. Such discussions are attended by the teachers’ organizations and, as appropriate, the scientists involved in each of the disciplinary fields. The decisions on dimensions to be assessed, on the other hand, are generally a more closed product drawn up by technicians who, at least institutionally, enjoy a high level of autonomy. In both cases, there is a manifest absence of the voice expressing the learning needs of the pupils.

How far is it possible and necessary to open to public discussion both the curriculum designs and the assessment instruments? This is a question to be faced as the context dictates. In the case of curriculum designs, formulae have been tried, without much success, for involving the voice of various stakeholders by means of consultative councils. Assessment bodies are also in the habit of holding consultative councils of experts, playing a more or less active role according to the countries or times, but in any case not forsaking the ambit of education specialists. The need for articulation between the contents of curricula and the contents of the assessment tests cannot and should not be underestimated, since it is one of the ways in which the measuring of results can serve as a factor for improving quality. Such articulation is not only institutional but also political. From this point of view, the teachers and directors must be associated with and committed to the assessment process. International experience indicates that the climate in which assessment is carried out is an important factor for explaining the results. On one hand, we have cases of deep commitment to the test. The Cuban example in our region or the experience of the Asian countries, where activities come to a standstill on the day the assessment is conducted, can be mentioned in this category. At the other extreme, we have examples of indifference or even resistance to measurement, notably Argentina, where some of the administrations of the PISA tests drew a high percentage of total absence of replies.
 
A delicate point: the validity and reliability of the instruments.
 
This is a “taboo” matter in the discussions on measurement. Any attempt to question the validity of the instruments is seen as a manner of rejecting the results, which is so in quite a number of cases. On the other hand, however, the measurement technicians are averse to being assessed themselves. The at times somewhat esoteric language surrounding the design of the tests and the presentation of the results brings about distortions in the use of the information compounded, as already stated, by the precarious handling of the communication media. As a telling example of this situation, we can recall the analysis of Ruben Klein on the problems of comparability between the universes used in the PISA tests of Brazil in 2000 and 2009, a problem that is also found in other countries and affects the significance of any improvements in performance announced by those responsible for the tests. To this type of problem arising in tests intended to gauge cognitive achievements, we have to add those arising from the challenge regarding the design of assessment instruments for measuring other dimensions of performance, such as citizen skills, representations and values. In this respect, it will be necessary to surround these activities with high doses of experimentation and rigour in information for the public.

Closing comment

Nobody doubts that assessment and gauging are very important components of the process of improving quality, but it needs to be assumed that a great many dimensions are involved, both outside and within the education system that are both material and cultural. The complexity of the gauging process obliges us to adopt a systemic or integral approach with the definition of sequences of action suited to each context. The systemic approach involves placing the gauging of learning achievements in the context of the meaning of educational policy. Gauging in order to promote rankings and competition among schools is not the same thing as gauging to promote higher levels of equality and social justice. The systemic approach also makes it possible to establish procedures in which there is an appropriate combination of external and internal assessment, assessment and teacher training, and assessment and processes of improving quality, together with defining the times at which it is most appropriate to apply the said instruments. In short, the purpose is to stop regarding gauging as an end in itself and, instead, to look upon it as a tool serving the aims of social justice.
 
These notes were basically drawn up for the international seminar Más allá de las notas. Ampliando fronteras en evaluación de aprendizajes”, organized by the Education Quality Assurance Agency, Santiago de Chile, 15 June 2016.
Juan Carlos Tedesco, at present a professor at the National University of San Martín (Argentina), was Deputy Minister and Minister of Education of Argentina between 2005 and 2010, having previously served as a UNESCO official.
In this respect, we need to remember the pioneering studies of Ernesto Schiefelbein on repeating grade levels in Latin America.
 
Steiner, G. and Ladjali, C. 2003. Eloge de la transmission. Le maître et l’élève. Paris, Albin Michel.
Blais, M-C., Gauchet, M. and Ottavi, D. 2014. Transmettre, apprende. Paris: Stock.
See, for instance, Raczynski, D. y Muñoz, G. 2005. Efectividad escolar y cambio educativo en condiciones de pobreza en Chile. Santiago de Chile: Ministerio de Educación.
On the concept of subjectivity policies, see Juan Carlos Tedesco. Educación y Justicia Social en América Latina. Buenos Aires, Fondo de Cultura Económica, 2012. Cap. X.
Rosanvallon, P. 2012. La sociedad de los iguales. Barcelona, RBA.
Dubet, F. 2014. La préférence pour l’inégalité. Comprendre la crise des solidarités. Paris, Seuil.
In the case of Argentina, the National Education Act sanctioned in 2007 establishes the institution of a Curriculum Updating Council, with the participation of personalities representing the various cultural and social ambits. However, apart from its formation and a first meeting, the Council has not been used as a policy instrument for increasing social participation in defining the contents of curricula.
Klein, R. 2011. “Uma re-análise dos resultados do PISA: problemas de comparabildade”, in Ensaio, Vol. 19, Nº73.