Acessibilidade / Reportar erro

School Improvement Trajectories in Chile: a case study

Abstract:

Progress in educational improvement research has generated consensus about the need to pay closer attention to the context within which it develops as well as to the multiple levels and participants involved in it. Within that analytic framework, a study was conducted to identify how the improvement trajectories experienced by schools can help understand their current scenario of strengths and weaknesses. Eight case studies were carried out in different types of Chilean schools, considering their socioeconomic and academic achievement characteristics. First, this study assessed the presence of educational improvement attributes in the schools. Following this, in-depth analyses of the trajectories of the four schools that attained the highest scores in the assessment were conducted. With the results obtained, an explanatory model was generated to describe the educational improvement trajectories of the schools that were analyzed. This model involves three stages (bring order; achieve organization, and improve teaching-learning), each of which includes one main process.

Keywords:
Educational Improvement; Improvement Trajectories; Educational Evaluation; Case Study; Educational Management

Resumo:

O avanço na pesquisa sobre melhoria educacional tem gerado consenso acerca da necessidade de prestar mais atenção ao contexto no qual se desenvolve, bem como aos múltiplos níveis e participantes envolvidos. Nesta estrutura analítica, foi conduzido um estudo para identificar como as trajetórias de melhoria vivenciadas por escolas podem ajudar a compreender seus pontos fortes e fragilidades. Foram realizados oito estudos de caso em diferentes tipos de escolas chilenas, levando em conta suas características socioeconômicas e de desempenho acadêmico. Inicialmente este estudo avaliou a presença de atributos de melhoria educacional nas escolas. A seguir, foram conduzidas análises em profundidade das quatro escolas que alcançaram os escores mais elevados na avaliação. A partir dos resultados obtidos, foi elaborado um modelo explicativo para descrever as trajetórias de melhoria educacional das escolas analisadas. Este modelo envolve três estágios (trazer ordenamento, alcançar organização, e melhorar o ensino-aprendizagem), sendo que cada um inclui um processo principal.

Palavras-chave:
Melhoria Educacional; Trajetórias de Melhoria; Avaliação Educacional; Estudo de Caso; Gestão da Educação

Introduction

In this paper we present the results of an investigation that aimed to recognize trajectories of improvement in Chilean schools of different characteristics. We sought to identify patterns that describe the paths of school improvement, as well as possible differences in the trajectories that these institutions follow, considering their particularities. It is necessary, in the first place, to address the discussion of the literature regarding educational improvement and their trajectories.

Background

Research pertaining to the concept of educational improvement aims to describe and analyze favorable changes in schools or educational systems as well as the factors, scenarios, and stakeholders that explain them, and the relations these elements share (Hopkins; Harris; Stoll; MacKay, 2011HOPKINS, David; HARRIS, Alma; STOLL, Louis; MACKAY, Tony. School and System Improvement: state of the art review. In: INTERNATIONAL CONGRESS OF SCHOOL EFFECTIVENESS AND SCHOOL IMPROVEMENT, 24., Limassol, 2011. Presentation... Limassol: ICSEI, Cyprus, 2011.). The process of development of this concept and its associated research have rendered possible the consolidation of certain points of consensus and identification of aspects that require further precision or a deeper analysis at the conceptual and methodological level.

Components of Educational Improvement

Hallinger and Heck (2011)HALLINGER, Philip; HECK, Ronald. Exploring the Journey of School Improvement: classifying and analyzing patterns of change in school improvement processes and learning outcomes. School Effectiveness and School Improvement, Abingdon, v. 22, n. 1, p. 1-27, 2011. define improvement as a journey that entails positive implications for a school or an educational system. With regard to this process, consensus exists pertaining to the notion that changes associated with educational improvement should translate into positive changes at the teaching and learning level (Hopkins, 2001HOPKINS, David. School Improvement for Real. London: Routledge; Falmer, 2001.; Elmore, 2004ELMORE, Richard. Instructional Rounds in Education: a network approach to improving teaching and learning. Boston: Harvard Education Press, 2004).

Several studies on educational improvement have sought to characterize this process and identify its fundamental variables, which include dimensions such as teaching and learning methods, organizational change, cultural change, competence training, curricular development, or system-level policies. In this case, some variables tend to be relevant such as leadership, teacher competences, or professional learning communities, to name a few (Fullan, 2002FULLAN, Michael. Los Nuevos Significados del Cambio en la Educación. Barcelona: Octaedro, 2002.; Levin; 2007LEVIN, Ben. Sustainable, Large-Scale Education Renewal. Journal of Educational Change, Dordrecht, v. 8, n. 4, p. 323-336, 2007.; Datnow, 2011DATNOW, Amanda. Collaboration and Contrived collegiality: revisiting Hargreaves in the age of accountability. Journal of Educational Change, Dordrecht, v. 12, n. 2, p. 147-158, 2011.).

A major portion of the variables related to educational improvement can be understood in terms of capacity building, a process that ought to be systemic, implying that it must include all actors and contexts, according to Harris. In addition, research has drawn attention to the collective improvement capacities or the social capital of the educational institution (Fullan, 2011FULLAN, Michael. Choosing the Wrong Drivers for Whole System Reform. Victoria: Centre for Strategic Education, 2011.; López; Sánchez, 2012LÓPEZ, Julián; SÁNCHEZ, Marita. Levers for Sustainable Improvement of Spanish Schools in Challenging Contexts. Journal of Educational Change, Dordrecht, 14, n. 2, p. 203-232, 2012.; Hargreaves; Shirley, 2009HARGREAVES, Andy; SHIRLEY, Dennis. The Fourth Way: the inspiring future for educational change. Thousand Oaks: Corwin Press, 2009.). This implies that the process of improvement can transpire at the individual, collective, and organizational level.

Sustainability and Educational Improvement Trajectories

Educational improvement is a continuous process; therefore, one of its the central challenges associated with it is sustainability, a goal regarded as both necessary and difficult to achieve (Hargreaves; Fink, 2006HARGREAVES, Andy; FINK, Dean. Estrategias de Cambio y Mejora en Educación Caracterizadas por su Relevancia, Difusión y Continuidad en el Tiempo. Revista de Educación, Madrid, n. 339, p. 43-58, 2006.).

In his review of the successful educational reforms carried out in Ontario in the 2000’s, Levin (2007)LEVIN, Ben. Sustainable, Large-Scale Education Renewal. Journal of Educational Change, Dordrecht, v. 8, n. 4, p. 323-336, 2007. observes that a strategy will foster sustainability if it is conducted in a comprehensive, coherent, and aligned manner, promotes respect for staff members and professional knowledge, and associated with adequate resources and incorporates all the parties involved in the teaching-learning process.

Sustainability can be regarded as a conclusive step in the process of educational improvement process. Therefore, research in this topic has increasingly focused on educational improvement trajectories to analyze the ways in which an institution can progress towards stages characterized by greater sustainability (Muijs; Harris; Chapman; Ross; Stoll, 2004MUIJS, Daniel; HARRIS, Alma; CHAPMAN, C.; STOLL, Louise; RUSS, Jennifer. Improving Schools in Socioeconomically Disadvantaged Areas - A Review of Research Evidence. School Effectiveness and School Improvement. An International Journal of Research, Policy and Practice, 15, n. 2, p. 149-175, 2004.).

Hopkins (2001)HOPKINS, David. School Improvement for Real. London: Routledge; Falmer, 2001. emphasizes that the analysis of two decades of research on this phenomenon provides some insights about the identification and description of these stages. Even though such stages may not happen in the same order in every school, they follow a certain sequential logic.

Based on this rationale, the author defines the following three stages of improvement that constitute his improvement quality of education for all (IQEA) model:

  • 1) Establishing the process: This involves definitions about the work structure, approach, and people necessary for the implementation of the improvement process.

  • 2) Going Whole School: This entails the generation of a strategy oriented towards the development of the teaching-learning process and the related individual and institutional competences required to attain its objectives.

  • 3) Sustaining momentum: This implies the consolidation of the capacity for change within the school and permanent expansion of teaching strategies.

For Chile, Bellei, Valenzuela, Vanni, and Contreras (2014)BELLEI, Cristián; VALENZUELA, Juan; VANNI, Xavier; CONTRERAS, Daniel. Lo Aprendí en la Escuela: ¿cómo se logran procesos de mejoramiento escolar? Santiago: Universidad de Chile; UNICEF, 2014. describe improvement trajectories based on differences between the notions of short-term tactics, strategic response, capacity building, and development. Based on these concepts, the authors identify four stages of improvement: 1) specific improvement; 2) incipient improvement; 3) school improvement on its way to institutionalization; 4) institutionalized school improvement.

Despite the existence of the patterns described, which explain the prevalence of certain phases in improvement trajectories, the identification of a single improvement trajectory is not possible. Several authors emphasize that improvement processes may not necessarily be standardized or replicable (Hargreaves; Shirley, 2009HARGREAVES, Andy; SHIRLEY, Dennis. The Fourth Way: the inspiring future for educational change. Thousand Oaks: Corwin Press, 2009.; Harris, 2012HARRIS, Alma. Liderazgo y Desarrollo de Capacidades en la Escuela. Santiago: Fundación Chile; Fundación CAP, 2012.; Anderson; Mascall; Stiegelbauer, 2012ANDERSON, Stephen; MASCALL, Blair; STIEGLEBAUER, Suzzane; PARK, Jaddon. No One Way: differentiating school district leadership and support for school improvement. Journal of Educational Change, Dordrecht, v. 13, n. 4, p. 403-430, 2012.). This evaluation presents a major challenge for the analysis of this phenomenon, because it encourages a more in-depth study of the specific characteristics and the differences in the schools’ trajectories.

In the context described, the research on educational improvement identifies other attributes characterizing this process and provides in-depth analyses of the implications of the aspects studied, especially under the rationale of the examination of improvement trajectories.

First, it has been observed that further specific and contextualized definitions of educational improvement are required. Guhn (2009)GUHN, Martin. Insights from Successful and Unsuccessful Implementations of School Reforms Programs. Journal of Educational Change, Dordrecht, v. 10, p. 337-363, 2009. emphasizes that it must be considered that policies and programs do not operate in controlled conditions, but in complex scenarios and culturally differential contexts.

Second, research acknowledges that the educational change proposed by reforms must be analyzed through a multi-level perspective, considering all stakeholders (Gallucci; Knapp; Markholt; Ort, 2006GALLUCCI, Chrysan; KNAPP, Michael; MARKHOLT, Anneke; ORT, Suzy. Standards-Based Reform and Small School of Choice. In: HARRIS, Alma; CHRISPEELS, Janet (Ed.). Improving Schools and Educational Systems: international perspectives. New York: Routledge, 2006. P. 186-216.; Loogma; Tafel-Viia; Ümarik, 2012LOOGMA, Krista; TAFEL-VIIA, Kulliki; UMARIK, Merik. Conceptualizing Educational Changes: a social innovation approach. Journal of Educational Change, Dordrecht, v. 14, n. 3, p. 283-301, 2012.).

The consideration of these important elements also requires that we consider the wide scope available for the conduction of studies on the improvement process. If we accept its multi-level nature, the effect of context, and the notion of trajectory, we can recognize that the structure of the improvement process is not singular, but multiple, and that answers associated with certain realities or components will not possess the same explanatory capacity as others.

Methodology

This article reports the results of a study that sought to describe the characteristics of the improvement process in diverse types of Chilean schools to establish whether the recognition of certain improvement patterns or trajectories was possible, considering several contexts for the implementation of this process.

Employing the case study methodology (Yin, 1989YIN, Robert. Case Study: research. Beverly Hills: Sage, 1989.), the research analyzed at the implementation of the educational improvement process in different socio-educational realities, considering a sample of cases that will be described below, together with the procedures followed for their selection.

Sample

We sought to extract case studies from Chilean public educational establishments, which are administered by municipalities (local government). Cases were considered from four regions of Chile, which are in the central zone of the country, which is the territory with the highest population density.

Cases were selected to constitute a comprehensive representation of the various social and educational realities of Chilean schools, considering the fact that educational research has identified the relevance of the socio-economic context and the level of educational development of schools as essential aspects to explain differences in policy implementation and educational improvement processes (Mourshed; Chijioke; Barber, 2010MOURSHED, Mona; CHIJIOKE, Chinezi; BARBER, Michael. How the world’s most improved school systems keep getting better. In: MOURSHED, Mona; CHIJIOKE, Chinezi; BARBER, Michael (Ed.) How the world’s most improved school systems keep getting better. London: McKinsey and Company, 2010.; Thrupp; Lupton, 2006THRUPP, Martin; Lupton, RUTH. Taking School Context more Seriously: the social justice challenge. British Journal of Educational Studies, Oxford, v. 54, n. 3, p. 308-328, 2006.). Two variables were employed to segment the schools and select the cases. First, we considered the classification of performance of Chilean schools employed by the national educational system. This classification identifies three types of schools: autonomous, emergent, and recovering.

According to the Ministry of Education’s (MINEDUC, 2008MINEDUC. Ministerio de Educación (2008). Ley 20248. Establece Ley de Subvención Escolar Preferencial. Santiago, 2008. Available at: <https://www.leychile.cl/Navegar?idNorma=269001>. Accessed on: 15 Sept. 2018.
https://www.leychile.cl/Navegar?idNorma=...
) evaluations, autonomous schools (high performance) have systematically obtained superior educational results. Emergent schools (mid or mid-low performance) have not displayed systematically good educational results according to the MINEDUC assessments. Recovering schools (insufficient performance) have produced permanently deficient educational results according to the same evaluation criteria.

The second segmentation variable concerned the socio-economic aspects involved. Schools were classified based on whether their percentage of priority students, as defined by Chilean educational authorities, was higher or lower than the category to which they belonged. Priority students are defined by the Chilean Ministry of Education as those whose socio-economic situation can hinder their chances of responding to the educational process.

In addition, two rural schools were considered. In total, the combined application of these variables led to the selection of the following eight schools:

  • a) Autonomous with a low-high percentage of priority students (Autonomous 1);

  • b) Autonomous with a high percentage of priority students (Autonomous 2);

  • c) Emergent with a low-high percentage of priority students (Emergent 1);

  • d) Emergent with a high percentage of priority students (Emergent 2);

  • e) Recovering with a lower-high percentage of priority students (Recovering 1);

  • f) Recovering with a high percentage of priority students (Recovering 2);

  • g) Rural school 1 (schools located in rural areas, usually with a small population of students);

  • h) Rural school 2.

Instruments of Data Collection and Analysis

To describe and analyze the improvement processes of the category of schools mentioned, a two-stage study was conducted. First, a methodology was designed to evaluate the presence of the attributes of the educational improvement process in the schools analyzed. This objective was pursued with the Analytic Hierarchy Process (AHP) (Saaty, 1997SAATY, Thomas. Toma de Decisiones para Líderes: el proceso analítico jerárquico, la toma de decisiones en un mundo complejo. Pittsburgh: RWS Publications, 1997.). The AHP aims to generate, within a tree-like model, a set of factors and values that systemically constitute a certain concept or object of evaluation. Its structure is based on the hierarchical decomposition of complex structures into their components. Numerical values are assigned to the definitions and valuations made by expert participants that facilitates the definition of the relative importance of each component to explain the phenomenon.

For this study, the tree-like model aimed to identify the attributes of educational improvement to assess its presence across Chilean schools. To accomplish this, first, we conducted focus groups with four sets of individuals with different specializations in the field of educational improvement (teachers, principals, technical-educational consultants, and academic advisors). They were asked to identify and describe the elements that characterize a Chilean school undergoing educational improvement. With this information, a primary model of educational improvement attributes was proposed that contained strategic criteria, sub-criteria, and terminal criteria relevant to assess its presence in Chilean schools.

The second stage consisted the validation of the hierarchical structure with the groups of experts, considering the fulfillment of the four axioms defined under the AHP method. Subsequently, the weights of each of the criteria identified were determined to acknowledge their specific contribution to the assessment of the attributes of educational improvement. This task was performed with Expert Choice software package, a specialized tool for AHP methodology. The methodology demands a maximum level of disagreement among participants as a condition of validity. In this case, the degree of disagreement was found to be 0.1%, a value that is considered to be valid.

The third part of the process entailed the definition of the indicators that render the collection of information possible. This allowed us to design the data collection instruments and/or identify secondary data that could provide information about each indicator of the assessment model. These instruments were subjected to a content validity evaluation by two expert raters.

Subsequently, the instruments of information collection were administered and the presence of the attributes of improvement was assessed in the eight participating schools.

After the completion of this initial phase, the second part of the study was undertaken. Considering the structural relevance of the notion of trajectory in the process of educational improvement, an implementation study was developed to describe the presence and the central characteristics of the improvement process in the eight schools, a task that was accomplished through in-depth interviews.

The interviews were conducted with key stakeholders who should intervene and/or foster school improvement: the school’s administrative team, its financial supports (district), the Ministry of Education at the local territorial level, parents’ representative, and a representative from an institution providing technical assistance to each school.

Specifically, the interviews sought to provide a retrospective view of each school’s improvement trajectory. To accomplish this, the interviewees were asked to characterize the school’s functioning in three moments within a five-year period (2009-2013). The moments selected formed the beginning, the middle, and the end of this period. The interviewees were requested to identify the aspects that fostered or hindered the school’s improvement and to state their specific role in this process for each moment of analysis. These retrospective interviews were conducted between October and December 2013. The interviews were aimed at reconstructing, upon the basis of a set of the participants’ experiences, the trajectory of each school during the period analyzed.

The discourse in the interviews was analyzed by employing the grounded theory (Strauss; Corbin, 1990STRAUSS, Anselm; CORBIN, Juliet. Basic of Qualitative Research: grounded theory procedures and techniques. California: Sage, 1990.), with the aim to identify causal, conditional, contextual, and intervening factors in the educational improvement processes of the schools studied which could be identified in the participants’ descriptions.

Subsequently, connections were identified between both stages of the study to analyze the current state of the schools and understand their improvement trajectories.

Findings

Phase 1 of the Study: assessment of educational improvement attributes

The results of the initial stage of the study will be presented first. They concern the assessment of educational improvement attributes in the eight cases examined.

The multi-criteria model constructed with the help of the experts comprises three strategic criteria: a) external improvement stimuli, i.e., the characteristics of the support and resources received from the educational system and the school’s use of them; b) improvement-oriented management by the school’s financial backer, i.e., the characteristics and conditions of the administration and management of the school in charge of the financial backer; c) internal mobilization of improvement, i.e., the processes, practices, and competences that the school develops directly and that foster its improvement. These strategic criteria are divided into sub-criteria and indicators. Figure 1 reveals the strategic criteria and sub-criteria that comprise the assessment model.

Figure 1
Model for the Assessment of Educational Improvement Attributes: strategic criteria and sub-criteria

The application of the multi-criteria model rendered it necessary to consider a wide variety of approaches to identify improvement attributes, considering the expected requirement to apply multiple evaluation strategies at several levels of the system to provide an integrated and fair assessment (Thomas, 2010THOMAS, Sally. Assessment and the Evaluation of Institutional Effectiveness. International Encyclopedia of Education, Oxford, v. 3, p. 172-180, 2010.; Schwartz; Hamilton; Stecher; Steele, 2011SCHWARTZ, Heather; HAMILTON, Laura; STECHER, Bryan; STEELE, Jennifer. Expanded Measures of School Performance. RAND Corporation, Santa Monica, p. 1-62, 2011. Available at: <http://www.rand.org/pubs/technical_reports/TR968.html>. Retrieved on: March 11, 2012.
http://www.rand.org/pubs/technical_repor...
). Table 1 summarizes the instruments utilized in the assessment of each strategic criterion.

Table 1
Information Collection Instruments: stage 1

Subsequent to the application of the data collection instruments and systematizing of the secondary data of the schools (early 2014), the information obtained was analyzed. This process consisted the calculation of the level of achievement of each indicator of the assessment model.

Table 2 summarizes the general results for each school in terms of their level of achievement of improvement attributes, as well as their achievement of the three strategic criteria. According to this rationale, a school may have a higher or lower presence of each attribute included in the assessment matrix on a continuum ranging from zero (absence of improvement attributes) to one (an absolute presence of the attributes). The sum of the values of each of these indicators (considering their specific weights, also agreed upon by the experts) provides a general achievement level associated with the educational improvement attributes defined.

Table 2
Achievement Level of Educational Improvement Attributes in the Eight Schools Studied (scores between 0 and 1)

It is important to consider that, with this methodology, we conducted a descriptive analysis of the scores obtained by the schools. An analysis of the statistical significance of these differences was not performed due to the small number of cases considered in the study. Considering this limitation, the experts defined that exceeding 0.15 points implied significant differences for the achievement of each indicator of educational improvement by the schools.

The information presented in the table reveals that autonomous schools display higher achievement levels than all other schools in terms of the presence of achievement attributes. However, in numeric terms, this difference only amounts to a few hundredths in the general result of this index in compares with an Emergent school and a Recovering one with better performance in this assessment.

In contrast, greater differences can be observed between the four schools among the highest scores and the four lowest-scoring ones. The difference that separates the first from the last school in the final evaluation of improvement results was 0.23.

A glance at the results according to the three strategic criteria of the evaluation structure, (a) external stimuli, b) financial backer’s management, and c) internal mobilization of improvement), reveals variability in the schools’ achievement levels. It implies that the schools that attain the best and worst results vary depending on the strategic criterion considered. In the case of the External improvement stimuli criterion, autonomous schools and emergent ones with a high percentage of priority students obtained the optimum scores. In contrast, the recovering school with the highest percentage of priority students attained the lowest achievement level. This situation suggests that in this case, the degree to which good results are achieved is not a sufficient condition to determine the presence of the multiple attributes that constitute educational improvement.

Regarding the strategic criterion Improvement-oriented management by the school’s financial backer, the autonomous school with a lower percentage of priority students and the emergent one with the highest percentage of such students obtained optimum scores. In this criterion, the schools with the lowest achievement percentages obtained considerably lower scores than the schools mentioned. Finally, a different achievement distribution is observed in association with the Internal mobilization of improvement criterion, as the autonomous school with the highest percentage of priority students obtained the highest achievement level. In contrast, the two recovering schools and the two rural ones obtained the lowest scores.

This information suggests that this process, which comprises multiple variables and is affected by contextual factors, can be configured in many ways and is attainable via several paths. Considering this, the second part of the study focused on the analysis of the improvement trajectory of a group of these schools, based on interviews with stakeholders to describe, the way in which they attained their current characteristics through a procedural logic, measured using the index described above.

Considering the results of the first phase of the study, the decision was reached to study the educational improvement trajectories of the four schools in further depth, with the best scores obtained in Stage 1, to obtain a clearer picture of the processes that led them to achieve the attributes identified in the assessment. We further sought to analyze the patterns and the differences between the trajectories of the schools that could be similar (considering the results of Stage 1 only).

The four selected cases were as follows: Autonomous 1, Autonomous 2, Emergent 2, and Recovering 1. Table 3 describes the strengths and weaknesses of these schools, identified in the assessment of Stage 1 in further detail.

Table 3
Summary of Improvement Attributes in the Schools Selected for Phase 2 of the Study

Phase 2 of the Study: analysis of improvement trajectories

The analysis of the improvement trajectory of the four schools selected was conducted based on in-depth interviews with the people involved in each case. The following is a description of the schools’ attributes in the three stages analyzed, considering their classification.

Autonomous Schools (1 and 2)

In the first period of analysis (2008), the two schools displayed good SIMCE scores and received social recognition due to these achievements. Both schools were described as possessing a well-structured functioning and a good organization of tasks. The school with a lower percentage of priority students (Autonomous 1) is funded independently with central government policies. In contrast, the second school (Autonomous 2) possesses less economic resources and is mostly funded by State subventions. In both cases, students’ parents closely monitor the schools; results and performance and exert pressure for them to maintain a high performance. The technical assistance group explained the following:

I’d say that a factor that I think has been really important is how, over the last five or six years, the financial backing entity has been very strongly results-oriented, their administration has focused on that, I’d say that causes part of the problems because the administration has focused very strongly on students’ academic results (Technical-Educational Assistance group [ATE], Autonomous School 1).

Both schools are characterized as weakly led by their financial backer, fundamentally due to the volatility of their policies. In addition, the existence of a vertical and undemocratic relationship is described in the first school. Regarding the internal processes of the school, the counterpoint to their strong structure is a weaker tendency to generate pedagogical innovation and technical discussion among teachers. This assessment was conducted by several participants from both schools, as described below:

We find this interesting because this municipality has a very strong communicational area. We are surprised that the parents-municipality/education department relationship or that between parents and the school lacks fluidity in basic tasks, such as transmitting official information (Parent, Autonomous School 1).

This is a very small corporation. So, the coordination office itself isn’t a reference point or a technical-pedagogical consultant for the school (Financial backer, Autonomous School 2).

In the second analysis period, the characteristics of the schools’ functioning remained unchanged for the most part. The main change during this stage is observed in the second school, whose budget increased tremendously, because it became a beneficiary of a new State subvention. These funds allowed the school to invest in technical assistance, teaching resources, and more staff.

Finally, the third period is characterized by the preservation of the characteristics described in the previous stage. In the case of Autonomous School 2, increased pressure from the financial backer also becomes a factor that intends to make the school increase its level of self-criticism and orient its functioning towards other objectives, apart from the SIMCE performance. Nevertheless, this pressure is not implemented through a clear policy.

Emergent School 2

According to the interviewees, at the beginning of the period analyzed, the school was going through a time of poor technical functioning, weak educational results, and low enrollment levels. Located in a social context of economic vulnerability, the school was only administered, not educationally managed, a dynamic shared with its financial backer. In addition, the institution had limited funds and lacked parents’ support. The financial backer describes this complex situation in the following words:

So, there used to be another principal, old principals, with a more social approach, you know? Less interest in the students’ performance according to the SIMCE or other standards used at the time [...] in those days there was little progress in terms of results (Financial backer, Emergent School 2).

In the second analysis period, the school had a new principal. He was chosen through the Higher Public Management system, which involved a more demanding and transparent hiring process. In addition, the financial backer was also changed. In this situation, there was a person in charge of the schools’ educational processes who intends to implement several policies. The school received a new State subvention that resulted in the availability of a substantially larger amount of funds. In the context described, the principal started the implementation of the policies advanced by the financial backer and defined basic work processes, normalizing the school and organizing its administrative, management, and teaching processes. In addition, he invited parents to participate more actively in the school’s activities. In this period, the school’s results improved slightly. The principal describes this stage as follows:

I think these changes have been due to my work, because I’ve been very incisive, constantly transmitting what I want teachers to do in the classroom. Obviously, together with my work team. The technical aspect has allowed us to implement a work style that at the beginning may have lacked strong teacher support, but as it progressed it has become part of the school (Principal, Emergent School 2).

In the third analysis period, the changes introduced in the previous stage were consolidated. They were catalyzed by the greater capacity of the school to identify and determine the strategy to use its economic resources at a time when it was receiving external technical assistance and had also decided to work without the support of the Ministry of Education in this area. During this period, the school’s results displayed another moderate increase and a change in its image was observed, as the seriousness of its work started to become noticeable in its surroundings. Regarding the development of its internal processes, the school focused on the rationalization and perfection of its teaching and evaluation processes, even though some early progress could already be observed in this area.

Recovering School 1

During the first analysis period, the school described that it completed its initial functioning phase after eight years’ time. Since it is located in a small town, the school had a relevant place in its community. Nevertheless, its academic results were poor, and its organizational work was week, considering the expectations defined by the educational community.

In the second period, the founding principal of the school became its financial backer. The new principal was regarded very lowly by all interviewees. A deterioration in human relationships within the school was observed, along with a weakening of the link with the financial backer, a decline in the school’s processes, and an impoverishment of its educational quality. In addition, the school’s image worsened, and more vulnerable students enrolled in the school, increasing its homogeneity and rendering educational processes more difficult. This situation is described by the financial backer in the quotation below:

Now, we had a very unfortunate time in the school. There was a principal in the years 2010, 2011, 2012. We took a chance [...] And that was a mistake. Two student strikes against the principal, the teachers also opposed him after some workplace accidents. The parents denounced him. Our scores worsened, and we didn’t have great scores before (Financial backer, Recovering School 1).

In the third period, a new principal was elected, a professional who built his career in the school. He, along with the financial backer, managed to organize the school adequately that made it possible to solve the situation described in the previous period. The financial backer introduced new educational policies and a system to support and monitor the school, valued by the participants. In addition, the principal introduced a set of basic procedures and routines to organize and assign internal tasks. Although the school is progressing towards normalization, measurements and tests have revealed no impact in terms of educational results. The principal narrates the arduous and incomplete improvement efforts that he has led:

First of all, I had to improve the school’s image, improve the relationship with parents and students, respond to all queries and doubts, and immediately solve any situations that came up [...] My team had to lead the school’s work in four management areas, curriculum, leadership, climate, and results, and apart from that the diagnosis of the school’s academic results [...] So, now all the teachers are committed to achieving certain goals (Principal, Recovering School 1).

Discussion

The following analysis intends to establish connections between the results of the evaluation of improvement attributes and the description of the four schools’ trajectories.

There is a certain overlap between the schools that started in a favorable position in the period analyzed and those that attained the highest levels in Phase 1 of this study (Autonomous 1 and 2). Both schools preserved their initial positive characteristics: work structure, orientation towards results, and image. However, the school trajectory analysis revealed that only Autonomous school 2 improved more clearly in different organizational processes, with specially in the principal’s leadership. In addition, neither of the two cases were characterized throughout the period by solid policies implemented by their financial backers or by the pedagogical leadership of their principals, reflected in the results of the assessment of improvement attributes. For this reason, in the case of the autonomous schools, the situation observed indicates stability rather than change in their improvement processes, because important transformations are not observed, especially in pedagogical processes.

In contrast, Emergent school 2 and Recovering school 1 started their trajectory in this period from a much weaker position than the autonomous ones. Nevertheless, the analysis of their trajectories more clearly revealed improvement processes during the period analyzed. Despite this, it is also necessary to consider that in these cases it was necessary to install and improve more basic processes or practices than the previous group of schools.

Considering this and the results of the assessment of the improvement attributes (Phase 1), we could deduce that during this period, these schools closed the gap with the first two that has mainly been a result of improvements in the financial backer’s and the principal’s leadership, two elements that are not as strongly developed in the autonomous schools. Despite this, their management processes still possess weaknesses and, most importantly, they are yet to fully reach the classroom. In this regard, the interviews reveal that the autonomous schools possess an effective method, even though it is mostly oriented towards academic performance under a competitive logic. In contrast, Emergent school 1 and Recovering school 2 have not even achieved this.

The weaknesses of Emergent school 1 and Recovering school 2 are linked to technical-pedagogical work and teaching processes. This deficit is evidenced by the results of the assessment of attributes, and thus, it is also present in the schools’ SIMCE test scores and the national teacher evaluation. It is interesting to observe the way in which their differing trajectories failed to adequately tackle these relevant processes. The improvement processes that have been installed, have had greater relationship with organizational aspects of the school, but have not yet seen relevant changes in the pedagogical work. These attributes are more present in the autonomous schools, but in these cases, no relevant efforts are identified to improve them during the analyzed period, which resembles the cruising schools, described by Stoll and Fink (1996)STOLL, Louise; FINK, Dean. Changing our schools: Linking school effectiveness and school improvement. Buckingham: Open University Press, 1996..

Continuing with their weaknesses, none of the four schools obtained noteworthy results in the teacher evaluation, not even those with a clearer improvement route.

In the case of the schools that started off the analysis period in a better situation, it is surprising that their SIMCE scores and their teacher evaluation results were not especially good, consistent with the picture of relative stagnation conveyed by the implementation study. This situation contradicts their public image of efficiency and it raises the question of whether it is possible to maintain an advantaged public position as a school, without necessarily requiring improvements or important achievements in the academic field.

A strength common to all the schools examined is an ability to implement school management processes and practices aimed at organizing and structuring work and responsibilities. In the case of Autonomous schools 1 and 2, this process predated the time frame analyzed and is determined by expectations associated with the school’s excellence in terms of academic performance. In contrast, the other schools have worked to define the sources that allow them to organize their work, identifying other points of reference to face this challenge. It is precisely this position that differentiates both groups of schools. While the first group previously found a form of organization, but did not improve it clearly in this period, the other developed and strengthened its internal processes in this period but was initially at a much less developed stage.

From another analytic focus, the comparative analysis of the cases studied reveals the weight of socioeconomic contexts in the development of the schools’ improvement trajectories. In the case of poverty, its effect is noticeable in the practices of Emergent school 1 and Recovering school 2 (especially in the latter). In some cases, it was necessary to invest time and energy to establish regular functioning routines. These processes began to develop within the period analyzed and had to deal with disadvantaged social contexts. This situation contrasts with the autonomous schools, whose previous success implied less pressure to address the socioeconomic disadvantage, given that their students do not live in poverty conditions, nor are they located in vulnerable territories.

As can be observed, the schools that achieved better results in the evaluation of attributes of educational improvement reached the present considering different trajectories, marked by different dynamics and with differentiated effects of their environment. These phenomena show the existence of diverse ways of improvement, as well as different requirements of schools to follow their trajectories in the future.

Towards an Explanatory Model of the Improvement Trajectory of the Schools Analyzed

As the above given information has revealed, the study of improvement trajectories and of the presence of attributes of this process in schools, apart from revealing the presence of some common elements, also provides evidence of the large number of specific traits and elements that influence schools’ trajectories. Considering this disquieting conclusion for those who study this process, the question of whether it is possible to define the characteristics of these schools’ improvement trajectories arises. The literature review conducted for this article covered conceptual aspects and proposals to explain educational improvement trajectories.

The analysis performed revealed the requirement to identify more specific processes that characterize the stages of educational improvement trajectories. For this reason, the proposal of a new explanatory model of improvement trajectories was advanced upon the basis of the results of the present study. This proposal seeks to describe the trajectory, considering the main process that the school focusses on during each stage and the phenomena necessary to progress from one stage to the next. The main process of each stage encompasses all the functions performed by the school, both in terms of management and in the teaching and learning sphere. The improvement trajectory involves surpassing the initial main processes and progressing towards processes that advance student learning and technical expertise pertaining to the educational organization.

We consider that the improvement trajectory resulting from this analysis comprises three main processes. They can be labeled as follows: 1) bring order; 2) achieve organization; 3) improve teaching-learning.

The first main process concerns the school’s requirement to orient itself towards the conduction of processes essential for its minimal regular functioning. To accomplish this, it must put in the effort to function regularity, while attempting a minimal articulation of a set of problems and tasks that affect the school individually, guided by rationales or expectations not wholly related to education. The results of the study reveal that the protagonists of this stage are constituted by the principal and the financial backer, and that there is an emphasis on compliance with the norms that ensure the school’s regular operation. In addition, this stage is rendered especially complex when the school operates in a context of social vulnerability, or when it has progressed only minimally in educational matters. This stage was identified in Emergent school 2 and Recovering school 1, during the second and the third moments of analysis during which, it was necessary to define priorities that could stabilize their regular functioning in the face of a critical situation. The second central process takes place when the conditions that guarantee the school’s survival are fulfilled, that is, when the participants realize that they must put their expectations at the service of those of the school. The main process involves the definition of work routines or practices that appear logical to the organization and function in a relatively coordinated way. The transition to this stage is effected by the protagonists in the first phase (the principal and the financial backer) by working in a stable manner and aligning themselves with the principal’s decisions, while the latter supports the orientations and policies introduced by the financial backer. In this context, the application of policy resources and definitions becomes relevant, as they help to channel this incipient organization and guide it to a place where it can be useful for the school. In this study, both autonomous schools were in the first moment of analysis in this phase. The retrospective narrative of their trajectory revealed that they remained in this phase during this period. In contrast, the emergent school and the autonomous one advanced to this phase during the period studied; they defined a more concrete way to organize themselves regularly and oriented their efforts towards institutional and pedagogical processes in an increasingly stable manner.

Unfortunately, the third main process could not be clearly observed in the cases analyzed, but can be inferred upon the basis of the weaknesses observed. This stage focusses on concretizing the school’s educational aspirations. The evaluation of improvement attributes reveals that in this process, the central process is geared towards building teachers’ competences. From this perspective, clear indications, orientations, or policies are required that advance from the logic of seeking organization to effecting the integration of the organized parts to achieve an educational goal. In addition, it is necessary to improve the technical aspect of the performance of work, that is, engagement in professional development. Proceeding to this stage requires more than the leaders of the system and the school: the school’s work culture and identity also have a role to play in this regard, as the institution must be able to locate a place within it that emphasizes on the importance of its own specific goals.

As can be observed, each stage is affected by the three strategic criteria that bring together the improvement attributes assessed in the eight participating schools (external impulses, financial backer, and internal impulses). Even though each stage involves its own complexities, the data reveals that the consolidation of the process occurs in the transition between stages 2 and 3, because this requires collective work and the building of concrete capacities, two elements that cannot be obtained by simply defining a way to organize work. Therefore, the actual leap in improvement involves the passage from the individual to the collective level and from disjointed tasks to interconnected processes at different levels.

The effect of the context on this transit has been clearly observed in the evidence collected. Schools located in more vulnerable contexts (Emergent 2 and Recovering 1) require more time and work to survive the first main process and proceed to the second. Due to the qualitative differences between stages 2 and 3, it is difficult to demand that a school reach the latter point without understanding the vicissitudes and time constraints associated with this road of development that also includes stage 1.

In addition, another element that was found to influence the passage from stage 2 to stage 3 is the signal that the school receives from the educational system regarding its achievements. Autonomous schools 1 and 2 did not improve their pedagogical processes during the period analyzed; however, they preserved their successful status in the Chilean educational system, which is organized according to a market logic. This reveals that the political environment and the system’s dynamics are the forces that obstruct the schools’ path to improvement. Figure 2 summarizes the dynamics of the main processes of school trajectory that can be derived from the analysis conducted.

Figure 2
Stages and Main Processes of Educational Improvement Trajectories

Considering the improvement trajectories described for these schools, it is apparent that the improvement processes required should be different in terms of the impulse, strategies, and support involved, depending on the central process the schools undergo and the types of constraints that the context imposes on them.

Taking into consideration all that has been mentioned before, the results of this research comply with the literature regarding the identification of the key aspects that influence educational improvement trajectories. Similarly, this study further indicates that the problems and challenges that a school must face in these areas may differ from those of other schools; therefore, more research is required to determine the ways in which these attributes may be clarified (and strengthened) through the identification of the social and economic characteristics of a school, along with its current stage of improvement. In addition, the study supports the notion of improvement trajectory described in the literature; however, adding that movement during this process relates to the objectives prioritized by the school.

The results of the study revealed that the sustainability goal, highlighted in the background section, may be affected by varied factors, such as change in leaders, or changes in policies affecting the school. In addition, the Autonomous Schools maintained their processes in a stable way (in a cruise phase), but that affected their impulse to continue improvement. For this reason, it is necessary to differentiate between stability and sustainability. Sustainability includes fulfilling the challenge of further improvement, as highlighted by Hopkins (2008)HOPKINS, David. Hacia una Buena Escuela: Experiencias y Lecciones. (C. Santa Cruz, Trans.). Santiago: Fundación Chile, Fundación CAP, 2008..

The progress of the improvement process towards the instructional core does not imply that discussions about learning must be postponed until organizational matters are resolved, because pedagogical work also requires efforts to consolidate basic processes (for example, ensuring that lessons begin or end on time) and to generate work routines or structures that introduce organization to the task of teaching. In fact, many of the school’s basic processes (stages 1 and 2) should be developed through the discussion of the way in which the teaching-learning processes should be developed.

Regarding the limitations of this retrospective study, it was decided that the description of the processes should benefit from the perspective of those involved in them. Despite its strengths, this alternative also presents risks such as other data that reflect the results of the schools’ trajectories being left out. Delving into these aspects may be relevant to attain a further comprehensive vision of the schools.

Considering these results, it might be relevant for future research to concentrate on the identification of the developmental patterns of educational improvement trajectories, while considering the differential effects of the social and the educational context. In addition, it would be necessary to identify the internal and external factors that augment the school’s central processes and thus facilitate improvement.

References

  • ANDERSON, Stephen; MASCALL, Blair; STIEGLEBAUER, Suzzane; PARK, Jaddon. No One Way: differentiating school district leadership and support for school improvement. Journal of Educational Change, Dordrecht, v. 13, n. 4, p. 403-430, 2012.
  • BELLEI, Cristián; VALENZUELA, Juan; VANNI, Xavier; CONTRERAS, Daniel. Lo Aprendí en la Escuela: ¿cómo se logran procesos de mejoramiento escolar? Santiago: Universidad de Chile; UNICEF, 2014.
  • DATNOW, Amanda. Collaboration and Contrived collegiality: revisiting Hargreaves in the age of accountability. Journal of Educational Change, Dordrecht, v. 12, n. 2, p. 147-158, 2011.
  • ELMORE, Richard. Instructional Rounds in Education: a network approach to improving teaching and learning. Boston: Harvard Education Press, 2004
  • FULLAN, Michael. Los Nuevos Significados del Cambio en la Educación Barcelona: Octaedro, 2002.
  • FULLAN, Michael. Choosing the Wrong Drivers for Whole System Reform Victoria: Centre for Strategic Education, 2011.
  • GALLUCCI, Chrysan; KNAPP, Michael; MARKHOLT, Anneke; ORT, Suzy. Standards-Based Reform and Small School of Choice. In: HARRIS, Alma; CHRISPEELS, Janet (Ed.). Improving Schools and Educational Systems: international perspectives. New York: Routledge, 2006. P. 186-216.
  • GUHN, Martin. Insights from Successful and Unsuccessful Implementations of School Reforms Programs. Journal of Educational Change, Dordrecht, v. 10, p. 337-363, 2009.
  • HALLINGER, Philip; HECK, Ronald. Exploring the Journey of School Improvement: classifying and analyzing patterns of change in school improvement processes and learning outcomes. School Effectiveness and School Improvement, Abingdon, v. 22, n. 1, p. 1-27, 2011.
  • HARGREAVES, Andy; SHIRLEY, Dennis. The Fourth Way: the inspiring future for educational change. Thousand Oaks: Corwin Press, 2009.
  • HARGREAVES, Andy; FINK, Dean. Estrategias de Cambio y Mejora en Educación Caracterizadas por su Relevancia, Difusión y Continuidad en el Tiempo. Revista de Educación, Madrid, n. 339, p. 43-58, 2006.
  • HARRIS, Alma. Liderazgo y Desarrollo de Capacidades en la Escuela Santiago: Fundación Chile; Fundación CAP, 2012.
  • HOPKINS, David. School Improvement for Real London: Routledge; Falmer, 2001.
  • HOPKINS, David. Hacia una Buena Escuela: Experiencias y Lecciones. (C. Santa Cruz, Trans.). Santiago: Fundación Chile, Fundación CAP, 2008.
  • HOPKINS, David; HARRIS, Alma; STOLL, Louis; MACKAY, Tony. School and System Improvement: state of the art review. In: INTERNATIONAL CONGRESS OF SCHOOL EFFECTIVENESS AND SCHOOL IMPROVEMENT, 24., Limassol, 2011. Presentation... Limassol: ICSEI, Cyprus, 2011.
  • LEVIN, Ben. Sustainable, Large-Scale Education Renewal. Journal of Educational Change, Dordrecht, v. 8, n. 4, p. 323-336, 2007.
  • LOOGMA, Krista; TAFEL-VIIA, Kulliki; UMARIK, Merik. Conceptualizing Educational Changes: a social innovation approach. Journal of Educational Change, Dordrecht, v. 14, n. 3, p. 283-301, 2012.
  • LÓPEZ, Julián; SÁNCHEZ, Marita. Levers for Sustainable Improvement of Spanish Schools in Challenging Contexts. Journal of Educational Change, Dordrecht, 14, n. 2, p. 203-232, 2012.
  • MINEDUC. Ministerio de Educación (2008). Ley 20248 Establece Ley de Subvención Escolar Preferencial. Santiago, 2008. Available at: <https://www.leychile.cl/Navegar?idNorma=269001>. Accessed on: 15 Sept. 2018.
    » https://www.leychile.cl/Navegar?idNorma=269001
  • MOURSHED, Mona; CHIJIOKE, Chinezi; BARBER, Michael. How the world’s most improved school systems keep getting better. In: MOURSHED, Mona; CHIJIOKE, Chinezi; BARBER, Michael (Ed.) How the world’s most improved school systems keep getting better London: McKinsey and Company, 2010.
  • MUIJS, Daniel; HARRIS, Alma; CHAPMAN, C.; STOLL, Louise; RUSS, Jennifer. Improving Schools in Socioeconomically Disadvantaged Areas - A Review of Research Evidence. School Effectiveness and School Improvement. An International Journal of Research, Policy and Practice, 15, n. 2, p. 149-175, 2004.
  • SAATY, Thomas. Toma de Decisiones para Líderes: el proceso analítico jerárquico, la toma de decisiones en un mundo complejo. Pittsburgh: RWS Publications, 1997.
  • SCHWARTZ, Heather; HAMILTON, Laura; STECHER, Bryan; STEELE, Jennifer. Expanded Measures of School Performance. RAND Corporation, Santa Monica, p. 1-62, 2011. Available at: <http://www.rand.org/pubs/technical_reports/TR968.html>. Retrieved on: March 11, 2012.
    » http://www.rand.org/pubs/technical_reports/TR968.html
  • STOLL, Louise; FINK, Dean. Changing our schools: Linking school effectiveness and school improvement. Buckingham: Open University Press, 1996.
  • STRAUSS, Anselm; CORBIN, Juliet. Basic of Qualitative Research: grounded theory procedures and techniques. California: Sage, 1990.
  • THOMAS, Sally. Assessment and the Evaluation of Institutional Effectiveness. International Encyclopedia of Education, Oxford, v. 3, p. 172-180, 2010.
  • THRUPP, Martin; Lupton, RUTH. Taking School Context more Seriously: the social justice challenge. British Journal of Educational Studies, Oxford, v. 54, n. 3, p. 308-328, 2006.
  • YIN, Robert. Case Study: research. Beverly Hills: Sage, 1989.

Publication Dates

  • Publication in this collection
    28 Oct 2019
  • Date of issue
    2019

History

  • Received
    25 May 2018
  • Accepted
    02 May 2019
Universidade Federal do Rio Grande do Sul - Faculdade de Educação Avenida Paulo Gama, s/n, Faculdade de Educação - Prédio 12201 - Sala 914, 90046-900 Porto Alegre/RS – Brasil, Tel.: (55 51) 3308-3268, Fax: (55 51) 3308-3985 - Porto Alegre - RS - Brazil
E-mail: educreal@ufrgs.br