Colección: La Educación
Número: (132-133) I,II
4. Results of Previous Evaluations
The most comprehensive evaluations of the New School program have been conducted by Psacharopoulos, Rojas, and Velez (1993) and Rojas and Castillo (1988).7 Both utilize a data set collected in 1987 in 11 Colombian departments, which contains results of instruments measuring basic skills in mathematics and Spanish, civic behavior, self-esteem, and creativity. The data also contain information on schools, teachers, and students in 168 New Schools and 60 traditional schools. Some characteristics of the sample should be pointed out. Departments and schools were not selected randomly. First, departments were selected where the New School program was relatively more developed. Secondly, in collaboration with regional coordinators of the program, New Schools were selected where the program had been in place for at least three years, and which had faithfully implemented the methodology (i.e, teaching-training, use of self-instructional guides, library). Areas were excluded based on the small number of New Schools there or the difficulty in reaching them. This tended to favor the “best” New Schools and probably biased the results above what a random sample of schools might have yielded. Thus, evaluations conducted with these data are a better approximation of the ideal New School’s effects on educational outputs. They may not be entirely applicable to the universe of New Schools.
In a comparison of means, Psacharopoulos, Rojas, and Velez (1993) found statistically significant differences between types of schools in third grade Spanish and mathematics achievement, creativity, civics, and self-esteem. In the fifth grade there were only significant differences in Spanish achievement. Then estimating a standard production function with ordinary least squares, they find that a New School dummy variable is positive and statistically significant for Spanish and mathematics tests in grade 3, Spanish in grade 5, and civic behavior for the pooled group of students. They find positive, but insignificant coefficients for tests of grade 5 mathematics and a pooled regression with the tests of creativity and self-esteem.
Their model specification is parsimonious and excludes school and classroom level variables representing availability of New School inputs such as the library and textbooks. Because these inputs are usually applied as a package, they are likely correlated with the New School dummy. Assuming that these inputs influence outputs positively, it is reasonable to expect that their exclusion exerts an upward bias on the New School dummy coefficient. Thus, the coefficient indicates an overall New School effect. But if some traditional schools adopt New School-type inputs and some New Schools are lax in implementing reforms, the dummy variable would reflect an ‘average’ effect; this would probably understate the program’s effectiveness in increasing educational outputs. It also prevents analysis of which program inputs are relatively more effective (or cost-effective, if information is available concerning unit input costs) than others, or the examination of interaction effects among inputs. These issues are further addressed in Part 6.
Rojas and Castillo (1988) present a variety of qualitative data showing that New Schools have higher levels of participation in community activities and high levels of teacher satisfaction with the New School methodology, training courses, and self-instructional learning guides.