Colección: La Educación
Número: (132-133) I,II
6. How Effective are New Schools in Raising Student Achievement?
Mean scores on tests of mathematics and Spanish achievement are given in Table 7. There are statistically significant differences on both tests at the third grade level in favor of the New School. In fifth grade, New School scores are higher than traditional schools, but the differences are not statistically significant. Scores are standardized and presented in Figure 1, which highlights the apparently declining effectiveness of New Schools at the fifth grade level. Nonetheless, taking raw means can be misleading since there are not controls for confounding variables.
To further test the hypothesis that New Schools were effective in raising student achievement, regression models of the following general form were specified and estimated with ordinary least squares:
achievement score = f (school characteristics, principal characteristics,
teacher characteristics, family characteristics,
Two different models were specified, one that followed the basic form of Psacharopoulos, Rojas, and Velez (1993). This simpler specification, leaving out variables representing New School inputs, allows the replication of the previous evaluation and the estimation an “overall” New School effect on achievement. Results are presented in Table 8.11 The New School dummy variable was positive and highly significant for third grade mathematics and Spanish. It was positive but less significant for fifth grade Spanish and negative and insignificant for mathematics. These coefficients are compared with those of the previous evaluation in Table 9.12 Results are strikingly similar between the two evaluations, despite the use of different samples. Both studies point to fifth grade mathematics as an area where New Schools have little effect on student achievement.
Various reasons might explain the narrowed achievement gap at the fifth grade. New Schools have lower dropout rates than traditional schools (see Table 10). Since these retained students probably achieve lower levels, overall achievement declines. This is difficult to control since it is problematic to empirically define which students would have dropped out had they attended traditional schools. But if this explanation is correct, a narrowed achievement gap is not necessarily evidence of ineffective New Schools. Students are being educated in New Schools who would normally drop out. Assuming their achievement levels rise, albeit less than for other students, the overall stock of education in rural areas also increases. Associated outputs of education presumably rise as well.
Secondly, the presence of New School fifth graders who previously attended another school that used traditional methodology could bias downward fifth grade scores. The models of Table 8 included a measure of the number of schools attended by students; coefficients were positive for both fifth grade models, apparently rejecting the second explanation. Thirdly, if a New School has not used the methodology for a child’s entire primary education, scores more closely resemble those of students educated in traditional schools. This is not controlled for in Table 8. A final explanation for the fifth grade gap is that the New School methodology, especially in mathematics, is less effective in upper grades. This cannot be readily answered by available data, but should be a topic of inquiry in future evaluations.
Because the New School coefficient is of primary interest, other results are not reviewed in detail, but some merit highlighting.13 Significant manipulable determinants of student achievement include electricity access, a proxy for overall quality of school facilities, except in fifth grade mathematics. Attending a school located in Valle, the department with the highest GDP per capita of the three under study, increases student achievement in the third grade.14 University education of principals and teachers is either negatively related and significant or insignificant. This could indicate that low teaching salaries relative to those of other careers attract the lowest quality university attendees to administration and teaching. A lower student-teacher ratio, as in much of the developed and developing country literature, is generally not a significant determinant of achievement.15 Among characteristics of the families and students, the highest level of education attained in the family is significant, most clearly so in third grade. Student repeaters are lower achievers in every level and subject, while students who hold jobs are significantly lower achievers in the fifth grade.16
The second model specification is presented in Table 11. The model adds four variables that represent inputs associated with the New School reforms: a measure of textbook availability (although this does not distinguish between the kinds of textbooks available), the presence of a library, a dummy variable identifying whether or not group learning is used “almost every day”, and the number of supervisory visits to the school in the past year.17 The New School dummy variable is left in the model to act as a proxy for other inputs, such as learning corners, teacher training, and perhaps motivation that the New School has inspired in students and staff. In third grade Spanish, the textbook variable is positively and significantly associated with achievement; in the fifth grade, coefficients are positive but insignificant. The library coefficient is positive and significant in third grade Spanish, and negative and significant in fifth grade mathematics. The negative coefficient may indicate that the presence of a library shifts teacher emphasis away from mathematics. The number of supervisory visits was not significantly related to achievement. The use of group learning was only significantly related to achievement in fifth grade mathematics.18 As suggested above, the New School coefficient declines slighly magnitude in third grade regressions. It declines and becames insignificant in fifth grade Spanish and remains unchanged in fifth grade mathematics. Cofficients of family and student varibles are quiet similar in both models.