EFFECT OF STATE INTERVENTIONS ON PUPIL'S ACHIEVEMENT
National Council of Educational Research and Training New Delhi
ABSTRACT This paper attempts to study effects of Operation Blackboard (OB) and incentive schemes on pupil's achievement in mathematics and language. The data collected from 1,746 schools for the Baseline assessment Studies of DPEP formed the basis of this study. Hier*** linear model and meta-analysis are used to analyse the data. The OB scheme has shown a positive and significant impact in Assam, Kerala and Madhya Pradesh. The average impact among states is also positive and significant. The items supplied under the OB scheme have also contributed significantly in Haryana, Karnataka and Madhya Pradesh. Supply of free textbooks, and mid-day meals have indicated a positive and significant impact on pupil's achievement in Madhya Pradesh. Whereas scholarship for regular attendance has shown significant impact on language achievement in Kerala
Introduction Article 45 enshrined in the Indian Constitution has set a noble goal of providing free and compulsory education to all children up to the age of 14 years. Concerted
72 IER: SPECIAL NUMBER 1993
efforts have been made by the government through planned schemes to provide educational facilities on universal basis. These efforts were further supplemented by introducing a number of incentive schemes such as mid-day meals, free uniforms, free textbooks, scholarships for regular attendance, etc. A substantial progress has been made since then in providing access to education and enrolling the children. The Fifth All India Educational Survey (NCERT, 1992) has revealed that while primary schooling facility was available to 94.5 per cent of the rural population within a convenient walking distance of one km, only 75.9 per cent of the children of the age group 6 to below 11 years were enrolled in schools. Further, according to Programme of Action (1986) the dropout rate during the year 1985-86 was 47.6 per cent at the primary stage. In spite of the stated achievements, the goal of Universalisation of Elementary Education (UEE) remained unachieved. The National Policy on Education (1986) reaffirmed the national commitment of achieving UEE with the following objectives
(i) Universal access and enrolment;
(ii) Universal retention of children up to 14 years of age;
(iii) A substantial improvement in the quality of education to enable all children to achieve essential levels of learning.
As conceived in the Programme of Action (1986) the scheme of Operation Blackboard (OB) was launched in the year 1986 to improve school environment. Under this scheme provision was made to provide two reasonably large rooms usable in all weather conditions, at least one more teacher to a single teacher school, toys and games material, blackboards and other learning material.
The District Primary Education Programme (DPEP) was launched in 1993 to further strengthen the efforts of achieving the UEE. To provide a scientific base for launching this programme, the Baseline Assessment Studies (BAS) were undertaken in eight states, viz., Assam, Haryana, Karnataka, Kerala, Madhya Pradesh, Maharashtra, Orissa and Tamil Nadu. To fulfil the objective of the BAS, it was considered necessary to find out the impact of the OB scheme and incentive schemes on pupils achievement.
The present paper attempts to study
Research Questions (i) To what extent schools under OB scheme differ from non-OB schools in terms of pupils achievement in Mathematics and language?
(ii) To what extent incentive schemes influence the achievement of pupils in mathematics and language?
This study uses the data collected for the BAS. The data are analyzed for each of the eight states separately. Since the data are hierarchical in nature, i.e., pupils are nested within schools, use of "Hierarchical Linear Model" (HLM)
STUDIES ON. DPEP 73
(Bryk and Raudenbush. 1992) in data analysis was considered appropriate to take care of the in-built data structure. The results of different states are synthesized by applying the meta-analysis method with known variance.
Review of Related Researches To evaluate the OB scheme several studies have been recently conducted by different agencies. Most of these studies (e.g., Gupta and Gupta. 1992: SANDHAN. 1992) attempted to find out the status of the implementation of the OB scheme. The study conducted by Muralidharan et. al.(1994). however. examined the impact of OB scheme on pupils achievement in different subjects including mathematics and language. This study was confined to Maharashtra and Tamil Nadu states only. The findings of the study revealed that the achievement of OB school pupils in all the subjects covered in the study was significantly higher than those studying in non-OB schools in Maharashtra. Whereas in Tamil Nadu pupils of non-OB schools performed better than their counterparts in OB schools.
The studies conducted in the area of incentive schemes are mainly concerned to assess their impact on enrolment and retention of children in schools. For example, Saxena and Mittal (1985) reported that the Gross Enrolment ratios of boys and girls were higher for the districts having mid-day meals programme than those without the programme. The study did not indicate definite impact of the programme on retention rates of boys and girls.
Sample of the Study The sample consisted of 1,746 schools, of which 1,045 schools are covered under the OB scheme. These schools were selected from 43 districts of the eight states covered under the BAS. Detailed information about school and pupil characteristics were collected through school and pupil schedules respectively. To estimate achievement of pupils on mathematics and language, standardized achievement tests (Shukla, et.al., 1994) were used.
Variables Used in the Study The input and outcome variables used in this study are defined in Table 1. It may be noted that the dummy and composite variables are either standardized or centered about the sample mean within each state.
Impact of Operation Blackboard Scheme Before assessing the impact of OB scheme on pupils achievement it is considered
74 IER: SPECIAL NUMBER 1995
TABLE 1
Variables at Pupil and School Levels
S.No. Label Variable Description Construction of Variable
I Pupil Background Variables
1. GIRL Gender Girl=1, Boy=0; centered at the pupil level
2. SCST Caste SC or ST=1, Non-SCST=0; centered at the pupil
level
3. DADED Father education Applied Logit
4. MUMED Mother education Applied Logit
5. DADOCC Father occupation Applied Logit
6. REPEAT Ever repeated a class Repeater=1,non-repeater=0; centered at the
pupil level
7. SES Socio-economic status Composite and standardized
II Contextual Variables
1. MEANSES School Mean SES Aggregated from pupil to school level; cen-
tered at school level
2. PCTSCST Per cent SCST Per cent of SC and ST in the school; centered
at school level
III Intervention Variables
1. OPBLACK Operation Blackboard OB Scheme= 1, Non-OB=0; centered at school
level
2. FACILOP Material available under Available=1, Not Available=0; Added all
OB scheme available items; centered at school level
3. MDMEAL Mid-day Meals Percentage of beneficiaries; centeredat schooI
level
4. UNIFORM Free Uniforms Percentage of beneficiaries; centered at
school level
5. TEXTBOOK Free Textbooks Percentage of beneficiaries; centered at
school level
6. SRATTEND Scholarship for regular Percentage of beneficiaries; centered at
attendance school level
7. OSCHOLAR Other Scholarship Percentage of beneficiaries; centered at
school level
worthwhile to compare the mean achievement scores of OB and non-OB schools. Table 2 gives the mean and standard deviation (s.d.) of language and mathematics achievement scores of pupils in OB and non-OB schools.
STUDIES ON DPEP 75
TABLE 2
Mean and S.D. of Pupils Achievement Scores
Sl. State School Number Math Scores Language Scores
No. Type of
Schools Mean S.D. t Mean S.D. t
1 Assam OB 74 19.93 5.00 2.13 21.79 5.13 2.93
Non-OB 87 18.02 6.19 19.60 4.39
2 Haryana OB 109 14.91 3.26 0.90 36.90 7.20 0.52
Non-OB 35 15.48 3.05 37.65 8.14
3 Karnataka OB 88 15-21 6.53 0.30 16.01 5.74 0.18
Non-OB 88 14.93 6.07 15.86 5.07
4 Kerala OB 54 15.66 2.15 2.62 21.21 3.39 1.40
Non-OB 59 14.36 3.00 20.33 3.31
5 Madhya OB 395 11.05 4.06 2.05 28.76 10.88 3.43
Pradesh Non-OB 338 10.45 3.77 26.00 10.86
6 Mahara- OB 95 10.83 4.67 2.04 15.43 5.72 2.24
shtra Non-OB 40 12.67 5.01 17.87 5.92
7 Orissa OB 136 13.53 5.47 0.02 32.87 8.83 0.66
Non-OB 29 13.50 4.98 34.13 11.40
8 Tamil OB 94 10.96 3.00 0.01 30.32 6.99 0.70
Nadu Non-OB 24 10.97 2.50 29.24 5.52
The mean achievement of OB schools is significantly higher than that of non-OB schools in the states of Assam and Madhya Pradesh in both mathematics and language. In Kerala, although the mean achievement of OB schools is higher for both the subjects, it is significant (t = 2.62) for mathematics only. In Maharashtra, howerver, the mean performance of non-OB schools is higher and significant than that of OB schools in both the subjects. No significant difference is observed between the mean achievements of OB and non-OB schools in the remaining states.
We now examine the impact of OB scheme on pupils achievement. Three input-process-output models(Willms, 1992), are used to analyze the data. Using broad terms, the models are described below:
Model 1
Pupil's achievement score = the average score of all pupils
+ the effects of pupil background
76 IER: SPECIAL NUMBER 1995
+ the effect of OB scheme
+ any unmeasured effects unique to the child's school
+ random error
Model 2
Pupil's achievement score = the average score of all pupils
+ the effect of pupil background
+ the effects of MEANSES+PCTSCST
+ the effect of OB scheme
+ any unmeasured effects unique to the child's school
+ random error
Model 3
Pupil's achievement score = the average score of all pupils
+ the effect of pupil background
+ the effects of MEANSES+PCTSCST
+ the effect of OB scheme + FACILOP
+ any unmeasured effects unique to the child's school
+ random error
Table 3 presents the results of the analysis based on the three models. Columns 4 and 7 of the table show the estimated effect size of the OB scheme on mathematics and language achievement respectively. These models use standardized achievement scores in mathematics and languages as the dependent variables.
Let us first consider results of model 1 which provide us estimates of the effect of OB scheme in the school, after adjusting for the effects of pupil background variables. It is evident from the table that there is a substantial gap in the performance of OB and non-OB schools on both mathematics and language achievements in the states of Assam, Kerala and Madhya Pradesh. The estimated achievement gap is statistically significant at less than .05 level in Kerala and Madhya Pradesh and at less than .10 level in Assam. It implies that in these states, on average, pupils in OB schools perform better than their counterparts in non-OB schools in both the subjects. The table further reveals that the OB scheme improves the mean performance of schools on mathematics achievement by 19 .3 per cent, 26.6 per cent and 13.2 per cent of their respective standard deviations in Assam, Kerala and Madhya Pradesh respectively. The corresponding percentages for language achievement are 27.5, 16.1 and 22.2. It indicates that the OB scheme did have a discernible impact on the mathematics and language achievement in these three states.
STUDIES ON DPEP 77
78 IER: SPECIAL NUMBER 1995
In the remaining states there is no evidence of OB scheme having any significant impact on pupils achievement in both the subjects. However, OB schools do perform slightly better than non-OB schools in Karnataka (in both mathematics and language) and Orissa (in mathematics only). In Haryana and Maharashtra, the mean performance is relatively low in OB schools as compared to non-OB schools. The estimated effect size is statistically significant at .10 level for Maharashtra only.
The model 2 adds school contextual variables, viz., school mean SES and proportion of SC/ST in the school, to the input variables of model 1. The analysis gives the estimated effect of the OB scheme on school mean achievement, after adjusting for the pupil background and the school contextual variables.
The achievement gap between OB and non-OB schools reduces and becomes insignificant at .10 level, after adjusting for the contextual variables, in Kerala on language achievement and in Assam on mathematics achievement. Further, the gap is still highly significant in Assam on language achievement and in Kerala on mathematics achievement. In Madhya Pradesh, the gap between OB and non-OB schools is significant at less than .05 level on both mathematics and language achievement.
In Maharashtra, where the gap between OB and non-OB schools was
STUDIES ON DPEP 79
significant at less than .10 level before modeling the school contextual variables becomes insignificant after adjusting for the effect of these variables.
We now examine the effect of OB scheme on school mean achievement Graphically. For this purpose two states, viz., Maharashtra indicating no impact and Kerala showing significant impact of the scheme, are chosen. The graphs are presented for Maharashtra in Figures 1 and 2 and for Kerala in Figures 3 and 4. In the figures, school mean achievement scores are plotted on the ordinate against OB and non-OB schools shown on the abscissa.
Figures 1 and 2 show a downward regression line indicating low performance of OB schools than that of non-OB schools. The figures clearly indicate that the performance of a few non-OB schools is much above the state average whereas a few OB schools are performing much below the average. It appears that probably the impact of OB scheme on achievement in Maharashtra is not visible due to these few schools. Figures 3 and 4 indicate an upward regression line both for mathematics and language achievement in Kerala, the OB schools performing higher. However, there are a few OB schools that are performing much below the average.
Table 4 presents the estimated achievement gap between OB and non-OB schools across the states.
80
STUDIES ON DPEP 81
TABLE 4
Achievement Gap between OB and Non-OB Schools across States (Meta-
Analysis)
Mathematics Language
Coeff. SE P-value Coeff. SE P-value
Average 0.091 0.039 0.04 0.095 0.051 0.075
Achievement
Gap
Random Variation Across States
Variance P-value Variance P-value
Average 0.004 0.142 0.010 0.029
Achievement
Gap
The meta-analysis of the effect sizes of all the eight states indicates that the average achievement gap between OB and non-OB schools in mathematics and language is 0.091 and 0.095 respectively, the OB schools performing hi-her. These achievement caps are statistically significant at .05 and .10 level respectively. It further indicates that while there is no variation among the states on average achievement gap in mathematics between OB and non-OB schools, there is a significant variation (p<.05) among the states on language achievement gap.