ACHIEVEMENT DIFFERENCE AND SCHOOL EFFECTS
National Council of Educational Research and Training New Delhi
ABSTRACT This paper attempts to study the effects of school related variables on pupil's achievement using the data of Baseline Assessment Studies recently conducted in the eight states of India. Twenty-five indicators both for pupil and school levels are used in Hierarchical Linear Model separately for each state. The contribution of factors of teacher quality, school resources and school academic climate are studied after adjusting for pupil's background and school context variables. The eight states results are also synthesized. It is found that there exist gender and SC/ST gap in achievement. Mother's and father's education and father's occupation are positively associated with the pupil's achievement. The school level factors of academic climate (test and feedback, homework, etc.) and teacher quality (teachers stay, teaching experience, etc.) are the prominent contributors as compared to those of school resources (educational and physical facilities.)
Introduction Universalisation of Elementary Education being the national priority, the National Policy on Education, 1986 emphasised that new thrust in elementary education
2 IER: SPECIAL NUMBER 1995
will focus on three aspects: (i) universal access and enrolment, (ii) universal retention of children up to 14 years of age, and (iii) a substantial improvement in the quality of education to enable all children to achieve essential levels of learning. In order to achieve these, an array of micro-planning-based strategies are being formulated for application at the grassroots levels. One of them is the district specific planning to develop the educationally backward districts as envisaged in the Programme of Action 1992. In this direction, in 1993 a new initiative, viz., the District Primary Education Programme (DPEP) was taken.
The District Primary Education Programme The DPEP focussed on development of district specific plan by keeping in view the following parameters:
(i) The emphasis on local area planning while formulating the district plans.
(ii) Greater rigour and infusion of professional inputs in planning and appraisal.
(iii) More focussed targeting-educationally backward districts with female literacy below national average and where Total Literacy Campaign has been successful.
(iv) The coverage will be focussed on primary stage with stress on girls and socially disadvantaged groups.
To begin with, the DPEP has been taken up in 44 districts in the states of Assam, Haryana, Karnataka, Kerala, Madhya Pradesh, Maharashtra, Orissa, and Tamil Nadu. As a part of the project formulation process, Baseline Assessment Studies (BAS) were undertaken to provide research based support to the district plans. The main thrust of the study was to assess the learning achievement of students approaching the end of primary school cycle, i.e., grade IV or V in reading (language) and mathematics based on grade III or IV curriculum as per the state pattern.
The present paper attempts to study the effects of school policies and practices on learner's achievement using the BAS data. The main objectives of this paper are to study-
(i) The variation in achievement in mathematics and language within and between schools;
(ii) The effects of pupils background on their mathematics and language achievement;
(iii) The between schools variation in mathematics and language achievement after adjusting for student's background and school context;
(iv) The effects of teacher quality on achievement in mathematics and language across schools;
(v) The contribution to mathematics and language achievement between schools by the school resources;
STUDIES ON DPEP 3
(vi) The association of school academic climate with the pupil's achievement between schools.
Review of Related Literature In India, Kulkarni (1970) conducted the first major study on achievement survey in mathematics covering the three levels of education, viz., end of primary (grade V), end of middle (grade VIII) and end of secondary stage (grade X.) in 15 states. The number of students in the sample varied from more than 28,000 at primary level to nearly 20,000 at the secondary level. The major findings of the study for the primary level were: boys achieved higher than girls; the socioeconomic conditions of the parents of school type (e.g., government or private management) provided better teaching-learning situations; and no relation Was found between school achievement and teacher qualifications. This study was conducted about 29 years back (in 1966-67) and since then considerable changes have taken place in the Indian system of school education.
The Fourth Survey of Researches in Education in India, 1983-88 (Buch, 1991) reported a few studies addressed to the achievement at the primary stage. Most of these (83 per cent) were doctoral studies. Besides, all were based on small samples and confining to a limited geographical area. The same pattern of researches continued as indicated in Fifth Survey of Educational Researches (Dave and Murthy, 1993).
Dave (1988) and his colleagues conducted an evaluation of achievement of primary school children studying in classes I to IV in 22 states/union territories in India. Although the main objective of the study was to compare the impact of inputs of the special project, Primary Education Curriculum Renewal, it also had samples of pupils from non-project schools varying from 1800 in class I to less than 200 in class IV. Large differences between states were observed in this study as well.
Recently, Shukla (1994) conducted another study to find out the level of attainment of primary school children in various states in India. A sample of about 66,000 students (studied up to grade IV) was covered for 25 states and the union territory of Delhi. These students were administered an Arithmetic Test and a reading Comprehension Test. The study revealed that over the states, the difference between the mean achievement of boys and girls did not have the same direction. In some states boys did better than girls whereas in some other states girls did better than boys. For the entire country the SC/ST pupils performed lower than the non-SC/ST ones. Further, the pupil's achievement was found to be positively related with father's education, facility for learning and educational environment at home. The variables related to schools and teachers indicated somewhat weak relationship with achievement.
4 IER: SPECIAL NUMBER 1995
Jangira (1994), while synthesizing the results of BAS of the eight DPEP states, found that students performed low in reading as well in mathematics. There was a marked difference in achievement across schools as well across states. With the help of regression analysis, a few individual and group level variables were found as significant predictors. The individual level. predictors for language achievement were : educational and occupational aspiration; teacher coming to class regularly; TV-watching; receiving dictation and feedback on tests; opportunity to read material other than textbooks; language of instruction and difficulty in understanding teacher's language in the classroom; availability of language textbook. For prediction of mathematics achievement, the individual level variables were : father's education, reading other material, correcting homework regularly, educational and occupational aspirations, and understanding teacher's language. The group level predictors, viz., asking children to read from textbook on their own and reading from the textbook and explaining were predictors for language achievement. For mathematics achievement, class size and teacher's expectations were found to be the significant predictors. The variables identified at individual as well at group level for mathematics and language did not indicate any consistent behaviour.
It can be summarised from the findings of the above studies that there was no consistency of pattern in relationships between the achievement and pupil and school related variables. However, the achievement differed widely not only between schools but also over the states. It is, therefore important to study the school related variables having effects on learner's achievement.
Data and Methods The 44 DPEP target districts, having female literacy below the national average, were selected from eight states. About 20 per cent of rural blocks and urban areas identified in 1991 census were randomly selected. Wherever tribal block did not appear in the selection one tribal block was added to the sample. Further, 3545 primary schools were randomly selected from each district on the basis of proportionate allocation considering the rural and urban population. Up to 30 all class IV or V students, and in case of more than 30, only 30 students were randomly selected. Students of class V were selected in the states of Haryana, Madhya Pradesh Orissa and Tamil Nadu; and students of class IV were selected in Assam, Kerala, Karnataka, and Maharashtra. Up to 5 teachers including the head teacher all, and if the number exceeded 5, five teachers were randomly selected. Following this procedure, the data relating to 1,746 schools, 23,700 students and 4,879 teachers were finally included in the analysis. The state-wise details are given in Table 1.
STUDIES ON DPEP 5
TABLE 1
State-wise Sample of Districts, Schools, Teachers and Students
State Districts Schools Teachers Pupils
Assam 04 161 439 1706
Haryana 04 144 531 2489
Karnataka 04 176 437 2568
Kerala 03 113 502 3089
Madhya Pradesh 18 733 1701 8010
Maharashtra 09 135 409 2143
Orissa 04 165 515 1364
Tamil Nadu 03 119 345 2323
Total 43* 1746 48792 3700
* One district data of Madhya Pradesh are excluded due to non-
availability of achievement scores on language.
Students of class V were administered the standardized tests on mathematics (40 items) and reading (84 items). For students of class IV the tests in mathematics and reading had 40 and 44 items respectively. In addition to the achievement scores other information related to pupil, school and teacher were collected through the structured interview schedules, namely, Student (Present) Schedule (SS), School Record Schedule (SRS) and Teacher Schedule (TS).
In all twenty-five indicators consisting of six at the pupil- level and nineteen at school-level were developed. In addition to these, two variables, viz., mean SES and per cent SC/ST at school- level were also included as school intake composition. The scores in mathematics and language were standardised with mean zero and standard deviation one. The details about indicators are given in Appendix A. Further, the state-wise basic statistics, viz., mean and standard deviation for each indicator are given in Appendix B.
A multilevel regression analysis fits a hierarchical linear model (HLM) to data that are organized hierarchically (Bryk & Raudenbush, 1992); in this case students are nested within schools. It enables the researcher to partition the variation in a variable into within and between schools, and to examine the relationships among variables both within and among schools. The first HIM analysis called a null model because it does not include any student background co-variate, provide the variation within and between schools. The standardised scores in mathematics and language were independently included for the purpose. Then, the pupil's background variables are included to explain the within as well between school variation. The student background variables which did not
6 IER: SPECIAL NUMBER 1995
indicate significant variation across schools were constrained. The reduction in the within as well between school variance for pupil's background variables also provide the adjusted school means.
The mean SES and per cent SC/ST of the school are considered important as the intake composition of a school, which can have a "contextual effect" on student achievement over and above the individual characteristics. These two variables were included in the analysis for further adjustments of the school means.
The three sets of school level variables were included in the HLM analysis independently after adjusting for the pupil's background and contextual variables. This approach was adopted for all the eight states. In order to have synthesis of the state results the regression coefficients and their standard errors with probability level less than 0.20 for each variable was taken for applying meta- analysis with the help of HLM computer programme.
Results
EFFECTS OF PUPIL'S BACKGROUND VARIABLES The variation within and between schools before and after adjusting for pupil's background variables for mathematics and language in respect of all the eight states are given in Table 2. The achievement in mathematics vary substantially
TABLE 2
Unadjusted and Adjusted Within and Between School Variances for Pupil's Background
In Mathematics and Language
Variance Assam Har- Karna- Kerala M. P. Maha- Orissa Tamil
yana taka rashtra Nadu
Mathematics - Unadjusted for Pupil's Background
Between School 48.3 31.3 60.6 19.6 45.6 39.3 44.1 36.0
Within School 51.7 68.7 39.4 80.4 54.4 60.7 55.9 64.0
Mathematics - Adjusted for Pupil's Background
Between School 44.8 27.1 53.6 19.6 42.6 35.6 40.4 31.1
Within School 43.7 62.0 32.6 69.0 50.5 42.5 52.5 56.8
Language - Unadjusted for Pupil's Background
Between School 35.0 35.6 45.2 13.7 45.4 37.7 29.3 32.8
Within School 65.0 64.4 54.8 86.3 54.6 62.3 70.7 67.2
Language - Adjusted for Pupil's Background
Between School 30.6 32.4 37.8 14.7 42.4 32.4 25.7 30.3
Within School 55.8 59.5 39.8 74.9 48.8 47.1 65.6 56.1
STUDIES ON DPEP 7
within schools from highest 80.4 per cent in Kerala to the lowest 39.4 per cent for Karnataka. In language the corresponding position is 86.3 per cent for Kerala and 54.8 per cent for Karnataka. It indicates that the variances in achievement within schools are substantial and they vary across states. Mostly the within schools variances are uniformly lower in mathematics than the language.
The variation in achievement between schools for mathematics and language for each state are statistically significant. These variances also vary across states. Kerala indicates the lowest percentage for variances for both mathematics (19.6 per cent) and language (13.7 per cent); the highest percentage of variance between schools for mathematics is in Karnataka (60.6 per cent) and for language 45.4 per cent in Madhya Pradesh. However, these variations are uniformly lower in language as compared to mathematics.
The within schools variances adjusted for pupil's background are reduced substantially for mathematics, ranging from 6.1 to 29.7 per cent, and the reduction for language is ranging from 7.3 to 28.8 per cent. Similarly, the reduction in variances between schools for mathematics and language are ranging from 1.7 to 13.6 per cent and 6.6 to 16.4 per cent respectively with the exception of Kerala state having no such reduction.
EFFECTS OF PUPIL's BACKGROUND AND SCHOOL CONTEXT VARIABLES The effects of pupil-level variables on mathematics and language are given in Table 3 and Table 4 respectively. They also include the effects of school intake composition.
The results show that there are large and statistically significant differences between boys and girls within schools in their achievement in mathematics in the states of Assam, Haryana, Karnataka, Madhya Pradesh and Orissa. These differences are also found statistically significant in language achievement for all the states except Haryana and Kerala. These differences vary significantly among schools in all the states. Mostly the gender differences are greater in mathematics than in language.
There are large and statistically significant gap in mathematics achievement of SC/ST and non-SC/ST students with the exception of Haryana, Orissa and Tamil Nadu. In language achievement the gap was significant in Haryana, Kerala and Madhya Pradesh. However, in most states these do not vary significantly among schools. Karnataka and Kerala are the two exceptions in language achievement.
Father's education within schools has shown a positive and highly significant association with pupil's achievement in mathematics and language in all the states except Karnataka, Maharashtra and Tamil Nadu for mathematics, find Haryana for language. Similarly, the mother's education has displayed highly significant
8 IER: SPECIAL NUMBER 1995
STUDIES ON DPEP 9
10 IER: SPECIAL NUMBER 1995
mathematics, and Assam and Maharashtra for language.