 READABILITY ASSESSMENT OF PRIMARY
 LEVEL TEXT-BOOKS

National Council of Educational Research and Training New Delhi

ABSTRACT

Grade III textbooks in language, environmental studies and mathematics in the major language of each of the six DPEP States namely Assam, Haryana, Kerala, Karnataka, Maharashtra and Tamil Nadu were evaluated in terms of their Readability level with the objective of providing information for inservice teacher training in use of existing textbooks and guidelines for preparation of new textbooks. Readability was considered in terms of Vocabulary, Conceptual- difficulty and Syntax. The study investigated -- (a) extent of readability of each textbook, (b) areawise differences, (c) parity in readability across three subject areas. A triangulated research design was adopted to include - (i) linguistic-analysis of each textbook, (ii) direct assessment of children's comprehension level for each textbook at the end of the session, (iii) teacher interview. Results of the study indicate state and area-wise differences in readability of textbooks. Variations in parity in readability level across subjects were also observed. Implications of results in terms of textbook preparation, teacher training and methodological issues are discussed.

Context

Any organised teaching-learning situation can be basically construed as consisting of a specific curriculum which, in turn, would include the syllabus, the instructional materials and the transactional strategies. The ultimate efficacy of the learning situation would thus be dependent to a large extent on the relevance of each of these components in the context of the learner and the learning

STUDIES ON DPEP 251

outcomes. In the national context today the significance of an appropriate curriculum becomes even more critical in view of the efforts towards universal primary education wherein the diversity of students' needs will inevitably become still greater and so will the need to adapt educational approaches to match curriculum, instruction and learner.

In the Indian situation, at the primary stage the textbook continues to be the most essential and in majority of cases the only aid in the hands of the teacher and the learner through which the given curriculum is transacted. This situation consequently places a heavy responsibility on the quality of the textbook for ensuring effective teaching-learning interactions and outcomes. For any instructional material to be effective, it has to be necessarily planned keeping in view the requirements of the learners particularly with reference to their age, their educational and cultural background, their immediate environment as well as their proficiency in the language which is the medium of instruction.

A large number of textbooks are being prepared in India every year, both at the central level and in the different states. Recent researches, however, have led to the speculation that the effectiveness of these textbooks is often restricted because of a lack of parity between the skill levels required by the textbooks and those available with the students. Often the distance between the home language and the standard language used in the textbooks also serves as a contributing factor. There is a dearth of research input or base to facilitate informed decisions regarding appropriateness and sequencing of content to match the linguistic competence of the target group. As a result neither any state nor any central agency has specified any language content to be used in the textbook in linguistic terms. Norms of linguistic competence of our children are also not defined. The authors do not get any specific guidelines or training and are often not even familiar with the needs of the stage of education for which they are writing.

From the point of view of the quality of a textbook its level of readability becomes a critical factor which could help or hamper the understanding or learning by the learner depending on the extent of match/mismatch between the linguistic competence demanded by the text and that available with the learner. This factor is critical not only in the language readers but also in the textbooks for other subject areas like Environmental Studies and Mathematics particularly in grades 3, 4 and 5 wherein higher level of conceptual understanding is required. To facilitate this understanding the readability of the text used must be ensured in these textbooks. However, often the level of EVS and mathematics textbooks are observed to be pitched even higher than the language readers that are in use for the same grade. There is an imperative need, therefore, for an in-depth empirical study of the existing textbooks in the above context in the three main curricular areas.

Recognizing this need, NCERT was commissioned by the World Bank in

252 IER : SPECIAL NUMBER 1995

February 1995 to conduct a quick study of a short duration on these very fines in collaboration with CIIL, Mysore.

Objectives and Scope of the Study

The objectives for the study were "to evaluate the readability level of primary level textbooks, provide information for inservice teacher training in the use of existing textbooks; and guide the preparation of new textbooks so that they are more readable."

Readability has been defined as, "the sum total (including the interactions) of all those elements within a given piece of printed material that affects the success, a group of readers have with it. The success is the extent to which they understand it, read it at an optimum speed and find it interesting" (Dale and Chall, 1949).

It could thus encompass several quantifiable and non-quantifiable dimensions including legibility of print, illustrations and colour, vocabulary, conceptual difficulty, syntax, text organisation, text redundancy, etc. For the purpose of this study, however, considering its limited scope and focus, readability has been considered in terms of Vocabulary, Conceptual difficulty and Syntax.

This study was limited to Grade III textbooks in language, EVS (I and II) and Mathematics in the six DPEP states, viz., Kerala, Karnataka, Tamil Nadu, Haryana, Maharashtra and Assam.

Research questions

The study has primarily addressed the following research questions that have emanated from the objectives :

1. To what extent are the textbooks "readable" (in terms of the specific components, i.e. vocabulary, conceptual difficulty and syntax) and cater to the diverse linguistic needs and levels of children in different geographical contexts of a state?

2. Is there parity in terms of readability of the textbooks across the three subject areas?

Review of Related Studies

Studies conducted in the area of readability have been guided by three main purposes - (i) to identify factors which validly distinguish easier from difficult material, (ii) to find a reliable means of measuring these factors, and (iii) to formulate some kind of an index through combination of these factors for predicting readability. These have led to the following three main categories of research in the area of readability

STUDIES ON DPEP 253

I. Quantitative Associational Studies : Quantitative associational studies have been the most characteristic of readability measurement. The typical product of these studies has been the various readability formulae based on the counting and weighing of several significant factors in the printed material, to predict the reading skills necessary to understand it. The first phase of these studies emphasized primarily the vocabulary factor (Patty and Painter, 1931; Stone, 1938). Later the focus shifted to other factors which could be significant predictors of difficulty in addition to vocabulary. Initially a large number of factors were identified, but were reduced to only a few in view of the high intercorrelations among them. Large, (1949), Yoakam (1948), Flesch (1948), Dale and Chall (1949), Fry (1990), etc., used specialised criteria in their studies on readability. The two factors common to all formulae had been vocabulary difficulty and sentence length.

Studies conducted in the last two decades have continued to delve further into factors affecting readability including discourse structure (Cooper, 1970; Moir, 1970; Alliende, 1987, 1990; Gourley, 1984). Gross and Sadowski (1985) gave a microcomputer programme called FogIndex but also, concluded that no readability estimate can replace commonsense and individual judgement.

II. Surveys of Experts and Reader's Opinion : Surveys of experts and reader's opinion in the area of readability have been very few but significant, These surveys have, while confirming the importance of stylistic factors also called attention to the importance of other factors not measured by formulae, i.e., content, format and organisation features and factors of readers' interest and motivation (Gray & Leary, 1940; Strang, 1948; Kukemelk & Mikki, 1993).

III. Experimental Studies : Experimental studies have generally attempted to assess the impact of planned intervention on readability. These have investigated effects of factors like redundancy, style of narration, inclusion of direct conversation, etc., on readability (Englemann, 1936; Burk, 1936; Wilsons, 1948; Horning, 1987; Currie, 1990 and Siegal et. al., 1974).

Very few studies have been conducted in the area of readability of textbooks in India as yet. A major study of direct relevance is that of Sharma (1993) which investigated the comprehensibility of language of class 3 textbooks of language, Science and Social Studies of Rajasthan. The study included a linguistic analysis of the textbooks as well as a collection of spoken and written language of children of grade 3 and made a comparative analysis. The study did highlight some inconsistencies between the language used in the textbooks and the spontaneous language of children, particularly in terms of vocabulary and sentence length. It also brought forth differences between urban-rural groups and reiterated the need to cater to both.

Agnihotri and Khanna (1991) and Kapoor et. al. (1993) applied the well-known Fog formula (Gunnings, 1952) and Smog formula on Hindi textbooks

254 IER : SPECIAL NUMBER 1995

using different samples. But the result invariably places the readability level of the examined textbooks to be much above the intended level. They have also reported a high correlation between performance of children on the cloze and comprehension tests.

The dearth of studies specifically in the area of readability of textbooks in India, however, only reinforces the need and priority for such studies, since these could contribute significantly to the improvement of quality of instructional material being produced for the primary grades.

Methodology

Design : A triangulated research design was adopted to include

- content (linguistic) analysis of each textbook;

- assessment of readability through direct testing of comprehension level of children for each textbook with respect to the linguistic content.

- interviews of teachers to invite their opinions on readability of the textbooks and the ways in which they use the textbook in their classroom transactions.

Procedure

Identification and Content (Linguistic) Analysis of Textbooks : For the purpose of content analysis of textbooks for Grade III prescribed in six DPEP states in the areas of Language, Mathematics and Environmental Studies (I and II) were identified. The languages covered were Assamese (Assam), Hindi (Haryana) Marathi (Maharashtra), Kannada (Karnataka), Malayalam (Kerala) and Tamil (Tamil Nadu).

Content analysis of each textbook was done on sample basis for Vocabulary (nouns, adjectives, adverbs and verbs), sentence type and average length of sentence. For the purpose of applying the Readability Formulae, specifically the Fog Index and Fry's Extended Readability Graph, three separate samples were selected from each of the language and EVS I and II textbooks and appropriately analyzed.

Test Construction : Comprehension Tests - For direct assessment of readability based on performance of Grade III learners, curriculum based assessment tests were constructed. "Curriculum based Assessment is asking students to perform tasks that have been drawn directly from the curriculum and then using assessment results to adapt instruction to reflect the learners' needs" (Hargis, 1987, Shinn 1989). The tests included

(a) Cloze tests

(b) Passage Comprehension tests and

STUDIES ON DPEP 255

For Mathematics only vocabulary test was used.

Reading and Writing Test: For this test, ten most frequently occurring words were identified on the basis of the linguistic analysis of each textbook. These ten words were listed out. These were dictated to a sample of ten children selected randomy who were subsequently asked to read the words written by themselves.

Sample

Comprehension Test : The sample drawn was purposive to certain extent since only those schools were selected which were known to be relatively more regular in their functioning. This was done to ensure that poor schooling may not confound the readability factor as reflected in the children's achievement for this study. All students in grade III of selected schools were covered to optimize the possibility of getting a normal distribution. Both boys and girls students of each area, i.e., Urban, Rural and Tribal were included in the sample. Table I gives a complete picture of the sample from the concerned states.

256 IER : SPECIAL NUMBER 1995

Reading and Writing Test : A sample of 10 children of grade III were selected on random basis from each school. In all 70 children constituted this sample, of these 40 children were urban (10 each from four states Assam, Haryana, Maharashtra and Karnataka), 20 Rural (10 each from two states Assam and Haryana) and 10 were tribal (Assam).

Interview Schedule for Teachers : A semi structured Interview. Schedule was constructed for teachers to elicit their opinions regarding the readability level of the textbooks with reference to the target group, and the ways in which the textbooks were used by them in day-to-day teaching. In all 84 teachers were interviewed accross the six states.

Analysis

The data received from the six states were scored and tabulated in terms of:

(i) Content Analysis : (a) consolidated frequencies of nouns, verbs, adjectives, adverbs, (b) total number of simple, complex and compound sentences, (c) average number of words per sentence.

(ii) Readability Assessment : The data have been analyzed into: (a) cumulative frequencies which have been drawn up for comparison of distributions across subjects/groups, (b) significance of difference between means of different sub-groups, (c) correlations between scores obtained in different subjects/item-types.

(iii) Teachers' Views on Readability and Use of Instructional Material Frequency analysis was undertaken to get a profile of teachers' views on readability.

(iv) In addition to this the Fog Index and Fry's Extended Readability graph were applied to sample text drawn from each textbook to arrive at a ready indicator of the readability level of each textbook.

Results and Discussion

Prior to discussion of the results obtained from the study it is imperative to emphasize that this study is essentially exploratory in nature and its results can at best be indicative only of possible trends, not definitive judgements or conclusions. The reason for this is that due to acute time and budgetory constraints, the study had to be necessarily limited in scope in terms of

(a) Its treatment/definition of the concept of readability so as to include only the minimum possible aspects.

(b) Selection of sample which was essentially purposive in nature and in no way representative of the larger population.

STUDIES ON DPEP 257

Yet, the triangulated nature of the design of the study serves to validate the findings to some extent through supplementing/complementing the information from one source of data to another which thus emerges as a strength of the study.

The results of the study are discussed in terms of the two research questions specified earlier.

How READABLE ARE THE TEXTBOOKS ?

Readability of the textbook for the six states in each subject is discussed taking the consolidated view as provided from the three sources of data for the total sample as well as sub-groups.

(a) * Direct assessment of children on comprehension tests

(b) Content analysis of textbooks

LANGUAGE TEXTBOOKS

With respect to children's performance (Table 2) it is evident that from the frequency distribution pattern language textbooks tend to be toward easier side for Maharashtra, Assam, Kerala and Tamil Nadu (31 per cent to 70 per cent scoring over 75 per cent) and relatively difficult for Haryana and Karnataka (more than 50 per cent children scoring less than 50 per cent). Area-wise differences (Table 3) have also emerged on children's performance on readability tests. While urban children scored significantly higher in Assam, Tamil Nadu and Haryana. In Tamil Nadu rural tribal differences were also significant in favour of the rural sample. In Maharashtra and Karnataka the rural sample significantly outperformed the Urban and tribal samples with the urban being lowest in Maharashtra and tribal in Karnataka (p<.01). Interestingly, in Kerala the tribal children scored highest followed by rural and then urban (p < .01). In terms of actual readability of the language textbook in Assam, Karnataka, Haryana and Tamil Nadu the readability is low for the rural sample only while in Karnataka and Tamil Nadu it is low for all the three sub-groups. In Maharashtra, it is possibly on the easier

READABILITY ASSESSMENT OF PRIMARY LEVEL TEXT-BOOKS