(VI)APPENDIX F--MEMORANDUM ON ITEM 7-- THE ESTABLISHMENT OF STATE EVALUATION UNITS TO PROMOTE EXAMINATION REFORM
It is a well-known fact that a programme of examination reform has been adopted by successive conferences of Chairmen and Secretaries of the Boards of Secondary Education since 1957. An Examination Unit consisting at present of 13 evaluation officers is functioning in the Directorate of Extension Programmes for Secondary Education under the Ministry of Education. This programme of examination reform is phased over a period of ton years and seeks to bring about a closer and more integral relationship between educational objectives, teaching procedures and learning processes. When implemented fully, it will not only have reformed the system of examination but will also have brought about a revision of the curriculum needed for the realisation of educational objectives.
A country-wide programme of this magnitude involving as it does nearly 13,000 Secondary schools and 14,000 teachers, requires a great coordination of effort between the different States. During the past sixteen months the Central Examination Unit has been conducting a number of workshops in evaluation and acquainted an appreciable number of teachers with the new evaluation approach. This is, however, only a small portion of the task to be done. Test items have to be prepared in accordance with the new objectives in the entire syllabus in each subject, these items have to be tried out and analysed, and finally the selected test material has to be made available to the different State Departments and Boards for internal and external assessment. This, in turn, will in natural course lead to a consequential reorganisation of the curriculum and adjustments and improvements in classroom teaching. It was recognised very early in the programme of examination reform that success would be possible only if the States continued the work of the Central Unit with their schools and teachers. The 1956 Bhopal Seminar on Examinations, therefore, recommended that "in each State the Board of Education of the type suggested by the Secondary Education Commission should set up a committee with a bureau attached to it for the constant appraisal and review of tests and other instruments and procedures of evaluation". The Seminar suggested that the work of the Bureau should include :
(i) The study and investigation of problems connected with tests and procedures of evaluation used at all stages of Secondary education, including the internal examination as well as the public examination.
(ii) The preparation of suitable examination papers which may, with refinement, attain the status of standardized achievement tests for the different subjects for use at the final examination. The following alternative procedures for the preparation of these tests are suggested :
(a) The Bureau may prepare a large number of suitable objective questions covering the entire syllabus, from which questions may be selected for the preparation of the examination papers to be given in a particular year.
64
OR
(b) The Bureau may prepare a number of alternative complete test forms (say, not less than five) on the entire syllabus in certain subjects. One of the alternative forms could be used as a test for a particular year.
OR
(c) The Bureau may prepare a number of alternative forms for each aspect or unit of a subject (e.g., the Physics portion of the General Science syllabus); the paper to be set for any particular year being prepared by putting together a suitable combination of the alternative forms from the required units.
(iii) The preparation of manuals containing detailed instruction for the administration and scoring of the tests and the interpretation of the results.
(iv) The preparation of manuals containing detailed instruction for drawing up questions of the essay type in the different subjects, with a view to reducing to the minimum such defects as subjectivity, lack of comprehensiveness and vagueness. The manuals should contain examples of good essay-type questions for the guidance of the paper-setters and instructions for marking and grading candidates on the basis of their performance.
(v) A scientific study of the problems of converting marks into grades and of combining the grades earned on the essay-type questions with the score on the objective tests.
(vi) A study of problems which are likely to arise when the school record and the marks at the public examination are combined in the final assessment of the candidate.
(vii) An investigation into the problems of relating the scores in the external examination with the scores on the internal tests and other school records for the purpose of the final assessment.
(viii) A scientific study of the questions set at the external examination and the reports of the examiners on the performance of the candidates.
(ix) A scientific study of the results subjectwise, schoolwise, districtwise, etc., with a view to effecting improvements.
(x) Exploring the possibilities of associating internal examiners with external examiners, as practised in certain European countries.
(xi) Exploring the possibilities of including the oral examination as an appropriate instrument of assessment.
In the programme of action which he outlined in 1957, Dr. Benjamin Bloom went further and suggested the establishment of State Examination or Evaluation Units for the purpose of carrying out the above work. He counselled, however, that the State should institute such Units only after a sufficient number of technically competent persons could be found or trained. He envisaged that within a three- year period State Examination Units would
65
become the foci of this work. The Conference of Secretaries of State Boards at its meeting last year resolved "that the State Examination Units should be set up as early as possible, preferably before 1961".
Although certain States, e.g., Kerala, Orissa and Bombay have set up Research Bureau on Examinations, no other State except Uttar Pradesh has so far reported the establishment of an Evaluation Unit. If our plan for the reform is to progress as steadily as scheduled, it is of primary importance that in the year before us we concentrate our efforts on setting up these Units so that at the commencement of the year 1960-61, the State Units may be in a position to implement the programme.
Personnel for the State Units As Dr. Bloom pointed out, the success and efficiency of these State Units depended largely on the personnel that ran them. It is obvious that such efficiency should be ensured by finding or training the required number of technical persons. A training programme for this personnel has already been envisaged in the scheme. Such training courses will be organised by the Central Unit in co-operation with experts from universities, training colleges and other interested bodies in the country under the guidance of the expert in evaluation from the Ohio State University team of Consultants.
It is expected that each State Unit will, at the initial stage, have the following posts :-
One Director
Three Evalution Officers- one for language subjects,
one for Humanities and one for Science
One Statistical Officer
Technical Assistants such as a translator
or a computor, and other clerical staff
Out of the above, it will be necessary to give training in the new techniques to the first four officers. States like Orissa, Kerala and Mysore have also made requests for providing training for the officers of their Examination Bureau, and their paper-setters. It may, therefore, be taken that on an average four to five officers from each State may be given training in the new technique as a first step. This will work out to about 60 individuals in all the States and Territories.
The provision of training for these persons would necessitate
(i) the immediate creation of new posts by the respective governments;
(ii) the selection of persons required to fill up the posts in such a manner that, they cover the major areas of knowledge, namely, languages, natural sciences and social studies.
As for the training courses, the Conference may decide whether it would be convenient to have them on a regional or on an all-India basis. The above measures shoud be taken well before June, 1960 so that the new and duly trained personnel is ready to man the State Units from the academic year 1960-61.
Nature of Training The course of training should cover :-
(i) the new approach to evaluation and theory of evaluation;
(ii) processes and techniques involved in this new approach;
66
(iii) some training in the statistical aspects of evaluation;
(iv) practical training with the evaluation officers in different subject fields;
(v) participation in workshops; and
(vi) a course of special training.
Functions of State Evaluation Units The functions of the State Evaluation Units will fall under the belowmentioned categories.
The problem upon which the Units will turn their immediate attention will be :-
(i) Preparing tests, objective and essay-type, based on educational objectives in all the subjects and manuals containing detailed instructions about evaluation, making suggestions about their use for external examinations and internal assessment;
(ii) Standardising such tests;
(iii) A scientific study of the system of internal assessment, determining procedure and methods of incorporation;
(iv) Carrying out research on problems related to examinations, and studying examination results; and
(v) Helping schools in using standardised tests for internal assessment.
Co-ordination with the Central Unit As the State Units get established and engage themselves in the activities mentioned above, the Central Examination Unit would gradually assume a somewhat different role than at present :-
(a) It will engage itself in the preparation of test material on objectives other than those covered at the initial stages;
(b) The introduction of reform and the study of test results or other relevant problems by the State Boards would require exchange of ideas and experiences for mutual benefit. It would function as a clearing house for the State Units in collecting and disseminating information on the subject;
(c) It will undertake research in problems referred to it by the State Units or the Boards;
(d) It will also assist and coordinate research work in the field of examination;
(e) It will bring out publications on
(i) guides or manuals illustrating test materials and learning experiences in relation to selected new objectives for the use of teachers and examiners; and
(ii) pamphlets or papers presenting the results of research on various aspects of the reform.
The Third Conference of Secretaries of the Boards of Secondary Education which was held in September 1959 at Delhi resolved "that State Evaluation Units should be established by the year 1960-61 and that training
67
courses may be organised by the Directorate on an all-India basis for the officers of the State Units in suitable batches." The information so far received from the different States is consolidated in the attached annexure, (Annexure C).
Provision has also been made in the Third Plan for the establishment of such State Units as Centrally aided schemes.
ANNEXURE C
Information about the, Establishment of State Evaluation Units
I. ORISSA An Evaluation Unit has been set up by the Secondary Education Board since 28th February, 1959. It also functions as a Research Bureau. Its specific functions are to promote measures of reform in examination at the State-level as suggested by the Centre, and to carry on independent research ancillary to such reform.
Staff : The Unit is in the charge of one Research Officer who holds a Master's degree in Education. The scale of pay of the post is Rs. 200-15-260-25--435-EB-25-610-EB-30--700.
Work done so far : At present the Unit is carrying out item analysis of the two tests on General Science (comprising 90 items), and Social Studies (comprising 80 items) administered on 320 students of nine selected schools in the State.
Co-ordination between the Unit and the State Education Department The necessary co-ordination exists through the Board, the President of which is also the Director of Public Instruction of the State.
Requirements : The Centre may give more concrete suggestions to accelerate the work of the Unit and supply the test materials as early as practicable. The Board has made a provision for the appointment of a Research Assistant for the State Evaluation Unit. If the Unit is to have the staff suggested by the Conference of Secretaries of Boards, it will need assistance from the Centre to the extent of 50 per cent of the expenditure of the Unit.
II. DELHI The Board of Higher Secondary Education, Delhi considered the question of setting up an Evaluation Unit at its meeting on 10th November, 1959, and resolved to recommend to the Delhi Administration that an Evaluation Unit be appointed so as to bring about reform in the examination system, as recommended by the Directorate of Extension Programmes for Secondary Education.
The State Education Department (i.e. the Delhi Administration) has not appointed an Evaluation Unit or a Research Bureau so far.
III. ANDHRA PRADESH No State Evaluation Unit has been set up. Resolution XX of the Conference is under consideration of the Director of Public Instruction.
The State Government has set up an "Examination Reform Committee" for suggesting ways and means for implementing the recommendations made on Examination Reforms by the Bhopal Seminar. It also examines any recommendations made on the subject of evaluation and examination reforms by different seminars and conferences.
The Committee consists of-
(1) The Director of Public Instruction;
69
(2) The Principal, College of Education, Osmania University, Hyderabad;
(3) One representative of the Osmania University;
(4) Two experienced headmasters;
(5) Director, State Bureau of Educational and Vocational Guidance;
(6) Three coopted experienced members.
The Committee met in August-September, 1959 and its recommendations have been submitted to the Director of Public Instruction for circulation among the teaching faculty and collecting their views.
The State Bureau of Vocational and Educational Guidance is functioning in the State since 1st April, 1958. It is engaged in the work of constructing objective tests of attainment for the purpose of making them available to the Secondary schools in the State. Objective tests constitute one of the items in the cumulative record in the State. The Bureau also takes up research on topics relating to examination reform. Special Subject Inspectors have been appointed in the office of the Director of Public Instruction to visit schools and guide them in the teaching of the respective subjects. They also help in the work of the Bureau. The Bureau is staffed with a Director, two Counsellors, and two Technical Assistants.
IV. S.S.C. BOARD, POONA No Evaluation Unit has been set up by the Board.
The Board has set up a Research and Investigation Section. This section is functioning since July, 1955.
Functions : Its special function is to institute investigations pertaining to the various problems connected with the S.S.C. Examination, as suggested from time to time by the Examination Reform and Research Committee of the Board.
Staff: One Investigation Officer on Rs. 350.
Work done so far : Completed eight projects related to a statistical analysis of the results of the S.S.C. Examination. No coordination between the Research Section of the Board and that of the State Education Department has been thought of, for the Research and Investigation Section has been concerned with statistical analysis of the results of the S.S.C. examinations and the Research Section of the Department of Education is mainly concerned with administrative problems.