VIEWS ON PUBLIC EXAMINATION PROBLEMS
 

National Council of Educational Research and Training New Delhi 16

December 1970

NOTE: This Appendix contains a selection of views expressed on several aspects of Public Examinations

        
             No.            Item                                Page
        
             I.   The Problem of Public Examinations              1
                  (Sir David Anderson)
        
             II.  Reliability of Essay Type Examination 
        `         Results                                         1
        
             III. An Analysis of an Examination Paper             2
        
             IV.  Significance of Results of Public 
                  Examinations                                    3
        
             V.   Examining Project Work-Physics                  3
        
             VI.  Examining Project Work-Chemistry                7
        
             VII. Reservations on Internal Assessment            10
        
                                          

I. THE PROBLEM OF PUBLIC EXAMINATIONS *1

SIR DAVID ANDERSON

"The problem of examinations is giving rise these days to much speculation, concern, and even alarm amongst students, parents, employers, and the public generally; and not least amongst those responsible for setting and running the examinations.

"Examinations and their results are no longer taken for granted: in some quarters they have become suspect. though much of this suspicion may be unwarranted. Nevertheless, our more accurate knowledge of the complexity of the educational process and the human mind justifiably shakes the confidence previously felt in examination results.

"We are now asking ourselves what it is that we are trying to measure? How accurately are we able to carry out the measurement? What is the relevance and correlation between the results of this measurement and the future academic and professional performance of the person concerned? These are the questions to which we have become alerted, and we cannot yet answer them to our satisfaction.

"In addition to all this, there is the interconnected though totally different problem of the sheer magnitude of the task of organising the whole examination procedure for the numbers now involved.

"All these problems require serious consideration and discus- sion.:"

II. RELIABILITY OF ESSAY TYPE EXAMINATION RESULTS *2

"The early investigations on examinations dealt mainly with the reliability of examination results. Hartog and Rhodes in their

1 Conference on Examinations, April 1968, Scottish Certificate of Education Examination Board, page 4.

2 Conference on Examinations, April 1968, Scottish Certificate of Education Examination Board, pp. 6-7.

2

classical study 'An Examination of Examinations' revealed the weakness of assessment by the essay type examination: irrespective of the subject, markers gave widely different assessments to the same scripts and could riot guarantee to award anything approaching the same marks to the same scripts after the lapse of a period of time. In one investigation, 14 examiners awarding a pass/fail verdict to 15 history scripts after an interval of about one year gave a different verdict in 92 cases out of the 210 on the second occasion. These experiments have been repeated over and over again. For example, I demonstrated this by taking a 12 year old's essay from our transfer tests in Fife and sending it round a large number of teachers asking them to mark it on the scale A to E; this essay received every mark from A to E. A similar investigation with a leaving certificate essay did not however have a wide range of marking. Teachers with leaving certificate experience awarded it nearly the same mark. Quite clearly the teachers at this stage of schooling could assess fairly accurately what mark an essay of that nature would likely receive at the leaving certificate stage. Whether it was a good assessment of the work done is difficult to state because reliability does not ensure validity: or to put it another way, the markers may be consistently wrong".

III. AN ANALYSIS OF AN EXAMINATION PAPER *1

"A Moray House research worker who is studying ability in Chemistry analysed the Chemistry papers of Higher Science from 1949 to 1962. Adopting a modified version of Bloom's Taxonomy, he lists the four main objectives in the teaching of chemistry in schools as (a) information, (b) application of principles, (c) quantitative application of principles (calculation), and (d) scientific method. Each question was analysed and a percentage of the total number of marks in each question allocated to the various categories. His analysis showed that the allocation of marks was distributed as follows:

                  Category
        
                      a                         66%
        
                      b                         11%
        
                      c                         14%
        
                      d                          9%
                                          

1 Conference on Examinations, April 1968,Scottish Certificate of Education Examination Board, page 8.

3

While it call be admitted that his classification does not provide an accurate assessment of the papers, it does reveal that a preponderance of the questions demand Rom the pupils knowledge and information. Clearly, this is imposing a wrong emphasis upon the teaching of chemistry, and illustrates the failure of the examiners and curriculum-makers to work closely together."

IV.SIGNIFICANCE OF RESULTS OF PUBLIC EXAMINATIONS *1

R.K. Mathur while studying the effect of uncertainty in scoring upon the degree of misclassification of students showed that any classification attempted on the basis of scores whose reliability is less than 0.9, under certain assumptions, leads to misclassification of more than 23% of the students and so may be sufficiently mislead- ing. Reliability coefficient of 0.63 as obtained for School Final Mathematics Examination of West Bengal can lead to a misclassification of about 440% of students. Of these, 23% stand the chance of being placed in a group higher than they truly belong to. It follows, therefore, that excessive deference paid to the published examination results is hardly justified. Published list of 'first class' students contains many 'second class' students and possibly a few of even a lower rank. Conversely, many 'first class' students are there in do 'second class' list.

V. EXAMINING PROJECT WORK-PHYSICS *2

Pupils should develop the ability to:

(i) extract, interpret and classify information (use of libraries, journals, graphs, simple statistics, estimations of experimental uncertainties, etc.)

(ii) plan appropriate experiments; devise and construct and/or handle apparatus; tackle the practical problem of measuring some physical quality;

(iii) present a lucid, coherent report on a piece of their own individual experimental (Project) work.

1 R.K. Mathur, Effectiveness of Board and University Examinations ill India; Bulletin of the Institute of Public Administration, Patna University, 1962, p. 46.

2 Certificate of Sixth Year Studies, Scottish Certificate of Education Examination Board pp, 55-57.

4

The Examination

The examination will consist of two parts, each of which will be subject to assessment by the Board:

(i) A Report on a project or projects founded on experimental work conducted by the pupil during the session.

(ii) A written examination based on the syllabus.

Equal weight in assessment will be given to the two parts. The project Report will require to be submitted to the External Assessor appointed by the Board by 1st April, 1971. To help in the assessment of the Reports schools will be required to provide an estimated mark in respect of each candidate's project work. Candidates may be required to discuss their projects with a visiting assessor. Arrangements for each visit will be made with the schools concerned (see paragraph 20).

The presentation requirement is a pass on the Higher grade in Physics (I and II).

Units

SI Units must be used.

Project Work

Choosing a Project

The Board appreciates that this kind of work presents difficul- ties and it is hoped that teachers in the schools will take every opportunity of consulting the Board when such difficulties are en- countered. A teacher with a large number of pupils in his charge may have to confine the project work to one branch of Physics. The pupils can then be given common instruction on the theoretical principles involved before engaging on their projects. A certain amount of team work can be encouraged in pursuing the common aim, each member of the team being responsible for one part of the assignment. Each member of the team would, of course, be required to prepare a Report on the particular part of the work he bad done. However, the greatest possible latitude will be allowed to the teacher. For example, there is no insistence on a policy on one project per pupil; it may be more convenient for the pupil to be engaged in a small number of projects, Where the number of pupils is small some teachers may prefer to depart from any formal teaching and to direct the pupil to a course of reading from which projects may arise in various fields of Physics.

5

Experiments of historical importance could be considered and repeated using modern techniques which could give improved results.

A project must involve experimental work and must be something new for the student (it need not be a piece of original research). The best projects usually involved identification of some measurable quantity, controlling variables and obtaining numerical results which are then treated with sonic understanding of their errors. The weakest projects are those involving the building of sonic complicated device using a given set of instructions.

The Teacher's Role

It is the task of the teacher to advise against projects involving concepts or techniques likely to prove too difficult, expensive or time-consuming, and to give special help to students in difficulties. The amount of help should be limited to that which will suffice to keep the project going.

Experience has shown that the work of candidates benefits from an introduction to:

(a) the writing of a report (structure, use of graphs, making references, etc.) (see below) ;

(b) the planning of a project (reading, constructing or assembling, selection of apparatus) ;

(c) techniques of measurement and recording of data;

(d) treatment of errors (uncertainties), significant figures,

Notes on the Writing of Project Reports

Structure of Report

The structure of a report will vary with the length and subject matter of the report and the proposals given below are for general guidance only. These proposals are intended to illustrate an approach which would be acceptable, but for any particular report they would be subject to modification as required.

1. Cover and title.

2. Table of contents.

3. Lists of tables, graphs and illustrations.

4. Summary (one paragraph is usually sufficient, preferably written in the present tense).

6

5. Main body of report (in past tense, third person)

(a) introduction

(b) apparatus and experimental procedure;

(c) readings, measurements made.

6. Conclusion (s), interpretation of results.

7. Diagrams, tables and graphs.

8. Bibliography, references, acknowledgements.

It is to be noted that neither the chronological order of the work done on the projects nor the writing of the report need be the same as the order of presentation given above. The sequence of writing might well be:

tables, graphs; results; main body; diagrams; summary; title; contents; bibliography, etc.

Presentation of Report

1. The front cover should display the title of the report, the name of the author and the name of the school.

2. The table of contents should include page numbers.

3. Only one side of each sheet of paper should be used.

4. Diagrams, graphs and tables should be placed either all together at the end, or near the appropriate part of the text.

5. Diagrams, tables and graphs should be captioned and numbered. Appropriate references to them should be made throughout the report.

6. Pages should be numbered for case of cross-reference.

7. The pages should be fastened together in some way, pre- ferably along the left-hand edge and the use of a looseleaf folder is recommended.

8. Bibliography, references to books and periodicals should be given in standard form.

It is recommended that references be given at die end of the report and indicated by numbers in the text.

Further considerations

1. The first requirement of any report is that it should be easy to read and understand. This implies an accurate title, a logical structure, adequate cross- references and a

7

clear statement regarding the conclusion (s) reached. Particular attention should be paid to the presentation of diagrams, tables and graphs.

2. Long passages of theoretical material are not required unless the material is original in some way. Otherwise, in most cases, the appropriate reference to a standard text will suffice.

3. Descriptions of apparatus should include the manufacturer's name and the type number where applicable. Diagrams or photographs are preferable to lengthy written descriptions.

4. If a project is carried out jointly by two or more students, each individual report must state clearly what part the writer played in the endeavour and must also give the name (s) of his colleague (s).

5. All measurements should be shown-either in tabular or graphical form.

6. Estimations or errors (uncertainties) in measurements and the accuracy of any final result must be included. Error bars should be drawn when plotting graphs.

7. Details of calculation such as logarithms should be included. The formula or method and the result only should be stated.

VI. EXAMINING PROJECT WORK-CHEMISTRY *1

Objectives of Project Work

The aim of project work in chemistry is the development of:

(i) skill it) devising an appropriate scheme for studying a problem in Chemistry;

(ii) skill in handling, classifying, interpreting and presenting information (including graphs and quantitative results) ;

(iii) ability to apply previously acquired understanding to new situations and to show creative thought;

(iv) resourcefulness on the part of the pupil with a corresponding lessening of dependence on the class teacher.

1 Certificate of Sixth Year Studies, 1971, Scottish Certificate of Education Examination Board, pp. 17-18.

8

Selection of Subject for Project

A project should be something new for the student but need not be a piece of original research. It should normally involve experimental work but need not necessarily do so.

Local industries, universities and colleges may be willing to help in suggesting topics and such assistance should be used wherever possible.

The teacher should not at any time hesitate to advise a pupil to change the project if the original appears likely to be unproductive.

Where there is any doubt about the suitability of a proposed project the Board should be consulted.

Responsibility of Pupil and Teacher

The subject of the project should normally be arrived at as a result of discussion between pupil and teacher. The teacher should discourage a pupil from adopting a project involving concepts or techniques likely to prove too difficult, expensive or time-consuming. The teacher would be expected to take part in regular discussion with the pupil on the progress of project work. The first draft of the project report should be the sole responsibility of the pupil but the teacher may give advice regarding the production of a final draft.

Group projects may be adopted provided that each member is to play a specific part and submit a separate report.

The Report

The Report must be submitted to the External Assessor appointed by the Board by 1st May, 1971. It should include:

(a) a clear statement of the objectives of the study involved;

(b) an outline of the method of investigation and full details of the results obtained (including failures)