Instruction, evaluation and certification in web based instr
abstract:
student evaluation is the key to the success of the learning and certification process. web based instruction does not have the flexibility of the classroom based instruction in regard to the evaluation process and presents certain difficulties. student evaluation is a multi-step process, with each stage having its own objectives and strategies. one of the major limitations of the on-line testing is that it restricts the evaluation to objective questions due its convenience, in spite of its limitations. the paper suggests the possible approaches to evaluation at different stages and for different purposes, in the context of web based education.
the paper considers student evaluation for learning, for self-testing, for the purposes of formative evaluation, for continuous evaluation and for the purpose of final summative evaluation needed for certification. the paper suggests an evaluation model for web based education and also suggests some alternative methods of incorporating the problem solving and short or long answer problems in the web based evaluation.
key words: web based testing, web based education, testing methods, instructional evaluation, testing methods, web based evaluation.
introduction
every education system assures the users of the products of its system, i.e. the employers, about the proficiency or capabilities acquired by its students as a result of going through its educational programmes. for this it uses some form of testing and evaluation leading to certification. often, the certification awarded by the educational institutions in the form of degrees, diplomas and certificates is accepted by the society for various purposes and is taken as quality certification. the conventional university system assures the quality of its product by controlling the inputs, resources, process of instruction and the student evaluation. for this, it uses the teachers as a key resource of the system. the teachers make use of the knowledge of the domain, and that of the students, to guide the students through complex instructional processes and then evaluate them. three important processes closely linked to each other have influenced the quality of education.
they are
a) integration of instructional objectives [1]and instructional strategies[2] to ensure congruence between these elements to ensure the development of desired competencies to the desired degree of proficiency;
b) spreading out the process of evaluation [3] through various activities over the whole period of learning to improve the reliability of the evaluation;
c) focusing on the evaluation of learning process rather than simple competencies.
to improve the reliability of the assessment of the students, teachers constantly evaluate the students through close observation in the classroom, analysis of the questions and responses from the students, and conduct of a variety of formal evaluationor test procedures like written, oral and practical examinations. education systems have also designed a variety of test items such as different types of objective items, short answer questions, descriptive questions, numerical problems, skill demonstration etc. each of the evaluation procedures and each type of the test items are designed to perform a specific function.
most educational systems consider that the objectives, content, instructional strategies, instructional interactions and student testing must form an integral unit in any instructional process, though in practice the amount of integration varies considerably from institution to institution and teacher to teacher. this is reflected in the fact that almost all the instructional design models [3,5] suggest that, the student evaluation plan and test items be best developed based on the objectives and the content. for this reason, the test items are written immediately after defining the objectives and writing the content out lines. so also the content, instructional strategies, and the instructional interactions are designed, based on the objectives and meeting the evaluation requirements. thus the educational system ensured congruence between the objectives, content, instructional strategies, instructional interactions and testing. the integration of instruction and evaluation is a result of the influence of the behaviorist philosophy and the competency based instruction on the instructional system.
instruction & evaluation in web based education
computer based instruction (cbi) [6] promoted the concept of interaction in the instruction by emulating the good teaching practices. the web based instruction inherited this concept from cbi. the interaction is basically implemented by eliciting an answer from the student, evaluate the answer and provide a feed back to the student. the interactivity is one of the important desirable features of the web based education (wbe) system. this feature influences the delivery options and also influences authoring and evaluation processes.
since, increasing number of persons are opting to undertake education through web based distance education programmes, it is useful for the web based education system to adopt some of the time tested and best practices of the conventional education system in regard to instruction and evaluation. though the web technology holds a lot of promise as an educational tool, the current generation of tools are designed for browsing and information retrieval, and not for supporting active learning systems. therefore, they currently lack a number of features that an advanced educational environment requires. since web based education is going to stay and since it is fulfilling an important need of the society, it is necessary to strengthen the wbe to a reasonably good level, where it matches the conventional education. for this, wbe must adopt the good features of the time-tested methods.
evaluation model
the educational systems evaluate students for different purposes such as certification, to encourage a student to learn, to find conceptual difficulties and rectify them, and so on. one of the important characteristics of an advanced learning environment is its ability to evaluate the knowledge acquisition and retention rate of its user, and to adapt to student needs. therefore, it is necessary for the wbe to have an adequate evaluation system [4] to take care of different needs. the subsequent paragraphs of this section examine different types of tests and their characteristics.
normally, the end of term test is conducted either at the end of the semester or the end of the year for certification purposes. these end-of- term tests are comprehensive in nature in the sense that they judge the total curriculum. if it is normative test, it tries to find the amount of subject knowledge the person possesses and the depth of understanding of the person. it also tests the ability of the student to integrate different aspects of the subject matter. in the case of criterion referenced test, it tries to find the achievement of the student in terms of competencies achieved or objectives attained. these tests do not normally perform the function of testing integrated knowledge unless it is specified as either as an objective or as a competency. most often, the end of the term examination is not meaningful in these contexts. the end-of-term test tends to be summative in nature and thus do not provide any feedback to the students. the tests tend to be representative sample with good ability to test the extent of knowledge a person possesses. the table of specifications normally governs these tests. thus the evaluation using either norm referenced testing or criteria referenced testing will test the achievement at the end of the term. it may be possible to operate this test on self-improvement basis, which means that the test can be repeated. a repetitive test requires large number of questions in the bank and a reasonable assurance that the items are not known to the students after a few try outs and test cannot be aborted without being marked.
a safe approach to conduct this test on-line is to conduct it under supervision at certain centres. it may be possible at a future date to conduct this test on-line using a wide-angle camera and audio facility integrated into the system. these facilities together with finger print analysis system could be used to authenticate the user by using the data collected over multiple sessions.
however for two entirely different reasons, a number of tests[6] are conducted spread out through out the year instead of depending entirely on the end-of-term evaluation. the first purpose is the reliability of the test results and consistency. multiple tests will take care of any temporary aberrations that might arise for any reason. the second reason is the pressure exerted by the tests on the students to study regularly. these periodic tests, also called unit tests cover limited areas and thus tend to be more exhaustive in testing the student and may use table of specifications. these tests fall under the name continuous evaluation and are both formative and summative in nature. the test evaluation normally provides exhaustive feedback to the student. this means that the teacher goes through the answer script. it is possible to provide standard answers to the student along with the standard feedback provided by the system. it may be possible to operate this test on self-improvement basis, with all the implications mentioned earlier. the unit tests are timed and they provide the opportunity to redo a problem or an item and records the number of revisions made to the item.
self-evaluation tests are periodic tests that are meant for evaluating oneself. these tests can be taken at any time but generally cover the subject area of a unit or a module. these also make use of the table of specifications and inherit most properties of the unit tests mentioned earlier. the differences are in regard to the extent of feedback provided to the student. while the unit tests provide exhaustive feedback, the self-evaluation tests provide feedback related to the correctness of the answer. since the self-evaluation tests also make use of the question bank, the number of questions in the bank has to be large.
assignments are questions that are to be answered by the student and submitted to the teacher. normally the teacher without any reference to the table of specifications prepares the assignments. the teacher may draw the questions from the bank, if chooses to do so. if the questions are not drawn from the bank, the teacher has to correct the assignments and provide the feedback. if the questions are drawn from the bank, the standard answers with some feedback can be sent to the student. normally the assignment marks or scores are not included in the student records as the assignments are learning tools. however, the self-assessment tests and assignments are not assessed for direct contribution to the overall score of the student. instead, the assignment submissions, and the promptness of assignment submissions etc. could make some contribution to the overall score.
besides these, the exercises provide sufficient practice to master the skills and to unearth misunderstandings, wrong concepts etc. these exercises need to be supported by good diagnostic ability so that the students can be guided in the learning process. quizzes provide quick assessment of the achievement of the students and are supported by minimal feed back. the exercises and quizzes contribute to the overall score through participation.
the students contribute to the group learning process by participating in various activities. in the case of wbe, the students participate through chat room discussions and newsgroup participation and intethe tests on the students to study regularly. these periodic tests, also called unit tests cover limited areas and thus tend to be more exhaustive in testing the student and may use table of specifications. these tests fall under the name continuous evaluation and are both formative and summative in nature. the test evaluation normally provides exhaustive feedback to the student. this means that the teacher goes through the answer script. it is possible to provide standard answers to the student along with the standard feedback provided by the system. it may be possible to operate this test on self-improvement basis, with all the implications mentioned earlier. the unit tests are timed and they provide the opportunity to redo a problem or an item and records the number of revisions made to the item.
self-evaluation tests are periodic tests that are meant for evaluating oneself. these tests can be taken at any time but generally cover the subject area of a unit or a module. these also make use of the table of specifications and inherit most properties of the unit tests mentioned earlier. the differences are in regard to the extent of feedback provided to the student. while the unit tests provide exhaustive feedback, the self-evaluation tests provide feedback related to the correctness of the answer. since the self-evaluation tests also make use of the question bank, the number of questions in the bank has to be large.
assignments are questions that are to be answered by the student and submitted to the teacher. normally the teacher without any reference to the table of specifications prepares the assignments. the teacher may draw the questions from the bank, if chooses to do so. if the questions are not drawn from the bank, the teacher has to correct the assignments and provide the feedback. if the questions are drawn from the bank, the standard answers with some feedback can be sent to the student. normally the assignment marks or scores are not included in the student records as the assignments are learning tools. however, the self-assessment tests and assignments are not assessed for direct contribution to the overall score of the student. instead, the assignment submissions, and the promptness of assignment submissions etc. could make some contribution to the overall score.
besides these, the exercises provide sufficient practice to master the skills and to unearth misunderstandings, wrong concepts etc. these exercises need to be supported by good diagnostic ability so that the students can be guided in the learning process. quizzes provide quick assessment of the achievement of the students and are supported by minimal feed back. the exercises and quizzes contribute to the overall score through participation.
the students contribute to the group learning process by participating in various activities. in the case of wbe, the students participate through chat room discussions and newsgroup participation and interaction with the teacher. the quality and quantity of participation can be evaluated and can be considered for the overall performance of the student.
question bank and types of questions
a good question bank with adequate number of questions forms the basis for the evaluation system. to build a substantial question bank, one may choose a collaborative approach. in this model, an independent bank is developed and maintained. every teacher and every institution can make use of this bank in their own environments and they can also contribute questions to the bank. using this collaborative approach a fairly large bank can be built in a short time.
a second dimension to building the question bank is making use of templates for questions [4]. using each template a large number of questions can be generated. thus the question bank will store a number of templates rather than the questions and the actual questions are generated at run time.
most web based testing efforts will end up using multiple choice, fill in the blanks and true false types of questions as they are more easily prepared, implemented, and evaluated. web based education systems must provide necessary flexibility in instruction and evaluation and use a variety of instructional and test strategies if they have to become serious alternative to conventional educational system. in such a context, it is useful to accommodate as much variety of questions as possible in the bank since each type of question is best suited for a given purpose. the types of questions that are to be accommodated are as under:
multiple choice
true or false
fill in the blanks
single word - short answer
multiple words - short answer
matching
pointing/selecting
dragging
numerical answer
procedural steps in solving a problem / procedures
long descriptive answer/essay
comprehension
case study / critique