Evaluation of the computer in education in-service course

The purpose of this study was to evaluate the effectiveness of the in-service course on Computer in Education(CIE) for teachers at  a Technical Teacher Training College. Specifically, the aspects of the in-service course investigated were the objectives, planning process, usefulness of the course, quality of teacher trainers, instructional methods and aids and mode of evaluation. A questionaire was designed to elicit information related to the various aspects of the in-service courses indicated above. The sample consisted of 45 teachers from one particular school in Kuala Lumpur. Descriptive and inferential statistics were used to analyse the data. The results of this study indicate that the perceptions of CIE teachers regarding the importance of in-service course objectives were generally favourable. 55 % teachers found the planning of the in-service course to be fairly adequate although a large proportion of them felt that prior knowledge or skill of the teachers be considered before implementation of the course curriculum. 55 % the teachers felt that they were not given enough time to learn the skills at their own pace. 50 % of the teachers felt that the practical skills were not delivered systematically. Thus a longer time frame for the course is needed. On the usage of audio visual aids, it was found that the trainers involved used LCD Projector, a laptop computer and screen during demonstrations and lectures. There was active participation by the teachers. The teachers felt site visits and seminars be held to enhance learning. On the aspect of evaluation, it was felt that a written assessment be held besides the individual assessment on practical work. The participants suggested that a follow-up study be done to evaluate their progress when they are back to their respective schools after the course. With regard to the usefulness of the in-service course 80% of the teachers expressed the opinion that it is useful and helped them cope with their professional tasks more successfully. A general opinion the participants felt that they be given incentives and certification after following the course for three months.
By: Ragupathy Rajah S. Poopala Rajah and Selva Ranee Subramaniam





INTRODUCTION




Malaysia, being a developing nation with a vision to achieve an industrialized status in 2020, has been striving towards rejuvenating existing education towards quality education. To realise Vision 2020 to attain a developed nation status, the Ministry of Education introduced the Smart School concept which is one of the flagships in the development of the Multimedia Super Corridor or MSC. As the first step, all schools will be equipped with at least one computer with 21 personal computers and other related devices. Thus the use of computers is surely scaffolded and becoming a reality in the Malaysian school system in pace with the effects of gobalisation. Educational development has been of utmost importance and catalysed the use of technology in education over two decades. The Ministry of Education has embarked in two phases in the implementation of computers in the school system. The first phase was learning about the computer. The second was   learning with the computer (Zoraini Wati, 1989). However, the question which remains, is the extent that the teachers have equipped themselves with the necessary skills and knowledge to ensure the effective implementation of the Computer in Education syllabus in schools. In implementing innovations, a certain amount of change from the normal practice has to be exercised. Implementation of innovation consists of changes from current  practices in order to achieve targeted learning outcomes. Many a times, innovation is interpreted differently by different teachers, policy makers and of course stakeholders. 

With the effects of globalization and rapid technological change, there is a critical need for in-service training courses for Computer in Education to be conducted regularly to keep teachers both novice and experienced teachers to keep abreast with the changes in the pedagogical content knowledge. Teaching is a complex activity, and question of the depth and the width of the components that need to be incorporated in the teaching and learning process besides emphasising the content can be a dilemma for practitioners. If routine and outdated teaching strategies and approaches are still being practiced, the products of the system would lack skills and competencies to face the challenges of this millennium.


PURPOSE

Many in-service courses are conducted by the Teachers Training Division Ministry of Education, to give teachers an exposure to the “new” technology integrated curriculum. Hence, a systematic study has to be made to continuously evaluate the effectiveness of the computer in education course from the participants’ point of view. This study is to investigate and explore the perceptions of the Computer in Education in-service course provided for teachers. Specifically, the study aims to answer the following research questions:

To what extent was the planning of the in-service course effective?
To what  extent were the teachers able to understand the objectives of the course and were the objectives attained at the end of the course?
3.         To what extent was the course content useful to the teachers?
4.         Do the trainers have the relevent expertise to conduct the in-service course?
5.         What were the instructional methods and teaching aids used in the course and how effective and adequate were these methods and aids?
6.         What were the assessment techniques used to assess the teachers during the course and how effective were they?





METHODS

The subjects of this study comprised selected teachers from a urban school in the Federal Territory of Kuala Lumpur. Forty five teachers from this school were selected to undergo the in-service course on Computer in Education for two weeks. The selection of the teachers was done by the State Education Department. The teachers, were graduates teach science and social science in their school. It is first time formal exposure in the use of the computer in teaching for all the participants. The instrument used by the researchers in this study is a set of questionnaire. The instrument was designed to obtain specific information from the teachers who have attended the course in a local Teachers’ Training College in the Federal Territory of Kuala Lumpur. Basically the questionnaire (Teachers In-service Course or TISC) was intended to collect the following data

(1)        Background information of the teachers.

(2)        Perceptions of the teachers towards the aspects of the in-service course and recommendations to improve the in-service course.

Section 1 of the questionnaire consists of questions relating to race, sex, age, highest academic qualification, highest professional qualification, number of years of teaching experience, number of years teaching computer and the number of in-service courses on computer attended to date (items 1- 8). The perceptions of the respondents towards the various aspects of the in-service course was obtained from the questionnaire which was divided into six sections, each of which contained questions relating to the following aspects Section 2 : Perceptions of the respondents towards the objectives of the course (items 9-22). Section 3 : The planning process of the in-service course (items 23-28). Section 4 : The usefulness of the course (items. 29-33). Section 5 : The quality of the teacher trainers and suggestions to improve their performance (items 34-45). Section 6 : The instructional methods and aids used by the teacher trainers and suggestions how they could be improved (items 46-55). Section 7 :The evaluation of the in-service course and suggestions to improve methods of evaluation (items 56-59).

All of the questions in Sections 1-7 required the respondents to mark ( V, ) in the spaces against each of the statement or item which best reflected their perception. For example in Section 2 the response follows a 4 - point Likert Scale the number '1' '2' ,'3 ' ,'4' which indicate 'not important at all ", 'fairly important', 'important' and 'very important' responses respectively. Against the items from Section 3 to 7, the number ' 1 " ,'2' ,'3' ,'4' ,'S' indicate 'strongly disagree',' disagree', 'uncertain' ,'agree", strongly agree' respectively. The questionnaire was written in one language that is English as the teachers were more comfortable with English. The questionnaire was administered to the teachers a year after they attended the course.The researchers were facilitators in the in-service course conducted for the teachers. The teachers were not required to write their names on the questionnaire. This was to ensure anonymity so that the teachers would give their honest and frank answers without worrying of any undesirable consequences. The teachers took 30 minutes to complete the instrument.

RESULTS AND FINDINGS
Students responses to each item and its mean score in the six subscales are shown in Tables 1 to 5. From Table 1the mean score of the item in the subscale Aims (Importance of the aims of the course) range from 3.222 to 3.6000 with an overall item mean 3.498 (items 9-15). This shows that all the respondents place importance on the objectives of the course. Whilst for the subscale Aims (How successful were the aims attained) showed an overall mean score of 2.459 indicating the attainment of the objectives of the course was fairly successful only.

Table 1:    Frequency and Mean Score of teachers' Responses to its Items Subscale AIMS
                    (Importance)
Item
Very
Important
Fairly
Not Important
Mean

Important

Important
At All
Score

9
10
11
12
13
14
15
31 (68.9)          14 (31.1)
30 (66.7) 15 (33.3)
27 (60.0) 18 (40.0)
28 (62.2) 17 (37.8)
25 (55.6) 20 (44.4)
18 (40.0) 27 (60.0)
17 (37.8) 23 (51.1)
5(11.1)
3.622
3.600
3.533
3.556
3.556
3.400
3.222
3.498
MEANSCORE OF SUBSCALE

AIMS How successfully were the aims of the CIE


Very                 Successful                Fairly              Not successful
Successful                                       successful                  at all

16
5(11.1)            26 (57.8)            14 (31.1)
2.800
17
1 (2.2)                   22 (48.9)            22 (48.9)
2.533
18
19 (42.2)                19 (42.2)             7 (15.6)

2.267
19
1 (2.2)                    17 (37.8)            27 (60.0)
2.422
20
1 (40.0)                                           25 (55.6)                     2(4.4)
2.356
21
1 (2.2)                    17 (37.8)            26 (57.8)                      1 (2.2)
2.378

MEAN SCORE OF SUBSCALE       2.459


Table 2:        Frequency and Mean Score of Teachers' Responses to its Items Subscale PLANNING    
                        PROCESS
Item
Strongl
Agree
Agree
Uncertain
Disagree               Strongly
Disagree
Mean
Score
22
3
(6.7)
20
(44.4)
11
(24.4)
10 (22.2)
1 (2.2)
3.311
23
1
(2.2)
11
(24.4)
5
(11.1)
20 (44.4)
8(17.8)
2.489
4
1
(2.2)
14
(31.1)
16
(35.6)
12 (26.7)
2(4.4)
3.000







MEANSCORE OF SUBSCALE
2.933

From Table 2 the mean score of the subscale 'Planning Process' range from 2.489 to 3.311 with a mean score of 2.933. All of the items except for item 23 have a mean score of less than 3.0 indicating the course objectives did not take into account the previous knowledge of the participating teachers. As for items 22 and 24 the  mean score of 3.311 and 3.000 respectively indicates the teachers were unsure whether or not the course took into account the teachers needs and their interest in computer.

Table 3:        Frequency and Mean Score of Teachers' Responses to its Items Subscale USEFULNESS
                        OF THE COURSE
Item
StronglAgree
Agree
Uncertain
Disagree
Strongly
Disagree
Mean
Score
       25
2(4.4)
14 (31.1)
9 (20.0)
15 (33.3)
5(11.1)
2.822
26
1(2.2)
19 (42.2)
14 (31.1)
10 (22.2)
1(2.2)
3.178
27
3(6.7)
31 (68.9)
5 (11.1)
5(11.1)
1 (2.2)
3.644
28
3(6.7)
21 (46.7)
10 (22.2)
10 (22.2)
1 (2.2)
3.311




MEAN SCORE OF SUBSCALE
3.239

From Table 1.3, the item mean score of the subscale 'Usefulness of the Course' range from 2.822 to 3.644 with a mean score of less than 3.0 indicating many of the participants felt the objectives of the course did not follow previous knowledge and the skills of the teachers. The mean score for item 27 (3.644) indicates that more than half the teachers felt that the time-table for the course is set to the convenience of the teachers. Overall mean score for the subscale (3.239) indicates more than half the participants felt the course was useful to the teachers. From Table 4, the item mean score of the subscale 'Quality of Teacher Trainers' ranged from 2.578 to 3.511 with an overall mean score for the subscale as 3.063. For item 32, the mean score of less than 3.0 indicating that 60% of the teachers felt the lectures were not conducted well and systematically. Item 33 with a mean score of 3.511 indicates that at least half the teachers felt the relationship between the participants and the trainers were close.

Table 4:        Frequency and Mean Score of Teachers' Responses to its Items Subscale QUALITY OF
                        TEACHER TRAINER


Frequency and Mean Score of teachers' Responses to its Items


Subscale
QUALITY OF TEACHER TRAINER

Item
Strongly
Agree
Uncertain
Disagree                 Strongly
Mean

Agree


Disagree
Score
29
4 (8.9)
17 (37.8)
12 (26.7)
        9(20.0)       3(6.7)
3.156
30
2 (4.4)
16 (35:6)
16 (35.6)
11 (24.4)
3.200
31
2 (4.4)
16 (35.6)
13 (28.9)
14 (31.1)
3.044
32
1 (2.2)
11 (24.4)
19 (42.2)
14 (31.1)
2.889
33
4 (8.9)
24 (53.3)
10 (22.3)
7(15.5)
3.511
34
3 (6.7)
10 (22.2)
22 (48.9)
        8(17.8)    2(4.4)
3.067
>> 
1 (2.2)
23 (51.1)
7(15.6)
          14 (31.1)
2.578




MEAN SCORE OF SUBSCALE
3.063

From Table 5, the item mean score for the subscale 'Instructional Method and Aids' range from 1.978 to 3.800 with an overall mean score for the subscale as 3.074. The item 37 is reversely scored and this indicates that slightly more than half the teachers agree that they have not been given enough opportunities in the selection of the activities for the course. About 38 participants are in favour with small group discussion. There is greater interactivity among.teachers in the form of small group activities organised during the course.

Table 5:        Frequency and Mean Score of teachers' Responses to its Items Subscale
                        INSTRUCTIONAL METHOD AND AIDS

Item
Strongly
Agree
Agree
Uncertain
Disagree                  Strongly
                             Disagree
Mean
Score
36

10 (22.2)
14 (31.1)
11 (24.4)             10 (22.2)
2.489
37
7(15.6)
24 (53.3)
6(13.3)
                        8(17.8)
1.978
38
1 (2.2)
37 (82.2)
3(6.7)
 4(8.9)
3.733
39
2(4.4)
36 (80.0)
5 (11.1)
 2(4.4)
3.800
40
1 (2.2)
30 (66.7)
6(13.3)
                        8(17.8)
3.489
41
3(6.7)
33 (73.3)
4(8.9)
                        5(11.1)
3.711
42

16 (35.6)
19 (42.2)
9(20.0)           1 (2.2)
3.089
43

13 (28.9)
7(15.6)
20 (44.4)              5 (11.1)
2.600




MEAN SCORE OF SUBSCALE
3.074

From Table 6, the mean score of the subscale 'Evaluation' range from 2.444 to 3.955 with an overall mean score of 2.993. Many of the participants were evaluated based on the individual participation in class sessions. No written test was used to evaluate the participants. Their honest opinions and comments concerning the course was sought in the form of a questionnaire given at the end of the course.

Table 6:        Frequency and Mean Score of teachers' Responses to its Items Subscale EVALUATION


Frequency and Mean Score of Teachers' Responses to its Items



Subscale
EVALUATION

Item
Strongly
Agree
Uncertain
Disagree                 Strongly
Mean

Agree


    Disagree
Score
44
2 (4.4)
5 (11.1)
10 (22.2)
28 (62.3)
2.444
45

11 (24.4)
13 (28.9)
15 (33.3)                  6(13.4)
2.578
46
7(15.6)
33 (73.3)
3 (6.7)
2(4.4)
3.956
47
3(6.7)
33 (73.3)
4 (8.9)
5(11.1)
3.711




MEAN SCORE OF SUBSCALE
2.993
                   
DISCUSSION & CONCLUSION
It may be convenient to discuss the major findings under the following headings: objectives of in-service courses, planning process, usefulness of courses, quality of teacher trainers, instructional methods and aids, and mode of evaluation.
Objectives of In-Service Courses
It is believed that an educational programme is successful only to the extent that the objectives of the programme are understood, accepted and perceived as important by the teacher trainers and the teachers concerned. Thus it is essential that the teachers are convinced of the importance of the objectives of in-service courses because their perceptions are likely to influence the extent to which these goals are attained. The results of this study indicate that the perceptions of teachers regarding the importance of in-service course objectives were very favourable. The seven objectives of the CIE in-service course in MPT were ranked in the following order based on the mean score.

(1)  To understand the objectives of the Computer in Education Program.
(2)  To update the knowledge on computer.
(3)  To discuss the various methods and process in learning and teaching computer.
(4)  To improve on the practical skills.
(5)  To discuss the use of audio-visual aids in the teaching of Computer in Education.
(6)  To discuss the right method of assessment.
(7)  To provide opportunities for the participants to exchange views in relation to  their own classroom experience in the school.

Furthermore, the findings of the study indicate that all of these objectives were perceived to have been attained at least 'fairly successful'.

Planning Process
Another important aspect studied is the planning of in-service courses. Careful planning is essential for the achievement of objectives of in-service courses. A summary of the findings regarding selected aspects of the planning process of in-service course is indicated below. The majority of the teachers agreed that generally their needs were identified. This is because the syllabus was designed to teach basic to moderate level computer knowledge and skills so as to cater for the majority of the teachers. However, a rather large proportion of the teachers indicated that the objectives of the course did not follow the previous knowledge and skills that they obtained. The teachers selected for the course were identified by the Headmistress of the school. All of the 45 teachers were from one school only and there was not much choice in the selection made. However, it is suggested that the educational authorities concerned consider the possibility of handing out a questionnaire to all the selected participants prior to them attending the course and the data analysed and discussed by the planners of the in­ service course. In this way the trainers will be well prepared and thus participants will be more comfortable following the course. There will definitely be a greater percentage of success at the end of the course for the organisers and the participating teachers.

A fair number of teachers, about 45% - 50% agreed that the interest of the teachers in the usage of the computer software was considered. This number of teachers had previous knowledge and wanted to enhance their skills. Those who disagreed on the statement felt that more time be allocated for the teachers to learn the skills at their own pace. Based on the findings of the survey, it is recommended that CIE in-service courses be conducted during the long vacation of seven weeks. Alternatively, in­service training should be provided in the way of a one-year full time course so that both lecturers and participants are able to discuss adequately problems and their solutions.

Usefulness of the Course
A majority of the teachers were of the opinion that the CIE in-service course was useful to them. Participation in the course helped them to cope with their professional tasks more successfully and they were able to use some of the results of their training in classroom teaching. The course also received the support of school administrators. However, a majority of the teachers felt that they ought to be given certification in recognition of their participation. Such recognition should be considered for purposes of promotion. A considerable proportion, about 45% of the teachers indicated that they were acquainted with new teaching techniques in the course. Many teachers felt the teaching techniques used will vary according to the availability of the necessary equipment and materials available in their school. Perhaps, it is necessary to provide teachers who attend in-service courses with some incentives. These incentives may take the form of enhanced promotional prospects or salary increments. These measures are likely to motivate teachers to greater efforts in in-service courses. To ensure the usefulness of the in-service courses, it is necessary to inform heads of schools with regard to programme objectives and expectations. This will minimise the frustrations that might be felt by some teachers who after the course find it difficult to introduce changes because of the unwillingness of administrators and staff to move away from the status quo.

Quality of Teacher Trainers
The effectiveness of a programme depends largely on the quality and capability of the teacher trainers who implement it. In view of this, the study examined whether teacher trainers concerned had the necessary expertise and were familiar with course aims and to transfer skills and knowledge to the participants. A fair number of the teachers, about 46% agreed that the teacher trainers had the necessary expertise in content and were familiar with course aims. The rapport between teacher trainers and trainees was close. The teachers felt that the lectures were not boring. However 50% of the teachers felt that the practical skills could be delivered more systematically. This could be due to the fact that the teacher trainers had just enough time to finish the course syllabus at the stipulated time.

Instructional Methods and Aids
The majority of the teachers felt that they had not been given the opportunity to select the kind of activities to be undertaken. Some small group discussions were organised by the teacher trainers. In addition, audio-visual aids were used effectively by some of the teacher trainers and there was active participation by the teachers. The teachers felt that the activities done did not cater to the interest of the individual teacher. About 40% of the teachers felt that on the whole the lessons presented were clear and effective. Due to the time constraint set for the course there was insufficient time for any discussion. Apart from classroom instructions it was felt that individualised guidance be given for the slow learners such that eventually they could follow the lessons. Such opportunities provided will enable the participants to clarify concepts and raise any issues that need further collaboration and discussion. It was also suggested that a team among the participants be formed to prepare instructonal materials and modules relating to the Computer In Education curriculum. In this way the learning and teaching process could be carried out effectively. The above findings suggest that teacher trainers tend to place greater emphasis on the traditional teaching approach - the chalk and talk method. In view of this, there appears to be a need for the teacher trainers to play down the traditional method of teaching and adopt a wider variety of appropriate teaching strategies so as to bring about more effective learning and understanding among CIE teachers. Activities such as seminars, panel discussions, role play, project work, excursion and visits should be encouraged so as to provide opportunities for active learning. According to Walter and Nolan (1950, in Sukumaran, 1984) stated that no one method of teaching could be used equally well by all teacher trainers.

Evaluation
Evaluation of teachers' performance is useful to the teacher as well as the teacher trainers. The type of evaluation techniques used will also determine the extent to which such an exercise will enhance learning and teaching. A majority of the teachers expressed the opinion that a written assessment be carried out to evaluate performance. As a form of evaluation, they were sometimes observed on an individual basis in relation to class participation. There was no follow-­up study to evaluate their progress in school from time to time. There seems to be a consensus of opinion among the teachers that an adequate follow-up study should be built into the course programme to evaluate the performance of teachers. Members of the school inspectorate and key personnel need to visit the participants in their respective schools so as to evaluate their progress from time to time.
REFERENCES


Sukumaran, S. (1984). Commerce Inservice Course iiT Peninsula Malaysia Perceptions of some Participants. Unpublished master's thesis, University Malaya.
Vaughan, E. D., Wang, M.C. & Dytman, J.A. (1987). " Implementing An Innovative Program: Staff Development and Teacher Classroom Performance." Journal of Teacher Education, 38(6),
pp. 40-47.
Walter & Nolan (1950) as cited in Sukumaran (1984). Commerce Inservice Course in Peninsula Malaysia Perceptions of some Participants. Unpublished master's thesis, Universiti Malaya.
Zoraini Wati Abas, (1989a). Moving towards the Computer Age : The Malaysian experience. In K.S. Wong, Zaiton Osman, Bing Selamat, Ainun Md. Hashim (Ed). 18 th Annual Conference of the International Association of School Librarianship Conference Proceedings (pp. 346 - 369). MI: ASL.</div>














.