Description

Four computer games on the classification of Chinese herbal medicine according to the names and appearances (Game 1); the nature (Game 2); the flavour (Game 3); and the indications of the herbs (Game 4) were developed. 

The first three games share similar characteristics in terms of logic. For Game 1, students were challenged with a sequence of photographs of herbs in a random order and they have to choose the correct name of each herb. For Game 2 and Game 3, students need to indicate the nature and flavour of the herb in a similar way. For Game 4, clinical scenarios were presented in a step-by-step manner. Students are asked to decide what is wrong with the person and what sort of herbs should be prescribed for the person with these symptoms. After a particular herb is chosen, the person's conditiono might improve or get worse so students have to decide which herb should be prescribed for the subsequent situation.

A 3-minute time limit is set for the first three games. If students can tackle the challenges faster, more herbs would be shown. For Game 4, there is no time limit because the ,major thing is being rated is students's clinical decision. Students would be asked if they want to continue the game if there is no response for more than 2 minutes. 

Three marks will be given to all correct answers in the first attempt; two marks for the second correct attempt; and one mark for the third correct attempt. The correct answer will be shown if students fail to answer the question correctly in their third attempt. At the end of the game, the total score will be shown on the screen. A top ten highest score will also be listedn on the screen. 

Evaluation

Summative and formative evaluations had been carried out. These included system generated spontaneous data, learning outcome evaluation, as well as user feedback.

System-Generated Spontaneous Data

Data were collected in Semester 2, 2014. Out of 331 students, 228 had played one or more of these games. Two hundred and seventeen played Game 1, 143 for Game 2, 90 for Game 3; and 210 for Game 4. It was noted that more students from the 53055 class played the game compared to the 53455 cohort. The 53055 students also spent longer time on average on these games, 1798 seconds per person, compared to 1221 seconds for the 53455 cohort. There was a general trend for both groups to spend more time on the game on the first week and the week before examination and less in between. Most (86%) of the students played the games only on personal computers, six percent on mobile devices, and eight percent on both personal computer and mobile devices,

Learning Outcome Evaluation

Among the 331 students, about two-thirds of them had played the games (players) and the remaining one-third had never played the game (non-players). Their examination results were compared to see if playing the game improved their examination scores. For the players, their average scores on herbal medicine was 62±14% compared to 54±13% among the non-players. Specifically, for the 53055 cohort, players scored 66±15% and the non-players scored 58±15%. For the 53455 cohort, players scored 60±13% and the non-players scored 54±11%. Thus, it might be concluded that the game contributed to a small increment in the examination scores overall.

Analysis was further conducted to see which factors influenced the examination scores. It was noted that among all performances in the game, only time spent on the game impacted on the examination scores. The following diagram (Figure 1) shows the dispersion of examination scores and time spent on the game. It must be stressed that playing the game was not a guarantee for high scores, but there was a trend that the more the student played the less likelihood for him/her to obtain a low score in the examination. 

The scores obtained while playing the game was not directly relevant to the examination scores. Relationship between the highest scores obtained in the game and the examination scores is shown in the following diagram (Figure 2). There is no significant correlationship between students' game scores and their examination scores. 

Student Feedback

A telephone survey and focus group interviews were conducted to obtain student feedback. In the telephone survey, students were asked to respond to three groups of questions to see if they find the game to be useful in helping them learn, evaluate the appropriateness of the game contents, and the design of the game on a scale of 0-10. The mean score for each question is presented in Table 1.

In general, the games were perceived to be helpful in facilitating student learning (scores 5.9-6.9); contents were not too hard to comprehend (scores 4.0-5.9); the design of the game was found to be good (scores 6.7-7.8). Scores for each individual question could be found in Table 1. Specifically, 90 percent of students in the 53455 cohort and 98 percent in the 53055 cohort indicated that the games were helpful in helping them study Chinese herbal medicine. Game 4 was ranked the most interesting by over two-thirds of the students; over half of the students liked this game most.

During the telephone interview, several problems were identified. Several students complained about the difficulty in logging in the game; latency of response; and difficulty in playing on the tablet or mobile devices. Some students pointed out that they did not like repetition of the same question. However, other students commented that repetition fo the same question helped them memorize the related knowledge better. In respect of the level of difficulty on the questions, students had mixed responses. Some found the questions too difficulty while some too easy and some found them just on the right level.

Six groups of students were invited to attend a focus group interview. These included playes of high, medium and low usage and students who had high, medium and low scores in the examination. High and medium players found the games help them learn. Low or non-players said they did not have time to play the games or they were not interested in playing the games. The non-players pointed out they would rather do revision instead of learning by playing computer games. The low and non-players also said they had to study well before they embarked on the games. They perceived that they did not study well enough for the examination so they should not spend time on the games. On the contrary, high and medium achievers believed that the games helped them learn the course material. On the other hand, the low achievers said they managed their time poorly and complained why they had to study Chinese medicinal nursing.

Recommendations for Improvement

• Increase the number of questions so as to elongate students' interest.
• Introduce the difficulty gradient within each game to improve the goodness of fit between students' level of existing knowledge and the challenges provided by the games.
• Encourage students to use mobile devices to play the games.
• Develop new games to address other aspects of the subject SN3208, such as the topics on meridians and acupunture points.