How do I evaluate my eCourse?
1. eCourse evaluation
Monitoring of your eCourse continues long after its launch. You should evaluate the performance and effectiveness of your eCourse after some time. This will help you decide what action to take with your eCourse. Will you retain it as is; revise or update content or archive or delete it? (Please refer to the next section for more details about these three options.)
So, what is the most effective way evaluate your eCourse? As a training intervention, your eCourse may be assessed using Kirkpatrick’s Four- Level Training Evaluation Model. These metrics were developed by Donald Kirkpatrick, of the University of Wisconsin in his 1959 book “Four-Level Training Evaluation Model.” The video below provides a short introduction.
WATCH: The Four Levels Training Evaluation Model (2:37)
Kirkpatrick identified four areas of assessment (Mindtools, 2016) which may be applied to your eCourse.
What to evaluate: User experience or how learners respond to the course topic, content, course navigation, moderator of facilitator, materials, etc.
How to do evaluation: Feedback survey, interview or focus group discussion
How to use results: For improvement of eCourse content and form.
What to evaluate: Knowledge gained by learners from the eCourse.
How to do evaluation: Learning activities such as quizzes or games to ensure that learners achieved your learning objectives.
How to use results: For improvement of eCourse content.
What to evaluate: How learners apply what they learned from your eCourse.
How to do evaluation: Self assessment of learner; interview of supervisors and colleagues of learner.
How to use results: For improvement of eCourse content and/or for decision to retain, archive or delete eCourse (if found not to be helpful).
What to evaluate: If eCourse is able to contribute to organizational requirements and goals.
How to do evaluation: Use indicators of success identified by organization to gauge success of eCourse.
How to use results: To justify retention of the eCourse, its deletion or the development of a different eCourse.
Important things to consider when evaluating your eCourse
1. Feedback from learners
Using Kirkpatrick's 4 areas of assessment above, you may revise the online survey and send it back to the eLearning team via email with the message title: "Revised feedback survey for my eCourse".
Aside from the feedback survey, you can gather learner feedback through
- Online group discussions among learners using online message boards.
- Personal email from learner to the eCourse owner.
- Interview with learners on the impact of the eCourse on their job performance.
2. Learner performance
You will regularly receive eCourse completion reports from the eLearning team with the following information about your learners:
- Number of enrolled learners
- Number of learners who have completed and not completed your eCourse
- Activity/content completion
- Quiz scores
- eCourse completion
- Submission of feedback survey
- Generation of course certificate
What should you be looking out for?
- Slow or rare completion of the eCourse
- Difficulty of learners in passing the quiz
- Difficulty of learners in getting past a certain section of the eCourse
If you encounter any of the above, you should do further probing to determine if you need to immediately revise your eCourse.
Below are more materials to guide you.
3. Review by a Subject Matter Expert
eCourse evaluation should also be performed by a Subject Matter Expert (SME) to ensure that content remains accurate, relevant, up-to-date and complete. You may ask your SME to fill in this form (same as the Peer Review form used while developing your eCourse).
Your decision on what to do with your eCourse should be based on these three considerations (feedback from learners, learner performance and review by an SME). The next section of this toolkit discusses next steps you can take with your eCourse.
If you need further guidance on eCourse evaluation, please send an email to the eLearning team with the message title: "Inquiry about eCourse evaluation."