How do I evaluate my eCourse?

1. eCourse evaluation

Monitoring of your eCourse continues long after its launch. You should evaluate the performance and effectiveness of your eCourse after some time. This will help you decide what action to take with your eCourse. Will you retain it as is; revise or update content or archive or delete it? (Please refer to the next section for more details about these three options.) 

So, what is the most effective way evaluate your eCourse? As a training intervention, your eCourse may be assessed using Kirkpatrick’s Four- Level Training Evaluation Model. These metrics were developed by Donald Kirkpatrick, of the University of Wisconsin in his 1959 book “Four-Level Training Evaluation Model.”  The video below provides a short introduction.

WATCH: The Four Levels Training Evaluation Model (2:37) 

by Hendri L. Tobing

Kirkpatrick identified four areas of assessment (Mindtools, 2016) which may be applied to your eCourse.

1. Reaction 

What to evaluate: User experience or how learners respond to the course topic, content, course navigation, moderator of facilitator, materials, etc. 

How to do evaluation: Feedback survey, interview or focus group discussion

How to use results: For improvement of eCourse content and form.

2. Learning

What to evaluate: Knowledge gained by learners from the eCourse.

How to do evaluation: Learning activities such as quizzes or games to ensure that learners achieved your learning objectives.

How to use results: For improvement of eCourse content. 

3. Behavior 

What to evaluate: How learners apply what they learned from your eCourse. 

How to do evaluation: Self assessment of learner; interview of supervisors and colleagues of learner.

How to use results: For improvement of eCourse content and/or for decision to retain, archive or delete eCourse (if found not to be helpful).

4. Results

What to evaluate: If eCourse is able to contribute to organizational requirements and goals. 

How to do evaluation: Use indicators of success identified by organization to gauge success of eCourse.

How to use results: To justify retention of the eCourse, its deletion or the development of a different eCourse.

Important things to consider when evaluating your eCourse

1. Feedback from learners

Just like in developing your eCourse, your learners are your primary consideration when deciding to retain, revise or discontinue an eCourse. ADB eLearn courses capture feedback from learners through an online survey (provide link to survey monkey here).  This allows you to ask learners what they think about the course in terms of user experience, topic, scope of discussions, quality of the materials used, areas of improvement, and other comments that would help you make the eCourse better. Results from this survey combine objective and open-ended questions. Thus, learners are able to rate your eCourse as well as include their comments and suggestions.

Using Kirkpatrick's 4 areas of assessment above, you may revise the online survey and send it back to the eLearning team via email with the message title: "Revised feedback survey for my eCourse".

Aside from the feedback survey, you can gather learner feedback through

    • Online group discussions among learners using online message boards.
    • Personal email from learner to the eCourse owner.
    • Interview with learners on the impact of the eCourse on their job performance.

2. Learner performance

You will regularly receive eCourse completion reports from the eLearning team with the following information about your learners:

    • Number of enrolled learners
    • Number of learners who have completed and not completed your eCourse
    • Activity/content completion
    • Quiz scores
    • eCourse completion
    • Submission of feedback survey
    • Generation of course certificate

What should you be looking out for?

    • Slow or rare completion of the eCourse
    • Difficulty of learners in passing the quiz
    • Difficulty of learners in getting past a certain section of the eCourse

If you encounter any of the above, you should do further probing to determine if you need to immediately revise your eCourse.

Below are more materials to guide you.

READ:  How To Measure eLearning Performance: The New Way To Build Evaluation Into eLearning

READ: 5 Elements of measuring engagement in training evaluation

READ: The streetlight effect: when your training evaluation Is guided by what’s easy to measure 

3. Review by a  Subject Matter Expert 

eCourse evaluation should also be performed by a Subject Matter Expert (SME) to ensure that content remains accurate, relevant, up-to-date and complete. You may ask your SME to fill in this form (same as the Peer Review form used while developing your eCourse). 

Your decision on what to do with your eCourse should be based on these three considerations (feedback from learners, learner performance and review by an SME). The next section of this toolkit discusses next steps you can take with your eCourse.

If you need further guidance on eCourse evaluation, please send an email to the eLearning team with the message title: "Inquiry about eCourse evaluation."