Measuring The Success of E-Learning Initiatives

by Carmine Porco — “The pressure is on e-learning initiatives to prove their salt before any investment is made”, says Julie Kaufman, Research Manager, IDC Canada. “Enterprises are demanding a business case before the purchase of a LMS (Learning Management System) or any other software or service behind e- learning initiatives.
While proof of return on investment (ROI) is a necessity, organizations are also requiring proof that e-learning delivers on its promise. “As well, executives are now demanding proof that e-learning will also improve employee learning, retention and satisfaction.”
 
Many corporations substantiate their e-learning initiatives after the fact by eliminating related jobs or reducing travel expenditures for training. Although these are viable cost savings, they should not be the sole motivation for an e-learning initiative. E-learning must demonstrate that employees are learning more efficiently and retaining more of the curriculum compared to in-class training, thereby improving employee productivity.
 
“While you may use a lot of numbers to convince executives, deployment of e-learning must be focused on people,” says a recent study out of Brandon-Hall research, which emphasizes the ‘learn’ in e-learning.  “E-Learning is not about technology, tools, content, etc. It is about training people and the resulting effect on your organization.”
 
Simply throwing courses online is not a viable model for success. Before one can measure the benefits of an e-learning initiative, one must understand what it intends to measure. Planning the e-learning initiative is of paramount importance if an organization is going to reap the rewards.
 

Planning questions that require answers:

  • What are your specific e-learning goals? (e.g. train more in less time, increase sales, etc.)
  • How will you align e-learning to the needs of the business?
  • What groups and how many employees require or desire access to the courses?
  • What kinds of e-learning courses are desired?
  • Do employees want or have an appetite to learn online?
  • As e-learning will not supplant all training, how will the two relate?
  • What are the critical success factors?
  • How will results be measured?
  • Who will responsible for the rollout and subsequent maintenance?
  • Which executive will be the champion of the e-learning initiative?
 
To answer these questions and to ensure success, a number of planning activities are required:
  • A  needs assessment to determine if the initiative is a “must have” or a “nice to have”
  • A requirements analysis to document and understand internal stakeholder needs
  • User research to determine to what extent employees welcome and will use e-learning
  • A feasibility study to determine the potential barriers to success and whether the initiative can be implemented in a reasonable time, at a reasonable cost
  • A strategic plan that documents specific objectives and goals and how they align with organization goals
  • A communication plan on how the organization will market the initiative to ensure use
  • A business case proving the cost benefit analysis and ROI
  • A measurement plan detailing the critical success factors and performance measures (e.g. use, ROI, etc.)
 
“For these improvements to occur, an e-learning strategy needs to be based on an analysis of your business requirements followed by sound planning and implementation,” says Brandon-Hall. “Good e-learning can reap great benefits. Bad e-learning can’t.”
 
“Without the proper due diligence early in the process, many e-learning initiatives are doomed to fail,” adds Kaufman. “We have seen many enterprises putting the cart before the horse and get trampled”.
 
“Also, implementing e-learning can require a substantial effort for your company. Courseware may need to be developed, technical platforms may need to be purchased or upgraded, key people need to become allies and supporters of the cause,” says Brandon-Hall.
 
Once planning is complete and e-learning is ‘live’, there are many metrics that can be utilized and would have been defined in the measurement plan. Different metrics are applicable to different types of e-learning which includes:
  • Video on demand
  • Virtual live classroom
  • Multicasting
  • Interactive simulation
  • Content authoring & management
  • Skills and knowledge assessment
 
Examples of e-learning metrics that could be utilized in proving your business case include:
  • Enabling product to market faster
  • Reassigned or reduced head count
  • Travel and accommodation cost reduction
  • Increased revenue
  • Reduce training time and costs
  • Reduced printing and distribution costs
  • Improved employee retention and satisfaction
  • Enhanced customer satisfaction
  • Easier tracking and measurement
  • Improved organizational performance
 
To enhance existing training programs and to find cost savings Compaq introduced online training in mid-1995. Compaq’s Learning Utility is a single interface and search and retrieval tool to access all training courses and information resources at the global computer company. The Utility supports both novice and expert users and provides access to proprietary, vendor and local content and is divided into three categories: Licensed Courseware (from market vendors such as SmartForce and business specific companies such as Microsoft, Cisco, etc.), Compaq Specific Courseware (Compaq product specific courses, etc.) and Documentation (downloads of hardware/software manuals, sales advisory guides, etc.).
 
In total Compaq’s Learning Utility offers over 3000 courses and over 5000 documents. In 2000, total e-learning downloads (and courses taken) equaled 342,412 with the average download cost reduced from $9.59 in 1999 to $7.78 in 2000. Excluding travel costs that would have been saved by employees having to previously travel to attend training, the Utility achieved total annual savings surpassing US$13 million in 2000. (Source: Jim Melanson, Director, Compaq, “Workforce Planning & e-Learning”, February 28, 2001).
 
Success stories and case studies abound. There is no longer a question as to whether e-learning is a winning technology, but whether it can produce winning results at your organization. Plan ahead, document proposed metrics, and measure performance from the outset. When built well and used effectively, e-learning has proven that it not only saves money but can also enhance employee productivity and satisfaction. 
A Senior Internet Business Consultant and a regular writer and speaker, Carmine Porco is the Vice-President of Prescient Digital Media. For more information on Prescient’s CMS Blueprint service, or for a free copy of the white paper “Finding ROI”, please contact us.