Success Stories:   Online Testing, Measurement & Evaluation

Business Objective

To ensure the success of a Global SAP Training Program for a Fortune 500 Manufacturing Company.

Background

With the assessment content pre-determined, our client needed an effective and secure online assessment system in place and tested in less than three weeks. In order to provide the necessary level of security and privacy, we recommended a secure hosted solution that protected the privacy of the employees' responses and evaluations without impacting their internal IT systems.

To ensure that our client was able to analyze data to determine effective instruction, we chose a system that provided powerful, real-time reporting tools that were fully customizable and easily accessible over the web. Providing our client with the administrative support, programming and interface design for this solution allowed them to focus on the actual training.

Challenges

 
  		• Ensure the readiness of employees to use the SAP system through assessment and certification.
  		• Create one evaluation system to be used by a culturally diverse and multilingual audience.
  		• Enable a smooth introduction to workplace skills testing.
  		• Implement first assessment within three weeks.

    Analysis

    The implementation of an Enterprise Resource Planning (ERP) system, such as SAP, is complex and resource intensive. Best practices indicate that successful implementations are best performed in stages. Some companies have been very successful in their launch, others experienced losses when their employees were not properly trained in the functions and processes related to their roles and responsibilities. This assessment program was required to accommodate four primary functions:

     
     
  		• Assess the level of knowledge acquired during classroom training.
  		• Provide further instruction via feedback to the students.
  		• Provide feedback to trainers.
  		• Maintain the confidentiality of the users.

    Design

    With these four guiding principles, we designed a solution to meet their needs.

     
     
    1. 
    		 Assess the Level of Knowledge Acquired

    We designed an intuitive interface that allowed the user to focus on the content. Each question and answer had a weighted score. A pre-defined cumulative score was required for successful completion.


    2. 
    		 Provide Further Instruction via Feedback

    The objective of the project was to ensure the successful training for an SAP implementation. To further the classroom training that was provided, the online assessment program instantly notified students of their scores, and provided customized instructional feedback for each answer.
    3. 
    		 Provide Feedback to Trainers

    A Kirkpatrick Level 1 Evaluation was administered online at the end of each class. Daily reports were given to each trainer providing a means for continuous improvement.
    4. 
    		 Maintain Confidentiality

    A secure hosted solution provided the framework to develop an ID and password protected site. With the administration of the user information and tracking being handled by KDG, the security of the system and the confidentiality of the users was further protected. By defining specific assessments that were available to each employee, additional security was provided.

    Development

    Our client worked with SAP experts and internal trainers to develop appropriate assessment content. KDG then used this content to create online assessments.

    We selected a hosted assessment program provided by one of our technology partners that provided the security and rapid deployment capabilities that our client required.

    We designed an intuitive interface that allowed users to concentrate on the program content instead of having to learn how to use it.

    KDG also created a series of online reports that our client used on a regular basis.

    Implementation

    Since the rollout of this program was occurring in Europe, we worked with our partner to support the solution through their European office thereby providing timely technical support.

    Incorporating existing and familiar employee identification (ID) numbers, we implemented a secure system that allowed users to change their passwords on their first login. The multi-tiered secure access enabled us to differentiate levels of access to administrators, offering varying levels of support and reporting capabilities.

    Evaluation

    The initial rollout was implemented as a pilot. Even though the concept of work skills assessment was a challenge to overcome, employees appreciated the opportunity to test their learning. The instructional team was able to modify training courses and methods based on employee feedback.

    Based on the success of this pilot, our client is moving forward with additional releases of the training and evaluation programs. The first rollout consisted of 60 unique assessments in two languages that were taken by 450 employees. The next release will have 80 unique assessments, available in five languages that will be taken by 2500 employees.