8 Keys to Effective Training

In my 15 years in the language training business, I’ve seen many different approaches to language training – some successful, many not so successful.  In this article I’d like to highlight some of the success factors I believe contribute to an effective training program.  Although these principles are specifically about language training, they also apply to other types of training.

1.      Have a clear objective.

If you don’t know what you expect your training to accomplish, it probably will not accomplish much.  You might think that obviously improving language skill is the objective of language training.  However, that is really not a clear enough objective because it does not provide any guidance on how best to structure or measure your program.  Here are some better examples of training objectives: 

• Enable our customer service reps to deal with English language phone inquiries. 
• Improve average TOEIC score of our staff by 100 points within a year.
• Improve English ability of our staff by one CEF* level in one year.

   * Common European Framework – a language proficiency scale for all EU languages

These objectives are specific enough to allow measurement of your program’s performance and to develop a timeline to achieve your objective.  Notice that the first objective focuses on a work skill, whereas the second two focus on developing overall language ability.  You need to decide which of these types of objective suits your organization because the first type can be achieved with a short course, but the second type requires a longer term program. And that brings us to point number two.

2.      Have a realistic time line.

You need to have a realistic time line to achieve your objective. The time line keeps the training focused, and you might be surprised at how unproductive training can become when there isn’t one.  If you focus on a particular work skill, you may achieve your objective with a short course of perhaps 30 hours.  Thus, you can plan for all new hires who need that skill to participate in a 30-hr training course.  If your objective is to improve the overall language skill of your staff, then you’ll need a longer time line. For example, to improve average TOEIC scores by 100 points requires 100-120 hours of focused training (but it could be much longer if the training is not focused on both the objective and the time line).

3.      Use the right measurement.

In all types of teaching, the test that’s used to evaluate learners has a big influence on the course itself.  We call this the “wash back” effect.  Here’s an example of negative “wash back.”  One company decided their staff needed to write better emails and participate more effectively in meetings.  So, they set up a large-scale training program to develop these skills.  However, they decided to use the TOEIC test as a progress measurement, and they set unrealistic targets for every participant to achieve within 6 months.  Well, after about two months nearly all email writing and meeting skills had disappeared from the course content, replaced by TOEIC test preparation (at the request of both the trainees and the training manager).  What a shame, because TOEIC preparation does almost nothing to improve email writing or meeting skills, which were the initial aims of the program.  If your objective is a particular work skill, then your measurement should be based on that skill – i.e., if your objective is telephone skills, you should evaluate by telephone role plays; if your objective is email, evaluate by writing tasks.  By using the right measurement, you turn the “wash back” effect to your advantage.

4.      Get feedback from trainees – and use it.

Adult learners are not just big kids, and they don’t want to be treated the way their schoolteachers used to treat them.  Adult learners need to feel motivated, and they need to feel the training is worth their time.  They will dislike pressure situations or anything that makes them risk failure in class.  Most importantly, an adult learner who feels negatively about a training course will not learn.  So it’s important to know what trainees think about their training program and to respond to their views.  The three best questions to ask your trainees are 1) What is the most useful part of the course, 2) How can the course be improved, and 3) Do you have any other comments about the course? Most importantly, once you’ve collected the feedback, use it to improve the training.  Try to fix any problems that are revealed, but also be sure to note positive aspects and incorporate these into the design of future courses.

5.      Personalize it.

Learners respond better to examples that they can relate to.  For example, a group of accountants will be bored by an email lesson about requesting a hotel booking because they never do that.  How much better it would be if the instructor, who has been provided with numerous authentic emails from the company, instead presents a realistic example of requesting an overdue inventory report.  And he highlights some errors that he has seen in their actual emails!  It’s an unfortunate fact that many of the situations presented in English textbooks are irrelevant to your trainees. It takes effort on the part of the trainer, the training provider and the client company to personalize the training content as much as possible.  You should expect that effort from your training provider but also do your part by providing as much input as possible, e.g. email/report samples, product information, etc.

6.      Focus on skills, not on knowledge.

The ability to use a language is about practical skills, not knowledge.  I know Thais with TOEIC scores over 850 who cannot participate effectively in a meeting or write a professional email in English.   They know English, but they can’t do English.  So focus your training on doing English. Do not allow it to become simply a series of grammar and vocabulary lessons; make sure that these inputs always lead to speaking or writing tasks, and that the inputs help trainees to do the task better.  Otherwise, you’re just wasting time and money.

7.      Increase the frequency of lessons.

A 30-hour course completed in five weeks will be more effective than a 30-hour course completed in 10 weeks, which is still better than a 30-hour course completed in 15 weeks.  The frequency of training sessions increases the effectiveness by allowing more “scaffolding”, a process where each lesson builds on the learning of the previous lesson.  The longer the gap between sessions, the more learning is forgotten during the gap. To avoid the problem of “two steps forward, one step back” a class should meet at least two times per week (three is even better).  If it is not possible to conduct classroom training this frequently, you should consider combining self-study or distance learning methods with your classroom training.  And that leads to my last point.

8.      Consider homework.

I say consider homework, because in my experience about 90% of adult learners do not want to do homework.  They are busy, and it makes them feel like they’re back in school again.  Nevertheless, any additional exposure to the training content outside of class will improve its effectiveness.  This is especially true if the classroom sessions are infrequent (less than twice per week).  So, it’s worth discussing whether your trainees will accept a limited amount of homework as a requirement of their training (it needs to be a requirement; otherwise it will not be done). I would recommend not more than 15 minutes of homework per classroom hour.  The following are some useful types of homework:

•  Revision of grammar/vocabulary.  Typical activities can be found in self-study workbooks or CD-ROMs that come with many textbooks.
•  Personalization.  Students apply language presented in class to personal contexts in short written assignments (may be presented orally in the next class).
•  Email to the instructor.  Each trainee emails the instructor during the gap between classes (may be an assigned task as mentioned above or free writing).
•  Prepare short class presentations. Trainees prepare to present something to the class (a news item, personal story, recipe, instruction, etc) for 3-5 minutes in next class.

I hope these observations about language training are useful for you.  As always, if you have any questions about your particular training needs, please feel free to contact me at sean@inlinguabangkok.com or 02 254 0955 ext. 101. 

Measuring the Impact of Language Training

Measurability, benchmarking, accountability….these are the buzzwords of the “ISO” era. In the last decade, many companies have overhauled their standards in order to qualify for the ISO certification. English training managed to escape analytical scrutiny for many years, but that has gradually changed over the years.

We started to see the impact of the “ISO” era about eight years ago, when clients first started asking if we could measure the improvement of their trainees after an English course. We met this requirement by giving the trainees a test on the course content both at the beginning and at the end of the course and comparing the results. Nowadays, this practice of pre- and post-testing has become commonplace. Around 70% of our courses now involve some form of pre-/post-testing.

In recent years, we’ve seen an increased focus on the meaning of the test results. Employers want to know what it means if a trainee scored 85%, or got a B grade, or achieved Level 2. How does the result relate to their communication skill on the job? Companies are looking for more and more sophisticated measurements of their trainees and their training programs. This presents challenges for both training providers and for the companies themselves because measuring the impact of language training is no simple matter.

HIERARCHY OF TRAINING MEASUREMENT

There is a kind of hierarchy in evaluating training programs, from the simplest measures (1) to the most sophisticated (4).

1. Trainee reaction. Collect the trainees’ feedback about the program.

2. Trainee performance on the course. Test how well the trainees learn the training content.

3. Transfer to job performance. Evaluate the impact on real work communication.

4. ROI (return on investment). Assess the financial benefit of improved performance compared to the cost of training.

Let’s look at how these measures apply to language training.

1. Measuring trainee reaction. This is the simplest evaluation of an English training program. Using a questionnaire, trainees are asked to rate the program on criteria such as instructor’s knowledge and methods, learning materials, interest level, etc.

A good questionnaire will also include open questions like “What is the most useful part of the course?” and “How could this course be improved?” Answers to these open questions are usually the most useful feedback. However, the numerical results from questionnaires are not always a reliable indicator of training performance. For one things, trainees may hesitate to give low scores to a teacher they like, even if his/her teaching performance is poor. In addition, the trainees may have different objectives than the company – for example, they may be more focused on having fun together than doing challenging assignments, and this will affect how they rate a course.

2. Trainee performance on the course. The way to measure how much trainees have learned or improved during a course is to conduct pre- and post-training assessments. (A final test alone may show what the participants know or can do at the end, but cannot show how much impact the course had on it.) There are two approaches you can take to pre-/post-testing: curriculum-based testing or proficiency-based testing.

§ Curriculum-based testing. In its most basic form, the pre-/post-test is a progress test – often right from a textbook– which is administered both before and after training. The good thing about this type of evaluation is the results will clearly show an impact from the training. And it will be in statistical form, so that performance of different trainees (or teachers) can be compared.

The drawback is that the result doesn’t tell you much about the trainees’ ability to use English in the real world. She improved her score by 50%….but can she deal with an English caller on the phone?

§ Proficiency-based testing. If you want a measurement that means something in the real world, you should consider proficiency-based testing. This type of pre-/post-test is not linked to the content of any specific training course. Instead, it measures language skills on a proficiency scale. TOEIC is a well-known example of a proficiency test that measures reading, grammar and listening skills. Other types of proficiency tests can measure speaking skill or writing skill. This type of testing provides a more meaningful measurement of both individual performance and overall program performance. Using a proficiency scale, you can set performance targets for different job positions, and you can estimate training hours for staff to achieve targets.

However, there are two important considerations. Firstly, language proficiency improves slowly. It may take 80-120 hours of training to produce a measurable improvement in proficiency scores, so using proficiency tests involves setting longer range goals. Secondly, proficiency testing must be conducted by experts in order to ensure consistency and reliability. Using proficiency testing will certainly add cost and complexity to your training program. A professional training provider can help you do this in the most cost-effective and efficient way.

3. Transfer to job performance. The next step up from measuring trainee performance on a training course is measuring how they apply it to their work performance. This is very difficult to measure directly. If a trainee writes emails in English, you could analyse a sample of their email writing before and after training. But just collecting the samples poses admin challenges, not to mention who would evaluate them and how. Trying to evaluate telephone skills or meeting skills presents even greater challenges. The only practical option is to measure performance indirectly through work-related assessment tasks, which are scored by a trained evaluator. For example, trainees who receive customer enquiries by phone could perform authentic role-play tasks as an assessment. Trainees who make presentations can be evaluated on a presentation. It is possible to devise a proficiency scale for any of these skills and then set performance targets for trainees.

This type of task-based measurement has very high validity, but it requires a lot of development to create authentic tests, and it requires expert evaluators in order to ensure consistency and reliability. A professional training provider can provide this service.

4. Return on investment. ROI is the ultimate measure of any business investment. Wouldn’t it be great if you could say to your top management, “This year we invested 250,000 baht in English training. The resulting improvement in language skill added 1.8 million baht to our revenue through added sales and customer retention.”

Unfortunately, that’s not going to happen. It is difficult to measure the financial impact of a training program because you usually cannot separate that impact from other factors (e.g., market forces, the economy, changes in staff, other training programs, etc). Moreover, many of the staff who participate in English training are not frontline sales or customer service staff, so their language skills do not directly affect sales or customer retention – they may only use English for internal contacts within your company. The impact of their language skill has more to do with how much confusion or frustration occurs in communication within your company, rather than making or losing money. Can we measure confusion, or put a money value on it? Probably not.

CONCLUSIONS

Training is an investment, and like other investments it must provide measurable benefits to justify the cost. While it may not be possible to measure the financial benefits of language training, it is certainly possible to measure the practical impact of language training on the knowledge and skills of the trainees. At a minimum, a curriculum-based pre- and post-training test will show you that the participants learned something on a course. If you have a longer range goal of improving the language skills of your staff over several years, then proficiency-based testing is the way to go. If your people perform specific communication tasks, such as making sales presentations, receiving telephone calls from customers, participating in project meetings, then task-based assessments will give you the most meaningful results.

I welcome any questions you may have about your particular situation or requirements. I hope you will feel free to contact me.