The TopClass Blog

Your association has an abundance of useful data in your LMS, AMS, website’s CMS, community platform, email marketing platform, and elsewhere. This data represents answers to questions— answers that lead to insight from learning analytics and insight that leads to action.

What questions would help you understand and improve the performance and value of your education programs?

What questions would help you make decisions about your education portfolio, marketing campaigns, or specific programs?

What questions would help you spot connections between different behaviors? For example, are people who enroll in a specific course format more likely to become repeat customers or renew their membership?

What questions would help you see correlations between product characteristics and outcomes? Are programs with a specific topic more likely to be completed or score higher on learner satisfaction?

When you ask the right questions of your data, the information you glean can help you decide where to focus your efforts and how to improve your offerings. A recent Association Analytics webinar, Beyond Basic Learning Metrics: Using Analytics to Uncover Insights & Education Gaps, filled my head with intriguing learning analytics scenarios and inspired this post.

Gaps in your association’s curriculum

By observing behavioral data, you can learn what people in different audience segments are interested in. Gather data from:

•    Email clicks
•    Website, community, and LMS searches
•    Website page visits
•    Community discussions
•    Conference session and webinar attendance
•    Video view
•    Surveys and program evaluations
•    Interest inventory updates

Talk to employers too as you identify gaps in your program portfolio and decide on educational products to develop. You may learn you need more content for seasoned professionals or content about new industry trends or regulations.

Product performance

Look at your products’ market and program performance. First, analyze the market performance of each product, program type/format, and your entire curriculum by customer segment:

•    First-time customers
•    Members vs. non-members
•    Career stage
•    Job function
•    Other relevant demographic categories

Examine popularity (registration numbers) by product, topic, format, and member/market segment. Are particular formats more popular? For example, compare live instruction vs. self-paced, instructor-led training vs. cohort programs.

What times of year is enrollment lowest and highest? Does the variance have to do with seasonal factors, exam dates, or membership expiration dates?  

How is each product performing? What learner behavior or engagement metrics would help you understand the quality of the learning experience?

•    Watching videos
•    Taking quizzes
•    Doing assignments
•    Participating in online community discussions
•    Hours spent in program

Can you identify leading indicators of learner success? Perhaps a mid-course learner survey would provide helpful data.

Program or course completion rate is a useful performance metric, but consider how you define completion rate. If you base it on learners submitting evaluations, what if some complete the course but not the evaluation?

Pass percentage and test metrics provide useful data for course and exam design.

Collect data that helps you identify why learners don’t complete the course so you can take steps to improve learner retention. Was the course too difficult? Irrelevant? Or did life get in the way?

Post-program evaluations provide data that helps you improve course content, format, and instructor selection and training. How do you choose instructors for the course? Do you pick whoever volunteers? Do you train them on adult learning principles?

Product value to learners

Traditionally, learners evaluate the program upon completion. But did the learning stick? Was it valuable in the long run? You’ll only know the answers to these questions—and the true value of the program—if you follow up several weeks later.

Check in with learners to see if they retained and applied their new skills and knowledge. What was the impact on their job and career? Did it lead to improved performance, promotion, a raise, new job, or new direction? This is your opportunity to collect testimonials for marketing.

Product value to association

Which programs are helping the bottom line? Analyze revenue by product, topic, format, member/market segment, and non-member vs. member.

Which products generate the highest long-term value for your association? Which should go on the sunset review list?

What percentage of members and of member segments are participating in courses or other educational programs?

Learner acquisition and retention

Behavioral data helps you identify prospective learners. Their clicks and views demonstrate their topic interests. Create targeted marketing campaigns based on their interests and career stages.

Deepen your understanding of learner needs by inviting them to participate in a pre-program survey. What led them to this product? Is something going on at work, in their career, or in the industry?

Are there any market segments who are not enrolling? What might your portfolio be missing?

Identify gateway programs that are more likely to lead to repeat business among members and non-members. Are there any common characteristics?

Member acquisition and engagement

Which programs are more likely to lead to member acquisition? Is there a correlation between specific topics or formats and member acquisition?

What percentage of members (and different member segments) attend online education programs? What type of programs do first-year renewing members take?

Is retention higher for members who participate vs. members who don’t? Is there any correlation between specific types of programs, topics, or formats and retention?

Reports vs. analytics

Association Analytics made a good point at the end of their webinar. With analytics, you don’t just look at a report, your curiosity prompts you to dig deeper, to compare and contrast, and draw conclusions. Learning analytics leads to decisions and action if you leave your preconceived notions at the door and let the data lead you to insights based on facts.