Learning and Certification Platform
My role
Timeline
01
Context
Twilio has a rich ecosystem of SI, GSI, and ISV partners. To help customers navigate this ecosystem and help partners demonstrate their competency and capability with various Twilio platforms, a Partner Practice Development (PPD) team I was part of was tasked with building a certification program.
This certification program would become an essential part of Twilio’s partner tiering mechanism.
02
Problem statement
Before this platform existed, the PPD team drove partner enablement through a generic Learning Management System (LMS) with role-specific learning paths.
That was a manual process for the team. Furthermore, partners lacked visibility. The individual consultants could see learning paths they had completed, but there was no way to help them decide which learning path to take next, provide any overview to how the whole partner organization is doing or orchestrate the content in any meaningful way.
03
Goals
Orchestrate content
Allow dependencies between learning paths.
Automate administering exams and exam retakes.
Segment content
Allow learners to specify what roles and technologies they are interested in and promote content that matches those.
Localization
Support content and exams across multiple languages.
Cross-platform content
Allow learners to consume content from various external systems (LMS, practical exercise system, proctored exam system, etc).
Single source of truth
Allow us to have a single source of truth so we don’t have to reconcile data from all the individual systems the platfrom integrates with.
04
Design process
To get a headstart, Twilio engaged a vendor with an existing learning orchestration platform that we could use as a foundation to build upon.
This platform came with basic organization of learning into learning paths, search capabilities and filters. Essentially the first half of the first goal listed above. The rest had to be built.
Orchestrating content
The vendor’s platform gave us the ability to make a learning path dependent on the completion of another learning path. So to be able to take Advanced Messaging, a learner first has to complete Basic Messaging.
But there could only be a dependency on one specific Learning Path. To make this dependency mechanism more flexible, I designed a new one.
Every learning path can award a badge for its completion, so my idea was to rebuild the dependencies around these badges. This gave us much more flexibility in building the dependency tree. Turned out to be a powerful mechanism that later also proved essential for localization.
The aforementioned badges became an important part of the visual identity of the platform. The initial requirement was to award badges only for certifications. But to help gamify the experience and motivate learners, we decided with the platform’s key stakeholders to also add badges for minor achievements.
I designed two visually distinct types of badges: large badges awarded only for certifications and smaller ones for minor achievements like completing a knowledge check. Below are some of my early prototypes.
Segmenting content
The PPD team was already building e-learning content for multiple personas: developers, sales specialists, marketers, solution architects, etc.
So one of the main requirements going into this project was to be able to clearly segment the content for different personas. Since this was the main attribute that everything else depended on, I proposed tying the different personas to various accent colors from Twilio’s brand palette. So for example all developer content would have a blue accent, all solution architecture would have a green accent, etc.
This proved popular with the users and formed the basis of our UI language for the rest of the platform.
Localization
This turned out to be a surprisingly challenging area. The badge dependency mechanism described above solved a lot of the admin challenges. But UX-wise, there were a lot of challenges I needed to address. This was caused by the fact that we didn’t translate all of our content and the majority was only available in English. Here’s an example of some UX challenges I had to solve:
- Users could select their preferred language in their profile
- If some content was available in that language, platform should show that version, otherwise show English version
- But what if the user started taking a learning path in English and later a version in their preferred language became available? Should they be allowed to enroll in that?
- Moreover, should users be able to complete multiple versions of the same learning path (in their preferred language and in English)? What does that mean for the number of exam attempts they’re allowed to take and the badges they’re awarded?
- What should the UX be if users switch their preferred language?
The logic driving this proved quite challenging to define and it had to be updated multiple times in QA to account for unforeseen edge cases.
Cross-platform content
The platform itself didn’t host the learning content, that was done by an LMS. It also didn’t store the learner data, that was stored in a CRM. And lastly, it also needed to integrate with some external systems like TwilioQuest which provided learners with hands-on exercises.
To facilitate all this, I had to define interfaces with all of these platforms. This usually involved looking at the integrations each platform provided, typically it’d be some REST API, and specifying the primitives that should be used for each operation. Then I worked with the development team to build custom connectors that would implement these interfaces.
Single source of truth
We wanted the platform to use a single source of truth for all of its data about learners, completed content, exam results, and certifications awarded. This would help us with internal reporting, but it also allowed us to give our partners better visibility of all of this information.
This requirement meant that all of the above-mentioned interfaces connecting all of these systems had to update data in the one system we designated as the single source of truth.
This put additional requirements on those interfaces and added design challenges around availability and data consistency. For example, what happens when the interface to the LMS system becomes temporarily unavailable and learners complete some content during this period?
05
Impact & Lessons learnt
After its launch, the platform became the cornerstone of Twilio’s partner certification program. During its first year, over 10,000 partner consultants used it and over 3,000 certifications were awarded.
After the launch, I continued working on the platform as its Product Manager. I drove its roadmap and worked with the development team on implementing new features. I launched 1 major new version a year after going live.