Did AI select you as a candidate for a promotional offer?
Let us take a use case where a company that owns a popular, leading job portal with millions of users wishes to roll out a new training course on — How to prepare for Data Science interviews?
The company wishes to identify a pilot user group of say few hundreds of users who will be contacted regarding this new course offering.
A given user’s activities on the job portal typically comprises of — updating skills or completion of courses or certifications gained in the Resume, connecting to more users, following key Influencers in a chosen field, publishing new content be it posts, articles, audio clips, videos etc., following hashtags, volunteering for initiatives, recommending other users, receiving recommendations, receiving skills endorsements and so on.
These lead to the different attributes that give information about the user. These attributes across all the users makes up the data that will help in selecting the pilot user group. Note that data also includes (as a label) information about whether the user responded to a similar course offering in the past or not.
How can a pilot user group be selected from the vast user base?
To crunch this data, derive insights and select a pilot user group, a Machine Learning system, specifically a Recommendation Engine is deployed.
What is a Recommendation Engine?
Recommendation Engines or Systems, also referred to as Information Filtering systems are a class of algorithms that aim to provide the most relevant information of interest to the user by filtering it from of a much larger information base. Recommendation Engines discover data patterns in the data by learning users’ choices and produce results that closely match the users’ interests. A couple of examples are Netflix’s movie recommendations and Amazon’s product recommendations as shown below.
Given the data, the Recommendation Engine arrives at the short list of hundreds of pilot users who may be contacted about the new course offering. As the e-mails are getting ready to roll out to the selected pilot user group, a pertinent question is asked — Why has User A been chosen to be contacted for the promotional offer and not User B?
The Black Box nature of the Recommendation Engine model mechanics offers no help to answer such a question.
What would help explain the recommendation given?
If there were answers like one or more of the following, it would explain the recommendation and certainly help in sending out customized e-mails to members of the pilot user group.
· User A was selected as, this user completed an industry certification recently and has made updates to his or her Resume more than 7 times in the last month.
· User A was selected as, this user has grown his or her connected social network by more than 10 % in the last two weeks.
· User A was selected as, this user has increased by more than 15 % his or her following of Data Science, AI, ML groups and Influencers in the last 3 months.
How could these explanations be arrived at?
To address the gap in explainability of the decisions taken, consider these features grouped into buckets that have some semantic meaning. These buckets could be imagined as follows for this use case.
Achiever: number of skills updated, number of additions to accomplishments, number of certifications added, number of overall updates to Resume etc.
Creator: number of posts written, number of articles written, number of engagements on created content, number of comments given, number of recommendations given and received, number of skills endorsements given and received etc.
Socializer: number of Influencers followed, number of new connections made, number of hashtags followed, rate of growth of network etc.
Now, we could have separate Component Models (models that are trained on a set of related features only) that would classify whether a user is selected or not in the pilot group and give a score for the same.
The Decision Tree Classifier may be used for building Component Models in this case as the goal is to address explainability. A Decision Tree is a simple but powerful learning and classification model that is used in supervised machine learning. A Decision Tree is a classifier in the form of a tree that contains decision nodes and leaves. As shown above, each internal node represents a test on an attribute, each branch represents the outcome of the test, and each leaf node represents a class label (decision taken after computing all attributes). The paths from root to leaf represent classification rules. These classification rules provide the explanation of how a decision was reached i.e. why an instance was classified with a certain class label.
Now, the results of all the component models are compared for each prediction as shown in the table below. The prediction score is a score that indicates the probability of a user selected for the pilot user group. This is typically expressed as a probability between 0 and 1. To illustrate this use case better, consider that a score of 0.6 and above may be taken to mean that the user is selected in the pilot user group and anything lower means that the user is not selected.
What more can be done with explanations?
The table above offers an explanation for why a certain user was selected to be a part of the pilot user group. With this at hand, such a selected user may be contacted with a personalized e-mail (that wouldn’t be possible without the explanations from component models). Some examples could be —
User A is contacted with an e-mail that this course will offer a completion certificate that could be showcased on his or her profile in the portal.
User B is contacted with an e-mail about the promotional offer further stating that a certain number of people in his or her networked community have also been contacted for the offer.
User C is contacted with an e-mail that this course will have an active discussion forum where Q & A will be encouraged and a user will get credits for active engagement in the forum.
Such explanations thus give a meaningful way to engage with the user. With such personalization, there may be a higher likelihood for users from the selected pilot group to sign up the promotional offer.
Note: The component models do not replace the Black Box Recommendation Engine but supplement it for better explainability. In this approach, the Black Box model provides the accuracy and performance whereas the component models provide the necessary explanation for the objective at hand.
