Credits -

Product - Nishant Tyagi, Rahul Aggarwal, Ankit Dhanda, Setal Patel

AI - Vivek Sharma, Nilesh Hirani, Rishabh Mehrotra

Engineering - Sonu Verma, Shubham Choudhary

Blog Illustration - Haritha NV

Introduction

At ShareChat, we send more than 2.5 billion notifications a day. Just like all other notifications, we are also trying to engage our users through it. That’s what 'conversion' is, right?

When we chase and try selling something to the user, and they buy it, then we've managed to convert the user. Life would be simpler if all of us were content with what we have. However, there is a constant need for more. All our users at ShareChat want more, and we understand that.

Users, conversions, stickiness - not because we are greedy but because we know we have got something so good that we believe everyone should try on. So we wake ourselves up and aspire to do that job, of improving user experience. However, given the scale, it isn't easy because one size doesn't fit all.

Our Problem

Not everyone likes cheesecake, but everyone loves desserts. Some of us drool over ice creams, but others may like brownies, apple pies, tiramisus, or gulab jamuns. If you are hosting a party and want to be the best host, you would want to have what everyone likes.

So to know what everyone's guilty pleasure is, you start your research. You go down memory lane, check their social media, or even call their exes (too far?) and come up with answers. And then you wait! Once the dessert is served, you anxiously wait for their reactions. Is it a smiling face with heart eyes or a grimacing face?

That is what we, at ShareChat, are trying to do. Scratching our heads, building hypotheses, writing logic, and running experiments to understand what our users want. What is the type of content they are looking for, because we want to be the best host out there.

Want to know what we do, and how we do it?

We personalize and experiment. The Artificial Intelligence team is constantly trying to use explicit and implicit signals generated from posts, interactions, and behavior on the platform to personalize the user experience. They have built different sets of services/models which have their purposes of powering feeds across ShareChat.

To compound the gains we get from these services, the AI team has democratized Personalization As A Service(PAAS) amongst other teams. One such initiative is personalizing all notifications sent by ShareChat. To kickstart this effort, we experimented by personalizing 1 out of 4 different kinds of notifications. We call them “Sticky” notifications.

Intervention

Sticky notifications are a collection of six tag names in carousel format sent as notifications. When clicked, this notification lands a user on the page specific to that tag which contains posts from that tag. Previously Sticky notifications were based on a common Trending Tags logic that was driven by a Top Tags service (AI service for top tags in a language/demographics) and the Content Operations Team. This logic was targeted but not personalized.

We replaced the powering logic behind this type of notification. The AI team has built a lot of affinity-based models which take into account content, user signal, and features to understand a user’s affinity to a certain content/post/tag category. It then predicts a confidence score with which it thinks the user might interact with that type of post/tag again.

One such model/service is the tag-affinity service, which predicts a score that indicates the propensity of the user to interact with that tag in the coming days. Hence, we ran an experiment that used a tag-affinity service to power sticky notifications at a user level.

Experiment Lifecycle

• We launched an experiment with multiple test/treatment variants + control variants to test our hypothesis.
• Variant definition: We experimented with different combinations of Affinity proposed) and Trending (existing) logic-based slots.
• The winning variant was scaled-up to 100 percent once users reverified the impact on the key metrics. A comparative analysis for the relative change in metrics for five percent variants and 50 percent variants was done.

Impact

One of ShareChat’s values is integrity which translates to Commit earnestly, Measure honestly, Growth follows. And we follow our values very closely.

It's crucial to stand by this line and should be followed to every extent possible.

Our scale and experimentation setup allows us to measure the impact we derive out of our interventions. The winning variant for us was the one with six affinity-based tag notifications.

A few of the key metrics which we observed were:

• Notification Click-through Rate (CTR)
  • Event Level - 35 percent relative increase in event CTR at an impression level.
  • User Level - 40 percent more users clicking on notifications now with respect to control.
• Incremental Users (DAU)
  • On a daily basis, more than a million users are opening ShareChat solely because of this change. The compound effect of this change is already there to be seen.
• Session per User - Two percent relative increase. This is over and above the incremental DAU impact. This could be attributed to better event CTR for test variants.
• Time spent per User - Five percent relative increase in average time spent per user.

The win to celebrate here is not just the fact that we were able to improve CTR and incremental DAU but we also managed to increase the number of sessions and time spent per user. It is extremely gratifying that we not only brought more users to the platform but improved their experience on the app as well. The increased session duration and engagement on tag feeds originating from the test group is a testament to the fact that the new personalized experience is more entertaining.

Conclusion

Why next steps? Because we want more, our users want more, and we continue to thrive to achieve perfection.

For notifications and personalization services, the next step is to personalize the remaining three categories of notifications using PAAS. Theoretically, it does make sense to use signals specific to notification CTRs and sessions originating from notifications as a feedback signal to further optimize user targeting and experience.

Using PAAS not just for notifications but also for other use cases. What if we are already working on it? You'll be able to see the result soon!

We are hiring!

At ShareChat, we believe in keeping our Bharat users entertained in their language of preference. We are revolutionizing how India consumes content and the best in class technology is at the forefront of this transformation. In addition, you'll also get the opportunity to work on some of the most complex problems at scale while creating an impact on the broader content creation ecosystem.

Exciting? You can check out various open roles here!