What TikTok Can Teach Any Company About Data Analytics and User Engagement

Data scientists and Marketing Departments: you can learn a lot more from TikTok than just how to dance

Everyday I talk to companies trying to emulate what the social media giants are doing. They collect data, they look at social graphs, and they build algorithms to determine how to improve customer engagement. Perhaps it’s time to look at how TikTok does it. Honestly, it’s very easy (and fast) to incorporate their methods into your analytics. Let’s look at TikTok’s differentiators and the lessons we can learn from them.

For a company using traditional social media to improve user/customer engagement, the workflow looks like this:

The “explosion” is not a good thing. What really happens is your network gets too big and your followers won’t be shown all of your content all of the time.

At some point you begin to use paid ads, but often these ad networks fail to provide lift, and this is proven many, many, many times.

If you don’t get enough followers on your channel then you’ll just become a “viewer” regardless of the quantity and quality of your content. The graph determines the direction the content travels (from creator through the follower graph). This shapes the future of the social network. If you don’t get followers, your content gets no distribution.

TikTok is different. They in fact do have a “Following” tab which is akin to the “follower graph” but it’s just a secondary pathway for content. It gets just a fraction of the traffic of the FYP (“For You Page”) which is algorithm-based. The FYP shows you content from creators you wouldn’t normally be exposed to because you don’t follow them. We call this algorithm an “interest graph”.

Lesson Learned #1 : TikTok has both the follower and interest graphs. I’ll bet they used A/B testing to determine which was better for them. A/B testing is a good way to test a controversial hypothesis. A good data scientist can help you with this. Don’t rely on your gut instincts.

Traditional social media uses the follower graph to approximate the interest graph, but that rarely works well. Your friends don’t always have the same interests as you. Eventually the signal-to-noise level becomes so bad that people just tune out. They are constantly shown the same boring content. How many times have you sat next to someone who was doomscrolling their Facebook feed? Does that seem engaging?

Lesson Learned #2: The interest graph is much more important than the social graph. When I need a recommendation for a new TV I don’t ask my social graph, I let my e-commerce vendor recommend one based on my interests and behaviors.

In a nutshell, this is what TikTok does after you upload a video:

This is important and novel, let me put it into words:

take the video, determine what the content is, tag it accordingly. Let’s call this “the Video AI”
get users to watch a few videos and determine their engagement. Let’s call this the interest graph algorithm
use the interest graph to show you future content, but carefully balance showing you what you like today vs what you might in the future so you don’t tune out. Let’s call this the exploit vs explore algorithm. This is classic reinforcement learning that most data scientists will immediately recognize.
TikTok doesn’t badger users to disclose personal information like other social media outlets. Let’s call this No Explicit Profiling. They do ask you for your contacts, but it definitely feels less intrusive than the other big social media outlets.

Let’s dig deeper on each of these items:

Differentiator B: Video AI

How do you design an algorithm to determine what the content of a video is? Visual AI for object detection is (mostly) a solved problem. But determining the purpose of a video is much harder. Example: we detect a human and a cat in a video…is the purpose of the video a “woman dancing with her cat in a humorous way” or “a woman training her cat to use the litterbox”? Totally different interpretations.

But it turns out if we have the list of objects we can make initial guesses at the content if we semantically-enrich the objects detected with other data ancillary to the video. Examples:

if a given content creator makes a lot of dance videos, it’s likely this new video is also about dancing
if a given consumer watches and likes a lot of dance videos, TikTok can show them a video that might be about dancing and then see what the feedback is. If this is done enough times we get a better likelihood of its purpose.

Video analytics is hard, but it’s also untapped. More people will engage with video for a longer amount of time than they will with text. I see people on their phone doomscrolling through their social media feed barely reading the first sentence of most posts…but most users will engage with a TikTok video for much longer.

Lesson Learned #3: Video is hard, but more engaging. Video crosses the language barrier, opening new markets. Everyone is doing text, maybe it’s time to focus your efforts on video. Start with basic tagging (another mostly-solved problem) and let your users stealthily help you to train the tagging algorithms. Microsoft’s Cognitive Services can help.

Differentiator C: The Interest Graph Algorithm

Most social media outlets bombard you with multiple pieces of content at the same time. Your screen probably has a ton of content surrounded by a ton of ads, and a scroll bar to let you see even more.

In that model, no algorithm can determine what has caught your interest on that page if/when you pause and focus. Are you looking at an ad or content? And which piece of content? Certainly if you “Like” something it’s easier but a lot of people will never click “Like” and others “Like” everything they see. The “Like” becomes almost meaningless.

Facebook and graph-based social media expose us to “weak ties”. Weak social connections. People you never really see or interact with on a regular basis are suddenly your “friends”…a college acquaintance from years ago, a work colleague from a previous job, a long-lost cousin you reconnected with 7 years ago and have never met face-to-face. You have far more “friends” on social media than you do in the physical world and they likely have ZERO influence over you.

TikTok does it differently. They show you a single video at a time, full-screen. Were you engaged? Did you watch the whole video, or click away after 2 seconds? Did you watch it twice? Did you share it? Did you search for the underlying music? Did you comment and what was your comment’s sentiment? Did you look for additional content by that creator? These are important feedback loops. TikTok is getting a much better idea of your true interests.

Lesson Learned #4: Your UX should be algorithm and feedback-friendly. Most aren’t. This might be controversial, but…stop bombarding your users with content. Slow down and get your user to focus. Be more engaging at a deeper level. You are adding friction for the user, but your analytics will be better. Minimally, this is worth a simple A/B test.

Lesson Learned #5: Don’t rely on your user to give you explicit feedback (say, by “liking” it). It’s meaningless. Instead learn to analyze their behaviors. Analyze what your users “do”, not what they “say”

If you tend to engage with dance videos and you were shown what my Video AI thinks is a dance video and you did in fact engage, that helps my algorithm recognize the true content of the video.

Lesson Learned #6: Whenever possible get creative and have your algorithms automatically re-learn. I always advocate that all algorithms should re-train after collecting human feedback. This usually means collecting batches of feedback before you can re-train. With reinforcement learning, TikTok’s algos are re-learning in real-time.

Differentiator D: No Explicit Profiling

Tiktok doesn’t have the typical profiling step like other social media platforms when you create your user account.

Wait a sec…with Tiktok you don’t even need a login. You can just start watching videos and it will start applying its algorithm. They know that if they can show value QUICKLY you might really want to provide your personal information, eventually. Brill-yunt!

TikTok doesn’t ask you up-front to add your email contacts or tell them your demographics or ask you about your interests. Some users might create an account, others just start watching content. Your tastes can change daily or hourly and the algorithm can respond to that feedback quickly, showing you different content.

Lesson Learned #7: Don’t badger your users to disclose information that you can just as easily get from analyzing their behaviors. Again, worry about what your users “do”, not what they “say”.

Differentiator E: Explore vs Exploit

Facebook Groups scare me a little bit, but not for the reasons you may think. Twenty years ago fringe groups didn’t have an easy outlet to voice their views. I’m a free speech advocate and I’m glad our society has evolved where non-traditional groups have a place to congregate. It doesn’t bother me that these Groups become echo chambers where members have a hard time having their viewpoints challenged. Although a better algorithm could help these Groups weight “explore” a little higher than “exploit”. That sounds like an opportunity for the next social media outlet.

When I hear about something like “Pizzagate” or “Flat Earthers” my inquisitive nature makes me want to learn why people think like they do. Mainstream media is focused on debunking these conspiracy theories and resorting to ad hominem attacks. I want to be empathetic of others’ beliefs, so the best place to do that is on the Facebook Group for these fringe groups. But if I do that, Facebook will label me as member of the group and now, suddenly, every post, viewpoint, and opinion I’m shown is overwhelmingly skewed to these alt-viewpoints. But a one-time inquiry is not my interest, at most it is a passing curiosity. When a “learning” algorithm (instead of a social graph) sees that I’m (temporarily) interested in Pizzagate it will exploit that fact and show me more of the same (like Facebook Groups would) but after it sees I’m not truly interested it will explore my other likes and forget about my older proclivities. The social graph doesn’t move quite as fast.

Lesson Learned #8: Build your algorithms to leverage “exploit vs explore”. Exploit what you know about your customers, but occasionally explore their other interests…you may land on some valuable information.

How you can apply TikTok’s methods to your business

Certainly, better user engagement. “What do you appear to be interested in? Then, let me show you more of that”…is the basic engagement model. This is easier to “cold start” and it feels less intrusive. This engagement model works well in many other use cases.

Let’s say you develop an internal enterprise application. User engagement is critical. You should know what your users are interested in and then adjust your application accordingly. User feedback loops can give you valuable insights into what your users think about your application.

Apply personalization, then scale. This is a compelling business model for any application. Build what your users actually want.

Consider new and novel ways to integrate “exploit vs explore” in your applications and experiences. Your application should be learning.

How can we help you today?

Are you convinced your marketing analytics can drive your revenue goals? Would you like to try something new and compelling like the TikTok models?

I work for the Microsoft Technology Center (MTC) as a data scientist and Design Thinker. Our mandate is to be the Trusted Advisor for our customers. We know that Digital Transformation is more about data monetization and creating new product lines than it is about deploying a new tech stack. The technology is easy, what’s hard is understanding the patterns that actually work.

Reinforcement learning (exploit vs explore) is probably the most misunderstood branch of machine learning. It’s difficult to code and there aren’t many frameworks to simplify the code. We need our preferences to be learned not from a social graph but from an algorithm that is rewarded when it successfully learns our preferences. “Exploit vs explore” is a big component of reinforcement learning. Our Azure Machine Learning Service helps you build and deploy these algorithms faster than ever.

Give us 3 days. We like to start with a short Design Thinking session to determine your most pressing business problems and some novel solutions we can build with you. Then we build a rapid prototype, together. We show you just how easy it is using our modern patterns and the Azure Cloud. Yes, we can actually build you a prototype customer engagement model that is close to how TikTok does it in just a few days. We want our customers to walk away saying, “Yeah, this process will work for us!” or “this seems risky for us right now”.

In 2021 IT is more about taking “small bets” with calculated risks versus large capital projects with high failure rates. Building a new approach to user engagement is an example of how we do things differently. The Microsoft Cloud is simply an enabler to make small bets, faster.

Data analytics, like building a TikTok model, is perfect for a Rapid Prototype Workshop. And we offer it as a service to our Microsoft customers. Does that sound like a good investment of your time? Let’s partner on your future. Contact me or your account team today.

Are you convinced your data or cloud project will be a success?

Most companies aren’t. I have lots of experience with these projects. I speak at conferences, host hackathon events, and am a prolific open source contributor. I love helping companies with Data problems. If that sounds like someone you can trust, contact me.

Thanks for reading. If you found this interesting please subscribe to my blog.

Dave Wentzel 2021-05-16 CONTENT
Digital Transformation Data Science

Differentiator A: algorithms vs social graphs