INTRODUCTION
TikTok is a wildly popular social media application with over 1 billion users. With its ever-growing popularity, it has also become a source of dermatological information and misinformation for the public. A call has been made to encourage dermatologists to join the platform to combat the spread of misinformation. The dissemination of dermatological information on TikTok is important and needs to be studied. Such studies published to date rarely account for TikTok's content algorithm and how it will impact their results. The information currently published on TikTok's algorithm reveals that it caters videos towards each user, based on perceived viewing preferences. In this commentary, we propose various mechanisms by which the features of the algorithm may bias data collection, leading to results that lack objectivity, reproducibility, and reliability. We suggest authors acknowledge how the nature of TikTok's algorithm can lead to variability in results. Currently, we do not believe there is an effective method to obtain representative, reliable, and reproducible data regarding dermatology content on TikTok.
The social media app TikTok has amassed over 1.6 billion users since its inception in 2016 and boasts over 1.7 billion monthly active users this year alone.1 In 2022, it generated $9.4 billion in revenue.1 To date, public academic search engines report over 240,000 results with the keyword "TikTok." Many dermatologists, dermatology residents, and medical students have joined the platform and posted content regularly. Understanding TikTok and its value in dermatology is important, especially since medical misinformation is exceedingly common on the app. In recent years, a call was made for medical dermatologists to join social media to combat dermatological misinformation.2 With its ever-increasing popularity, research is needed to assess information disseminated on the application. Numerous studies have been published analyzing various dermatological concepts on TikTok.3-5 However, investigators rarely consider the unpredictable nature of TikTok’s algorithm, and how it may produce unrepresentative, inconsistent, and unreliable results.
Although much of the algorithm remains elusive, a leaked document provides some information.6 Additionally, the company has released reports on how it uses data, mainly covered by news outlets. TikTok reports that content recommendations are based on a variety of factors including user interactions with content (likes, shares, comments), content previously created by the viewer, and device data.6 Accounts the user follows and watch time are utilized as well.6 The algorithm also predicts what type of content a user will like, even before the user indicates with the previous specifications that they do. The app presents users with videos it believes they might enjoy and then gauges their responses.7 In this way, the TikTok algorithm manipulates the user's experience from the instant they open the app.
A survey of dermatological-based studies focusing on TikTok revealed various ways of measuring data. Some study methods focused on the first "X" number of videos under a hashtag, others analyzed "top" videos by searching a specific term, and some analyzed the most popular videos under a hashtag but did not specify what metric they used to determine which videos were most popular. Notably, the top "X" amount of videos under a hashtag are not sorted according to popularity; this is made evident by the random variation in number of views, likes, and shares from video to video. There is no consensus about the best way to obtain and sort data on the app. Additionally, many studies do not account for algorithmic intervention in which content is served to the investigator, or that the act of data collection itself may be biasing the results by altering which content is subsequently presented. For example: if an investigator spends longer amounts of time analyzing videos containing misinformed content, they are likely to be served more similar content by the app, skewing the data toward misinformation. The moment a user opens the app, TikTok caters videos to the user. Interestingly, even if a user is not logged into their account, TikTok will still collect data on the user.8
Inspired by previous study methods, we investigated variation in user experiences due to the algorithm by comparing in-app search results between users. It appears the search categories "top," "videos," and "shop" differ between users. The categories "users," "sounds," and "hashtag" seem to be the same from user to user. Although not much information is available about how content under "users," "sounds," and "hashtag" is generated, TikTok states that, "the hashtag page displays the videos that started the trend first, and then other popular videos relevant to the trending hashtag."9 After our informal investigation,
