Young people are online for many reasons: for school, social activities, and for playing and being creative. One of the fastest-growing creative play platforms today is Roblox, a massive multiplayer online game (MMOG) that allows users to develop their own games, play games that other users have made, and connect with friends to chat and play together.
Roblox has grown tremendously in the past year, in part as a result of the coronavirus pandemic and ensuing school closures and stay-at-home orders. Player spending more than doubled from September 2019 to September 2020, rising from $44 million to over $94 million per month1 and the number of monthly active users passed 150 million in July of 2020.2
More than a game, Roblox has become a valuable tool in the new distributed world, allowing kids to keep in touch and have safe, socially-distanced interactions.
Joel Silk, Senior Director of Content Moderation at the platform, understands the danger that a badly-moderated system would pose, and takes his responsibility to monitor and manage the content on the platform very seriously.
“What really drives me about brand safety is being able to provide a safe digital neighborhood for my kids to explore and pass time with friends,” he noted. “And I want that neighborhood to be as safe and comfortable as mine when I was a kid.”
One of the complexities of managing content on the Roblox platform is due to the sheer volume of user-generated content (UGC) that is created on the platform every day. Additionally, as Joel noted, online usage in general exploded too quickly for clear behavioral expectations to be set and communicated.
“We inherently know and understand how to act in certain settings, right? We know what is or is not appropriate in an office, as opposed to a church or museum or a playground. But we haven't yet figured out as a society that the digital equivalent of that.”
Listen and subscribe to the Brand Safety Exchange podcast via your favorite podcast service, or check out the interview video and transcript below.
Listen on Apple Podcasts
Listen on Google Podcasts
Listen on Spotify
The following interview transcript has been edited to make it more concise.
Tiffany Xingyu Wang
Hi everyone, welcome to Brand Safety Exchange where you will hear unique perspectives from experts and leaders within the world of brand and user safety. Brand Safety Exchange is hosted by Oasis Consortium, a think tank to unite stakeholders and create actionable, measurable standards for the protection of online brand safety and I'm Tiffany Wang, the GM of Oasis Consortium. Today, I'm very honored to have my friend Joel Silk, Senior Director of Content Moderation from Roblox. Welcome, Joel!
Joel Silk
Hey Tiffany, thanks for having me.
Tiffany
So, before we start, tell me about yourself and Roblox, and your journey to becoming the Senior Director of Content Moderation at Roblox.
Joel Silk
Sure. So I have been with Roblox for about a year now in this role. Prior to that, I was a security product manager for a tech company that did data backup and storage. And then for the two decades before that, I was an Air Force intelligence officer. I actually found those skills transferred very well into my current role. So let's talk about Roblox. Roblox’s mission is to bring the world together through play. And we have a really strong community of players, game developers, and creators who work together to build and play these rich immersive experiences in what we call our digital map metaverse. As the Senior Director for Moderation, I lead a global team of dedicated moderators, and our trust and safety organization and we ensure that our entire community can interact in a safe and civil environment. To do this, we use a team of approximately 1600 agents around the globe, and they protect our users. We monitor for safety and detect inappropriate content 24/7/365. We use a combination of machine scanning and human moderation. And this includes 100% of human review on user-generated content so things like uploaded images or videos or audio files for the games that are being created, 3D models, that kind of thing. We also employ cutting-edge AI and machine learning to assist. And when and where it makes any sense to do so. We also constantly monitor world events, current slang, and vernacular. Our largest demographic of users is teens, 9 to 12 or 13, so really tweens almost. And as you can imagine, slang terms and teen vernacular move very quickly so we have folks dedicated to trying to stay on top of all that. And of course, internet memes as they come out and manifest on our platform, we want to make sure that the ones that violate our community rules are dispatched quickly, so we have a team that does that. When it comes to brand safety specifically, my role is really twofold and it's two sides of the same coin. First, I help to ensure that our special events and partnerships are in-line with our company values, and that the content surrounding them stays within our community rules. And then the flip side of that is I ensure that our partners and collaborators feel safe trusting their brand with us by providing a platform free of damaging content.
Tiffany
Interesting and what is the latest number, 160 million users on the platform globally. Wow. And there's a lot to unpack here, I heard you say having humans in the loop for keeping online trust and safety, your crazy background as an intelligence officer in the Air Force prior to this role, and now you are in the private sector to keep the team safe. As a general rule, we start this whole podcast with a rapid-fire round I called Oasis Refresher. So, 30 seconds per question, we’ll run through those to set the foundation for the rest of the podcast. So, why do you think the topic of brand safety is of growing importance today?
Joel
So I think in the past, brand association with troubling content was clearly a problem but there was a little bit of randomness to it. The odds that your intended target audience saw random bad content associated with your brand was relatively low, but I think today, in the world of precision-targeted marketing, the saturation of intended target, as opposed to bystander random viewer is much higher rate so each instance of your brand being associated with bad content hurts your brand much more than it ever used to because it is so targeted. Right. And I will tell you when I retired from the military and joined the private sector, I never again thought I would talk about precision targeting and collateral damage so thank you for bringing back history.
Tiffany
I was about to say, it takes an officer to say precision targeting here. Alright second question, why are you personally invested in this topic?
Joel
So for me, you know, when I was a kid, my parents would kick me out of the house and send me off into the neighborhood, and they would say “don't come back until the streetlights come on.” I don't know that that's really possible these days. And so, what really drives me about brand safety is being able to provide a safe digital neighborhood, for my kids to go and explore and pass time with friends and do those things. And I want that place in that neighborhood to be as safe and comfortable as it was for me when I was a kid.
Tiffany
No, that's very true. We don't really have the guardrails in a virtual world. I think that's why we first talked about that and then the next thing we know we started Oasis Consortium together to set the guardrails for the virtual world. So the third question is that, if we didn't have Oasis, if we didn't have guardrails for the virtual world, what would happen?
Joel
So I think internet use and online communities really exploded before we had an opportunity as a society to establish clear behavioral expectations. In the physical world, we inherently know and understand how to act in certain settings, we know what's appropriate in an office building as opposed to a church or religious site or a museum or a playground, that kind of thing. We just kind of understand that, collectively. But we haven't yet figured out as a society the digital equivalent of that. And so without agreed-upon industry standards, I don't know that we'll ever really get to a place where it's commonly understood that hey you just don't do that here. I think additionally, platforms, not just users, are becoming more and more interconnected. And of course, it creates huge opportunities for businesses but that interconnectedness also means that we have to rely on each other, to ensure safety across multiple platforms. And without industry safety norms and standards, we really can't build the trust that's necessary to enable that level of interconnectedness.
Tiffany
Perfect! I love the mention of words like trust, safety, security, you just threw all those out there. So with that, Oasis has the five principles: Openness, Accountability, Security, Innovation, and Sustainability. Obviously with your background as an intelligence officer in your past career and now in the private sectors doing moderation for Roblox, I'm really curious to hear your perspectives about his term Security, because it has so many meanings and everybody from different perspectives could define it differently. And especially when it comes to cybersecurity, for the longest time people always think, “that's a nation-state issue” but now we are living it. How in your career has your definition of security evolved?
Joel
So, as you can imagine as a 20 year veteran of the intelligence community, security is something that I spent a lot of time thinking about and talking about. But as you mentioned, that term security means totally different things to different people. And even for myself over the course of my career, it's changed as I've been involved in different projects and different aspects. So harkening back to the lieutenant days, my first security focus was on personnel security, ensuring that folks had the right security clearances to do the work that they needed to do. Making sure that they got their background investigation paperwork done correctly and on time, that kind of thing. Then I did a little bit of work in physical security, so looking at alarm systems and the physical access controls, surveillance capabilities, making sure that we had coordinated responses anytime an alert was triggered; that kind of thing. And then I moved into operational security, and I worked to ensure that even our unclassified information was properly protected. I think in the private sector, the closest match to that would be like ensuring that product details don't leak before the product is released, right, so think about the iPhone 14 and making sure that those particulars stay secret until you're ready to release them. I think one of my favorite examples of operational security in the private sector is when you see the pictures of prototype vehicles out doing road tests, and they always have those weird painted lines on them to kind of obscure their shape; it’s a great example of operational security. So, having done all of that, it wasn't until the second half of my intel career that I got deep into cybersecurity. I did defensive cybersecurity, so firewalls, intrusion detection, threat intelligence, that kind of thing. I also did offensive cybersecurity, which I’ll leave to your imagination for that one. But as an intelligence officer, my main focus was on doing the intelligence collection and analysis from everything from mapping what adversary networks look like to understanding how their users and their administrators think and act, and really everything in between. All of that experience has taught me that when it comes to cybersecurity, you really have to operate with the assumption of breach. The concept of a completely secure system or network really just doesn't exist. And with a focused and determined attacker, it's just a matter of time. And whether it's a nation-state or, as we've seen a bored kid, it's just a matter of time. And so you have to rely on other tools in your toolbox to help maintain that security. And this is where the fundamentals of security really come into play, the concepts of least privileges and zero trust, robust infosec training programs, red teams. I think one of my favorites is only keeping the data that is absolutely critical for your operation, and then only keep it for as long as you need it. Don't hang on to it unless you absolutely need to; these are all things that are critical to ensuring security. And then, in turn, user and brand safety. And so, I just rambled on, and not until the last sentence did I mention the word safety. I think there's a linkage there that that is important to call out: How does security relate to safety? If you look at the definitions, security is focused on the protection of resources; personnel, organizations, information - that kind of thing and mainly protecting from external factors. Safety, on the other hand, is more the feeling you get when all of those external risk factors have been mitigated. You can kind of relax and feel safe. And so, in my current role on the moderation side, I'm responsible for the safety of our users, our creators, and our business partners. I really can't provide that feeling of safety until I'm convinced that we have taken care of those external factors that seek to do harm. And that's why security is one of the Oasis principles of brand safety and why it's so critical that we establish these industry standards and protocols together because it's a team sport, something that you can't do alone.
Tiffany
Yeah, I love that's very true even, even the term “brand safety” - it is already a combination of potentially two departments.
Joel
Right, absolutely.
Tiffany
It's very true, whenever I step into a cybersecurity community, this is a quite new notion to a lot of CISOs. Right. It’s become so important because people start to realize, exactly like you mentioned, that if you can create the notion of trust of your platforms, then you can drive more users and more user engagement and retention, and it directly translates to the business goals like revenue, etc. And I think the CISO is very well positioned to take that responsibility on.
Joel
Absolutely, I think we're just starting to see an awakening by consumers and they are coming to the realization that cool integrations and slick user interfaces aren’t enough to keep your users anymore. So if you can't make them feel safe, giving you their data, if they can't trust you, they're going to go elsewhere.
Tiffany
Yeah, it's really true. Trust and safety become a differentiator, a value proposition of platforms. Let's talk a little bit more about Roblox. I've been working with different platforms, and what is very unique about Roblox is that you are dealing with teens. What is so special about Roblox, specific to trust and safety. Why is it so important to your job? Please use this platform to tell the audience how important your role is for Roblox, and even for the whole of society.
Joel
Sure. As I mentioned before, we are creating this safe digital neighborhood for kids to go and explore and play. When you think about it, Roblox is really one of the last bastions of free play that exists. I don't let my kids out in the neighborhood without somebody going with them, right. And I want to make sure that we can create a place in the metaverse, in this digital environment, where kids can go and explore those boundaries and those social norms that we need to create. That’s one of the big initiatives for us, is we try really hard at Roblox to not focus so much on punishment and saying, “Hey you violated our rules and go take a timeout.” We're trying to shift to more of an educational response and say, “Hey, maybe that isn't the nicest way to respond, maybe there's a better way to say what you want to say.” And teaching those social norms at a young age, helping to understand what is acceptable and what is not acceptable in the digital environment. Yes, there's this layer of anonymity between my computer screen and yours, but that doesn't make it okay to act inappropriately. And that's something that we take very seriously, that responsibility of helping to teach our users and our communities moving forward.
Tiffany
Yeah, it's fascinating what you were saying. When COVID hit, a couple of my friends told me you should write an article about some investment trends you see. So I wrote five predictions and the last prediction was about the line between education and gaming getting more and more blurry. You said that you are actually teaching the teens and the kids how to behave in society, and I think COVID accelerated this trend. And we'll see kids probably spend more time on these platforms more than at a school, so you have crazy responsibility Joel, on your shoulders there.
Joel
It is, and fortunately, I have a great team. And we have the support of our executive leadership from the company and it's a great, great time to be a Roblox, so I'm really excited about what we're doing.
Tiffany
I have one last question, you told me that your US market is saturated with 160 million users around the globe. What's the percentage of that for the US and what is International?
Joel
It’s now about three-quarters of US 9 to 12-year-olds are on Roblox.
Tiffany
And then the rest is international. So, as the audience might imagine that different cultures, different languages, you try to have players from all those different backgrounds come together to play. And then you try to moderate the whole platform, and try to keep everybody safe. So tell us a little bit about the challenge of growing globally, from the perspective of moderation.
Joel
I think, as you can imagine, one of the biggest challenges for expanding a market into a global community comes language. At first blush, you may think like, Spanish versus English versus Korean versus Chinese, but we have difficulties even in English. Probably the most glaring example is in the UK, the word “fag” is used to describe a cigarette. Right. Well, here in the US, of course, that is discriminatory and that's just one example of how from a moderation policy and even when we get into the AI and machine learning tooling that we're developing is, how do we grab the context to know that hey, this phrase is okay in this language, country in culture; but it’s not okay in these other places. How do we segment the population into those geographic groups, and is that something we really want to do? Maybe somebody from the UK really wants to play with somebody from the US, we should encourage that, right, to make it a global community. And how do we make sure that we have the standard bar set at the appropriate levels so that folks are free to communicate and share ideas, but at the same time maintaining safety for everyone?
Tiffany
Wow. Hard job. That's why I'm so excited about having you be on the advisory board because you really see the challenge and you demonstrated some best practices in the industry. I think brand safety has been very led by a lot of brands, but I think the brands need to understand the constraints and the challenges you are dealing with. Only when both sides can come together around the table, and realize how hard it is, and really encourage all the platforms to set roadmaps and resources and the talents like yourself to lead the initiative, will brand safety ever happen. So really, on this note, thank you so much Joe for coming on board.
Joel
Thank you so much for having me today, it's been a pleasure.
Tiffany
Thank you, Joel.
1https://www.statista.com/statistics/1077442/roblox-player-spending/
2https://corp.roblox.com/2020/07/roblox-developers-expected-earn-250-million-2020-platform-now-150-million-monthly-active-users/#:~:text=Roblox%20also%20announced%20it%20now,immersive%20and%20massive%203D%20experiences.
