Snap Knew of Sextortion Risks, State Lawsuit Alleges

[00:00:18] [SPEAKER_02]: Welcome to Tech News Briefing, it's Thursday, October 3.

[00:00:23] [SPEAKER_02]: I'm Zoe Thomas for The Wall Street Journal.

[00:00:26] [SPEAKER_02]: A Silicon Valley pioneer says artificial intelligence will drive down the costs of services, ushering

[00:00:33] [SPEAKER_02]: in an age of abundance that can support a robust social safety net, if we let it.

[00:00:39] [SPEAKER_02]: We'll find out about Vinod Khosla's take on AI's potential impact.

[00:00:45] [SPEAKER_02]: And then, how do you warn users about the risks of sending explicit photos without scaring

[00:00:51] [SPEAKER_02]: them off your platform?

[00:00:53] [SPEAKER_02]: That's an issue Snap grappled with and failed to effectively address, according to an unredacted

[00:01:00] [SPEAKER_02]: version of a lawsuit brought against the company by New Mexico.

[00:01:04] [SPEAKER_02]: The state says Snap put growth over safety and failed to disclose design features that

[00:01:09] [SPEAKER_02]: make its platform a haven for abuse and sextortion.

[00:01:13] [SPEAKER_02]: Our reporter Jeff Horwitz joins us to talk about the case.

[00:01:20] [SPEAKER_02]: First up, venture capital pioneer Vinod Khosla is all in on AI.

[00:01:26] [SPEAKER_02]: Here's Khosla on The Wall Street Journal Pro's Executive Insights podcast, available on WSJ

[00:01:32] [SPEAKER_02]: Special Access.

[00:01:33] [SPEAKER_03]: Vinod Khosla, Executive Insights Podcast, Executive Insights Podcast, Executive Insights

[00:01:33] [SPEAKER_03]: Most expertise in the world, whether you're talking about structural engineers or oncologists,

[00:01:40] [SPEAKER_03]: mental health therapists or primary care doctors or journalists, teachers, that expertise will

[00:01:48] [SPEAKER_03]: be near free.

[00:01:49] [SPEAKER_02]: That voice you hear interjecting is Stephen Rosenbusch, one of our WSJ Pro Bureau chiefs.

[00:01:55] [SPEAKER_02]: He spoke with Khosla and Stephen is with us now.

[00:01:58] [SPEAKER_02]: So tell us more about Vinod Khosla.

[00:02:01] [SPEAKER_02]: Put his perspective on AI into context for us.

[00:02:05] [SPEAKER_04]: Vinod Khosla has been a force in Silicon Valley almost since it came into being.

[00:02:11] [SPEAKER_04]: He was the co-founder of Sun Microsystems in the early 1980s and he was an investor

[00:02:18] [SPEAKER_04]: at venture capital pioneer Kleiner Perkins in mid-1980s where he returned something on

[00:02:24] [SPEAKER_04]: the order of $10 billion to the firm.

[00:02:28] [SPEAKER_04]: And he's made a career out of taking massive all-in bets on new technology, new ideas

[00:02:37] [SPEAKER_04]: that sound improbable at the start.

[00:02:39] [SPEAKER_04]: He's willing to make these bets on areas where there's a high degree of failure but

[00:02:47] [SPEAKER_04]: the rewards of success are really, really outsized.

[00:02:51] [SPEAKER_04]: So if one thing doesn't work out, there are many, many other opportunities for him

[00:02:56] [SPEAKER_04]: to drive a huge gain that compensates for the many things that don't work out.

[00:03:02] [SPEAKER_04]: In this particular area, he is very, very confident though.

[00:03:08] [SPEAKER_02]: What is his take then on the pace of AI innovation?

[00:03:11] [SPEAKER_04]: He thinks that it's going to cause massive disruption.

[00:03:14] [SPEAKER_04]: About 10 years out, he thinks that AI will replace about 25 percent of today's current

[00:03:22] [SPEAKER_04]: labor.

[00:03:23] [SPEAKER_04]: At that level, we'll need a much, much broader social safety net.

[00:03:28] [SPEAKER_04]: But he argues that if we let AI essentially do its thing, that the productivity gains

[00:03:34] [SPEAKER_04]: from allowing AI to play a much bigger role in business and healthcare, in medical science,

[00:03:43] [SPEAKER_04]: in education will be so profound that productivity growth will begin to really accelerate and

[00:03:51] [SPEAKER_04]: we'll be able to afford a much broader and a much deeper social safety net than what

[00:03:57] [SPEAKER_04]: we have now or what's possible now.

[00:04:00] [SPEAKER_02]: Can you say a little bit more about that?

[00:04:01] [SPEAKER_02]: How does Koslis say this will drive down costs?

[00:04:05] [SPEAKER_04]: In his view, AI will take on the characteristics of what we consider an expert in today's world

[00:04:13] [SPEAKER_04]: and will come to rely on AI to provide not just productivity gains, but also advice,

[00:04:24] [SPEAKER_04]: services, judgment in areas from science to education to business.

[00:04:33] [SPEAKER_04]: This will allow us to reduce the cost of elder care.

[00:04:38] [SPEAKER_04]: It will allow us to reduce the cost of education because we'll have AI-based agents that will

[00:04:47] [SPEAKER_04]: be as good or even better than human teachers in the classroom in the not so distant future.

[00:04:57] [SPEAKER_04]: And that because the cost of labor in all of these areas will be driven down to something

[00:05:02] [SPEAKER_04]: close to zero and productivity will go up, we'll have an enormous amount of productivity

[00:05:10] [SPEAKER_04]: gains that can be reinvested elsewhere.

[00:05:14] [SPEAKER_02]: Is there anything that Koslis thinks could throw off this vision for how AI could sort

[00:05:20] [SPEAKER_02]: of fundamentally change our economy?

[00:05:22] [SPEAKER_04]: He thinks that what we do with AI is a political decision and a social decision more than a

[00:05:29] [SPEAKER_04]: technological decision.

[00:05:31] [SPEAKER_02]: All right, that was Stephen Rosenbusch.

[00:05:33] [SPEAKER_02]: He's a bureau chief for WSJ Pro.

[00:05:36] [SPEAKER_02]: Coming up, New Mexico claims Snap failed to warn users about sextortion risks on its platform.

[00:05:43] [SPEAKER_02]: We'll tell you about the state's case and Snap's response to the lawsuit after the break.

[00:05:58] [SPEAKER_02]: Snapchat, a social media app known for its vanishing messages, is popular with teens.

[00:06:04] [SPEAKER_02]: But the app's parent company, Snap, grappled internally with how to warn of sextortion

[00:06:11] [SPEAKER_02]: risks on its platform without striking fear in users and their parents.

[00:06:16] [SPEAKER_02]: That's according to an unredacted version of a lawsuit New Mexico filed against the

[00:06:21] [SPEAKER_02]: company in September.

[00:06:22] [SPEAKER_02]: Here to tell us more is our reporter Jeff Horwitz.

[00:06:25] [SPEAKER_02]: So Jeff, tell us more about what Snap allegedly knew about the risks of sextortion and how

[00:06:31] [SPEAKER_02]: it supposedly knew it.

[00:06:33] [SPEAKER_00]: Snap is a mixture of direct messaging with pictures and augmented reality features and

[00:06:39] [SPEAKER_00]: then a little bit of the regular social media feed that you're used to from Facebook and

[00:06:44] [SPEAKER_00]: other things.

[00:06:44] [SPEAKER_00]: And so it turns out because of its emphasis on person-to-person communication and because

[00:06:51] [SPEAKER_00]: of its emphasis on ephemeral communications, meaning that messages that literally disappear

[00:06:57] [SPEAKER_00]: and because of its young user base, that it was a particularly attractive forum for sextortion.

[00:07:04] [SPEAKER_00]: Basically, this would be someone convincing a person, frequently a minor, to share nude

[00:07:11] [SPEAKER_00]: pictures and then blackmailing them.

[00:07:14] [SPEAKER_00]: And this is something that every social media platform deals with.

[00:07:18] [SPEAKER_00]: But Snap was discovering that they were getting 10,000 reports a month and they were trying

[00:07:24] [SPEAKER_00]: to build up their defenses and also figure out what was going wrong and why their platform

[00:07:30] [SPEAKER_00]: seemed to be more vulnerable to this than others.

[00:07:34] [SPEAKER_02]: Despite the company's concerns about sextortion, Snap told The Wall Street Journal that sexting

[00:07:39] [SPEAKER_02]: accounts for a minuscule fraction of the messages exchanged on the platform.

[00:07:44] [SPEAKER_02]: But Jeff, what did Snap's internal reports say about how well the company was handling

[00:07:49] [SPEAKER_02]: this risk?

[00:07:50] [SPEAKER_00]: In terms of moderation and just responding, not great.

[00:07:55] [SPEAKER_00]: And I've written about stuff similarly with Meta with this.

[00:07:58] [SPEAKER_00]: They were pretty much failing all the time in a sample of cases when sextortion was reported

[00:08:04] [SPEAKER_00]: to them through the official Snap channels.

[00:08:06] [SPEAKER_00]: This is not unusual.

[00:08:08] [SPEAKER_00]: But so setting that aside, they were also looking at their product and trying to figure

[00:08:13] [SPEAKER_00]: out what about its product features might be making Snap a bit more of a home to people

[00:08:19] [SPEAKER_00]: who were bent on committing abuse and also why kids felt comfortable taking pictures

[00:08:27] [SPEAKER_00]: and sending them via Snap that they might not be willing to send elsewhere.

[00:08:32] [SPEAKER_02]: What did Snap find according to the documents cited in New Mexico's lawsuit?

[00:08:36] [SPEAKER_00]: So a few things.

[00:08:37] [SPEAKER_00]: Per New Mexico, the disappearing messages actually may have lulled users into a false

[00:08:43] [SPEAKER_00]: sense of confidence that this wasn't going to stick around and bite them.

[00:08:47] [SPEAKER_00]: And Snap did create a feature that alerted users when someone took a screenshot of their

[00:08:52] [SPEAKER_00]: phone.

[00:08:53] [SPEAKER_00]: Thing is, that doesn't always work depending on how that screenshot is recorded.

[00:08:57] [SPEAKER_00]: And also it's only after the fact, right?

[00:08:59] [SPEAKER_00]: So getting notified after someone already has compromise on you is not good.

[00:09:04] [SPEAKER_00]: So New Mexico is alleging basically that Snap failed to disclose this, that the product

[00:09:08] [SPEAKER_00]: features were sort of high risk and that the company didn't adequately tell teenagers and

[00:09:14] [SPEAKER_00]: their parents about these things.

[00:09:15] [SPEAKER_02]: What did New Mexico find when their investigators went and made a fake profile of a 14 year

[00:09:22] [SPEAKER_02]: old?

[00:09:22] [SPEAKER_00]: Yeah, so they found that commitments that are sort of what Snap had said publicly in

[00:09:27] [SPEAKER_00]: terms of how things worked and what safety features they had in place, that there were

[00:09:32] [SPEAKER_00]: at the very least significant exceptions to that.

[00:09:35] [SPEAKER_00]: For example, Snap was saying that this quick ad feature where they basically recommend

[00:09:40] [SPEAKER_00]: people that might be in your orbit to you to connect with, that it was in fact recommending

[00:09:46] [SPEAKER_00]: strangers and in fact some definitely inappropriate strangers to kids and to vice versa.

[00:09:53] [SPEAKER_00]: So it wasn't working.

[00:09:54] [SPEAKER_00]: And in fact, per New Mexico's discovery, Snap realized this in 2023, that in fact what they'd

[00:10:00] [SPEAKER_00]: been saying about their precautions, it wasn't working the way it was supposed to.

[00:10:05] [SPEAKER_00]: The company says, by the way, that it has worked to fix that, to address that and continues

[00:10:09] [SPEAKER_00]: to work on this stuff.

[00:10:11] [SPEAKER_02]: What has Snap said in particular about this lawsuit?

[00:10:14] [SPEAKER_00]: Snap has noted that the platform does have certain restrictions that are very much in

[00:10:23] [SPEAKER_00]: keeping with attempting to prioritize safety over growth.

[00:10:26] [SPEAKER_00]: For example, unlike Instagram or a lot of Twitter, you can't look and see everyone that someone's

[00:10:32] [SPEAKER_00]: connected to.

[00:10:33] [SPEAKER_00]: There are ways that you can, particularly if a user isn't trying very hard not to be

[00:10:39] [SPEAKER_00]: found.

[00:10:40] [SPEAKER_00]: There are ways that you can find people on there, but it's harder.

[00:10:44] [SPEAKER_00]: And Snap does view itself as being unfortunately the recipient of some of the bad behavior

[00:10:50] [SPEAKER_00]: and perhaps worse design features that are on other platforms.

[00:10:55] [SPEAKER_00]: Basically that by creating a place where kids feel comfortable and confident to some degree

[00:10:59] [SPEAKER_00]: that they end up being a venue of choice when people who are malicious are trying to get

[00:11:05] [SPEAKER_00]: kids to switch over to a different platform.

[00:11:07] [SPEAKER_02]: And where do things stand with the case?

[00:11:10] [SPEAKER_00]: I mean, this just got unredacted.

[00:11:12] [SPEAKER_00]: The case was filed last month, but this is the first time we were able to see what New

[00:11:16] [SPEAKER_00]: Mexico actually had from internal documents.

[00:11:19] [SPEAKER_00]: And a big question is whether these cases are designed in a way that's going to get

[00:11:24] [SPEAKER_00]: them past the traditional liability protections that platforms have.

[00:11:28] [SPEAKER_00]: Obviously, New Mexico is coming at this in a way that is intended to bypass that roadblock.

[00:11:36] [SPEAKER_02]: All right, that was our reporter Jeff Horwitz.

[00:11:39] [SPEAKER_02]: And that's it for Tech News Briefing.

[00:11:42] [SPEAKER_02]: Today's show was produced by Julie Chang with supervising producer Kathryn Millsop.

[00:11:46] [SPEAKER_02]: I'm Zoe Thomas for The Wall Street Journal.

[00:11:49] [SPEAKER_02]: We'll be back this afternoon with TNB Tech Minute.

[00:11:52] [SPEAKER_02]: Thanks for listening.