OpenAI Has a Tool to Detect When Students Use ChatGPT to Cheat

[00:00:00] If only life had a remote control, you could pause or rewind. Well, life doesn't always give you time to change the outcome, but prediabetes does. Take the one-minute risk test today at doihaveprediabetes.org. Brought to you by the Ad Council and its prediabetes awareness partners.

[00:00:18] Welcome to Tech News Briefing. It's Thursday, August 8th. I'm Zoe Thomas for The Wall Street Journal. At least three once-hot artificial intelligence startups have been rescued via a type of deal that many in the tech industry say are acquisitions in everything but name.

[00:00:35] We'll tell you what's going on. And then, Teachers Want Help Detecting When Students Are Using ChatGPT to Cheat. OpenAI has built a tool that can do just that with 99.9% certainty. So why hasn't it released the tech yet? Our reporter Deepa Sitaraman will join us with that exclusive reporting.

[00:01:01] But first, AI startups raised billions of dollars last year, aiming to become winners in the latest tech-driven boom. But as we've talked about before, the tech is expensive to build. Many startups are discovering they don't have the resources and the runway to get there.

[00:01:19] Now, some are asking Silicon Valley's biggest companies to bail them out. Joining us now is our reporter, Berber Jin. Berber, these so-called acquihires, when a large company buys a startup mostly to hire its employees, have been common in Silicon Valley for a long time.

[00:01:35] But something slightly different is going on with these AI startups, right? Yes. So essentially, these big companies are doing acquihires but with a really big twist. Companies like Google and Microsoft and Amazon, they've each done a particular type of deal where,

[00:01:52] yes, they are trying to acquire the talent behind these startups, but they're also paying these really big licensing fees to get access to the startup's underlying AI technology and also buy out the early investors in these startups. So typically, acquihires are much cheaper than traditional acquisitions.

[00:02:13] But what you're seeing here is that big tech companies are essentially paying the price that would essentially be a traditional acquisition price, and they're getting almost everything they would get through a traditional acquisition. And the prices they're paying are so high is that investors are essentially getting

[00:02:31] paid back and sometimes at a premium from what they invested in. The startup investors are getting paid back. Exactly. Can you give us an example of this? Yeah. So last week, Google struck this type of deal with Character AI.

[00:02:45] So Character AI was an AI startup that was really hot out of the gate. They were building their own language models to power a consumer chatbot. Basically, they were creating avatars that people could chat with so you could chat with anyone ranging from Elon Musk to The Beatles.

[00:03:01] But they ran into this problem where they realized that building these models would just be way too expensive for them to do. And so what happened was that Google came knocking. Both of Character's co-founders, Noam Shazir and Daniel DeFratis,

[00:03:16] were at Google and actually left Google to start Character, and Google really wanted them back. So instead of just paying a flat acquisition price, they paid $2 billion in the form of a licensing fee. They would give them access to Character's technology.

[00:03:32] That came on top of a $500 million investment they had also recently made into Character. And they essentially paid the same acquisition price they would have paid if they were pursuing a straight acquisition. And what we've reported actually is that

[00:03:46] the two sides were actually considering a traditional acquisition at the same price, but they essentially decided that the antitrust risk was too high behind that type of deal. So they resorted to this creative workaround that has become increasingly common in the past year.

[00:04:01] So what's left of Character AI? Character AI still exists. They put in an interim CEO. They're looking for a new CEO. And some of the money that Google paid to Character is being sent directly to the bank account to help fund the new startup's growth.

[00:04:16] How are regulators reacting to this type of deal? They're looking at these deals closely. We've reported that the FTC is examining similar types of deals that Amazon and Microsoft have done with two other startups. What regulators are really concerned about now is whether the monopoly

[00:04:36] big tech companies have on vast swathes of the tech industry, whether that monopoly will grow in the era of artificial intelligence. All right, that was our reporter, Berber Jin. Coming up, OpenAI is trying to decide what to do with a tool that can reliably detect

[00:04:53] when chat GPT has been used to write an essay for a research paper. We'll tell you about the company's concerns after the break. Well, life doesn't always give you time to change the outcome, but prediabetes does. Take the one minute risk test today at doihaveprediabetes.org

[00:05:20] brought to you by the Ad Council and its prediabetes awareness partners. Concerns about students using AI programs like chat GPT to cheat on their assignments have existed since almost the moment the tool was released. A recent survey by the Center for Democracy and Technology, a technology policy non-profit,

[00:05:42] found 59% of middle and high school teachers were sure some students had used AI to help with schoolwork, up 17 points from the prior school year. OpenAI, chat GPT's maker, has a method to reliably detect when someone uses the AI chatbot to write an essay or a research paper.

[00:06:03] It's been ready to be released for about a year, according to people familiar with the matter and internal documents viewed by the Wall Street Journal. So why isn't it out yet? Here with more on that is our reporter Deepa Sitaraman.

[00:06:16] And just before we get started, we should note that Wall Street Journal owner News Corp. has a content licensing partnership with OpenAI. But Deepa, tell us how does this anti-cheating tool work and just how good is it at detecting when chat GPT has been used?

[00:06:31] To understand how it works, you kind of need to understand how large language models like the one that powers chat GPT in general work. It works by predicting what is the most likely snippet of text to come next.

[00:06:48] How this technology works is it changes how those snippets and fragments, which are technically called tokens, it changes how the tokens are selected. So it creates a pattern and that pattern can be legible to OpenAI as a watermark. It's not foolproof, just to be clear.

[00:07:11] You need a certain amount of text to make it work and it needs to be largely original. So if you ask chat GPT to spit out the Gettysburg Address, it's not really going to tell you anything.

[00:07:26] But if you're asking it to write a five-page essay about the Gettysburg Address, then there's more opportunity for chat GPT to make decisions about what word would come next. And that would allow the watermark to really shine through.

[00:07:45] Are there ways to cheat the anti-cheating tool, maybe get around this watermark system? Sure. You could put it through Google Translate. It can remove the watermark that way. There is something called a carrot attack where you can ask chat GPT to begin with

[00:08:02] to insert an emoji in every other word and then delete that emoji. The teams within the company that have been advocating for the release say, look like, yes, there are ways to get around this, but this technology under these circumstances can work.

[00:08:20] Meaning, you know, if you're a student cheating and you just take the output from chat GPT and deliver it, it is going to catch that. Your reporting shows OpenAI has been working on this project for roughly two years, and it's been ready for release for about a year.

[00:08:38] Walk us through the company's efforts to create it. So a lot of it comes from this one researcher named Scott Aronson. Scott pretty much created the mechanism for doing all of this two years ago. So it's been ready to go in that time.

[00:08:58] But part of what has given OpenAI pause is that they didn't really know how watermarking would affect what has really become their flagship product, chat GPT. So the debates start in early 2023 where they're writing down pro and con lists.

[00:09:21] They then commission two different studies, one just to understand how the general public might view these kinds of detection tools. And from that they learn that generally speaking people are aware that tools like watermarking or any kind of AI detection tool that there are going to be drawbacks.

[00:09:45] But as a whole, they prefer having these tools to not by a margin of four to one. But at the same time, OpenAI has commissioned a second study that looks at chat GPT users, about 2,500 of them.

[00:10:02] And they found that 29% of these users would use chat GPT less if it were watermarked and other chat bots or similar products were not. So that really gives the company pause because they don't want to scare off their users.

[00:10:25] What has OpenAI said publicly about why it hasn't released this tool? After our story published, they updated one of their blog posts and confirmed that they've been working on this kind of technology and they confirmed that it works well.

[00:10:39] But they said they were looking at alternatives because they wanted to fully explore the space and truly understand what the options were. Deepa, OpenAI does have a tool to determine whether an image was created using its text to image generator, Dolly 3.

[00:10:58] That was released for testing this past spring. Why was it OK to put that out? When I asked the company that, the perspective I got was that OpenAI as a whole believes that images could be potentially more damaging, especially in an election year.

[00:11:14] And so that became a priority. Should we expect that this anti-cheating tool from chat GPT is going to come out? And if so, when? The company hasn't said anything about that. They haven't committed to releasing it at all.

[00:11:28] What they've said is that they are going to very carefully explore the space, ensure that they've tested every option and thought through even the unproven technologies. And if in the end text watermarking is the best they've got, this is what they'll launch. That's our reporter Deepa Sitaraman.

[00:11:48] And that's it for Tech News Briefing. Today's show was produced by Julie Chang with supervising producer Catherine Millsop. I'm Zoe Thomas for The Wall Street Journal. We'll be back this afternoon with TNB Tech Minute. Thanks for listening.