‘Distillation’ Is AI’s New Buzzword—and a Scary One for AI Companies

[00:00:03] Welcome to Tech News Briefing. It's Thursday, February 6th. I'm Pierre Bien-Aimé for The Wall Street Journal. Amazon may be number one in online retail, but its bricks and mortar stores aren't doing quite as well. We'll talk to real estate reporter Kate King about the company's decision to shut some of its storefronts. And DeepSeek, the buzzy artificial intelligence company out of China, may have trained one of its models by having it ask questions of its American counterparts.

[00:00:30] Tech reporter Miles Krupa says so-called distillation has some investors spooked. First, after a decade-long experiment with real-life stores, Amazon is pulling back. In recent years, the e-commerce company has closed all kinds of shops. Its portfolio of Amazon Go convenience stores, where you can just grab an item and walk out and you'll be charged electronically, has shrunk by about half since early 2023 to 16 stores in four states.

[00:01:00] Sales at its stores, including Whole Foods Market, have grown annually, topping $5.2 billion in the third quarter of 2024. But that compares with about $61.4 billion at Amazon's online store. Kate King covers real estate for The Wall Street Journal, and she joins me now. Kate, Amazon Go stores, they do seem pretty innovative. There's no traditional checkout counter, so there's no line, and no one likes a line. They use these cameras and sensors to track your purchases. Are customers just not into it?

[00:01:29] I'm sure some are, for sure. I did interview one person who said that it was convenient in the moment to go into the store and walk out without waiting in line, but it didn't necessarily change his life. And he said that while maybe he saves a few seconds not having to pull out his credit card, paying by credit card is so quick now, especially with Apple Pay, which allows you to quickly just scan with your phone at the checkout aisle. So for him, it wasn't a huge game changer. And it's not just Amazon Go stores, right?

[00:01:57] Amazon's tried a bunch of different bricks-and-mortar concepts. That includes, of course, their bookstores, which they started with about 10 years ago in Seattle, their four-star locations, which were stores that were stocked with their best-selling items from the website. They had their Amazon Style stores, which were basically fashion stores with a lot of technology. And, of course, they have their grocery portfolio. And the other types of bricks-and-mortar stores, the bookstores, the four-star, the fashion stores, Amazon has closed and kind of abandoned those projects.

[00:02:27] What they're really focusing now on is their grocery portfolio. Namely, Whole Foods, which it owns. Yes. Amazon bought Whole Foods Market in 2017 for over $13 billion, and it's doing really well. Sales are growing. It's expanded the locations of Whole Foods. And Amazon also has launched a concept called Amazon Fresh, which is a high-tech grocery store. It sells a lot of mass-market items at maybe more affordable prices than you'd find at Whole Foods, and it uses technology.

[00:02:57] And here, Amazon's really experimented and made some major changes. When it first opened the Amazon Fresh stores, they had the Just Walkout technology that you see at Amazon Go convenience stores. But this didn't really work, and so they paused expansion of Amazon Fresh. They removed the Just Walkout technology. And now they have shopping carts that are equipped with scanners. So when you pick up an item off the shelf, you scan it, you put it into the cart.

[00:03:20] And even some of the harshest critics that I interviewed do say that the Amazon Fresh redesign has worked. But at the same time, it remains to be seen how successful it will be in terms of growth and customer adoption. So is Amazon scrapping its Just Walkout tech? Amazon is still using its Just Walkout technology. While it might be walking away or paring down its focus on the Amazon Go stores, it's not reducing its emphasis on Just Walkout technology.

[00:03:50] Amazon licenses this technology to over 200 third-party retailers. That includes colleges and universities, hospitals, those convenience-type stores in airports, stadiums. And someone I interviewed who was really critical of Amazon's physical locations said Amazon Go, while it didn't really work as a standalone store, it wasn't a waste of time or money because it allowed Amazon to really research, develop, and refine this technology,

[00:04:18] which it's now selling to other retailers. That was our reporter, Kate King. Coming up, training cutting-edge AI is massively expensive. But if the next best AI models are super cheap, where does that leave the pioneers and their investors? That's after the break.

[00:04:45] The Chinese company DeepSeq turned heads last week with a powerful problem-solving AI model, R1. And it was a lot cheaper to make than models made by deep-pocketed American companies. But could DeepSeq have piggybacked on other artificial intelligence? That's the idea behind distillation. By drawing on the results of others' work, distillation can create a model that's almost as good quickly and more cheaply.

[00:05:10] OpenAI, the maker of ChatGPT, says it's seen indications that DeepSeq distilled from the AI models that power ChatGPT. DeepSeq didn't respond to emails seeking comment. But the company has said it used distillation on open-source AIs released by Meta and Alibaba in the past. Myles Krupa covers tech for The Wall Street Journal, and he joins me now to talk about this. Myles, how does distillation work? It's basically taking the outputs of one AI model

[00:05:38] and using that to sort of teach another one how to solve problems and ask questions. And so it's actually a technique that's been around for a while in AI. We've seen examples of distillation in the past from big US AI labs, but DeepSeq certainly is making everybody much more aware of the practice now. Ali Goetze, who runs this company called Databricks that does a lot of AI work,

[00:06:08] he compared distillation to basically getting to ask any question you want of Einstein and becoming almost as knowledgeable as he is in physics. That's what the process looks like for these AI models. And we should quickly note that News Corp, owner of The Wall Street Journal, has a content licensing partnership with OpenAI. Myles, we reported that OpenAI is probing whether DeepSeq used its models to train its chatbot using distillation. That's right.

[00:06:37] OpenAI began as this company that wanted to democratize AI research and make it available for all as part of its mission. And over time has gotten a lot more closed in terms of protecting its IP. It stopped sharing as much research. And that's true of a lot of AI labs. But OpenAI in particular is very sensitive to its trade secrets, or it's the technology that it spent billions of dollars to create

[00:07:04] being improperly used in competing AI, as DeepSeq appears to have done, according to OpenAI. And how effective has distillation proven to be so far from what we've seen? It appears to be really effective. It's an easy way for smaller developers in particular to recreate some of the capabilities

[00:07:26] of much larger models in a much more cost-efficient manner and doing it in a way that also produces AI models that themselves are a lot smaller and more cost-efficient. So if distillation works so well, what effects could that have on the hyperscalers, the companies that are pouring vast sums into AI and their investors? Yeah, it just puts even more pressure on them to stay at the cutting edge to justify

[00:07:56] the billions of dollars they're spending. You know, it just raises the question of what's the point if everything can be replicated so quickly and so cheaply? I mean, it was a matter of months between when OpenAI announced this model called O1. That was a very powerful model. It's something called reasoning. And then a few months later, DeepSeek came out with its own version, R1, that was basically just as good.

[00:08:23] And so when investors are looking for justification for the likes of Google, Meta, OpenAI spending billions on chips and data centers and salaries for researchers to stay on the cutting edge of AI, it just raises questions about whether all of that is really necessary. And in fact, overall, what could this do in the arms race in terms of changing the costs of AI? Overall, everything is coming down in cost.

[00:08:51] A lot of people are talking about the commodification of the AI models that power things like chat GPT, that basically all of the sort of secrets are going to become secret no longer. OpenAI and open source developers like DeepSeek will only sort of increase the pace of replication and innovation. And that will all help to drive down costs in the underlying technology.

[00:09:18] That's where a lot of people see the puck traveling. OpenAI's terms of service forbid using its AI to develop rival products the way that OpenAI alleges DeepSeek did. But can OpenAI really do anything to enforce that? Yeah, it's hard. They said that they've closed down an account they believe was tied to DeepSeek. So that's sort of the only real mechanism. It's like a game of whack-a-mole, right? Because if you close down one account, somebody can always open up another one.

[00:09:45] But speaking to people about this, it sounds like the way companies like OpenAI and Google provide their AI models through what's called an API, it's very hard to prevent up front the sort of misuse that OpenAI is alleging. That was our reporter, Myles Krupa. And that's it for Tech News Briefing. Today's show was produced by Julie Chang with supervising producer Catherine Millsop. I'm Pierre Bien-Aimé for The Wall Street Journal. We'll be back this afternoon with TNB Tech Minute. Thanks for listening.