Why AI Giants Are Turning to Small Language Models

[00:00:00] What is dedication? People ask, how your children learn how to ride a bike and you didn't. I just created an environment where they taught themselves and all I had to do was be there. That's dedication. Visit fatherhood.gov to hear more.

[00:00:12] Brought to you by the U.S. Department of Health and Human Services and the Ad Council. Welcome to Tech News Briefing. It's Tuesday, July 9th. I'm Zoe Thomas for The Wall Street Journal. Personalized schemes to dupe internet users are on the rise.

[00:00:29] The trouble is, it is harder than ever to know whether that person showing up in your messages is real or not. There are ways to protect yourself though and some tech companies are beginning to help too.

[00:00:41] And then, companies are embracing the benefits of less powerful artificial intelligence models. Our reporter Tom Dutton will tell us why for AI giants, smaller is sometimes better. But first, in an era of fakes online, it's getting harder to know who to trust. AI can replicate photos and voices.

[00:01:08] It can help bad actors write more convincing messages. And social media can help scammers find information on their targets. U.S. consumers lost $1.1 billion in romance scams last year according to the Federal Trade Commission. While business scams cost people $752 million. So, how do you stay safe online?

[00:01:31] Cordelia James from our personal tech team is here to tell us. Cordelia, can you give us a recent example of how somebody has been duped online?

[00:01:40] Earlier this year in Hong Kong, an employee ended up handing over $25.5 million to an attacker who used AI to pose as the company's chief financial officer. It appeared as though these were legitimate people that he knew in a meeting.

[00:01:56] But it was just AI-generated deepfakes that was impersonating his colleagues. It's really complex, whether it's deepfakes to that level or even just maybe a photo that's been AI-generated to look like someone. It can be really difficult for people to tell the difference.

[00:02:12] All right, so let's talk about some ways you can protect yourself. What are step security professionals say you should do to check if somebody is real? One of the ways that security experts are saying that you can check is by asking a lot of questions.

[00:02:28] Because sometimes these are bots on the other end, sometimes these are like real people. But once you ask a bunch of questions, you might be able to find some holes in terms of what they're saying. Also having a code word.

[00:02:40] So sometimes these bad actors might want to impersonate like a family member or a friend or someone that you know. And so you want to have – go ahead and prearrange a code word with that person, that loved one.

[00:02:51] If you want to send somebody money for used furniture on Facebook Marketplace or someone reselling concert tickets, what is the safest way to pay? You want to make sure you're using a goods and services transaction rather than a peer-to-peer transaction. So things like Apple Cash.

[00:03:06] Apple Cash does not have a goods and services option. So if the person is asking you to send it through Apple Cash, that's a red flag. Use something such as like PayPal or Venmo that has a goods and services option for like transactions with businesses.

[00:03:20] So that way if it turns out that the other person on the other end isn't legitimate, then you can at least get that money back. All right. Well, that's some of the steps that we can take as individuals.

[00:03:30] What tools are the tech companies adding to help protect us from scams? So Google is working on building a tool for Android phone users that essentially detects when someone is saying something that might sound off or like scammy.

[00:03:44] So if someone – like you're talking to someone on the phone and they were to be like, oh, I need your password in order to access your bank account, you'd get a notification that says that this might be a scam.

[00:03:56] And so it says it in real time because when something's urgent, it can be easy to kind of overlook those red flags. So that's in the works. It's not here yet, but that's something that Google is currently working on. Users are going to have to opt into this.

[00:04:10] And LinkedIn already has a feature that's very similar that already exists on its platform and it's on by default for US users.

[00:04:17] Essentially, if someone were to reach out to you, someone that you are not connected with, and they send you some kind of message that seems kind of scammy, it automatically – it's marked as spam and you don't even have to worry about it.

[00:04:30] What about for dating apps where people can often be lured into tricks? Bumble, for one, has a deception detector tool that it launched earlier this year. And so essentially that's one of these tools that kind of works on the back end.

[00:04:45] So it doesn't even necessarily require a user to do anything. It screens different profiles and assesses the authenticity of them. And so far it seems like it's getting good results. Bumble said that two months after launch, member reports of scam accounts have fallen by 45%.

[00:05:03] All right, that's Cordelia James from our personal tech team. Coming up, less is sometimes more. We'll tell you why tech giants and startups are thinking smaller when it comes to AI models. That's after the break. What is dedication?

[00:05:25] People ask, how your children learn how to ride a bike and you didn't. I just created an environment where they taught themselves and all I had to do was be there. That's dedication. Visit fatherhood.gov to hear more.

[00:05:35] Brought to you by the U.S. Department of Health and Human Services and the Ad Council. Giant models trained on mountains of data characterized the start of the artificial intelligence arms race. Now, tech giants and startups are thinking smaller.

[00:05:55] They're building AI models trained on less data and often designed for specific tasks. Here to tell us more about the reason behind the shift is our reporter Tom Dutton.

[00:06:06] So, Tom, listeners have likely heard the term large language model referring to the deep learning algorithms that train AI programs like ChatGPT and BARD. But we're talking about a category of AI software called small or medium language models. What are these models used for?

[00:06:23] Right now they're being trained for two things. Either they're on devices so they're small enough that they can fit on a computer and don't need to run off site on, you know, giant supercomputers and warehouses miles and miles away. And they're also often trained on specific data sets.

[00:06:40] So maybe a company's internal emails or their own data or just legal documents. So it's more specific to tasks rather than just a be all end all know it all model that the large language models typify.

[00:06:53] These models need less data, but how much less data are we talking about? Oh, significantly less. Open AI GPT-4, which is like the standard right now of these large models. They have more than one trillion parameters.

[00:07:06] It's estimated parameters are like the different knobs or variables that get tuned and tweaked to turn out the answers that large language models spit out. So that's in like the trillions level.

[00:07:16] At these smaller levels, we're talking 10 billion parameters or often even less, sometimes even less than 1 billion parameters. So one one hundredth of the size at times. What concerns have prompted companies to work on these smaller models? That the big ones are expensive.

[00:07:30] When you are dealing with a model that ingested that much data that has to basically run every query through all of these different parameters, it's really expensive to run. It's just a simple matter of cost. A lot of the times. How significant is the cost difference?

[00:07:44] Well, just the development cost, which is a huge part of these things. We're talking about, again, less than one one hundredth of the total amount. So GPT-4 costs one hundred million dollars to develop, to create something of that size.

[00:07:56] And a lot of these small and medium models, they can be done for less than 10 million. I just want to take a moment to say that News Corp, the owner of The Wall Street Journal, has a content licensing partnership with OpenAI, the maker of GPT-4.

[00:08:09] But Tom, what companies are producing these smaller AI models? It's all types of AI model developers have basically jumped aboard this small-medium train in the last couple of months.

[00:08:22] So everything from Google, which has developed large models, Gemini, at par with Microsoft and OpenAI in terms of the size of their models. There are companies like Mistral, which is an AI startup, Cohere, another AI startup.

[00:08:36] All of these guys are building models that are at that small-medium size. And so basically everyone in the game right now. Where might people who don't work in the tech world encounter these smaller models? Like all of this technology, it's still fairly experimental.

[00:08:51] So you're not seeing a ton of it out in the wild. But if you work for a business and they are trying out generative AI, there are a number of companies that we talk to that have said,

[00:09:02] actually, we think we can do a lot of the heavy lifting that we need to do with smaller models. So this one executive that I spoke to at Experian, the credit agency, they are building chatbots that can do things like credit checking or financial advice.

[00:09:18] And that's using smaller models. And how important are these small AI models to the makers of laptops and phones like Apple and Microsoft? Because these models are small, they can fit on a specific device.

[00:09:30] So something like GPT-4, so unwieldy, it needs to run on supercomputers in the cloud. Whereas these small media models, they are small enough to fit on even a phone or a laptop.

[00:09:41] A couple of months ago, Microsoft rolled out a new line of what they're calling AI laptops, which have a ton of neat little AI features that can create drawings and search things on your computer.

[00:09:52] All of those things require AI models, but they do not run in the cloud. It's all done on the device. And that wouldn't be possible with these large language models.

[00:10:01] So if you are trying to build something that's quicker, that is cheaper, and then arguably more secure, because it's not transferring a bunch of data out into the cloud, it needs to run on the device and it needs to be small.

[00:10:12] So that's another important use case for this. So what does this shift tell us then about the AI race and maybe the future of AI development?

[00:10:20] There were a few people I talked to that said the reason we're having this small to medium model moment is because the frontier of the large models is fairly stagnant right now. So GPT-4 came out, was released, the training was finished mid-2022.

[00:10:37] So we're talking about two years in which the frontier of large language models really hasn't moved all that much, not in a substantial significant way.

[00:10:46] And so at a moment of kind of stasis at the frontier of the technology, the small and medium models have gotten a lot more attention. And so we're seeing on one hand the need to create a business. That's one engine behind this moment.

[00:10:58] But the other thing is right now at the most impressive level, we're not seeing a ton of breakthroughs yet. That was our reporter Tom Dutton. And that's it for Tech News Briefing. Today's show was produced by Julie Chang with supervising producer Catherine Millsop and deputy editor Chris Zinsley.

[00:11:15] I'm Zoe Thomas for The Wall Street Journal. We'll be back this afternoon with TNB Tech Minute. Thanks for listening.