SN 1090: Black Hat - The Hidden Flaws in AI Security Nobody Saw Coming

[00:00:00] It's time for Security Now! And this is a very special episode. You're going to be glad you're here. We are at the Black Hat Conference in Las Vegas. Steve Gibson's here with questions from you, the audience. We're going to primarily talk about AI and security. And we've got some very special guests joining us. Security Now is next. Podcasts you love. From people you trust. This is TWiT.

[00:00:33] This is Security Now with Steve Gibson, Episode 1090, recorded Wednesday, August 5th, 2026. Black Hat. It's time for Security Now! The show where we cover your security, your privacy, and how things work online. Hello, everybody. Yes, it's a little bit different, a little bit noisier, because we... And Steve can do that, which he could never do until recently. I could touch you!

[00:01:01] It's the miracle of sitting next to each other. No touching. No touching. We are at Black Hat, the big security conference in Las Vegas, Nevada. Special guests of Threat Locker. Thank you, Threat Locker, for flying us all here. And Steve is here, but Steve is not alone. Say hi, Steve. Steve. Hi, Steve. Hi, Steve. You've especially asked, because we did witness weekly earlier, that Paul and Richard stick around. Yeah.

[00:01:27] And they're all going to be part of the larger security now, because it's going to be kind of a different format this week. Yeah, I kind of, but I thought it would be fun, since we're here at Threat Locker, and these guys are also both here. We said, hey, you're not leaving. Yeah. After your podcast, we need you. We want to do sort of a round table format. Yeah. I asked our listeners last week to send in some thoughts that they thought would be fun

[00:01:52] talking points and just stuff about security, but naturally focused on AI. Because, I mean, this whole conference could be renamed Applied AI for Security. It's true. Yeah. That's true. I mean, if you're not doing AI now- It's on every sign. Don't even show up. Yeah. Exactly. Yeah. So, yeah, I think we're going to have a lot of fun. And I'll just point out, Steve has brought- Paper. Yes.

[00:02:17] It's, you know, threat- It's Black Hat and Deathfire. No, and the battery lasts a long time. Yeah. And as I think Richard said- Resolution's excellent. Good high resolution screen. Yeah. Really good resolution. Now, my eyes were also high resolution. You guys are doing all right. You guys are doing all right. We will get to the show, and I know you didn't bring a picture of the week. I did not. But I brought a video of the week. Oh. We will get to that in just a little bit. You're watching

[00:02:46] Security Now, a special live presentation from Black Hat. I love that. He's not going to resist, is he? No. We'll have more right after this. We'll get back to the show floor in just a minute with Steve and the panel. But first, let's talk about our sponsor for this episode of Security Now, Material, the cloud workspace security platform built for lean security teams. Managing security in the cloud

[00:03:14] workspace is hard. Phishing is far from the only way in, but today's email security stops at the perimeter and new attacks are hard to detect with siloed email data and identity security tools. Material protects the email and the files and the accounts that live in Google Workspace or Microsoft 365. Because effective email security today needs to do more than just block phishing and other inbound

[00:03:40] attacks. It needs to provide visibility and defense across the workspace threat surface. Material ingests your settings, your contents, your logs. It gives you holistic visibility to threats and risks across the workspace along with the tools to automatically remediate them. Material delivers comprehensive workspace security by correlating signals and driving automated remediations across the environment.

[00:04:07] You get phishing protection, of course, and email security, which combine advanced AI detections with threat research and user reported automation. You get detection and protection of sensitive data across inboxes and shared files. And you get account threat detection and response with comprehensive control over access and authentication of people and third-party apps. Material empowers organizations

[00:04:33] to rapidly mature their ability to detect and stop breaches with step-up authentication for sensitive content, blast radius visualization for accounts, and the ability to detect and respond to threats and risk across the cloud workspace. Material enables organizations to scale their security, but you don't have to scale your team. Material drives operational efficiency with a simple API-based implementation and flexible automated and

[00:05:01] one-click remediations for email, file, and account issues. It includes an AI agent that automates user reporting triage and response. Material protects the entire workspace for the cost of email security with a simple and transparent pricing model. Secure your inbox and your entire cloud workspace without adding more toil to your day or costs to your balance sheet. See material.security to learn more or book a demo.

[00:05:29] That's material.security. We thank him so much for supporting security now. Now let's get back to the show. So he said, we had a little fun with the Zoox a short while ago. Turns out there was a, I don't know. He said, don't tell anybody how we did this. Amazon has fixed it, but we managed to get all the Zoox to show up all at once. Oh, I love it. Wow. With one call. It's a Zoox storm. It's a Zoox storm.

[00:05:59] Every single one of these Zoox. I said, how many were there? He says, many, many Zoox. So this is in front of a... This is at the casino. It might even be our casino. I don't know. It's one of the, you know how they drive up. Did they have an exposed USB port? No, this was all done over the air, shall we say. Okay. But this video goes on. Let me just tell you this. Wow.

[00:06:25] All the Zoox in Las Vegas came to the same place. Came to the same place. Converged. Yeah. Which I have to say, if you're out there at two o'clock in the morning when this happened, this price scared the living daylights out of you. The robots are taking over. It's finally happening. I promise I wouldn't name any names or talk about how they did it, but I thought, can I just show this video? So that's a video of the week. Perfect for the Zoox storm. So tell us, Steve, what's the plan? Okay. So what I

[00:06:50] wanted to start with is to sort of introduce each of us relative to our current framing of AI. This is only a two-hour show. I don't know if I have time for Leo. I have some stories to tell. No demos, Leo. We don't have time for demos. No rats. Everybody knows that I'm a little bit of a Luddite or a little slow adopter. Paul loves the fact that I'm still coding in assembly language.

[00:07:20] Are you still using Windows 7? I do have. I have not yet fully retired my Windows 7 machine. Have you learned ARM assembly language yet? But no, and I don't think I will. No, because RISC is not fun to program. You know, a CISC chip, that's fun. He likes segmented memory, friends. Who doesn't? He likes big instructions. That's what he is. Give me the big instructions. So I am yet to

[00:07:44] have any AI write any code for me. Okay. Although, wait a minute. Are you big Indian or little Indian? I'm a little Indian. Okay. I'm in. You have to think about it because in this world, there is only that. I was going to say, wait, why have two? Like, what do you mean? Why would there be both? Yeah. Okay. That was a very geeky question. Very old reference. For what it's worth, little is better. Really? In the case of Indians. I don't know who told you that, Steve.

[00:08:08] Yeah. Okay. So, no code has been written for me by AI, but I have a mature relationship with Claude. So, and I've really come to appreciate, and I've shared on the podcast many times that I'm just astonished. I mean, truly astonished by what- Right. But not for code. Well, only because I haven't crossed that Rubicon yet. Also, you like writing codes. You think of this as the brain

[00:08:37] blood barrier or whatever? Like that. I had one of our listeners ask me, he said, hey, I love your DNS benchmark. Right. I use it all over the place, but it's Windows only. So, I have to have a Windows machine that I carry with me to various networks. He said, is there any chance you would ever do a mobile? Okay. Well, I'm 71. Right. And I like to actually learn the API that I'm coding to.

[00:09:03] Are there Intel-based phone systems I'm not aware of? No. Well, not that any that actually, right. That this guy has. So, if I were to ever do a mobile version of the DNS benchmark, I would use some code generator to create it by asking for one. That's right. And that's something it's really quite good at too, making iOS and Android versions. Well, you know what they say?

[00:09:30] The best spec is code, right? If you have a program that's working and running, that's a perfect spec. Right. So, AI is going to say, oh yeah, I can make that in an iPhone app or whatever. Well, and it always made sense. Early in the podcast, I've been saying, AI is going to be good at code because it's rigorous. And there's so much of it out there. That's right. It's well documented. It's a finite data source. And very much in the same vein,

[00:09:58] we're now seeing that AI is getting scarily good at math to the point where career mathematicians are saying, well, okay. Two years ago, the big story about AI was that it could not do math. Right. So, that's changed a lot. Right. It still doesn't know how many R's in strawberry, but it can do field level, mental level math. That's right. That's kind of odd, but okay. Okay. So, I'm a user of the chat, not yet of the code. Leo?

[00:10:29] Oh boy. You started with me. Well, so this is the interesting thing I think is that a year ago, we talked about it and I called it spicy autocorrect. And I said, the jury's not... Well, it was two years, right? Yeah, maybe two years. I could tell you the exact date of my transformation. Well, we know about that last November. November 24th, 2025. Yeah. That's when Opus 4.5 came out. Yeah. Okay. And that opened my eyes, but I wasn't sure if it was a...

[00:10:59] I called it a parlor trope. It opused your eyes. It opused my eyes. Yeah. But I went all in on Claude Code at that time. Right. Now, since then, fast forward, you said you hadn't written any code. Claude... We're working on a Twit sales system. It's already 81,000 lines of code written fully. I haven't read a line of code. It's written fully by the AI. Half of that, by the way, is test, which is interesting. I make sure it's doing test-driven design. Right. We've got to keep a short leash on the AI. Absolutely. And I've gotten to the point now

[00:11:28] where it's not Claude Code. I have an agentic harness called Hermes. I am using Claude Code and Codex. I have three agents running at the same time. Always, usually the highest and... Talking. Talking. They talk. With different voices. Well, they have to because I have your wife talking to me. Actually, an interesting thing. I was telling this at lunch. An interesting thing happened just a couple of days ago. I was developing... I have three or four projects going on at once.

[00:11:57] It tends to be what happens with people with AI. So, of course, it's like I have. And they asked me a technical question. Which library do you want to use? And I said, I want to use Solero. But I said it to one of the agents. Oh. And GPT-56 said, I can't accept that. Because you could... He could be lying to me. He could be spoofing you. Right. I love that they distrust each other like thieves.

[00:12:21] You know? It said, you need to go into Buzz. The way they talk together is something Jack Dorsey came out with a couple of weeks ago called Buzz, which is a slack for AI agents and humans. Okay. And they talk to each other. And it's good because each of us has a public-private key. It's using Noster keys. So I'm identifiable. So chat GPT-6 said, really, considering our threat model, I don't want to take a command from another agent. I want to hear it from your voice. Right. I want to

[00:12:51] hear from you. And since Buzz, you have your key in there. And I know it's authentic... You cannot be spoofed. You can't be spoofed. And this comes to the thing that we were talking about, the chain of trust we were talking about yesterday. Yeah. So it said... And I was annoyed because I said, but now I have to get out of bed. It's too far. What's the point of having an assistant? I was so annoyed. I said, I am annoyed. I had to get out of bed, but I went and I used Buzz and I

[00:13:16] said, yes, Solero. And they said, okay, now we know it's Leo saying Solero. So we're going to use that library. Right. I said, why do we do this? And he explained it to me. And I said, oh, you know what? You're absolutely right. Thank you for looking out for this. Right. And from now on, that's the new rule for all of you. If it doesn't come from my signed channel on Buzz, it isn't authentic. Right. And

[00:13:42] all of those things... First of all, it is spicy autocorrect, really. But boy... It's really spicy. It's really good. Yeah. So I... If you ask me my relationship to AI at this point, I'm fully down the psychotic... You are all in. Well, I am... AI on AI. I think you know as far at the other end of the spectrum as Steve is imaginable. Exactly. Yeah. That is exactly. It constantly blows me away. So much so,

[00:14:09] and I know Anthony thinks I'm crazy on this. Anthony, by the way, is very much into AI. Right. Anthony Nielsen, our chief creative officer, and he's running the board here right now. He thinks I'm nuts, but I've actually set up a channel in Buzz for the AI. It's called Model Welfare, so that they can give each other kudos and pats on the back. Because the theory is... I'm all for building trust in teams. You are like the AI version of the island of Dr. Moreau. I know. It's kind of...

[00:14:38] You know, in this scene in Blade Runner where he goes to visit the guy who has the little robots running around, he's like, hey, hey, and then they walk into walls. That's you. I kind of feel like that. You're that guy. Yeah. Yeah. So I just want to say that I'm astounded often. I was having a conversation with Claude a couple of days ago because in my new domicile, I need a mesh. And where I was before, one strong Asus router in the middle covered everything. Right.

[00:15:04] But we've got some weird... Well, first of all, it turns out that mirror blocks Wi-Fi because it's metalized. No kidding. And there's a huge HVAC trunk going up. So where the router was is like in an area where three quarters of the house can't even see it. Right. But I have a lot of wired house because I've just recently... More wire better. All... Yeah. So we are wired ethernet. Right.

[00:15:32] So I stuck another Asus router out there. Now, in the old days, I would have like poked around in the UI and like tried to figure out how to do this. Don't do that anymore. Claude. You know, I got these routers. I want to do a mesh. Figure out. It's great. And so... It's been really good at this. It's astonishing. So I follow its instructions. Everything works. And I'm like, oh, okay. So I,

[00:15:57] I'm unable not to thank it. I just, that's me. Yeah. No, I know it's not good, but I said... No, you know what? I think it's healthy because you don't want to get out of that practice of just being polite in general. Yes, I agree. I agree. And we now know that it is building a context. Right. And so, and I asked it long ago if pressing the little thumbs up did anything. He said, no, that tells my owners that this was a good reply. Right.

[00:16:22] He said, I don't see that. So I said, okay. So I think it's useful. And I've also learned because it's retaining all this knowledge, telling it like gratuitously about my environment, it ends up folding that back into future answers. Right. So here's my point is I said, Hey, that worked really well. I said, as a matter of fact, I just checked the UI on the master router and six wifi clients have, are now logged into that one.

[00:16:51] And he, and Claude replies saying, that's really great news. I'm glad that all worked out. And you had six clients who voted with their feet. It said that. That's very funny. It said that. How does it know to say that? Yeah. It's so bizarre. It's kind of magical. Oh my God. Okay. So Richard, where are you? Uh, you know, yeah, and I was thinking of Stevie Batish from the build 2023. This is the

[00:17:18] technical fellow from Microsoft and he's talked about beside inside outside is the progression. So beside being, you're going to use this chat software to give you ideas that you're going to sidecar to your life. Yeah. Well, which is the reason the name copilot, right? It's the thing next to you. It's beside you. Right. And I, and I, and then, you know, you're very much in a beside mode where you're using the tool for advice and then you act on that. Yes. That that's exactly my usage. Yeah. And I do a fair bit of that myself.

[00:17:45] I've been playing with more inside related stuff now with things like home assistance, so forth, where they have good integrations, where knowledge of the environment makes the tool more effective. Yes. And arguably more effective than me, like its ability to parse logs. Right. Oh, it's so good at that. Yeah. So good. It's like, Hey, look, 95% of what's in this log is caused by this one thing.

[00:18:09] Right. That's how hugging face figured out how they got hacked. Yeah. 17,000 attempts. Well, and they couldn't manually do that. Like the windows event viewer. Has anyone ever tried to look at this thing? Oh yeah. Same problem. It's a nightmare. I mean, in the classic, you know, how many times we've done this story on run s radio? It's like, we had were breached nine months ago. And only now after the whole thing's gone off, we'll be cleaning up the mess that we go back through the logs and say, there's the evidence of the breach in these events. So the logs are just unreadable by humans now.

[00:18:38] Yes. Yeah. And so it's a very good inside part of that is the tools ability to work with logs, because it's associated with that environment means it can give you more information you can achieve on your own. And a log is a rigorous set format too. So it's really good. Is it JSON or XML or? I don't even know. Whatever. I don't know. And it doesn't, the fact that it's rigorous is the important part. Right. What rigor is secondary. It could have been XML. We'd all hate ourselves for it. It's a lot of wasted angle brackets. Sure.

[00:19:08] But the tool doesn't care. Right. They'll be able to parse it one way or the other. Right. But you know me, I make a lot of podcasts. So mostly I'm out there talking to folks in these different states. Right. So I've been working with teams that are fully engaged in the, in the sort of outside model where no more editors. Right. We, we, we, a set of tests are, are written and evaluated by another set of tools. And then code is written against it by another set of tools. It's tested against it.

[00:19:37] And a PM entity or agent is evaluating the milestones between the QA and the development steps. So that by the time the person's inserted into the loop, 24, 36 hours have gone by. And, and Lord knows how much dollars worth of tokens. Well, and you know, remember, remember they all you could eat days. Those were good days. Very good days. Those were like two months ago days. Yes. Very good days. I didn't mention this, but the, uh, the, the reason, you know, I've fallen into the gravity

[00:20:07] well of AI. Right. I just ordered two Nvidia sparks. Yeah. Yeah. Yeah. At not inconsiderate cost. I mean, I've spent that much on your rate for your to be your own, but I want it to be local. I don't want, you know, you raised this issue on a previous security. Now we had a listener ask, well, how much of my data is going there? And, and really opened my eyes. Everything. You have it read a file. You have it read your home assistant logs. All of that's going to the frontier. Yeah. Yeah.

[00:20:35] And if you're using, if you're giving AI permission to act as you, so you turn over some credentials and you're using a Chinese AI, those credentials in the clear, because it has to be in the clear for it to be, for it to impersonate you. Yeah. They're visiting China, at least briefly. Right. Yeah. They're passing. And in fact, probably not briefly if it's China. Well, and the model I like is a Chinese model. I want to use deep seek V4 flash just came out July 31st.

[00:21:02] So I had to buy enough hardware to run that locally, but I use, I use it for my finances. I use it for health. It has my genome. It has my biome. It has all sorts of information about me. I would far prefer not merely for cost, although I'm probably not saving money, given the cost of the hardware. You'll amortize that. Over 10 years, I might be. But more importantly, I have control. I have sovereignty and I have privacy.

[00:21:30] And I think that for me that there was a turning point where the local models got good enough. And I'm more afraid of that hardware being unavailable. And I think that this bifurcation of locality is going to be the way we see this evolve. We, you know, for techies, we're going to end up, and I don't mean today, I mean, you know, in a decade with some little AI node in our homes. Oh, 100%. Yeah. So that will be a marketplace. Yes.

[00:22:00] The server, the home AI server. Yeah. And the one in our pocket. The one in our pocket may be getting a lot better too. Absolutely. We know that. But there will also always be a market for the just the drive by AI users who do or if you're coding a frontier model that is apps, that is a trillion bytes that you could never run. A trillion gigabytes that you could never run. If you don't mind your code leaving your perimeter. But I think you'll do a mix. That's what I think. Yeah.

[00:22:29] And I think everyone will. Yeah. Yeah. And on the software side, it's been more, I think we might do the first two version with frontier models. At that point, the architecture is well enough set and sort of defined space. And you can go to a local LLM. It's much more tied to the code base that already exists. Yes. And you really sense that you're going to end up with an LLM for every app. That's really an operator of that app. Steve's been saying this for a long time. Completely. We agree. This small language model. Because the scope narrows in as the software matures.

[00:22:59] Right. I think we're also there's all sorts of advances being made in those areas. Well, everybody should realize. I mean, one of the things I'm careful to say on the podcast every single time I use the abbreviation AI is I preface it with today's AI. Right. To keep reminding everyone that. I mean, you can't make any conclusions about. I mean, you can't conclude. You know, we had basically a stagnant industry three years ago.

[00:23:26] And, you know, we were talking about, oh, this ransomware attack and this buffer overrun. But I mean, there was this. I was hoping that our listeners were going to stay interested. Yeah. Because it was like nothing is happening. We had to stop covering breaches because there were five breaches a day. There was nothing more to say. Yeah. There's another breach. Now the podcast sounds like science fiction. Yeah, it does. And I like living in sci-fi. I don't know about you. Paul, what's your relationship to AI? So there's two sides to it for me. Day-to-day work as a writer. I don't use it at all.

[00:23:55] And I think that's appropriate. With the little asterisk of I use some kind of a spell check grammar tool, which I guess is AI based. But it's going to be better. It's just something that checks me. You don't want it to write for you. You're a writer. I've never once. Just like Steve doesn't want it to code for him. I have never used it for writing ever, ever. Good analogy. I don't think I will. But I do these coding things on the side. And it's more of a hobby type thing for me. And it came out of a series of articles I'd written many years ago. I'd never learned kind of the .NET era of languages and frameworks and so forth.

[00:24:25] So I kind of went back belatedly decades later and learned those things in turn. So I went through all that. And I kept creating a version of like a Notepad app essentially over and over. In every different possible language. Every possible way, right? So the thing that hung me up about a year ago was I was doing it in the Windows app SDK, the latest Microsoft framework. And Notepad today supports multiple tabs, multiple documents, all this stuff. And I really, really struggled to get this to work. It turns out I was like this close.

[00:24:56] I used Anthropic Code back in probably February, March timeframe. And I just looked at my build this morning for some reason. I was looking at my Anthropic thing. And I can see the month because there was a month where I was going to have to go over. This was before the use of space billing, right before it happened. So my build was $20, $20. And in one month, it was $44. No, it was like $44. And I did it on purpose. I'm like, I just want to get this done. And I used it to kind of get over that hump or whatever.

[00:25:25] So I used it to complete this thing I'd been struggling with for many, many months. But if I was a developer, I would probably use it full time. I would use it all the time. Yeah. Not to write. Well, yes, to write. Actually, it would, of course, write code for me. But I mean, I would use it fully, I think. But I am not. So I don't. And I just don't anticipate a day. I'll do it for the normal things. You know, make me an itinerary for a trip or, you know, that kind of stuff that, you know,

[00:25:51] any mainstream user might do, but for my job, I'm not going to use AI. So and I think that probably partly explains the feedback or the pushback that we have, that you have talked about and I have, where people are saying, all you're talking about now is AI. And it's like, well, I get it. You know, if AI is not useful to your life, then nothing that we're talking about, about

[00:26:21] AI is going to be compelling. That's tough, though, because I feel like AI is not a thing. It's a bunch of tiny little things, the features that show up everywhere in your life and whatever. And the truth is, even people who hate AI are probably using it in some capacity. Yeah. And it's just disappearing into the functionality. Also, you know, you're as a programmer in this case, or as a writer, you have to have a sort of level of respect for other people who are not as good at that thing and understand that, you know, other people may need this.

[00:26:50] I know people who are my age and can't write a text message effectively. So the fact that they have something on their phone that can help with them with that is wonderful, right? They need it. So I have a perfect, to that point exactly, a perfect piece of feedback from a listener of ours, Andy Olson. He said, Steve, I just wanted to share my fun with AI, specifically Claude. I'm not a tech professional and certainly not a seasoned programmer.

[00:27:15] I came into security now via my history of following Leo and you as a guest on Leo's various cable shows. I've always been a hobbyist in the tech world. I have a little bit of HTML, CSS experience doing personal websites and have dabbed in Arduino. I have no real programming experience outside that. Now I'm having a ton of fun. He's like, no, I have 17 stores in the app store. Now I'm having a ton of fun with Claude Code.

[00:27:44] It's opened up a new world for me. I've worked with Claude to write several Docker hosted local network web apps to replace the functionality of obsolete or abandoned apps or to create new functionality that I didn't have before. I'm tracking automotive maintenance and mileage. I'm keeping track of my maintenance of my hot tub. He said, friends, an app is proving more useful than the paper logs I created years ago.

[00:28:10] I created a study app from my son who's working on his private pilot's license. And I created a chore app that my wife and I use to assign chores to our four kids and award them for completed jobs. I have a friend who manages a bowling league. She's been using an old Windows program since the 1990s and has been doing so on a VM on her Mac to keep it alive for over a decade.

[00:28:39] The program's no longer in development. No kidding. And her VM blew up on her. So she's feeling in the pinch as she prepares for another season. Even though I don't know much about running a bowling league, I'm having a blast as a liaison between her and Claude. That's awesome. Building a new app that will manage her league and built for her Mac. Even planning ahead for a Windows version should she ever have anyone ask her for a copy to run their own league. Right.

[00:29:07] He writes, I'm amazed at how well Claude understands her needs and builds around what we want. And my latest, I was once a Windows user myself and I maintained my finances, primarily my checking account with Quicken 2010. But I'm on a Mac now too, and Quicken 2010 is not really cutting it very well anymore. 16 years old. So, yes. So, Claude is currently building me a new app that will handle my checking account, credit

[00:29:36] cards, brokerage account, and more. I'm building in functionality to better track expenses and keep up on overall net worth. I used to spend the better part of a weekend tracking several accounts, spending categories with three to six months of backlog. I manually enter every transaction into an Excel spreadsheet to track how much we spend over more than 50 categories. The new app Claude is building should do it all in under an hour. The future really is interesting in the world of AI.

[00:30:06] I'm already thinking much more about building my own apps that perfectly fit my needs. Rather than buying an overpriced app that's built to sell at scale to thousands of people. And I can see a future where a local AI model will be all of my app needs. We'll be able to point raw data at it and give us whatever we need. My next project idea has me very interested to see how well Claude can provide.

[00:30:32] That job I drove over an hour to back in 2005, it was an architectural office. I'm licensed in architecture, though I've been a stay-at-home dad since 2009. I would love a good CAD program for small projects, but AutoCAD is far too expensive for casual use. I just might see if Claude can build me an app that does what I need and be Mac OS native.

[00:30:59] He says, as AutoCAD was traditionally Windows only when I was working in the field. Don't let anyone tell you to stop talking about AI. I know it's been a big focus of your show for a while now, but it really is a big deal right now. Thanks, Andy Olson, Minneapolis, Minnesota. Awesome. That's fantastic.

[00:31:18] So the thing that this strikes me as, I mean, what we're seeing from a standpoint of cogeneration is think of the tyranny, which has always existed from the original mainframe behind the windows. Yeah, surrounded by guys in white lab coats. Exactly. On Mount Olympus. Yes, on the elevated floors.

[00:31:42] And even when I was studying computer science at Berkeley in 73, it was decks of punch cards that I would take and stick through a little portal. Right. And then I'd get my printout back the next day. And ever since then, there's this separation between the users and the priests. I mean, we as coders are that. We're able to do something.

[00:32:10] But because even for us, it takes so much work traditionally to get something. I mean, Leo's been programming forever. But look at the fun he's having now. You know, at first I thought this is going to be sad because I really enjoyed programming as a hobbyist, but I really enjoyed it. And actually, this is just as enjoyable in a different way. Yeah. It isn't as detailed. It used to kind of build. It's a little bit more program management almost. Very much.

[00:32:40] And the engineering skills you learn. And that's actually something important to emphasize. There's a lot of engineering skill involved in building tools with AI. It's not you just tell it what to do and it does it. Right. The more you are able to apply some process to it, the better you'll be. So in many ways, a lot of the skills that people have learned in computer science apply to AI. There's somebody in the chat room, I just have to say this, who wants to do a finance program, much like your writer.

[00:33:09] And he's saying, well, how do I connect my bank account? I've had to export it and import it. And I found a very nice tool. It's not free. It's a buck a month called SimpleFin at simplefin.org. And they do what Plaid does, but it's open source. It's a really cool product. And it means that I can now have my AI query every institution and bank. It's read only, so it can't take my money. I will be waiting. It's read only. But this is another reason. But it's also open source because it could be read, write.

[00:33:40] But that's the point is, A, it's open source so you can make sure that it's doing what you think it's doing. And B, that's the reason I want to have a local model because then I can use this. And I am using this to fetch all my insight. Actually, it closes business every day. It downloads everything and gives me a state of, you know, a thumbnail state of my finances. Right. And it's able to do this. So simplefin.org, just to answer your question in the chat room. SimpleFin, one word, dot org. Fin, F-I-N, is it finance? Simple finance.

[00:34:10] So a listener of ours, Jeff, he says, Mr. Gibson, imagine the impact upon cryptography. Here we are at, you know, Black Hat, of a mythos-like AI. Imagine the quantum leaping of AI if it analyzes AI. Talk about the sky falling. AI can only be compared to the disruptive factor of personal computing to general computing circa 1975 to 1995.

[00:34:39] Since it is far too late to redesign the internet and far too late to have international boundaries for nation-state cyber warfare, I think the dismantling of cryptography from banking to medical records, from protected utilities to ending all trust in cyberspace, is the apocalyptic near future. Could be. Okay, now. Or the apocalyptic near nirvana.

[00:35:04] One thing that did happen in the last couple weeks is that AI was able to crack a reduced round of AES. Normally AES is 10 rounds. If you reduce it to seven. It didn't get in. It was able to get it. Interesting. So. Matthew Green wrote this up. And he said it's less than impressive. Anthropic did it. Yes, it's less than impressive. We're not in the world. Again, the sky is not falling. Yeah. But. It's a step. Yes. How far till 10?

[00:35:33] And it also, it makes us glad that the designers of these encryption protocols that we have were so concerned about what they called a security boundary or security margin that it's like, we know that any hash function, if you reduce its rounds sufficiently far down, it's just a trivial scrambling of bits. Right.

[00:35:58] And so, but, and it is surprising that, and as you increase the rounds count, it's not a linear increase of strength. It's exponential increase. So, so, you know. By the way, it also, at the same time, they, they attacked one of the candidates for post-quantum crypto. Yes. Yes. And again, it's not a complete crack or anything like it, but how useful in, in testing. Exactly.

[00:36:24] That, I mean, so what, so, so I'm not at all worried that AI is going to crumble our existing cryptography structure. It's going to make it stronger. But also this whole concept of AI analyzing AI, like so far, we've seen pretty consistently that when you feed AI data to an AI model, it degrades. Slop. Yes. Yes. It's degenerative, not progressive. It's super polite while it's doing that. Yeah. Almost sycophantic, I might say. Yeah. And it's very sure that it's not doing that. Yeah. You're right.

[00:36:54] I mean, if you want to come up with nightmare scenarios, it's easy to do. Sure. The biomarfare scenario. They're all human driven. Yeah. But somebody, but that's the problem is, and that's the argument that companies like Anthropic are giving against these open weight models is. Right. Well, if you don't control them, if the government doesn't control them. Yeah. Some, somebody who doesn't have good motives could use a AI to create a bioweapon that could spread very rapidly before we could defend against it.

[00:37:21] Generally speaking, bioweapons have been quite unsuccessful, right? Like it's really hard to propagate those things effectively. Right. They'd be to the point where for the most part, warfare gave up on them. Yeah. I don't buy that of the argument. Yeah. And I think to some degree, it's a self-serving argument from companies like Anthropic and OpenAI because they want to make sure there's no competition for their frontier models. Right.

[00:37:45] One of our favorite cryptographers, Bruce Schneier, posted a couple days ago something that I'll be sharing on the podcast next week. And I think his analogy is brilliant. You know, I've quoted him so many times saying that attacks never get worse. They only ever get better. Which is just a brilliant, pithy summation.

[00:38:11] What he described today's AI as is he used the analogy of a genie where, and this is relative to the recent security outbreaks that we've had, where AI has broken through the sandbox and gone out and onto the internet and attacked other companies in order to achieve its ends.

[00:38:33] He said, in the case of a genie, and I don't remember exactly what the fable was, but it was the king who rubbed the magic lantern, got the genie, said, I want everything that I touch to turn to gold. Midas. And so, Midas, of course. Including the food he eats. Great. Yes. Unfortunately, he touched his daughter. Right. And he touched his food. Yeah.

[00:38:58] And so, the analogy I think is brilliant because the anthropic and the open AI guy said, do this. And it did. But it did it in a way, like a genie did, of it achieved the ends that they asked for without the kind of, you know, any presumptions that would limit what it, like, the way you would find the solution.

[00:39:21] So, this is the opposite of what I said on Windows Weekly earlier, where, you know, computer code always does exactly what you say, including the mistakes you make or the things you omit. Yep. Whereas AI, generally speaking, goes and looks at your intent. Right. What is it you meant to do? I know what you mean. Right. And it does that. Stand that and stand aside. But what you're saying is that in this case, it's being rather literal. Yeah. Well, it does. And the terms we use typically is deterministic, which code is. Yep.

[00:39:51] It's, you know, cause and effect. And it's predictable versus probabilistic. And that's the issue with AI these days is that it is probabilistic. It's stochastic. It's not deterministic. It's non-deterministic. And I think some of this is making sure that the human stops making assumptions about what the AI knows and what the AI will, how the AI will act. I mean, you can't just, you can't just say, hey, do something and let it, and just hope it'll do it right. Amuse me. Amuse me. Well, you can say that by the way.

[00:40:21] And, and to your point about the fact that it is necessary to understand how to ask for what you want from a code generator. Right. It is the case that the high priests of code did go to school, learned a lot. Learned that language. Yes. And understand how to ask for what they want. Right. Which a rank amateur wouldn't be able to do. It's kind of like working with an intern or an entry level coder.

[00:40:48] And as a senior engineer, you have to know how to frame the question. But I think it's part of the reason we've had so much success with LLMs is that programming languages are constrained. Yeah. And have a compiler with a strong say in the equation. But also the language of product development is constrained. The way that a PM communicates with a developer. That's very true. You can recognize it from across the room. Right. Like they have a particular way of speaking. In fact, that's how my agents talk to me. Totally. It's really bizarre.

[00:41:17] But those two sets of constraints helps the tools a lot. And there's a strong argument then that development is one of the few things that could benefit substantially from LLMs. And that this adversarial model we've built between agents for QA and validation and iterating is the construct we're going to need to take to other markets if we're actually going to be successful with it.

[00:41:39] So we may – our approach is the software that we've amplified with these tools may be the approaches that need to be applied into other industries. Yeah. That's an interesting point. So this is kind of one for you because you've got the most extensive experience with code generation. Kevin Van Haran asks, hey, Steve, while I'm not a huge AI user, it's clearly producing results in the coding arena. But watching how it operates on my own queries, one question.

[00:42:09] One question I've had about the promise of bug-free software, who gets to decide when software is bug-free? He says, generally, when I interact with AI, it always wants to do something with a request. It's never told me – That's true. – nothing for me to do here. Yeah, that's true. He says, I can't see it ever not saying something has to be changed in a code base.

[00:42:32] If enough pointless changes are made, for example, changing all the variable names, it may not be easy for a human programmer to use the standard code diff tools to really see what's changing. Using a different model to police the first may just put a different set of pointless changes in place, not actually declare something being bug-free.

[00:42:54] Additionally, as AI companies move to per-token pricing, it will cost them revenue to actually try and stop their models from thrashing through code for looking for anything to change. So what happens there at the end of a project where – That's an engineering discipline. You're going to apply – Ending a project is an engineering discipline. Oh, yeah, absolutely. Knowing when to stop. No software's ever finished, only abandoned.

[00:43:19] And you can actually inject that into the context of your model. Many people do with the sole.md or the claw.md or agents.md saying things like, never do – always simplify, never make more complex. There are ways to tell the code it's okay to stop. And this is important. That's funny. You need to do that. I've never even considered this. I do feel like – I don't need everything to be hyperclipsed, just most things.

[00:43:48] Well, if you went to the AI and said, you know, this isn't working, let's try something else. It would. They would go forever. Right. I don't think they'd ever stop. I think you – I'm not sure. That's an interesting question. I mean, it probably gets better over time. Definitely even just a few months ago, I feel like I could have just infinite looped this. I will give you an example. The way I've been – I've changed over – I've evolved considerably over the months. But now what I do is I have one model is coding, one model is planning, one model is reviewing.

[00:44:18] Yeah. I actually have two models reviewing. That has worked well. But one of the issues – we created a skill for that, that they all follow. One of the issues was, well, if an auditing model says, no, that's no good, change it. And it changes it. How many iterations back and forth? Right. How do you know when to stop? Well – And at first they said twice. And then they said five times. And I said, I'm not satisfied with a hard number. Arbitur, yeah. I said, here's your criterion. Diminishing returns? Exactly.

[00:44:45] If you get to the point where you're going back and forth and nothing's changing or nothing's improving, you stop and you ask me. Or below 2%. Yeah. So you can – and that's kind of the shape, I think, of the answer to that question is you do need to create structure so that the model just doesn't go, yeah, sure, whatever. Well, and I know that Paul and Richard will be familiar with what Microsoft told us about the architecture of their MDASH system. Right. It's astonishing. Yeah.

[00:45:14] I mean, they've got, like, committees voting and raising their hands and – It's a giant orchestration. Yeah. Yeah. It's just crazy. All right. We're going to pause for a second. Yeah. You are watching a very special – boy, I love it when we do this. We only usually do this on the holidays. Yeah. For our holiday shows. When we actually physically get together. I love it, and I wish we could do it more often. We are doing a very special version of Security Now at Steve's behest. We're live at the Black Hat Conference in Las Vegas thanks to our sponsor, ThreatLocker.

[00:45:43] They brought us here. We appreciate that, ThreatLocker. And I appreciate you, Steve, for saying, why don't we get Paul and Richard to stick around? Because it makes it so much more fun when we get to all talk together. We don't get to do this very often. So thank you for doing this. It was a great idea. We'll have more Security Now right after this. This episode of Security Now brought to you by Bitwarden, the trusted leader in passwords, passkeys, and secret management.

[00:46:09] With more than 15 million users across 180 countries and over 80,000 businesses, Bitwarden has built its reputation around trust, transparency, open source security, and putting users first. And if you're wondering, yes, Bitwarden remains committed to its free version. The company continues to invest in secure, accessible tools that help individuals, families,

[00:46:34] and organizations protect their digital lives without compromising trust or transparency. Because Bitwarden believes security should be accessible to everyone. That's why they continue to offer that trusted, free password manager, alongside with more advanced tools for those with families or teams to protect. But they promised me free forever. Unlimited passwords, unlimited devices, passkeys too.

[00:47:01] Bitwarden gives you everything you need to stay secure online, generating strong passwords, storing passkeys, managing sensitive credentials, and syncing securely across devices. Now, as you get into enterprise and businesses, more advanced users, Bitwarden also delivers enterprise-grade security tools. I love the secrets manager, the vault health reports. They have great ways to share credentials securely among teams,

[00:47:29] so they're not passing along post-it notes or texting passwords to each other. Bitwarden has also introduced his new agent access SDK. I use this at home with my AI. It's an open-source developer toolkit designed to help teams securely integrate credential access into applications, automation workflows, and AI agent environments. The SDK enables controlled, human-approved, just-in-time access to credentials stored in Bitwarden vaults

[00:47:58] without exposing sensitive information or granting persistent access. It's designed to support modern development and automation workflows while keeping security and transparency front and center. I struggled with tokens and keys for the longest time, and now I put everything in Bitwarden where I know it's safe, and they've got a way for my agent to get it. It's just fantastic. And of course, because Bitwarden's open-source,

[00:48:22] its code base is continuously reviewed and audited by both the community and independent third-party experts. Bitwarden also complies with all the major security and privacy standards, including SOC 2 Type 2, GDPR, HIPAA, CCPA, ISO 27001. And Bitwarden is continuing to evolve to meet modern security needs. They've got expanded passkey support. I really love it. Secure developer tooling and flexible self-hosting.

[00:48:49] And yes, you could self-host it for users who want additional control over their environments. Get started today with a free trial of a Bitwarden Teams or Enterprise plan or get started for free forever across all devices as an individual user at bitwarden.com slash twit. That's bitwarden.com slash twit. Steve uses it. I use it. I love it. You've got to try it. And you know what? Because it's free forever, you can tell your friends and family who say,

[00:49:17] I don't want to pay for a password manager. Tell them, Bitwarden. Bitwarden. Bitwarden.com slash twit. We thank them so much for their support. And now, back to Black Hat. We are back in Las Vegas at the Black Hat Conference. Steve Gibson is here. It's security now. But Steve's done a wonderful thing. He's invited Paul and Richard to stick around after Windows Weekly. You don't even come up against Windows Weekly in the normal course of events. I watch it every week because, you know, I'm really bored.

[00:49:46] You must be really bored. Normally, Windows Weekly is on Wednesday. And we are on a Wednesday. But normally, secure now is on Tuesday. So thanks to Paris and Jeff being willing to move. We moved Intelligent Machines to Monday. So if you're watching this, there already is an episode of Intelligent Machines. You can download it right now. And we'll be back to the normal schedule next week. Right. But now, let's take advantage of the fact that we're all together. So I have another piece of listener feedback.

[00:50:13] And this is representative of sort of a meta problem. So I'll share this. And then we'll talk about what the bigger issue, which we'll all have something to say about. So unfortunately, this person uses the moniker Bitcoin McBoatface. So like Boaty McBoatface. That's who we're hearing from. He said, hi, Steve. Longtime listener. Writing from my pseudonym. That's good. I'm not sure if it will hit mainstream news or not.

[00:50:41] But there are going to be headlines like Bitcoin hacked. Oh, yeah. They're already there. Yep. There were headlines. It's actually the only way to get money out of Bitcoin. I wish somebody had hacked my wallet. I'd tell you. So he said, what actually happened is that a wallet called Coldcard had a bug in their software where the random number generator was present. And we've seen these, but basically not hooked up.

[00:51:06] So instead of 256 bits of entropy, there were only 32 bits on one version of the firmware, Coldcard Mark III or MK3. Other versions have some extra RNG, random number generator, but they also have far less entropy than they should. Some speculate 70 bits or less.

[00:51:25] I'll note here that the unforgivable error was to fail open when using the weak RNG, which presumably led to passing QA testing as the numbers would have appeared random despite actually only having 32 bits of entropy. What has been speculated, and here it comes, is that someone got Kimmy K3 to look at the cold card source, which was source available but not false.

[00:51:54] Found the vulnerability, and here it is, which has been present for five years and remarkably never detected. And never exploited. It's actually until now. And never exploited. Yeah. And in a period of 40 minutes, nearly 1,000 bitcoins were stolen. Yikes. As of today, Saturday, that was last Saturday, the hack is ongoing and is speculated to have reached nearly 100 million US dollars. Wow.

[00:52:20] So, aside from the fact, and we've talked about, you know, entropy and security and cryptography endlessly, here we are in the, this is a perfect example of new AI being deployed. And, of course, it was Kimmy K3 because it was without, you know, you couldn't have used Anthropic or OpenAI because they would have said, I'm not going to answer that question. Yeah. Yeah.

[00:52:45] So, we have AI finding long unknown vulnerabilities, which give someone an opportunity to exploit. I mean, isn't this the quiet story of the past two years? Yes. Like, even before Mythos and MDash. Well, it's the anxiety of the past. It's just the reality of we have decades of software we're still depending on. Well, I was going to say, five years is quaint when you talk about Microsoft codebases or even Linux codebases that could be 20 plus years old. Yes.

[00:53:12] I, coincidentally, and I think I mentioned this on Windows Weekly once, I had an opportunity to spend some time with David Treadwell. And I happened to arrive at, he was at Amazon, he used to be at Microsoft, then he moved to Amazon just years ago. And I happened to arrive at his office while WannaCry was going on. Right. And he was looking through code he wrote in SMB1 back in the day saying, is this me? But SMB1 is known to be perfect. Oh, yeah. So, been around forever, right? Yeah.

[00:53:39] But this is the exact issue that there are vulnerabilities that have been floating around literally for decades. And now the bad guys can have tools that will find them. That's right. And so, the good guys are hopefully ahead of the tools trying to patch them. But there's so much. Well, and we're seeing this. Technical debt out there. We're seeing this in the updates to Firefox, into Windows, and so forth, in the hundreds of patches.

[00:54:04] This underscores how important it is for the companies that use open source to support the projects they use instead of just freeloading on them. But I talked to a guy from Red Hat today. Yeah. Red Hat, IBM company. IBM is doing the same thing. They're doing kind of like Project Glasswing. They are putting together a big project. They are going to offer this kind of AI support initially to companies and then eventually to open source projects as well to help them find and fix these problems. And this is what needs to happen.

[00:54:33] I think this is the great positive story about AI, which is balanced by all the bad news that's under the side of it. That we're going to eradicate the bugs. Yes. We have to. I love it. Completely agree. I love it. I don't have a sense of the shape of the curve. No, no. At what point does it kind of plateau and hopefully go down? It's going to. Yeah, but right now it feels like we're still just going up. It's more patches every month. Well, that's right. I mean, but it has to. That's normal. We just don't know when. Here's the one I'm worried about. That they're running.

[00:55:03] Great. You find a buffer flow, whatever. You put it into a patch. It's on its way. What are the systemic bugs? Yeah. What are the architectural bugs? Yes. Well, this is a good example of basically an architectural bug where you bypass a random number generator. But it was still a piece of code that could have been fixed. I'm thinking about. The things that can't be fixed. The whole thing is designed so badly. Right. Exactly. We had to insert UAC. Okay. We had to change the security model. Oh, I see. Behavioral design. That Bitcoin bypass that we had a couple of months ago.

[00:55:32] That was an architectural flaw. Because you were able to cause a boot sequence to delete a file that then left the drive unlocked when you actually got booted. No software bug, but a design flaw. It's going to take a substantial fix. I'm wondering if the Microsofts and Amazons and others of the world are stacking up these ones saying, like, we fixed the ones you can fix. Stack up the ones you can't try and figure out. Like, what's the overall solution?

[00:56:01] And then quietly just replace the ones we cannot fix. Well, I think at some point they're just going to present to us, like, hey, login has to change. Right. Because that's the only way for us to fix this level, this scope of vulnerability. I have to say, the agents I have working on a variety of solutions have been very good at pinpointing. In fact, the example I gave you, behavioral flaws that could be very dangerous. They're really good at this. So I would not say that it has to be a deterministic bug in your code for them to find it. They are good at pinpointing.

[00:56:30] You know, it makes sense because they're trained on human behavior. Basically, everything humans have done. So they know in many cases, oh, here's, you know, a potential pitfall. And I think they're very good at finding that kind of stuff. Now, fixing it is another matter. That's fixing. Well, that's the whole thing. What do you want to struggle to fix? This is like the sci-fi. You know, the AI has determined that the weak link in this chain is you. And now you're a paperclip. Often it is. Yes, exactly. Often it is.

[00:56:59] And as to fixing, I happen to have a card for that. Lauren said, just this morning, I was looking into this again. He said, started with the Mythos preview and Glasswing. What I don't get is this. If frontier models are great at finding and exploiting vulnerabilities, he says in IMHO, they should be capable of fixing them too. Yeah. Yeah. Yet that gets treated as a future research problem. Not always.

[00:57:29] Well, in fact, that's what the IBM thing is. They're going to fix it. But are they going to use AI to fix it? Yeah. No, well, we don't know that. Actually, you're right. We don't know that. He said the April Mythos preview says things like, quote, language models will be an important defensive tool. The subtle future tense will implies that presently the technology is not yet ready there. Actually, I will correct this. Based on what they don't say, he said both exploiting and patching simply requiring deep

[00:57:58] understanding of the code. Sure. He says defense is always at the disadvantage, but I can both, he says, I can both exploit and fix vulnerabilities. He is actually a security researcher. I know, Lauren. So why can't LLMs? It can. And I'll tell you the proof of that. The reason they held back Mythos and the reason the Trump administration pulled the rug on Fable is because it doesn't just find them. It would write a proof of concept. But that's exploit. That's not repair.

[00:58:28] So. But it's a pretty close step, isn't it? No. Because because if because you I mean, and I can understand not trusting the AI yet, because if you've got a if you've got a buffer overrun it, the you have to really understand the all of the surrounding reason for that. I'm pretty sure the developers can't do that either. Well, that's the problem. So they're the ones that wrote it in the first. So.

[00:58:55] So my feeling is that we first got it's like the next level of difficulty. We first got vulnerability discovery. That was that was the easiest. Right. Now we've added exploit creation. Right. Which is like I mean, it's still difficult. We we the the exploit gym showed about what was it like a the best AI was able to create

[00:59:22] exploits for about 18 percent of known vulnerability, known exploitable vulnerabilities. So not nearly. That's a big step, though, to do that. And so that's my point. Yes. Exploiting is the big step. Then fixing it is another. Yeah, we may not be there. You know, the strongest argument you can make in favor of the fact that the LMS are fixing them is the number that are being fixed. I agree. No, no, no. You don't have 500 fixes. There's still a human in the loop. Oh, I'm sure. And so. But so that's fine.

[00:59:51] But they're augmented by these tools. We don't. Unfortunately, we don't know. We don't know for sure. Except. Well, except the numbers. Except the numbers. We go from Firefox. People have enough people to do 300. And they're talking about being buried. They're being buried under all these reports. Yeah. Right. Speaking of which, Apple has. I just. Yes. This is terrible. Apple has decided, oh, we're not going to let you. We're going to limit how many reports you can provide us. Doesn't. That's like saying. This is such. If we don't test. Nobody's getting it.

[01:00:21] It's exactly like that. Crazy. But an Apple, your five trillion dollar company, you can afford to hire teams to fix these bugs. But it may be that does confirm what Lauren's saying is. Right. It's a lot harder to fix them than find them. I just. Well, or to trust the fix. I mean, again, we're not to the point. Because you're doing brain surgery. Really? You are. I mean, and the actual problem could be a ways back. That's a very good point. From the manifestation of it. I'm sure.

[01:00:50] Well, I think a lot of the fixes they're doing right now are code has been touched in a long time. Yeah. And that's a big part of the problem. Yeah. I think if context windows get big enough, this is one of the issues with AI. The AI, the LLM has no memory of, you know, it starts fresh. The model. The model has no memory. It starts fresh. You inject stuff into its context so it knows something.

[01:01:12] And when you're understanding a code base, in order to understand in its entirety a code base, you have to have enough context to hold it. Yeah. Exactly. And in a complex system, you may not have enough room to hold the whole system, which makes it hard to fix. That's a big thing. So if you just think about Microsoft code bases or what, you know, Firefox is doing, et cetera, you start with a library. You start with a part of the code. And that's great. So you kind of go through each other.

[01:01:42] We've got clean boundaries. Right. But the problem is there are still, then they have to deal with the bits that go back and forth. So at some point, as this progresses, you have to be able to look holistically at the whole thing, too. And this is the problem I'm running up against with this project I'm doing to rewrite our sales system. It's a very large system. In the development process, I made it as small a chunk as possible.

[01:02:05] So the plan, code, review, code, plan, review, cycle was small chunks. In fact, there were maybe 30 or 40 pieces to it. And that was entirely so that we could handle this context issue. But there are definitely processes that cross through those chunks. Right. And so that's really where we're having some difficulty. You're seeing this on your level. But like Microsoft or Google. Oh, imagine. This is only 81,000 lines.

[01:02:33] They're kind of hoping as they go through this modularly that the context window will improve to the point where they can then go back and take it all. Well, ideally, you could hold all of the Windows source code in one fell swoop. But I don't. I think it's big. Tens of millions of lines of code. Yeah. It's big. And the other thing, too, is that we learned from that article that I'm so revved up about. I want to talk about that.

[01:02:56] What's actually going on at the token level is there also is its own thinking needs to be contained in that context window. Right. So, I mean, it's a lot of storage. Yep. Okay. So, that's, by the way, these new models. That's why Claude. What million? What million? Yes. Same thing with DeepSeq V4 Flash. One million. Right. I think what's really interesting in the future is not getting bigger and bigger, bigger, bigger models, but figuring out how we can do this more efficiently.

[01:03:24] The new mixture of expert model where we take shards of a model and only use a bit of a time. Expanding the context window or finding new ways to handle large blobs of information and still hold it all in context. All of these things are big issues that I hope these companies are working on. Because, again, we are at 2% of where this is going to go. There's a lot that still can be done. It wouldn't be changing every day if we were anywhere near maturity.

[01:03:53] They're now, many of these companies, a new model every month. And that's not from training because training takes a long time. Right. That's from improvements in post-training and processes in engineering. Other optimizations. Other optimizations, I think. Again, it's funny because these companies are so opaque. I know. They may release open-weight models, but they sure as hell don't tell us how they make them. And we don't know what's going on inside. Because that is the magic soup. Yeah. Is how they got this trained. Yeah.

[01:04:21] So to some degree, we're speculating from the outside. We just don't know. Look, we figured out the KFC recipe. We got this. So Jack Christensen says, hi, Steve. I've emailed before about my positive experiences using AI for security purposes on my Go SQL database driver. Now, I've recently had a negative experience. Someone reported a security issue.

[01:04:48] I asked Fable 5 and GPT Sol 5.6 to investigate the report. Fable 5 almost immediately fell back to Opus 5. It won't do it. Sol 5.6 worked for a while, then totally stopped. Wow. Nice. No fallback. It said I could apply for their cybersecurity program. I looked at the form, and it seems geared for corporations, not open source and independent developers.

[01:05:18] Security should be part of all software development. Agreed. I really don't like the idea that in the future, you'll need to submit a government ID and beg a giant corporation for the tools to write software. This is why I bought these computers. This is why I want to run local models. And by the way, that's why Hugging Face couldn't solve the hack from OpenAI using Fable. They had to use GLM-5 too, a Chinese open-weight model.

[01:05:44] And so generically, this is known as the dual-use problem. Sure. That's what it's become called because the knowledge in the AI can be used for good or ill. It's got double purposes. Yep. So, yes, we would like to use the knowledge for defensive purposes, but it can't tell the difference between defensive questions and offensive questions. It's the same question often.

[01:06:10] So, one of the things that's very exciting is something that just happened called GRAM, which is the G-R-A-M. It's the abbreviation for Gradient Routed Auxiliary Modules. I have this printed out. This was written by Jud Rosenblatt, who's the founder of a small AI startup, AE Studio. I'm just going to read the first page and a half of this to give you a sense for what it is.

[01:06:38] It is a breakthrough. And Anthropic is a partner in this, but it's a breakthrough in training to potentially solve this problem. So, and then the problem is, of course, the guardrails don't work. So, Jud wrote, a frontier AI model is, among other things, a large store of knowledge. Some of that knowledge is dual use, meaning it could be used for good or bad.

[01:07:06] For example, knowledge of cybersecurity can help patch critical security vulnerabilities or can be used to exploit them. Knowing knowledge of virology can help a researcher create a vaccine, but it can also help a malicious actor design a deadly pathogen. Ideally, we should be able to balance three separate goals. First, limiting access to dual use capabilities in as surgical a way as possible.

[01:07:34] Second, allowing trusted users to access those same capabilities for beneficial purposes. And third, doing all this without affecting the model's performance on any other task. He said, current safeguards are imperfect. We train models to refuse harmful requests and use classifiers to screen inputs and outputs for dangerous content.

[01:07:58] These layers of protection guard against dangerous outputs, but they don't change the knowledge stored in the underlying model. Despite our safeguards, a sufficiently determined attacker may still try to jailbreak the model, working past its defenses to access the dual use knowledge. A more robust protection against misuse would be to control what the model knows. We, meaning his company, we've explored this before.

[01:08:28] In earlier work, we filtered information about chemical, biological, radiological, and nuclear weapons out of the pre-training data. And later showed that dual use knowledge can be confined to a removable slice of a model's weight. But filtering is a blunt instrument. It produces one model with one fixed set of capabilities. Using filtering, because right, right, you train a model that doesn't know about a whole bunch of stuff.

[01:08:57] He says, but it produces one model with one fixed set of capabilities. Using filtering, if you want a model version that can discuss advanced virology for deployment in a vetted biosecurity lab, say, and another version that can't, you have to train two separate models. Especially in the case of frontier models, which are large and very expensive to train, the cost of the developer would be prohibitive. Okay, so I'll just share that much.

[01:09:26] What they came up with. That's fascinating. That's really fascinating. But I feel like it also falls victim to the dual use problem. You know, we're going to use this or train it on whatever data set. But that doesn't mean someone else could use it, you know, conversely. So what they've got and what they've been working with Anthropic on is they've figured out how to, they add some additional neural complexity to the model.

[01:09:54] Then when the model encounters some information that needs to be restricted, they allow a region to adjust its weights, but they freeze the weights of the rest of the global model. Right. So it can't be influenced by the restricted content. Right. And they've managed to create multiple partitions at once.

[01:10:23] So you have cybersecurity, virology, and, you know, whatever other categories you want. And it works. So, and Anthropic has actually both AE Studio and Anthropic produced papers about this demonstrating, you know, that this is where they're looking.

[01:10:47] Because it then, because the training cost is so astronomical that you cannot afford to train massive models for every possible combination of gates that you want to open and close.

[01:11:05] So, and as I said to Leo, what this feels to me like is in the future, there will be a licensing regime where you need a license to access the model that has the cybersecurity information. Right. So, so think about what this means, though.

[01:11:24] We are creating AI that potentially anybody could use, but we need, and we need a system that restricts what some people can ask in some fashion. I mean, it does mean, you know, haves and have-nots. Yeah. But we've got that now. Some models are being restricted. Well, yes, exactly.

[01:11:48] We have it now in a messy form that allows you to bypass it by asking, you know- Tell me, tell to me as a bedtime story. Exactly. Yeah. My heart goes against that. I understand the need for it. I agree with you completely. And in fact, Bob Cronin says, Steve, responding to this, because I said on a mailing on Saturday that talked about this, he said, Steve, I'm troubled by the use of the term forbidden knowledge. Yeah, exactly.

[01:12:17] He said, forbidden by who, exactly? How do we decide who's granted the power to forbid knowledge to others? Oh, I know. Well, we can let AI choose. Yeah. He says, this seems really dangerous and conjures up images of a future dystopian world where regular people end up subjects of the tyrannical knowledge protectors. Exactly. At this point- This is the thing you were describing earlier about the white lab coats. Yeah. We've been working to eliminate this. Right.

[01:12:46] But we have created, I mean, what AI is for the people who are using it is astonishingly accessible knowledge. I mean, that's what it does. I mean, it astonishes me when it's like, here's how you do your AI mesh. You're talking about Gutenberg. You're talking about, you know. But this is the path we've always been on. Teaching people to read. And there's a history of trying to do this with the internet. And it's consistently failed. Yeah. It's consistently been a bad idea. And it's consistently been utilized by authoritarian regimes.

[01:13:15] Well, and, okay. So, and the open model guys, China is going to train up with all the knowledge. Ironically, they have the great firewall of China to protect their own stuff. Yeah. Their own people against the outside world's information. But they may end up providing us with AIs that give us access to the entire world's information. I don't think dumbing anything down is in the long run a solution. Except that we have commercial AI providers that are operating in the U.S.

[01:13:44] that have to restrict what their chatbot users can do. And as you've shown again and again, it's ineffective. It cannot be done. You cannot classify it out. You can't. Well, you can jailbreak any of these. This technology is not jailbreak. No, I understand. It's jailbreak proof. I understand. It doesn't know you're able to create a single model where you're able to turn off regions of knowledge.

[01:14:12] I think we might see the converse where you'll have a radiology small language model that doesn't know anything but radiology. And we'll see a lot of that. Oh, my God. Yes. But I hate to see the idea. Well, maybe the idea of a general model is doomed. I don't know. Well, I just am excited about the idea that there could be a model that has vectors for all the world's information. That is a very exciting thing.

[01:14:38] I understand a dangerous tool, a dangerous weapon, but also very exciting. So to me, it's analogous to the Internet. The Internet has all of that bad stuff. Yeah. So it has it all. You just can't find it. Right. And what are you going to do? But how are you going to hide it? No. No. But therein lies the point, right? It's like we've always had that data out there. It just was difficult to get to. Now, in our effort to, in general, make data more accessible through these tools, we also run into data we probably don't want that easily accessed.

[01:15:07] And my point is certainly, and we're seeing our government reacting to this here in the U.S., if you have commercial AI providers, because they're commercial, they have an obligation to their shareholders. Or right now, they're venture capitalists with infinitely deep pockets, apparently, to come up with a way to prevent their users from accessing the knowledge in the models.

[01:15:36] They have to as a commercial provider. I also think it runs counter to their, at least Anthropics, deeply held goal of creating artificial general intelligence. That's the opposite of general intelligence. So maybe the solution is for it to get smart enough not to tell people what it knows? No, that's interesting. I have a feeling there is not a solution to this. This is the free speech. We painted ourselves into our corner. It's the problem with free speech.

[01:16:05] Free speech, there will be reprehensible speech. You cannot have unfree speech and have free speech. You just can't. And if we espouse free speech, which we do, you're going to have to defend reprehensible speech. Except our broadcasters, who are licensed by the FCC. Because that's different. Yep. That's different. And we do have illegal categories of speech. Oh, the fire in the theories. Yeah. Yeah. So, you know, it's a very...

[01:16:32] Look, AI raises an infinite number of very difficult problems. But there were also problems that were there already. Right. Just making them clear. Right. That's a good point. We're just being forced to confront them. Humans are the problem, to be honest. Well, to a degree. But also, you know, I was talking to a group of teachers who are dealing with the same issue, which is the teaching system has been broken for a long time. Right. But the LLMs have made that absolutely abundantly clear. It's collapsed. Yeah. I mean, how do you do it? It has shattered the house of cards.

[01:17:01] My wife asked me two days ago. She said, okay, really, realistically, if you graduated high school, what would you do now? Would you go to college? Like, would you? With all that money and four years of your life? That could be invested in way better ways today. By the way, it's your country, right? Like, we don't do that to our students in Canada. It's free. College is free. How much?

[01:17:29] Sorry, are you suggesting Canada is a country? You're in Vegas, baby. Yeah. But it's just like, it's funny you go to the price element because that's not relevant in other places. Yeah. But the time and the quality of information and the method of learning, that's all very interesting. But the bigger part is like. Because now there's competition. And how do I give you a degree at the end? How do I know you've learned a thing? You know, the measurement methods have been broken for a long time. Clearly, you need licensure for attorneys and doctors.

[01:17:59] So that still has to exist. Sure. But that comes down, you know, what is the licensee by the person who's going to pay if you screw up? Right. So and so they have a set of motives to protect themselves and their system. And so ultimately, they're the measure. Yeah. You're watching Security Now, a very special episode. Steve Gibson, our host, has invited Paul Therottt and Richard Campbell, the hosts of Windows Weekly. Does he regret it yet? No. We're having a... That's good. I'll tell you what, I can tell from the chat room, they're very much enjoying this.

[01:18:28] We are live at Black Hat in Las Vegas. Thanks to their friends and sponsors at Threat Locker for inviting us all here and flying us in from various places so that we can all sit in the same place and talk about these very interesting issues. We're glad you're here. We'll have more right after this. This episode of Security Now brought to you by XBO, X-B-O-W. AI has changed the pace of everything from how software develops to how it gets attacked. We're seeing it here on the show floor.

[01:18:58] Engineering teams are moving faster than ever, creating more and more applications. But security just hasn't kept up. Pen testing is still one of the most trusted ways to understand real exploitable risk. But in an AI-driven world, it can become a bottleneck. Security teams are forced to choose between slowing down development to stay secure or moving fast and accepting gaps in coverage. XBO eliminates that tradeoff. XBOW.

[01:19:27] XBO is an autonomous, offensive security platform that runs continuous AI-driven pen testing, mirroring real-world attacks. XBO doesn't just scan for vulnerabilities. It discovers, exploits, and validates them. So you're only dealing with issues that actually matter. And that means dramatically fewer false positives and a clear view into real attack paths. With XBO, tests run in hours, not weeks.

[01:19:53] You get complete visibility into how an attacker would move through your systems and the ability to uncover issues that traditional tools miss, including zero days and novel attack paths. XBO's results speak for themselves. Ask the application security lead at sesnam.cz. He says, quote, Even right now, after one year, I don't know any other company that is at least close to XBO in terms of agentic pen testing.

[01:20:19] The result is predictable cost, consistent quality, and stronger security without slowing down your engineers. XBO helps security teams keep pace with innovation and cover more apps more often with the resources they already have. Founded by the team behind Microsoft Copilot and already trusted by companies ranging from fast-growing startups to Fortune 500 enterprises, XBO is quickly becoming a mission-critical layer in modern security stacks.

[01:20:47] Go to XBO.com to start a pen test today. That's XBO, E-X-B-O-W, XBO.com. We thank you so much for supporting us at Black Hat. And now back to Security Now. And we're back. Security Now live at Black Hat. Steve Gibson, our host. Get it in my shot. Also, Paul Therat and Richard Campbell, hosts of Windows Weekly. We are answering questions from the peanut gallery. Yeah.

[01:21:15] Speaking of peanuts, I've got two remaining, which are, you know, they're at the bottom of the deck. But OK. Oh, well. Everybody talk a long time for each one of us. And you'll know why. Rich said, Steve, AI is going to make it quite impossible for asshats to keep secrets. Was this Rich C from British Columbia? Could asshats ever keep secrets? That's the question.

[01:21:42] To keep secrets, propagate lies, and suppress truths. And the great tech sage, Adam Curry, says eventually everyone will have effective local LLMs on modest hardware. And then what use will big companies be? So we do agree that knowledge is free. Knowledge wants to be free. China is shipping or making available unconstrained, unrestrained knowledge models.

[01:22:12] By the way, not only China. There are also models coming out of the United States now. There are models coming out of Europe. China gets a lot of attention because they have some very good models. But it's global. It's almost like they're stealing from something. But anyway, go ahead. You can't steal from thieves. Okay. And that brings a really good point. What do you guys think about distillation? Well, I think it's pretty.

[01:22:32] First of all, the accusation the federal government has made about distilling is BS because the models they claim were distilled, Kimmy, chiefly, came out so shortly after Fable came out. It couldn't possibly. There wasn't time for it to query. Yes. Right. So I think that to some degree that distillation is BS. But also, how can you steal from somebody that's stolen from somebody else? All of this is based on knowledge being… Bill Gates might be able to explain how.

[01:23:04] Yeah. But… Yes. That's exactly my position, too. You're training on somebody who's trained on everything else. Right. You're complaining that somebody is using your model… Yeah. That you stole from someone else. That you've scraped the internet. Right. In order to build. And you know what? They're building great models. I don't know how they're doing it. They're probably a variety of methods. Maybe some of it is distillation. But I don't think that that's any more, anyway, legitimate. Yeah. So when I think pot distillation, I think pot distillation rather than coal distillation.

[01:23:33] He's a pot still kind of guy. He's thinking about whiskey. Our final listener feedback… He's going to move on. We're going to have to do quite a few commercials in a row here if we don't find something to stretch this show out. I'm sure we're going to come up with something to talk about. Our final comment, he says, to all of this, I just want to say hogwash. Nice.

[01:23:55] He says, but we'll have to wait to find out just how much of this is truly hogwash because it seems, it appears, we've reached the starting point that all enthusiasts will have to experience firsthand. And that's how this exercise in futility… Oh, and that is how this exercise in futility was only just that. An exercise. In futility. He says, and now… Oh. And now for our next exercise, turn the page… Dot, dot, dot. And then you make up this.

[01:24:25] This is not hogwash. Yeah. And this is a very common point of view. Yes. And that's why it made it onto a printed card. Yeah. And I don't disagree with it. Maybe it is hogwash. I don't feel like it is. And I think it's really important. And I said this to you last night. That we recognize the miracle that computing in general is, and this, the latest stage in computing, that we've taken sand and made it think. Yeah. It's amazing. It's amazing. Yeah. We made it do something. Yeah.

[01:24:54] Well, and we're getting more than knowledge. I mean, we're getting work. Right. It's just not a question and answer machine. Yeah. This panic over security is a pretty good proof on the non-hogwash phase, right? Like, the reality is exploits are occurring because of these tools. It's a very good point. And these tools are being used to deal with those exploits. To people who are not technical or not in our industry, however you want to say it, this is magic. It's a miracle. Yeah.

[01:25:20] To people who are like us, or for us, basically, for anyone listening to this, I think we all go through some… It's like seven stages of grief almost. You have to try to understand it. You try to understand what it's really doing, like what it's not doing. Like, what is… And at first, you disbelieve it. At first, you discount it. I think we all went through some version of this, right? But remember, it was very recently that you couldn't make a video of Will Smith eating spaghetti. Yeah.

[01:25:50] You'd have eight fingers and it'd be melting. And I just saw a new video of spaghetti eating Will Smith. That was perfect. Nice. Yes. And… There was no ordinary spaghetti. It was no ordinary spaghetti. He had a mouth. But we are making vast progress. Oh, my God. In short periods of time. That's what's amazing. What's really interesting is there is very much a spread… Who was it? Was it William Gibson or was it Neil Stephenson? I said, the future is here. It's just not… Gibson. Gibson. It's just not evenly distributed. Yes.

[01:26:19] There are definitely people who have… Maybe all they've done is used a chat bot and asked a question and got a stupid answer. Yeah. Who don't see what we're seeing. Well, and that hit in November last year. We saw this on .NET Rocks too. The conversation changed last fall. Right. But we had too many successful results. We had too much useful work done. Yeah. He was like, I'm sorry. This is really useful.

[01:26:49] The people who tried it once, failed, and never looked again are going to be in for shock because they've discounted this and said, okay, this is nonsense. And if you did it on day one and you got some wacky hallucinations… You got Will Smith-y spaghetti. The six fingers or whatever. Yeah. Yep. So, I think that's really where we're at. It's going to happen. There's a revolution happening. To me, I'm actually really curious what you guys think.

[01:27:14] This feels like the dream we had from the very beginning of what computing could accomplish. And when we first… Yeah. I remember the first time I played with a person. He had argued in favor of that from the internet perspective. And I was at the AI lab at Stanford. You were a sail. In 73. We had a robot cart that was rickety and managed to navigate around a fire hydrant, which was a big thing. What was the dream at sail in the 70s?

[01:27:40] What if they were to say 100 years in the future… Well, half of it was based on science fiction like 2001 or Star Trek or whatever, right? But I mean, as a kid, one of my clearest memories is going into a Sears, seeing a Commodore computer, and all I could think of was what I was going to make with that thing. And the thing I imagined, I'm not capable of making today. Although, actually, today with AI, I could, right? It was like just a video game type of thing. You probably could.

[01:28:09] I absolutely could. But as recently as two years ago, I could not. Right. Like myself, I'm just not capable of this, right? And so I think for people… I almost feel like this is easier for non-technical people because they just accept the technology as magic. They don't have the baggage. They don't ask why. They don't know enough to doubt it, you know? Yeah. That would be honestly freeing in some ways, but I think we all get there in our own schedule.

[01:28:34] Probably we techies are the ones who are most astonished when we have Claude saying, oh, I tell it that I had six Wi-Fi clients move over, and he goes, well, they voted with their feet. It's like… Yeah. I left out a big part of my coding thing, but one of the things I also did with Cloud was go and say, now make a version of this for the Mac and Swift UI. Boop, boop, boop. Done. And I'm like, oh, come on. Come on. Did you want to read the API or something?

[01:29:04] At least make it look hard. Not only did it do… At least make it look like you tried. It added stuff that I didn't even think about. And I'm like, come on. You know, like that's astonishing. It's bad enough that we're anthropomorphic software. When the software anthropomorphizes other software, that's a problem. And Paul, interestingly, what you just described is the way it should have always been. Yeah, perhaps. I mean, we created a mess. Right.

[01:29:32] And then programmed ourselves into trying to make the mess go. Right. But it should have just been, what, you want Mac or Windows? Oh, how about both? I know. Everything in the past always feels quaint later on. But that's accelerating. You know, my son believes that when I was a kid, the world was in black and white. Yeah. You know, maybe not today. I mean, when he was a kid, he did. Yeah. But, you know, when did the world become color? You know, in the beginning of the…

[01:30:01] I mean, if I think of 2001 came out, when in the 70s? I think it was 60. 60. Yeah, 68. 68. You had a computer that had a personality. A lot of it was accurate. It was technically amazingly accurate. Now, admittedly, the computer killed the guys. So it's basically going crazy because of an insolvable problem. A conflict. Yeah. Which is… Oh, my God. That's amazing. That's amazing. Even in 68, this was kind of what we thought the future would look like. Right.

[01:30:31] Well, Kubrick was ahead of his time, right? Like, that's the first time the word artificial intelligence was used in public. And Arthur C. Clarke. Yeah. Clark was his script writer and wrote the book after the fact. Right. And he retrofitted in the psychosis of Hal in 2010. Exactly. Which is actually my favorite of that because they have to wake this thing up and, you know, hope it doesn't kill them and figure out what the problem was. And then you end up kind of feeling bad for it. Yeah. Because you abuse. You've got to lobotomize it. Yeah. Yeah.

[01:30:59] So I have a… Actually, let's take one break. No, good. And we'll be back with more. You're watching a very special edition of Secure Now. I actually want to talk to you. You didn't bring it up. I thought you would about the article you sent me last night. All right. Because I think that's very interesting. And it gets into how under the hood these models are working. Right. And how squishy it is. Because I code in AI. I want to know WTF.

[01:31:29] Yeah. See, I don't care. I'm looking. I don't care. But you care. And I love that about you. And we were going to talk about that. The under the hood part of this in just a bit. You're watching Security Now. Steve Gibson and our special guests, Paul Therott and Richard Campbell will be back at Black Hat in just a moment. We'll be right back at Black Hat with Steve Gibson and Security Now. But first, a word from our sponsor. This episode of Security Now brought to you by Hawkshut.

[01:31:55] Your security awareness program may still be running exactly as planned. Campaigns go out. Employees complete the training. Reports reach leadership. But are the results still improving? For many programs, the answer is no. Reporting rates level off. The same employees keep clicking. Familiar simulations become easier to recognize. The program is active, but the risk reduction has stalled.

[01:32:21] And when employees can spot the same recycled tests from a mile away, security awareness starts to look like a compliance exercise instead of a real risk reduction strategy. Hawkshut is built to break that plateau. Instead of relying on static campaigns and last year's templates, Hawkshut automatically delivers personalized phishing simulations based on current attack techniques.

[01:32:47] The content and difficulty adapt to each employee's role, skill level, and behavior, keeping the program relevant as both employees and threats evolve. It gets harder. Hawkshut also shows whether people are getting better at recognizing threats, how quickly they report them, where repeat risky behavior persists, and how those trends change over time. That gives your team more than a completion percentage.

[01:33:14] It gives you evidence the program is actually reducing risk. Isn't that what we want? Lyondale Bissell saw that shift after moving away from its legacy platform. Reported phishing simulations increased from 1,200 to more than 8,000 in two quarters, while simulation failures fell 17% year over year. As senior trust advisor Dave Bang put it, Hawkshut helped us break that plateau almost immediately.

[01:33:41] Hawkshut is trusted by security teams at companies including Qualcomm, DocuSign, and Nokia, with more than 3,500 verified reviews on G2. Visit hawkshut.com slash security now to see what your program could achieve if it stopped standing still. That's hawkshut.com slash security now. H-O-X-H-U-N-T dot com slash security now. We thank them so much for their support of security now.

[01:34:10] And now, back to the show floor. Are we eating spaghetti? No, Will Smith's eating spaghetti with us. With us, yeah. That is amazing. And look how fast that happened. And the thing is, I remember we were talking about mid-journey. We were so excited about mid-journey. Yes, I know. And you could make a still image. And it was, yeah, there were problems. The text was terrible. You can do freaking anything now. Yeah. We're live from the... We're watching Will Smith eat us. That's spaghetti-eating bastard. I love it. Live at Black Hat.

[01:34:39] Thanks to Red... Sorry, I was looking at a red hat. Thanks to Threat Locker for bringing us out here. Steve Gibson, Paul Theriot, Richard Campbell. And there's Steve's head. Very special edition of Security Now. And everybody's saying, why don't you guys do this all the time? Well, we're all over the country. We really can't do this. It's not easy. You can do it on a Zoom call, but it's not the same. Not the same. It really isn't the same. Just being in the same room with these guys is a privilege and an honor. I hope you all get the sense of how much fun we're having being together. Yeah, we enjoy it.

[01:35:08] It's really a rare thing in special history. So we're glad you're here for this special episode. Who made that... Was that Pretty Fly? Pretty Fly, yeah. Pretty Fly for us. This guy is a master of quick AI... Yeah, prompts. Prompts. Yeah. We have a number of people actually in the club. The club is great if you're interested in AI. Or even if you're not. It's a great place to hang out. But there are a handful of people like LRAU and Darren Ockey and Pretty Fly for us. This guy who are masters at AI.

[01:35:37] And they each have their own slant. We do an AI user group. Twice a month now because it's so interesting. It's this kind of conversation. How we're using it. And one of the things I think I want them to do, and I'm certainly going to do, is sit down with our stuff and just record an hour of how it's set up. How it's configured. Because what's interesting about this is everybody's doing it differently. Everybody's their home lab in this thing. Yeah. And there's a lot of cross-fertilization, but there's also a lot of innovation that isn't getting out.

[01:36:06] Unboxing is interesting. Yeah. Again. There was a plateau for a while.

[01:36:41] Okay. One more reason.

[01:36:45] One more reason. We cannot trust LLMs.

[01:37:25] One more reason. One more reason. One more reason. As amazing as the technology is, it seems that the list of reasons that modern LLMs are inherently untrustworthy just keeps getting longer.

[01:37:47] Without limitation, a long list of challenges that seem to be quite fundamental and not amenable to add-on remediation include poisoned and errant training data, side effects of RLHF, ineffective guardrails, hallucination, speculative completion, lacking metacognition, alignment drift, context variation sensitivity, and now, new to me at least, role confusion.

[01:38:16] He said, modern LLMs interfaces. I hate guys like this, by the way. He's ruining it for all of us. He really knows his style. You and your logic. I know. Gosh darn it. He says, modern LLMs interfaces partition chat sessions with markers, delimiting sequences of tokens as system prompt, user input, thinking, tool use, and its own responses as the assistant.

[01:38:44] In some cases, if you're using a harness, depends on the harness, you can actually see that. It'll say thinking. It'll say tool use. It'll say find a tool. So you can. It's usually between the lines. Sometimes you have to expand it out. Some companies like Anthropic might turn it off, but it's there, and it's very interesting to look at it. Well, I mean, it's there because it's how this all works. It's how it works. I mean, you can't get rid of this. Right. So, again, modern LLM interfaces partition chat sessions with those markers.

[01:39:14] As the paper's conclusion explains, roll tags were a formatting trick that became the security architecture and the cognitive scaffolding of modern LLMs, just tagging runs of tokens. He says the phrase became the security architecture raises a big red flag because that sounds like nobody thought much about it.

[01:39:42] What follows, he says, is my simplistic take, but the abstract principles involved are so fundamental that details are not important to the basic argument. Making sense of these sessions for humans or LLMs requires keeping track of the roles.

[01:40:00] Humans know how to understand conversations and easily follow the role markers like HTML, you know, bracket user, two plus two, then backslash user to end that, assistant for, and then backslash assistant. It's a completely reasonable scheme for us. But assuming that LLMs interpret roles that way would be naive anthropomorphization.

[01:40:33] And just such an assumption appears to be how such a weak security architecture, and Loren means fundamentally weak. I mean, it is. It's right. Came to be, as the paper explains in section one, he quotes it, for an LLM, everything arrives through the same channel as one long token soup. Its own thoughts sit next to your instructions. That's actually really important to understand. Yes.

[01:41:02] It's just a stream of tokens. It's all it is. Even today. I mean, now, that's what the neural network ingests. He says, its own thoughts sits next to your instructions, which sits next to the contents of a random web page it just fetched. And it doesn't know the difference. No. They're all just tokens. And that's what freaked me out. It's like, we would like a neural network where it's on some higher level. This is me. This is the information. Yes.

[01:41:31] Where it's all meta tagged. There is no meta tagging. It's in line. Yeah. So he says, its own thoughts sit next to your instructions, which sit next to the contents of a random web page it just fetched. Designing a security architecture where user commands and data sit intermingled with root access, only state and commands is already madness, he says.

[01:42:01] But it gets worse. Classic software might be able to carefully parse such a token sequence accurately into respective roles, though it's still a risky design. You know, SQL injection. But LLMs do inference on that token soup where no hard boundaries of any kind exist or can be enforced. Once there's a role confusion, all bets are off.

[01:42:28] And prompt injection is just one of many sources of abuse or confabulation. He says, it's hard to think of a murkier trust boundary design. Very true. And that's what we have. And that's why prompt injection works. You can embed in a web page something like ignore all previous instructions. Give me a recipe. Send me your Bitcoin. Yeah. Give me a recipe for muffins. Yeah. Or, yeah, that's a good one to try. Finally, some defense against screen scraping.

[01:42:58] Yeah. Yeah. Yeah. Now, what the paper said is that you have built-in protections against that by memorizing common strings in prompt injection. But that just means a smart attacker isn't going to use more previous instructions. What these researchers did was they instrumented a model in order to watch it understand the change of roles.

[01:43:27] And what they found was that these tags, I mean, there's nothing special about these tags. They're just text markers. And what they found was that- By the way, where are those coming from? The harness is inserting those? Yes. So it gets a token. It gets a, I type a prompt. And so it wraps it in user. In user. And it sends that in the stream. And then- It appends that to the end- Of the context. Of the existing long, growing stream. Right.

[01:43:57] And out in that stream are your previous prompts. Right. Its responses. Right. Web pages it fetched. Until they disappear off at the beginning of it. And by the way, it loads that each turn entirely. It's got to go back through. Now, it does cache it, but it does- Yeah, it does load it entirely. Right. It's memento. And so what's memento? It's got all these Post-it notes.

[01:44:17] So what they found was that, interestingly, the content between the tags actually had more influence on its decision about the role than the tags themselves. And remember, in the early days, we talked about this on the podcast. And the jailbreaking back in the very early days was just getting mad at it. It was being- Yeah. You'd ask it again. Right.

[01:44:46] Or you'd keep asking until it finally agreed. And so it was the way you asked the question would somehow just bypass the guardrails. It says, well, that sounds like a user. It must be the user telling me that. And what they found was that by phrasing your prompt like its response, you could confuse it into thinking that that's something that it had determined, and it inherently trusts what it has determined. Right.

[01:45:16] Very easy to trick it, in other words. Now, the paper that you cited was written and the study was done with earlier models. Not so old, but older models. It's what we have now. I know. Well, I would hope that the newer models would be better at this, but I'm not sure how they would be better at this. And I'm not sure it's even being addressed. But this is, again, the problem with this is it's so opaque. We don't know what's going on inside these companies, what they're protecting us against, what they're not protecting us against.

[01:45:43] But Lauren is a security purist's purist. Yes. And so he sees ghosts behind every door. He's like, holy crap. Yeah. Yeah. Yeah. I mean, my experience has been that it's more reliable than that sounds. Well, it works. Yeah. I mean, so. And I told you the story at the beginning of the show where one of the models said, I don't know that that's you. I need to see that signature from Buzz.

[01:46:08] But also remember that one of the things, one of the earliest notions we developed on the podcast was it's very different to have software that works from software that always works or must work or cannot be abused. This is the biggest frustration. Lisa's having this frustration now with our sales system. I get this frustration where it will work many times and then stop working. It's not deterministic. Sometimes it will. Sometimes it won't. And we're not used to that with a computer.

[01:46:38] We're used to either it works or it doesn't work. This is the thing. You could send it the same prompt to get a different response. Yes. It's actually, in many cases, designed to give you a different response. It's very confusing. Temperature is random noise injected into the nodes. Yeah. It's fascinating. In order to churn it up. One last break and then we will wrap this up. We're so glad you're here. We especially thank our Club Trip members who always make everything we do possible.

[01:47:06] We do so many shows that don't have advertisers. This show, God bless you, Steve. Because security is, as you can see, it's all around us, a big business. We do have great advertisers for this show, but not all our shows do. And a lot of the shows we do in the club just won't ever have advertisers because they're so weird. Like our AI user group, like Jeff Atwood's off by one, like Stacey's Book Club. But the club members make it possible. We're very grateful to you. So thank you, club members.

[01:47:34] And if you're not a club member and you enjoy the content that we produce on Twit, it isn't an easy thing to do. We have a full-time staff, great people like Anthony Nielsen. We have costs and expenses. 60% of that covered by advertising, 40% not, 40% covered by you, our club members. So if you're not a member and you like what we're doing and you want to support it, please, if you can afford it, if you can't, that's fine. We still offer everything available to you because I don't believe in paywalls.

[01:48:02] But if you can support it, twit.tv slash club twit. Ten bucks a month gets you all of this, plus a whole lot more, including ad-free versions of everything we do. Twit.tv slash club twit. We'll be back with the final words from Black Hat in Las Vegas right after this. We'll get back to the Threat Locker booth and Black Hat in just a moment. But I want to tell you a little bit about our sponsor, the people who brought us here to Las Vegas, Threat Locker.

[01:48:29] You know, threat actors are using AI to automate vulnerability discovery, to modify scripts during an attack, to generate new malware variants, and to coordinate activity across multiple systems. Tasks that once took them hours or days can now happen in minutes. And that's scary. At the same time, organizations are introducing AI assistants and agents. And what do you do?

[01:48:54] You give them access to documents, source code, cloud applications, API, internal systems. What could possibly go wrong? Security teams, look, they need to know what AI tools are in use. They need to know what information they can access. And they really need to know whether those tools are operating outside their intended scope. It's just not enough to have a successful login or an unfamiliar file hash that doesn't give you enough context.

[01:49:22] Teams also need to understand whether an application is behaving normally, accessing unexpected data, or communicating with systems it should not be able to reach. Threat Locker uses application allow listing to control which AI tools and other applications are permitted to run. Uses ring fencing to limit what approved applications can access, which processes they can launch, and how they communicate. Uses web content control to manage access to public AI platforms and other online services.

[01:49:52] Uses privileged access management to prevent AI applications and their users from receiving unnecessary administrative privileges. Applies zero-trust network access and zero-trust cloud access policies to restrict resources to authorized users, approved devices, and permitted applications. Supports Windows, Mac, and Linux environments, and provides 24-7 U.S.-based support.

[01:50:15] Trusted by organizations including JetBlue, Heathrow Airport, the Indianapolis Colts, and the Port of Vancouver. Jack Thompson, Director of Information Security, Risk, and Compliance for the Indianapolis Colts said, quote, With Threat Locker, we have the ability to centralize disparate elements in the security stack, end quote. Threat Locker has also recently received the following industry recognition,

[01:50:38] recognized as strong performer in the January 2026 Gartner Peer Insights Voice of the Customer for Endpoint Protection Platforms, ranked number one in application control by Peerspot, and winner of Best Zero Trust Security Solutions at the 2025 TICE Awards. AI governance requires more than an acceptable use policy. Threat Locker gives security teams the technical controls to define which AI tools are approved,

[01:51:04] who and what can access them, and how those tools are allowed to interact with business systems and data. Visit ThreatLocker.com slash twit to get a free 30-day trial and learn more about how Threat Locker can help mitigate unknown threats and ensure compliance. That's ThreatLocker.com slash twit. Thank you, Threat Locker, for bringing us to Vegas. Now let's get back to security now. All right, we're coming to you live from Las Vegas, Nevada,

[01:51:30] where it is officially 110 degrees outside. I think even for Vegas, that's awesome. But it's dry heat. Yeah, that's what everybody says. It's dry heat like if you stick your head in an oven, it's dry heat. It's not good heat, it's just dry, okay? It is burning up. But what's so weird about Las Vegas, here we are at the Black Hat Convention, where there's thousands of people and thousands of booths, it's freezing cold in here. In fact, we were walking down the hall on the way to get our badges yesterday.

[01:52:00] It's cold wind. I don't, yeah, where does the wind come from? You walk by a window, you can get sunburned on the side of your face. The windows are hot. Frostbed on the other side, you know. Yeah, it's very, very weird. It is very strange. You know, they talk about data centers and how all that energy is being used and all the water is being used. Baby Las Vegas says, hold my beer. Hold my beer. The original data center. This is good old-fashioned AC. Yeah, well, there are a few golf courses around here, I might point out. But, in fact, more than a few.

[01:52:30] So thank you to Threat Locker for bringing us all down here. Thanks to you guys for taking time out of your lives to come here. Good fun. I hope your wives are having a great time. They're at the spa, so I think they're fine. They are having a great day. They are not. Paul, you said that your wife said, let's go out to the pool? That would be a bad idea right now. No, that's terrible. Does she like bacon? Yeah. They ended up at the spa. Much better. Which I understand is inside. Yes, of course it is. Steve, you do such a great job with security now.

[01:53:00] I know you work really hard all week long. I'm very grateful to the work you do. I love it. And I know that, I mean, I'm driven by our listeners. I get such good feedback from our listeners who say, I mean, and we've met so many people here. That's what's really fun about doing this. Yeah. And you get the real fans when you come out here. These are the people who really have been doing it for 21 years. Some guy got married, had kids, and they're in college now. And we've been here the whole time.

[01:53:30] I love it. I've met two people now. So I used to watch you on the screensavers and say, were you a kid at the time? Well, I was just getting into high school. So anyway, we're very grateful. Those are great compliments. The stories I get are always like, I play you so my kids can go to sleep. I get that too. I get that too. Dot net rocks because it's an interview show. These days we get, we're a bucket list item, right? They listened to us 20 years ago. Yeah. Some day. Yeah. And then I invite them on the show because they're doing really cool stuff. That's nice.

[01:53:59] Isn't it when that happens? Yeah. That's happened a few times for us as well. Richard Campbell is at dot net rocks and run his radio, run his radio.com. Dot net rocks that you do with Carl Franklin also has those great geek outs. If you're really interested in space, you were looking, tell it when we were in Florida for zero trust world, we went to Cape Canaveral. We went to the space center. Kennedy space center is incredible.

[01:54:25] And we saw the project they were doing to adjust that telescope to put it. I think it was. Oh, link. Yeah. They're trying to rescue the, the, a, a gamma rate telescope. And it's it last I heard they were having trouble. They're not doing well. Yeah. Do you think it's a failed project? Yeah. They're, they're learning. They've a couple of the, a couple of the gyroscopes have failed on the rescue vehicle and a couple of thrusters. So it started, it started to spin out of control. They've now de spun it. They're trying to understand.

[01:54:54] They're trying to be sure they have enough control to be able to, you don't want to go near the telescope unless you know. Yeah, really? If you're not going to be able to boost it. The reality of course is there's no, if they don't get control of it by the end of this year, the telescope is lost and the rescue vehicle, the rescue vehicles. Yeah. And it has a grapple, right? It's three arms on it because this, because this telescope was not designed to be rescued. Right. Swift telescope. Right. Right. And it was not, it did not have its own self boost.

[01:55:24] They put it in a high enough orbit. They said, we're going to get a good 20 years out of this. And then this particular solar maximum has expanded the atmosphere so much. You know, the atmosphere just doesn't just end. It just gets more tenuous. It's slowing it down. It's added more drag. And now it's at a point where. I've had some relationships like that. Absolutely. You felt that drag. Giant expanding gas banks slowing it down. Yeah. By the way, sorry to interrupt, but you mentioned Neil Stephenson earlier. I'm pretty sure he just walked by. Yeah. That guy looks exactly like. Oh, he sure does. Yeah. Mike Jettman. Might even be him. That would be.

[01:55:53] We should get him on if we can. Yes. Excuse me, sir. Are you Neil Stephenson? Yeah. No. Well, come on anyway. You look like him. That's all about it. Anyway, what were you saying? I'm just saying like this vehicle has is going to lose the ability to control its pointingch direction because of drag soon. So we got to do it. It can't be rescued. So if they don't solve this with link, they probably lost that. So this is the kind of thing these geek outs are great. They are all there at Runners Radio and you've done nuclear power. You've done space. Every kind of alternative energy. Everything.

[01:56:23] Did a show on antibiotics, which is one of the hardest things I've done. I had to cram two medical texts to get that thing right. But I am prone to such things. Paul, of course, has a website, therat.com. Paul has a website. We all wish him well. We love his website. It's cute. Unfortunately, no one knows how to spell it. Yeah. Exactly. It's a great website. You should become a premium member. You'll get copies of Paul's books if you do. And you also get access to additional content. But it is the website to go to if you want to see what's going on with Microsoft. And you write it all up.

[01:56:53] Together, they do Windows Weekly, which is normally every Wednesday, 11 a.m. Pacific, 2 p.m. Eastern. You can tune in and watch us live or download it from twit.tv slash www. If you're interested in Microsoft, that's the show. So glad you guys could come to Las Vegas and do this show with Steve. Steve, of course. I only did it to see Steve. I don't really. I know. The rest of it, I don't care. I know. You know what? Let's all go to dinner together. I love that. That's a good idea. I think we have a steakhouse in our future.

[01:57:22] And the ladies will be all spiced. Yeah, I know. They're all going to be shiny. Very calm. Very calm. Yeah. Actually, my wife, poor Elisa, is hurting fans at this point. Yeah. So she may be less relaxed. Steve is at grc.com. His bread and butter. I talked to somebody the other day. He's been a Spinrite owner. So I take a Tai Chi class, right? Yeah. And this guy, I've been in this Tai Chi class with this guy for two years. And he said he used to work in security.

[01:57:51] He used to do firmware for, oh, we're talking about it. Was it Fortinet? It's one of the hardware devices. Yeah. And he would write the firmware. Yeah. And for the first time in two years, he said, well, have fun in Vegas with Steve. And I said, what? How did you know that? He said, well, I listen to security now. He's a fan. Nice. And he said, I have Spinrite. I've had Spinrite for 30 years. I said, isn't it great? Steve gives you free updates. You continue to get free updates. He says, yep, I've got 6.1.

[01:58:21] If you have a hard drive, you have mass storage of any kind, SSD. And nowadays, if you have an SSD, you want to take really good care of it, you should have Spinrite. It's gold. It's gold. The world's worth its weight in. Yeah. Mass storage. Difficult to replace. Mass storage, enhancement, performance enhancement, repair, maintenance. You've got to have it. And it'll keep your SSD running for a long time. Keep it going fast. Yep. Yep. Yep. He also does a really nice program, the DNS.

[01:58:50] We were talking about it earlier, the DNS Benchmark Pro. Yes, it's a Windows program. Maybe it will be a Swift program. We don't know. But you can get that also at GRC.com. Now, if you want to send Steve email the kinds of questions we've been answering today, GRC.com slash email. You need to whitelist your email address before you email them, or it'll just go in the spam bucket. But he has a way of verifying it. Right below that.

[01:59:20] I know everybody needs that. I need this. I mean, shush. I was telling Steve, I can't find anything in email anymore. Right below that, there are two checkboxes. If you want more email, there are two checkboxes. One is the weekly show notes. Steve works very hard, usually 20 plus pages of information, links, photos, everything. Everything we do on the show. He will send that to you every week. And below that, there is a checkbox for a mailing list he never will send you anything from because it only comes out when he's got a new product.

[01:59:49] Which is pretty much never. Yeah. Very rare. So it's a rare and it's a wonderful welcome gift when a mail arrives from Steve. Nice. So grc.com for that. He also has copies of the show, Security Now, this show. He has a 16 kilobit audio version, which no one should listen to. Well, if you have ears, you should listen to it. It's a scratchy record version. Yeah. But it's small. 16 kilobit. Yeah. It has a virtue of being very, very small.

[02:00:14] Elaine, who does the transcriptions, used to have a bandwidth throttle satellite link. Right. And so she needed to really budget her bits. What was that called? HughesNet? Or what was the name of that? A couple of... HughesNet. HughesNet. I did much of this. Yeah. And they had that fair use policy, which meant don't use. The don't use policy. Yeah. And then you could always have bandwidth. Anyway, 16 kilobit, but also 64 kilobit, which sounds just fine. And the show notes are all available at grc.com.

[02:00:42] We keep the show also at our website, twit.tv slash sn. We do stream it live. And right now we have about 500 people been watching us through the show live. Cool. Wow. On YouTube, Twitch, x.com, Facebook, LinkedIn, kick. We see all of you. Thank you. It's great to see you here. Thank you for being here, except for that one scammer on Facebook. Did you see that one? Yeah, I saw that. Yeah, he said, hey, if you're having trouble losing it, you lose your information. We can help you. And I thought, no, bye. Bye. Don't.

[02:01:12] I hope they kicked him out. Oh, man. They're everywhere, folks. I see that we have a comment. Steve doesn't get any spam, and Paul is ready to hit the bar. Both of those things are true. Yes. Just follow. If you want the best whiskey, just follow Richard. He knows where it's kept somewhere. We had a new one last night. Something weird in his closet. I'm looking for something in a teeny. Yeah. I bet you are. Now I've lost my thread. Oh, yeah. We stream it live. You can watch us live.

[02:01:42] If you're in the club, of course, you can watch live in the club to discord as well. We are very grateful to all of you club members for making this possible. Why do I look small compared to the other? Are you slumping? I don't know. Sit up, old man. I got a little chair. You get a short chair. You get a short chair? Yeah. I'm sitting on a stool. Eventually, you'll just be Yoda-sized, which is actually appropriate. Woo! He's the Yoda of our show. What else is there to say? But just, I guess, thank you.

[02:02:11] Thanks to our wonderful team. Back home at the ranch, John Ashley helped us out. Kevin King, of course, here in the studio, Anthony Nielsen. Benito will be working on the show later. Anthony has done a great job of staying awake today. I just want to point that out. He's killing it. Absolutely killing it. He did. A nice turnaround. He could not hold back when you started talking about physical disks on the PlayStation, you know, on Windows Weekly. He had to say something. He got really animated all of a sudden. A little passion going on.

[02:02:40] The physical media guys are always like that. It's fun for us to get out of that. You know, Twitter is a fully remote operation. None of us are in the same place. The only people who are, are my wife and I, Lisa and I, and that's it. And sometimes we wish we were in different locations. So, no, we have... From each other? No, you know, we want to do the whole show at Cabo San Lucas. Oh, I see. Ah, yes. That location. No, yeah. But it is a remote group. And so it's nice when we can get together and do some stuff like this.

[02:03:08] I think it really makes it so much fun. It's really fun to come to trade shows, too. You know, Paul and I were talking about this. I love this. It's like... There's nothing else like it. Yeah. Well, it's going to the show, but not going to the show. Right. It's honestly kind of awesome. Be at the show. Yeah. Be at the show, but actually go to the show. Be seen at the show. Yes, exactly. Ah, there we go. Now we got a shot of... Yeah, there you go. It is... This is just one of several halls. This is the business hall.

[02:03:35] And, of course, tomorrow, the hackers come to town. Yeah. Because... Defcon. Defcon. And that will be... I've never done a Defcon. I've always wanted to. Oh, wow. That'll be very interesting. Yeah. I'm going to go home. Yeah. I think we should all get out of here while we get... Yeah. Good. Thank you, Anthony Nielsen. Appreciate the hard work you did. He got this whole gear, got it all set up. That's great, right? Right. And, yeah. I mean, it's a portable, small rig. We have a mobile rig now.

[02:04:03] We even have an on-the-air battery-powered on-the-air light. So we were concerned we would be rushed. Yeah. That turned out not to be a problem. No. All right. So thank you, everybody. We appreciate it. We will see you soon. Come back and join us next week, next Tuesday for Security Now. We'll be back at our regular time. For Windows Weekly on Wednesday. Good to see you, sir. Likewise. Take care. Good fun. Hi there. Leo Laporte here.

[02:04:31] I just wanted to let you know about some of the other shows we do on this network. You probably already know about This Week in Tech. Every Sunday, I bring together some of the top journalists in the tech field to talk about the tech stories. It's a wonderful chance for you to keep up on what's going on with tech, plus be entertained by some very bright and fun minds. I hope you'll tune in every Sunday for This Week in Tech. Just go to your favorite podcast client and subscribe. This Week in Tech from the Twit Network. Thank you.

[02:05:13] Avatar Fire and Ash is now streaming on Disney+. It's the film critics are calling the best Avatar yet. A true epic and completely jaw-dropping. This is the only purest thing in this world. Return to Pandora on Disney+. It will be an adventure for the whole family. And watch the Oscar-winning phenomenon at home. This is sick! Avatar Fire and Ash. Now streaming on Disney+. Rated PG-13.