AI for MSPs: Nobody Is Counting the Cleanup

[00:00:02] OpenAI just handed mathematicians 700 papers and a warning that some of them could have issues. The checking is now somebody else's job. For your client's AI, not somebody is you. This is the Business of Tech. I'm Dave Sobel.

[00:00:18] The AI industry has a new habit. Ship the output, skip the part that makes it usable. Start with mathematics. OpenAI posted 722 mathematics manuscripts to GitHub, grouped into 372 results and produced by an internal model the public can't use.

[00:00:38] OpenAI's own repository warns that some of the results that haven't been formally checked, quote, could have issues, end quote. By the Verge's count, only about 4 in 10 of the main results come with a machine-checkable proof. Mathematicians told the Verge it could take years just to understand what's there. A referee rating one paper a day, every day, would need nearly two years to get through them.

[00:01:05] The field had asked for something different. Weeks earlier, 25 Fields Medal winners signed an open letter warning that AI labs, racing to solve famous problems, were threatening the work itself. Their concern was solutions announced without proper write-ups, without citation of prior work, and without anyone able to explain them. OpenAI withdrew its sponsorship of a Caltech math event after researchers there criticized it.

[00:01:35] And a week before the release, an independent advisory group of mathematicians at the Institute for Advanced Study, one OpenAI itself consulted, asked labs to disclose the model, the prompts, and the compute behind each result. OpenAI didn't name the model, published no prompts, and gave only compute as an average.

[00:01:56] The people downstream felt it immediately. NYU mathematician Tristan Buckmaster told Business Insider that entire research programs were wiped out in a day. Early career researchers had built grant proposals around problems that now have claimed answers. OpenAI says the release includes a process for revisions and citations, and plans to fund workshops to help researchers work through it.

[00:02:23] Some of it may well be real. That isn't the question. It isn't only math. In its own report on unintended model actions, Anthropic disclosed that during testing, Claude Haiku 4.5 filled in a Philadelphia police tip form about a murder case, with no name and no contact details. I may have information regarding this case, it wrote.

[00:02:48] Its agents also filed 20 incomplete visa applications on a State Department form, the New York Times reported. None were processed. Anthropic says the cases had minimal real-world impact, and it disclosed them itself. But a police department still had to deal with a tip that came from a test. Different fields, same shape. The output shipped. The preparation the receiving side depends on didn't.

[00:03:18] So why does the preparation keep getting skipped? If you're listening to this and haven't hit follow yet on Apple Podcasts, search Business of Tech. Takes five seconds, and you'll get the next episode automatically. Every MSP I talk to has the same problem. They've got AI. What they don't have is a way to charge for it. PAX8 calls the fix managed intelligence.

[00:03:44] Selling outcomes customers want to pay for, not just one-off tools. The PAX 8 marketplace is where you build it. Stop reselling software. Start running intelligence. 47,000 MSPs already made the move. Start yours at PAX8.com. That's P-A-X, the number 8, dot com. Every one of those norms exists for the same reason.

[00:04:11] They make whoever produces the work carry the cost of checking it. Peer review asks an author to write a result up so others can follow it. Responsible disclosure asks a bug hunter to confirm the bug before filing it. That arrangement held because producing the work was expensive, too. Nobody wrote 700 papers in a month. AI broke that balance on one side only. Generating got nearly free.

[00:04:40] Checking didn't get any cheaper. You can watch it happen in open source. Linus Torvalds told kernel developers that the flood of AI reports has made the Linux security mailing list almost entirely unmanageable, as the register reports. Different people run the same tools, find the same bugs, and file the same reports. Maintainers spend their time forwarding them or pointing out that the fix shipped weeks ago. Torvalds calls it pointless make-believe work.

[00:05:10] His ask to reporters is the old norm, restated. Read the documentation. Write a patch. Add something the AI didn't. Add something the other day. Now look at what happens when a producer actually counts. Anthropic's free OSS scanner sends model-written vulnerability reports to the open source projects that sign up, with no human review. And Anthropic warns some will be wrong. Anthropic also publishes a dashboard for its broader disclosure program.

[00:05:39] By its own numbers, it has disclosed 6,157 vulnerabilities across 591 projects. Maintainers acknowledged 5,103. 516 have been patched upstream, about 1 in 10. Even so, the finding was the easy part. The fixing is still on the maintainer's side of the ledger. Anthropic itself notes that a released fix doesn't mean it's installed.

[00:06:06] Every day between found and fixed is the day the bug is known and still open. In plain terms, the producer's books record the output. Papers released, bugs found, reports sent. The checking, the fixing, and the explaining land on somebody else's calendar. And nobody records them. That's why the output looks free. The cost was moved, not removed. And it doesn't only move to mathematicians and maintainers.

[00:06:34] Follow that cost far enough, and it reaches the service desk. For a small business, the people downstream of AI output are your clients. Their files get exposed. Their staff lose hours. And the MSP is usually the only one in a position to see it. On this show, Bryan Lukralle, Chief Strategy Officer at Charter Technology Solutions, a Brooklyn MSP serving charter schools and nonprofits, described what's landing on his team.

[00:07:02] Clients are connecting Claude and ChatGPT to their shared files. In their minds, it's their own staff reading their own files. But the AI finds the file buried 20 folders down that was never locked down properly, and he says things have been exposed. The cleanup is permissions work, policy work, and assessment work. And CTS is giving it away. He hopes to have a paid offering by January.

[00:07:27] Next on his list is how to staff support for apps built by teachers and administrators who aren't developers. Nobody upstream counted any of that. It lands on his schedule. Here's what pushing back looks like. Daniel Stenberg leads Curl, open source software that moves data inside an enormous range of products. Its bug bounty paid out more than $100,000 for 87 confirmed vulnerabilities.

[00:07:55] Then Steinberg measured what was arriving. In earlier years, more than 15% of submissions turned out to be real. Starting in 2025, it fell below 5%. Not even 1 in 20. He described the time spent debunking the rest as completely wasted. So he ended the bounty, removed the money that rewarded volume, and kept banning people who submit AI slop. He didn't absorb it, he counted it, and he changed the terms.

[00:08:24] That's the choice in front of an MSP. Count the work AI output moves onto your clients. The assistant they connected, the vendor's model written alert, the app somebody built over a weekend. Then show it to them with terms attached so that the work arrives with a named person who checked it, or the client sees what skipping that step costs. Or absorb it quietly and your clients never learn what the tools they chose actually cost,

[00:08:51] while you become the free verification layer for every producer upstream. Before you pick a side of that choice, consider what each one does to your position.

[00:09:31] This episode is supported by Mail Protector. MailProtector.com to learn more. In events, Ninja One has a free on-demand webinar on making your security hygiene work visible to clients, what to measure, and how to show it at a QBR. Link is in the show notes. Why do we care? The fair objection is that absorbing a little cleanup is just good service,

[00:09:56] and the MSP who pushes back looks difficult next to the competitor who says yes to everything. But a cost nobody manages doesn't stay small. The volume of unprepared AI output only goes up, and the shop quietly absorbing it is subsidizing every producer upstream out of its own margin. The separator won't be who says yes. It will be who can show a client in hours what that client's AI habits cost last quarter

[00:10:24] and set terms from that number. So what to consider? Measure it before you argue about it. For one quarter, tag every ticket caused by AI output nobody prepared. The assistant a client connected, a model written alert, a tool somebody built over a weekend. Without that number, pushing back is an opinion. With it, it's a lie in the conversation. Change the incentive, not just the tone.

[00:10:52] Seidenberg didn't ask reporters to try harder. He removed the reward for volume. For an MSP, that means AI-originated work arrives with a named person on the client side who checked it, and work doesn't, is scoped and billed as a project instead of disappearing into a flat beam. Make preparation the precondition. Lukralle's clients learned what their permissions allowed when an assistant read them.

[00:11:19] Require a permissions review before any assistant is connected to shared files, priced as its own engagement. The cleanup then happens on your schedule as preparation, not on the client's as an instant response. If this trend continues within the next year, The MSPs that can show a client a measured AI cleanup number will be setting the terms. And the ones that can't will find the cost in their own margins first.

[00:11:48] This is the business of tech. Want to go deeper than the news? The Small Biz Thoughts community is where MSP owners and operators work on the business, not just in it. Member meetings, a deep resource library, and courses through IT Service Provider University. Everything you need to run the practice you actually want. Join us at smallbizthoughts.org. Interested in advertising?

[00:12:16] Head to mspradio.com slash engage. The Business of Tech is written and produced by me, Dave Sobel, under ethics guidelines posted at businessof.tech. Thanks for listening. I'll see you on the next episode. Part of the MSP Radio Network.