AI Errors in Law, Databricks' AI Databases, Zoom's CEO Avatar, and Gender Employment Risks

[00:00:02] It's Wednesday, May 28th, 2025, and I'm Dave Sobel for Things to Know Today. AI's hallucinations are landing lawyers in hot water. AI Not Humans now code 80% of new Databricks databases. Automation threatens to deepen the gender employment gap. And Zoom's CEO uses an AI avatar raising questions of authenticity and trust. Plus, we have data on Apple's new modem. This is the Business of Tech.

[00:00:31] I'm going to lead off with some AI downsides. Judges are increasingly identifying fake legal citations, with a notable rise in misuse attributed to lawyers relying on artificial intelligence tools. Since May 1st, judges have reported at least 23 instances of AI-generated inaccuracies in court records, highlighting a growing trend where legal professionals are at fault for these errors.

[00:00:55] Damian Chartland, a legal data analyst, has documented 120 cases where courts have recognized AI's hallucinations, instances where the technology produced fabricated legal quotes or cited non-existent authorities. In a significant shift, data indicates that in 2023, three out of 10 cases of AI citation errors were attributed to lawyers, contrasting with the majority of mistakes previously made by individuals representing themselves in court.

[00:01:24] The implications of the findings are significant, as courts worldwide are imposing fines for AI misuse, with penalties exceeding $10,000 in some cases. Databricks has revealed that 80% of new databases created on their NEON platform last month were generated by artificial intelligence agents rather than human developers. This trend highlights the growing influence of non-developers utilizing vibe coding platforms,

[00:01:49] which enable users to create applications simply by providing prompts that AI agents then transform into code and project specifications. The article discusses how these AI-driven tools are changing the landscape for development tools, suggesting that companies must adapt to remain competitive. It emphasizes the importance of optimizing products for visibility to large language models and stresses the need for seamless integration, with companies like NEON and STRIPE leading the way in providing user-friendly solutions for AI agents.

[00:02:19] A recent article in Futurism highlights the alarming trend of artificial intelligence potentially increasing the gender employment gap, particularly affecting women. According to a report from the United Nations International Labor Organization, the risk of women facing high automation potential jobs rose to 9.6% in high-income countries like the United States, up from 7.8% just two years prior.

[00:02:44] In comparison, the risk for men is significantly lower at 3.5%, which has also increased from 2.9% in 2023. The report reveals that one in three workers in wealthy nations face some degree of exposure to automation, with jobs traditionally held by women, such as administrative and clerical roles, being especially vulnerable. Sociologists note that while the gender gap in labor hours has narrowed, the gender pay gap continues to persist,

[00:03:12] meaning women's work often shifts toward household tasks rather than formal employment. With the rapid adoption of artificial intelligence, experts warn that without substantial changes in the labor market, women may face increased challenges in securing stable employment. Why do we care? AI's propensity to hallucinate isn't a quirky bug. It's a high-stakes liability, especially when professionals blindly trust its output.

[00:03:39] The legal profession serves as a proxy for broader regulated industries, such as finance and healthcare, where trust, compliance, and audibility are non-negotiable. For providers supporting clients in these sectors, this is a cautionary tale. Any AI deployment must come with robust guardrails paired with accountability. If providers focus only on surface-level AI enablement, like cheaper operations, faster development, better marketing,

[00:04:05] they risk walking clients straight into governance disasters, ethical landmines, and workforce disenfranchisement. Ignoring these undercurrents may preserve short-term margins, but erodes long-term trust. And in the AI services business, trust is the currency of growth. And note the shadow IT and shadow AI problem. Procurement and software creation are moving from IT teams to business users wielding AI tools.

[00:04:33] Finally, MSPs advising clients on automation strategies need to incorporate a human impact lens, particularly as they assist in transforming business operations. Questions of re-skilling, up-skilling, and equitable impact should be baked into the conversation, not bolted on as later PR cleanup. There's also reputational risk for clients and you if these impacts are ignored.

[00:05:01] Are your customers getting the most from their Microsoft 365 Business Premium subscriptions? Are you delivering maximum value while ensuring best-in-class margins? Nerdio's new modern work features let you streamline the management of Microsoft technologies like Azure Virtual Desktop, Windows 365, Intune, and Defender. Reduce the need for multiple tools, consolidate your vendor stack, and deliver greater value to your customers.

[00:05:27] Help your customers maximize their investments, free up your team for strategic tasks, and drive meaningful business outcomes. With Nerdio Manager for MSP, a single, flexible platform with mix-and-match plans ensures a flexible, perfect fit for you and your customers. Deliver solutions that achieve real business impact. Visit GetNerdio.com to find out more.

[00:05:53] ChatGPT's Deep Research feature can now integrate with Dropbox and Box, allowing users to pull data directly from these cloud storage services. This enhancement enables the AI to conduct research more efficiently by accessing documents stored in users' accounts. Deep Research, an artificial intelligence agent developed by OpenAI, automates the research process by crawling the internet and compiling information based on user prompts.

[00:06:16] Previously, it relied primarily on internet sources, but this new integration allows for the inclusion of user-specific documents. The addition of Dropbox and Box comes after the successful integration of GitHub and OneDrive, expanding the capabilities of ChatGPT's research tools. OpenAI has indicated plans to support custom applications and services in the future. GoTo has announced a new strategic partnership with Acronis. The collaboration aims to enhance endpoint management and security capabilities for IT professionals,

[00:06:47] cross-managed service providers, and small to mid-sized businesses, introducing the LogMeIn data protection suite powered by Acronis. The new suite integrates unified endpoint management with business continuity and disaster recovery tools, allowing IT teams to manage backups, automate recovery processes, and extend protection to cloud services like Google Workspace and Microsoft 365. Joseph George, general manager of the IT solutions group at GoTo,

[00:07:14] emphasized that the partnership addresses the complexity faced by IT teams, highlighting the risks associated with using siloed platforms for backup and disaster recovery. And Salesforce has launched Agent Force for Financial Services, designed to help financial institutions address staffing shortages and meet rising client demands by automating front office operations with role-based artificial intelligence agents. The offering aims to enhance efficiency and regulatory compliance,

[00:07:41] while allowing human employees to focus on relationship building and complex problem solving. With 50% of the insurance workforce expected to retire within the next 15 years, and a projected shortfall of 100,000 financial advisors by 2034, the pressures on firms to maintain service levels without significantly increasing staff. Salesforce's solution includes pre-built agent templates for various roles,

[00:08:06] such as financial advisors and loan officers, which can automate tasks like meeting preparation and client inquiries. According to Salesforce, only 21% of consumers feel fully satisfied with the level of personalization they receive, highlighting the need for solutions that can improve client experiences while adhering to compliance standards. Why do we care? If you're not building your own integration logic or playbooks, you risk becoming a reseller of someone else's intelligence.

[00:08:33] Whether it's Salesforce, OpenAI, or GoTo, the vendors are pushing further into your value chain. That's great if you're riding the wave, but risky if you're not actively positioning your services as how these tools are implemented, governed, and evolved. Also, AI agents accessing client data, especially in regulated sectors, opens a compliance minefield. If you can't explain what the AI saw, why it acted, and whether it was right, you're going to lose trust.

[00:09:02] Notable in corporate communication, Zoom's chief executive officer, Eric Wan, has begun using an artificial intelligence avatar during quarterly earnings calls, follows a similar move by chief executive officer of Klarna, who recently employed an AI avatar for his investor call. Wan's avatar was created using Zoom Clips, the company's tool for asynchronous video production. During his address, the avatar emphasized Zoom commitment to innovation, stating,

[00:09:40] Why do we care? Someone tell Eric Wan no. It's easy to dismiss this as a gimmick, but it exposes some real flaws here. How do we know that was won in those calls, and what security was put in place ensuring they aren't being misused? The ability to do script, polish, and present a message without the speaker ever needing to deliver it live does introduce new dynamics in corporate authenticity and consistency.

[00:10:10] There is both opportunity and risk. Over on the opportunity side, small business leaders can look and sound polished even with limited time or PR resources. It does level the playing field, and MSPs can help set it up. But on the risk side, there's fake CEO messages, impersonation, and misuse, which are all going to rise. Providers must educate clients about secure identity and communication practices, especially as deepfakes move from security concern to business feature.

[00:10:38] Was this actually you? Is a key question to be able to ask. The continued AI theme is about accountability, and too many solutions offer none. Qualcomm has challenged Apple's claims regarding the performance of its in-house C1 modem, asserting that it falls short compared to Qualcomm's existing modems. A recent study conducted by Cellular Insights found that Android devices powered by Qualcomm technology

[00:11:06] outperformed the Apple iPhone 16e in both download and upload speeds, particularly in urban environments like New York City. The report indicates that download speeds for the Android devices were 34.3 to 35.2 percent faster than those of the iPhone 16e, while upload speeds were 81.4 to 91 percent faster. Qualcomm emphasized that the performance differences were especially pronounced under less-than-ideal signal conditions,

[00:11:34] highlighting the advantages of modern features in their modems that the C1 lacks. This comes as Apple seeks to reduce its reliance on Qualcomm. Why do we care? Apple wants full-stack control from silicon to software. This may seem disastrous, but it's not. Not every client will notice a 35 percent drop in theoretical mobile speeds. Many operate on Wi-Fi most of the time or don't push their upload speeds heavily. Also, Apple will iterate fast.

[00:12:02] This is their first-gen modem, and they've proven capable of rapid improvement. Qualcomm should be worried. That's not bad for a low-end, entry-level version 1. Thanks for listening. Today is National Brisket Day, National Hamburger Day, International Burger Day, and World Hunger Day. All those meat-related days have me very hungry. The Business of Tech is written and produced by me, Dave Sobel,

[00:12:28] under ethics guidelines posted at businessof.tech. If you've enjoyed the show, make sure you've subscribed or followed on your favorite platform. It's free and helps directly. Give us a review, too. If you want to support the show, visit patreon.com slash msbradio, and you'll get access to content early. Or buy our Why Do We Care merch at businessof.tech. Have a question you want answered? We take listener questions, send them in,

[00:12:58] ideally as a voice memo or video, to question at mspradio.com. I answer listener questions live on our Wednesday live show on YouTube and LinkedIn. If you've got a comment or a thought on a story, put it in the comments if you're on YouTube, or reach out on LinkedIn if you're listening to the podcast. And if you want to advertise on the show, visit mspradio.com slash engage. Once again, thanks for listening, and I will talk to you again on our next episode.

[00:13:28] Part of the MSP Radio Network.