AI Truthfulness, Open Source Standards, TeamViewer & GitHub Innovations, Cisco's Partner Program

[00:00:02] It's Thursday, October 31st, 2024 and I'm Dave Somel. Four things to know today.

[00:00:07] New study finds that LLMs encode truthfulness internally, offering a potential to reduce hallucinations.

[00:00:14] The open source AI definition 1.0 sends a new benchmark for transparency targeting open-and-name-only models.

[00:00:23] TeamViewer GitHub and LinkedIn launch AI and Cisco transitions partners to solutions-based specializations with their new program.

[00:00:32] This is the Business of Tech.

[00:00:35] A new study from Technion, Google Research and Apple reveals that large language models possess a deeper understanding of truthfulness than previously recognized.

[00:00:45] Traditionally, LLMs have been known to generate errors, referred to as hallucinations, which include factual inaccuracies and biases.

[00:00:52] The study analyzed the internal workings of LLMs by focusing on exact answer tokens, the specific response tokens that determine correctness, across four variants of MSTRL-7b and LAMA-2 over 10 datasets.

[00:01:08] Results indicated that truthfulness information is concentrated in these tokens, suggesting that LLMs encode their own truthfulness.

[00:01:17] The researchers developed probing classifiers that significantly improved error detection, demonstrating that large language models have a multifaceted representation of truthfulness, but do not generalize well across different tasks.

[00:01:32] An OpenAI's AI transcription tool, Whisper, has been criticized for its excessive hallucinations.

[00:01:38] A University of Michigan researcher reported that eight out of ten audio transcriptions contained inaccuracies.

[00:01:44] An engineer analyzing 100 hours of transcriptions found hallucinations in about 50% of cases, while another developer noted issues in nearly all of the 26,000 transcripts he examined.

[00:01:57] Experts warn that this could disseminate misinformation across various industries, including healthcare, where Whisper is increasingly used to transcribe patient consultations.

[00:02:07] Why do we care?

[00:02:09] Well, this suggests that the models have a structured internal way of encoding truthfulness, and focusing on these exact answer tokens could improve reliability and potentially filter out hallucinations.

[00:02:20] For providers, this offers a new angle for advising clients on model selection, focusing on tools with embedded error detection mechanisms for more accurate deployments in those business-critical applications.

[00:02:32] Now, businesses should be cautious about fully automating processes that rely on AI-generated outputs, especially those involving sensitive information.

[00:02:41] Implementing policies for review and verification, alongside robust logging to track AI-generated data, can help mitigate the risks, and that's all solution provider offerings.

[00:02:52] The Open Source Initiative has released version 1.0 of its Open Source AI definition, a standard that aims to clarify what constitutes Open Source AI.

[00:03:05] To qualify, an AI model must provide sufficient information about its design and disclose details about its training data, including its provenance and processing methods.

[00:03:16] OSI emphasized the need for consensus among policymakers and developers, especially as regulators begin to scrutinize the AI space.

[00:03:25] Despite this new definition, many AI models labeled as Open Source, such as those from Meta and Stability AI, do not fully meet these criteria.

[00:03:34] A Signal Foundation and Carnegie Mellon study found numerous so-called Open Source models are essentially open in name only, highlighting concerns over access and transparency in AI development.

[00:03:46] The OSI plans to monitor the implementation of this definition and propose updates as necessary.

[00:03:53] Why do we care?

[00:03:54] Well, IT leaders and developers integrating AI into products now have a more explicit framework to assess models for true openness.

[00:04:03] Prioritizing models that adhere to OSI standards can reduce reputational risk and legal liabilities tied to unverified or biased datasets.

[00:04:12] Providers serving clients in sectors sensitive to data providence, such as finance, healthcare, or government, could benefit from aligning with truly open models to address client concerns about data integrity and regulatory compliance.

[00:04:25] For organizations that rely heavily on Open Source AI, the standard could be used as a guidepost to filter out models that lack true transparency.

[00:04:35] TeamViewer has launched AI-powered features called Session Insights, designed to enhance remote support efficiency for IT teams.

[00:04:43] The tools automatically summarize sessions and provide analytics, allowing quicker handovers and improved decision making.

[00:04:50] A recent survey by TeamViewer revealed that 75% of IT, OT, and business decision makers believe AI is crucial for increasing business efficiency,

[00:05:00] with 61% anticipating positive revenue impacts in the coming year.

[00:05:06] IT professionals save an average of 16 hours monthly through AI, compared to just 6 hours for public sector employees.

[00:05:14] As AI adoption grows, the majority of decision makers are now seeking practical applications, with 61% agreeing the AI hype cycle has ended.

[00:05:23] GitHub has announced that its coding assistant, GitHub Copilot, will soon support four new large language systems.

[00:05:37] The update was revealed during the GitHub Universe conference, promising to enhance developer choice across various functions in Copilot.

[00:05:45] The OpenAI models are currently available in Copilot chat, with Claw 3.5 Sonnet expected soon and Gemini 1.5 Pro to follow in the coming weeks.

[00:05:54] GitHub aims to provide multi-model functionality to cater to diverse programming needs, emphasizing its commitment to being an open developer platform.

[00:06:04] The company also introduced GitHub Spark, an AI tool that allows users to create micro-apps using natural language, highlighting the evolving landscape of AI encoding.

[00:06:14] And LinkedIn has announced the launch of its AI hiring assistant, designed to streamline the recruiting process by connecting recruiters with potential candidates more efficiently.

[00:06:24] The AI agent can analyze job descriptions and prompts from recruiters suggesting suitable candidates based on specific criteria.

[00:06:32] Hari Srinathson, Vice President of Product for LinkedIn Talent Solutions, emphasized reducing time spent on administrative tasks, allowing recruiters to focus on actual hiring.

[00:06:44] Why do we care?

[00:06:45] Well, adopting tools that offer AI-powered insights and summaries could be a differentiator, allowing providers to provide quicker and more accurate support.

[00:06:54] Directionally, this is where support is going with more automation beyond.

[00:06:58] Offering options such as the different models from OpenAI, Anthropic and Google empowers developers to select the models that might excel in language understanding, code generation or specific needs.

[00:07:09] It also breaks the OpenAI locked in.

[00:07:12] And if this is the trend of the future, it's directly an area for providers to help.

[00:07:17] Which model should customers pick?

[00:07:20] Well, answer that question.

[00:07:24] Cisco has unveiled its new Cisco 360 Partner Program, aimed at enhancing the value partners deliver to customers by addressing their needs and modernizing infrastructure.

[00:07:33] The program, developed in collaboration with partners and customers, will transition over 15 months, including with its launch in February 2026.

[00:07:41] Cisco is investing $80 million into partner initiatives, including $60 million for skill development through all access subscriptions, and $20 million for quarterly training events focused on AI, security and networking.

[00:07:56] The program will introduce a streamlined framework with two designations, Cisco Partner and Cisco Preferred Partner, emphasizing a shift from traditional architecture to solution-based specialization.

[00:08:09] PIXM Security, PIXM, has launched a managed service provider program for zero-day phishing protection.

[00:08:16] The program aims to combat the threat of phishing attacks occurring over non-email channels, including SMS and social media.

[00:08:24] With over 500,000 end users protected, PIXM utilizes AI-driven computer vision to provide real-time defense against phishing attempts.

[00:08:32] Notably, over 50% of phishing links are clicked on outside corporate email.

[00:08:38] The program offers a low-cost solution at just $1 per user per month and features immediate deployment as a browser-based plugin.

[00:08:46] PIA, the AI-driven helpdesk automation platform, has announced its integration with TimeZest, a scheduling solution for MSPs.

[00:08:53] The new integration aims to streamline meeting coordination within helpdesk workflows, significantly reducing ticket resolution times.

[00:09:01] By determining early in the ticket lifecycle whether a meeting is necessary, the integration prevents unnecessary scheduling, thus enhancing operational efficiency.

[00:09:11] Why do we care?

[00:09:12] Well, for providers working with Cisco, the program's emphasis on training and solution-specific skill development could directly impact their service quality and expand their market potential.

[00:09:22] Additionally, as Cisco prioritizes specializations, partners may be incentivized to deepen competencies in key areas like AI or network security, which hopefully align with client needs.

[00:09:33] Are you ready to get your brand in front of the tech leaders shaping the future of managed services?

[00:09:41] Here at the Business of Tech, we offer flexible sponsorship opportunities to meet your needs, whether it's live show sponsorship, podcast advertising, event promotion, or custom webinars.

[00:09:53] From affordable exposure options to exclusive sponsorships, our offerings are designed to fit businesses and vendors of all sizes looking to make an impact.

[00:10:02] Prices start at just $500 per month, making our packages a fraction of typical event sponsorship costs.

[00:10:11] Be a part of the conversation that matters to IT service providers worldwide.

[00:10:17] Join us at MSP Radio and amplify your message where it counts.

[00:10:22] Visit MSP Radio.com slash engage today to explore all the ways we can help you grow.

[00:10:31] Thanks for listening.

[00:10:32] Besides Halloween, it's also National Caramel Apple Day, National Magic Day, and National Doorbell Day.

[00:10:39] If you've got a comment or a thought on a story, put it in the comments if you're on YouTube, or reach out on LinkedIn if you're listening to the podcast.

[00:10:45] And if you like the show, give us a review, make sure you've subscribed and followed on your favorite platform, and make sure to tell a colleague.

[00:10:52] Share is the number one thing you can do to help the show grow.

[00:10:56] I will talk to you again tomorrow.

[00:11:00] The Business of Tech is written and produced by me, Dave Sobel, under ethics guidelines, posted at businessof.tech.

[00:11:08] If you like the content, please make sure to hit that like button, follow or subscribe.

[00:11:13] It's free and easy and the best way to support the show and help us grow.

[00:11:18] You can also check out our Patreon, where you can join the Business of Tech community at patreon.com slash MSP Radio, or buy our Why Do We Care merch at businessof.tech.

[00:11:31] Finally, if you're interested in advertising on this show, visit MSP Radio.com slash engage.

[00:11:38] Once again, thanks for listening to me.

[00:11:40] I'll talk to you again on our next episode of the Business of Tech.

[00:11:47] Part of the MSP Radio Network.