Exploring AI Trends with Yusuf Khan, Head of Data Science at Constellation

[00:00:02] Clearly, the best way to learn about something is to talk to somebody who's deep in it. And in fact, I met somebody at the opportunity to go deeper with who focuses on a social listing tool that's pulling AI together. So I had to know a little bit more.

[00:00:18] Yousuf Khan joins me as the head of data science at Constellation on this bonus episode of the Business of Tech. You're looking for security solutions for your MSP and Bitdefender has new ones for you. With enhanced protection, simplified management, 24 by 7 analyst led security, threat hunting

[00:00:37] and end to end protection options. It's time to check out Bitdefender's new offerings. With the ability to customize the security solution for what you and your customers need. You'll find a cost effective selection with adaptive and scalable security. Want to check it out?

[00:00:54] Bitdefender would love to schedule a demo for you. Just visit Bitdefender.com or the link in the show notes. Well, Yousuf, thanks for joining me today. Thanks, Dave. Thanks for having me here.

[00:01:09] So I'm going to start with a kind of high level pit because you're head of data science at Constellation and you're spending a lot of time looking at the market. Give me a quick high level of some of the most significant trends you've observed in

[00:01:21] the past year and where you think we're going with this. Yeah, that's a really good question. So the trends are really converging more around the AI space. So I mean, there's such a huge increase in not just research, but funding that's going into productizing AI.

[00:01:42] So kind of like if you are on LinkedIn all the time, you're just going to find AI, AI, AI stuff. But there's a lot of noise kind of embedded into that, right? For companies to really leverage this technology, to be able to build it into their solutions

[00:02:00] and add additional value is the biggest challenge. And the trend that I've been seeing and following for a long time is how is the evolution of these AI technologies and those solutions around them sort of improving that even we

[00:02:15] at Constellation constantly talk about and try to figure out how do we embed that into our own core product so that it sort of like becomes a much beneficial value add for our end customers. Now, where are you seeing like the most value?

[00:02:31] Because there's so many interesting stats, I cover them on the show all the time about the adoption rates are significant, right? Like we've got significant numbers of Americans that have used them. A lot have said they're using them at work.

[00:02:43] We're seeing uptake from, in particular, Microsoft is showing real revenue growth in that. But at the same time, it's also sometimes difficult to get into really specific use cases of saying like, well, what is it being used for?

[00:02:58] So what are you finding are the real impactful use cases and business implementations that you're tracking that are working? Yeah, again, really good question. I think at the end of the day, it really comes down to really re-evaluating your business,

[00:03:15] your industry, and trying to figure out if AI is right for you or not. What systems like ChatGPT and these large language models have done is that they have democratized the use and access to using general purpose AI systems.

[00:03:33] They're not perfect by any means, but they're still good enough to help automate certain processes that might otherwise require other types of inefficiencies to be put in place. And overall, automate the entire process, not only from an end-to-end perspective, but from

[00:03:53] a standpoint of adding that additional customer value. So as an example, I can say, in the automotive space and Constellation is known for our automotive clients and the types of solutions that we provide.

[00:04:07] We're looking at how do we incorporate these AI technologies to try to not just streamline the end process, but also to add additional tools and systems for our end clients, whether that's tier one or tier three, depending on the use cases that we have in mind.

[00:04:26] So I think the trend is there is always going to be these general purpose large language models, but the key value add will always sort of come down at looking at those models and trying to fit it into your product ecosystems to improve efficiency and productivity.

[00:04:44] And sort of to like last one point that I want to add to that as well is that training and building foundation models is also an expensive task, right? Like there's going to require a lot of data, but most importantly, it also requires a lot

[00:04:59] of compute storage and those things. A lot of small to medium sized businesses are not necessarily going to be able to do this. And in order for us to reach a certain threshold where we can achieve a higher productivity

[00:05:14] level, we need to be on the shoulders of these genes which have these types of LLM models for us to further customize it with our data for provide those downstream value that I've been talking about.

[00:05:27] So it's a little bit of like playing around with these systems, but making sure that they could all sort of help add that additional productivity boost. So I want to ask because I always get nervous when we talk about playing around with tech

[00:05:41] because that becomes a little too nebulous. And I want to offer a premise that I've been working on and get your reaction to it. Because when I'm looking for parallels of where the value for an IT services organization,

[00:05:54] a consulting organization, a managed services provider, I know that they're not going to be building large language models. That's not going to make any sense. So if I think about each of these large language models and each of the various implementations,

[00:06:07] kind of the way that I might think of varietals and grapes, right? And so that they create wines and that there are very... Wine is a category, but there's a bunch of different wines because they're different.

[00:06:18] Just like there are lots of different large language models and lots of different pairings. But if the large language model and the AI companies are the winemakers, that makes solution providers and consultants and such into sommeliers.

[00:06:33] They are taking those models as created, those wines, and they're pairing them with customers' needs, i.e. the food to make sure that it matches. So what's the key... First off, it sounds like you agree with that premise.

[00:06:50] So from there, how do you differentiate to be a really great sommelier? 100%. I definitely agree. And I actually want to take it a step further to kind of contemplate on this. And this was an interview that I heard Shamath give an analogy around this idea where...

[00:07:09] Think of LLMs and these foundation models as refrigerator, right? Well, with the refrigerator, there's some money that's to be made, right? But the actual money that was made were from companies like Coca-Cola and others, which

[00:07:23] use the refrigerator as sort of their factor in making their profits and things like that. So with that same thing in mind, one of the key things to keep in mind is an LLM, if it's

[00:07:39] given 10 different or 10 same inputs, all of these LLMs in a way will converge. Machine learning in general works the same way. But if you have some data that is your own data, which you and only you have access to,

[00:07:55] all of a sudden the LLMs have another way to sort of work with and address different types of pain points and questions that might happen. And that sort of becomes very unique to you. You know what I mean? And that's your value add and proprietary.

[00:08:08] So it's sort of like these LLMs are your level one tool set that you have. And data becomes this integral part, which adds a ton more value when sort of fused with these LLMs.

[00:08:23] So we just kind of have to be a little bit smart about data in general. And how do you want to like... And we do see this, a lot of companies are rethinking about how do they monetize their data?

[00:08:36] How do they start opening up their data such that once the data has been made available to the LLM, then it's a lot harder for the LLMs to not have access to data, but compete with other LLMs in a different way.

[00:08:51] So I think we need to see how this whole thing plays out a little bit because there's also questions about will countries kind of gatekeep their data? Like if China decides to gatekeep their data, then how is it going to sort of flow in and out?

[00:09:07] What happens to research? What happens to PII data? And there's a little bit of work around, like people are thinking along the lines of clean rooms or just some type of masking and anonymization and proxy type data to still allow the LLMs

[00:09:21] to pass through all of this information. But we need to allow a little bit of that to sort of play out. So I want to ask a little bit more about be smart about your data because that feels like

[00:09:32] one of those areas that there's a lot in there. And I would say that is particularly highly valuable and that most companies have probably done a really bad job at. What do you mean by being really good at data?

[00:09:48] Yeah, so keep in mind that almost a good majority of the companies that we would talk to, they would all will agree that they have some sort of data that they've captured, whether it's hard fact data or some database that's lying around. Data is still data.

[00:10:05] Sometimes we only think about it as transactional tables, but it could be this interview as well like videos, multi-modal. We need to start thinking multi-dimensional when we're talking data. And the reason why I say we need to is also generalizing it because there's a group of

[00:10:22] people who do think multi-modal. And this is important because these language models are not necessarily tied down to just a single mode. They are multi-modal. They are vision based. They could understand or parse through different types of modality of data.

[00:10:38] But being smart with data sort of comes down to taking a step back, assessing what the data is, spending some time to come up with a strategy to try to understand, is my data proprietary but more than proprietary, does it add value not just to my industry, but

[00:10:56] even outside of my industry? Keep in mind, so at Constellation, we have a ton of data on various different things, but we break it down by specific industries. So when we personalize a product, the data is not completely broad.

[00:11:11] It's very contextualized to that specific industry, which sort of allows you to get better responses from the LLMs. And that's the key reason why companies want to work with us, not only because we have

[00:11:23] that access to the data, but the technical skill set to deliver on that data based on the different types of value props that the customer might be looking for. So tell me a little bit about the data that you're pulling in, because you've been doing

[00:11:36] a lot of work around contextual social listening as part of that. Tell me what that means and what the data is that you're collecting. That's a really good question. I love the word contextual social listening.

[00:11:47] And if I were to summarize contextual social listening, I also need to start off with the history of social listening. Social list listening itself is the idea of looking at everything that's happening online digitally and trying to derive metrics from it.

[00:12:04] The problem, the biggest problem with social listening is it's too broad. It has some type of entry points. So for as an example, you might have a keyword or hashtag or an account that you're

[00:12:15] looking for. But where it sort of tends to fall off is when you need to contextualize it to a specific industry. So what we do at Constellation is our pool, Aurora, specifically does contextual social

[00:12:30] listening. So as an example, it could mine through a lot of these online informations like how some of these other social listening companies do. But most importantly, connect that data back at some type of an identifier which then contextualizes and zones it down even further.

[00:12:47] Now, there's two things that sort of come out of it. One more precise classification of qualitative to quantitative data, which is industry specific. Second, you have now an added layer of factual information checking because we're joining the data back on different fields.

[00:13:05] It allows to like, you know, the system sort of look at it and try to like factually make sure if the information is correct or not. What that prevents is, and you would have probably heard this as well, there's a lot

[00:13:17] of AI generated content out there and that's a big factor. But keep in mind, just before AI generated content, we were and we still are fighting false information online as well. So now you have double trouble problem and then you need a way to hone in on social

[00:13:35] listening in a way which is not just industry specific, which is both factually right, but also contextually relevant to the types of clients that we're working. So we have that laid out both on the auto side as well as on the pharma side.

[00:13:49] So, for example, one of the examples in pharma that I can give you is, you know, if a doctor is being interviewed online, we're able to connect that doctor back and de-anonymize the information in a way where we know just from the name what the

[00:14:04] doctor's ID is, who they affiliated with and things like that, which is, by the way, this type of information is also publicly available. Any doctor information such as that. So we don't necessarily do any PII level type of information.

[00:14:18] Anything that's publicly available is kind of what we work with. And depending on the client engagement, the data that they have, it's easy to marry with our data and then overall improve their knowledge graph.

[00:14:30] So how do you deal with, I'm going to do broadly, you know, bad information and I'm intentionally not using any of the bugs because I can envision this both as false information that is intentionally placed out there, AI generated information which you may want to discard.

[00:14:48] And then the generative AI itself also sometimes makes stuff up. So you have several categories. How do you deal with making sure that the information you're looking at is valid? That's a really good question.

[00:15:03] And it's a way difficult task to solve as well, because if you think about it, I just brought up the concept of multimodality as well. Right. And we're familiar with deep fakes and things like that.

[00:15:16] You could just start an entire company trying to focus on solving and catching deep fakes. But whether you do it or not, that's up for debate. I'm more so on the fact that because these AI systems are getting better and better,

[00:15:32] there is nothing that we would be able to do right now that would want to reach the accuracy of the future of these AI systems. So it's sort of like a, it's still up for grabs in terms of how much information that

[00:15:46] you want to do. But in our systems, what we try to do is we look at information, we do the right de-identification and joins to try to make sure that what is pulled out on that information content that we retrieve, does that sort of make sense?

[00:16:03] So we have AI systems that look at the AI systems as well to kind of have like a scoring mechanism built out. That's one of the reasons why our systems are a lot more accurate because we have

[00:16:13] multiple filtrations to enhance the data and to be able to sort of connect that with clients' data as well. And we do this filtration depending on the use cases as well. And there's always a way to flag non-compliant or non-accurate information.

[00:16:30] And it's constantly a work in progress as we continue to evolve it. But we do think that using generative AI to sort of sit and these agentic approaches to sort of look at data and try to distinguish what is right from wrong, it doesn't

[00:16:46] necessarily solve the entire puzzle, but it does get us a step further ahead in fighting false information. So what are the one or two key kind of emerging AI trends and technologies that you think services companies need to be tracking right now?

[00:17:03] Yeah, I think agents as an example is a huge trend that's coming out. So just to give an example, like the LLM systems, they're good at retrieval. They could in making sure if you provide the right context, there are techniques like

[00:17:20] RAG, which is called retrieval augmented generation, which is this idea that, well, I don't want the system to be too broad. I want to give it the right context, have my prompt and it could pull in the information. Agentic RAG is a similar concept.

[00:17:34] And the concept behind agentic RAG is that, well, I want a decision making support, which can do some type of planning and things like chain of thought, prompting and things like that. The agents can take up a level above that.

[00:17:49] Not only can they do planning, but they can have access to things like external functions or tools, if you may, to try to do a little bit more complex planning. Now, the way that I break these things down is based on Daniel Kahneman's system one

[00:18:05] and system two thinking. What we human beings generally perceive as easy might not necessarily be the case. It might not necessarily be easy for these LLMs or even these agents. So system one is anything that you would just ask and it doesn't really require

[00:18:23] your mind to like really go into some crazy computation, where system two is a little bit more computation intensive. So that allows you to just retrieve the thing. We think system one and two, system 1.5, which is a little bit blurry between one

[00:18:37] and two, these agents can help solve these types of things. Definitely one of the trends that I'm looking out for and trying to bring into consolation, which can help sort of streamline some of these pulling off information and doing a little bit of planning.

[00:18:54] So I think the service industry will definitely benefit a lot by having those types of algorithms we put in place to automate that in a much more precision manner and then overall improve the productivity. Yusuf, this is fascinating.

[00:19:10] Yusuf Khan is the head of data science at Constellation, where he leads the development of innovative AI driven solutions like Aurora that he mentioned. They're cutting edge social listening tool. He's got extensive experience in AI and data science and Yusuf is focused on

[00:19:24] integrating these technologies to enhance business strategies and combat that misinformation. Yusuf, thanks for joining me today. Thanks for having me, Dave. Are you ready to spot opportunities by aligning IT with your client's business strategies? Get in sync equips MSPs and IT professionals with the tools, methods,

[00:19:43] and training to deeply understand client strategies, ensuring IT investments directly support key business objectives for tangible outcomes. With get in sync, you gain critical insights that empower decisive actions, enhancing your competitive offering. This solidifies your role as a trusted advisor and supports your client's

[00:20:04] strategic needs, bringing greater success. Test your readiness to become a certified get in sync trusted business advisor with our free online assessment. Accept the challenge and discover if you have what it takes to become an indispensable strategic partner for your clients. Begin your journey with get in sync.

[00:20:23] Visit getinsync.ca slash MSP radio to learn more. The Business of Tech is written and produced by me, Dave Sobel, under ethics guidelines posted at businessof.tech. If you like the content, please make sure to hit that like button and follow or subscribe.

[00:20:44] It's free and easy and the best way to support the show and help us grow. You can also check out our Patreon where you can join the Business of Tech community at patreon.com slash MSP radio, or buy our Why Do We Care merch at businessof.tech.

[00:21:02] Finally, if you're interested in advertising on this show, visit MSP radio.com slash engage. Once again, thanks for listening to me. I will talk to you again on our next episode of the Business of Tech. Part of the MSP radio network.