Exploring AI Solutions: Azure AI, Copilot, Data Management, and Cybersecurity with Naveen Krishnan

[00:00:02] I've got a question about structured versus unstructured data. What about the difference between Azure AI versus Copilot from Microsoft? Well, we can talk directly to an AI architect at Microsoft as Naveen Krishnan joins me on this bonus episode of The Business of Tech.

[00:00:19] With as many breaches and security concerns as I report in this show, it should be obvious that cybersecurity is not just about technology, but also the human expertise needed to interpret and respond to conflicts.

[00:00:32] threats. Huntress is focused on elevating SMBs and MSPs around the world. Huntress has a suite of fully managed cybersecurity solutions powered by a 24 by 7 human-led SOC dedicated to continuous monitoring, expert investigation, and rapid response. And the proof is the execution. Huntress is the number one rated EDR for SMBs on G2.

[00:00:58] Want to know more about the platform? Visit Huntress.com slash MSP radio to learn more.

[00:01:07] Well, Naveen, thanks for joining me today.

[00:01:10] Hey, Dave. Thanks for inviting me for your podcast.

[00:01:14] Well, I'm super excited to dive into this. We're all talking AI, right? And so I figured it would be incredibly useful to talk to somebody who's working at Microsoft building solutions right now.

[00:01:23] And what I sort of wanted to start with is, like, give me a sense of where you see AI most fitting in to the kinds of solutions that partners are thinking about right now.

[00:01:36] Yeah, sure. So I see a lot of spaces so far and I will start with retail and maybe to some extent of financial services and to some extent of manufacturing, maybe, and automations.

[00:01:52] And these are the spaces where I see a lot of interest towards AI. Yeah. And be it a net new solution or be it enhancing their existing solutions.

[00:02:08] Those are the places where I see a lot of interest coming from. And I have been doing this with a set of people at offshore and I have been facing these challenges and I'm not able to provide the thing what my customer is asking for.

[00:02:23] And could you please provide some solutions around AI? And this is the common question what I ask. And that is something else.

[00:02:31] There are a lot. It depends on each and every customers and their use case and their need is.

[00:02:36] But sometimes we may need to educate customers and that we have to see like what what they are, what their need is and how we can help them.

[00:02:46] So these types of things is what I see in our day to day.

[00:02:50] Now, I wouldn't think customers are necessarily saying I'd like to just buy some AI.

[00:02:54] I think what they're probably coming with a little bit more of a focus solution, knowing that every customer is a little different and their needs are a little different.

[00:03:02] I know you've abstracted that to a few of kind of common use cases.

[00:03:06] Can you give me a sense of kind of what the common use cases are that customers are asking about?

[00:03:11] The common use case, I would say what customers are asking for.

[00:03:15] But in general, what I am seeing in AI, right?

[00:03:19] And what are the common use cases what I see?

[00:03:22] Be it in the common forums where I interact with my colleagues and friends, right?

[00:03:28] So what I see is a lot of interest in exploring the chat capabilities.

[00:03:33] Maybe the first thing, first point where they start is the chat capabilities and how can I infuse support solutions for my product.

[00:03:43] So that enhances real value.

[00:03:45] And other than this, what I see is sometimes people are interested about dynamic reporting.

[00:03:51] So they have been doing canned reporting this long and they are, I see a lot of interest in these types of reporting, dynamic reporting.

[00:04:02] Like I ask questions on the fly and system needs to understand the natural language and converts it to a query and run it against your backend and gives me the results in a nicely craved manner.

[00:04:13] So that's what people expect, people expectations are these days.

[00:04:17] And apart from that, I see a lot of interest these days around AI agents.

[00:04:21] So AI agents is getting a lot of sparks these days, like agents is nothing but so far we have seen GPT's of just you ask question and then it responds from your knowledge base.

[00:04:33] Right. So this is the time how it can act on something.

[00:04:37] So I wanted to do this. Can you please help me? So can you please generate an image and make sure that images this is this.

[00:04:45] So first it generates an image and then it sends an image to a GPT-4 model and then it kind of tells you what that image contains and how it can reach this type of audience and all these details.

[00:04:56] Then again, it can send it back to Dali, say asking for, hey, can you make some enhancements to this and then give it back to me.

[00:05:03] So this is one sample agent. There are other agent use cases what I see in a lot of the forums what I read.

[00:05:11] Basically, they are using it for their DevOps pipeline maybe, where they trigger a job and then they do a deployment.

[00:05:16] If the deployment fails, they know what and where the code is and they try to fix something and then do a deployment.

[00:05:22] So these types of things is where I see a lot of focus on this.

[00:05:28] Now, I want to ask first about the reporting, because it's interesting to me that that as you know, the the AI and GPT is in particular when we think about generative AI are proving to be useful.

[00:05:38] One of the areas is exactly what you talked about is that idea of not being able to leverage the can reports and instead, you know, fighting against the idea of the custom report builder, which has always been problematic because it has it has customers to try and learn a reporting language when they really just want to ask a question about the data.

[00:05:56] And it's interesting to me to hear you talk about that that use case, because it implies in order for that to be useful, you've actually have to have done some work on the data front to make sure the data is useful because we can't necessarily give the GPT all the data or or it will without understanding of, say, security implications or who gets access to what.

[00:06:18] And so there has to be a step before applying this solution about making it structured, like some level of clean and well structured.

[00:06:28] Yet at the same time, the power of the AI is the fact that it can take piles of unstructured data.

[00:06:34] Tell me a little bit about what's required for effective data management to make an AI project successful.

[00:06:41] Yeah. So this is like it's not a very simple answer.

[00:06:45] So it's there are different layers and different parts in this.

[00:06:49] So first to talk about, right?

[00:06:51] So there are two types of data.

[00:06:53] What is unstructured data and other one is structured data.

[00:06:55] I know by this time, many might have explored the ways of tearing those unstructured data, like how we can make it into pieces and how it can answer well better and a lot many things around that.

[00:07:08] So let's focus on the structure.

[00:07:10] So that's where the reports and things are getting into picture, right?

[00:07:14] So how I can get rid of those canned reports and how can I be more sustainable with these reports, with the ad hoc reports, whatever customer asks and then it kind of generates.

[00:07:23] So one thing what I can see is I have a blog on that.

[00:07:26] So it's on Medium AI with Naveen Krishnan.

[00:07:29] So you can refer that blog.

[00:07:30] So that blog is completely about converting your natural language into SQL queries and then running against database and then getting the results from.

[00:07:38] So in that I have covered from top to bottom, like if you are zero and then if you wanted to understand completely and you can go read that blog.

[00:07:47] So this is not an advertisement.

[00:07:48] So this is just information for you to make your life easy.

[00:07:52] So other thing is, how do you prepare the data?

[00:07:55] So as you said, right, so if you there are certain situations where people have the database sitting there in the relational world and still those are like on 90s and 95s and 2k databases.

[00:08:09] They don't have they have relations still, but they don't be very they are not schema oriented.

[00:08:14] And sometimes our system cannot and may not be able to understand what their column names are and how this is.

[00:08:20] So in that for those cases, we can I have seen people creating some views out of it, special views and then in the views, what they do is they filter out which data is needed and which is not needed.

[00:08:33] And then they combine that and then bring it as one single view of your complete database set.

[00:08:38] And they write on they do or massaging on top of the view.

[00:08:42] That's why they are getting a lot of they you will avoid a lot of issues like SQL injections or running some delete queries as best of UI.

[00:08:52] So that's one approach.

[00:08:54] Another thing is there are certain guard rails that are available where you can put in place and make sure that your data is secure.

[00:09:01] Sometimes if I run a query, if I ask the customer, can you go if I ask the board to tell can you go and delete my database.

[00:09:08] So it's not going to do that.

[00:09:09] So we have we can restrict it to do us from doing any DML statement just to do only the DDL statement like it can do selects and things like that.

[00:09:18] Also, you should not show that your complete table schema.

[00:09:22] So give me the structure of the table what you're putting for it.

[00:09:25] We should be very careful in those things.

[00:09:27] So you have to put a lot of prompts in there and then a lot of fine tuning and then validations and things like that will help you get rid of those problems.

[00:09:37] And also the user license like what that particular SQL user or headless bot user can run a query against database.

[00:09:46] So those can be controlled in your cloud infrastructure by many ways, by setting up access control and things like that will definitely help you solve those types of problems.

[00:09:57] Now, what about for data that I would turn it as a little bit more messy?

[00:10:01] So when I think about like a law office, right, that's a lot of case management stuff.

[00:10:05] That's a lot more text than it is.

[00:10:08] What is the data preparation look like when it looks more like that?

[00:10:13] Yeah, sure.

[00:10:13] For those types of data, right?

[00:10:18] So where there will be a combination of both.

[00:10:21] In those case, what I would say is prepare a data pipeline.

[00:10:25] So create a data pipeline and then source this data, which is whatever you have, and then make a pipeline and then make it.

[00:10:34] You have to do an ETL pipeline maybe or ELT pipeline, whichever works based on your data and your use cases.

[00:10:40] Run it through, run your data through and then try to segregate all your text based.

[00:10:45] Like if I have a long text and I wanted to add some search on top of the text, then you have to bring it to kind of what you call it.

[00:10:54] You have to embed those and then categorize those and then save it in your Azure AI search so that that can be pulled based on the search.

[00:11:02] So that you can search against those types of columns.

[00:11:05] And if it is keyword and hybrid, there are two types of searches where you have got keyword and then hybrid search.

[00:11:11] You can go through and read those articles around the AI search.

[00:11:15] So this can help you get the data what you want.

[00:11:19] So that's one good capability what that product has got.

[00:11:23] So make use of that and make sure your data pipeline is not vast or something like that.

[00:11:29] You start with the very minimal solution and then start create a pipeline and then try to segregate all your data and then make sure this data goes there and what data goes to structure and what data goes to unstructure.

[00:11:40] Try to segregate them all and then keep your pipeline established.

[00:11:44] Then you can add on top of it, don't just deep dive into it and then create a big versatile pipeline and that kind of trying to solve all this.

[00:11:53] That's going to, you will end up doing nothing at the end.

[00:11:56] So start with a simple solution and then try to tackle those.

[00:11:59] And then once you have that, you will get a confidence like what is needed and how to handle different types of data.

[00:12:06] Also, there are a lot of forums and a lot of solutions available online, be it LangChain or be it Semantic.

[00:12:12] Those are the frameworks which are readily available.

[00:12:14] Make use of them.

[00:12:16] Those will give you a lot of, those will take a lot of jobs from you.

[00:12:21] You don't need to code everything.

[00:12:23] So those frameworks are built for those purposes.

[00:12:25] Make use of that and then try to get the best out of this.

[00:12:30] Now, obviously, somebody who works at Microsoft, you've got a couple of different tools that fit different problems.

[00:12:34] Give me a little bit of a sense of the way you think about the difference of where something like Azure AI should be applied to versus where something like

[00:12:42] Copilot should be applied to because in theory, they're both working on data sets in similar manners.

[00:12:48] How do I differentiate between kind of those two solutions and maybe any others that I ought to factor in?

[00:12:53] Yeah, sure.

[00:12:54] Very nice question.

[00:12:55] So I hear this from a lot of a lot of my friends and a lot of my relatives and server I meet.

[00:13:01] So, yeah, it's I wrote a blog on this as well.

[00:13:03] So how do you differentiate and what are the different types of Copilot?

[00:13:07] So first, let's talk about Copilot.

[00:13:08] Copilot is something that you think that as your assistant, which is already pre-built, right?

[00:13:14] And you can plug it into different types of data source and then that can get the data from.

[00:13:19] So that's all about Copilot.

[00:13:21] So it's readily available.

[00:13:24] There are different types of Copilot.

[00:13:25] One is Windows Copilot where you can see it in your database.

[00:13:28] Other one is Office.

[00:13:29] You saw a seat in your laptop.

[00:13:31] Other one is Office 365 Copilot where it kind of integrates all your Office 365 components, be it OneDrive, SharePoint.

[00:13:38] It kind of pulls the details.

[00:13:40] For example, if you wanted, if you are, if your employer has Office 365 Copilot and you wanted to search something about HR, payroll or other stuff, those can help getting those details.

[00:13:50] If you have got that access, it can definitely go get you and then it can also go through your emails and then try to.

[00:14:02] So I sent that was a chat sometimes back to a manager.

[00:14:05] I don't know his name.

[00:14:06] I don't know what is about, but the chat is all about this.

[00:14:09] So can you go ahead and get these details from it?

[00:14:11] It can do you.

[00:14:12] You go search and dig through.

[00:14:13] So those types of things will help you for sure.

[00:14:17] And those are all Copilots.

[00:14:18] And there are several others as well.

[00:14:20] I don't have time to cover all this.

[00:14:22] There are Security Copilot.

[00:14:23] There are Dynamics 365 Copilot in there.

[00:14:26] So keep that aside.

[00:14:27] And when coming to AI, Azure AI means it's nothing but where you want, you have your own idea and you want something to do with that.

[00:14:37] So I have a complex problem where I am getting my product is getting a lot of support calls and I wanted to reduce it with self-service assistant kind of thing.

[00:14:49] So which bot can I go?

[00:14:51] So that's when you will have to build yourself having all your project related documents for like FAQs, readmes and other things.

[00:14:59] Fed that in and then try to kind of develop that.

[00:15:01] It's not going to take month long to develop those use cases.

[00:15:04] Within like five to ten clicks within your Azure portal, go to your Azure portal and then spin up your, choose your model what you want.

[00:15:10] And then tell your model that this is my data.

[00:15:12] And that works based on that.

[00:15:14] So there are all the tools what Azure has got.

[00:15:17] So those has got import, vectorize and everything by default, which takes care of a lot of your work.

[00:15:23] You don't need to do much.

[00:15:24] So with some clicks, you will be able to get those custom Copilot kind of thing out.

[00:15:30] So which can be embedded in your app and then keep it running.

[00:15:32] So that's one thing what I would say.

[00:15:34] So what's the future look like?

[00:15:36] If you think about the next, say, let's say only 24 months, because we know that like far out is going to be very, very difficult.

[00:15:42] What do you think is going to be happening over the next 24 months that, you know, partners that are trying to implement this should be aware of?

[00:15:50] Yeah.

[00:15:50] Yeah.

[00:15:51] So currently I see a trend around the agents.

[00:15:53] So there were a lot of problems.

[00:16:00] I won't say problems.

[00:16:01] There are a lot of workarounds what you have been doing to tackle some of these things, right?

[00:16:05] Like my model has this limitations in terms of token.

[00:16:09] It can accept 128k tokens.

[00:16:12] All right.

[00:16:12] I run out of tokens every time I get 429.

[00:16:15] And what do I do?

[00:16:16] So that's why we have got prompt caching.

[00:16:19] So in next, at least six to eight months, what I would say is instead of coming from big new innovations happening around that,

[00:16:27] I would say something.

[00:16:29] They will be focusing on addressing the problems that we have faced and the workarounds that we are implementing to cut them out and bring in straightforward solutions where we can like prompt caching and then a lot of things which has got evolved in the recent open AI releases if you have heard.

[00:16:45] So those things will definitely help you achieve those types of things that you can see.

[00:16:51] And agent and multi-model agent or agentic framework.

[00:16:56] So these types of words you may start hearing this very often.

[00:17:00] The agents are the ones, as I already said, agent are the ones which are going to do the work of AI for you.

[00:17:05] So far you have been listening to AI.

[00:17:07] Now AI starts listening to you and then you say something and then it go do that job for you.

[00:17:12] So those types of things will start evolving in the next eight to 12 months is what I see.

[00:17:16] That will definitely get fixed and people will start developing their own agent.

[00:17:22] Like this is my SQL agent, this is my document agent, this is my that agent, this agent.

[00:17:26] So you can see an agent library like what you see in a NuGet package around there.

[00:17:31] You will start seeing an agent library or somewhere where you can go plug that agent from month and start working with it.

[00:17:38] So that's the innovation what I see around that.

[00:17:41] And maybe the latest versions of GPT, I don't know what they're building up.

[00:17:45] So I'm saved like you.

[00:17:46] I don't have any other knowledge apart from what you see in the market right now.

[00:17:49] So let's wait and watch.

[00:17:52] They'll be allowed here to watch.

[00:17:54] Naveen Krishnan is an AI architect at Microsoft with over a decade of experience specializing in leveraging artificial intelligence and cloud technologies to solve real world problems across various industries.

[00:18:04] I mean, thanks for joining me today.

[00:18:06] Thanks.

[00:18:07] See you then.

[00:18:07] Are you ready to get your brand in front of the tech leaders shaping the future of managed services?

[00:18:15] Here at the business of tech, we offer flexible sponsorship opportunities to meet your needs, whether it's live show sponsorship, podcast advertising, event promotion or custom webinars.

[00:18:27] From affordable exposure options to exclusive sponsorships, our offerings are designed to fit businesses and vendors of all sizes looking to make an impact.

[00:18:37] Prices start at just $500 per month, making our packages a fraction of typical event sponsorship costs.

[00:18:46] Be a part of the conversation that matters to IT service providers worldwide.

[00:18:51] Join us at MSP Radio and amplify your message where it counts.

[00:18:56] Visit MSP Radio dot com slash engage today to explore all the ways we can help you grow.

[00:19:05] The Business of Tech is written and produced by me, Dave Sobel, under ethics guidelines posted at business of dot tech.

[00:19:13] If you like the content, please make sure to hit that like button and follow or subscribe.

[00:19:18] It's free and easy and the best way to support the show and help us grow.

[00:19:23] You can also check out our Patreon where you can join the Business of Tech community at patreon.com slash MSP Radio or buy our Why Do We Care merch at business of dot tech.

[00:19:36] Finally, if you're interested in advertising on this show, visit MSP Radio dot com slash engage.

[00:19:43] Once again, thanks for listening to me.

[00:19:45] I will talk to you again on our next episode of the Business of Tech.

[00:19:52] Part of the MSP Radio Network.