The TED AI Show: How open-source AI will reshape power dynamics in tech w/ Hugging Face CSO Thomas Wolf

[00:00:00] TED Audio Collective.

[00:00:36] I'm not Tickle Me Elmo, I'm talking about Tamagotchis.

[00:00:40] You know, these were those little digital pets that lived on egg-shaped key rings and never let you sleep.

[00:00:46] They were super needy, and if you didn't feed them long enough, you'd wake up to find a gravestone where your pet used to be, with a little ghost floating next to it.

[00:00:57] Personally, I was very happy to co-sign the Tamagotchi to the annals of history.

[00:01:01] But believe it or not, they've got a weird connection to some pretty cool AI stuff happening today.

[00:01:10] I'm Bilal Siddhu, and this is the TED AI Show, where we figure out how to live and thrive in a world where AI is changing everything.

[00:01:24] How will humans and machines work together in the future?

[00:01:28] We spend so much time discussing how the world's changing.

[00:01:32] It would be absolutely absurd to believe the role of the CEO is not.

[00:01:36] This is Imagine This, a podcast from BCG that helps CEOs consider possible futures for our world and their businesses.

[00:01:45] Listen wherever you get your podcasts.

[00:01:52] You've probably heard about artificial intelligence and ChatGPT, but do you know the person in charge?

[00:01:58] On our podcast Good Bad Billionaire, we tell the stories of how the world's billionaires made them money.

[00:02:03] We're telling the story of Sam Altman, the boss of OpenAI who make ChatGPT.

[00:02:07] He became a billionaire this year, but his wealth has nothing to do with artificial intelligence.

[00:02:12] He actually got rich investing in other tech startups.

[00:02:15] Listen to Good Bad Billionaire to learn how he did it and whether he's good or bad.

[00:02:19] That's Good Bad Billionaire wherever you get your BBC podcasts.

[00:02:22] So there's a company called Hugging Face, like the emoji, smiling with outstretched hands.

[00:02:31] Back in 2016, the co-founders Julian Chamond, Thomas Wolfe, and Clement DeLong had this idea.

[00:02:39] They could use AI to create an online Tamagotchi for teenagers.

[00:02:43] A chatbot.

[00:02:44] A friend that was always around for them.

[00:02:47] A constant companion that responded to their input.

[00:02:51] While they were tinkering with their AI chatbot, they started thinking about the actual infrastructure underpinning their product.

[00:02:57] Like, what goes into an AI chatbot?

[00:02:59] You need natural language processing, right?

[00:03:01] Which means you're going to need data.

[00:03:03] Massive amounts of data.

[00:03:05] You're going to need a library of pre-trained models to mediate the interactions between computers and human language.

[00:03:12] Basically, you need a lot of complex parts just to run a simple chatbot.

[00:03:16] And right around this time in 2017, Google dropped their seminal,

[00:03:21] Attention is all you need paper, which introduced the world to transformers.

[00:03:26] So, while Hugging Face was tinkering with their fledgling product, they had a lightbulb moment.

[00:03:32] This software stack they were creating to build and improve their chatbot was perfect for the shift to the transformer architecture.

[00:03:39] And they wanted to share it with others.

[00:03:40] So, they shared their code, their best practices, and their findings with the public.

[00:03:45] And each time they shared something, they got such a positive response from their followers and users that they didn't want to stop.

[00:03:52] They decided to build a public library of AI software.

[00:03:56] So, anyone, anywhere, building anything with AI can access the information they needed.

[00:04:03] Next thing you know, Hugging Face made a full pivot away from the chatbots and put all their resources into this mission.

[00:04:10] And today, their site is the go-to resource for all sorts of folks interested in AI.

[00:04:15] From software engineers, the weekend coders, to computer science students.

[00:04:19] I use it myself every single week.

[00:04:22] And Hugging Face includes so much more than code.

[00:04:25] It's a common space for anyone using, building with, or even just interested in AI to learn more and do more with this amazing tech.

[00:04:33] Want to create your own comics?

[00:04:35] Someone made a comic book generator and shared it on Hugging Face.

[00:04:39] Looking to include images from other countries in your datasets?

[00:04:42] Yup, they've got that too.

[00:04:44] Want to share the latest release from IBM and NASA on AI-enhanced climate predictions?

[00:04:49] There's a message board for that as well.

[00:04:51] It's pretty wild, right?

[00:04:53] Of course, not everyone wants to go open source.

[00:04:57] And yeah, there are definitely advantages to keeping your cards close to the vest.

[00:05:00] But I spoke with Thomas Wolfe about why open source works for Hugging Face.

[00:05:05] And why it matters not just for the major players in the industry, but for the rest of us too.

[00:05:12] Well, Thomas, welcome to the show.

[00:05:14] It's a pleasure to be here.

[00:05:15] Thanks for having me.

[00:05:16] Cool.

[00:05:17] Alright, so let's talk about your origin story.

[00:05:19] You know, I obviously want to chat about Hugging Face.

[00:05:22] But I'd actually love for you to start with your personal background because you are what I would call a triple threat.

[00:05:28] You're an engineer.

[00:05:30] You're a scientist.

[00:05:31] And you're a lawyer.

[00:05:32] Could you chat a bit about who you are and your journey into artificial intelligence?

[00:05:37] So, I grew up in France.

[00:05:40] And that's also where I met one of my co-founders, Julien Chemin.

[00:05:43] We were playing together in a band, rock band in Paris during our engineering school.

[00:05:49] And after engineering school, I went to do research in quantum physics, statistical physics, basically working on superconducting materials.

[00:05:57] I really liked that.

[00:05:58] It was really nice, but it was moving a bit too slowly for me.

[00:06:02] And so, at the end of basically a postdoc after my PhD, I was like, I need to do something else.

[00:06:09] And I really liked writing at this time.

[00:06:11] And so, one of my friends was a lawyer and told me, if you do law, basically you write a lot.

[00:06:16] So, I was like, okay, why not?

[00:06:18] And so, I moved to patent law, which is half technical and half legal, I would say.

[00:06:24] And so, I did that for six years, started to get progressively my own clients bigger and bigger.

[00:06:32] And a lot of them were startups and a lot of founders.

[00:06:35] And at the end, in the years like 2014, 15, some of them were starting to do deep learning.

[00:06:42] So, some of the intellectual property strategies that was designed were around, okay, how can you protect, you know, stuff that were around.

[00:06:49] At that time, image recognition, object segmentation, they were really the first thing that they work with, modern AI techniques, right?

[00:06:57] And I was very surprised that I didn't know anything about AI because this was basically the same equation as physics equation, but just written by ML people.

[00:07:09] And Anchorist in Time, what year, around what year was this when you're looking at these kind of perception AI patents?

[00:07:14] So, that was 10 years ago, 2014, 15, yeah.

[00:07:18] So, like right in the early days of like ImageNet and all that stuff sort of starting to pop off.

[00:07:23] Exactly. Yeah, exactly that time.

[00:07:25] And so, that's where I started to do some evening class, but on my own, reading, reading papers, reading books.

[00:07:31] And so, that's when I basically contacted back to Julien.

[00:07:35] Well, we actually just ended up posting roughly the same thing on AI and on Facebook.

[00:07:41] And I was like, oh yeah, I'm also reading this.

[00:07:43] And he said, yeah, maybe we're going to start a startup in New York.

[00:07:47] So, why don't you join and do some science back?

[00:07:50] And then you'll find maybe a more serious job in a couple of months.

[00:07:54] And basically what happened is I never moved out to find a more serious job because, you know, the game company that was Hugging Face at the beginning.

[00:08:03] So, the early idea, and that's why we have this non-serious name, was to make kind of a modern version of Tamagotchi.

[00:08:11] So, modern version of this kind of, you know, funny little being to interact with.

[00:08:17] And when I say modern, I mean powered with all these early Gen AI things.

[00:08:22] So, basically at that time we thought the only thing we could do was, you know, understand image better.

[00:08:27] And we were like, that's still really cool.

[00:08:29] You can recognize selfie, you can maybe react to how someone is, you know, looking at you on emotions.

[00:08:35] And, but quickly came some of the early NLP breakthrough in generation.

[00:08:41] So, with LSTM at the time.

[00:08:43] And we were like, oh, maybe we can even generate texts.

[00:08:46] And so, basically we were exploring all of this, building this game.

[00:08:51] And that's a bit when happened, I would say, the change or like the morphing of Hugging Face in what it is today.

[00:08:59] Which happened basically by, you know, trying to, on our side as well, give back.

[00:09:05] So, if we do some research, we find some stuff, we should also write our first paper.

[00:09:10] We should also publish our code.

[00:09:12] And after a couple of months, we were like, yeah, when we look at all the curves we have, you know, in a startup, you're always looking for this exponential.

[00:09:19] You're always looking for where is something growing crazy.

[00:09:22] And so, the game was working nicely.

[00:09:24] We had a very nice linear increase.

[00:09:26] We were reaching 300 million messages.

[00:09:28] But the open source indicator or the stars on GitHub, the issues, they were really growing exponentially.

[00:09:35] And we also realized we were very much excited about this idea of open source.

[00:09:40] That's something Julien is also a very, very strong proponent, even more than me, I would say.

[00:09:45] Like radical open sourcing, radical open sharing.

[00:09:49] And this, I think, made a lot of sense.

[00:09:52] And that's where basically the mission of Hugging Face kind of started to come.

[00:09:56] So, not really from the beginning, but it became kind of obvious that that was both something that was growing very strongly and at the same time something we believed a lot in.

[00:10:05] So, basically, that's when we pivoted for our Series A 2019, actually.

[00:10:12] Yeah.

[00:10:13] Talk to me about that pivot.

[00:10:14] You know, clearly, y'all were one of the earliest companies that were looking at these sort of machine learning primitives.

[00:10:18] At your disposal, let's say for visual understanding.

[00:10:21] And trying to build like a consumer experience around it.

[00:10:24] I'm sure that involved building your own stack to kind of do that.

[00:10:28] This is probably the early days where there were no companies such as yourself making that easier.

[00:10:33] And so, kind of packaging that up and tackling open source.

[00:10:37] That must have been interesting because usually you hear about the reverse where you've got a developer-centric company that goes consumer and boom, they get suddenly a lot of interest in adoption.

[00:10:47] Y'all feel like the complete reverse case.

[00:10:49] Can you talk a little bit more about that pivot and shift as it happened?

[00:10:53] Yeah.

[00:10:54] So, in this time we were doing like a lot of trials in many, many directions.

[00:11:00] Both on the interaction we wanted from the user.

[00:11:03] Okay.

[00:11:03] Is it better to, you know, be on mobile?

[00:11:07] Is it better to sound like NLP?

[00:11:09] Is it better to use images?

[00:11:10] What is the most interesting thing?

[00:11:12] But also more generally, you know, how do we want to be as a company?

[00:11:17] How can we get a little bit of excitement?

[00:11:19] You also have to picture that back in the day we were really underdogs.

[00:11:22] We were basically, you know, three French founders.

[00:11:25] So, still, I mean, still already like in the US, but for the rest we had no PhD in ML.

[00:11:32] We were not in the field at all versus Google people.

[00:11:35] And so, we had a lot of kind of imposter syndrome and wanted to build some credibility.

[00:11:42] And it was also the time of the chatbot company, you know, the early versions.

[00:11:46] And we kind of wanted to steer away from this and not be put in the same bag.

[00:11:50] We were like, okay, we're trying to build something serious.

[00:11:53] We believe in the AI.

[00:11:54] We're not just, you know, repurposing a couple of templates and making that into a chatbot.

[00:11:59] So, that was part of this thing with the idea of, okay, if we publish, if we communicate about what we do, we're going to be taken more seriously.

[00:12:09] And it's going to be easier to raise, it's going to be easier to hire people.

[00:12:12] So, it was really exciting, which actually will end up being very true.

[00:12:16] And I think being open source and having an open source approach is still really a great, great idea to hire great people, to be visible.

[00:12:24] When you're starting, you know, a little bit outside of the traditional path.

[00:12:29] And yeah, it doesn't mean you have to do everything open source, but it can be a very big part of the mix.

[00:12:35] You know, a big part of why y'all have so much adoption, especially in the early days, was, you know, some of your open source libraries, right?

[00:12:42] Like the famous Transformers library.

[00:12:44] Now, for the audience that might be uninitiated, how would you explain what a transformer is and the importance of these open source libraries and even data sets?

[00:12:53] So, a transformer is basically the software of AI.

[00:12:57] So, it's the, you know, the workhorse is the back end that kind of make all the AI models we see today.

[00:13:05] And they define, you know, how you combine the numbers.

[00:13:09] An AI model is basically just a big set of matrices, so a big set of numbers, and you need to combine them in a certain way to get the output that you want.

[00:13:17] So, it can be an image created from a text prompt.

[00:13:20] It can be text generated from text.

[00:13:22] So, the first step is usually you convert the input in numbers.

[00:13:25] You convert your text in numbers by associating a number to each word.

[00:13:29] And then the way these numbers are combined are defined by the architecture.

[00:13:33] So, Transformers is one way to combine them.

[00:13:35] And basically, Transformers is also the name for the big libraries that we have.

[00:13:41] Like, we have many, many open source libraries now because we expanded much wider than just text.

[00:13:46] But that's kind of where we started.

[00:13:48] And this library, I would say what was really special about it was two things.

[00:13:53] The first thing was it was very, very easy to use.

[00:13:57] And so, basically, I remember when I met the researcher who created this BERT model, which was the first, I would say, widely used model.

[00:14:06] So, before GPT-1, right before it was a bit around the same time.

[00:14:11] It was the first model that people would, you know, people who have been working the field for a long time would basically drop everything they were doing to try this new model and run to it and try to test it.

[00:14:22] And so, when you reduce the entry barrier, it helped them a lot.

[00:14:26] So, we had a lot of people thanking us, okay, I could try this model and I could modify it and try to tweak it and understand how it was working and basically switch my whole research around this new model in a minute, thanks to the very easy access that your library gave to us.

[00:14:45] So, the first power user of our libraries were really ML researcher, AI researcher.

[00:14:50] And that's really how it started.

[00:14:52] And then the second big ingredient was that we started to include a couple of models in it.

[00:14:58] So, you have one model released by a company and we see that a lot today.

[00:15:02] And basically, a couple of weeks later, a couple of months later, you know, their competitor released their own model, which is slightly better.

[00:15:10] And so, you have this kind of, I think, quite healthy and generally nice competition between a researcher or team building models to get the best model.

[00:15:20] So, it's very nice.

[00:15:21] So, it's very nice.

[00:15:22] But when you are participating in this competition, you want also to quickly be able to try the new model and to compare them together and to understand, okay, what are the differences?

[00:15:30] Why is the new model from Google better?

[00:15:32] And the nice thing about the libraries, the transformer, when I created it was it was really flexible.

[00:15:37] And so, I could quickly add new models in it.

[00:15:40] And basically, I remember when OpenAI released GPT-2, it was one day in basically a weekend.

[00:15:48] So, I worked on it on Saturday.

[00:15:50] And Sunday, I had added it in the library.

[00:15:52] It was really, I think, on a Friday.

[00:15:55] And on Monday, everyone could use it in the library.

[00:15:57] So, basically, people just keep the same library and they just switch, you know, like one arguments, basically, and they could use the GPT-2.

[00:16:06] So, this was the beginning of the idea of the hub.

[00:16:09] And then we added a lot of social feature and more things on top of it.

[00:16:14] But that was this beginning of an idea that, you know, there will be a diversity of models.

[00:16:19] There will be more and more models, AI models coming.

[00:16:21] And it's very nice if you can easily move, you know, between them and try them and compare them in this single environment.

[00:16:29] Yeah, it's super exciting because, like you mentioned, the ML community, at least historically, has just been so transparent and forthcoming.

[00:16:36] And I think that creates those opportunities for virality, right?

[00:16:41] If people come across a cool new thing, oh, BERT's super cool or GPT-2 is super cool.

[00:16:45] They're going to jump into it.

[00:16:46] They're going to post about it.

[00:16:47] Other ML engineers and researchers are going to look at it.

[00:16:50] And then on and on.

[00:16:52] And that's sort of the flywheel continues.

[00:16:53] And that's quite a beautiful thing because, like, the way my first exposure to Hugging Face was even the next layer of making this more accessible was Hugging Face spaces.

[00:17:03] Every time a new, you know, computer vision paper would drop or something like that or a new, you know, like, a diffusion paper or diffusion transformer thing would drop, I'd go to the Hugging Face space and try it out.

[00:17:14] Oh, segment anything model by Meta is out.

[00:17:16] Let me just upload an image and see what it does.

[00:17:19] And that was exciting, right?

[00:17:20] Because it suddenly contextualizes some cool new research and it's not the same five videos associated with a paper that get reposted again and again and again.

[00:17:30] You can see it with your own data.

[00:17:33] So if you have to describe what Hugging Face does now and the role that it plays in the open source community, how would you describe that?

[00:17:41] It's really this, I think, this place where people who make models and the people who use them that are actually, a lot of them are now the same.

[00:17:50] And I think it's very nice.

[00:17:51] That was also our idea would be if we democratize ML and AI enough, everyone could be a model builder.

[00:17:59] And actually, just before talking with you, I was playing with my kids and we were trying with my son, we were trying the new Flux model.

[00:18:05] So the image generation for Black Forest, which is totally amazing.

[00:18:09] So good.

[00:18:10] And so we went just exploring spaces, trying it and then fine tuning it on a couple of images.

[00:18:16] And you think, OK, 11 year old can now fine tune the ML model.

[00:18:21] And so the frontier between, you know, ML builders and ML users is really, really thin.

[00:18:27] And that's very good, I think, because our idea was always, we think the very long term vision is we think AI should be a common good.

[00:18:36] Or like for me, as I said in my physics view is everyone can learn about general relativity, quantum physics.

[00:18:43] Anyone can learn about this.

[00:18:45] These are like common good knowledge from all humanity.

[00:18:48] And I think it's great if, you know, only, I don't know, Microsoft would know everything about quantum physics.

[00:18:53] That would be very sad, right?

[00:18:55] And I think AI is a bit the same.

[00:18:57] It's a very fundamental, you know, revolution.

[00:18:59] It's a technique that's going to change, obviously, a lot of things we do.

[00:19:03] And it should be something that anyone could understand.

[00:19:06] And that if anyone would, OK, I want to understand now how this thing works, they should be able to do that because it's going to control so many things in our life.

[00:19:14] And the best way to understand something is, I mean, first to share it in an open source way, but also to be able to use it very easily.

[00:19:22] You know, you go to a space and you're like, OK, how does this model work?

[00:19:26] Well, you try a little bit.

[00:19:27] You try a couple of examples of prompts.

[00:19:29] You try, if it's a chat model, you try to ask a couple of tricky questions.

[00:19:34] And spaces has helped us a lot to democratize this aspect and make Huggy Face this kind of place where you can find models and you can also understand them.

[00:19:44] And the third thing we added is data sets.

[00:19:46] Because in the end, what we are discovering is that the data that you put in these models is basically the core thing that makes everything works.

[00:19:54] You know, the quality of the data and the quantity.

[00:19:57] And so a couple of years ago, maybe three years ago now, we started this second approach around data sets.

[00:20:04] So we also started a second library called data sets.

[00:20:08] And we started to host data sets on the hub.

[00:20:10] And the idea was, let's open this new black box.

[00:20:13] OK, so we have the models.

[00:20:14] We've opened roughly the black box of the model.

[00:20:16] We've opened weights.

[00:20:17] We've opened a source around the model code.

[00:20:20] But the data set should also be accessible.

[00:20:23] If you want to understand AI, you have to understand how to make good data sets.

[00:20:27] So we have to push people to share them in an open way.

[00:20:30] And this has been growing also very strongly on the hub.

[00:20:33] We have, I think, roughly maybe 300,000 data set.

[00:20:37] But I would have to check because this is exponentially increasing.

[00:20:41] And this is very exciting, I think.

[00:20:43] Because when you have all of these things open, basically, you can really say, OK, now anyone can train a model and basically join the fan.

[00:20:54] I think that's really, it's so interesting to hear you talk about openness both on the model side.

[00:20:59] And I think the data set side gets less attention.

[00:21:01] You know, to your point, yeah, these models are like, if they're open weights, you know, it's like this inscrutable set of matrices.

[00:21:08] Like, what do you like? All you all you can do is use it. Maybe you can fine tune it or whatever.

[00:21:12] But when it comes to the real, I guess, like, I don't know if oil or gold is the right analogy that is like distilled wisdom into these models.

[00:21:21] It comes from the data and making that openly available is so huge, too, especially if you've got an eager community.

[00:21:28] That's like kind of rather than just, you know, a select group of people at various labs working on this stuff, you can have the whole community, including indies and hackers working on it.

[00:21:37] Which sounds exciting, but it almost makes me want to go back to the bigger question here, which is, you know, in the past, in previous interviews, you've said that, you know, we need to fight for open source.

[00:21:47] Why is open source important and why should we fight for it?

[00:21:53] Yeah, it's a big question. It's important for many reasons.

[00:21:57] Maybe the most practical one is because of balance of power. I think open source is a great way to make sure that you can have many people participating in the AI revolution.

[00:22:07] And not only, you know, like a small set of, you know, well-founded labs, but basically anyone, you know, any smart person from anywhere in the world who have a nice idea can join.

[00:22:20] And because they can start from this openly accessible weights data, they're almost at the level of the top lab from the get-go.

[00:22:29] So that's really nice. I think it brings, in my opinion, it brings a lot more people around the table.

[00:22:35] So it's really great to catalyze, you know, new research ideas to basically, you know, accelerate knowledge in the field, but also to include other voices.

[00:22:44] We also tend to think sometimes that AI is just built by, you know, a small set of ML people, you know, math-focused white male, for instance.

[00:22:55] But I think it's really nice if everyone can join. So basically, if everything's open, a lot more people can join, participate.

[00:23:02] I do want to talk about the other side of this, right? When a lot of people push back against open source, especially in the recent discourse around, you know, regulation in California and just like America at large.

[00:23:14] You know, how do you think about balancing that need for innovation versus like the potential risks of making these like some very powerful AI tools extremely accessible?

[00:23:24] Hmm. So I remember maybe one year ago, there was a lot of fear around, okay, AI is going to be like a Terminator or is going to kill us all in the next month, right?

[00:23:37] And there was a lot of like, I would say needlessly worries around existential risks.

[00:23:42] And we would say, okay, in January, I heard some people say, we're going to be all dead in January, 2024.

[00:23:47] AI is going to be killing us.

[00:23:50] Obviously, it didn't happen.

[00:23:51] We're not paperclips yet.

[00:23:53] I think a lot of this was kind of pushed also by this, you know, obviously this interest from the Pioneer Lab to maybe get some early regulation that would, you know, slow down a bit.

[00:24:03] So a lot of this was, I think, some kind of lobbying pressure.

[00:24:07] And I think thankfully people kind of understood that this is also another tool.

[00:24:12] It's going to be not like, should we forbid this technology or not?

[00:24:16] But it's going to be more, okay, there is a set of usage of this technology that should be forbidden.

[00:24:23] And so regulation, I think, is very nice.

[00:24:25] And it's something we welcome.

[00:24:27] We've been participating also, giving some feedback to the various regulators.

[00:24:33] We don't actively lobby for anything, but definitely we think some regulation is great.

[00:24:38] The only thing we would like to avoid, I would say, is some kind of regulation that would be regulatory capture, basically.

[00:24:45] So, I mean, to explain, it's basically if you make like a process to be approved by the government, which is so complex that basically only a couple of very large companies with the lawyers and, you know, the money to do that will be able to do that.

[00:25:00] I think it would be a very sad outcome because basically you will end up having this oligarchy of like this very small monopoly of company deploying the technology for everyone.

[00:25:11] I don't think it's a great outcome for us.

[00:25:15] I think it's kind of the beginning of any very good dystopian movie where you have basically one company, you know, controlling the core technology that's used.

[00:25:23] It's how easily we start a very good sci-fi movie.

[00:25:25] But if we stay clear from that, if we have regulation that makes sense and that's also open to smaller teams, smaller companies that's accessible, then I think it will be very nice.

[00:25:36] I think you're totally right.

[00:25:37] These tools are very dual use, right?

[00:25:39] And it's about time rather than trying to regulate the tools themselves, we got to focus on the usage of those tools and what people are doing with them and hold the people accountable, right?

[00:25:48] Because ultimately this comes down to power dynamics, as you said, right?

[00:25:52] Do you want to have this technology be the dominion and sort of the purview of a handful of extremely large companies, you know, oligopoly, as you said, or do you want there to be a balance in power?

[00:26:04] And what's interesting, particularly about the open source space that you're a huge, you know, kind of I would say like community builder and accelerator of is how well open source is actually done.

[00:26:17] I mean, I remember the narrative like a year and a half ago, you know, it's like maybe even a year ago, Ilya Sitzkevira was in Tel Aviv and he was like, well, I think these models are always going to be the domain of the big labs.

[00:26:29] And, you know, open source will never be able to catch up, et cetera, et cetera.

[00:26:34] And this past year, there's just been so much excitement.

[00:26:37] Like personally, Llama 3.1 by Meta, which by the way, is kind of funny that Meta is like the bastion of open source.

[00:26:44] Maybe again, economic incentives can sometimes be a good thing, I guess.

[00:26:48] But how do you think that like power dynamic has evolved over the last year, year and a half?

[00:26:52] Because it feels like open source was the underdog and now it's neck and neck, it almost feels like.

[00:26:57] And it's also cool how accessible that's gotten.

[00:26:59] Then you mentioned, you know, playing with your kid, fine tuning, you know, like in creating like clones of just taking a couple of photos and suddenly you can have so much fun.

[00:27:07] It's so accessible now.

[00:27:09] And that's all way easier than I think people think, which is like open source isn't something you need to be super developer savvy to take advantage of anymore.

[00:27:19] Yeah, but you're right.

[00:27:21] There is still a, I feel it and I felt it today with myself.

[00:27:25] There is still a little bit of entry barrier.

[00:27:27] I think we could remove with open source.

[00:27:30] There is still like a little bit things that are tech savvy.

[00:27:33] It's getting better, but it will be nice to see this becoming more and more accessible.

[00:27:38] With the goal, I think one of the very nice goal is also, like you're saying, if you have your use case, it's maybe specific to you.

[00:27:44] I don't know, you take your notes on your phone and then you want to convert them and put it.

[00:27:49] It would be very nice if you could just plug a couple of these things together and get your thing that, you know, do the things for you and tailor to you.

[00:27:57] So I don't think like, you know, it's going to be tackled by the big players very soon.

[00:28:01] But if you have enough, you know, simple bricks that smaller companies can make fit together and this is going to be very nice.

[00:28:09] And that's where you have real democratization where basically, you know, it will start to be very easy to automate some part of the boring workflow that you have and basically concentrate on cool stuff.

[00:28:22] Talking about bullshit tasks, that's also something I'm very excited about today, which is open source robotics, which I think is going to be big probably next year or maybe the year after, but very soon.

[00:28:35] So when you think about robotics, what do you mean by robotics?

[00:28:38] Yeah, it's funny because I don't really like humanoid robots.

[00:28:41] I find them quite scary.

[00:28:42] I don't really want any of them in my house.

[00:28:46] I mean, I like robots that I can understand.

[00:28:48] So I've really partly built myself or they're made of open source parts that I know.

[00:28:54] And maybe they don't even have the human factor, but they can have, you know, fun factors.

[00:28:59] And you can have one that you just made yourself, you train it to, you know, take care of your dishwasher for yourself or like this or fold your clothes.

[00:29:06] That's a project we have this week, which is like two small arms that you put and they just fold your clothes.

[00:29:12] And this is much less scary, I think, than actually buying a very large humanoid robot that costs 10, you know, 10 grants and that sits in your house.

[00:29:21] So I think there's a whole range of robotics, I think, which is going to be quite cute, quite fun, quite excited.

[00:29:29] Pete?

[00:29:29] It's such a good point because, you know, I'm seeing your philosophy manifest itself.

[00:29:34] Like there's an interesting parallel where the way you're thinking about models, right?

[00:29:37] You've got, you're going to, yes, you're going to have some very large models, but you're going to have this like massive long tail of small Taylor specific models.

[00:29:44] Some are going to be running in the cloud.

[00:29:46] Some are going to be running on the edge, closer to your devices.

[00:29:49] Like kind of the models you mentioned, you initially mentioned you were focusing on NLP, natural language processing, understanding, you know, language.

[00:29:57] And then, you know, vision stuff started popping up.

[00:29:59] How do you understand and perceive the world, right?

[00:30:02] And now you can kind of take those primitives and put them into these robots using like Arduinos and like off the shelf stuff, which, oh gosh, it must be so exciting to be a kid in engineering school these days.

[00:30:14] It's like the plurality of options are just staggering.

[00:30:18] Yeah, it's a crazy time to be.

[00:30:20] I think what is crazy is you see all the things you can do and it's like, it's like an endless greenfield.

[00:30:27] You're like, nobody tried this yet.

[00:30:28] No, nobody tried because just we can't do that since just last month.

[00:30:32] And you're like, wow, let's try it.

[00:30:34] I mean, I saw that with, yeah, with my son just now scrolling through the spaces around Flux.

[00:30:39] You can do that?

[00:30:40] Yeah, okay.

[00:30:41] And oh, and you can also do that.

[00:30:43] It's like, it's so exciting.

[00:30:45] Yeah.

[00:30:46] I think probably the main problem for kids today is just to choose one project to do because there's like 10,000 possibilities.

[00:30:54] Yeah.

[00:30:55] Yeah, the problems are plenty, as they say.

[00:31:14] So here in the last bit of this interview, how do you see the relationship between open source AI and commercial closed source AI evolving over time?

[00:31:22] Yeah.

[00:31:23] Yeah.

[00:31:24] I think probably, in the end, probably rather similar to the dynamic we have in software where you have basically both, you know, coexisting, you know, you have open source, but definitely you still have a lot of closed source software.

[00:31:36] And that's fine.

[00:31:37] I mean, even on the hub, right?

[00:31:38] We have a lot of open model.

[00:31:39] People also use a lot of the hub for private models.

[00:31:42] So like maybe half of it is closed source model, actually.

[00:31:46] And that's fine.

[00:31:46] I think it's quite healthy, right?

[00:31:48] If you took a Lama and you fine-tuned it on your enterprise data, you shouldn't have to open source this model again.

[00:31:56] Okay.

[00:31:56] It's fine.

[00:31:57] It's your private data.

[00:31:58] You manage to gather it or like whatever.

[00:32:00] I think it's very healthy.

[00:32:02] So we have both coexisting, I think, on both climbing together with this competition of like this healthy competition.

[00:32:09] Obviously, open source is very nice because it allows people to combine stuff together.

[00:32:13] So this idea of plugging models with, you know, image, adding speech, like you were saying, you can do that very easily with open source model.

[00:32:21] And that's quite exciting.

[00:32:22] Open source model will also be the place where the kind of smaller community or like the more hacky community will continue to strive.

[00:32:29] I would say right now the situation is rather healthy.

[00:32:32] I would say we have a mix of both of them.

[00:32:34] I hope it stays like that.

[00:32:36] But the same with access to compute, right?

[00:32:39] As access to computers are getting easier and easier.

[00:32:42] And so, yeah, we have something that it will be like computers, you know, like we now all have these very small laptops and we used to have this very thin MacBook.

[00:32:51] But you probably grew up like me in a time where we used to have very large thing.

[00:32:55] And we were like, oh, that's how a computer should be.

[00:32:57] And now when I show, I still have one desktop with GPUs.

[00:33:00] I show it to my son.

[00:33:01] It's like, oh, that's weird.

[00:33:02] It's very big.

[00:33:02] Yeah.

[00:33:03] It used to be the standard.

[00:33:04] So, we have this very small and very versatile and very smart models that we can use a bit everywhere.

[00:33:10] And we still have also this very large model.

[00:33:13] But yeah, for the rest, I would say the very interesting thing now is also that we are moving out from just focusing on building models.

[00:33:21] But kind of taking models as a given.

[00:33:24] We're like, okay, we have LAMA.

[00:33:25] We know it can do a lot of things.

[00:33:27] And now, what are the most interesting verticals?

[00:33:30] What are the most interesting use cases that we can apply it to, right?

[00:33:34] Because chatting is obviously one use case, but chatbot is also not the end of the goal, right?

[00:33:41] And people, I think in the early AI, they're like, okay, making models is going to be the big business.

[00:33:46] That's where everything's going to be.

[00:33:48] I very much disagree with this.

[00:33:49] I think models are going to be really commoditized, but it's very exciting to watch, to follow.

[00:33:54] So you're totally right.

[00:33:56] That vibe shift has been so clear because even, again, a year, year and a half ago, all the startups that were hitting the scene were like, well, we train our own models.

[00:34:06] It's like we're not a GPT wrapper company.

[00:34:08] Whereas over the last year, as you've seen some of these startups kind of go back and return to larger labs, like Character AI perhaps being a recent example, but also like Pi with inflection and others like it, right?

[00:34:21] Similar thing with Amazon as well.

[00:34:23] What you're seeing is sort of like, it's almost desirable to be a wrapper company.

[00:34:28] Because again, like the model layer might not be the piece where all the value will be created.

[00:34:33] It's the use cases, it's the products and experiences.

[00:34:36] And so I have to ask, even though you can't tell me the killer app, since you are running, you know, the largest open source community, I would say, you know, people literally call you the GitHub of AI is the phrase that I've seen used.

[00:34:49] What are some of the most exciting things you're seeing these days that you're like, holy crap, like, give us like a flavor, a hit list, if you will.

[00:34:58] Yeah.

[00:34:58] No, I think everything around the AI is very, around image is very interesting.

[00:35:03] So you are playing just also before this interview with a new space that actually makes you wear, you know, a piece of clothes.

[00:35:11] So you put the image of clothes, you put yourself in whatever position, like from side, from the back, and it's just really great.

[00:35:17] And you put this cloth on you.

[00:35:19] So, and this is obviously something, I mean, there's been a couple of companies already trying that, right?

[00:35:24] But that's one of the reasons where I'm really like, oh, okay, now the tech is really, really good.

[00:35:30] So this is going to be deployed everywhere, virtual try-in, and probably also integrated.

[00:35:35] And I think would be, yeah, integrated also in daily life experience.

[00:35:38] Okay, you move around and you just actually see how it's going to look on you and this type of thing.

[00:35:42] So around image, I think there is a lot of really cool thing beyond just the artistic mid-journey thing we started with.

[00:35:53] So I think there's going to be a lot of very interesting use cases.

[00:35:57] Around text, it's quite tricky because I think a lot of this is in the UX and the UI and how you put that, how you integrate that in your workflow.

[00:36:06] So a good example, if you're a coder, is the recent cursor code helper.

[00:36:10] That's all my feed is, by the way, is clawed cursor.

[00:36:13] Oh, yeah.

[00:36:13] Yeah.

[00:36:14] That's all I see.

[00:36:15] So it's funny because it means if you're carefully focusing on the user experience, even if you don't build your own model,

[00:36:22] you can really make something that gets people extremely excited about.

[00:36:26] Even if there is like a huge competitor like Copilot, which is pushed by Microsoft, integrated directly in the thing,

[00:36:33] so it is the obvious option you want to try and start.

[00:36:36] But the entry barrier for people who care deeply about their users is still quite small.

[00:36:42] So it's very exciting.

[00:36:44] Video is very exciting.

[00:36:46] I think we work also on video.

[00:36:48] I think video is very nice because, yeah, there's a lot of content.

[00:36:51] Like sometimes you don't want to watch a movie.

[00:36:55] There is some need.

[00:36:56] Sometimes you just want a summary of something.

[00:36:58] Sometimes you just want to jump to a part of a movie or a meeting or something.

[00:37:02] So being able to have something that understands process video in a real way is going to be game changing for so many applications.

[00:37:11] So AI could unlock really amazing things here.

[00:37:14] Yeah.

[00:37:15] One thing I'll say is like what you're talking about, just like all these modalities, these different mediums of understanding and generation are all sort of stacking on top of each other in a cool way.

[00:37:23] Even as you were talking about the example of video, like I got so excited because, gosh, you're totally right.

[00:37:29] We can certainly summarize like massive reams of text, like 12-page papers and give me the distilled summary or highlight a section.

[00:37:36] And then for video, that's a little bit more challenging, like being able to take a video and like kind of create the summary version of that.

[00:37:42] I think there's so much latent knowledge sort of sitting around waiting for us to access, which is very exciting.

[00:37:49] And even let me just open on something maybe you didn't thought about, but modalities we can't use, see, or access.

[00:37:54] So I have a very cool project we are pushing right now on a public cluster, which is around modeling molecules.

[00:38:00] It's a foundational model for quantum chemistry.

[00:38:03] So, I mean, I can see images, I can hear a speech, but I cannot feel a molecule.

[00:38:10] That's something I will never be able to do.

[00:38:11] But AI model can do that and can access this thing that I can generate.

[00:38:16] Making the unseen scene in a sense.

[00:38:18] Basically like this, yeah.

[00:38:19] I love it.

[00:38:20] That was a great conversation.

[00:38:21] Thomas, thank you so much for joining us.

[00:38:23] Thanks, Ben-Hawel.

[00:38:24] It was great.

[00:38:28] So if there's one thing you take away from today's episode, it's this.

[00:38:32] Open source is absolutely killing it in the AI space right now.

[00:38:36] Remember our GitHub deep dive?

[00:38:38] Well, multiply that impact by about a thousand for AI.

[00:38:41] The innovation happening out in the open is mind-blowing.

[00:38:44] I mean, think about it.

[00:38:45] You've got what?

[00:38:46] A small number of product managers and engineers in these big labs, right?

[00:38:50] They're working on a tiny slice of what's possible.

[00:38:52] But throw these models out into the wild, and boom.

[00:38:55] You've got an entire community going nuts.

[00:38:58] Pushing boundaries we didn't even know existed.

[00:39:02] Take Meta's Llama 3.2, for example.

[00:39:04] Meta is one of the biggest labs out there.

[00:39:06] And as of this recording, in October 2024, they just released a huge LLM to the general

[00:39:12] public.

[00:39:13] And yes, you can find it on Hugging Face for free.

[00:39:16] This thing is nipping at the heels of GPT-4.

[00:39:19] And in some cases, it's even pulling ahead.

[00:39:22] Folks are using it to build all kinds of things, right in their own homes, from rudimentary

[00:39:28] chatbots that become increasingly sophisticated as you use them, to custom voice-activated assistants

[00:39:33] on their phones and computers.

[00:39:35] And that's just scratching the surface.

[00:39:38] Or look at AI image generation.

[00:39:39] Yes, Midjourney's still the top dog in terms of usage.

[00:39:43] But Flux?

[00:39:44] The free open-source image generation model?

[00:39:47] In just a month, it's grabbed so much mindshare, it's not even funny.

[00:39:51] And remember, this is from the same folks who gave us stable diffusion.

[00:39:55] The OGs of open-source AI.

[00:39:58] And it's like clockwork.

[00:39:59] First, you get scrappy, duct-tape projects.

[00:40:02] Next thing you know, this tech is baked into products we use every single day.

[00:40:05] And let's not forget about data privacy.

[00:40:08] If you're running a business and your data is your lifeblood, you don't have to take

[00:40:12] anyone's word for it.

[00:40:14] Just fire up your own models on your own hardware.

[00:40:17] No sketchy terms of service.

[00:40:19] No trust issues.

[00:40:20] You're in control.

[00:40:22] That's the beauty of open-source.

[00:40:24] It's all about what you can build, not what some gatekeeper allows you to do.

[00:40:28] In fact, I'd argue that in the race to push AI boundaries, open-source isn't just

[00:40:34] keeping pace.

[00:40:34] It's setting the pace.

[00:40:38] The TED AI Show is a part of the TED Audio Collective and is produced by TED with Cosmic

[00:40:43] Standard.

[00:40:44] Our producers are Dominic Girard and Alex Higgins.

[00:40:48] Our editor is Banban Cheng.

[00:40:50] Our showrunner is Ivana Tucker.

[00:40:52] And our engineer is Asia Pilar Simpson.

[00:40:55] Our researcher and fact-checker is Christian Aparthe.

[00:40:58] Our technical director is Jacob Winnick.

[00:41:00] And our executive producer is Eliza Smith.

[00:41:03] And I'm Bilavul Sidhu.

[00:41:05] Don't forget to rate and comment.

[00:41:07] And I'll see you in the next one.