AI in the Cloud

Today we have Thomas curian along with Our moderator Frederick lardenois please Welcome them to the [Music] Stage all right full house we just spent 20 minutes talking about soccer and Cricket so got to switch up the Conversation a a little bit here Um when you joined Google cloud in 2019 You and I sat down at some point at Google next and we talked about Kubernetes and multicloud and kind of Infrastructure things and servers the World has changed quite a bit since then And I was at Google Cloud next a few Weeks ago and I was joking we should Just call it Google AI next um at what Point did you realize that AI was going To be the next big thing for You probably as far back as 2020 you know our our vision for cloud Computing has always been super simple It's been about Simplifying technology so that people Can use it to build great Applications the first problem we solved Was using Cloud to deliver Infrastructure instead of having to have People buy data center space and Machines and manage them we said we can Just give you infrastructure as a Service uh then we built manage services On top of it so that if you had Challenges getting access to people uh

For example to manage a database or to Do cyber security or to build Collaboration tools we could give it to You as a fully managed service so when AI came Along we started our work at Google back You you know almost 20 years ago on AI And we saw that the application of AI Into Applications uh whether it was search or YouTube or Gmail was the central way That people were going to experience Ai And so we felt that if we can give People a Platform one where they can either Develop applications using models Extremely easily and where the platform Would simplify how they used AI or for End users where our products themselves Incorporate AI to represent all the Skills they needed it would hugely Transform the way they experienced Ai And so there have been many many moments On that Journey but we've been working On it now for almost four years and when You're talking about AI right now you Talking about generative AI already at That point or we more thinking about the Machine learning as we used to call it Back then we've worked through four Evolutions of AI you know the first Version of AI was what we were doing Called classification classification for Those of you who use Google search if

You go and type into Search and say find Me an image of a cat uh it returns an Image of a cat and not a dog because It's doing Classification if you then say find me An image of a cat in my drive but you Don't have image of a cat in your drive But it returns image of an animal and Not a plant it's doing something called Categorization the third flavor that we Brought out was prediction prediction is Hey I'm Managing uh supply chain or Inventory or demand picture tell me how My inventory is going to look like if Trends continue like this or if they Change and that's using statistical Methods or what we call basian models to Do Prediction and then generation is where You train a model on a set of data and Then the model can generate output based On the data was trained on and so those Are multimodal models where you can talk To it using either chat or text and it Can respond in text chat it can generate Code it can generate images audio video Different modalities so for us this is Our fourth generation many of you use Our products like for example if you use Gmail Uh we're helping you write and we're Helping you write at scale um we use Underneath when you say you get

Suggestions like autocomplete something For me or type in things we do that 180 Billion times a year which is half a Million times a day half a billion times A day 500 million times a day and so We've been using AI inside our products From as early as 2014 so that's you know A lot of history in how we brought both Built AI models but also brought it to People was there a moment for you when You first saw a demo of generative AI Where you said yes this is something we Have to go all in On we we had I mean there were many Different Applications which we saw were truly you Know Transformative uh one of our teams for Example we run many of the largest Contact centers in the world And contact centers are fairly Sophisticated in the sense of the number Of different types of questions you get And how sophisticated your understanding Of a user's intent and being able to Respond to it um you know one of our Teams has built a model which is Actually being used by many companies Now to understand questions and we had Uh an Organization provided a billion user Inputs meaning there's potentially a Billion different combinations of Questions that could come in and the

Model was remarkably accurate in Responding and that's when you have the Moment that you know a normal person Would have a hard time remembering a Billion combinations and being able to Handle all of them and here's a model That's handling a very very Sophisticated set of questions and being Able to do it with remarkable Fidelity most people can hand no just Kidding so Um let me talk business for a second Here cuz you know when you took over Google Cloud Google Cloud was kind of The number three Cloud you still are Yes when you think of AI now is that a Moment for you to kind of reset this Game you know just if you look at our Growth when I Joined Google you know our Cloud Business was the 28th largest software Company in the world today we are third In Cloud but we are the fifth largest Software company in the world there are Only three other four others larger than Us and they were all founded before 20 Before the year 200000 so in four years We've created one of the five largest Software companies in the world uh we Look at our growth rate as a metric of Our Success a big part of it has been Finding The problem that customers are going to

Want to solve and being able to solve it Better than anybody else so when we look At AI we saw people wanting to have four Capabilities one is really Worldclass infrastructure for training And serving Models and you know the kind of systems That you can give people to be able to Train models faster and cheaper and also To inflence We today run 50% of all AI startups in the world and We run 70% of all the AI unicorns and so The companies that really know the Technical detail of what is needed to Serve and you know to train and serve Models choose us because we have the Most optimized infrastructure so that Was one the second thing we saw is that Companies founding on AI particularly Those that want people people to use Them as scale you need a set of services Around the model itself and so we built A platform and through that platform we Make models available not just our Models Google models but also models From third parties and startups but we Surround them with a number of Capabilities that people need if they're Going to build applications take a Practical matter models are known to Hallucinate meaning they answer and the Answer may not be factual so how do you Make sure you check and look at where

The model derive the answer from that's Called grounding we have services for Grounding for watermarking for Responsibility controls for synthetic Data generation reinforcement learning Feedback automation remember Google Invented most of these things all the Things that we use in our SE search in Our uh YouTube and other platforms the Technology that Engineers use at when The when we use models internally we're Exposing through a platform for Developers so that you get the benefits Of all of that so that's number two and The third thing we've always felt is Models are evolving on a multi-year Journey to do all the skills that humans Have and so when we look at that the the The idea we had with models was to take Each role that people had and have a Model augmented so for instance if You're a software engineer at Google you can use a model to write Code you can use a Model to generate the documentation for The code you can have the model do code Inspection which is I submit code before I'm allow to check in I'm going to have The model actually inspect my code to Make sure I don't have a security Vulnerability in my code for example you Can have the model generate tests I know Engineers love to write tests right Nobody likes to write tests so the model

Can generate unit tests and so this is An example of something that we built Our people are using a scale but we're Also making available to other companies Now so that you can have an expert po Pair programmer work alongside you and So when we looked at AI we felt some People will want amazingly cost Effective scalable infrastructure to Train the models on some people will Want a platform where they've already Built a model but they want to reach Developers through the platform that's The second piece and the third piece is Some people will want to use models to Build solutions for end users and one Type of profile is what we're doing for Programmers but we're building these end User profile solutions for a variety of Other uh users as well so want to talk About the open ecosystem and programming For a moment but to get back to business For a second Um do you do you think AI is going to be The main driver of growth for You you know we're seeing Growth you can't grow from where we were To one of the five largest companies Just depending on one driver we do do Think AI will Infuse a lot of our Solutions and just to give you an Example we've taken every part of our Portfolio and built an AI powered Experience for people so think of I'll

Just give you two Examples if you use our collaboration Tool we took each Persona in the Collaboration tool and built an AI part Experience so if you're using our Document authoring tool Google Docs we Can actually help you write and an Author can assist you Writing if you're a people who love to Do slides but you don't know how to do Graphics very well you can actually type In text and we can generate the graphics For you many advertising agencies are Using it to actually build ad campaigns So we we took each of the profiles if You attend video conferences I know all Of you love to do video conferences and If you you want to take you want to have A person who Can transcribe the meeting summarize the Action items of the meeting generate a Summary of the meeting translate the Summary into all the languages because Many people are working across different Countries you can do that automatically So we've taken each of the personas that People typically want because they say I Wish I had an administrative assistant Who could take meeting notes and now We've built it with AI it's the same Thing for using a platform if you're a Developer who wants to access our Platform you can go in and say take as a Very simple example many people have to

Do data Analysis and they get questions all the Time what are the numbers how are our Numbers doing so we wanted to simplify It so we said you can go in into a Search box and say tell me how revenue Is Trending the system will one calculate Your Revenue Trend two it'll tell you Why revenue is trending that way third It'll generate a Narration fourth it'll build a chart Fifth it'll create the slide deck and It'll write the slide deck for You that's what is available now and so A lot of what we've done with AI is to Take Ai and Infuse it into all the tasks That people want to do to make it so Much easier to get them done and by Doing that we want to be able to open up Technology for more people because the Easier you make it the more people than Get access to it and you've brought up Workspace quite a bit now you know the Consumer office suite side of Google Cloud I'm not sure everybody's aware That those all fall under your uh regime But um did you ever think of splitting Those two up and did kind of this AI Revolution change that discussion in any Way so workspace for those of you use Gmail or Google Docs we offer an Enterprise version of that called Workspace it's used by over 10 million

Companies around the world and Government agencies and other People what we're really trying to do With it is twofold one to make the Experience of using it if you if you Think about what email systems word Processors Etc were a word processor for The last 30 years was just a replacement For the Typewriter a spreadsheet was a Replacement of a Calculator a slide deck was a Replacement for what people used to do Called slide Projector we felt that with AI you Should be able to do More we should make the word processor An author to help you write we should Make the spreadsheet an analyst that Will help you Calculate we should take the slide Projector and replace it with a digital Graphics artist an AI powered graphics Artist that you can collaborate in real Time and that's why we call it duet Because you are working with somebody a Digital somebody in real time and it can Ass you in doing tasks that you couldn't Do yourself now we've integrated that Into the the Google Cloud platform Because many developers say hey I'm I'm Working on a project I want to chat with Somebody to get some help I'm working on A project I want to generate the

Documentation for my application and I'd Like to use the document authoring Facility to generate the documentation For my project so we've integrated and Made it extremely easy for for people to Use these pieces together and AI brings All the skills that you need into one System so people can do it the thing With AI is that it's it's expensive to Run and you've got these consumer Services that are being used by billions Of Users is AI going to be one of the main Differentiators in between the Enterprise versions and the consumer Versions where the Enterprise version Will have more sophisticated features in GMA and docs or how are you thinking About that so the the models that we use Today if you go to Google Search we have Something called search generative Experience which is our new generative AI powered search experience the models That sit under search we're making Available to everybody so every model That we use internally we're making Available to customers as well to build Their applications and we're making it Available at the very same time as we're Doing internally now to make models work Well we've been very clear based on our Own experience that you want to choose The right model for the task and when I Say the right model models can be of

Different Sizes the largest model is not always The right one let me give you an example Imagine you're a soft engineer and You're writing Code if you want to generate a function You say please generate this function And you wait if you're typing code and You want it to autoc complete you want That code fragment that you're autoc Completing to happen in real time Because otherwise you type and then You're typing over something the model Is generating so the smaller Models have better Latency they can be lower Cost they can also answer many questions That you may not need the Lara model to Solve for so most clients that we work With we actually allow them to use a Combination of models so you can start For example talking to the mod smaller Model if it can't answer the question it Can transparently fall back to the Larger model and for those of you use Gmail you can actually see it in the Preview that we're running right now for Customers so you can for example on your IOS or Android device you can bring up The Gmail app if you're part of our Preview and you can say help me WR it Talks to a model that's running on the Phone you can imagine that's a highly Compressed highly distilled model the

Reason we're doing it on the phone is Both to make latency low and if you're In a place that you don't have great Connectivity you can still write if you Want the model to do more sophisticated Things it can fall back transparently to A larger model that runs in our cloud And so with we're trying to give people The choice of the best model which can Be a combination of size cost latency And a variety of other Characteristics but just to go back to The question does it Mean the free user won't get that choice And we get the smaller model or what is That going to look like we are there is A set of features that are generative That we made available generally in Gmail to everybody we have not yet Announced consumer because we're still Testing with consumers uh we're also Testing in different languages you know Because we've got a large user base Outside the United States we've said Publicly that we will be making the Product available to Consumers very Small businesses small businesses and Large Enterprises and there'll be a set Of capability that we just include in The Product all right let's go back to the Models for a second there again because When I was at Google Cloud next a few Weeks ago this open ecosystem of models

Was something you were highlighting Quite a bit that that that means you're Working with some you not competitors But other like Facebook for example with The Llama models uh which is somewhat Unusual right but why is that open Ecosystem so important to you so what we Work with three types of models Obviously people ask can I get Google's Latest models yes and as I said the same Models that sit under our consumer Properties are available at the same Time as that available inside the Consumer properties we also work with a Variety of Open Source models open Source models like stable diffusion Llama 2 uh code Llama uh Falcon there's a variety of Them the way we do it is we actually PLL Our developer Community to see what they Really want us to work with and the Popular ones we actually package in a Pie torch container and make that're Available for people so that's number Two number three there are some Companies that work with third-party Models third party models are not open- Source models but third party models Like Claude for example from anthropic We do support Claude we do support uh Cohere A1 AI 21 Labs is available Runway Which is a super popular media uh you Know video model is available uh we will With mid Journey so we want to allow

Customers to choose the best one in some Cases the best one will be from Google In some cases the best one may be from Somebody else and we offer you one Platform where you can use all of it uh So that you don't have to choose Different tools every time you want to Choose a different model and that's Somewhat different from some of your Competitors do you think that's a major Differentiator for you is that part of Why you're doing this we we definitely Feel that customers should choose a Platform to build with AI and we always Tell customers if you look at the large Companies we work with whether that's Wendy's or Price Line or we've had Hundreds of customers do announcements With us Mayo Clinic and others they're Deploying it across a range of different Environments and we always tell them the Most important thing is you need to Choose a platform that provides you the Ability to manage models test them Evaluate them handle change management All this stuff and so when we did that People said can you allow me to use a Range of models not just yours with the Platform and we felt that was the right Thing run those models and run them on Google Cloud it's a that's you can run Those Models Super efficiently because They're getting the infrastructure to Run that we're using to run our search

Models our Gmail models so you can Imagine we've spent years optimizing Performance and cost to serve those Models EXP extremely cost effectively We're making that available to third Parties to use to serve their models to The developer ecosystem just as cost Efficiently as our models and talking About that infrastructure a lot of the Industry is suffering from just not Being able to get the gpus and the chips They want is that a problem you're Facing as well you know we've seen this The evolution of infrastructure for many Years and as AI moves from the realm of Data science and Research into Applications you actually need a range Of different kinds of accelerators so as An example if you train a model Depending on if you're building a sparse Model or a dense model if you're using Something called mixture of experts Versus a single large model the kind of Chips and system you need are very Different uh because you need to decide How much coess memory do you need uh Versus how much memory per chip do you Need Etc same thing for serving you need There are many many techniques now being Used when you serve a model uh for Example if you've got data in a data Store people use something called Vectorization to expose it uh depending On how much is textual versus numbers

You may decide how much floating Point Optimization you need so what we have Seen at Google because we're in our 10th Year of building ml systems is you will Need a range of these different kinds And so we offer 13 different kinds of Accelerators so that people can choose The one that's really optimized for Their needs part of the value of that is We're not bottleneck on the supply chain Constraint right now the supply chain Constraint is around a manufacturing Process called chip on wafer on Substrate many accelerators don't have That bottleneck and because we offer the The diversity customers are we don't Have the same constraints in offering People the ability to use this Infrastructure you said earlier on your Constraint is more in the number of People you have that can install those Chips in your data obviously the you Know we've not yet built a system where A chip just walks into the data center And plugs itself in right so there's Lots of logistical work we have going on To get the machines in we use a water Cooling for some of our larger system Systems because they really provide a Maturely better performance and cost of Training so there are many many people At Google working on making sure that Part of the ecosystem works well all Right well we'll head out and install

Some h100s somewhere in the data center Thank you very much thank you all thank [Applause] [Music] You