Chasing Generative Video

We have co-founder and CTO of Runway Anastasis germanitis and Kyle wiggers as Our moderator please welcome them both Back to the stage [Music] [Applause] Well anastasis am I pronouncing that Name correctly perfect yeah awesome just Wanted to make sure before we get too Far into it thank you for joining me on Stage you're running a really Interesting company Runway it's found incredible success in Recent years we'll talk about all of That first of all though I'd like to Know where the name came from because as You're probably aware there's there's a Couple of startups named Runway and they All do very different things so what What was that brainstorming session like Yeah it's actually not uh It's not a very sophisticated process That uh where we arrived for the with a Name I would say that like the initial Version of Runway was in a platform that Allowed creatives to use a variety of Open Source models so it's a place when You when models run so that's that's Kind of the the whole context of it it Doesn't it's not that much more Complicated no I was just curious um Makes perfect sense Um now you and your co-founders come From really interesting backgrounds um

From what I understand you met at NYU And art school Um so what was that transition like from Art students if I have that correct to Startup founders Yeah so Um the the origin story of Runway is Interesting that art schools are not Really the place where you start AI Companies not a lot of folks go to Art School to start an AI company and in Fact I went to Art School to escape Startups I used to being in the Bay Area And I wanted to take a break from Startups and technology and just going To art schools my way of doing that then It got sucked back in exactly Um so the way we started was so all Three of us met in our art school we Were in the same program and grad school At NYU and we were very fascinated at That point generative models were there But other much earlier stage in terms of The Fidelity of the results in terms of What you could do with them but we were Very interested in this idea of like Those models were really difficult to Use so just by making them easier and Give them the hands of filmmakers and Artists and creators just figuring out Like what they can do with them I Remember a very early project that we Made just before Runway was was a thing Or

Um just as a kind of uh school project Uh we there was there was an open source Model that Nvidia had released that was Trained on this academic Benchmark data Set for self-driving cars so it was a Generative model that could generate Essentially street views so we just Sudden the idea of just let's just make A simple drawing tool around this and Then just put it out for like at like Our classmates and like other artists to Use it and we saw even with such a Narrow domain and such narrow concept Like people created all kinds of Serial Imagery of uh like giant humans and Small and tiny cars like street signs That fall from the sky so all these Things that were like you wouldn't you Wouldn't expect that the generative Model for in set train for such a Specific task to be able to be used in Those creative ways and that principle Of The best way to understand the Capabilities of those models the best Way to understand the use case is to Give them to the hands of artists as Early as possible and kind of Incrementally build a product with them Has been kind of really key in how we're Building kind of Runway since then right For sure Um on the subject of you know starting Runway all those years ago like we

Probably have a lot of um you know Founders in the audience who would like To hear what was it like putting Together your first pitch deck um and And getting ready to kind of like Showcase what you'd build to investors Um you want from this prototype to some Kind of MVP I assume and you probably Had ideas of maybe maybe ideas of where You wanted to take the platform so what Did that first pitch deck look like Yeah so I think the important thing for Every Founding team is really understand what Their strengths and like what they're Really good at and like kind of build The narrative at least initially when You don't have like too much traction You have too much of a product uh to Really understand that and kind of Utilize that so for us coming from like The program we're in our school was had A very demo driven culture of just Really building the best way to explain An idea the best way to demonstrate an Idea is to actually build a prototype And just test it out and so that's what We did we just built like build a series Of tools and like showcase them to People and then uh and then just the Whole page like we had this pitch deck That we ended up never really using just Because we just showed like how the how The app worked and how people like all

The kinds of things you could create With this early version of Runway and That to prove to be a much better way of Explaining what it can do since we were You know like three International Arts Students not like not a track record of Being able to do these things so that Was the best way for us to show that we Could Yeah is that the approach you use to Raise 141 million dollars this summer as Well not quite a lot has changed since Then I would say Um one early realization we had was that Like for us it was very clear if we Looked at the trajectory of where Researching generative models was back In like when we started experimenting With that in 2016 uh like even even then You could see like how the Fidelity the Resolution the quality we're getting Better every year and so for us it was Very important to figure how do we not Just wait for the technology to develop But how do we kind of get involved in Building that technology and and so very Early in the history of the company we Start operating a research team to be Able to kind of work in generative image And video models [Music] For for some time there was this drift Between what was on the product and what Was in research just because the

Resource wasn't really production ready Uh and some something happened I would Say at some point uh like last year Where we saw this step function increase In terms of the quality of the Imagination and it just became very Clear that now this was becoming really Useful to people and so we started Building this new series of tools we Started building Um a new series of models this year we Placed a lot of emphasis on video Generation with models like gen 1 and Gen 2. and those really Um not just became a big part of what We're working on today but also it Really showed I think the world of the Potential alternative video it really Created this new kind of this almost Nuclear Community around those models That uh and I think grew like our Product in round went to like a new Level right Um now I'm sure the research is Expensive and a variety of respects Right you probably need a lot of compute You need to hire the right Talent is That part of the reason Um you you have raised so much money From from Big names like Google and Salesforce does it just require that Much Capital to to keep up with where The market is going or even you know get Ahead of your competition

Yeah so it's an extremely competitive Space and and unlike earlier like SAS Companies of last decade it's very much Like the balance between uh how much you Spend on infrastructure for so much you Spend to build a team is much different Than it was in the past uh so a big part Of uh this round was kind of uh bringing Bringing companies that have deep Expertise in AI to help us really like Grow our infrastructure and grow grow The ability to train larger and larger Models uh like Foundation models take a Lot of resources to train and and they Require a very specialized teams so it Was important for us to be able to Continue to be able to develop those Right yeah are these investors expecting To see returns anytime soon or you know Profitability or are they kind of Playing the long game with us The the focus for us is like what we've Seen with a model like Gen 2 is like It's been a year in the making and it Was at least six months of like a lot of Uh investment in research and uh in Training those most and until they were Arriving the product uh and the Investment paid off very very quickly Once we released those models so for us This is how we approach like we think We're still very early in terms of like What uh generally video models can Enable and what those models can enable

And so our focus is very much on uh Investing and building kind of the next Generation of models to be able to stay In kind of competitive right and you Have Um another initiative Beyond uh the Tools I guess in charging you know for You to use them right Um uh Runway studio Um is it appears to be from from what I Can tell you can correct me sort of a Consulting Wing business where you Partner with companies to to help them You know create yeah using generative AI Is that going to be an increasingly Bigger part of your business you think Going forward Um what was the impetus for creating That Yeah so I would say a lot of the impetus For creating Runway Studios was really Formalizing what we already had like we Had like for us it was really important To have Um creatives researchers scientists at The same kind of tables that is in the Office and like working together to Build those tools uh and so we we spent Uh we we built a creative team in-house That was uh was being used to test the Models to be able to make all the Announcements that you see from Runway And that uh kind of grew and grew and Became a big part of I think what makes

Runway different in terms of how we Develop those models so we decided to Kind of grow that further and build a Production agency around it and the goal Is less to support other companies for This for Runway Studios is more to Support creators and to figure out like For us developing those Technologies not Just like releasing new models is really Showing people how to use those models Like really supporting artists that were Using those most interesting ways so we Um like one of the initiatives Runway Studios was the AI Film Festival which We did at the beginning of the year and Then we're doing another one next year Like ways of showcasing because I think The Techno there's a misconception that Technology kind of Will Blow by itself I Think it really you need to work uh with People on co-develop it and really uh And have the way uh creatives use those Models inform their research and like The way you develop those models Definitely Um so you mentioned Gen 2 earlier which Is an improved version of a video Generation model that she released what Two years ago at this point maybe one And a half gen one Um January things are moving so fast Anyway the text to video is a Fascinating field I think like very

Young nascent Um players like Google have demoed Technologies they're working on in this Area Um I think you're probably the furthest Along in terms of like commercialized Product that I've seen Um So what improvements have you been Making to it lately and actually before That Um how did you train this model using What kinds of data how long did it take To train the thing I'm sure many in the Audience including myself are curious About what went into gen 2. Yeah totally so Gen 2 is as you Mentioned improvement over gen 1 Um so that has been in the making for Since at least a year I think the vision Of Texas video was from the beginning of The company actually Um the So we released Gen 2 back in March and It was really and it was a very gradual Rollout from initial kind of to a small Set of creators to a larger Discord Group and then to our product and the App and we've been making kind of Continuous improvements since then so One big like maybe one of the most the Biggest improvements was when we Released image to video two months ago And that allows you to kind of take any

Pre-existing image and then basically Predict and generate uh the remaining Component there's still a lot of Limitations to address like the control Ability of those models the Fidelity of The results there is I think we're still I guess very very early in terms of like All the use case for the those models But for us again it's it's really Important to kind of incrementally Develop and develop this technology so So kind of moving forward I think a big Folks for us is further uh improving the Controllability more so being able to Like right now the alignment of those Models is not as good as it could be so It might ignore some parts of the Problem or it might uh not be able to Really follow uh uh fall follow the kind Of the intention of the user so that's a Big Focus for us for further Improvement Um and and beyond that uh I would say Just further discovering new use cases For them that like I think a big a big Part of the development of the model Kind of was uh towards more uh for Storytelling purposes and to build a Larger narrative to build a story with Consistent characters uh it takes a lot More than to get individual shots right And that's that's going to be a big of a Focus moving forward right um having Used it um it is really impressive uh When the prompt is specific you know

Obviously building a generalized tool For text to video is I don't know if It's even possible but Um when when you mention what kind of Shot you want the style of the video and If like it's relatively short in Duration Certainly it's usable What I'm wondering about though a lot of Generative AI companies are taking Different approaches to training right Like some are licensing the data that They're using to train their models Others not so much because they argue That fair use protects them from Lawsuits in the US at least what is Runway's approach to this Um and uh you know no matter the Licensing Arrangements Arrangements you Have with contributors to the data set Um Can people can artists opt out of Training if they wish to when it comes To your models Yeah so that has been a like a big Conversation in the past year uh and We're kind of working closely with Artists to figure out like one of the Best uh the best approaches to address This Um so we're Um I would say there's a combination of Different efforts on Iran uh where we're Exploring various data Partnerships to Be able to further grow I think I think

Um for training those models like model Model Styles and compute is as important As the the size of the data set and so We're exploring a lot of dark data Partnerships and upfront to be able to Further grow and kind of further build The next generation of those models Um and so like we're I think like for us Being coming from a creative background Has been a really key to how we built This company and like and so we're kind Of like in terms of figuring out how to Um how to move this this technology Forward where artists feel that those Those products and tools work for them Is is there really is a really key piece Of like how how we're thinking about Further kind of growing the product Right so a related question is uh Copyright Um the status of AI generated media in Terms of copyright in the US is a little Up in the air at the moment I think it's Safe to say Um we're waiting on the copyright office To make some decisions around that Um but uh what what does Runway stance Then because you know I'm sure everyone Using your platform wants to know that They can copyright whatever is produced By those tools what would I would Whatever they use your tools to produce So will you stand behind creators Um if you know the copyright status of

Their media generated by Runway is Challenged in court for example Yeah so there was a I guess this provide A bit of context there was there was a Cardis card decision that was made that And it was in a very I think it's been Taking a bit um Uh I think it's been a bit misunderstood Where a work that's entirely generated By AI it applied to works that are Entirely generated by AI that are not Copyrightable that's not how you build Uh like work within Runway it's a Co-creation process so it's not as if Like uh the like our models generate Like content on their own uh they're Generating content with the intention of The artists and so that so so that core Decision doesn't apply uh so we're gonna As the regulation is an early space the Regulations evolving like every week so We're gonna adapt to whatever change the Regulation to make but artists should Feel confident using the platform that Uh that the content I create like we Stand behind that and and and it belongs To them like right right for sure Um and you know to your point big Studios are using your tools already so At least some of them feel confident it Seems Um so you mentioned ways in which you See the platform improving down the line Are ways in which you want to improve it

And you know focusing perhaps almost Entirely or largely on generative video Um do you want to get to the point where Somebody can type in I don't know a Sentence a paragraph and generate a Commercial maybe even like a television Show is that even technically feasible Um I mean given enough time I guess Anything is but I'm curious to know your Perspective on that Yeah so the like one of our North Northstar metrics is being able to Generate uh to our foam with Runway like A feature-like foam but the way we see That goal is less you're typing uh like Generating me a science fiction film and Basically generates to our foam like the Way people are using the platform today Is they they go through like hundreds And hands of iteration it's actually Actually something that we track how Many users uh Um generate at least a hundred Generations or at least a thousand Generations as a way to determine the Success of the tool and so like as People generate longer and longer videos As you generate Um uh and they generate Um like potential future length foam Using Runway it's not going to be once The process the iterative loop is a big Part of like the way you use Runway You're you're going the main the main

Goal of Runway is to really allow you to Explore ideas faster it's not to take Away your current intention and Creativity and vision and to actually Refine your ideas and a really big Important part of like making any work Because you really you don't fully know What you want to create until you start You start the process and you kind of Iterate with the tool yeah I was going To ask I mean with your ongoing writer Strike Um you know what what do you what you Just described sounds amazing from a Technical perspective but maybe like Professionals in Hollywood that might Have a different opinion or be concerned About like what this might mean for Their jobs what would you tell them do Would you say they have to reskill or Just kind of adapt to The Changing Times Yeah what would you tell them Yeah so I think again there is Um Um one one of the one one of the issues Right now with the with the discourse Around around this is there is a public Narrative around this and then there is Like what we're seeing every day and Like the way people are using those Tools Um and the public narrative is kind of Very much influenced by Science Fiction It's influenced by like what's the

Latest Terminator movie like AI wasn't Particularly portrayed uh positively on That uh but what we're seeing from Artists is that these tools actually Allow me to multiply the creativity uh Of course it's like some things will Change I'm not saying the creative Workflow will remain exactly as it is uh Today like in the future as those models Further evolve uh like when uh in age of Like silent films you had an orchestra Uh inside the cinema and you no longer Have the orchestra those those uh those Those jobs have evolved to something Else but it's not a zero-sum game it's Not something that like we uh It's usually like we when they is kind Of leaps in technology happen that Changed like that inventing your medium That invent and you are form uh we see It's actually Um like the the amount of uh the amount Of jobs the amount of creative output Actually increases in the world and so We I don't see any reason why it would Be different with generative models Gotcha fair enough Um so I did want to wrap up with uh with This question Um as you're well aware Reception to AI generated media hasn't Always been extremely positive depending On the context right Um I the recent Netflix controversy

Comes to mind with like AI generated Generated backgrounds being used in an Anime series right and people seem to be Up in arms about that so how do you Think How do you hope that opinions will Change uh going forward and do you think You'll encounter or your customers will Encounter any resistance for using tools That the tools that you're building and Producing Yeah again I really empathize with those Those concerns I would say the a lot of What we try to do on our end is really Um By building like useful tools around Those models kind of demystify them like I think a lot of the Um a lot of the concerns actually come From the again the public narrative Around around Ai and around the Misconception that Ai and kind of like Performs the whole task for you and what We see from creators is it's still like A very iterative it's still a very time Consuming in some ways process but what Those tools allow you to rather than Rather than replace the creative Workflow it actually allows you to Explore ideas faster to like to to tell More stories to do to explore ideas that Um you didn't have kind of resources to Explore in the past and so as those Tools become more integrated like part

Not just of our product but of many Different products like people will kind Of I think naturally start to use them More and and they're going to face the Limitations of them and they're going to Face what they're good and what they're Not and so they're going to see them Increasingly as tools rather than like Uh individual agents or models that kind Of have their own intention and like and And I think so I think it's just Naturally going to happen as people uh As people kind of become more habituated Through this technology and like we Become as part of a larger set of tools Right fair enough and on a more Optimistic note for me um as someone who Has no uh camera skills whatsoever or Editing for that matter Um using Gen 2 has been Interesting very interesting and I look Forward to seeing where it goes from Here well um thanks again for joining me On stage anastasis and thank you I'm Sure we're all eager to see where Runway Goes from here thanks everyone Foreign [Music]