AI Can Write Words — But Can It Understand Them? | TechCrunch Disrupt 2023

Today we have May khabib from Ryder as Well as ofier Kowski from Deep Pub along With our moderator haa Jean camps please Welcome them all to the [Applause] [Music] Stage hello everybody thank you for Joining us so um Karin says she hasn't Seen whole articles written by chat GPT Yet but she definitely edits my stuff so We'll see how that goes I'm just kidding So I actually every couple of weeks try To get chat gbt to write articles for me And every time I there's just no heart And no passion and um I fear that that's About to change Mary for how long will I Have a job May um sorry I think um You'll have a job for a long time it's Like just absolutely not in question um We uh last month worked with Kyle shika At the New Yorker and fine-tuned one of Our writer models on you know yes a Bunch of stuff they had written at the New Yorker but maybe like 50,000 words Of his like Corpus and he did find it Like super Eerie uh but you know is it Going to write fullon New Yorker Articles no uh we did have like some fun Experiment where we would take Tech Crunch articles and rewrite them as the New Yorker and they would start off with These like rambling Anecdotes uh so yeah you can have a lot Of fun with it it'll help you but uh

You're you're good I think you're safe I'm very glad to hear that so what are AIS particularly bad at right now it's Bad at knowing things that only you know MH you know um and that you know may Sound like tological but um the llms are Predictive machines like I think we all In the room already know that uh AI can Write words but can't understand them so Uh we've got a lot of exciting Techniques though and a whole ecosystem Of tooling um to solve what I'm calling The last mile problem uh which is Actually 80% of the work but llms are a Great utility to get you started on a Lot of these fun Generation Challenges But you know alone aren't super good at Human quality reasoning or generation Yeah There is an interesting thing that's Happening Because essentially the llms are Statistical models we're talking about That um backstage for a moment and there Are things you can say in one language That you absolutely cannot say in Another because it's just the wrong Thing to say how do you get AI to deal With things like cultural differences or Language differences so actually this is A a great way to to emphasize the the Limitation of AI For example everybody in the audience Would understand the phrase hold your

Horses but a straight translation into Any language would be wrong right um so What we're training the machine is to Understand some of the idioms but still In the process of translating into Different languages we use human humans To curate the end results for example uh Another example is a joke you know a Joke is a very cultural thing and when You take a joke and you trans straight Translation of it it would be wrong it Won't work and currently translating Jokes with AI is a difficult to Impossible uh uh task for AI currently Yeah but air you could take um a model And say replace every joke that is Specific to a culture with like an Anecdote that is like Generic use the models to solve those Yes definitely but the kind of work that We're doing is a very high quality work For example we're doing work for the big Studios so in a movie you don't want to Yeah I know you don't want it to be an Anecdote so obviously AR are very Accurate and very good on the average Side of things but when you want higher Quality for example writing an article a Very cool article that everybody would Love to read then you would need a Person to curate it so it's a great tool As you said yeah what's interesting I Mean uh English is my third language and Uh your idiomatic example was

Interesting in Dutch if something is Very surprising you might say now comes The monkey out of the sleeve which you Know is funny to us because it makes Absolutely no sense but it stops being Funny when you realize that really gets In the way of of communicating um I'm Curious with your tools that are trained For different use cases could you say More about um how that actually where The rubber hits the road like what is That um who is using it for what and why Yeah great question so we are a full Stack generative AI platform so what That means is we combine a large Language model ours with Knowledge Graph So think of it as like rag type of Solutions and AI guard rails together so Um uh let's say you are a wealth Management company and you've got wealth Managers who spend a lot of their week Putting together like proposals for Individuals and they're looking at you Know a Charles Schwab statement and a Bunch of like kyc documents and a bunch Of intake forms it's actually a Wonderful use case for being able to Take structured and unstructured data Together um to put together you know two To three pages of of Pros there is a lot Of Last Mile work to actually go from Yay I have an llm in my own environment And I've got an API to it to I've got Something that 2,000 wealth managers can

Now use and so that's what we call a use Case and breaking down that use case Into what's going to be required what Tooling is required to get the accuracy That you need what am I going to um ask The user for in creating this new Workflow because as ofier said you have To have the expert really driving that Workflow you're not going to press a Button and you suddenly have this Proposal that you're going to send to a Client M so the use case based approach Really is is very practical and rooted In reality and then all of the data work Isn't some kind of theoretical big thing It's let's get this number of examples Of proposals that look like this and Bank statements that look like that and Kyc documents that are you know like Handwritten versus typed and so you you Break down the problem um and can Actually ship something that is that is Use useful mhm and um you had some news Recently uh y'all raised a god aful Amount of money at a half billion dollar Valuation thanks we didn't disclose the Valuation but we we did raise $100 Million yeah uh what's next yeah Right thank you uh so what's next is We're growing every team uh so if you or Folks you know would like to work at a Super fun like real um uh use case based Generative AI Company please hit me up I'm May at writer uh and what's next is

Uh multimodal what's next is Agents What's next is multi-lingual um but you Know the the multilingual approach for Me is like all about because we used to Run a localization company is really all About um generating a native language Versus translating which I think has to Be a human approach for it to really get To the Quality that's required for our Use cases but yeah some some fun stuff Coming down the pike awesome thanks ha Um multilingual multilingual is kind of Your bread and butter um can you say a Little bit about what you do it gives a Little bit of context yeah so what we Actually developed is a platform that Enables uh to localize content Simultaneously into more than 65 Language while preserving the original Experience this allows people even in The US to enjoy content International Content M and so that is both subtitling And dubbing right yes we're doing both Uh uh kind of work for and we're working With the big studios Distributors and Enterprises around the globe MH so can You give some examples of who you're Working with Um for example we worked with the stream Platform probably you don't know them But topic.com because they're Specializing in darkar kind of drama TV Series there are only specialized in Drama series that are coming from

International source sources that were Not created and they are bridging this Uh uh uh language barrier because if you Watch a Danish kind of a Dr Noir it's Very hard to for an English audience to Watch this kind of content so with with Our solution they can do it very Efficiently and uh in a very high Quality MH and so the thing that comes Out of your platform do you consider That like okay this is done now or is it An early draft of a translation how how Good is it right now so actually taking A content from one language into Multiple languages requires uh different Uh uh uh um stages first you have to Transcribe it so the model needs to Transcribe and to understand the Original language and he should do it Very well secondly he has to translate It which is most cases a straight Translation and then there goes another Part which is adaptation adaptation is Adapting the translation to the local Culture we talked about jokes and idioms And this is the third part and the last Part is creating the voices and creating The voices also have to relate first to The original because you want it to be a Simless experience but also to the Culture that you are referring the Target language that you are referring To so we're doing all all of these Processes this is a very convoluted

Process and a very convoluted workflow But the use case is you know as simple And is well understood what is for Actually we're enabling people to Overcome the language barrier Fe Actually free people from this Barrier yeah no I love that Um I was playing around with writer and Kind of uh trying to figure out how it All works and I noticed that you have Specifically something that you called Ai guardrails and and I thought that was A really interesting Concept in terms of Keeping it on the right track can you Say a little bit more about like why did You decide to develop that and how has That evolved over time yeah the the the Last smile means that the the the Content or the answer really has to um Uh be correct and and fit into workflows Where you know where we now don't have Hundreds of people checking this process So the accuracy really has to be high And what we found is for you know Certain kinds of brand and editorial Guidelines um if you are United Healthcare and they're a big writer Customer and you've got a right at a Fourth grade reading level because of State guidelines for Medicaid content And uh you put it into a prompt you are Going to fail those State reviews so the NLP postprocessing is is really Important and they'll be you know we'll

Use smaller helper for llms to do a lot Of those microservices but there are Hundreds of different AI guard rails Under the writer umbrella for everything From tone and character count to um uh Grade level and Clarity uh and then You've got accuracy and compliance and There's a whole um uh host of categories Where it's just not good enough um and Not accurate enough 49es of uh uh uh Consistency to be able to you know put It into the prompt or you know uh have It run through the llm only yeah and I I Imagine one of the things that comes you Mentioned data accuracy right I asked a Um uh a chat GPT 4 I think to write me a Bio and the bio was incredibly accurate And really sure about absolutely Everything and said I had a different Birthday and so it's interesting right Because most of it was absolutely 100% Correct and so I could easily be led to Assume that everything was correct and I Imagine if you're doing stuff for Healthcare Like does it flag hey I'm not completely Sure about this piece of data or is There a way to know whether or not it's Hallucinating yeah we call it claim Detection so anything that is a Statistic or a fact or we can't find it In you know something you put into the Context window or the the knowledge base That you are associating with uh that

Use case we will we will highlight in Purple for the end user and they got to Go through and like click that they have Citation needed yeah yeah and there's an API for that too so if you've integrated The use case into some workflow you can Actually call the claim detection API so You know the folks who are most advanced Um in in the Enterprise are thinking About you know what kind of AI literacy And AI education do we do for our Workforce because you know if you are That person who is writing uh if you're The wealth manager who is is using AI Now to write a proposal um you've you Still got to read every line of it Because you're going to get on the phone And present that to your your client or They're going to come into your office And talk about it and so the um to it Has this really are customer too they do This fun like 10 slide um watch outs Like before anybody can use writer and They also have an internal tool um you Got to like go through this test right And it's very simple but uh it is a real Important step in someone's adoption Journey to say you know like here are The things that you really do need to Watch out for or things that we need to Make sure folks are educated on what the Models can do and what they possibly Could not know so you've got to check Yeah well and this is Beyond like

Writing as Hemingway or making a pretty Picture you're literally talking about Wealth managers so this is millions if Not billions of dollars and people's Healthcare thing so it absolutely has to Be correct is there a way to report it's Like hey this isn't right or where did This come from kind of thing so um there Is a a whole body of research um around You know explainability in um in Generative Ai and there's some really Promising research including things that We are doing around like literally going To um the weights of where something may Have may have come from um but the you Know the most important thing is really To show somebody the source so where in Your knowledge graph or in the content That's been you know associated with the Use case did something come from uh and In that case really help develop the Muscle memory of like tracing back an Answer to the kind of the derivative Pieces of content or kind of Foundational uh type knowledge that um Came in from um the uh the database yeah I was working with one startup Who had Who tried to train their um uh language Models on like a whole knowledge base Which is wonderful except the knowledge Base included three generations of the Software right and so you end up Reporting like yeah you can definitely Do that by clicking this button which

Was removed six years ago kind of thing Um antology is a big problem yeah yeah No I imagine and then you end up with Very interesting data hierarchies to try And even figure out what that means Means for sure so as a journalist I use A bunch of different tools to transcribe My um interviews the prep calls and that Kind of thing and it is astonishing how Bad it is if anybody speaks in any Different way like in our interview you Speak with a slight accent the the it Was gibberish if I interview somebody Who is Scottish may as well forget about It like why are AI still so terrible at This I believe that AI is currently not As good as we want it to be because it Is trained on the average kind of data It's a statistical model so so it works On the average data so if I have an Accent this is not an average on when we Speak English most of the English Speakers speak with a fluent accent so This is why those models are are failing Actually what we do is we fine tune the Models on accents or different languages This is why they become multilingual and Then they can understand better accents So what does that actually look like Training like do you feed it a bunch of Audio with that has already been Transcribed or like how do you train it Maybe I'll give a great example that you Probably uh uh won't understand about

Llms but for example when we're using Our technology the the the pre-trained Multilingual model uh and we fit it with With a sentence for example that that Has the word Chanel Chanel is the the Famous you know fashion house the llm You know just say it as Channel because In English it reades as Channel this is You know this is the obvious average What an English person English speaking Person would say but if you want to find Unit what you have to do is to find unit On French language or French words and Then he can understand that there might Be French language out there yeah well And the interesting thing in in a lot of Languages you intentionally mispronounce Things like like uh like uh names for Example the Norwegian name Kel nobody Gets right so it is pronounced Kel but In English that would then be correct so Now you have to train the language both The way it's meant to be pronounced and The way you're meant to mispronounce it Like that just breaks my brain and we do That as humans we do that all the time Yes uh what we use in our technology is Building leries it's not AI it's like All technology but this to solve this Problem because current llms cannot cope With this kind you know even a joke take A joke you know it's just taking Something which is sad and flipping Around this is why it makes us laugh

Because it's like the flip side of it It's just making things nonsense uh but When you're in your case what you just Said what we build is a glossery that The AI is complimentary to the model and Then the model can you know just Understand this is a word that I can Take it just as in the glossery not do Anything it's like trying to learn German where you go here's the rule and Here's the 5,000 exceptions yeah totally Um what is currently hard for um for Actually building the models and Building the exceptions that live in in Not like a knowledge base or a Generation tool like Writer I think some of it is what ofir Said you know um sometimes we'll come in And uh a customer hasn't been able to Get a solution Beyond like a 50 or 60% Accuracy rate and they're just like Trying too hard and you know like that Example of using the glossery instead of Like trying to get the llm do it to do It is is a is a decent example um and There's research now that um shows that A lot of the smaller models are actually Higher performing on you know certain Language tasks than than bigger models And some times like switching to um a uh A combination of tasks that are Associated with uh different models that Are doing you know specific things Actually produces a better output so I

Think you know the the general challenge Is um a a lack of familiarity of both The llm and the NLP kind of techniques That are required to get some of the the Results and you know sometimes an Overbuilding or an over Reliance on Prompt engineering yeah well I imagine Also if you if you limit the the sets You actually have much higher chances of Success like if it's a drive-thru you Know you don't you're not expecting them To understand everything you can't ask Every question to a drive through Attendance it's mostly about do you want Fries or not and that's like having Those options makes it a lot easier than Saying hey what's the what's the capital Of Zambia they're like look I don't know I'm just here to sell you fast food kind Of thing right so how do you limit the This the scope of the the piece that They do need to know about or is it Actually much easier than it sounds so It's a good question the the use case Based approach has us asking folks a lot Of the time like you know building a Generic um you know company X GPT uh is is fun to get everybody um uh Into the technology and you know their Their their finger their fingernails Dirty with um uh with how stuff works But in general are probably too generic To do anything so excellently well that You know it it replaces whole workflows

Um and so we think the answer is is Really the ability to build multiple Digital Assistance or Q&A at scale uh And so you know if you are building a um A sales enablement uh bot that is Actually associated with a different Knowledge Graph than uh a bot that helps Folks understand uh the policies of HR Or um kind of the internal company um uh And so really thinking about what do you Actually the jobs to be done what do you Actually want the thing to do uh and Then working backwards and a lot of Times you know we're actually if the Answers are wrong um we're taking a look At the data and we're maybe using an llm A small one to transform the data right And then trying again with the same Prompt the same model and it works so Much better so you know the taking a Real 360 approach is um is is fun and And we transfer that to our customers we Want them to learn alongside us as well Mhm what is the what is the what is the Bleeding edge for you right now like What is the thing that is like the Furthest reach into uh AI generated Tech Yeah I think it's probably what we're Calling level three question and Answering so um you guys might have seen The CNBC headline yesterday that Morgan Stanley built you know a a chat GPT like Thing to query like 100,000 research Documents except the catch is

Everybody's got to write in perfect English full sentences or the model gets Confused which is hilarious like who Does that uh and so you know I do think Kind of bleeding edge is sort of level Three like you know can I actually talk To the model um and ask a question that Is really like multi-concept that Combines you know a couple asks in one And uses like contextual knowledge that Is really um uh specific to the company Right referring to databases or products Or concepts that are really company Specific and not that General that That's bleeding edge for us yeah I love That I think it's so interesting because A lot of the time the the AIS and stuff That we as consumers are um get to touch Like Siri and like some of the Google Assistant stuff they're surprisingly not Very smart I mean you can't even be like You're in San Francisco what's the Weather in London and it gets it right Like I yeah I mean who knows now Apparently um Amazon Alexa is is llm Powered I haven't been home to try it Yet so right this is the problem of Fundraising and coming here you know Yeah what's the what's the Cutting Edge For you like CU I can imagine there's a Whole bunch of really interesting things With speech generation and translation And yes so we talked about translation Translation is still something that

We're working on and in terms of it's Not only getting it right but also Understanding the emotion understanding The emphasis the right emphasis where You need to put the emphasis in Different places but I think that the Future holds doing it in real time mhm Can you do a sport event or News this is something that currently is Not available due to the Restriction of The Translation mostly yeah I mean I guess Simultaneous interpretation is one thing But not to multiple languages and not AI Powered yes yes and and also enabling You know humans to be more effective This is something that you Know current AI models are what what They are doing doing is that are just a Tool they're enabling people to be more Productive more effective and more Creative in a way you know you can use The model just to get the ideation part Faster so even us we're working with our Technology to enable people to have a More creative approach content very cool And how does that work with so I I have Worked with text to speech and most of The time you know you get a little bit Of Melody a little bit of color in the Language that feels very good but in in If you're doing acting for example Sometimes they're sad sometimes they're Angry like there's emotion behind it and

A director can say do that again but you Know try that again with emotion can you Do that with the AI voices now yes so This is this is a good question that you Ask because this is a very difficult Task for for current text to speech Models what what we have developed is a Way that you can direct the machine to To say something in a different way in a Different emotion you can do it by Shouting by singing by Whispering it's Pretty cool the technology to do it and But it's still to a certain limit right Because emotion is varied from 100 to From zero to 100 there's a lot of gray Areas right still the models can do it In a very uh limited way right it could Go really good actors are able to Pretend to be angry and as a as a viewer You're like oh he's pretending to be a Angry right and now you're into like Multiple layers of different types of Expressions of voice right this is a Good notion because it's it's built on Different or a multimodality problem Because it's built in the language that You you use but also on the intonation Of how you use it so so it's not only The voice it's also the language the the Translation that you you spoke and how You spoke it yeah and is that something That like how how would you do that do You have to write a prompt do you Highlight it and bold it or something

Like how does that work in your tool so Actually what the machine is doing is Looking at the original trying to Imitate the way the original was it's Just preserving the original experience But obviously it not 100% accurate as Most of the llms and most of the the the AI models that you have out out there so We have language expert within the Region that go over the the uh the end Result and then fix in the places where It needs to be fixed and the the those Fixes are fed into the machine so it Learns for the next time MH one of the uh narratives that often Comes up around AI is they took our jobs Um could you both speak to that like to What degree is what you're doing uh Reducing the amount of jobs that are Available Yeah uh you Know honestly like I really can say this With a straight face we create jobs Because there's so much to do to Actually like put these use cases into Production that a lot of our customers Can't fill those jobs fast enough um AI Program directors and managers people to Really think about data people who are Developing use cases people who are Training folks on how to you know trans Form these these workflows and you know Some of that is people companies have Budget now to experiment with AI it's

Like seemingly the only thing there's Extra budget for is figuring out Generative AI but they're not actually Putting that towards headcount um to to Make the the gains real and so I I do Think there's going to be a bit of a Reckoning um with that in in in a few Quarters where you know CFOs are going To be like you know show me what you did With a few million bucks um but in terms Of like actually you know we work with a Lot of CMOS cdos Chief digital officers Like the jobs are certainly transforming How we do the jobs is transforming but You know nobody is cutting heads who Really believes in content or digital or Brand um I think that's different in SMB World um I I do think it's different uh But you know in in our space I feel like You know we've got um uh we've got a Long way ways before there aren't going To need to be people at the helm yeah There's currently an actual writer Strike on right you in this world where People are going to be nervous what's Your take on this um as of today nobody Lost his job because of what we do Actually most of the our customers are Looking to monetize on content that was E not economically viable to monetize on So actually we are enabling them to do More work and as I explained we use the Same professional from within the Industry to curate the result of the

Machine so actually we're bringing more Job to the same people right so so Current phase actually there is more Work to be done than by those Professional than was before all right Well let's leave it at that I guess AI Creates Jobs thank you so much for joining me on Stage and thank you all for coming today Have a great show thank You Oh