Facilitating Human-Robot Interaction | TC Sessions: Robotics + AI 2020

Foreign [Music] Good afternoon everyone all right let's Talk let's talk robots interacting with Humans So I've actually been re-watching lost In space on Netflix Clara I know you Actually haven't seen the show so this Referenced my fall little flag but you Have and so essentially it's this robot That befriends a child at least in Season one I haven't seen season two yet But you know there's like emotions Involved and they're getting along and They're protecting each other how close Are we to that sort of world Maybe let's start with you Mike since You've actually seen what I'm talking About as close as we are to Alpha Centauri um we're really far away Um I think there's elements of that that Were there's discrete elements that we Can do but that whole picture of what That robot's doing and then and and the Physical part way way beyond our means The emotional part Um I think you are seeing more there There's more because you can do that Through gesture and interaction and There's some interesting startups doing Things in that space But I I think and That's why people are going for more the Emotional side so far rather than the Functional side

Yeah I mean I uh both of my children Were good friends with robots when they Were little Um my son and his Roomba liked to play Together a lot and they're toddlers my Daughter had a uh a agricultural robot But she asked me when I left that Company if when that robot got too old To work could we please keep him Um and we did and we lived in my life in My kitchen for a year or so Um but I think that that is you know for Fun as you were saying that the level of Uh understanding and ability and and you Know Consciousness I guess that sounds Like what you're describing Um yeah we're a long way from from Anything like that in the real world Today okay that is unfortunate but good To know well you haven't seen season two Yet you'll have to keep making friends With humans I'm really sorry okay and so Claire what what you do is you work on Technology for for robotic arms inside Factories yeah so what Veo does is we Enable flexible and efficient Manufacturing by making machines Responsive to humans so our first Product gives industrial robot arms eyes And brains so that they can interact Safely and effectively with people with Production workers in a manufacturing Context And so how do you take into account the

Fact that these are very large robots Interacting with humans like what what Type of safety precautions do you take Into account yeah so Um one of the main reasons you want to Use robots in manufacturing is for their Superhuman capabilities you want a Machine that can pick up a 200 pound Object it can move it at two meters per Second and set it down with sub Millimeter precision and the capab the You know the physics of the mechanical Capabilities behind that means that you Have a very dangerous robot and Typically in factories today these Robots are are kept in cages and we Always make it clear that it's not to Keep the robots in it's to keep people From getting close to the robots and Getting hurt Um so what we're building is a sensing And computer vision system that is Designed to meet the requirements of a Functional safety system the regulatory Requirements that are required for any Um any Industrial Automation system That's protecting people from from Dangerous machinery and that means Everything from designing custom sensors And custom computers with dual redundant Hardware to designing all of our all of Our algorithms from the bottom up so That they are they have well-defined Failure modes that we can detect and

Respond to so it's basically underlies Everything we do And what are the types of tasks that That robots should do on their own in Factories versus the tasks that they Should be working kind of side by side With humans we'll We'll so today because you have to have Basically robots in cages a Manufacturing engineer who's designing a Process step has to decide is this going To be a fully automated step or is this Going to be a fully manual step and if It's going to be fully automated that Means that there's a lot of extraneous Automations who essentially get parts in And out of those cages do a lot of sort Of auxiliary tasks if it's going to be Fully manual that means that it's Typically going to be something that's Hard for a person to do that may be have Ergonomic challenges require lift assist Devices a lot of sort of challenges and Often performance problems so an example Of something we'd like a human and a Robot to be able to work together on is For example a robot can lift an electric Motor weighing hundreds of pounds and Position it precisely so that a person Can come and attach Fasteners attach a Wiring harness to tasks requiring Dexterity and judgments and flexibility Nice and so and Mike so what what you're Doing is definitely in the same realm

But a little different so you're working On robots inside the home uh geared Toward people who need assistance yes Yeah and ironically we have similar Issues because our what we're trying to Do is take off the shelf components so Instead of having to do our own depth Cameras and all the sensors to get them At that factory grade level we're taking This the range of new sensors are coming Out from AR headsets step cameras uh Gyros all these things that pretty much Any part in our robot you can get Between stripping it out of something From Bed Bath Beyond or Best Buy you Know and that gives us because those Things are made in the billions of units And so we can close together a sort of a Proximity of what what Claire is doing But in a not as functional level but Also with with lighter capabilities but We're working with robots that help Carry help carry things or deliver Things for people who have some Impairment of Mobility themselves but You have all the same requirements Around safety that Claire is talking About just at a different scale and so Our you know one of the common themes That we've come up against is people's Expectations about Rosie the robot or C-3PO in their house is going to be are Just way way too high way too high Compared to the reality of what we're

Trying to engineer and make functional And work with someone yeah and I know That you're you're not quite ready to to Show what to show what your robot looks Like but but yeah just like you Mentioned oftentimes when we think of Robots in the homes they're kind of they Look like Androids with like these like Human elements to them and can you talk A little bit about how you decided on What your in-home robot will look like Yeah I can show some slides what we said What it wasn't going to look like so um So I think if I go to yeah Pepper but if you look at that there was A lot of hype around social robots and Again there's some really strong Examples there's intuition robotics There's a stealth Company by Paolo Pajani and that's doing some really Interesting things on things that meet Human needs but with pepper there is There was a whole range we had sort of This batch of robots dressed up to look Like people be able to gesture and Interact but what you what you look at That is the arms really aren't Functional they can move but they can't Pick up anything and what they get Relegated to is you know now they're Sort of retail greeters you know the Robot waves it tries to engage somebody With big eyes and then it points at the Screen to do that and we've had this

Generation of robots with screens on Wheels and what we're trying to do is Move beyond that to sort of what can we Do that's functional and so when we look At stuff this goes back to early Research with PR2 from a little garage And what and Willow garage was a great Company most of us wouldn't be at this Conference without Ross and everything That came out of there but that example This is like that robot right there with With doing pancakes that can't be done Safely and people go why not or I can See it being you know the motors and I'm Sure Clara could probably list 25 Reasons that's wrong there but I'll give You one that nobody thinks about is There's pancake batter on the back Counter the robot go turns on The Griddle and and goes to to pour the Pancake batter and if it's a way and a Five-year-old comes over and wants to Grab the pancake that's on the bat on The griddle there's no physical way that Massive robot can move that distance the Way a mom or dad could grab the child And protect them So we really anything you do on a robot You automate you own and we have to Think about all those consequences so The robots become more practical as we Look at this again this was interesting Research they did with seniors out of Georgia Tech and this is just delivering

A pill bottle and it was funny because It's it's the seniors wanted that they Want that use case but that massive Robot trying to navigate around just not Even figure out what its cost is is just A lot of of taking up space and everyone It wasn't even just PR2 it would be Everybody that I watched any University Do this research there was always this Weird grab like this person with back Problems is having to reach to grab the Robot and robot's not really sure when To let it go so we're really trying to Work at this more natural flow of how do You bring items Within Reach and it's It's and things but what you see out of All this is these large robots they over Top you know from an interaction point Of view they're overpowering people They're they're larger than them and so Our thing was okay let's let's take that All apart let's how do we focus on the Function make something that's a little Smaller where the person feels where the Robot is really subordinate to what the Person is doing and doing that I'll just Give you another example the reason I Throw these out is that when I go and Show people like our product and I go And Pitch to a senior living community They go oh I saw um care about four this Video is like well this is great we want That and that's a concept of it that's Carol about I think it has like 33

Degrees of freedom Um and or 31 something in that range and Is able to pick something off of a shelf And that's not reality I think Clara can Tell you like 31 degrees of freedom when You're making a robot is 31 things that Go wrong exponentially when you're Manufacturing so and now what it's done Is if you look up you know care about Now it's a retail greeter they took the Arms off they don't have a bend and it's In the store again so what we're ours is Doing something different but I'll just Give you an example I think someone that Troy is going to be talking later today You look at what they're doing with tri And they have a much more practical Example robots lower smaller Mass Circular and if you look at what it's Bringing over to someone it's bringing a Basket that means that risk that grasp Grasper can go and grab the basket hold That and let somebody take it out and All those things are really key and These are those types of interactions That you were trying to figure out and Do within that that envelope so I'm curious I mean how do You immense To training the robot how to interact With the human but like in terms of Actually training the human like how how Do you train the human to know exactly How to interact with the robot and

Really to trust that robot One of the things that I think is very Important in design of an automated System that's going to be interacting With a human is that you make the Automated system predictable and you Basically expose its mental model mental It's internal model of of what it Understands and what it doesn't Understand and I think that's you were Talking about the sort of Anthropomorphic robots and I agree that That's very deceptive because what you Need when you're interacting with a System is to have an again an accurate Model of what it knows and what it Doesn't Um and so that's something that when we Have these systems that um basically Control an industrial robot arm that There's a couple of pieces that we Include with that one is that we make The the way in which the robot arm will React to you will slow and stop as you Approach it needs to be very predictable So as a human worker interacting with That robot you know what it's going to Do based on what you do because then Humans are really good at figuring those Things out and optimizing around around Them the other thing we do is we provide A visualization a screen that actually Shows the robot's internal State key Pieces of that state in a way that's

Easy for a person to understand when You're looking at them so I think being True to the function of the robot and And accurately conveying what it knows What it doesn't know and how it's making Its decisions is actually the best way To create a functional interaction yeah Definitely and I think with them we're Dealing with people especially they're Untrained is just work within their Natural flow the metaphors that they Already have our robot is actually more Furniture-like than it is robotic and We've done we keep on reinforcing that In the industrial design too because you See something and okay there's water on A Surface I can pick it up and you don't Even have to explain you don't have to Get into okay wave your hand and tell You know tell me when it's okay to let Go of the water from the robot side just Present something in the right way so a Lot of it is trying to understand the User flow what's where is their mind and Their behavior and their Instinct Already going and then how do we adapt The robot behavior in the system to that Yeah I think there was is a Walmart they Had their little like aisle aisle bot And they found that a few like put a Walmart apron over it that like people Treated the robot a little bit better Um yeah I guess it had more human like Qualities like I'm not exactly sure what

They were doing before with it if they Were like kicking it or something but Um but yeah but but it sounds like that At least for like in the home and in the Factory like it's not it doesn't Necessarily need to look human in order For humans to feel okay interacting with It I think it raises the expectations so High when it's anthropomorphic it's it's What people want I you know when we did The original Lego Mindstorms we were Struggling with what the people wanted To like what would be the robot on the Front of the box and we eventually made We used a part of a Lego submarine with Pneumatics to make something that would Grab a can and that was to give this This chunk of plastic this ability to Sort of look anthroponographic and People could relate to it but at least In the Lego system you could build that And people knew and had an expectation Of it but when you make things look like People people think they're going to Work like people and and it's not that Does and then it's then it gets becomes Sort of devolves from there And then for for your in-home robot Mike Um is it are you envisioning it being Like voice command or would the person Kind of try to control the robot with Their with a smartphone or smart device Or what would how would be the primary Communication generally you command it

So one thing is you don't you could take Remote control but we don't want people To do that because that puts a burden of Usage on it it's a most somebody might Point to something or say I wanted to go Here a lot of its voice Um you know so it's one of the reasons You know we're funded by the Alexa fund Not just because we like Alexa but That's compared to 10 years ago now we Have this capability to give Complex Directions of go here get that and come Back and and so the voice is one just Simple buttons for some people may have Hearing problems or just prefer that I'm Waking up in the morning I just want the Robot here and I don't want to get into A discussion about it so I think the Idea is again follow the flow but There's just buttons there's I would say Voices first just radio buttons and then An app if you really need it but you you Don't put the burden of them one thing We realized was the amount of Calculations the robot's doing to move Itself in a space is actually pretty Amazing so we can't keep even when we Try to do it we can't joystick the robot As well as it drives itself now so we Try to sort of smooth that out and put Put abstract ways that people can Command it yeah and this uh yeah this is Definitely geared toward toward the both Of you but I guess yeah start with with

You Clara what what exactly are you Training the robot to see inside the the Factory so we aren't training anybody to Do anything Um that's actually one of the Um because of the safety concerns that We have you need to use as I said Algorithms with deterministic failure Modes and machine learning does not Qualify so our system is all built on Deterministic 3D modeling so we are Looking we're analyzing a 3D space Figuring out what our sensors can see What they can't see we know because it's An industrial work cell we know it's Supposed to be in there so when you're Designing a work cell and you're Building the machinery and you're Putting all the different pieces Together the robots and the conveyors And the fixed automation you actually Configure our system to know those Things so anything that is of sufficient Size and shape that it could be a person We're going to treat as if it is a Person Um and the design of the application and What that system does and like what Inputs and outputs it actually comes From is is built by a systems integrator Really thinking of they will call the Robot arms that you see and you know YouTube videos of of car factories we Call those robots but really the robotic

System in a manufacturing application is An entire work cell so our sensors for Example don't go on the robot for the Same reason you don't have your eyes on Your hands the sensors are monitoring The space and and the training is Essentially done during the development Of that work cell and then what we are Providing is is a intelligence basically A spatial awareness knowing what's going On in that space and where it is and is Not safe for the robot to go got it yeah And then and then Mike how does that Work work for you and in terms of what Your robot can we'd love to have Clearance sensors but we can't afford Them and the other thing we can't do is We can't put them in sort of God mode Around the environment that we're stuck With the robot for the most part needing The sensors on board and that creates This visual perception images like Things we really care about that would Advance robotics is how wide the field Of view on something like Intel Realsense becomes the wider that is man We were like you know like we have less You know blind spots and unless of all These edge cases that we're trying to do So we sort of move this space around us Like a force field and then people are Coming in and out of that and we're Interacting with pets and the babies on The floor all sorts of Wonder wonderful

Scenarios that scare me at night um Thinking about that could fail Um but the the important part is is that 10 years ago we didn't have these Sensors now we do and they're they're Rapidly coming down in price and they're Getting better and better in resolution So we we basically have this virtual Radar field around us that that really Is going to enable a new generation of Products whether it's from us someone Else there will be robots in the home in A much bigger way now Oh God was just going to Echo advances In sensing it have been huge and so we Like we do build our own sensors but we Actually use uh the chipset is not you Know we're not quite crazy enough to Build our own silicons Um so yeah the chipsets are are off the Shelf chip sets that we then build into A camera with redundant hardware and Fpga processing and other pieces but the The fundamental availability of that Kind of technology is just huge for what We can do with robots today yeah and one Thing we're talking about like sort of Stacking these things so like software And processing power is doing you know Is going on that you know the the curve And then sensors are doing are close Behind it but Claire you were talking About Motors and it was like she Understands you know what I was talking

About if you can repeat what we're Talking about with the way Motors aren't Quite at that same if you graph you know The motor power per dollar availability Over the past hundred years Those exponentials don't work for you as Well yeah no we're still dealing with We're still dealing with rotational Motors they could be servos they could Be actuators but the cost of those the Performance they're getting better their Their quality is better they're lower Cost but it's nothing like the like the Processor that runs our robot is Ridiculously affordable now compared to What it would have been 10 years ago Yeah Yeah and then even just kind of looking Forward I mean 10 years from now like Where do you see this industry really Really going especially I mean I guess We could just start first um Mike with Like in-home like right now I mean it's It's not gonna look like a human but Yeah maybe 10 years down the line it Could or even if it could should it If it looks like a human it won't be Functioning like a human I I in my Lifetime I'm not going to see something Like C-3PO walking around the house It's I had one of our investors who was Like it was like I realized I want this Guy to invest he said if you start as a VC in robotics and you've worked you

Know if you have a perspective after Working one year or three years in the Space you get one perception if you've Been in for 20 years you have a whole Different sense of the timeline and it Just isn't moving that fast and we Aren't changing fundamentally some Things like we do have the walking Robots we do have the back flipping Robots and and things like that but we Still aren't moving those legs any Differently we don't have you know the Other the other sci-fi thing is um Westworld we need Westworld we need 3D Printed muscle fiber and skin and the Whole thing to go to that and maybe There'll be something in the middle or It'll be Mannequin-like or or trying to represent Human but it won't be functionally what We do you think there yeah no I mean I Think the That idea or promise of some general AI You know robotic systems that that can Really understand the things that we as Humans understand and when you were Talking about you know manipulation These things they're incredible like you Get the things that you know a human can Do you know with their hands with their Eyes their understanding of the physical World is is really tremendous and so you Know we think about Um Manufacturing

You know that's not something that Actually I think a lot of people think About frequently but it really Manufacturing underlies the modern world As we know it today we think about the Planes Trains and Automobiles we we took To get here today the the appliances That washed our care for our clothes and Prepared our food the you know the Generators of power our cities the Construction equipment that builds our Cities all of those things are made in Factories and when people think about The future of a factory you know some People's ideas are you have that kind of The the lights out Factory and all this Equipment humming 24 hours a day and not A human Insight Um but whenever anybody's tried to build A lights out Factory for for durable Goods manufacturing it fails Um and we really believe that ultimately That's that's a technology dead end but In the in a world with the pace of Product Innovation that we have today There's no way you amortize the full Cost of automation over the life cycle Of the product that product's obsolete By the time you're done and so what we Really need is is a manufacturing Concept that is fluid and dynamic and Changes quickly and what that means is You want you want humans with their Intelligence and their flexibility and

Their understanding so that's why you Know that we started Veo robotics Because we really believe that the Future of manufacturing is going to be Humans doing what humans are good at and Robots doing what robots are good at and Working closely together and that if Anything I I feel like the uh the the Idea of General AI can almost holds Robotics back because instead of Thinking about how should robots work And what could robots do with what They're good at that will help people Whether it's in their homes or in a Factory we're trying to get robots to do What humans do And so really we need to be thinking About letting robots you know let the Robots do robots Um and that there's a huge amount of Value to be unlocked in that with the New kinds of technologies that are Available today yeah and I think we're Looking at the same way is that you know We're go like we're really clear when we Start talking to a Home Care Agency or Uh Independent Living we're we are not We're not out to replace caregivers with Robots what we are trying to do is one Give autonomy back to people that have Some need for care or have or some Impairment so they have more Self-control over the world even when There's a caregiver present but the

Other part is to extend the impact of a Caregiver so are there things that a Caregiver can set up that now they're Gone for the day but the robot can now Automate and deliver or take up or allow For like remote connection or some Monitoring aspect and all that so it's This interplay it's robot as assistant Is in that mix and it's heavy you know Small assistant big person in that Equation for us but um but that's the Way we we balance it and then so I think You'll see these co-robots sort of Flowing out in different ways as we Figure out those economies and the Benefits that we get out of it System you almost never have a situation Where the automation does exactly what a Human used to do automation will always Do there's parts of that task usually The less value-added parts of that task That can be automated and then you leave A person able to do the higher level Functioning it's never a build a machine To do exactly what the human did before Oh thanks nice well you've you've given Us a lot to think about so yeah this has Been awesome thank you so much thank you Thanks thank you [Music]