Why is Web3 Such a Cybersecurity Disaster?

Hello hi everybody well web3 was Supposed to be incredibly secure the Encryption lauded uh over and over again But really it's become a bit of a cyber Security disaster hasn't it Um the hackers have stolen more than Four billion dollars in cryptocurrency To date so what's to be done what is to Be done or is it just still early days For crypto who knows to discuss these Issues uh we're joined by Jesse Irwin CEO of amulet Stephen Tong CEO of zelik And uh interrogating them is our very Own moderator Lorenzo Francesca bicarai Give them a big round of applause Everybody thank you Thank you [Music] Hi everyone thank you for being here We're here to talk about crypto thank You Justin Stephen for being here I would like to begin with some numbers According to an estimate there are Around 38 billion dollars stored in Crypto today And last year hacker stole around 2.7 Billion in crypto that's almost 10 of All the liquid in crypto Uh Jesse what the hell like why are Hikers still in so much crypto the Incentives for being able to just go out And hack this code are wild we have a Built-in bug Bounty program where if you Find a bug the money falls out into your

Lap part of the reason that it looks This way is because The Primitives that We get to build on the code that we get To use isn't quite as trustworthy or Strong as we'd like it to be but also When you're looking at more conventional Technology we have these immense Security organizations and Frameworks And all kinds of tools and risk Management and threat modeling we can Put into place but those organizations Tend to have some sort of central Structure and in the blockchain space we Love decentralization but it adds Complexity and when you're dealing with Folks who are tinkering and hacking and Playing you're not always going to have A group of people who are thinking about The need to put an application security Program or a plan in place to keep these Catastrophic bugs from making their way Into code and then getting money out Stephen what do you think why is this Happening like why we keep losing money Yeah why why are we losing so much Crypto every day to hackers I think Jesse put it well Um if we look at on the surface level The market for bugs here is very liquid Because normally if you have a bug let's Say you have an Uber ATO like an account Takeover you have to find someone to Broker this bug and sell it and they're Gonna have to sell it to someone who can

Actually run that campaign whereas in Crypto it's extremely liquid because you Can just like immediately you know Exercise it for its market value like How much you can steal with it right and Then at the same time the technology is Not mature I think we look at more Fundamentally The design constraints that we see in The web3 space are you want it to be Permissionless and you want it to be Decentralized so decentralized is great For Censorship resistance but at the same Time it means that there's no Central Authorities and because there's no undo Buttons right so a mistake is Catastrophic when we wire money to the Wrong bank account if you email the bank Really fast usually they can fix it you Can't do that in crypto right and when You have a permissionless system there's No identity and this is great for Democratization right but it also means That there's no social costs for example There's lots of systems out there like Bgp which are not very secure but Everyone plays nice because everyone has Like a Delaware C Corp or something and They don't want to like be have other People be mad but in blockchain there's No such social cost yeah Um we had a panel on ransomware before That was great and we hear about

Ransomware all the time because Obviously it's a huge problem in 2022 The FBI received thousands of reports Um totaling at least 34 million with an M not a b And obviously the losses are are much Higher because we don't know about it But you know that's still much lower Than crypto hacks so Stephen do you Think that the public and even the cyber Security press maybe not talk about Crypto security enough I think they're Just decent desensitized to it Is it just because there's too many Hacks and we just lost track of them Yeah yeah what do you think Chester I Mean we get to a point where every time You hear news about crypto it's how big Is the latest amount of money if it fell Out of a blockchain and what did the Winner take home this time It kind of reminds me a lot of when You're you're watching the news and you Hear about the latest art Heist and you Know only so many paintings can get Stolen and only so many hacks can be Adventurous and impressive before it's Boring and we're ready to move on to Something else Is it also do you think it's also Because it's a sort of still a niche Industry and not a lot of people are Aware of the existence of crypto or Don't own crypto

Somewhat but I know that the news is Getting out there when I have a Grandparent call me and say that wasn't Your blockchain was it so I I think the Information or at least the blast impact Of these hacks and explosions it's out There a little bit but I think when you Look at all the other things going on in The world we have you know banking Crises and we have all kinds of Macroeconomic things going on a little Bit of a war for an attention as well So I've been covering this space for a Couple of years now and you know maybe I'm a pessimist but I don't have a Feeling that the security in crypto is Getting better do you think it's is that Fair Stephen am I being too pessimistic I think it is slowly getting better as People get more accustomed to building In such a hostile environment When you take let's just say web Developers and you dump them into Mission critical environment there's a Period where you had a cultural Culturally acclimated to that I also Think that the fundamental problems are Still there though decentralized and Permissionless it makes it very hard to Build a secure system for example There's it's very hard to think of a Plausible design for like a secure Platform or a firewall given those Design constraints

It's just a very hostile space in General and people have to get used to Building in that What do you think Jesse is it getting Better I do think that we are seeing Truly understand the reasons you want Security plans Security Programs and Coordination and incident response They're understanding why they want Those things and I see more demand for Them day by day that said some of the More fundamental things that we need to Do to alter the development process and Put architectural designs and threat Modeling in place that's not as Accessible as it could be for helping us Better manage risking code folks are Slowly coming around to it so I have Hope I get asked for much more Interesting security strategies and to Solve more exciting problems than I did Five years ago and being asked for a run Book for example is like yes you want to Plan so that's great but it is very slow Yeah is it also an issue of educating The people in the space because they Don't come from the cyber security Industry most definitely I've had the Privilege in my career to see and work With teams of all sizes across the Entire technology sector and it's been Amazing but a lot of folks in crypto They are self-taught they're tinkerers It reminds me of a lot of my hacker

Friends But when you haven't seen what it looks Like to push code into production that's High reliability and has all of these Things done to it to run bugs out it's Difficult to know what you should be Doing and there is a gap there so it's a Lot of trying to introduce things to Folks and make it accessible And sometimes they get it and other Times they call back a year and a half Later and say ah yeah that thing what Was it I want it now yeah so there's a Lot of teachers yeah you have to do a Lot of teaching to your customers Stephen like how much do you have to Explain to them how important your job Is depends on the customer Um and just out of curiosity any of you Got on your crypto stolen before Me Oh I mean I lost some Bitcoin it's not a Hard drive somewhere in a landfill But I haven't had it stolen before now Yeah how about in the audience has Anyone got under crypto stolen Yeah it is a big problem Um So what are you you know what do you What's your advice if someone comes to You and says you know I want to have Some crypto how do I secure it what do You tell your friends Maybe that not tacky about how to

Protect your crypto uh I guess depends On your threat model so I think if your Threat model is like cyber criminals Which I think is the most common one It also depends on how much money you Have in kind of like if you're a Corporation or not so if you're a Company you should probably have like a Multi-sig and all the bearers of the Multi-say keys should be Harbor wallets And you should make sure that you have Like you know M out of n where the Numbers M and N makes sense if you're Individual I think you should use a Hardware wallet you should disable Two-factor authentication you should Disable SMS based two-factor Authentication on all your services Um especially your email because your Email is kind of like a root password For your web identity and then make sure To turn on like app based two-factor for Everything and then you should be fairly Secure or you can deposit it in a Centralized custodian if you're like Acceptable if find acceptable for your Use case Generally um Worried about The time especially when I work with Core teams and we're dealing with these You know token plans and grants the Problem is that we have to protect an Address for a long term and you know if

You're married your spouse needs to be Able to use it and access the Information if you live in California You can't write it down in a notebook You have to have one of these metal Cryptocurrency wallet stamps just in Case it really depends but for the most Part really just the hardware wallets And being very strategic about how long You are willing to leave your crypto in An exchange or in someone else's hands Is really key because even if you do Leave crypto in a centralized exchange That you trust you know we saw FTX we've Seen tokens go away when reserves are Mismanaged so it's good to hold it on Your own it's a little more complex but Definitely store it offline and don't Let it live on the computer you use Every day yeah unfortunately exchanges Are the one of the favorite targets of North Korean government hackers so And Stephen I just wanted to go back a Little bit too you said have multi-sig Or multi-signature if you're an Organization can you explain a little Bit more about uh why that is important Um if you have a checking account for Your company you probably have dual Approval on it you probably need to have Multiple admins to approve transactions Above like a hundred thousand dollars For example these are policies that you Also probably want on your

Cryptocurrency treasury as well I also Wanted to mirror Jesse's thought I think That air gapping is an excellent Solution and not enough people use it so Some practical implementations of Multi-cigs popping the stack would be Agnosis safe that's very popular or if You want to get fancy you can like pay Fire blocks a very large SAS Subscription for some NPC stuff So it's you know it's it's we're Starting to see like cyber security Folks like you come to the industry come To the crypto industry what does the Industry need to do better still to Protect people's crypto I think one of the things that I've Experience is just in designing and Being part of developing this protocols There are some sharp corners and some Rough edges we definitely have Abstraction to do so that these tools And these platforms are more usable a Lot of that work you know we've seen in The normal web the Google Chrome team Did an amazing job on their usable Security work we need that in the crypto Space to improve outcomes for the people Who rely on our chains but we also just Need to really be intentional about what We're building and have a plan to Protect it at every step I mean we have Regulation breathing down our necks Every single day and having had

Experience in fintech already like you Either get regulated or you Self-regulate no matter how Decentralized we are though we've Created these amazing crazy wild Protocols and chains they do incredible Stuff so we just need to link arms a Little more often and figure out how to Really make it work for the people who Use these tools What do we need to do better I think it comes down to multitude of Factors we need to improve across the Board I think one would be better Developer education as Jesse has said Better culture around development Practices for example we still see a lot Of clients deploy code they say give Code to us that is not tested whatsoever Literally no unit tests so that needs to Not happen I know I'm being quite prescriptive but I would hope that software I'd deploy my Money into has unit tests I think better Ux for the user side of things extremely Important better user education is just As valuable A lot of people lose their money to Phishing this is still like one of the Top ways people lose their money not to Smart contract hacks or bad custodians And I think there's some other ones that you Know but they're not as important as

Developer education user education in my Opinion Yeah that's I think that's important and At the same time I think it's fair to Say that you know a lot of people like You have come to the industry there have Been some improvements so what do you Think has improved in terms of the Ecosystem of cyber security companies Coming to crypto How is that changing the crypto world Well I think there's definitely more of An acknowledgment that vulnerability Assessments need to be a part of the Process for putting code into production In the past few years we've also seen a Lot of security folks instead of rush to Make you know the meanest joke about a Blockchain really come to the front Lines and help people figure out how do We manage all of this user burden in all Of this risk that's on our shoulders so I see more collaboration and cooperation That said one thing that is a struggle Is sometimes vendors know that there Aren't enough security Engineers they Don't have an informed customer to push Back on them and every once in a while That can make my job or you know the Trust that folks using these chains put In them a little bit more difficult What do you think about the cyber Security ecosystem coming to crypto Is that a that's a good thing right I

Largely mirror what Jesse said yeah Actually I'm curious like what brought You to this space you you both come from Cyber security traditional and Quote-unquote cyber security what Brought you to crypto I guess oh yeah okay okay sorry Um I guess me and my friends were all Hackers that's how we started zelk we Really enjoy learning about new novel Technology and finding weird bugs in Them and we wanted to also apply our Skills where they're the most valuable We figured that IOS zero days are cool but people are Losing hundreds of millions of dollars That seems fairly important and we Should work on it What about you Jesse what brought you Here I think for me I've seen Traditional security teams of every size And shape you know servicing Organizations all kinds of complex Problems There's such a huge opportunity with all Of this decentralization we have to step Back to look at what's not working in Our existing cyber security and Engineering practices and to just tear It all down and start over we could have Developers pick up some of the security Practices that most companies struggle To put in place anyway we could get

Better security outcomes we could run More bugs out of code and we have an Opportunity to actually integrate how we Do security into development so that It's an optimization and not this extra Weird scary thing no one understands or Knows how to do and I think that is Probably one of the most exciting Opportunities with all the creativity And energy in the space that I see here There are many but the chance to just Rebuild what security could look like so It's easier or more natural just as Thrilling yeah that's interesting so I Remember talking to someone a couple of Years ago who works in the space as well And he told me that in his opinion we Should think of the code that goes into Smart contracts the same way that we Think of code that goes into airplanes Satellites Rockets you know it's So-called Mission critical meaning that You have to get it right from the get-go Because you know if you ship it with Serious bugs it's going to go tragically Bad you know as you said millions of Dollars are going to go away Steven is That a fair and accurate way to look at Crypto and coding crypto and why is it So hard to secure yeah I think like the NTSB States the chance of dying in a Plane crash is 1 in 30 million or so If you look at the amount of money in D5 And the amount of dollars have been

Stolen it's more like one in twenty Dollar per dollar so one in 30 million Versus one in 20 I think we should try To close that Gap And how do we do that what's the first Step The first step is to write unit tests For the smart contracts That's a good one I also think step two Is we're seeing this movement towards Interoperability in the blockchain space In a lot of places we are using smart Contracts to bridge one chain to another As load-bearing infrastructure when Frankly it's really risky to do that and We shouldn't be doing it if we're able To innovate more quickly in the area of Interoperability and of course I work in Cosmos we love our inter blockchain Communication we're able to do that more Securely these Bridge hacks that you're Probably tired of reporting on at this Point they go away we get to solve more Exciting problems around virtualization And cryptography maybe even Cryptographic agility that would be Amazing so I think if we just keep Innovating an interoperability a lot of The things that are brittle and break And are being misused just won't Be the leading story in the industry Anymore yeah Yeah so bridges are a big like week week Link here

So are you saying that like we should Move away from the bridges in you know Completely or from the way they are Right now Well I think Ultimately if we get strong Interoperability between all these Ecosystems the bridges we decide we want Might look a little different but we Might not end up needing them at all and I think that would be just amazing for The era of these bridges that explode And catch on fire to just be over so we Can worry about other things yeah what Do you think Stephen About what about the you know the issue With Bridges and securing them bridges Are load barrier infrastructure I agree With that They need to be really secure and it's Generally hard to secure them I think The track record speaks to that I don't know if that should exclude us From Building Bridges in general I just Think that we have to be very careful Just because Space shuttles are very hard to build Doesn't mean that we shouldn't build Them Yeah and you have a company that does Security audits Stephen which you know That's your specialty Um how do you think you know crypto Founders should think about audits

They should Seek them throughout the process in Early They should not be thought of in Afterthought where you wave the security One then you make the problems go away This is generally going to lead to worse Outcomes because the earlier integrate Security into your process the less Expensive it will be for example if you Catch the issues during development and Testing it's as cheap as having an issue And having a ticket to fix that issue if You find this in like a post development Review Sprint then you might have to go Back and make changes to code that You've already gone to code freeze on Which is more expensive if you get it During an audit you're going to have to Pay for the audit if you get it during a Bug Bounty you have to pay the Bounty Which is more expensive if you get White-headed or if you get blackheaded The cost increase as you go through this Building pipeline so it should be done Early and it's also very important to Remember that the security reviews are Meant to be consultative in best efforts And you can't expect external reviews to Catch everything so it's very important To have security across the board as a Process and not just a point in time You know magic wand yeah so audit early And often and don't just think that it's

A stamp of approval forever right that's One of the misconceptions I think that People have right you know I've been Audited or good Yeah I think to add to that sometimes in Crypto a security audit is the only Security treatment that something gets Before it goes into production if we Don't have unit tests then we don't Understand the correctness of the code And if it works as intended usually the Vulnerability is where we don't Understand the code and it doesn't work As intended to see a team sort of step Back and have a more holistic Perspective of how to execute on Security would definitely make the world Better but also just to see some vendors Step up and say hey like this is not It's not a health check it's not there Are no bugs left in your code would be Really really helpful because too often I see tweets fly by and marketing blogs Say oh well we passed our audit you know Audits don't work like that Yeah Yeah and I think it's hard to say that Sometimes crypto feels like the wild West of Finance you know with the Obscure companies with obscure business Models weird names And they get hacked but you know who are The actual victims Um just say like what are we you know

I mean a few years ago when we were all Stuck at home during covid I had friends Who I hadn't heard from since I was in High school call me and say what's this Crypto thing I'm gonna take my savings And I'm gonna make a ton of money and I'm gonna retire We're at the point now where people hear About you know this ability maybe in a Happier Market cycle to profit and it's Not just a bunch of Institutions who are Losing funds it's not just a bunch of Investors there are countries on this Planet where Bitcoins used more than Fiat currency And there are places where it's Immensely difficult to transfer money And that's what this is used for so it's Not just some rich guy who you know Drives the lambo and eats a bunch of Steak and makes Bitcoin carnivore jokes There are real people here and we need To be really thoughtful about the Outcomes for them I mean we need to be Responsible enough so that when people Do put money here it doesn't destroy Their entire financial future to have a Bug even if they took on more risk than They should have in the first place yeah This in the industry the crypto industry Maybe doesn't think about you know Individual investors like retail Investors as much History why there is a lot of focus on

Institutions because we are looking for Widespread adoption and we want to see Big things happen And what I always push back to say is When you look at large economies half of The economy is an Enterprise giant Institution and the other half is you Know small businesses Small retail investors really have to Think about what it looks like on both Sides Any thoughts uh Stephen I saw you not in I think that if you go on the Twitter Account coin sessions I've read enough stories of people who Have lost their life savings to know That retail is Usually the one who suffers in the end I Find that fact very unfair Because if you're the founder of like a D5 and it blows up If you raise money for that Venture That's not really your money but the People that lost money in that hack they Deposited their personal savings and Wealth and for a lot of people who live In countries that are not for example The United States That's a lot of money for them and it's Very unfair And that's also why like we do this job Yeah Yeah um I think something that also People don't don't realize maybe is that

You know web3 or D5 has a lot of web 2 Problems right I think we the industry Wants to pretend that you know it's Completely new and everything is Uh is different but there are Web Two Problems in web3 as well can you Stephen Can you talk a little bit about that oh Yeah fishing yeah fishing is so bad Um here like people will hijack the Social media accounts of board Apes I Know the director of security at like Board Yacht Club and he tells me about The insane like Steps they go through to secure just Their Twitter account because people Want to take it over to launch a Phishing campaign it's like oh new Airdrop and then there's a like a wallet Wallet trainer website it will link them To or another one is they'll like do bgp Or DNS stuff for example I think they Last year they hacked the front end of Curve using DNS to do fishing there it's Crazy I think better ux and do that Better education can help solve that but It's horrible and not just that there's The classic problems of securing Sensitive key material right like you Need to not have your private key be Leaked seed phrases are not great Um And it's not just a user prominence Institutional problem I mean if you Think back all the way to the very first

Large group of hack it's Mount gox That's not a fancy smart contract hack That's just their backend infrastructure Got poned and their keys got stolen and The Bitcoins are gone yeah yeah and Sometimes hackers just take advantage of Discord channels right they take over The Discord Channel or Discord bot And Fish Everyone on the channel yeah Yeah and like web 2 security is like the Foundation you cannot have web 3 Security that will do security yeah so You need to get both right yeah yeah There's so much opportunity in the space And the attacks that people pull off are Always going to be the most simple the Least expensive they can because most of The attacks are financially motivated That said I've been playing with blockchain since 2017. I have not seen a single security Problem across my desk that I did not Experience before I started playing with Blockchains in my career I mean people Talk about how difficult it is to you Know build a strong crypto wallet I Raise them that we know things about Browser extensions from password Managers Um there's so many application security Problems there are so many attestation Problems and there's so much we probably Need to figure out about adopting more Trusted Computing models so we have

Better things to work with but I don't See anything just so shockingly new that We don't have prior knowledge somewhere Else in the security field to help us Figure out how to reason about it yeah We have the knowledge we just need to Apply the right place in the right way So I guess just to finish I'm curious Like you know would you recommend your Non-techy friends and relatives to Invest in crypto or do you recommend it I'm not allowed to speak on investment Advice Yeah I do not have that position But um Say generally like I'm a security Engineer I'm not entirely risk Intolerant But the Market's not great right now and We don't know how things are going to Shake out I can't say during a period of Instability you know Hack all the things drop all your money In there because that's not what I'm Doing right now yeah that's fair well Thank you so much for being here thank You for coming and thank you for Listening everyone [Applause]