Alive and well in the worst way
By Alex Haynes, CISO, CDL
Let’s pretend you have offensive security skills and you want to use them for gainful employment. You attend a job interview and you listen to the benefits of what this company has to offer. First of all, most of the time you’ll be working for free – unless you find a vulnerability, and then they might pay you a few weeks later. You’ll also receive no paid sick days, paid holidays or days off of any kind because well, you’re working for free remember? The tools you’ll need for this job, that includes laptops, mobile devices, and any other widgets, you’ll have to provide yourself. As for a pension? Of course not. Nor any other kind of benefits you might expect – that means no subsidized gym memberships, health insurance, no discount vouchers, free breakfasts or subsidized food of any kind.
You’d be forgiven for thinking this company is made-up and doesn’t exist but what I’m describing is the reality of thousands of individuals who actually work on bug bounty programs for various crowdsourced security companies.
It’s hard to find a comparison to others in the current gig economy. Uber, Airbnb, and Deliveroo are all known examples of such companies, whereby employees work their own hours and forego traditional employee benefits (holidays, pensions, etc.) as a trade-off. While there are parallels in the way crowdsourced researchers work to other gigs, there is one crucial difference – gig economy workers are actually paid for their labor and can predict their income if they choose to invest 2 hours or 2 days a week.
That may sound insane, but let me elaborate. Researchers are only paid on bug bounties if they find vulnerabilities. To find vulnerabilities, you obviously have to invest your time. Sometimes you might be lucky and find critical, high-paying vulnerabilities in minutes – I was once lucky enough to find $6,000 of vulnerabilities in 30 minutes – not a bad hourly rate – but these findings are the exception, not the norm. Most of the time you don’t find any vulnerabilities at all. That’s hard to reconcile for most people, if you spent 6 or 7 hours trawling through an application and came up empty-handed, you get nothing for your time. Worse, you might actually find a vulnerability, but then it’s classed as a ‘duplicate’ meaning someone else before you have already found it, and you still get nothing.
If I was to attempt an analogy using Uber then it would be the equivalent of calling for an Uber, and then having two dozen Uber drivers simultaneously drive to your location to pick you up. You’d only jump in the first one that arrived, and he’d take you to your destination. The other drivers who arrived after the first would get nothing, even if they wasted their time and fuel getting to your pickup location. Oh, also the guy who drove you to your destination wouldn’t be paid straight away. He’d probably receive the payment a few days later, or if you’re unlucky, anything from a few weeks to a few months later. Maybe.
Of course, Crowdsourced security isn’t transportation, but the business model is similar – no full-time employees so you don’t have to burden yourself with the high-cost structure of recruiting and maintaining a workforce. When it comes to Cybersecurity, that workforce is highly skilled, hard to find and expensive to maintain so it actually gives them numerous advantages over pentesting companies who they now directly compete against.
You see Pentesters (or anyone with an offensive security skillset) are hard to find on the job market because yes, there’s a shortage and yes it’s getting worse. Once you recruit them you have to pay them top dollar so they stay, and you also have to keep them happy by sending them to conferences and allow them time to do their own research and attend certification courses (which you’ll also pay for). On top of that, you have to pay them traditional benefits like pension, a regular salary, etc. And since you’re sending them to various customer sites you obviously need to pay for expenses. While you have them in your employment you also have to make the best use of them, billing them out at 1000$ a day (or more!) so you can make some money off them. Not having them working on client engagements is very expensive as they are sitting around doing nothing.
Crowdsourced companies have leapfrogged these complications in a spectacular fashion by just removing that from the equation entirely. Your ‘employees’ can be anywhere in the world and as long as they are incentivized to participate in bounties, even if they aren’t paid unless they find something, then you’ve just made your business leaner. You don’t need to pay for their certifications, tools, upkeep, pensions or any of the costs that are associated with full-time employees. You pay them per vulnerability so it’s irrelevant how many there are – you don’t need office space to contend with nor worry about even reviewing their performance because it’s a self-fulfilling cycle – those that perform better get paid more, so are invited to more bounties, then get paid even more and so the cycle continues.
But who would actually sign up for this? Well, thousands in fact. First of all, there aren’t that many people working in this fashion. Forget the marketing statistics you hear – crowdsourced companies may claim anywhere from 150,000 to 300,000 people on their platform, but all they are doing is counting the number of sign-ups. When you drill down into the statistics, only a tiny percentage of those people have ever logged a vulnerability. I have my own anecdote to illustrate this – back when it started I signed up to HackerOne in 2014 just to see what the fuss was about. I also signed up to Bugcrowd and spent most of my time on that platform instead. I came back to HackerOne in 2018 and saw that I was ranked 5800 out of 120,000, without ever logging a single vulnerability. I would joke about it by saying I was in the top 5% of researchers on HackerOne without ever having done a single thing!
When I logged my first vulnerability my ranking jumped to 3000 odd, my second vulnerability jumped me another couple of hundred places. This all implies that very few people actively participate, and even geographically, a great deal of their researcher base is in India. It’s telling then that HackerOne has recently released the ability for you to be paid directly in Indian Rupees (payouts are in US dollars by default).
Most people on these platforms, like myself, don’t do it full time, especially if you live in Europe or the US. Salaries in the cybersecurity sector are good enough that don’t have to moonlight for extra money, which is why without exception, all the researchers I speak to do it for fun, the challenge or just the safety net of being able to hunt for bugs in applications without the threat of legal action.
So what’s been done about this? Well, crowdsourced companies are acutely aware of this criticism and are slowly trying to address this issue. Synack launched ‘missions’ a year ago which are short, focused tests for a single vulnerability, whereby if you find the vulnerability or not, you’ll get paid. Bugcrowd also has launched their ‘next-gen’ pentest which follows a similar vein – if you flow through a testing methodology but don’t find anything, you’ll get a lump sum – and if you find vulnerabilities then you get paid for those too.
Arguably the industry has a lot of work to do. While they have teams internally dedicated to ‘researcher success’, these are customer-focused. I’ve lost count of the number of times I’ve had a company, not payout (either by ignorance or on purpose), ignore a vulnerability or just generally misclassify the severity of something that’s found to pay less.
The one exception to this is Synack, who have solved this issue by having a slightly different business model – they payout from their own funds all the time, and negotiate with companies separately – this is also the reason they have the fastest payouts in the crowdsourced industry and often you can be looking at money in the bank 48 hours after submitting a vulnerability – a long stretch from the weeks and months it takes for other platforms to pay.
It’s hard to see this continuing into the future – bug bounties and disclosure platforms aren’t new anymore and it’s telling that the researchers you find on one platform are identical to the other platforms because simply put, those with a desire to do so now participate in bug bounties and the recruitment drive is over – there isn’t a neverending stream of researchers to pull from. This is problematic as their entire business model depends on two things – a continuous stream of people looking for vulnerabilities and having those people do it mostly for free.
As the bottomless pit of researchers hasn’t materialized, platforms have had to switch tactics. ‘Cycling’ researchers is common – for example if you have 30 researchers assigned to a private bounty program, and a lets say 20 of those haven’t logged a single vulnerability in a few months, it’s fair to say they aren’t looking anyway – so you cycle then out and invite 20 new people in to replace them. This is to generate that constant flow of researchers and a different set of eyeballs might spot something the others haven’t (this is one of the primary advantages of crowdsourced security over pentesting, so makes complete sense). The other technique is gamification – Payments are increased for certain companies and this is communicated out to everyone to re-kindle interest – the introduction of badges, achievements, t-shirts and all sorts of goodies as rewards are also common if certain criteria are met (Meeting certain targets or types of vulnerabilities for example). Some of the notifications appear almost like sales offer – “for this month only, all cross-site scripting attacks are paid out at DOUBLE the rate. Hurry, offer ends April 30th!”
This is essentially a race to the bottom since techniques like this will work in the short term, but will come up against the same long term boundaries – there isn’t an infinite supply of highly skilled specialist labor that works for free. Crowdsourced platforms have redefined pentesting, but have also redefined the gig economy, in the worst way possible.
About the Author
Alex Haynes is Information Security Manager at CDL.He has a background in offensive security and is credited for discovering vulnerabilities in products by Microsoft, Adobe, Pinterest, Amazon Web Services, IBM and many more. He is a former top 10 ranked researcher on Bugcrowd – a vulnerability disclosure platform with over 400 vulnerabilities to his name.
Alex can be reached online at [email protected]
