By Byron V. Acohido
Dissecting the root cause of Uber’s catastrophic data breach is a worthwhile exercise. Diving one level deeper into the scenario that led up to the popular ride-hailing service losing personal data for 50 million passengers and seven million drivers shows us why this particular type of hack is likely to recur many more times in 2018.
Related podcast: Why DevOps and security are destined to intersect
Hackers got deep into Uber’s Amazon Web Services platform. They did this by somehow obtaining, then using the AWS logon credentials of one of Uber’s software developers, who left those credentials accessible on GitHub. Though we don’t know nitty gritty details, security analysts say something like this had to have happened:
While working on an AWS coding task, the Uber developer took some of this code base and uploaded it to GitHub. No security sins to this point. ‘Git’ is a system for controlling the latest version of software programs; GitHub is an online repository where developers upload code for peer reviews and such.
Here’s the wider context: imagine the degree to which Uber, in order to connect riders and drivers, uses software to tie into services hosted by Amazon, Google, Facebook, Twitter, iPhone and Android. Uber is a prime example of an Internet-centric enterprise comprised of a collection of tools and services hosted by myriad partners. Think about how frenetic the software development process must be too keep Uber humming.
Root cause
It appears that Uber’s programmer was, understandably, hustling. While trying to be productive, he or she did not take into full consideration the dire security implications of routinely including AWS logon credentials in GitHub uploads — where hackers could find it, or finagle their way to it.
Was this a happenstance, inadvertent exposure? Quite the opposite. This precise type of security blunder, in fact, has become a routine software development practice — one that is being repeated hundreds, if not thousands of times a day, says Jeremiah Grossman, Chief of Security Strategy at SentinelOne.
Grossman
“The average piece of software today has to take data from multiple locations and bring it to one spot,” Grossman explains. “When you’re developing software, your code might have to authenticate to Amazon, or to Facebook, or to many other systems. So developers are including those authentication keys in their source code because the software needs to leverage those credentials.’’
If that sounds like a systemic poor practice, it absolutely is. It’s an offshoot of the rise of DevOps – the movement in tech circles to unify software development (Dev) and software operation (Ops.) DevOps encourages both to happen simultaneously, instead of sequentially, to speed up deployment of new programs.
“One of the consequences of DevOps and the increased need to automate everything is that your automation scripts and your automation tools need to have access to everything very fast,” explains Liz Rice, technology evangelist at Aqua Security. “So it has become very, very easy for people to put account credentials into those scripts, and it is that kind of mistake that ends up with people losing control of those credentials.”
Grossman puts this new type of oversight into perspective: “It’s a very huge mistake and it’s also a very unforgiving mistake. It’s like leaving your key under the doormat where anybody can easily find it and use it.”
Copy cats
It is reasonable to anticipate that given the success of the Uber hack, cybercriminals will step up schemes to find and take advantage of AWS credentials left exposed on GitHub. “If you know how to do the right searches on GitHub, you would find thousands of AWS keys lying all over the place,” Grossman says. “It’s a really common mistake that you’ll see out there all over the place, and it has been that way for years.”
The smart thing for companies to do is require their software developers to store access credentials separately from any code uploaded to repositories like GitHub, Rice says. “Putting the credentials inside the script just makes it much more risky,” she says.
Amazon is aware of this exposure and addresses it under its extensive ‘shared responsibility’ policy for AWS users, thus shifting the burden onto companies. It’s up to AWS users to jealously guard access to their AWS accounts. To it’s credit Amazon doesn’t leave users out on a limb without a safety net. Amazon also supplies the means for users to exercise fine-grained control over access privileges to their AWS accounts, Rice says. So companies have the technology to compel their developers take extra (less-convenient) steps to jealously guard their AWS credentials.
Sacrificing convenience
At the end of the day it comes down to developers pausing while rushing to push the latest, coolest new program into live deployment – and actually taking those extra security steps to limit AWS access to specific persons at specific times.
Rice
“It’s an extra step to think about how we’re going to keep those credentials secure and unfortunately people sometimes forget to do it,” Rice says. “It’s exactly the same as you as an individual having the responsibility to look after your own passwords and PIN numbers for you bank accounts. If you leave your passwords and PIN numbers out on a piece of paper on your desk, you bear some of the responsibility if someone were then to take your money. In the same way, Amazon can’t be held responsible if an employee leaves a root password lying around. The company and the employee bear some responsibility.”
Keep in mind this exposure is by no means restricted to Amazon and GitHub. DevOps style software development is moving fast and furiously on dozens of other code repositories, such as Sourceforge, Launchpad and Bitlocker.
Erlin
And the temptation to take security shortcuts with every popular web service looms large, says Tim Erlin, vice president of product management and strategy at Tripwire, a Portland, Ore.-based supplier of security configuration management systems.
“Organizations should consider how people access tools like Office365, Salesforce.com, Concur, Google’s application suite, and others. Additionally, cloud providers like Amazon, Azure and others are also potential attack vectors,” Erlin opines. “More broadly, this breach should drive organizations to evaluate the various points of access throughout their software development lifecycle, and their supply chain in general.”
