Docker Image Vulnerability Research

24% of latest Docker images have significant vulnerabilities

Conversations: Hacker News, Twitter, LinkedIn

Subreddits: r/sysadmin, r/security, r/docker, r/hacking, r/ubuntu, r/opensource, r/commandline, r/vrd, r/netsec

Intro

Understanding the vulnerability landscape in the container ecosystem is critical to our mission of securing the world, so we decided to put our technology to work to answer a key question: what is the current state of vulnerabilities in official Docker repositories?

The official Docker repositories consist mostly of images for open source and commercial software, are generally managed by the author or vendor, and contain the most widely used images in the Docker community. The 20 most popular images have been pulled tens of millions of times each. Increasingly, they are used as the base for distributed systems replacing some of the functionality of 3rd generation configuration management software.

Docker takes security seriously; this project focuses on the wider community and practices. While this article focuses on identifying problems, the next will propose some solutions.

On February 6th, we scanned 91 of the 133 official Docker repositories. This is every repository with a ‘latest’ tagged image consisting of a major linux distribution and a functional package manager. We used an open source scanner (vuls) that we modified, and then analyzed the data using tools we’ve built for our company, Federacy. Unfortunately, this removed alpine and static binaries because vuls currently supports neither. Scoring is CVSSv2 and priority follows the standard.

New to Docker? We wrote an addendum for you.

Observations

Percent of docker images vulnerable 55060281837e39550505a8543b0a5391acb558c1a127e3f3c53942227f82aab0

24% of Docker images tested have significant vulnerabilities (high or medium priority). 10.53% have high priority vulnerabilities.

Images vulnerable by distribution 816acdbdb06e21c1151b5751c4896462ec24b6fd8516e2f7a1360fe5b4e1ba79

Ubuntu images have significantly more vulnerabilities than Debian images. Debian is the least vulnerable major distribution. It has fewer moderate and high priority vulnerabilities (8%) than Ubuntu (27%). RHEL-based distribution sample size was very small (4% of all images).

Docker images by base distribution c4915981d8907ebb91da8904fd16b503bbfed666b89bc8104bdc9a2671c57cbc

Debian is the most widely used distribution — RHEL, the least, by far. Debian is the dominant base distribution among official docker repositories, and accounts for over 79% of latest-tagged images. Ubuntu accounts for ~16%, and RHEL-based distributions the remaining ~4%.

Average no high priority vulns 8a3e20b2fcaa30ed68a736ab925c1ce8b561f6c918b6ee4060e4ed551f3f8663

Older releases of Debian and Ubuntu have significantly more vulnerabilities. Correlating indicators are: a higher average CVSS score, more packages installed, and more updatable packages.

The number of high priority vulnerabilities in Debian 8.6 and Ubuntu 14.04 was considerably higher than in the new releases (8.7 and 16.04) and Average CVSS (v2) score shows the same.

Packages installed by release 1872fe8e9f73e8358da828c189f641a613c207a776598d882ca093356e2e293e

New distribution releases had fewer packages installed and updatable. While this is not a direct indicator of vulnerabilities, it means a smaller surface area, and fewer unapplied updates.

The most common vulnerability was SSL Death Alert (CVE-2016–8610) This is a DoS-able vulnerability affecting software compiled against GnuTLS, OpenSSL, and NSS, including nginx but excluding httpd.

An attacker can flood vulnerable software with malformed SSL ALERT messages, because a new thread is not allocated to parse them. IP-based firewalls and deep packet inspection can be used to mitigate the vulnerability.

Most common vulnerability per distribution

Debian: CVE-2016–7056 (86% of images). An openssl vulnerability that could result in the compromise of a private key. However, it requires local access and the use of cache timing attacks during multiple signatures, so while a severe vulnerability, it is unlikely to impact most people. Affects most linux distributions because openssl is widely used.

Ubuntu: CVE-2016–8610 (93% of images). Described above.

RHEL: CVE-2016–1248 (50% of images). A vim vulnerability that could result in code execution and privilege escalation. Affects any system with vim, and requires opening a malicious file.

Most common high priority vulnerabilities

CVE-2016–4658 (5% of images). A libxml2 vulnerability that is misleading. Affects most linux distributions, not just Apple products as stated in the CVE, and despite being rated moderately is a significant vulnerability for people who make libxml functionality accessible remotely, which includes nokogiri and most open source software that parses XML/HTML.

CVE-2016–1252 (5% of images). A vulnerability in how the apt package manager parses signed repositories could lead to altered packages and resulting code execution. Affects Debian 8.7 and Ubuntu 16.10, 16.04, and 14.04.

Conclusion

As engineers we have a responsibility to our users and other stakeholders. Being compromised isn’t just embarrassing, it can cause major damage to the trust in our brands. It can also have serious financial implications, as evidenced most recently by the $350m haircut Yahoo took on their acquisition price.

Managing known vulnerabilities is the first step towards a strong security posture. If we’re not updating our systems, and keeping an eye on emerging vulnerabilities that are yet to be patched upstream, we’re basically leaving the front door wide open.

The challenge, of course, is how to best roll out updates. Many startups have invested in CI/CD systems for their code, but many have not implemented a similar process for building images or validating third-party images. Likewise, very few startups run automated vulnerability scanning in production. At Federacy, we’re trying to make this so simple and useful that it’s a no-brainer for every startup.

Next Steps

1. More Images. Add more sources of images, especially cloud (AWS, Digital Ocean, Google Compute, etc).

2. More Scanners. Add clair and other proprietary scanners.

3. More Time (Series). It would be interesting to see how quickly images become vulnerable after being deployed.

4. More Automation. Fully automate generation of charts and data so they are always current.

Addendum: How to Scan Docker Images and Mitigate Vulnerabilities

Scanning Docker Images became significantly easier in 2016, and we now have a number of options. Docker Hub, Quay.io, and others offer scanning for repositories they host and there are now open source scanners (clair and vuls). At Federacy, we're supporting the latter approach by making the open source scanners easier to use and connecting them to a better vulnerability database. It only takes a few minutes and one command to scan your Docker images for free.

Mitigating vulnerabilities in Docker images is a bit more difficult, but here are a few suggestions: install package updates in your image build process. If you can, automate updating packages on your image while it is running. Add vulnerability analysis to your image build process. Use Alpine or a similar distribution. Build a static binary image.

Thanks to ant, fujin, kris, nick, btm, and william for reading drafts of this and making it much better.

Weekly Roundup

Updates on major vulnerabilities, compromises, and security news.