Learning Tree recently received an alarming message from Amazon reporting that an IaaS instance started for a course event “has been placing spam (unsolicited messages, typically advertisements) on websites hosting online discussions, such as Internet forums: check the information provided below by the abuse reporter.”
In Learning Tree’s Cloud Security Essentials course we work with real cloud servers, which means that they are just as exposed to attackers as any other systems.
Two of the lab exercises in the course involve vulnerability assessment. The first uses an automated script powered through a web page — it is relatively easy to carry out, but the tool is limited. It can only check a few settings and can do no mitigation.
In the second exercise, each student is given one server running in Amazon’s US-East cloud infrastructure. You then connect to that server and check patch level for the operating system kernel itself, and the patch level and configuration of multiple network services.
Here’s where it gets risky: to make the exercise more interesting, we intentionally use cloud server images with a number of problems. The instructor has to deploy the cloud resources the day before the course starts. The intent is that all the students will find the problems with their servers and shut down the vulnerable network services, leaving those cloud servers in far better shape.
The problem is that you start a spare machine or two, and then some students drop out of the course at the last moment while others show up but don’t finish this exercise. At the end of that exercise at least two vulnerable servers remain running.
What lessons have been learned from this?
First observation: The vulnerable server was found by some attacker’s automated scan, exploited, abused, and that abuse caught within two and a half weeks. I’m surprised it didn’t happen much faster.
Second observation: Configuration management includes taking down unneeded servers, but the end-of-course process fell apart.
You may ask why the biggest cloud provider offers vulnerable machine images. It’s useful for this class, but why are they there? Some Amazon customers must continue to run the outdated image to support some legacy software. With very restrictive settings on the virtualized firewall in front of it, what Amazon calls the security group, it could be quite safe to run outdated HTTP and MySQL services (as long as you trusted the very limited set of client hosts and the people using them).
This event was interesting, if a little embarrassing, and no major harm done. However, it demonstrates an enormous risk caused by so-called “shadow IT”, a risk present even if you are using the very latest and fully patched cloud server images.
I’ll explain that next week!