To Blog

A practical approach to microservice security

Security is a tough and often poorly understood topic. Many development teams lack the knowledge and skills to address it properly. Confusion creates a thriving environment where various security experts and vendors are making fortunes by selling their services and solutions. I’m not an expert, just a practical guy who spent most of his life in development trenches. Nonetheless, I’d like to share my practical view on this topic.

Do you really believe that the more security, the better? Well, a prison cell, perhaps, is the most secure place in the world. But is this the right place for you? Similarly, a microservices system is an area where you can imprison yourself if you are not smart about security. I’m not saying that security is not important and security experts are wrong. Not at all. However, most of their advice is out of context and assumes the worst case scenario, which may or may not occur. Additionally, you are the one who knows the context best and is capable of making the right calls.

To make things clear, let’s have a look at another analogy. Imagine an office building. A bunch of people work there. Each of them has their own agenda and different levels of security clearance. Also, there are visitors who are allowed to enter the building. No one knows what they have in mind. If you don’t put locks on every office door, you’ll soon find yourself missing equipment and leaking information. Now, imagine your home. If you were to adopt the same approach, install locks on every door, and issue keys to your spouse, kids, and pets, will you have a happy life afterwards? I highly doubt it...

In the examples above, we applied identical security practices. Structurally, the two systems looked very similar. They both included buildings with rooms, doors, people, and valuable assets that need to be protected. We were supposed to get similar results, but we didn’t. The cause of this is because the contexts of where we applied the security measures were different. In the first case, we had an unsafe environment, where additional security was justifiable and required. We could not trust anything or anyone in the office building, so we had to take extra measures to ensure the security of our valuables. However, in the second case, in our own home, we were dealing with a trusted zone. We protect the perimeter of our houses by installing a lock on the front door, building a fence around the backyard, and, perhaps, putting grilles on windows. When someone knocks on the door, we check them before letting them in. And as soon as they are in, we tend to trust them. So additional security measures do not add much value in this case. They do, however, create lots of additional hoops that all occupants have to jump through every few minutes.

The situation with security in microservices systems is very much the same. Security experts tell horror stories about what could happen if your containers get hacked, microservices are attacked, or if some other malicious event takes place. Then they give you well-thought-out recommendations on how you should protect containers, encrypt communication, perform authentication and authorization, and much more. They’ll sell you advanced and expensive tools. Then you’ll spend even more by putting all of that to use. In the end - all of those goodies will generate so much overhead, that your system will literally crawl. And performance isn’t the only area that becomes affected - your developers will have to jump through so many hoops to change or add something, that even a simple task may take what seems like forever to accomplish.

Does this mean that security experts are wrong? By no means! They simply tend to give advice that is out of context. They tend to treat microservices systems as unsafe environments. When this is true, all of their recommendations make very good sense. And such systems do exist. A good example of such a system would be a cloud platform that allows clients to run Docker containers. Who knows what clients are going to be running in their containers?! Exactly - no one. So it makes sense to assume the worst and act accordingly. But, on the other hand, most enterprise systems can be viewed as trusted environments. Or, at least, they can be made trustworthy with a few extra organizational steps. Once they become trustworthy, a paranoid approach to security becomes overly expensive and unnecessary.

So, if your microservices system can be viewed as a trusted environment, what would be considered a practical approach to security? Here are a few tips:

  1. Clearly define and protect the system’s perimeter. All instances and services should be placed within a separate virtual network. Only external ports on instances that provide services to clients should be open. Everything else must be closed.
  2. All external connections must have encryption (typically HTTPS), authentication and authorization. Logging external calls for auditing purposes is also a good idea.
  3. All databases and services that run on external networks must be connected via trusted peer connections.
  4. All code that is running in the system must be reviewed for security. Code, developed by subcontractors, must be handled with additional care.
  5. All components that are deployed in the system must come from internal secure repositories. You should never directly deploy from public repositories. If you decided to adopt a 3rd party component, carefully review it and place it into your private repository before deployment.
  6. Only a few trusted individuals (usually DevOps engineers) should be granted access to production environments. It’s a good idea to restrict remote access to a single host that is inside the secure perimeter (aka, a management station) and allow connection to that host only from the machines of DevOps engineers. Developers should not have direct access to production environments. Instead, give them limited access to system logs and metrics.
  7. Do not mix multiple systems if they are managed by separate teams. Also, do not share the same account for production and test environments. Most security holes exist in the grey areas of overlapping responsibilities.
  8. Be conscious about the value, information, and logic in your system. Operational data, production statistics, and work assignments are rarely interesting enough to justify breaking into a system. Things that represent the most interest to hackers are user accounts, credit cards, personal information, and access to remotely controlled devices. Deal with that kind of information and logic with extra care. PCI and Hippa compliance gives specific guidelines on what needs to be done.

If you follow these simple rules, your system can be trusted, and you can go light on internal security mechanisms. This will guarantee improved system performance, as well as reduce your development and operational costs.

Happy microservicing!

Sergey Seroukhov

Principal Consultant and Founder, Enterprise Innovation Consulting