Using AWS Organizations
AWS Organizations have been around for a few years now, but until recently most of our projects here at Truss haven’t used them; most of our projects are small scale enough that the added complexity of multiple AWS accounts didn’t seem worth the management overhead.
Last summer, though, we had the chance to lay out some greenfield infrastructure for a new project and do some experimentation. It didn’t take long before we found a lot of advantages to using AWS Organizations even for relatively small projects and with the right patterns, the overhead of using them is surprisingly manageable. We thought we’d share our findings and give folks some pointers on how best to use them in their own infrastructure. If you want to see how these all look in practice, you can look through our terraform-layout-example GitHub repo, which implements these patterns with heavily-annotated Terraform configurations.
Why use AWS Organizations?
If your infrastructure is relatively small and uncomplicated, you may be asking why you should bother with AWS Organizations at all. Do you really need multiple accounts if your infrastructure isn’t really that complex?
At the most basic level, AWS Organizations give you consolidated billing -- no longer do you have to juggle billing for individual accounts, instead you get a single bill that covers every account in the organization. In fact, if you want, that can be the only thing an organization gets you. However, the benefits of an organization extend far beyond just fewer billing headaches.
The biggest advantage of AWS Organizations is compartmentalization. For the project we were setting up last summer, one of our big concerns was security. We wanted to be able to give developers as much flexibility as possible experimenting with AWS services, but we needed to make sure that our production environments could be built to restrict access as much as possible. We could achieve these goals in a single AWS account or multiple AWS accounts without an organization (and we have in the past), but using an organization makes this much easier.
Instead of needing to use complicated IAM permissions to lock down AWS services, where a typo in your pattern matching could break functionality or expose data, the account boundary lets you give developers free reign in a sandbox account, with much more restricted access in other accounts. You could do this with multiple unlinked AWS accounts too, but Organizations make it easier to consolidate user management into a single account (see below) and place controls on services with service control policies (SCPs).
What is even better with an organization is that we can create accounts and add them to the hierarchy all with Terraform, which we’re big fans of here. This makes it easy to add new accounts for one-off projects and clean them up when you’re done. You also get the benefit of all the user management and security functionality you’ve already built in for the other accounts. As a result, the cost of the initial setup quickly pays for itself, especially in a larger project.
Organization patterns
We've come up with a number of patterns from working with AWS Organizations over the past year. You can read the full list in our Engineering Playbook, but here are some of the most important and helpful ones that we’ve found:
Your master account (we call it the “org-root” account) should have a very small number of users (usually a select group of infrastructure engineers or leads). This account should only be used to manage the organization or handle billing (which is all consolidated at this level).
The next account you create should be your “id” account. This account is where you’ll handle user management and nothing else. All of your IAM users should be set up here, added to groups here, and then you can manage their access to other accounts here by letting them assume roles in other accounts.
Finally, the last of your “skeleton” accounts will be the “infra” account. This account should have access limited to infrastructure engineers, and here you can define services that will be used by the organization as a whole. The two big examples for us have been handling our root DNS zone and an Atlantis deployment to handle Terraform applies for us.
Rather than granting delivery or project managers direct access to the org-root account, we can create a role that can be assumed in the org-root account by a group of users in the id account that grants access to only billing services.
Create two organizational units (OUs) to start at least, a “suspended” OU and a generic OU that will encompass all your accounts aside from the org-root account. The suspended OU should have an SCP that denies access to all services attached to it, and you can use this to quarantine accounts you believe have been breached or accounts which are otherwise frozen (we do this with GovCloud placeholder accounts, for instance). Having the other OU that contains all other accounts makes it easy to enforce blanket policies for the entire organization.
Cloudtrail, Config and GuardDuty can be set up to aggregate at the Organization level, and we recommend doing so if at all possible.
We’ve made some Terraform modules that help automate some of these patterns:
trussworks/org-scp/aws defines a set of common SCPs you can apply to OUs
trussworks/iam-cross-acct-src/aws sets up cross-account roles in the id account
trussworks/iam-cross-acct-dest/aws sets up cross account roles in your non-id accounts
Once you have your org-root, id, and infra accounts, the rest of the organization is more free-form. You can have a sandbox account that allows developers power user access to do almost anything there, an account for each of your dev, staging, and prod environments, an account with extremely limited access to archive your logs, or whatever suits your needs. Our best practices are still evolving on this, but it’s likely we’ll continue exploring the possibilities and adding new findings to our playbook.
Implementing AWS Organizations
You may be saying to yourself, “this all sounds great if I’m starting from scratch, but I have a lot of infrastructure already built in our AWS account already. Do I have to rebuild everything if I want to use Organizations?”
Luckily, the answer is that you can probably find a way to transition to using an Organization even if you already have existing AWS infrastructure. Here at Truss, we’ve been moving our company infrastructure to an AWS Organization and we did start over with a new organization and account structure. Since everything we build is set up with Terraform, it’s easy for us to stand up services in a new account by just copying the Terraform code over. It hasn’t been a painless transition, but it’s a lot less work than it might seem. If you aren’t using something like Terraform to define your infrastructure, this is a great time to start.
However, if that sounds like a nightmare, you can import an existing AWS account into a new organization, and slowly move services and resources between accounts. This is probably more difficult with things like S3 buckets (at least at this point, there’s no way to move an S3 bucket from one account to another), but this does let you start to use Organization features like SCPs while you migrate resources into new accounts. It also means that you can make your future projects in new accounts in your Organization, so that when it eventually becomes untenable to keep everything in a single AWS account, you aren’t trying to do all this with ten times the infrastructure.
Hopefully this post gives you some ideas for how you can benefit from using AWS Organizations, and shows that it’s not as difficult to gain those benefits as it might seem. We have some future blog posts coming up soon on some other AWS management patterns and tools we’ve been working on lately, so stay tuned.