There has been a lot of press recently about data breaches where poorly configured Amazon S3 buckets were implicated as the source of the breach.
Most recently, a care home in England exposed some 10,000 resident records, including care plans, public funding for individuals, and their decisions on the right not to be resuscitated.
In the USA, a software company that developed an app for managing marijuana dispensary customer data, left an S3 bucket open to the world, leaking data on 30,000 medical and recreational marijuana users.
And plastic surgery imaging company Nextmotion also exposes 900,000 plastic surgery images, including before and after photos of patient procedures.
Why is this happening?
We’re told that the cloud is just as secure as on-premises deployments, right? We’re living in a networked world where everything is connected to everything. We expect instant access to our data, documents, photos, banking information, shopping, and so on. But we also rightly expect, and are promised, a degree of privacy – we want instant access to everything, but we don’t want anyone else to be able to see it. This presents a complex problem for businesses – to give consumers the level of access they need, with the level of privacy they expect.
Privacy requires security, and due to the complex nature of modern online services, security must exist at many layers.
For your privacy to be protected, security must be assured at each layer. It’s no good having the best data security if, for example, your application can be easily compromised. Maybe this could be due to weak password policy, or some poorly written code. It’s also no good if you have the best perimeter security in your datacentre, but your network security is lax, and it allows someone to penetrate the network and exfiltrate your data quickly.
But in this article, we’re discussing data security only, specifically concerning Amazon S3 buckets, as they have been getting a lot of bad press recently.
What exactly is an Amazon S3 bucket?
In summary, S3 stands for Simple Storage Service, and it is an Object Storage solution from AWS. I wrote a detailed guide which you can read here. Think of file storage – a place to store things like documents, photos, video clips, etc. – very much the sort of personal information that when uploading to a web application, we expect to remain private. S3 storage is easily accessible via web APIs, and it is infinitely scalable. S3 customers are only charged for storage consumed, plus data transfer and requests made to the service. It is, therefore, very desirable for application developers due to its low cost and ease of use. It’s no wonder so many businesses have adopted S3 storage for the object storage component of their applications.
Who is responsible for the security of data stored in Amazon S3?
In AWS, security is a shared responsibility. Amazon is responsible for the security ‘of’ the cloud, and AWS customers are responsible for security ‘in’ the cloud.
Mostly AWS is responsible for the physical security of the datacenters and hardware for the software that they control – hypervisors and databases and for the physical network. AWS customers are responsible for network traffic security, operating system and firewall configuration, application security, identity, and access management, and, critically, customer data.
So, if you misconfigure an AWS S3 bucket and your sensitive customer data is exposed, you cannot blame AWS – it is your responsibility as an AWS customer to ensure that your data stored in AWS S3 is secure.
Suggested Read – Amazon S3 Pricing Simplified
So how do you Secure your Amazon S3 Bucket?
S3 Buckets can be set to ‘Public’ or ‘Private.’ Generally, when data is leaked, it is because admins or developers have mistakenly put ‘private’ data into a ‘public’ S3 bucket. You may ask why on earth you would ever want an S3 bucket to be public? There are, of course, use cases for an object store to be public – if you are hosting a public-facing website, for example.
The good news is that S3 buckets are private by default – you must go through a couple of steps, including an apparent confirmation with a warning to make an S3 bucket public. You can see below in the ‘Create Bucket’ wizard that the ‘Block all public access’ box is checked by default.
Unchecking the box results in this warning, which requires the user to check another box acknowledging that any objects stored in the bucket may be publicly accessible:
Here you can see the list of S3 buckets in my account – I’ve highlighted the ‘Access’ column which will either say ‘Objects can be public’ or ‘Bucket and objects not public’:
If you don’t want the contents of your S3 bucket to be visible to the world, always select ‘Block all Public Access’ then grant access to those users or services that need it using one of the methods discussed below – Access Control Lists, Bucket Policies, and IAM policies.
S3 Access Control Lists (ACLs) – avoid if you can!
Misconfigured ACLs are also another common reason for breaches of data stored in Amazon S3.
There are two types of Access Control Lists available in S3 – Bucket ACLs, which allow you to control access at the bucket level, and Object ACLs, which will enable you to control access at the object level. However, ACLs have long been considered a legacy access control mechanism that predates Identity and Access Management (IAM). The advice from AWS is not to use ACLs unless you already have them in place, and they are working as you need them to. The general advice is to control access to data using Bucket Policies or IAM Policies.
S3 Bucket Policies
S3 Bucket Policies are specific to individual S3 Buckets. They specify permissions of who can do what within the bucket and cannot be attached at the object level. If you have multiple buckets and require the same policy for each bucket, you’ll need to recreate the policy for every bucket. If the policy is updated, you’ll need to remember to update the policy on every individual bucket to which it applies.
Identity & Access Management (IAM) Policies
IAM policies specify what actions are allowed or denied on what AWS resources. IAM Policies can be attached to groups, users, or roles that are then subject to the permissions you have defined. IAM policies can be used to specify permissions across S3 buckets, and they are managed centrally within IAM – one IAM policy can be applied to multiple buckets, so if you make a change to an IAM policy it will automatically be applied to all buckets and/or objects covered by the policy. IAM policies can also be used to control access to other services rather than just S3.
Amazon S3 offers ‘Default Encryption’ for all objects stored in an S3 bucket – Default Encryption ensures that all objects stored in an S3 bucket are encrypted at rest, offering a further level of protection for your data. Objects are encrypted using either Amazon S3 Managed Keys (SSE-S3) or Customer Managed Keys (CMK) stored in AWS Key Management Service (KMS). The term has always confused me a little, as ‘Default Encryption’ is not turned on by default – the ‘Default’ part means that once turned on, all objects stored in the bucket (unless otherwise specified) will be encrypted using the default encryption algorithm for the bucket. Default Encryption can be enabled via the AWS console, Command Line Interface, or API via the AWS SDKs. Enabling default encryption will not encrypt any existing objects stored in a bucket, only new ones added. If you want to encrypt existing objects in a bucket, S3 batch operations can be used to encrypt them with a single request.
S3 also supports the HTTPS protocol, so you can encrypt data in transit when uploading to or downloading from S3.
How can I check if my S3 Buckets are Secure?
AWS offers several cloud-native tools to monitor access to your S3 buckets, plus there is a range of third-party tools and managed services to help you identify and rectify any potential problems.
AWS CloudTrail logs will track bucket level actions, so you have a record of any access to your S3 buckets. Object-level actions can be tracked using Amazon S3 data events.
AWS config can be used to monitor ACLs and Bucket Policies for any policy violations that allow public to read or write access to your buckets.
AWS Cloudwatch can be used to monitor when specific actions are taken on S3 resources – you can then trigger specific processes in response to certain events.
And since February 2018, the AWS Trusted Advisor S3 Bucket Permissions Check has been free for all S3 users.
Third-party Cloud Management Platform can periodically scan your S3 resources against over 25 best practice checks and display compliance in a Red/Amber/Green status to help you quickly home in on any potential issues.
So there you have it – if you are using Amazon S3 Object Storage, speak to your developers and admins, point them to this article, then have them reassure you that they understand how to secure S3 buckets and that you won’t be the next headline concerning an Amazon S3 data breach. Remember, the responsibility is yours and yours alone!