Amazon S3 storage is one of the AWS services that has been around the longest and is one of the most widely used. For those unfamiliar with S3, it enables individual data items, ranging in size from one byte to five terabytes to be stored in scalable storage. There is no limit to the number of items, referred to as objects, that can be stored in S3. Amazon have recently published the usage statistics for the end of Q1 2012 for this service. At this time there are 905 billion objects stored in S3 with Amazon routinely handling 650,000 requests per second for access to these objects. The number of objects is growing daily by more than one million. This is an incredible amount of data.
These results motivated me to write about the different ways in which users access S3 and the kind of ways they are using it. For those unfamiliar with S3, then storage buckets, which are URL addressable entities are created in S3 and the data objects placed in these. A simple usage scenario is storing Web application assets in S3. For example, a typical Web application has many images and handling the delivery of these images to remote clients, even when cached, places an unnecessary load on the applications servers. A cost-effective way to offload this work is to place the images on S3. Since these are URL addressable, the HTML pages can link to the images here. This means as pages are requested, the images are delivered from S3 storage, freeing the servers the Web application runs on from this workload.
Another way that S3 is being used is via the AWS Storage Gateway. This service, still in beta mode, enables a cost effective, secure, off-premise data backup and disaster recovery mechanism. On-premise storage appliances connect to the storage gateway, providing a secure communication channel to S3 storage. Data stored using the Storage Gateway is automatically encrypted. For disaster recovery, point-in-time snapshots can be taken of on-premise data and sent to S3 providing secure and immediate access to backups should the need arise.
What is interesting is the number of different ways Amazon AWS provides for accessing S3. The simplest is via the Web management console, but this is only really practical for small amounts of data. Other ways include a Web Services API, command line tools, AWS Import/Export developed for large data sets, the above mentioned storage gateway, direct connect pipes for performance as well as many third party backup services.
With a range of security settings on S3 buckets, controlling who has read/write/update access, S3 offers an incredibly safe and cost effective scalable storage solution. If you have a storage requirement that can be off-premise, needs to be low latency yet scale and be secure, I urge you to seriously consider Amazon S3. In my experience over the last two years its a fantastic service. The usage statistics are strong evidence that I am not the only one who thinks so.
For related course information check out Learning Tree’s course entitled Cloud Computing with Amazon Web Services™.