S3, Block storage, File storage, and Object storage

1. Overview

2. Object Storage

2.1 What is Object Storage

The term object storage, or object-based storage, derives its name because it packages data and metadata into objects. Metadata is essentially data that describes other data, or in the case of object-based storage, is information about the files that are typically stored within an object.

Objects are stored in a flat structure or address space. Objects are each assigned an object ID or unique identifier, enabling them to be retrieved from a single repository or pool of storage. Enterprises value this approach because it offers greater flexibility on where they can place their data beyond block- and file-based storage solutions.

The popularity of cloud object storage products from Amazon Web Services (AWS) and other providers have helped heighten object storage's profile in recent years.

2.2 Pros

2.1.1 Ease of use

Each object gets a unique ID and an HTTP URL that can be publicly accessible. Data read and write operations are very simple and maybe performed directly by a user’s browser, via representational state transfer (REST), without having to go through the control of the server app. The app is released from the rigid structure of database tables and file system hierarchy.

2.1.2 Scalability

Unlike classical storage using files and tables, the infrastructure to store objects doesn’t grow in complexity when data grows. Object storage can grow quickly, without limits.

2.1.3 Agility

File systems and databases are complex and require constant care by the sysadmin or database administrator. Thanks to the simplicity of objects, the developer or app owner doesn’t have to depend on these professionals, which eliminates bottlenecks in the app’s evolution. A developer has more freedom to change an app without the help or blessing of the infrastructure team. This agility aspect is what makes object storage so attractive for modern apps.

2.1.4 ETCs

• A simple HTTP API, with clients available for all major operating systems and programming languages
• A cost structure that means you only pay for what you use
• A built-in public serving of static assets means one less server for you to run yourself
• Some object stores offer built-in CDN integration, which cache your assets around the globe to make downloads and page loads faster for your users
• Optional versioning means you can retrieve old versions of objects to recover from accidental overwrites of data
• Object storage services can easily scale from modest needs to really intense use-cases without the developer having to launch more resources or rearchitect to handle the load
• Using an object storage service means you don’t have to maintain hard drives and RAID arrays, as that’s handled by the service provider
• Being able to store chunks of metadata alongside your data blob can further simplify your application architecture

2.3 Cons

• You can’t use object storage services to back a traditional database, due to the high latency of such services
• Object storage doesn’t allow you to alter just a piece of a data blob, you must read and write an entire object at once. This has some performance implications. For instance, on a filesystem, you can easily append a single line to the end of a log file. On an object storage system, you’d need to retrieve the object, add the new line, and write the entire object back. This makes object storage less ideal for data that changes very frequently
• Operating systems can’t easily mount an object store like a normal disk. There are some clients and adapters to help with this, but in general, using and browsing an object store is not as simple as flipping through directories in a file browser

3. Block Storage

3.1 What is Block Storage

Block storage is synonymous with storage area networks (SANs) and enables storage services that aren't possible with file storage technologies used in network-attached storage (NAS) systems. Block storage involves saving data in blocks, or raw storage volumes.

Each of these storage blocks can appear as an individual hard drive to an external server operating system. An operating system, in turn, uses the Fibre Channel (FC), Fibre Channel over Ethernet (FCoE) or iSCSI protocols to access these blocks.

The reason for block storage and therefore SANs are popular in enterprise IT environments is due to their flexibility and performance characteristics. Block storage supports a variety of workloads that require low-latency, network-based storage operations, including business-critical applications, virtual machines, RAID implementations and databases.

And while it's not to be confused with file storage systems—the type that enables organizations to offer their employees shared file services over a network using a NAS—a file system can be layered atop block storage since block storage appears as raw storage to server operating systems.

In the cloud, block storage is available from services like AWS Elastic Block Store, or AWS EBS, which provides scalable block storage that can be used by Elastic Compute Cloud (EC2) instances.

3.2 Pros and cons

3.2.1 Pros

• Block storage is a familiar paradigm. People and software understand and support files and filesystems almost universally
• Block devices are well supported. Every programming language can easily read and write files
• Filesystem permissions and access controls are familiar and well-understood
• Block storage devices provide low latency IO, so they are suitable for use by databases.

3.2.2 Cons

• Storage is tied to one server at a time
• Blocks and filesystems have limited metadata about the blobs of information they’re storing (creation date, owner, size). Any additional information about what you’re storing will have to be handled at the application and database level, which is additional complexity for a developer to worry about
• You need to pay for all the block storage space you’ve allocated, even if you’re not using it
• You can only access block storage through a running server
• Block storage needs more hands-on work and setup vs object storage (filesystem choices, permissions, versioning, backups, etc.)

4. Usage

4.1 Object storage use cases

• Cloud storage
• Unstructured data storage (documents, images, video, etc.)
• Big data storage
• Backup and recovery
• Archival storage
• Big Data Analytics

4.2 Block storage use cases

• Databases
• Business applications
• Virtual machines
• RAID
• Boot from networked storage

5. Comparison between Object, File, and Block Storage

5. Reference

https://www.enterprisestorageforum.com/storage-technology/object-storage-vs-block-storage.html

https://searchstorage.techtarget.com/definition/object-storage

https://www.ibm.com/blogs/cloud-computing/2017/02/01/object-storage-benefits-myths-and-options/

https://cloud.netapp.com/blog/block-storage-vs-object-storage-cloud

https://medium.com/harrythegreat/%ED%81%B4%EB%9D%BC%EC%9A%B0%EB%93%9C%EC%83%81-%EC%98%A4%EB%B8%8C%EC%A0%9D%ED%8A%B8-%EC%8A%A4%ED%86%A0%EB%A6%AC%EC%A7%80-object-storage-%EB%9E%80-9d9c2da57649

http://fakecineaste.blogspot.com/2017/12/block-storage-object-storage-and-file.html

https://www.digitalocean.com/community/tutorials/object-storage-vs-block-storage-services