Skip to main content
Back to Glossary
storage

Object Storage

A flat-namespace storage architecture where data is stored as discrete objects with metadata and a unique key, accessed via HTTP/S APIs (S3-compatible) — used for AI training datasets, backups, and data lakes at petabyte scale.

Unit: TB/PB capacity

Full Definition

Object storage organises data as objects rather than files in a hierarchy or blocks in a volume. Each object contains the data payload, variable-length metadata (user-defined key-value pairs), and a unique identifier (a URL key within a bucket namespace). Access is via HTTP/S APIs — Amazon S3 established the de facto standard, now supported by all major platforms including Google Cloud Storage, Azure Blob Storage, MinIO, Ceph, and NetApp StorageGRID.

Object storage is optimised for write-once-read-many workloads at unlimited scale: AI training datasets (hundreds of TB to petabytes), ML model checkpoints, database backups, log archives, and media files. It provides high durability (typically 11 nines / 99.999999999%) via erasure coding across distributed nodes. Object storage does not support in-place modification or POSIX file semantics — applications must be designed for S3-style access patterns.

In AI factory workflows, training clusters load datasets from object storage into local NVMe or shared parallel filesystem caches; inference services load model weights from object storage at startup.

Also Known As

S3 storageblob storageS3 compatible storage

Source Reference

Amazon S3 API Reference; SNIA Storage Networking Dictionary; Ceph Object Storage Documentation

Related Terms