Content-addressed storage

  **Content-addressed storage (CAS)** identifies stored objects using a digest derived from their representation. A request names the content expected, while a storage service or retrieval protocol locates a copy. This separates an object's identity from a particular server or pathname. RFC 6920 standardizes one way to name information with hashes; individual storage systems use their own object formats and identifier schemes. [RFC 6920](https://www.rfc-editor.org/rfc/rfc6920.html)    CAS is useful for archives, version control, and shared build artifacts. Its central promise is precise identity and verification under the chosen hash scheme. Persistence, trustworthy authorship, and successful reconstruction require additional mechanisms. This article covers the software storage pattern, representative implementations, and its limits for [digital preservation](/wiki/digital-preservation); it is not a survey of associative-memory hardware or a commercial product catalogue.    ## Addressing content  

− A simplified store accepts bytes, computes a digest, and records a mapping from that digest to the bytes. Repeating a write with identical input can reuse the existing object. Reading by digest allows the recipient to hash the retrieved representation and check it against the requested value. The Venti archival server is a concrete implementation of this block-level design. [Quinlan and Dorward, Venti](https://www.usenix.org/legacy/publications/library/proceedings/fast02/quinlan/quinlan.pdf)

+ A simplified store accepts bytes, computes a digest, and records a mapping from that digest to the bytes. Repeating a write with identical input can reuse the existing object. Reading by digest allows the recipient to hash the retrieved representation and check it against the requested value. The Venti archival server is a concrete implementation of this block-level design. [Quinlan and Dorward, Venti](https://www.usenix.org/legacy/publications/library/proceedings/fast02/quinlan/quinlan_html/index.html)

    The digest is not a compressed copy from which an arbitrary object can be recovered. A separate copy must exist. Nor is a content name necessarily a network address: a client may resolve it through a local index, an archive resolver, or a network lookup. RFC 6920 distinguishes identifying information from locating and transporting it. [RFC 6920](https://www.rfc-editor.org/rfc/rfc6920.html)    An illustrative workflow is:    ```text  … 33 unchanged lines …  Content addressing does not make editing impossible. An edit produces a new immutable object, while a separate reference can designate the new current version. Unchanged objects may be shared across versions. The storage and retrieval costs depend on object boundaries, representation, and implementation; whole-file duplication and chunk-level reuse are different designs, not universal properties of CAS.    ## Historical context and applications    ### Archival storage  

− The 2002 Venti paper describes a network block store designed for archival data, with write-once objects and duplicate-block coalescing. Its applications included logical backup, physical backup, and snapshot file systems. Its archival policy went further than merely assigning hashes: the authors deliberately designed an append-only repository. The paper calls its performance results preliminary and bases its feasibility discussion on data from more than a decade of use of two Plan 9 file systems; those are source-specific prototype and workload observations, not a general performance result for all content-addressed stores. [Quinlan and Dorward, Venti](https://www.usenix.org/legacy/publications/library/proceedings/fast02/quinlan/quinlan.pdf)

+ The 2002 Venti paper describes a network block store designed for archival data, with write-once blocks and duplicate-block coalescing. Its applications included logical backup, physical backup, and snapshot file systems. Its archival policy went further than merely assigning hashes: the authors deliberately designed an append-only repository. The paper calls its performance results preliminary and bases its feasibility discussion on data from more than a decade of use of two Plan 9 file systems; those are source-specific prototype and workload observations, not a general performance result for all content-addressed stores. [Quinlan and Dorward, Venti](https://www.usenix.org/legacy/publications/library/proceedings/fast02/quinlan/quinlan_html/index.html)

    ### Build caches    Bazel separates an **action cache**, mapping action identities to result metadata, from a **content-addressable store** holding output files. This is a useful distinction between “which computation's result?” and “which resulting bytes?” Reading an existing cached object is reuse, not an independent execution of the computation. [Bazel: Remote Caching](https://bazel.build/remote/caching)    A Nix implementer's 2020 discussion similarly distinguishes names derived from build inputs from names derived from produced content. If a changed recipe produces identical output, content addressing can reveal that equality and potentially avoid unnecessary downstream work. The article explains a design motivation at that time; it should not be read as a statement about every current Nix store mode. [Hufschmitt: Towards a content-addressed model for Nix](https://tweag.io/blog/2020-09-10-nix-cas/)  … 43 unchanged lines …    ## Relationship to reproducible builds    [Reproducible builds](/wiki/reproducible-builds) ask whether specified artifacts can be recreated from the documented source, instructions, and environment. Content addressing helps name and compare the inputs and outputs. It does not cause a nondeterministic build to become deterministic or recreate missing dependencies. [Reproducible Builds: Definition](https://reproducible-builds.org/docs/definition/)    For an archival handoff, a useful proposed record contains the identifier scheme, representation rules, retained locations, exact source revision, reconstruction instructions, and known verification limits. Such a record lets a later investigator distinguish byte identity, availability, rebuildability, and the truth of any claims made inside the artifact.