Why storage decisions fail when teams compare products instead of layers

Architecture teams do not usually make weak storage decisions because the feature matrix was missing a row. More often, they ask a product question before deciding what responsibility the product is meant to carry.

That is how a conversation becomes “S3 or Snowflake?”, “a data lake or backups?”, or “cold storage or archive?” Each sounds concrete. None is a complete comparison.

A product comparison is often a responsibility map with the labels removed. Put the labels back before making the choice.

Product names hide the contract

Product names are convenient shorthand. They are also lossy.

A name can refer to an interface, a managed service, a database, a file format, a query engine or an architecture pattern. Two products may overlap in capability while carrying very different operational contracts. One may bundle identity, metadata, execution and storage. Another may expose only a storage API. Comparing their monthly price without separating those responsibilities produces a precise answer to the wrong question.

The hidden contract includes at least five things:

the state the component is authoritative for;
the workload it must serve;
the failures it must survive;
the party that controls it;
the evidence that demonstrates those claims.

Until those are visible, “better” usually means “better at the dimension the vendor page made easiest to compare.”

S3 versus Snowflake is not one decision

S3 names an object-storage service and API family. Snowflake names an integrated analytical data platform. They can participate in the same system. They can also overlap in where analytical data is retained. That still does not make them equivalent units of architecture.

A fair comparison would separate the stack:

object persistence;
table and schema semantics;
catalogue and lineage;
query execution;
workload isolation;
identity and policy;
operations, recovery and support;
export and replacement.

The answer may favour a more integrated platform. It may favour storage and compute that can be operated separately. It may use both. The decision becomes defensible only when the team can say which responsibilities it is buying, which it is retaining and which it is deliberately coupling.

“Where should the data live?” and “Where should the query run?” are related questions. They are not one question.

A data lake is not automatically a backup

A data lake may contain a broad history of source and transformed data. That history can be extremely useful during recovery. It does not, by itself, make the lake a backup system.

The same identities may be able to delete production data and lake data. Corrupt transformations may overwrite both. Essential catalogue state may live elsewhere. Retention may be designed for analytical convenience rather than recovery points. The lake may contain the records but not the application configuration, keys, schema history or dependency order required to restore service.

A recovery system begins with a failure model. It asks which clean states must remain available after accidental deletion, malicious change, tenant loss, region loss or an untrusted control plane. It then protects the full reconstruction chain and proves it through restore.

A lake can be part of that design. Calling it a backup does not complete the design.

Replication is current state; recovery needs history

Replication is valuable because another copy can continue serving when a component fails. Its weakness is the same mechanism: it copies change quickly.

Delete the wrong records and replication can delete them elsewhere. Encrypt the live dataset through compromised credentials and the encrypted state may propagate. Introduce a logical error and every healthy replica can agree on the wrong answer.

Recovery needs historical distance from the event. That may come from point-in-time logs, versioned objects, immutable recovery points, independent exports or several mechanisms used together. It also needs an administrative distance: a path that still works when the production identities or control plane cannot be trusted.

Availability asks, “Can another current copy serve?” Recovery asks, “Can we reconstruct an acceptable earlier state?” An architecture needs both questions where the workload justifies them.

Cheap retention is not an archive strategy

A low-cost storage tier solves a useful economic problem. It does not decide what constitutes an authoritative record.

Archive design has to preserve more than capacity. It has to preserve context, integrity and authority across time. Which schema explains the record? Which key decrypts it? Which event placed it under legal hold? Which policy permits retention? Which person or process can authorise disposal? Which reader will still work after the originating application is gone?

The most expensive archive failure is not always lost data. It can be retained data that cannot be interpreted, cannot be proven authentic or cannot be disposed of safely.

Tiering and archive belong in the same cost conversation. They do not belong in the same definition.

Open formats reduce one dependency, not every dependency

Open formats matter. They can make inspection easier, broaden tool choice and reduce dependence on a proprietary reader. That is real architectural value.

But a portable file is not the same as a portable system.

Parquet files may be readable while the schema history is missing. Table data may be open while the catalogue, lineage and access model remain locked inside a service. A logical database export may omit extensions, roles or performance-critical behaviour. A vector index may be exportable while the source chunks, embedding model and preprocessing steps needed to reproduce it are not.

Portability should be tested as a chain:

export representative data;
export the metadata and configuration required to interpret it;
open it through an independent path;
verify meaning, completeness and controls;
record the work still required to operate it elsewhere.

“Uses an open format” is evidence for one step. It is not evidence for the whole chain.

Products should be compared inside a responsibility map

Before a shortlist, build a small responsibility map.

Responsibility	Authoritative state	Primary operator	Failure boundary	Required evidence
Live application state
Analytical data
Search, cache or vector index
Recovery history
Archive record
Catalogue, policy and keys

Now place candidate products into the map. Some boxes will contain one integrated service. Others will contain several components. Some products will appear in more than one row.

That is acceptable. The goal is not maximal separation. The goal is to prevent one product label from silently inheriting responsibilities that nobody designed or tested.

Compare failure domains before feature lists

Feature lists describe what a platform can do under expected conditions. Architecture also needs to describe how it fails.

Ask whether live data, replicas, backups, catalogue state and encryption keys share:

the same tenant or subscription;
the same privileged identity;
the same region or provider control plane;
the same automation path;
the same deletion authority;
the same operational team and escalation route.

Two copies in one destructive boundary may improve component availability without creating meaningful recovery isolation. Three services controlled by one compromised identity can fail as one system.

This does not mean every dependency must be separated. Separation has cost and operational weight. It means correlated risk should be chosen explicitly rather than discovered during an incident.

Compare ownership in operational terms

“Managed” and “self-hosted” are operating models. Neither is a complete ownership assessment.

A managed platform may be highly ownable when identities, keys, exports, policies, evidence and transfer routes are under clear organisational control. A self-hosted platform may be effectively unowned when only one engineer understands it, no restore has been tested and the configuration exists as undocumented state on a server.

Ownership becomes visible through verbs:

who can grant, revoke and review access;
who can rotate or recover keys;
who can export data and metadata;
who can restore outside the primary environment;
who can inspect logs and prove policy operation;
who can transfer the system to another operator;
who can shut it down without losing required records.

If the answer to every verb is a supplier, a single employee or an untested script, the ownership claim is weak regardless of where the hardware sits.

The review should end with evidence

Architecture language is full of claims that sound complete: durable, immutable, portable, compliant, highly available, recoverable.

Each claim needs an evidence object.

Durable may be supported by service configuration and failure testing. Immutable may require proof that the relevant administrators cannot shorten retention. Portable may require an export opened with another tool. Recoverable requires a restore report and validation result. Governed requires policy, ownership and access evidence from the path users actually take.

Where evidence does not exist, the honest output is not a stronger adjective. It is an assumption, a test or a risk with an owner.

That habit changes procurement discussions. Marketing language becomes a hypothesis to verify rather than a property inherited by purchase.

The practical rule

For every product under consideration, write one sentence with five parts:

This component serves this layer, is authoritative for this state, must survive these failures, is controlled by this owner, and is evidenced by these tests or records.

If the sentence cannot be completed, the team is not ready for a product comparison.

If two products complete different sentences, they are not direct alternatives.

If one product completes several sentences, review the coupling and the shared failure domain before calling the integration a simplification.

Start with the layer. Name the contract. Then compare products.