Storage Contract¶

This document is the normative application-to-storage contract for Hubuum. It defines what a selectable backend must provide, what application callers may assume, and which obligations are verified structurally or behaviorally.

The exact Rust surface is StorageBackend in crates/hubuum-storage-core/src/backend.rs. The semantic capability group map maps every required trait, and semantic-coverage.toml inventories every method, effect, tracked input variant, and evidence scenario. method-contracts.toml exhaustively specifies every page, candidate, batch, and complete-collection method. Those machine-checked sources and this semantic contract must change together.

Scope¶

The contract applies to every backend in StorageBackendKind::ALL. Focused models and test doubles may implement narrow capability traits, but they are not selectable backends and must not claim the guarantees in this document.

The contract is a statically linked Rust interface, not a dynamic plugin ABI. An external adapter can implement the capability traits from hubuum-storage-core, but Hubuum must still add it explicitly to exhaustive dispatch and the sealed application certification registry before operators can select it.

Contract Layers¶

The complete guarantee has three layers:

Rust trait bounds alone cannot prove that a method wrote an audit event, used one commit boundary, delivered an event, or emitted telemetry. Certification exists to test those obligations, and the sealed registry prevents structural conformance alone from making a backend selectable.

Boundary and Ownership¶

Values crossing the boundary are owned by backend-neutral crates. A storage API must not expose:

• Actix request, response, extractor, or application-state types;
• ApiError or public HTTP status decisions;
• Diesel traits, rows, schema modules, queries, connections, or errors;
• a native pool, client, transaction, cursor, or database type identifier;
• global configuration, metrics registries, or exporters; or
• unredacted credentials, payloads, filters, URLs, or claim tokens in diagnostics.

The application owns use-case orchestration, public validation, authorization policy selection, HTTP projection, and StorageError to ApiError conversion. The contract owns operation-shaped traits, validated requests, observable results, and bounded error classifications. Each adapter owns persistence layout, native transactions, locking, queries, migrations, driver errors, and conversion into contract values.

Errors move in one direction:

adapter error -> StorageError -> application or API error

Complete Backend Contract¶

StorageBackend is indivisible for application selection. Its traits are organized into 20 documented semantic capability groups covering:

• resource lifecycle and atomic transaction composition;
• identity, authentication, groups, principals, and authorization facts;
• catalog, computed, aggregate, relation, history, inventory, and search reads;
• tasks, imports, exports, backups, restores, and remote targets;
• audit reads, subscriptions, fan-out, delivery, and retention; and
• readiness, metrics snapshots, execution context, and other operational state.

A missing method is a compile error. Dummy success, empty-result, and generic unsupported implementations do not satisfy the semantic contract. Truly optional behavior is composed outside the aggregate. In particular, WorkerNotificationProvider may be attached for low-latency wake-ups because durable polling remains the correctness path.

Six-Part Audited Storage Contract¶

The following six semantic guarantees are mandatory and are tested together for every registered backend. The behavioral certification gate that follows controls whether an implementation may be selected by the application.

1. Attribution Is Mandatory¶

Every ordinary audited mutation requires an EventContext. The context names the immediate actor and carries request, correlation, task, and initiator provenance where applicable. It is not optional.

Code without a user actor must choose explicit system or worker attribution. Compatibility helpers whose historical names contain without_events do not disable auditing; they use system attribution. There is no ordinary event-suppression escape hatch.

2. The Result Proves the Audit Write¶

An ordinary audited mutation returns StorageMutationOutcome<T>:

• Committed { value, audits } means state changed and the non-empty audits set identifies every durable event written for that atomic change.
• Unchanged(value) means the requested operation was a semantic no-op. It carries no receipt, writes no lifecycle event, and must not advance revision or modification time merely to manufacture a change.

StorageAuditReceipt contains the stable event sequence, event UUID, entity type, action, and before and after revisions. It deliberately omits snapshots, metadata, actor details, and other permission-scoped audit content. Authorized callers retrieve those through AuditEventStorage.

Returning a receipt is not permission to synthesize evidence after commit. It must be derived from the event persisted by the same atomic operation.

Portable audit documents¶

Every adapter constructs the permission-scoped body of a new event with AuditDocument from hubuum-events-core, reexported by hubuum-storage-core. The document owns the summary, optional before and after object snapshots, metadata object, and schema version. Adapters add entity coordinates and provenance through NewEvent, but must not assemble those document fields in a native event row.

AuditDocument::try_new and AuditDocumentBuilder::try_build reject non-object snapshots or metadata. Schema selection is also backend-neutral: documents without a numeric resource revision use version 1, while a snapshot containing a positive integer revision uses revision-aware version 2. An adapter cannot select or advance that version independently. A new document schema requires a new construction rule and matching conformance expectations before any native serialization changes.

Entity snapshots are made from boundary projections when a canonical helper exists. Collection lifecycle events, for example, use StorageCollection::audit_snapshot; a native collection row is first validated into StorageCollection and never defines its own public audit shape. Persistence rows and backend serialization remain adapter-private.

3. State and Durable Side Effects Are Atomic¶

The state mutation and canonical audit append share one backend-native commit boundary. On failure or rollback, neither may remain visible. Transactional notifications may become visible only after the same commit.

TransactionStorage::with_transaction accepts one required EventContext and passes it to every transaction-scoped lifecycle mutation. A callback returning Err, a native operation failure, or a failed commit rolls back the complete unit of work, including its audit events.

The external sink call is intentionally not part of the domain transaction. After commit, EventFanoutStorage atomically claims the canonical event, matches subscriptions, creates durable delivery rows, and releases the claim. Delivery workers then use opaque claims and acknowledgements. Before transport, EventDeliveryWorkerStorage::begin_event_delivery must verify the token, in-flight status, and unexpired lease using the adapter's authoritative clock. It returns None for lost ownership or a scheduled rate deferral, and a StorageEventDeliveryLease for an admitted send; it must never renew an expired claim. Construct that lease with a monotonic timer started before acquiring storage resources and the remaining duration observed in storage, so query and dispatch delays cannot extend permission to send. Workers reserve the difference between lock and transport timeouts for acknowledgment and fence all result writes with the original claim. Delivery is at least once, may be unordered across events, and consumers deduplicate by event UUID. Worker notification is a latency optimization; durable polling remains the correctness path.

Sink admission is atomic across workers and subscriptions. Configured spacing and provider cooldowns defer work by clearing its claim and setting its next attempt time; they consume no failure attempts. A hot sink must not monopolize a claim batch. Permanent failures terminate the current delivery. System subscriptions require an event with no direct or related collection. Test requests validate sink and scope, append a real request audit, and queue a separate delivery purpose; tests never replace normal fan-out uniqueness.

4. Observation Is Application-Owned and Mandatory¶

The application supplies two complementary observers in production:

• StorageObserver receives bounded logical operation observations at the opaque storage handle; and
• an adapter-native observer, currently PostgresObserver, reports pool, transaction, query, failure, and other implementation-level signals.

Adapters and wrappers do not choose a global metrics registry or exporter. Production construction requires injected observers. Explicit unobserved or no-op constructors exist only for tests, benchmarks, and deliberate one-shot maintenance tools.

Capability, operation, backend, result, and other metric dimensions are static and bounded. IDs, names, queries, URLs, credentials, payloads, and error text must never become metric labels. Failed and rolled-back operations must still produce failure observation.

5. Portable Error Classification Is Consistent¶

Adapters must classify equivalent failures identically because the application uses the kind for HTTP status and retry behavior. This matrix is normative:

Native error text remains adapter-private. In particular, a pool checkout failure is Unavailable, while a failed query on an acquired connection is Backend. Corrupt persisted values are also Backend. Shared DTO validators return an unclassified StorageValidationError; applications map caller values to request errors, while adapters map rejected native projections to Backend.

6. Revision Conflicts Preserve the Current Revision¶

Optimistic-concurrency failures use StorageErrorKind::RevisionConflict and must carry the positive current ResourceRevision. Adapters must not discard, guess, or stringify this value while translating native errors. The application projects the same value to its API error so a caller can refresh or retry against an authoritative revision.

Selection Requires Behavioral Certification¶

hubuum-storage-conformance supplies the reusable BackendAuditFixture and verify_backend_audit_contract runner. For each StorageBackendKind::ALL entry, it verifies:

1. a committed receipt matches the durable event and its exact canonical AuditDocument;
2. a no-op returns no receipt and appends no event;
3. an injected failure persists neither state nor event;
4. the committed event creates durable fan-out and reaches a recording sink; and
5. logical, native-backend, and failure telemetry are observed; and
6. a stale precondition returns the exact current revision without persisting the attempted mutation.

The root compatibility registry additionally exercises every semantic capability group plus service, readiness, and authenticated HTTP behavior. Backend-native tests remain mandatory for isolation, lock behavior, cancellation, connection loss, claims, leases, recovery, migrations, and database-specific failure mechanics.

Only after these checks pass may the application implement its sealed CertifiedStorageBackend marker for the adapter. The marker records a reviewed certification decision; it is not a substitute for running the tests.

Maintenance Writes¶

Imports and restores implement the explicit ImportStorage and RestoreStorage capabilities. These workflows preserve or reconstruct durable state and history, so pretending each imported row is an ordinary user mutation would be incorrect.

Maintenance is not a general unaudited write API. It exposes only the typed import and restore state machines, including their validation, provenance, coordination, durable results, and rollback requirements. Ordinary request, service, worker, and fixture writes must use audited mutation APIs. Adding a new maintenance operation requires an explicit contract and review of how its history is preserved or recorded.

Backup and Restore Closure¶

BackupSnapshotStorage captures all logical sections from one consistent view. StorageBackupSnapshot::try_new validates section completeness, authoritative revisions, collection authorization coverage, live/open-history agreement, and event revision consistency. A history-inclusive capture must fail on missing or contradictory temporal state; capture must never manufacture history.

StorageRestoreDocument::at_restore_boundary prepares the complete logical replacement. With retained history, it preserves the supplied snapshots. Without history, it establishes one current system-attributed snapshot for each live temporal resource, retaining its ID, revision, data, and resource timestamps. Temporal validity starts at the supplied restore boundary, represented at microsecond precision. Its create operation marks entry into the new retained timeline. Task, audit, and delivery history remains empty until the restore-success event.

RestoreStorage::apply_restore must persist these prepared rows atomically with the state replacement, derived-state reset/rebuild scheduling, success event, and terminal receipt. Adapters map logical rows to their native representation; they do not reconstruct the history policy. Failure must expose none of the replacement state or provenance. A successful restore must permit another history-inclusive backup and restore, including after normal mutations.

Every selectable backend must pass the shared hubuum-storage-conformance::verify_backup_restore_contract runner, exercised by tests/restore_contract/mod.rs with both history modes and later mutations.

Event Retention and External Archives¶

Retention separates durable database coordination from external archival:

1. claim_event_retention_batch durably records one immutable batch ID and its exact event documents without deleting them;
2. EventArchiveSink::archive runs outside the database transaction and must be idempotent for that batch ID; and
3. complete_event_retention_batch deletes exactly the claimed events and records completion atomically.

The core execute_event_retention_batch helper owns this sequence. Adapters implement only claim and completion and cannot replace the archive-before-purge ordering.

An archive failure leaves the claim and events intact. Retrying must return the same batch ID and documents. Completion is idempotent, but it is valid only after the caller has durably archived that batch (or deliberately selected a discard archive). A count mismatch, malformed claimed document, concurrent maintenance state, or unavailable coordinator is an error; it must never be reported as a successful empty purge.

Transactions, Concurrency, and Cancellation¶

The application may compose existing safe resource primitives through the opaque StorageTransaction. It never receives the native transaction or connection. Hidden state machines, lock protocols, and consistency rules stay behind one operation-shaped backend method.

The adapter owns isolation, locking, uniqueness, optimistic revision checks, claim validation, and serialization when its native transaction cannot safely run operations concurrently. Snapshot reads must provide the consistency, visibility, filtering, counting, and paging semantics documented by their capability rather than combining unrelated observations.

Dropping a future requests cancellation. A backend must not report a successful commit after its callback returned an error. If native work can continue after cancellation, the adapter must preserve the externally visible atomicity contract and document and test that behavior.

Authorization¶

Storage supplies neutral facts and enforces local permission queries. The application selects and evaluates the configured authorization policy. Storage operations therefore accept neutral visibility, token scope, pre-authorized identifiers, or narrow bounded authorization callbacks. An adapter must not import a concrete external policy engine or silently replace it with local authorization.

Visibility is applied before filtering, counting, paging, aggregation, and audit projection wherever the owning capability requires it. Authorization inputs are enforced constraints, not hints.

Migrations and Schema Ownership¶

Migrations belong to the adapter that owns the schema. PostgreSQL migrations live in crates/hubuum-storage-postgres/migrations; neither hubuum-storage-core nor the root application owns them.

Diesel supports crate-local migrations. The PostgreSQL adapter embeds them with embed_migrations!("migrations"), resolved relative to that crate, while test and deployment tooling passes the same directory explicitly with --migration-dir. The adapter build script tracks the directory so embedded migration inputs rebuild correctly.

Schema changes must preserve the repository's adjacent-release compatibility policy and update generated schema, migration checks, change classification, container inputs, and native migration tests as required.

Packaging and Public API¶

hubuum-storage-core is the experimental public extension surface. It, its backend-neutral dependencies, and hubuum-storage-conformance form the exact-version storage adapter SDK. They must remain usable by an out-of-tree adapter without root or adapter-private access.

An external crate can consume the types, implement the traits, and use the published conformance harness. Method-specific query and collection behavior is normative and exhaustive. Neutral audit-document construction and validation by a second complete adapter remain follow-up work. See storage query semantics and the method contract registry.

hubuum-storage-postgres and the root hubuum crate are workspace-internal. The conformance crate is a public development dependency and must not enter production binaries. PostgreSQL schema, migrations, native clients, and telemetry types remain in the PostgreSQL adapter rather than leaking into hubuum-storage-core.

Performance Contract¶

Relation endpoint-set reads must enforce any supplied StorageRelationIdsQuery::max_results() in their native query before materializing rows. Apply the bound after endpoint and visibility selection, with ascending relation IDs; zero produces an empty result. An absent bound preserves complete results. Typed endpoint IDs do not inherit public integer-filter list limits. The shared adapter scenarios cover these bounds for both class and object relation reads.

The semantic guarantees do not prescribe SQL, but they must not conceal unbounded queries, repeated pool checkout, or accidental per-row work. Query-budget tests protect representative database shapes. Benchmarks measure the read boundary, audited mutation dispatch, and construction and consumption of committed mutation outcomes separately from database round trips.

Returning an audit receipt does not require a follow-up audit read. An adapter should derive it from the event write in the atomic mutation. Transactional composition reuses one native connection and unit of work rather than opening nested transactions for each constituent operation.

Change Rule¶

Any storage-boundary change must update, as applicable:

1. the owning trait and DTOs;
2. the StorageBackend aggregate and exhaustive application dispatch;
3. every selectable adapter;
4. semantic-coverage.toml, method-contracts.toml, and shared conformance or compatibility scenarios;
5. backend-native consistency and failure tests;
6. this contract and the affected capability or maintainer guide; and
7. changelog, OpenAPI, migrations, CI classification, and container inputs when their external contracts are affected.

If these artifacts disagree, the implementation is not complete.

Task lifecycle search¶

StorageTaskListQuery::searching attaches a validated StorageTaskSearch. Adapters must read search() before consuming the query with into_parts(); the latter keeps its existing signature. Apply every search predicate before counts, sorting, limits, and cursors, alongside submitter and excluded-kind restrictions. The PostgreSQL adapter uses the same predicates for rows and counts within its existing read snapshot.

Kinds and statuses are OR sets; different predicates combine with AND. TaskTimeRange uses inclusive lower and exclusive upper bounds, excludes null timestamps when bounded, and rejects reversed or empty intervals. Terminal selection includes all four terminal states, including partial success and cancellation. Trace identifiers are normalized to lowercase. Absent trace links and terminal reasons do not match their corresponding predicates.

Task discovery adds a storage SDK obligation: preserve the private-fielded, versioned StorageTaskMetadata on creation, completion, redaction and backup round trips. Validate persisted JSON once with from_persisted. Preserve unknown historical facts and update output summaries in the fenced finalization transaction. Artifact purge must not erase them.

Apply TaskDiscoverySearch predicates before counting and pagination, using its single captured evaluation timestamp. Project StorageTaskDiscoveryState from retained schema-work kind/status and artifact existence in bounded queries; never assemble schema reports during task polling. The PostgreSQL adapter requires migration 2026-09-18-000001_task_discovery.

Task authorization enrichment uses AuthorizationDataStorage::load_authorization_resources with a deduplicated StorageAuthorizationResourcesQuery. Adapters return lightweight StorageAuthorizationResource facts for all seven reference kinds, omit missing resources and relation endpoints, and bound queries by resource kind rather than page size. Preserve names and both relation endpoints for delegated policy checks; do not load template contents or remote transport/credential settings. Permission decisions remain with the configured authorization backend.

authorize_local_collection_batch returns one decision per input, in input order, using only the requested principals and collections. Preserve the single-check semantics: all requested permissions must occur in one group grant for that collection. Do not combine partial grants from different groups. Local permission batches use this operation for both endpoints of relations, including non-admin callers, without per-resource database calls.