Skip to content

API Reference

Complete reference of all public functions in the den package, organized by category.

Module: github.com/oliverandrich/den


Database

Function Signature Description
Open Open(ctx context.Context, b Backend, opts ...Option) (*DB, error) Open a database around an existing Backend. The context governs any setup work triggered by options (for example WithTypes)
OpenURL OpenURL(ctx context.Context, dsn string, opts ...Option) (*DB, error) Open a database using a URL-style DSN (requires backend import). The context governs connection dialing and any setup work triggered by options
Register Register(ctx context.Context, db *DB, docs ...document.Document) error Register document types; creates collections and indexes. The document.Document parameter is a sealed marker — only types embedding document.Base satisfy it, so passing a non-document value is a compile error
WithTypes WithTypes(docs ...document.Document) Option Open/OpenURL option: register document types at open time. Equivalent to calling Register(ctx, db, docs...) immediately after Open, but composes as a single expression. Registration errors abort Open and are returned as its error
db.Close (db *DB) Close() error Close the database connection
db.Ping (db *DB) Ping(ctx context.Context) error Healthcheck; delegates to backend
NewID NewID() string Generate a fresh 26-character ULID. Save calls this automatically for empty-ID docs; use it directly for pre-assigned document IDs, worker IDs, correlation IDs, or deterministic test fixtures

CRUD

Every CRUD function below takes a Scope parameter. Scope is a sealed interface satisfied by both *DB (operating outside a transaction) and *Tx (operating inside RunInTransaction). Pass whichever you have.

Save

Function Signature Description
Save[T] Save[T](ctx context.Context, s Scope, doc *T, opts ...CRUDOption) error Persist a single document. Empty-ID docs take the insert path (ULID assigned); ID-bearing docs take the update path. Hooks fire on whichever branch runs
SaveAll[T] SaveAll[T](ctx context.Context, s Scope, docs []*T, opts ...CRUDOption) error Persist every doc in the slice in a single transaction. Mixed batches (some empty-ID, some not) are supported. Fail-fast: any per-doc error rolls back the transaction
Replace[T] Replace[T](ctx context.Context, s Scope, fresh *T, opts ...CRUDOption) error Full-content replace (PUT): fresh's client fields overwrite the stored row (omitted fields reset to zero) while Den's server-owned fields (_id, _created_at, _rev, soft-delete audit) are preserved from the existing record. Last-writer-wins; does not resurrect soft-deleted rows. ErrNotFound if the ID is absent, ErrValidation if fresh has none
PreserveServerFields[T] PreserveServerFields[T](db *DB, dst, src *T) error Copy Den's server-owned fields from src onto dst — the building block behind Replace. ErrNotRegistered if T is unregistered

Read

Function Signature Description
FindByID[T] FindByID[T](ctx context.Context, s Scope, id string, opts ...CRUDOption) (*T, error) Find a document by its ID (direct key lookup). Bypasses the soft-delete filter — explicit-by-ID lookups always return the row if present. Callers can check Value.IsDeleted()
FindByIDs[T] FindByIDs[T](ctx context.Context, s Scope, ids []string, opts ...CRUDOption) ([]*T, error) Find multiple documents by their IDs. Same soft-delete contract as FindByID
Refresh[T] Refresh[T](ctx context.Context, s Scope, doc *T, opts ...CRUDOption) error Re-read the document from storage, replacing all field values
RefreshAll[T] RefreshAll[T](ctx context.Context, s Scope, docs []*T, opts ...CRUDOption) error Re-read every doc in the slice from storage in one transaction. Fail-fast

Delete

Function Signature Description
Delete[T] Delete[T](ctx context.Context, s Scope, doc *T, opts ...CRUDOption) error Delete a document. Soft-deletes if the document embeds SoftDelete
DeleteAll[T] DeleteAll[T](ctx context.Context, s Scope, docs []*T, opts ...CRUDOption) error Delete every doc in the slice in one transaction. Fail-fast
HardDelete HardDelete() CRUDOption CRUDOption for Delete that permanently removes a soft-deleteable document
IncludeDeleted IncludeDeleted() CRUDOption CRUDOption that makes by-ID lookups (FindByID, FindByIDs) consider soft-deleted documents. Mirrors QuerySet.IncludeDeleted()
SoftDeleteBy SoftDeleteBy(actor string) CRUDOption CRUDOption for Delete that records an actor on the document's DeletedBy field. No-op on HardDelete() and on types that don't embed SoftDelete
SoftDeleteReason SoftDeleteReason(reason string) CRUDOption CRUDOption for Delete that records a free-form reason on the document's DeleteReason field. No-op on HardDelete() and on types that don't embed SoftDelete

By-condition writes (find-and-update, find-or-create, bulk update, bulk delete, back-links) live on QuerySet — see the Query section below. There is no top-level FindOneAndUpdate / FindOneAndUpsert / FindOrCreate / UpdateMany / DeleteMany / BackLinks anymore: build a chain with NewQuery[T](db, conds...) and call the matching terminal (UpdateOne, UpsertOne, GetOrCreate, Update, Delete, BackLinks).


Query

Creating a Query

q := den.NewQuery[T](scope, conditions...) // scope is *DB or *Tx
Function Signature Description
NewQuery[T] NewQuery[T](scope Scope, conditions ...where.Condition) QuerySet[T] Create a new chainable query for type T. Scope is *DB (outside a transaction) or *Tx (inside one). The context is supplied later by the terminal method, so one QuerySet can be reused across contexts

Chainable Methods

All chainable methods return QuerySet[T] and can be composed in any order.

Method Signature Description
Where Where(conditions ...where.Condition) QuerySet[T] Add additional filter conditions
Sort Sort(field string, dir SortDirection) QuerySet[T] Sort results by field (den.Asc or den.Desc)
Limit Limit(n int) QuerySet[T] Limit the number of results
Skip Skip(n int) QuerySet[T] Skip the first n results (offset-based pagination)
After After(id string) QuerySet[T] Cursor-based pagination: fetch results after this ID
Before Before(id string) QuerySet[T] Cursor-based pagination: fetch results before this ID
WithFetchLinks WithFetchLinks(fields ...string) QuerySet[T] Eagerly resolve Link[T] fields on results. No arguments resolves every link field; passing JSON field names resolves only those (selective ?expand=-style hydration)
WithNestingDepth WithNestingDepth(n int) QuerySet[T] Override max link-fetching depth for this query
IncludeDeleted IncludeDeleted() QuerySet[T] Include soft-deleted documents in results
ForUpdate ForUpdate(opts ...LockOption) QuerySet[T] Acquire row-level locks on every matching row. Requires *Tx scope — terminal methods return ErrLockRequiresTransaction otherwise

Terminal Methods

Terminal methods execute the query and return results.

Every terminal takes ctx context.Context as its first argument, so the same QuerySet can be executed against different contexts (different timeouts, different cancellation scopes).

Method Signature Description
All All(ctx context.Context) ([]*T, error) Execute query, return all matching documents
First First(ctx context.Context) (*T, error) Execute query, return the first matching document. Returns ErrNotFound if nothing matches
Count Count(ctx context.Context) (int64, error) Count matching documents
Exists Exists(ctx context.Context) (bool, error) Check whether at least one matching document exists
AllWithCount AllWithCount(ctx context.Context) ([]*T, int64, error) Return matching documents and total count (for pagination)
Iter Iter(ctx context.Context) iter.Seq2[*T, error] Return a lazy iterator for streaming results with range. Terminates on the first error
Update Update(ctx context.Context, fields SetFields) (int64, error) Bulk update every matching document and return the count. Fail-fast: any per-row error rolls back the transaction and returns (0, err). Field names are validated before the tx opens
UpdateOne UpdateOne(ctx context.Context, fields SetFields) (*T, error) Atomic find-and-modify on a single matching row. Returns ErrNotFound on miss and ErrMultipleMatches if more than one row matches
UpsertOne UpsertOne(ctx context.Context, defaults *T, fields SetFields) (*T, bool, error) Find-or-create-then-update. defaults is used only on miss; fields is applied on both paths. Returns (doc, inserted, err)
GetOrCreate GetOrCreate(ctx context.Context, defaults *T) (*T, bool, error) Find-or-create with no post-find update. Equivalent to UpsertOne(ctx, defaults, SetFields{})
Delete Delete(ctx context.Context, opts ...CRUDOption) (int64, error) Delete every matching row and return the count. Drains the iterator before issuing per-row writes so cursor pinning on PostgreSQL is safe. Soft-delete routing and HardDelete apply per row
BackLinks BackLinks(linkField string, targetID string) QuerySet[T] Chainable: narrow the QuerySet to documents that reference targetID via the named link field. Compose with WithFetchLinks / IncludeDeleted / Sort / Limit etc. and dispatch with any terminal
Search Search(ctx context.Context, term string) ([]*T, error) Literal-terms full-text search (FTS5 on SQLite, tsvector on PostgreSQL): the term is treated as plain words ANDed together with FTS5 operators/punctuation neutralised, so raw user input is safe on both backends. Blank term returns no rows. ErrFTSNotSupported when the backend lacks FTS
SearchRaw SearchRaw(ctx context.Context, term string) ([]*T, error) Like Search but passes the term to the backend's native FTS mechanism (FTS5 query syntax on SQLite; plainto_tsquery on PostgreSQL). Raw user input is unsafe on SQLite — use Search for that

Note: den.LiteralFTS5(term string) string exposes the literal-terms transform Search applies — use it to build a safe FTS5 string by hand for composing with SearchRaw.


Aggregation

Aggregation methods are chained onto a QuerySet[T].

Scalar Aggregations

Method Signature Description
Avg Avg(ctx context.Context, field string) (float64, error) Average of a numeric field across matching documents
Sum Sum(ctx context.Context, field string) (float64, error) Sum of a numeric field across matching documents
Min Min(ctx context.Context, field string) (float64, error) Minimum value of a field across matching documents
Max Max(ctx context.Context, field string) (float64, error) Maximum value of a field across matching documents

Grouped Aggregations

Method Signature Description
GroupBy GroupBy(fields ...string) GroupByBuilder[T] Group results by one or more fields
Into Into(ctx context.Context, dest any) error Execute grouped aggregation into a target slice of structs
Project Project(ctx context.Context, dest any) error Project query results into a struct with a subset of fields
// GroupBy example
type Stats struct {
    Category string  `den:"group_key"`
    AvgPrice float64 `den:"avg:price"`
    Count    int64   `den:"count"`
}

err := den.NewQuery[Product](db).GroupBy("category.name").Into(ctx, &results)

Relations

Function Signature Description
NewLink[T] NewLink[T any](doc *T) Link[T] Create a Link from an existing document, extracting its ID
FetchLink[T] FetchLink[T](ctx context.Context, s Scope, doc *T, field string) error Fetch and resolve a single link field on a document
FetchAllLinks[T] FetchAllLinks[T](ctx context.Context, s Scope, doc *T) error Fetch and resolve all link fields on a document
WithLinkRule WithLinkRule(rule LinkRule) CRUDOption Set cascade behavior for insert/update/delete of linked documents
LinkFields[T] LinkFields[T](db *DB) ([]LinkFieldMeta, error) Enumerate a type's Link[T] / []Link[T] relation fields (JSON name, target collection, single-vs-slice, eager) — e.g. to allowlist expandable relations. ErrNotRegistered if T is unregistered
Marshal Marshal(v any) ([]byte, error) Output JSON marshaller: emits hydrated (Loaded) links as their nested object anywhere in the value graph; unloaded links and json.Marshal stay the bare id (storage format unchanged). Pair with WithFetchLinks for ?expand=-style responses

Reverse queries live on QuerySet: den.NewQuery[House](db).BackLinks("door", doorID).All(ctx) — see the Terminal Methods table.

Rule Value Description
LinkIgnore 0 No cascading -- only the root document is written/deleted
LinkWrite 1 Cascade writes to all linked documents (insert new, update existing)
LinkDelete 2 Cascade deletion to all linked documents

Change Tracking

Requires embedding document.Tracked alongside document.Base.

Function Signature Description
IsChanged[T] IsChanged[T](db *DB, doc *T) (bool, error) Check whether the document has been modified since last load/save
GetChanges[T] GetChanges[T](db *DB, doc *T) (map[string]FieldChange, error) Get a map of changed fields with before/after values
Revert Revert[T](db *DB, doc *T) error Restore the document to its last-saved state by decoding the stored snapshot over its fields. Returns ErrNoSnapshot if the document was never loaded or does not embed Tracked. Named Revert (not Rollback) to avoid name collision with the backend transaction's Rollback method

Transactions

RunInTransaction opens a transaction; the closure receives a *Tx. CRUD functions take a Scope (satisfied by *DB and *Tx), so the same Save/Delete/FindByID etc. work both inside and outside a transaction — pass the *Tx instead of the *DB. The APIs listed below are the transaction-only ones: they take *Tx directly because their semantics are tied to transaction lifetime.

Function Signature Description
RunInTransaction RunInTransaction(ctx context.Context, db *DB, fn func(tx *Tx) error) error Execute a function within a transaction. Commits on nil return, rolls back on error
LockByID[T] LockByID[T](ctx context.Context, tx *Tx, id string, opts ...LockOption) (*T, error) Find a document by ID and acquire a row-level lock (SELECT ... FOR UPDATE on PostgreSQL; no-op on SQLite). Held until the transaction commits or rolls back. Optional SkipLocked() / NoWait() modifiers
SkipLocked SkipLocked() LockOption LockByID and QuerySet.ForUpdate modifier: return ErrNotFound (or skip locked rows in multi-row queries) instead of blocking. PostgreSQL FOR UPDATE SKIP LOCKED. Queue-consumer primitive
NoWait NoWait() LockOption LockByID and QuerySet.ForUpdate modifier: return ErrLocked immediately if another transaction holds any row. PostgreSQL FOR UPDATE NOWAIT
QuerySet[T].ForUpdate ForUpdate(opts ...LockOption) QuerySet[T] Acquires a row-level lock on every matching row in one statement. Only valid when the QuerySet is bound to a *Tx; terminal methods return ErrLockRequiresTransaction if the scope is a *DB
AdvisoryLock AdvisoryLock(ctx context.Context, tx *Tx, key int64) error Acquire an application-level lock held until the transaction commits or rolls back. PostgreSQL pg_advisory_xact_lock; SQLite no-op
(*Tx).Transaction (t *Tx) Transaction() Transaction Low-level accessor that returns the underlying backend Transaction. Only for infrastructure code (e.g. the migration log) that needs to bypass the registry, encoding, and hooks. Application code should use Save / Delete / FindByID / NewQuery

Note: Standard CRUD operations (Save, Delete, FindByID, …) accept a Scope parameter; pass *DB outside a transaction and *Tx inside.


Metadata

Function Signature Description
Meta[T] Meta[T](db *DB) (CollectionMeta, error) Get metadata for a registered collection (fields, indexes, links, settings)
Collections Collections(db *DB) []string List all registered collection names

Attachments & Storage

Types and functions for embedding file references in documents and swapping the byte-storage backend. See the Attachments & Storage guide for the full walkthrough.

Option and Accessor

Function Signature Description
WithStorage WithStorage(s Storage) Option Open/OpenURL option that installs a Storage on the DB. Required for the hard-delete attachment cascade to actually drop bytes
db.Storage (db *DB) Storage() Storage Accessor for the configured Storage, or nil if none was installed

Storage Interface

type Storage interface {
    Store(ctx context.Context, r io.Reader, ext, mime string) (document.Attachment, error)
    Open(ctx context.Context, a document.Attachment) (io.ReadCloser, error)
    Delete(ctx context.Context, a document.Attachment) error
    URL(a document.Attachment) string
}

Implementations must be content-addressed enough that two calls with identical bytes resolve to the same StoragePath. Delete must be idempotent on missing paths.

Storage Registry

Located in github.com/oliverandrich/den/storage. The root package holds the interface + a scheme-based opener registry; concrete backends live in sub-packages that self-register on import.

Function Signature Description
OpenURL OpenURL(dsn string) (storage.Storage, error) Parses <scheme>://<location> and delegates to the opener registered for the scheme. The opener receives the full location verbatim, including any query string. Returns a clear error when the scheme is unknown (usually missing a side-effect import of the backend sub-package)
Register Register(scheme string, opener OpenerFunc) Registers an opener for a scheme. Typically called from a backend sub-package's init(). Panics on duplicate registration
OpenerFunc type OpenerFunc func(location string) (storage.Storage, error) Factory signature for backend openers. Receives the full location (everything after ://); backends parse their own query parameters
URLPrefixFromLocation URLPrefixFromLocation(location string) (stripped, prefix string) Helper for backends that honour the conventional ?url_prefix= query param. Returns the location with that param stripped plus the extracted prefix. Backends that ignore url_prefix (S3) can skip the call; URL-prefix-aware backends (file) call it before parsing the rest of their location
ErrEmptyContent var ErrEmptyContent error Returned by Storage.Store on a zero-byte reader

Filesystem Backend (storage/file)

Located in github.com/oliverandrich/den/storage/file. Reference backend that stores bytes on the local filesystem. Importing the package for its side effect registers the file:// scheme with storage.OpenURL.

Function Signature Description
New New(rootPath, urlPrefix string) (*Storage, error) Constructs a filesystem-backed storage.Storage (also aliased as den.Storage). Content-addresses paths to YYYY/MM/<sha256-prefix>.<ext>; uses os.Root to refuse path traversal
fs.Close (fs *Storage) Close() error Release the underlying file descriptor held for the storage root
fs.URLPrefix (fs *Storage) URLPrefix() string Returns the HTTP path prefix the storage serves its files under. HTTP-layer packages type-assert on a local interface{ URLPrefix() string } to decide whether to register a serving handler; remote backends (S3/GCS) deliberately do not implement this

Attachment Document Embed

Located in the document sub-package (github.com/oliverandrich/den/document).

type Attachment struct {
    StoragePath string `json:"storage_path"     validate:"required,max=1024"`
    Mime        string `json:"mime"             validate:"required,max=100"`
    Size        int64  `json:"size"             validate:"required,min=1"`
    SHA256      string `json:"sha256,omitempty" validate:"omitempty,len=64"`
}
Method Signature Description
IsZero (a Attachment) IsZero() bool Reports whether the attachment is empty (no StoragePath and no Size)

Embed alongside document.Base for IS-a-file documents, or declare as named fields for HAS-files documents. den.Delete(..., den.HardDelete()) walks the document via reflection and asks the configured Storage to delete every non-zero Attachment it finds.


Index Lifecycle

Function Signature Description
DropStaleIndexes DropStaleIndexes(ctx context.Context, db *DB, opts ...DropStaleOption) (DropStaleResult, error) Drop indexes previously created by Register() that no longer correspond to any IndexDefinition. Managed indexes (GIN, FTS) are never touched
DryRun DryRun() DropStaleOption Option for DropStaleIndexes; reports the plan without mutating the database

DropStaleResult contains two slices:

  • Dropped []StaleIndex — indexes that were (or would be, under DryRun) removed
  • Kept []StaleIndex — recorded indexes that are still referenced by a current IndexDefinition

StaleIndex has fields Collection, Name, Fields []string, Unique bool.


Migrations

Located in the migrate sub-package (github.com/oliverandrich/den/migrate).

Function Signature Description
NewRegistry NewRegistry(opts ...Option) *Registry Create a new migration registry. Pass migrate.WithLogger(l) to receive structured progress events
WithLogger WithLogger(l *slog.Logger) Option Registry option that emits per-migration slog events (start, success, failure)
Register (r *Registry) Register(version string, m Migration) Register a migration with a version string
Up (r *Registry) Up(ctx context.Context, db *den.DB) error Run all pending forward migrations
UpOne (r *Registry) UpOne(ctx context.Context, db *den.DB) error Run one forward migration
Down (r *Registry) Down(ctx context.Context, db *den.DB) error Roll back all migrations
DownOne (r *Registry) DownOne(ctx context.Context, db *den.DB) error Roll back one migration

Testing Helpers

Located in the dentest sub-package (github.com/oliverandrich/den/dentest).

Function Signature Description
MustOpen MustOpen(t testing.TB, docs ...document.Document) *den.DB Open a file-backed SQLite database in a temp directory; auto-registers docs and cleans up after test
MustOpenPostgres MustOpenPostgres(t testing.TB, connStr string, docs ...document.Document) *den.DB Open a PostgreSQL database for testing; auto-registers docs

Key Types

Type Description
DB Database handle; holds the backend and collection registry. Satisfies Scope
Tx Transaction handle; wraps a backend transaction. Satisfies Scope
Scope Sealed interface satisfied by *DB and *Tx. Parameter type for all CRUD entry points so the same function works inside and outside a transaction
Link[T] Generic reference to a document in another collection; stores ID, optionally holds resolved Value
SetFields map[string]any used for partial updates via QuerySet.UpdateOne, UpsertOne, and bulk Update. Field names are validated against the registered struct before the tx opens
Settings Document-level settings (collection name, revision, nesting depth, indexes)
QuerySet[T] Chainable, lazy query builder
SortDirection Sort direction: den.Asc or den.Desc
LinkRule Cascade behavior for link operations
Option Functional option for Open/OpenURL
CRUDOption Functional option for write operations (e.g., WithLinkRule)
FieldChange Represents a changed field with Before and After values
CollectionMeta Metadata about a collection: fields, indexes, links, settings
LinkFieldMeta Describes one Link[T] relation field: JSON name, Go name, slice/eager flags, target collection and type. Returned by LinkFields
IndexDefinition Index specification: name, fields, unique flag