Documents¶

Base Types¶

Every Den document embeds document.Base — the required anchor that carries ID, timestamps, and the revision token. The orthogonal features (soft delete, change tracking, file attachment) are available as separate composable embeds; combine whichever you need.

package document

type Base struct {
    ID        string    `json:"_id"`
    CreatedAt time.Time `json:"_created_at"`
    UpdatedAt time.Time `json:"_updated_at"`
    Rev       string    `json:"_rev,omitempty"` // populated when UseRevision is enabled
}

type SoftDelete struct {
    DeletedAt    *time.Time `json:"_deleted_at,omitempty"`
    DeletedBy    string     `json:"_deleted_by,omitempty"`     // populated by SoftDeleteBy()
    DeleteReason string     `json:"_delete_reason,omitempty"`  // populated by SoftDeleteReason()
}

type Tracked struct {
    snapshot []byte // not serialized
}

type Attachment struct {
    StoragePath string `json:"storage_path"     validate:"required,max=1024"`
    Mime        string `json:"mime"             validate:"required,max=100"`
    Size        int64  `json:"size"             validate:"required,min=1"`
    SHA256      string `json:"sha256,omitempty" validate:"omitempty,len=64"`
}

Typical compositions:

type Product struct {
    document.Base
    Name string `json:"name"`
}

type Article struct {
    document.Base
    document.SoftDelete
    Title string `json:"title"`
}

type User struct {
    document.Base
    document.Tracked
    Email string `json:"email"`
}

type AuditLog struct {
    document.Base
    document.SoftDelete
    document.Tracked
    Action string `json:"action"`
}

type Media struct {
    document.Base
    document.SoftDelete
    document.Attachment
    OriginalName string `json:"original_name"`
}

Den detects features structurally: soft-delete by the presence of the _deleted_at JSON field, change tracking by the Trackable interface, attachments by reflection over document.Attachment fields during hard-delete. Any type that carries the right fields / methods participates, even without these specific embeds.

Struct Tag Syntax¶

Document structs use two tags:

• json -- Sets the serialized field name (the key stored in JSONB). Standard Go encoding/json rules apply.
• den -- Carries Den-specific metadata. No field name, just options.

The den tag also appears on GroupBy().Into() and Project() target structs with a different set of values (count, sum:, from:, etc.). For the full inventory of every value across every context, see the den: reference table.

Available den tag options on document fields:

type Product struct {
    document.Base
    Name  string   `json:"name"  den:"index"`
    SKU   string   `json:"sku"   den:"unique"`
    Price float64  `json:"price" den:"index"`
    Body  string   `json:"body"  den:"fts"`
    Tags  []string `json:"tags"  den:"index,omitempty"`
}

Collection Naming¶

Den derives collection names automatically: lowercase struct name, no pluralization.

Override the default by implementing DenSettings():

func (p Product) DenSettings() den.Settings {
    return den.Settings{
        CollectionName: "products",
    }
}

ID Generation¶

Den uses ULID for document IDs -- lexicographically sortable, timestamp-ordered, 26 characters. The generator is in-tree and strictly monotonic within the same millisecond, so two inserts in the same ms keep their insertion order under Sort("_id", den.Asc) and cursor pagination.

p := &Product{Name: "Widget", Price: 9.99}
den.Save(ctx, db, p)
fmt.Println(p.ID) // "01HQ3K8V2X..."

ULIDs are preferred over UUIDs because they sort chronologically, which benefits B-tree indexes in both SQLite and PostgreSQL -- improving write and scan performance.

You can set an ID manually before insert. If ID is empty, Den generates a ULID automatically:

p := &Product{Name: "Widget"}
p.ID = "my-custom-id"
den.Save(ctx, db, p) // uses "my-custom-id"

For generating a ULID outside a save — pre-assigned document IDs, worker IDs, correlation IDs, deterministic test fixtures — call den.NewID() directly:

id := den.NewID() // "01HQ3K8V2X4XR1KPMM6N4G8J3P"

The document.Document marker¶

Every type that embeds document.Base automatically satisfies the unexported document.Document interface — Base contributes a marker method that propagates via Go's struct-embedding promotion. The marker has no runtime overhead and isn't something you implement directly; it exists so APIs that should only accept Den documents (e.g. validate.Document(doc)) can enforce that constraint at compile time. If you write helper functions that take "any Den document," declaring the parameter type as document.Document keeps the contract explicit:

func auditTouch(doc document.Document) {
    // doc is guaranteed to embed document.Base
}

Random structs without document.Base fail to compile — surfacing the contract violation at the call site instead of at runtime.

DenSettings Interface¶

Implement DenSettings() on your document struct to configure collection-level behavior:

type DenSettable interface {
    DenSettings() den.Settings
}

type Settings struct {
    CollectionName string            // override auto-derived name
    UseRevision    bool              // enable optimistic concurrency control
    Indexes        []IndexDefinition // compound indexes
}

Example with multiple settings:

func (p Product) DenSettings() den.Settings {
    return den.Settings{
        CollectionName: "products",
        UseRevision:    true,
        Indexes: []den.IndexDefinition{
            {Name: "idx_category_price", Fields: []string{"category", "price"}},
        },
    }
}

Index Definitions¶

Via Struct Tags¶

Single-field indexes are defined directly on the struct:

type Product struct {
    document.Base
    Name  string  `json:"name"  den:"index"`   // secondary index
    SKU   string  `json:"sku"   den:"unique"`  // unique index
    Price float64 `json:"price" den:"index"`   // secondary index
}

CREATE INDEX IF NOT EXISTS idx_product_name
    ON product(json_extract(data, '$.name'));

CREATE UNIQUE INDEX IF NOT EXISTS idx_product_sku
    ON product(json_extract(data, '$.sku'));

CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_product_name
    ON product((jsonb_extract_path_text(data, 'name')));

CREATE UNIQUE INDEX CONCURRENTLY IF NOT EXISTS idx_product_sku
    ON product((jsonb_extract_path_text(data, 'sku')))
    WHERE jsonb_extract_path_text(data, 'sku') IS NOT NULL;

Via Struct Tags (Compound Indexes)¶

For multi-field indexes, use unique_together or index_together with a shared group name. Fields with the same group name are combined into a single composite index:

type Entry struct {
    document.Base
    Feed string `json:"feed" den:"unique_together:feed_guid"`
    GUID string `json:"guid" den:"unique_together:feed_guid"`
    Body string `json:"body"`
}

This creates a composite unique index on (feed, guid) -- the combination must be unique, but individual values can repeat. The group name (feed_guid) becomes part of the index name: idx_entry_feed_guid.

For non-unique composite indexes, use index_together:

type Event struct {
    document.Base
    UserID string `json:"user_id" den:"index_together:user_date"`
    Date   string `json:"date"    den:"index_together:user_date"`
}

CREATE UNIQUE INDEX IF NOT EXISTS idx_entry_feed_guid
    ON entry(json_extract(data, '$.feed'), json_extract(data, '$.guid'))
    WHERE json_extract(data, '$.feed') IS NOT NULL
      AND json_extract(data, '$.guid') IS NOT NULL;

CREATE UNIQUE INDEX CONCURRENTLY IF NOT EXISTS idx_entry_feed_guid
    ON entry((jsonb_extract_path_text(data, 'feed')), (jsonb_extract_path_text(data, 'guid')))
    WHERE jsonb_extract_path_text(data, 'feed') IS NOT NULL
      AND jsonb_extract_path_text(data, 'guid') IS NOT NULL;

Nested Field Indexes¶

den: tags also work on fields of named-struct and pointer-to-struct fields at any nesting depth. The schema walker descends through Profile MyProfile / *Profile and emits dotted JSON paths into the generated SQL.

type Profile struct {
    Slug       string `json:"slug"       den:"unique"`
    Department string `json:"department" den:"index"`
    Bio        string `json:"bio"        den:"fts"`
}

type Account struct {
    document.Base
    Email   string  `json:"email"`
    Profile Profile `json:"profile"`
}

CREATE UNIQUE INDEX IF NOT EXISTS idx_account_profile_slug
    ON account(json_extract(data, '$.profile.slug'))
    WHERE json_extract(data, '$.profile.slug') IS NOT NULL;

CREATE INDEX IF NOT EXISTS idx_account_profile_department
    ON account(json_extract(data, '$.profile.department'));

CREATE UNIQUE INDEX CONCURRENTLY IF NOT EXISTS idx_account_profile_slug
    ON account((jsonb_extract_path_text(data, 'profile', 'slug')))
    WHERE jsonb_extract_path_text(data, 'profile', 'slug') IS NOT NULL;

CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_account_profile_department
    ON account((jsonb_extract_path_text(data, 'profile', 'department')));

Index names replace dots with underscores to stay valid SQL identifiers; the dotted form is preserved in the expression so the runtime lookup resolves the nested JSON value. unique_together / index_together groups can mix flat and nested fields in the same group.

Link[T] / []Link[T] fields and time.Time are treated as leaf scalars — the walker doesn't descend into them. Self-referential pointer types (Node { Child *Node }) are walked once at the top level and then short-circuit, so the schema stays bounded.

Pointer-to-struct fields behave the same as the value form. A nil parent pointer at insert time stores no value at the nested path; the unique index treats this as NULL (multiple nil parents don't collide).

Via Settings (Compound Indexes)¶

Alternatively, use DenSettings() for programmatic index definitions:

func (p Product) DenSettings() den.Settings {
    return den.Settings{
        Indexes: []den.IndexDefinition{
            {Name: "idx_category_price", Fields: []string{"category", "price"}},
            {Name: "idx_tenant_sku", Fields: []string{"tenant_id", "sku"}, Unique: true},
        },
    }
}

Index Creation Behavior¶

Dropping Stale Indexes¶

Register() is additive: it creates new indexes but never removes obsolete ones. When you remove a den:"index" or den:"unique" tag, or rename a field, the old index stays in the database.

To clean up, call den.DropStaleIndexes():

result, err := den.DropStaleIndexes(ctx, db)
if err != nil {
    return err
}
log.Printf("dropped %d stale indexes, kept %d", len(result.Dropped), len(result.Kept))

To preview what would be dropped without actually dropping anything, pass den.DryRun():

result, _ := den.DropStaleIndexes(ctx, db, den.DryRun())
for _, idx := range result.Dropped {
    log.Printf("would drop: %s.%s (fields=%v)", idx.Collection, idx.Name, idx.Fields)
}

Den tracks indexes it created in a private _den_indexes metadata table, so this operation only considers indexes Den knows about. Managed indexes (the PostgreSQL GIN index, FTS triggers and auxiliary tables, application-created indexes that Den did not create) are never touched.

Nullable Unique Constraints¶

When a pointer field is tagged with den:"unique", Den creates a partial unique index. Uniqueness is only enforced for non-nil values -- multiple documents can have nil for that field.

type User struct {
    document.Base
    Username string  `json:"username" den:"unique"`         // always required, always unique
    Email    *string `json:"email,omitempty" den:"unique"`  // optional, unique when set
}

// Both users have nil Email -- no conflict:
den.Save(ctx, db, &User{Username: "alice"})  // Email: nil
den.Save(ctx, db, &User{Username: "bob"})    // Email: nil

// But duplicate non-nil values are rejected:
den.Save(ctx, db, &User{Username: "carol", Email: ptr("carol@example.com")}) // ok
den.Save(ctx, db, &User{Username: "dave",  Email: ptr("carol@example.com")}) // ErrDuplicate

CREATE UNIQUE INDEX IF NOT EXISTS idx_user_email
    ON user(json_extract(data, '$.email'))
    WHERE json_extract(data, '$.email') IS NOT NULL;

CREATE UNIQUE INDEX IF NOT EXISTS idx_user_email
    ON "user"(((data->>'email')))
    WHERE (data->>'email') IS NOT NULL;