Migrations¶

When to Use Migrations¶

Documents in Den are schema-flexible: adding new fields with zero values works without any migration. Explicit migrations are needed for:

Field renames (e.g., name to title)
Type changes (e.g., string to int)
Data transformations (e.g., splitting a full name into first/last)

Running Migrations at Startup¶

Migrations do not run automatically. You are responsible for calling r.Up() at the right point in your application lifecycle — typically after opening the database and registering document types:

func main() {
    ctx := context.Background()

    db, err := den.OpenURL(ctx, "sqlite:///data.db")
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    // Register document types first
    if err := den.Register(ctx, db, &Note{}, &User{}); err != nil {
        log.Fatal(err)
    }

    // Run pending migrations
    r := setupMigrations()
    if err := r.Up(ctx, db); err != nil {
        log.Fatal(err)
    }

    // Application is ready
}

Note

r.Up() is idempotent — it skips migrations that have already been applied. It is safe to call on every startup.

Using Burrow?

If you are using Den through Burrow, migrations are handled automatically by the framework. See the Burrow Migrations Guide for details.

Defining Migrations¶

Migrations are Go functions registered with a version identifier:

import "github.com/oliverandrich/den/migrate"

func setupMigrations() *migrate.Registry {
    r := migrate.NewRegistry()

    r.Register("20250402_001_rename_name_to_title", migrate.Migration{
        Forward: func(ctx context.Context, tx *den.Tx) error {
            for note, err := range den.NewQuery[OldNote](tx).Iter(ctx) {
                if err != nil {
                    return err
                }
                note.Title = note.Name
                if err := den.Save(ctx, tx, note); err != nil {
                    return err
                }
            }
            return nil
        },
        Backward: func(ctx context.Context, tx *den.Tx) error {
            for note, err := range den.NewQuery[Note](tx).Iter(ctx) {
                if err != nil {
                    return err
                }
                note.Name = note.Title
                if err := den.Save(ctx, tx, note); err != nil {
                    return err
                }
            }
            return nil
        },
    })

    return r
}

Tip

Use a timestamp-based naming convention like YYYYMMDD_NNN_description for migration versions. This ensures consistent ordering across environments.

Running Migrations¶

r := setupMigrations()

// Run all pending forward migrations
err := r.Up(ctx, db)

// Run one forward migration
err := r.UpOne(ctx, db)

// Roll back one migration
err := r.DownOne(ctx, db)

// Roll back all migrations
err := r.Down(ctx, db)

Migration Tracking¶

Den stores migration state in a single document under the _den_migrations collection (key "log"), holding a JSON array of {Version, AppliedAt} entries. r.Up() loads this log, compares it against the registered migrations, and runs the missing ones in order.

Transaction Safety¶

Each migration runs within a Den transaction alongside the log update:

If the migration function returns nil, the entry is appended to the log and the whole transaction commits.
If it returns an error, the transaction rolls back — the log is unchanged, so the migration stays pending and the next r.Up() will retry it from scratch.
r.Up() stops at the first failure and returns; any remaining pending migrations stay pending until the failing one succeeds.

Observability¶

The Registry emits structured slog events for every migration lifecycle step. By default they go through slog.Default(); pass migrate.WithLogger(...) to route them through a dedicated logger:

logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))
r := migrate.NewRegistry(migrate.WithLogger(logger))

Events:

Message	Level	Attributes
`migration_start`	`INFO`	`version`, `direction` (`up` or `down`)
`migration_success`	`INFO`	`version`, `direction`, `duration_ms`
`migration_failure`	`ERROR`	`version`, `direction`, `duration_ms`, `error`
`ensure_table_failure`	`ERROR`	`error` — fires at most once per Registry (sticky)

The error field is attached as a string for structured-log pipelines. The original error value still bubbles up the call chain — logging it here does not swallow it.

Note

Passing a nil logger via WithLogger falls back to slog.Default() — keeps configs honest when a logger field is optional.

Warning

If a migration fails, fix the issue and re-run r.Up(). Den will retry only the failed and pending migrations.

Streaming with Iter()¶

For migrations that touch many documents, use Iter() to stream documents without loading them all into memory:

r.Register("20250410_001_normalize_prices", migrate.Migration{
    Forward: func(ctx context.Context, tx *den.Tx) error {
        for product, err := range den.NewQuery[Product](tx).Iter(ctx) {
            if err != nil {
                return err
            }
            product.Price = math.Round(product.Price*100) / 100
            if err := den.Save(ctx, tx, product); err != nil {
                return err
            }
        }
        return nil
    },
})

Error Handling¶

Migration errors are wrapped with den.ErrMigrationFailed:

err := r.Up(ctx, db)
if errors.Is(err, den.ErrMigrationFailed) {
    log.Fatal("Migration failed:", err)
}