Migration API

Database migration framework for large-scale Firestore transformations.

What It Does

The Migration API provides a sophisticated framework for executing large-scale database migrations, particularly for Firestore collections. It enables memory-efficient data transformations using pagination, event-driven architecture, and batch operations.

Key Capabilities

Paginated Migrations: Process large datasets in chunks
Memory Efficient: Configurable page sizes prevent memory overflow
Batch Operations: Efficient Firestore batching
Execution Control: Selective migration runs with ordering
Progress Tracking: Monitor pages and items processed
Flexible Transformations: Custom transformation functions
Context Management: Global and per-page context

Main Components

SimpleCollectionMigration

Simplified pattern requiring only execute() method:


class MyMigration extends SimpleCollectionMigration<T> {
  options = {collection: 'users', migrationId: 'upgrade-users'}
  async execute(doc: T) {
    return doc.version < 2  // true to migrate
  }
}

MigrationRunner

High-level runner for executing migrations:


new MigrationRunner(migrations, globalContextItems).run(pageSize, projectId, onlyIds)

FirestoreCollectionMigration

Core migration class implementing ETL pattern:

fetchPage(): Paginated retrieval
transformPage(): Custom transformations
writePage(): Batch writes

Common Use Cases

Database schema upgrades
Data format migrations
Large-scale data transformations
Multi-collection updates
Firestore to SQL migrations

What Customers Don’t Have to Build

Pagination for large datasets
Batch write operations
Migration orchestration
Progress tracking
Context management
Execution ordering
Memory optimization