Skip to Content
Engineering11 Documentation 🔥
BackendServer API PackagesDatastores

Datastores API

Multi-database ORM with unified interface for Firestore, SQL, BigQuery, Redis, Algolia, and Typesense.

What It Does

The Datastores API provides a comprehensive, type-safe repository pattern that abstracts database operations across multiple persistence backends. It enables applications to seamlessly switch between Firestore and SQL databases while maintaining a consistent interface, and provides built-in support for search engines, caching, and analytics datastores.

Key Capabilities

CapabilityDescription
Multi-Database SupportUnified interface for Firestore, PostgreSQL, MySQL, SQL Server, SQLite, BigQuery
Repository PatternType-safe CRUD operations with generic typing throughout
Search IntegrationBuilt-in Algolia and Typesense search engine support
Redis CachingFull-featured Redis client with type conversions
Real-time SubscriptionsRxJS observables for live data updates (Firestore)
Transaction ManagementSimplified transactions for both SQL and Firestore
Query BuilderFluent query API with type-safe filtering and ordering
Pagination SupportCursor-based pagination for large datasets
Batch OperationsEfficient bulk writes with automatic batching
Data SerializationAutomatic date conversion and field value handling

Configuration

Datastore Selection

  • Environment Variable: DEFAULT_DATASTORE='firestore'|'sql'
  • TOML Config: /config/config.toml
  • Default: Firestore

Connection Configuration

  • Firestore: Via Firebase Admin SDK
  • SQL: Knex configuration (connection string, pool size)
  • BigQuery: GCP project and credentials
  • Redis: Host, port, key prefix
  • Algolia: Application ID and API key
  • Typesense: Host and API key

Exports

Repository Classes

  • BaseFirestoreRepository<T>
  • CollectionRepository<T>
  • SubcollectionRepository<T, PathIdFn>
  • BaseSqlRepository<T>
  • BaseBigQueryRepository<T>

Service Classes

  • FirestoreService, FirestoreQueryBuilder
  • SqlExecutor<T>
  • RedisRepository
  • AlgoliaIndex<T>, AlgoliaSearchService
  • TypesenseIndex<T>, TypesenseSearchService
  • BatchService, FirestorePaginationService

Utilities

  • Ops - Query operators
  • FieldValue - Special operations
  • Mapper - Object transformations
  • registerDatastore() - DI utilities
  • registerMultiService()

Types and Interfaces

  • IRepository<T> - Repository interface
  • PaginationParams, PaginationResponse
  • Query types and operators
  • Error types

Common Use Cases

  • Multi-tenant SaaS: Switch between Firestore and SQL per customer
  • Real-time applications: Live data subscriptions with Firestore
  • Analytics platforms: Stream data to BigQuery for reporting
  • Search features: Full-text search with Algolia or Typesense
  • Caching layers: Redis for session and application caching
  • Data migrations: Stream large datasets between datastores
  • Audit trails: Track all database operations in BigQuery
  • Batch processing: Efficient bulk writes with automatic batching

What Customers Don’t Have to Build

  • Multi-database repository pattern
  • Type-safe CRUD operations
  • Query builder with operators
  • Real-time subscription management
  • Transaction coordination
  • Batch operation logic
  • Pagination with cursors
  • Redis type conversions
  • Search engine integrations
  • BigQuery streaming infrastructure
  • Object serialization
  • Connection pooling
  • Error handling abstractions
  • Datastore switching logic
  • Dependency injection for databases
  • Field value operations
  • Streaming for large datasets
  • Object transformation pipelines
Last updated on
ValidationModels