Sync Service

┌─────────────────────────────────────────────────────┐
│              TaskService Entry Points               │
│  requestCollectionPatch, requestDocumentPatch,      │
│  requestCollectionDeletion, requestDocumentDeletion │
└───────────────────┬─────────────────────────────────┘
                    │
                    ▼
         ┌──────────────────────┐
         │   Cloud Task Queue   │
         │  (Reliable delivery) │
         └──────────┬───────────┘
                    │
                    ▼
    ┌───────────────────────────────────┐
    │  SyncCollectionPatchController    │
    │    (NestJS Task Processors)       │
    └───────────┬───────────────────────┘
                │
    ┌───────────┴──────────────────┐
    │                              │
    ▼                              ▼
┌─────────────────┐    ┌──────────────────────┐
│PatcherTask      │    │DeletionTask          │
│Generator        │    │Generator             │
│                 │    │                      │
│- Streams pages  │    │- Streams pages       │
│- Creates tasks  │    │- Creates delete tasks│
└─────────────────┘    └──────────────────────┘
    │                              │
    └──────────┬───────────────────┘
               │
               ▼
    ┌──────────────────────┐
    │  Firestore Updates   │
    │   (Atomic writes)    │
    └──────────────────────┘

class TaskService {
  /**
   * Update all documents in a collection
   * WARNING: Documents must have 'id' field for pagination
   */
  requestCollectionPatch(request: {
    collectionId: string; // Collection path (leading/trailing slashes stripped)
    values: unknown; // Partial document update (merged with existing)
    collectionGroup?: boolean; // Query across all subcollections with this name
  }): Promise<void>;
 
  /**
   * Update a single document
   * For slow-changing data and high-volume updates
   */
  requestDocumentPatch(request: {
    path: string; // Full document path
    values: unknown; // Partial document update (merged with existing)
  }): Promise<void>;
 
  /**
   * Delete all documents in a collection
   * Creates individual delete tasks for each document
   */
  requestCollectionDeletion(request: {
    collectionPath: string; // Collection path
  }): Promise<void>;
 
  /**
   * Delete a single document
   */
  requestDocumentDeletion(request: {
    path: string; // Full document path
  }): Promise<void>;
}

BaseFirestoreCollectionStreamer features:
  - Automatic cursor-based pagination
  - Configurable page size (default: 100 documents)
  - Orders by document 'id' field for consistent pagination
  - Processes pages sequentially to avoid memory issues
  - Supports both regular collections and collection groups
 
Processing flow:
  1. Fetch page of documents
  2. Process each document in page
  3. Move cursor to next page
  4. Repeat until no more documents

Collection Group features:
  - Query all subcollections with the same name
  - Example: Update all 'members' subcollections across communities
  - Strips leading/trailing slashes from collection paths
  - Removes all slashes for collection group queries
  - Processes documents regardless of parent hierarchy
 
Use case:
  Collection: communities/{id}/members
  With collectionGroup=true: updates ALL members in ALL communities
  With collectionGroup=false: updates members in specific community

Phase 1 - Collection Task:
  - Receives collection patch/delete request
  - Streams through collection in pages
  - Creates individual document task for each document
  - Each document task is independent
 
Phase 2 - Document Tasks:
  - Process individual documents atomically
  - Can retry independently if failed
  - No cross-document dependencies
  - Guaranteed eventual completion
 
Benefits:
  - Partial failures don't restart entire collection
  - Individual documents can retry without affecting others
  - Progress persists across task failures
  - Natural rate limiting through task queue

Task queues:
  - SYNC_COLLECTION_PATCH_REQUEST → /tasks/sync/request-collection-patch
  - SYNC_DOCUMENT_PATCH_REQUEST → /tasks/sync/request-document-patch
  - SYNC_COLLECTION_DELETION_REQUEST → /tasks/sync/request-collection-deletion
  - SYNC_DOCUMENT_DELETE_REQUEST → /tasks/sync/request-document-deletion
 
Task infrastructure handles:
  - Automatic retry on failure
  - Rate limiting and throttling
  - Task deduplication
  - Scheduled execution
  - GCP project routing

Path processing:
  - Strips leading slashes: /collection → collection
  - Strips trailing slashes: collection/ → collection
  - Removes all slashes for collection groups: a/b/c → abc
  - Handles both top-level and nested collection paths
 
Examples:
  - Input: /communities/123/members/
  - Output: communities/123/members
 
  - Input: /members/ (collectionGroup=true)
  - Output: members

Update behavior:
  - Uses Firestore update() not set()
  - Merges new values with existing document
  - Only specified fields are modified
  - Unspecified fields remain unchanged
  - Supports nested field updates with dot notation
 
Example:
  Document: {id: '123', name: 'John', age: 30, city: 'NYC'}
  Update: {age: 31}
  Result: {id: '123', name: 'John', age: 31, city: 'NYC'}

Document operations:
  - Each document patch is atomic
  - Each document deletion is atomic
  - No partial document states
  - Guaranteed consistency per document
 
Controller handlers:
  - documentPatchRequested: FirestorePaginationFunctions.update()
  - documentDeletionRequest: FirestorePaginationFunctions.delete()
 
Both use atomic Firestore operations

Deletion uses Command/Handler pattern:
  - DeleteCollectionCommand: Request representation
  - DeleteCollectionHandler: Business logic execution
  - DeletionTaskGenerator: Streams and creates delete tasks
 
Benefits:
  - Separation of concerns
  - Easy to add pre/post-processing hooks
  - Testable business logic
  - Extensible for custom deletion logic

BaseFirestoreStreamer features:
  - Abstract consume() method for custom processing
  - Automatic page fetching and iteration
  - Cursor management between pages
  - Pages processed counter
  - Optional custom orderBy field
  - Deep copy of items before processing
 
Implementations:
  - PatcherTaskGenerator: Creates patch tasks per document
  - DeletionTaskGenerator: Creates delete tasks per document
 
Both inherit streaming logic, only implement consume()

Example 1 - Product name updates:
  UpdateProductSubscriptionsHandler:
    - Streams all subscriptions
    - Filters by productId
    - Creates patch tasks for matching subscriptions
    - Updates denormalized productName field
    - Processes in parallel with Promise.all
 
Example 2 - Community member cleanup:
  DeleteAllMembersHandler:
    - Builds collection path from communityId
    - Requests collection deletion
    - Sync service handles pagination and deletion
    - Guarantees all members removed
 
These demonstrate:
  - Filtering before patching for efficiency
  - Parallel task creation for performance
  - Integration with repository patterns
  - Cascade deletion coordination

Serialization features:
  - Converts JavaScript objects to Firestore format
  - Deserializes Firestore data back to objects
  - Handles special Firestore types (Timestamp, GeoPoint, etc.)
  - Preserves type information through round-trips
  - Used automatically by all update/delete operations
 
FirestorePaginationFunctions handles:
  - serialize<T>() before writes
  - deserialize() after reads
  - Type-safe transformations

Ordering requirement:
  - All documents MUST have 'id' field
  - Used for orderBy in pagination queries
  - Enables consistent cursor-based pagination
  - Prevents duplicate processing
  - Ensures deterministic iteration order
 
WARNING in TaskService:
  "this collection data to patch must have an 'id' property
   which is used to sort paginated data"
 
Impact:
  - Documents without 'id' may be skipped
  - Pagination relies on indexed 'id' field
  - Collection must support orderBy('id') query