Skip to Content
Engineering11 Documentation 🔥
BackendServer API PackagesPub/Sub

PubSub API

Simplified Google Cloud Pub/Sub wrapper with automatic retries and message archiving.

What It Does

The PubSub API provides a simplified interface for publish-subscribe messaging using Google Cloud Pub/Sub. It enables applications to send messages to topics and subscribe to them with automatic retry logic, message archiving, and intelligent message parsing that handles multiple encoding formats.

Key Capabilities

CapabilityDescription
Message PublishingPublish messages to Pub/Sub topics with automatic retries
Automatic ArchivingOptional message archiving to audit trail topic
Retry LogicExponential backoff with up to 5 retry attempts
Subscription ManagementManage topic subscriptions and message handlers
Intelligent ParsingAutomatic message parsing for multiple encoding formats
Ordering SupportMessage ordering via ordering keys
Topic ReachabilityCheck if topics exist and are accessible
Mock SupportTesting mock for unit tests
Buffer EncodingHandles Buffer-encoded messages (default Pub/Sub format)
Multiple HandlersSupport multiple handlers per topic subscription

Configuration

Constants

  • ARCHIVE_TOPIC_NAME: ‘engineering11_payload_archive’
  • MAX_ATTEMPTS: 5 (retry attempts)

Archive Topic

  • Automatic creation if not exists
  • Stores complete message metadata
  • UUID-based message IDs

Exports

Services

  • PubSubService - Main publishing service
  • MockPubsubService - Testing mock
  • PubSubSubscriptionManager - Subscription management

Functions

  • parsePubsubMessage() - Message parser

Types

  • Event<T> - Generic event type
  • UpdatedEvent<T> - Update event type
  • CloudFunctionEvent<T> - Cloud Function event
  • IEvent, IUpdatedEvent<T> - Deprecated types

Constants

  • CONSTANTS - Configuration constants

Common Use Cases

  • Event-driven architecture: Publish events between microservices
  • Async job processing: Queue jobs for background processing
  • Data pipeline: Stream data between processing stages
  • Notification delivery: Send notifications to subscribers
  • Audit logging: Archive all messages for compliance
  • Real-time updates: Push updates to subscribers
  • Webhook delivery: Reliably deliver webhooks
  • Message broadcasting: Send to multiple subscribers

Roadmap

Multi-Cloud Support

This API currently supports Google Cloud Platform. Future versions will add support for additional cloud providers and container-based deployment options, enabling greater flexibility in infrastructure choices.

What Customers Don’t Have to Build

  • Google Cloud Pub/Sub integration
  • Exponential backoff retry logic
  • Message archiving infrastructure
  • Intelligent message parsing
  • Buffer and encoding handling
  • Subscription management
  • Topic reachability checking
  • Message ordering support
  • Error standardization
  • Connection lifecycle management
  • Mock service for testing
  • Event type definitions
  • Protobuf message parsing
  • JSON message parsing
  • Handler registration system
  • Multi-handler support per topic
  • Graceful shutdown logic
  • Archive topic management
Last updated on
EventsPub/Sub Service