15. Google Cloud Pub/Sub

Google Cloud Pub/Sub is a fully managed, asynchronous messaging service. ArmoniK uses it as a task queue with topics and pull subscriptions, supporting optional message ordering and exactly-once delivery. For local development, ArmoniK deploys the Pub/Sub emulator.

15.1. Deployment

# Local development (uses the GCP Pub/Sub emulator)
just queue=pubsub build-deploy

# Production (against real GCP Pub/Sub — ensure credentials are set)
just queue=pubsub build-deploy

For production, GCP credentials must be available (Application Default Credentials or a service account key file).

15.2. Architecture in ArmoniK

ArmoniK creates one topic and one subscription per partition. Topics and subscriptions are created on demand (a 404 at startup triggers creation).

Naming pattern: a{Prefix}-{partitionId} (topic) and a{Prefix}-{partitionId}-ak-sub (subscription).

subscription:a{Prefix}-partition-0-ak-sub ──pull── PollingAgent[0]
        │
        topic:a{Prefix}-partition-0  ──── Submitter (push)

15.3. Configuration

All configuration variables and their defaults are documented in the Environment Variables reference.

15.4. Supported features

Feature

Supported

Task priorities

No — single subscription per partition, no priority routing

Message ordering

No — PubSub__MessageOrdering sets a subscription flag but the adapter does not set OrderingKey on published messages; Pub/Sub ordering guarantees do not apply

Exactly-once delivery

Yes — PubSub__ExactlyOnceDelivery=true (prevents duplicate deliveries at Pub/Sub layer)

KMS encryption

Yes — PubSub__KmsKeyName

Message retention

Yes — configurable via PubSub__MessageRetention

Automatic redelivery

Yes — via acknowledgment deadline (AckDeadlinePeriod)

15.5. Limitations

  • No priority queues. Pub/Sub has no native priority routing. All partitions use a single subscription.

  • MessageOrdering has no effect. PubSub__MessageOrdering=true sets EnableMessageOrdering on the subscription, but Pub/Sub only orders messages that carry an OrderingKey. The adapter does not set an OrderingKey when publishing, so delivery order is not guaranteed regardless of this setting.

  • ExactlyOnceDelivery overhead. This feature requires acknowledgment tracking on the Pub/Sub side and increases latency. Enable only when duplicate task execution is unacceptable.

  • GCP authentication required in production. The emulator does not require credentials; real GCP does. Ensure Application Default Credentials or a service account key is configured before deploying.

  • Topic/subscription naming: ArmoniK prepends a to the prefix to avoid names starting with goog (reserved by GCP). Keep Prefix short to stay within GCP’s 255-character name limit.

15.6. Health check

The emulator exposes an HTTP endpoint. The Terraform module waits for:

curl -fsSL localhost:8085