14. Amazon SQS

Amazon Simple Queue Service (SQS) is a fully managed message queuing service. ArmoniK uses it as a task queue with optional priority-based routing and batch operations. For local development, ArmoniK deploys ElasticMQ as a drop-in SQS emulator.

14.1. Deployment

# Local development (uses ElasticMQ emulator)
just queue=sqs build-deploy

# Production (against real AWS SQS — ensure AWS credentials are set)
just queue=sqs build-deploy

AWS credentials must be available as environment variables before deployment:

export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...

14.2. Architecture in ArmoniK

ArmoniK creates one SQS queue per partition per priority level. With MaxPriority=0 (default) a single queue is used per partition; with MaxPriority=N there are N queues per partition, one per priority level (1 = lowest, N = highest).

partition-0-1 ──┐
partition-0-2 ──┤── SQS
partition-1-1 ──┤
partition-1-2 ──┘

PollingAgents pull from the highest-priority non-empty queue first.

Queue names follow the pattern: {Prefix}-{partitionId}-{priority} (or {Prefix}-{partitionId} when MaxPriority=0).

14.3. Configuration

All configuration variables and their defaults are documented in the Environment Variables reference.

14.4. Supported features

Feature

Supported

Task priorities

Yes — one SQS queue per priority level per partition

Batch operations

Yes — up to 10 messages per batch (SQS API limit)

Long polling

Yes — configurable via WaitTimeSeconds

KMS encryption

Yes — via Attributes["KmsMasterKeyId"]

Automatic redelivery

Yes — via visibility timeout (AckDeadlinePeriod)

14.5. Limitations

  • Message size: SQS limits messages to 256 KB. ArmoniK task metadata stays well within this limit.

  • Visibility timeout cap: SQS maximum visibility timeout is 12 hours (43 200 s). AckDeadlinePeriod must stay below this.

  • Queue URL caching: ArmoniK caches queue URLs and refreshes on 404 or NonExistentQueue errors. Avoid deleting queues while tasks are in flight.

  • Batch size: SQS returns at most 10 messages per ReceiveMessage call. High-throughput workloads benefit from increasing DegreeOfParallelism.

14.6. Health check

The ElasticMQ emulator (local) exposes an SQS-compatible endpoint. The Terraform module waits for:

curl -fsSL "http://localhost:9324/?Action=ListQueues&Version=2012-11-05"