14. Adaptors

An adaptor provides a way to connect ArmoniK.Core to other services. They are used as abstractions for:

  • The queue whose role is to distribute messages representing tasks to execute to polling agents.

  • The database which is used to store the metadata of the tasks, tasks input and/or output data, partitions and authentication. It keeps track of the state of the computations and data managed by ArmoniK.

  • The object storage where the data used as task input and/or output is stored.

An interface lies between each service and the other components of Core. The adaptors provide implementations of the interfaces. The interface abstracts away all the complexities and the details of the underlying service (queue, database or object storage), and provides the features of the service that Core components may need. It is defined in the NuGet package ArmoniK.Core.Base. This package contains the interfaces and classes needed for the implementation of a custom adaptor.

When the components of ArmoniK.Core need to interact with services like the queue, database, or object storage, they don’t directly interact with these services. Instead, they only interact with the interface. The adaptors implement this interface and handle the specific logic for calling and managing the required service, whether it’s the queue, database, or object storage. We plan to provide adaptors implementations for on-premises installation and each cloud provider.

14.1. Adaptor dynamic loading

Adaptors will be loaded dynamically during startup (today, only the queue can be loaded dynamically but other adaptors will come soon). Adaptors must implement the IDependencyInjectionBuildable since we rely on the implementation of this interface to register the adaptors implementations through dependency injection. This interface exposes a Build method that will be integrated in the dependency injection. It enables adaptors to instantiate their classes and expose the implemented interfaces. Then, the interfaces will be used by the middleware to interact with the different storages.

14.2. Class initialization and health check

Adaptor interfaces inherit from the IInitializable that requires an Init and a Check methods. Therefore, it is mandatory to implement them. Init is used for deferred initialization of attributes in class instances (for instance, the creation of a connection after the build of the instances through the Dependency Injection). Init should also be reentrant and thread-safe to ensure that no issue arise during initialization. Check provides an health check interface used to request the status of the adaptor (for instance, are we still connected to the queue ? Can we still receive messages ?). These adaptor health checks form the health check of the application (polling agents, control plane, dependency checker, metrics exporters).

14.3. Queue Adaptors

Core components have two primary tasks when interacting with the queue: pushing messages into the queue and pulling messages from it. That’s why a queue adaptor needs to mainly implement two methods of the queue interface to define an adaptor: IPushQueueStorage and IPullQueueStorage.

Pre-included queue adaptors are also available such as:

  • ArmoniK.Core.Adapters.Amqp provides an AMQP connector for ArmoniK. It allows using any AMQP server such as ActiveMQ, ActiveMQ Artemis, RabbitMQ or IBM MQ

  • ArmoniK.Core.Adapters.RabbitMQ provides a dedicated adaptor for RabbitMQ. It uses RabbitMQ native client that supports priorities better than the AMQ protocol in our AMQP adaptor.

  • ArmoniK.Core.Adapters.Nats provides an Nats Jets Stream connector for ArmoniK. It allows using a Nats server.

Other adaptors for queue may also be available:

  • ArmoniK.Core.Adapters.SQS AmazonSQS (supports priority)

  • ArmoniK.Core.Adapters.PubSub GCP Pub/Sub

14.4. Object Storage Adaptors

The ArmoniK.Core.Base project does not expose interfaces for the object storage yet but it will be coming soon.

The current available object storage adaptors are:

  • ArmoniK.Adapters.Redis provides a RedisCache implementation.

  • ArmoniK.Adapters.S3 provides a S3 implementation.

14.5. Database Adaptors

The ArmoniK.Core.Base project does not expose interfaces for the database yet but it will be coming soon.

The current available database adaptors are:

  • ArmoniK.Core.Adapters.MongoDB

Other adaptors for database may also be available:

  • SQL

  • DynamoDB