The API and storage layers are pluggable and provide ability to work with different backends and queue service providers.
Conductor follows RPC based communication model where workers are running on a separate machine from the server. Workers communicate with server over HTTP based endpoints and employs polling model for managing work queues.
- Workers are remote systems that communicate over HTTP with the conductor servers.
- Task Queues are used to schedule tasks for workers. We use dyno-queues internally but it can easily be swapped with SQS or similar pub-sub mechanism.
- conductor-redis-persistence module uses Dynomite for storing the state and metadata along with Elasticsearch for indexing backend.
- See section under extending backend for implementing support for different databases for storage and indexing.