Skip to content

Basic-Data-Infrastructure/BDI-FEDeRATED-Distribution-Orchestrator

Repository files navigation

Federated Distribution Orchestrator

made with: python code style: black

Configuration file

The DO needs a configuration file to run. The location of the file is controlled by two environment variables:

  • PYC_ROOT: root directory of the software (default /)
  • PYC_CONFIG: config file relative to the root (default /config/application.yaml)

Note: all times in the configuration file are in seconds

Delivery

Delivery of message works as follows. The DO will determine to which nodes it needs to deliver messages. For each node it will spawn a worker to deliver those messages. The number of parallel workers is limited by the configuration value delivery.startup.workers.

Each worker will try to deliver a limited number of messages (specified by delivery.batch).

If delivery fails, the message will be rescheduled for delivery in an amount of time that depends on the number of deliveries already tried. The delay is delivery.retry.delay.fixed + [number of tried attempts - 1] * delivery.retry.delay.progressive.

If delivery fails delivery.retry.max times, the message will be frozen, and no further delivery will be scheduled. This behaviour can be disabled by setting delivery.retry.max to a value of 0.

Delivery attempts will be executed at delivery.interval intervals. When new messages arrive, a delivery will be executed immediately.

Database purge

The database will be cleaned at purge.interval intervals. The following data will be deleted:

  • Delivered messages older than purge.delivered seconds
  • Frozen messages older than purge.frozen seconds
  • Log records older than purge.log seconds

Purging may be disabled completely by setting purge.interval to 0. Or by setting purge.delivered, purge.frozen or purge.log to 0 to disable that particular purge.

Dependencies

Release 1.0 requires the folowing libraries.

library version
fastapi 0.115.6
PyYAML 6.0.2
python-multipart 0.0.20
gunicorn 23.0.0
uvicorn 0.34.0
uvicorn-worker 0.3.0
sqlmodel 0.0.22
psycopg2-binary 2.9.10
PyJWT 2.10.1
cryptography 44.0.0
pycryptodome 3.21.0
bcrypt 4.2.1
requests 2.32.3
aiofiles 24.1.0
aiohttp 3.11.11
aiodns 3.2.0
Brotli 1.1.0
charset-normalizer 3.4.1
uvloop 0.21.0
httptools 0.6.4
tabulate 0.9.0

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors