A micro-service developed in Symfony 4 that consumes data from an external API, processes them, publishes them to a RabbitMQ exchange, consumes them from a RabbitMQ queue and stores them into a MySQL database.
composer install
symfony server:start
Run the following command to publish 1 message to the exchange:
php bin/console publish-message 1
Run the following command to consume 1 message from the queue and store it into the database:
php bin/console consume-message 1
or use docker-compose (run from the docker folder):
docker-compose build
docker-compose up -d
docker-compose run php composer install
Run the following command to publish 1 message to the exchange:
docker-compose run php bin/console publish-message 1
Run the following command to consume 1 message from the queue and store it into the database:
docker-compose run php bin/console consume-message 1
Run the following command to start consuming messages from the queue and store them into the database, until you stop the service:
docker-compose run php bin/console rabbitmq:consumer consume
Stop the containers:
docker-compose down
PublishMessageCommand: A custom console command used to ivoke the PublishController and publish messages to the exchange. It takes one argument that defines the number of messages to publish. Use it in the following way: php bin/console publish-message 1, with 1 defining the number of messages.
ConsumeMessageCommand: A custom console command used to ivoke the ConsumeController and consume messages from the queue. It takes one argument that defines the number of messages to consume. Use it in the following way: php bin/console consume-message 1, with 1 defining the number of messages.
MessageController: Handles all the processing of the messages. This controller implements two methods:
- publishToExchange(int $num): Makes requests to an external API, processes the response and publishes a message to a RabbitMQ exchange with a valid routing key. It takes one argument that defines the number of requests and therefore messages to publish to the RabbitMQ exchange.
- consumeFromQueue(int $num): Consumes messages from a RabbitMQ queue. The callback ivokes the MessageService which processes the message and stores it into the database. It takes one argument that defines the number of messages to consume from the RabbitMQ queue.
MessageService: A service that is ivoked by the callback defined in the RabbitMQ queue. It implements the ConsumerInterface::execute method. This method consumes the message from the queue, processes it and stores it into the database.
HexConverter: A helper service that converts a big integer in hexadeciman format to its decimal format. This service is required as the default hexdec() PHP function can't handle big integers.
Our database only has one table named metric. The structure is presented bellow:
| Collumn | Type | Description |
|---|---|---|
| id | int(11) | Unique ID for every entry. This is our primary key. |
| gateway_eui | varchar(255) | The gateway_eui value we got from consuming the API. Because of its big size, it is stored in the database as a string. |
| attribute_id | int(11) | The attribute_id value we got from consuming the API. |
| value | int(11) | The value field we got from consuming the API. |
| timestamp | bigint(20) | The timestamp value we got from consuming the API. It is stored as a bigint in order be more easy to sort entries by this collumn. |
The values for profile_id, endpoint_id and cluster_id are the same for every entry and thus there is no point in storing them in the database.