Feedback for Version 0.0.1-alpha.3

[marcuspoehls]

Hey Romain,

Thank you for putting the work into creating this queue package. I’m using this package in a personal project. The project is an Adonis.js app, and I’ve integrated the queue to follow the Adonis structure with a config/queue.ts file and a QueueServiceProvider.ts registering the queue manager singleton.

I wanted to give you feedback after using this package for two weeks. This feedback might be helpful for future directions and contributions from the community. I’m interested in contributing to this package.

Questions

I ran into a couple of questions while implementing the queue jobs and the queue worker. Maybe some of the questions are missing features because of the early release state. Others could be bugs or wrong usage of mine. I’m happy if you could share your thoughts 🙂

• How should I handle E_JOB_NOT_FOUND errors? I didn’t see an export for the (named) errors and wasn’t sure if I would try/catch errors and then check for an error code?
• Is it possible to search for enqueued jobs based on attributes (e.g., filtering by specific payload data)?
  • I want to ensure that only a single job is on the queue for a given database entity. I’m running recurring jobs using the delay feature, and a queue job enqueues the next one.
• Can I clear jobs based on filters such as queue name, status, or payload fields?
• How exactly does worker concurrency work?
  • In my case, I used the worker configuration from this package’s readme. It uses worker: { concurrency: 5 }. Then I started one worker. The worker processed my job 5 times in parallel — is this expected behavior, or should a worker have locked the job, or is it a misconfiguration on my side?

Wishlist

I’m also having a wishlist for features or adjustments. Are you open to contributions from the community via pull requests?

• Lifecycle hooks
  • I’m thinking of a method that runs when a job is successfully processed or ultimately fails after reaching the maximum number of retries. Something like onJobSucceeded or onJobFailed or onMaxRetriesReached
• Fail worker startup when no jobs are found (e.g. invalid glob pattern or empty path)
  • my setup is an Adonis.js v6 application that I transpile from TypeScript to JavaScript and ship the JavaScript to production. My queue jobs glob pattern only had the entry with the TypeScript file ending app/jobs/*.ts. That means, no jobs were found in production. The queue worker started correctly and detected jobs in the queue. While attempting to process queue jobs, the related job wasn’t found because the glob didn’t match a queue job file. Ultimately resulting in cleared jobs.
• Access to job context
  • such as current retry count or number of attempts made
• Keep processed jobs in the storage instead of deleting them
  • for example, the status could be extended and used with a "processed" state next to "pending" and "delayed"
  • this would allow me to check processed queue jobs and also keep a reference from the past. Maybe we could build a UI to display statistics.
• export the QueueConfigOptions type
  • currently that type is not exported from the package
  • my workaround is creating a custom type in my application by retrieving the type from the Worker constructor
• throw an exception when the Queue Manager isn’t booted
  • currently a generic "undefined is not an adapter" error is thrown when trying to process a queue job and the queue manager is not booted
  • in my Adonis application, I only registered the queue manager to the IOC container and forgot to boot the queue. I didn’t think of the missing boot in the beginning and had to dig into this package’s code to spot my issue

Thanks for building such a clean and framework-agnostic queue library — it’s proving very useful already! I hope this feedback is helpful and I’m happy to contribute pull requests for ideas on my wishlist 🙂