Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to consume from primary and secondary job queue #190

Open
EwaRvr opened this issue Mar 1, 2019 · 3 comments
Open

How to consume from primary and secondary job queue #190

EwaRvr opened this issue Mar 1, 2019 · 3 comments

Comments

@EwaRvr
Copy link

EwaRvr commented Mar 1, 2019

Is there a way to consume from 2 queues in such a way that as long as there are messages in the primary queue, messages in the secondary queue are ignored?

Scenario:
Our API posts messages into a queue, and waits for a certain time for a response. If that response comes back within a certain time, it return that response, but if it times out due to the backend being flooded with requests, it sends back a 201 Accepted status. In addition, we want to take that request, and place it in a separate (lower priority) queue, probably using deadlettering. The reason is that as now there is no API client waiting for the response anymore, we might as well prioritize other requests that a client is waiting for, over this one, and come back to this one once the spike has quieted down.

But now our challenge is how to have a single process consume from our primary job queue first, and only consume from the secondary/lower priority queue when there is nothing left in the primary queue?

I know RabbitMQ has message priority within a single queue, which Rabbot doesn't support, but I'd much prefer using separate queues.

Using Rabbot, what is the best strategy for this?

Thanks

@woolite64
Copy link

Rabbot allows you to set maxPriority on queue creation. See maxPriority in the Topology documentation.

@EwaRvr
Copy link
Author

EwaRvr commented Mar 1, 2019

@woolite64 Yes, but as arobson replied to issue #13, it does not support setting priority on the messages. meaning they ALL go at priority level 0.

@alonisser
Copy link
Contributor

@EwaRvr while I would the pr to be merged, we currently maintain a fork https://github.com/Zappatta/rabbot by the talented @Zappatta allowing message priorities.. and we do use it successfully

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants