Apache ActiveMQ is a great workhorse full of features and nice stuff. It’s not the fastest MQ software around but fast enough for most use cases. Among features are flexible clustring, fail-over, integrations with different application servers, security etc. Apache Apollo is an attempt to write a new core for ActiveMQ to cope with a … Read more
Using Kafka for a task queue is a bad idea. Use RabbitMQ instead, it does it much better and more elegantly. Although you can use Kafka for a task queue – you will get some issues: Kafka is not allowing to consume a single partition by many consumers (by design), so if for example a … Read more
I would say that Gearman is better for queuing “jobs” and RabbitMQ is better for queuing “data”. Of course, they are both really the same thing, but the way it works out for me is that if you are trying to “fan out” work to be done, and the workers can work independently, Gearman is … Read more
There are not any hard-coded limits inside RabbitMQ broker. The broker will utilize all available resources (unless you set limits on some of them, they are called watermarks in RabbitMQ terminology). There are some limitations put by Erlang itself, like maximum number of concurrent processes, but if you theoretically can reach them on single node … Read more
I assume your system looks like the following. You have a REST service, which receives requests from the client. It converts the requests into commands which the business logic can understand. You put these commands into a queue. You have a single or multiple workers which can process and remove these commands from the queue … Read more
Personally I am quite fond of message queues and think they are arguably the most under-utilized IPC in the unix world. They are fast and easy to use. A couple of thoughts: Some of this is just fashion. Old things become new again. Add a shiny do-dad on message queues and they may be next … Read more
Years ago, I asked myself the same question. I was looking at NServiceBus to work with a different message queue, but the question was the same. I decided not to use NServiceBus. 6 Month later, I realized I had re-built half of what NServiceBus did… only much more poorly. The equivalent question of why would … Read more
RabbitMQ is probably doing persistence on those messages. I think you need to set the message priority or another option in messages to not do persistence. Performance will improve 10x then. You should expect at least 100K messages/second through an AMQP broker. In OpenAMQ we got performance up to 300K messages/second. AMQP was designed for … Read more
Well, let’s take a closer look at the scenario you are describing above. I think it’s important to paste the documentation immediately prior to the snippet in your question to provide context: Section 4.7 of the AMQP 0-9-1 core specification explains the conditions under which ordering is guaranteed: messages published in one channel, passing through … Read more