Tuesday, December 31, 2013

What is RabbitMQ and Celery?

RabbitMQ

key points from Wiki

  • Open source message broker software
  • AMQP (Advanced Message Queuing Protocol)
  • Written in the Erlang Programming Language
  • Built on the Open Telecom Platform framework
words from their official website
Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.
The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gavent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).



RabbitMQ 是一個仲介者角色(broker),使用AMQP所以開啓port 5672,啟動後開始聽別人分赴的工作。而Celery則是負責規劃工作,且是Multiprocessing,並由多個worker去完全安排的工作。

所以RabbitMQ和Celery是怎麼合作的?

這份stackoverflow的post中提到: Celery is a distributed task queue, which means the only thing that it should do is dispatching tasks/jobs to other servers and get the result back. And, RabbitMQ is a message queue, and nothing more.

由此可見,Celery是安排message怎麼到達RabbitMQ的一個角色,其中甚至可以做排程,而真正處理message的人是RabbitMQ。