Why would running scheduled tasks with Celery be preferable over crontab?

I’ve been using cron for a production website, and have switched to celery on a current project.
I’m far more into celery than cron, here is why:

  • Celery + Celerybeat has finer granularity than cron. Cron cannot run more than once a minute, while celery can (I have a task run every 90 seconds which checks an email queue to send messages, and another which cleans the online users list).
  • A cron line has to call a script or a unique command, with absolute path and user info. Celery calls python functions, no need to write more than code.
  • With celery, to deploy to another machine, you generally just have to pull/copy your code, which is generally in one place. Deploying with cron would need more work (you can automate it but…)
  • I really find celery better suited than cron for routine cleaning (cache, database), and in general, for short tasks. Dumping a database is more a work for cron, however, because you don’t want clutter the event queue with too long tasks.
  • Not the least, Celery is easily distributed across machines.

Leave a Comment