Skip to content

Commit e9a4b92

Browse files
committed
add tons of new resources to the celery page
1 parent 6d18bb9 commit e9a4b92

File tree

1 file changed

+71
-22
lines changed

1 file changed

+71
-22
lines changed

content/pages/04-web-development/30-celery.markdown

Lines changed: 71 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ meta: Celery is a task queue for executing work outside a Python web application
1212
implementation for [Python web applications](/web-development.html) used
1313
to asynchronously execute work outside the HTTP request-response cycle.
1414

15-
<a href="http://www.celeryproject.org/" style="border: none;"><img src="/img/logos/celery.png" alt="Celery task queue project logo." style="border-radius: 5px;" width="100%" class="technical-diagram"></a>
15+
<a href="http://www.celeryproject.org/" style="border:none"><img src="/img/logos/celery.png" alt="Celery task queue project logo." class="shot" width="100%"></a>
1616

1717
<div class="well see-also">Celery is an implementation of the <a href="/task-queues.html">task queue</a> concept. Learn more in the <a href="/web-development.html">web development</a> chapter or view the <a href="/table-of-contents.html">table of contents</a> for all topics.</div>
1818

@@ -48,18 +48,21 @@ every Sunday. When the interval or specific time is hit, Celerybeat will
4848
hand the job over to Celeryd to execute on the next available worker.
4949

5050

51-
### Celery tutorials
51+
### Celery tutorials and advice
5252
Celery is a powerful tool that can be difficult to wrap your mind around
5353
at first. Be sure to read up on [task queue](/task-queues.html) concepts
5454
then dive into these specific Celery tutorials.
5555

56-
* [Getting Started Scheduling Tasks with Celery](http://www.caktusgroup.com/blog/2014/06/23/scheduling-tasks-celery/)
57-
is a detailed walkthrough for setting up Celery with Django (although
58-
Celery can also be used without a problem with other frameworks).
56+
* [A 4 Minute Intro to Celery](https://www.youtube.com/watch?v=68QWZU_gCDA) is
57+
a short introductory task queue screencast.
5958

60-
* [Introducing Celery for Python+Django](http://www.linuxforu.com/2013/12/introducing-celery-pythondjango/)
61-
provides an introduction to the Celery task queue with Django as the
62-
intended framework for building a web application.
59+
* This blog post series on
60+
[Celery's architecture](https://www.vinta.com.br/blog/2017/celery-overview-archtecture-and-how-it-works/),
61+
[Celery in the wild: tips and tricks to run async tasks in the real world](https://www.vinta.com.br/blog/2018/celery-wild-tips-and-tricks-run-async-tasks-real-world/)
62+
and
63+
[dealing with resource-consuming tasks on Celery](https://www.vinta.com.br/blog/2018/dealing-resource-consuming-tasks-celery/)
64+
provide great context for how Celery works and how to handle some of the
65+
trickier bits to working with the task queue.
6366

6467
* [How to use Celery with RabbitMQ](https://www.digitalocean.com/community/articles/how-to-use-celery-with-rabbitmq-to-queue-tasks-on-an-ubuntu-vps)
6568
is a detailed walkthrough for using these tools on an Ubuntu VPS.
@@ -77,18 +80,19 @@ then dive into these specific Celery tutorials.
7780
are great reads for understanding the difference between a task queue and
7881
why you shouldn't use your database as one.
7982

80-
* [A 4 Minute Intro to Celery](https://www.youtube.com/watch?v=68QWZU_gCDA) is
81-
a short introductory task queue screencast.
83+
* [My Experiences With A Long-Running Celery-Based Microprocess](https://theblog.workey.co/my-experiences-with-a-long-running-celery-based-microprocess-b2cc30da94f5)
84+
gives some good tips and advice based on experience with Celery workers
85+
that take a long time to complete their jobs.
86+
87+
* [Checklist to build great Celery async tasks](http://celerytaskschecklist.com/)
88+
is a site specifically designed to give you a list of good practices to
89+
follow as you design your task queue configuration and deploy to
90+
development, staging and production environments.
8291

8392
* Heroku wrote about how to
8493
[secure Celery](https://engineering.heroku.com/blogs/2014-09-15-securing-celery)
8594
when tasks are otherwise sent over unencrypted networks.
8695

87-
* Miguel Grinberg wrote a nice post on using the
88-
[task queue Celery with Flask](http://blog.miguelgrinberg.com/post/using-celery-with-flask).
89-
He gives an overview of Celery followed by specific code to set up the task
90-
queue and integrate it with Flask.
91-
9296
* [Unit testing Celery tasks](https://www.python-celery.com/2018/05/01/unit-testing-celery-tasks/)
9397
explains three strategies for testing code within functions that Celery
9498
executes. The post concludes that calling Celery tasks synchronously to test
@@ -99,13 +103,37 @@ then dive into these specific Celery tutorials.
99103
[open source Git repository with all of the source code](https://github.com/ZoomerAnalytics/python-celery-unit-testing)
100104
from the post.
101105

106+
* [Rollbar monitoring of Celery in a Django app](https://www.mattlayman.com/blog/2017/django-celery-rollbar/)
107+
explains how to use [Rollbar](/rollbar.html) to monitor tasks. Super
108+
useful when workers invariably die for no apparent reason.
109+
102110
* [3 Gotchas for Working with Celery](https://wiredcraft.com/blog/3-gotchas-for-celery/)
103111
are things to keep in mind when you're new to the Celery task queue
104112
implementation.
105113

106-
* [Deferred Tasks and Scheduled Jobs with Celery 3.1, Django 1.7 and Redis](https://godjango.com/63-deferred-tasks-and-scheduled-jobs-with-celery-31-django-17-and-redis/)
107-
is a video along with code that shows how to set up Celery with Redis as the
108-
broker in a Django application.
114+
* [Dask and Celery](http://matthewrocklin.com/blog/work/2016/09/13/dask-and-celery)
115+
compares Dask.distributed with Celery for Python projects. The post gives
116+
code examples to show how to execute tasks with either task queue.
117+
118+
* [Python+Celery: Chaining jobs?](https://stackoverflow.com/questions/3901101/pythoncelery-chaining-jobs)
119+
explains that Celery tasks should be dependent upon each other using
120+
Celery chains, not direct dependencies between tasks.
121+
122+
123+
### Celery with web frameworks
124+
Celery is typically used with a [web framework](/web-frameworks.html) such as
125+
[Django](/django.html), [Flask](/flask.html) or [Pyramid](/pyramid.html).
126+
These resources show you how to integrate the Celery task queue with the
127+
web framework of your choice.
128+
129+
* [How to Use Celery and RabbitMQ with Django](https://simpleisbetterthancomplex.com/tutorial/2017/08/20/how-to-use-celery-with-django.html)
130+
is a great tutorial that shows how to both install and set up a basic
131+
task with Django.
132+
133+
* Miguel Grinberg wrote a nice post on using the
134+
[task queue Celery with Flask](http://blog.miguelgrinberg.com/post/using-celery-with-flask).
135+
He gives an overview of Celery followed by specific code to set up the task
136+
queue and integrate it with Flask.
109137

110138
* [Setting up an asynchronous task queue for Django using Celery and Redis](http://michal.karzynski.pl/blog/2014/05/18/setting-up-an-asynchronous-task-queue-for-django-using-celery-redis/)
111139
is a straightforward tutorial for setting up the Celery task queue for
@@ -117,15 +145,36 @@ then dive into these specific Celery tutorials.
117145
in your application. Note however there are other ways of integrating
118146
Celery with Django that do not require the django-celery dependency.
119147

148+
* [Flask asynchronous background tasks with Celery and Redis](http://allynh.com/blog/flask-asynchronous-background-tasks-with-celery-and-redis/)
149+
combines Celery with [Redis](/redis.html) as the broker and
150+
[Flask](/flask.html) for the example application's framework.
151+
152+
* [Celery and Django and Docker: Oh My!](https://www.revsys.com/tidbits/celery-and-django-and-docker-oh-my/)
153+
shows how to create Celery tasks for Django within a [Docker](/docker.html)
154+
container. It also provides some
155+
120156
* [Asynchronous Tasks With Django and Celery](https://realpython.com/blog/python/asynchronous-tasks-with-django-and-celery/)
121157
shows how to integrate Celery with [Django](/django.html) and create Periodic Tasks.
122158

123-
* [Dask and Celery](http://matthewrocklin.com/blog/work/2016/09/13/dask-and-celery)
124-
compares Dask.distributed with Celery for Python projects. The post gives
125-
code examples to show how to execute tasks with either task queue.
159+
* [Getting Started Scheduling Tasks with Celery](http://www.caktusgroup.com/blog/2014/06/23/scheduling-tasks-celery/)
160+
is a detailed walkthrough for setting up Celery with Django (although
161+
Celery can also be used without a problem with other frameworks).
162+
163+
* [Introducing Celery for Python+Django](http://www.linuxforu.com/2013/12/introducing-celery-pythondjango/)
164+
provides an introduction to the Celery task queue with Django as the
165+
intended framework for building a web application.
166+
167+
* [Asynchronous Tasks with Falcon and Celery](https://testdriven.io/asynchronous-tasks-with-falcon-and-celery)
168+
configures Celery with the [Falcon](/falcon.html) framework, which is
169+
less commonly-used in web tutorials.
126170

127171

128172
### Celery deployment resources
173+
Celery and its broker run separately from your web and WSGI servers so it
174+
adds some additional complexity to your [deployments](/deployment.html). The
175+
following resources walk you through how to handle deployments and get the
176+
right configuration settings in place.
177+
129178
* The "Django in Production" series by
130179
[Rob Golding](https://twitter.com/robgolding63) contains a post
131180
specifically on [Background Tasks](http://www.robgolding.com/blog/2011/11/27/django-in-production-part-2---background-tasks/).
@@ -140,5 +189,5 @@ then dive into these specific Celery tutorials.
140189
discussed in existing documentation.
141190

142191
* [Three quick tips from two years with Celery](https://library.launchkit.io/three-quick-tips-from-two-years-with-celery-c05ff9d7f9eb)
143-
provides some solid advice on retry delays, the -Ofair flag and global
192+
provides some solid advice on retry delays, the `-Ofair` flag and global
144193
task timeouts for Celery.

0 commit comments

Comments
 (0)