Using Celery with Django
Integrating Celery with Django allows you to handle background tasks like sending emails, processing images, or generating reports without blocking the main request-response cycle.
1. Installation
Install Celery and a broker interface (e.g., Redis).
pip install celery redis2. Project Setup
You need to define the Celery instance in your Django project. Suppose your project is named myproject.
Create celery.py
Create a file named celery.py inside your main project directory (next to settings.py and urls.py).
# myproject/celery.py
import os
from celery import Celery
# Set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myproject.settings')
app = Celery('myproject')
# Using a string here means the worker doesn't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
# should have a `CELERY_` prefix.
app.config_from_object('django.conf:settings', namespace='CELERY')
# Load task modules from all registered Django apps.
app.autodiscover_tasks()
@app.task(bind=True, ignore_result=True)
def debug_task(self):
print(f'Request: {self.request!r}')Update __init__.py
To ensure the Celery app is loaded when Django starts, modify myproject/__init__.py.
# myproject/__init__.py
from .celery import app as celery_app
__all__ = ('celery_app',)3. Configuration
Add Celery configuration to your settings.py.
# settings.py
# Celery Configuration Options
CELERY_BROKER_URL = 'redis://localhost:6379/0'
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'UTC'4. Creating Tasks
You can define tasks in a tasks.py file inside any of your Django apps. Use the @shared_task decorator so your apps don't need to import the specific Celery app instance.
# myapp/tasks.py
from celery import shared_task
@shared_task
def add(x, y):
return x + y
@shared_task
def send_welcome_email(user_id):
# Logic to send email
return f"Email sent to user {user_id}"5. Running the Worker
Open a terminal and run the Celery worker from your project root.
celery -A myproject worker -l INFO-A myproject: Specifies the Django project name (wherecelery.pyis located).-l INFO: Sets the logging level.
6. Django Database Access
Celery tasks can interact with Django models just like regular views.
from celery import shared_task
from .models import User
@shared_task
def count_users():
return User.objects.count()Note: Be careful passing model instances to tasks. It is better to pass the primary key (ID) and fetch the object inside the task to avoid race conditions and serialization issues.