Celery task results stored in the backend after execution can occupy storage space and affect performance if left for too long. Both Django-celery-beat and Celery offer options to clean up backend data in their respective ways to address this issue. In this article, we will explore the `CELERY_TASK_RESULT_EXPIRES` setting and the `celery.backend_cleanup` task in Django-celery-beat, along with recommendations for using both settings.

1. CELERY_TASK_RESULT_EXPIRES: An Option to Set the Expiration Time of Task Results

CELERY_TASK_RESULT_EXPIRES is an option to set how long the result of an individual task will remain in the backend, measured in seconds. This setting operates on a TTL (Time-To-Live) basis whenever a Celery task is scheduled, resulting in the automatic deletion of task results after the specified time.

• Example Settings:

  CELERY_TASK_RESULT_EXPIRES = 86400  # 24 hours (in seconds)

• This setting ensures that task results are automatically deleted from the backend after 24 hours. The TTL setting is useful for adjusting individual task expiration times and prevents unnecessary data accumulation in the backend.

2. Django-celery-beat's celery.backend_cleanup: Periodic Cleanup Task

When using Django-celery-beat, the celery.backend_cleanup task periodically cleans up old task results accumulated in the backend. This cleanup task is scheduled to run once a day by default, and you can adjust its frequency in the Django Admin to have it run more or less frequently. This periodic cleanup task effectively prevents old data from accumulating in the backend even without a TTL setting.

• Example of Setting Frequency: In Django Admin, you can adjust the frequency of celery.backend_cleanup; if there's a lot of real-time data, you can set it to run more frequently, or keep it at the default (once a day) if not.

3. Conclusion: Which Setting to Choose?

CELERY_TASK_RESULT_EXPIRES and celery.backend_cleanup settings perform overlapping functions, so using either one is sufficient for managing backend memory. However, you can enable both settings if necessary.

• If You Are Using Django-celery-beat: Since the celery.backend_cleanup task periodically cleans up the backend, it is usually sufficient to schedule this cleanup task without needing the TTL setting.
• If You Schedule Celery Tasks with Code: If you're writing your task scheduling code personally without using Django-celery-beat, it is recommended to set CELERY_TASK_RESULT_EXPIRES to manage TTL. This ensures that each task result is automatically deleted after the configured time, efficiently managing backend memory.

Summary: Backend management is possible with either the cleanup task of Django-celery-beat or the TTL setting alone. For Django-celery-beat users, scheduling the cleanup task is suitable, while for those scheduling with code, the TTL setting is recommended. You can also use both settings simultaneously for more detailed data management.