1.. _daemonizing: 2 3====================================================================== 4 Daemonization 5====================================================================== 6 7.. contents:: 8 :local: 9 10Most Linux distributions these days use systemd for managing the lifecycle of system 11and user services. 12 13You can check if your Linux distribution uses systemd by typing: 14 15.. code-block:: console 16 17 $ systemd --version 18 systemd 237 19 +PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN -PCRE2 default-hierarchy=hybrid 20 21If you have output similar to the above, please refer to 22:ref:`our systemd documentation <daemon-systemd-generic>` for guidance. 23 24However, the init.d script should still work in those Linux distributions 25as well since systemd provides the systemd-sysv compatibility layer 26which generates services automatically from the init.d scripts we provide. 27 28If you package Celery for multiple Linux distributions 29and some do not support systemd or to other Unix systems as well, 30you may want to refer to :ref:`our init.d documentation <daemon-generic>`. 31 32.. _daemon-generic: 33 34Generic init-scripts 35====================================================================== 36 37See the `extra/generic-init.d/`_ directory Celery distribution. 38 39This directory contains generic bash init-scripts for the 40:program:`celery worker` program, 41these should run on Linux, FreeBSD, OpenBSD, and other Unix-like platforms. 42 43.. _`extra/generic-init.d/`: 44 https://github.com/celery/celery/tree/master/extra/generic-init.d/ 45 46.. _generic-initd-celeryd: 47 48Init-script: ``celeryd`` 49---------------------------------------------------------------------- 50 51:Usage: `/etc/init.d/celeryd {start|stop|restart|status}` 52:Configuration file: :file:`/etc/default/celeryd` 53 54To configure this script to run the worker properly you probably need to at least 55tell it where to change 56directory to when it starts (to find the module containing your app, or your 57configuration module). 58 59The daemonization script is configured by the file :file:`/etc/default/celeryd`. 60This is a shell (:command:`sh`) script where you can add environment variables like 61the configuration options below. To add real environment variables affecting 62the worker you must also export them (e.g., :command:`export DISPLAY=":0"`) 63 64.. Admonition:: Superuser privileges required 65 66 The init-scripts can only be used by root, 67 and the shell configuration file must also be owned by root. 68 69 Unprivileged users don't need to use the init-script, 70 instead they can use the :program:`celery multi` utility (or 71 :program:`celery worker --detach`): 72 73 .. code-block:: console 74 75 $ celery multi start worker1 \ 76 -A proj \ 77 --pidfile="$HOME/run/celery/%n.pid" \ 78 --logfile="$HOME/log/celery/%n%I.log" 79 80 $ celery multi restart worker1 \ 81 -A proj \ 82 --logfile="$HOME/log/celery/%n%I.log" \ 83 --pidfile="$HOME/run/celery/%n.pid 84 85 $ celery multi stopwait worker1 --pidfile="$HOME/run/celery/%n.pid" 86 87.. _generic-initd-celeryd-example: 88 89Example configuration 90~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 91 92This is an example configuration for a Python project. 93 94:file:`/etc/default/celeryd`: 95 96.. code-block:: bash 97 98 # Names of nodes to start 99 # most people will only start one node: 100 CELERYD_NODES="worker1" 101 # but you can also start multiple and configure settings 102 # for each in CELERYD_OPTS 103 #CELERYD_NODES="worker1 worker2 worker3" 104 # alternatively, you can specify the number of nodes to start: 105 #CELERYD_NODES=10 106 107 # Absolute or relative path to the 'celery' command: 108 CELERY_BIN="/usr/local/bin/celery" 109 #CELERY_BIN="/virtualenvs/def/bin/celery" 110 111 # App instance to use 112 # comment out this line if you don't use an app 113 CELERY_APP="proj" 114 # or fully qualified: 115 #CELERY_APP="proj.tasks:app" 116 117 # Where to chdir at start. 118 CELERYD_CHDIR="/opt/Myproject/" 119 120 # Extra command-line arguments to the worker 121 CELERYD_OPTS="--time-limit=300 --concurrency=8" 122 # Configure node-specific settings by appending node name to arguments: 123 #CELERYD_OPTS="--time-limit=300 -c 8 -c:worker2 4 -c:worker3 2 -Ofair:worker1" 124 125 # Set logging level to DEBUG 126 #CELERYD_LOG_LEVEL="DEBUG" 127 128 # %n will be replaced with the first part of the nodename. 129 CELERYD_LOG_FILE="/var/log/celery/%n%I.log" 130 CELERYD_PID_FILE="/var/run/celery/%n.pid" 131 132 # Workers should run as an unprivileged user. 133 # You need to create this user manually (or you can choose 134 # a user/group combination that already exists (e.g., nobody). 135 CELERYD_USER="celery" 136 CELERYD_GROUP="celery" 137 138 # If enabled pid and log directories will be created if missing, 139 # and owned by the userid/group configured. 140 CELERY_CREATE_DIRS=1 141 142Using a login shell 143~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 144 145You can inherit the environment of the ``CELERYD_USER`` by using a login 146shell: 147 148.. code-block:: bash 149 150 CELERYD_SU_ARGS="-l" 151 152Note that this isn't recommended, and that you should only use this option 153when absolutely necessary. 154 155.. _generic-initd-celeryd-django-example: 156 157Example Django configuration 158~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 159 160Django users now uses the exact same template as above, 161but make sure that the module that defines your Celery app instance 162also sets a default value for :envvar:`DJANGO_SETTINGS_MODULE` 163as shown in the example Django project in :ref:`django-first-steps`. 164 165.. _generic-initd-celeryd-options: 166 167Available options 168~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 169 170* ``CELERY_APP`` 171 172 App instance to use (value for :option:`--app <celery --app>` argument). 173 174* ``CELERY_BIN`` 175 176 Absolute or relative path to the :program:`celery` program. 177 Examples: 178 179 * :file:`celery` 180 * :file:`/usr/local/bin/celery` 181 * :file:`/virtualenvs/proj/bin/celery` 182 * :file:`/virtualenvs/proj/bin/python -m celery` 183 184* ``CELERYD_NODES`` 185 186 List of node names to start (separated by space). 187 188* ``CELERYD_OPTS`` 189 190 Additional command-line arguments for the worker, see 191 `celery worker --help` for a list. This also supports the extended 192 syntax used by `multi` to configure settings for individual nodes. 193 See `celery multi --help` for some multi-node configuration examples. 194 195* ``CELERYD_CHDIR`` 196 197 Path to change directory to at start. Default is to stay in the current 198 directory. 199 200* ``CELERYD_PID_FILE`` 201 202 Full path to the PID file. Default is /var/run/celery/%n.pid 203 204* ``CELERYD_LOG_FILE`` 205 206 Full path to the worker log file. Default is /var/log/celery/%n%I.log 207 **Note**: Using `%I` is important when using the prefork pool as having 208 multiple processes share the same log file will lead to race conditions. 209 210* ``CELERYD_LOG_LEVEL`` 211 212 Worker log level. Default is INFO. 213 214* ``CELERYD_USER`` 215 216 User to run the worker as. Default is current user. 217 218* ``CELERYD_GROUP`` 219 220 Group to run worker as. Default is current user. 221 222* ``CELERY_CREATE_DIRS`` 223 224 Always create directories (log directory and pid file directory). 225 Default is to only create directories when no custom logfile/pidfile set. 226 227* ``CELERY_CREATE_RUNDIR`` 228 229 Always create pidfile directory. By default only enabled when no custom 230 pidfile location set. 231 232* ``CELERY_CREATE_LOGDIR`` 233 234 Always create logfile directory. By default only enable when no custom 235 logfile location set. 236 237.. _generic-initd-celerybeat: 238 239Init-script: ``celerybeat`` 240---------------------------------------------------------------------- 241:Usage: `/etc/init.d/celerybeat {start|stop|restart}` 242:Configuration file: :file:`/etc/default/celerybeat` or 243 :file:`/etc/default/celeryd`. 244 245.. _generic-initd-celerybeat-example: 246 247Example configuration 248~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 249 250This is an example configuration for a Python project: 251 252`/etc/default/celerybeat`: 253 254.. code-block:: bash 255 256 # Absolute or relative path to the 'celery' command: 257 CELERY_BIN="/usr/local/bin/celery" 258 #CELERY_BIN="/virtualenvs/def/bin/celery" 259 260 # App instance to use 261 # comment out this line if you don't use an app 262 CELERY_APP="proj" 263 # or fully qualified: 264 #CELERY_APP="proj.tasks:app" 265 266 # Where to chdir at start. 267 CELERYBEAT_CHDIR="/opt/Myproject/" 268 269 # Extra arguments to celerybeat 270 CELERYBEAT_OPTS="--schedule=/var/run/celery/celerybeat-schedule" 271 272.. _generic-initd-celerybeat-django-example: 273 274Example Django configuration 275~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 276 277You should use the same template as above, but make sure the 278``DJANGO_SETTINGS_MODULE`` variable is set (and exported), and that 279``CELERYD_CHDIR`` is set to the projects directory: 280 281.. code-block:: bash 282 283 export DJANGO_SETTINGS_MODULE="settings" 284 285 CELERYD_CHDIR="/opt/MyProject" 286.. _generic-initd-celerybeat-options: 287 288Available options 289~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 290 291* ``CELERY_APP`` 292 293 App instance to use (value for :option:`--app <celery --app>` argument). 294 295* ``CELERYBEAT_OPTS`` 296 297 Additional arguments to :program:`celery beat`, see 298 :command:`celery beat --help` for a list of available options. 299 300* ``CELERYBEAT_PID_FILE`` 301 302 Full path to the PID file. Default is :file:`/var/run/celeryd.pid`. 303 304* ``CELERYBEAT_LOG_FILE`` 305 306 Full path to the log file. Default is :file:`/var/log/celeryd.log`. 307 308* ``CELERYBEAT_LOG_LEVEL`` 309 310 Log level to use. Default is ``INFO``. 311 312* ``CELERYBEAT_USER`` 313 314 User to run beat as. Default is the current user. 315 316* ``CELERYBEAT_GROUP`` 317 318 Group to run beat as. Default is the current user. 319 320* ``CELERY_CREATE_DIRS`` 321 322 Always create directories (log directory and pid file directory). 323 Default is to only create directories when no custom logfile/pidfile set. 324 325* ``CELERY_CREATE_RUNDIR`` 326 327 Always create pidfile directory. By default only enabled when no custom 328 pidfile location set. 329 330* ``CELERY_CREATE_LOGDIR`` 331 332 Always create logfile directory. By default only enable when no custom 333 logfile location set. 334 335.. _generic-initd-troubleshooting: 336 337Troubleshooting 338---------------------------------------------------------------------- 339 340If you can't get the init-scripts to work, you should try running 341them in *verbose mode*: 342 343.. code-block:: console 344 345 # sh -x /etc/init.d/celeryd start 346 347This can reveal hints as to why the service won't start. 348 349If the worker starts with *"OK"* but exits almost immediately afterwards 350and there's no evidence in the log file, then there's probably an error 351but as the daemons standard outputs are already closed you'll 352not be able to see them anywhere. For this situation you can use 353the :envvar:`C_FAKEFORK` environment variable to skip the 354daemonization step: 355 356.. code-block:: console 357 358 # C_FAKEFORK=1 sh -x /etc/init.d/celeryd start 359 360 361and now you should be able to see the errors. 362 363Commonly such errors are caused by insufficient permissions 364to read from, or write to a file, and also by syntax errors 365in configuration modules, user modules, third-party libraries, 366or even from Celery itself (if you've found a bug you 367should :ref:`report it <reporting-bugs>`). 368 369 370.. _daemon-systemd-generic: 371 372Usage ``systemd`` 373====================================================================== 374 375* `extra/systemd/`_ 376 377.. _`extra/systemd/`: 378 https://github.com/celery/celery/tree/master/extra/systemd/ 379 380.. _generic-systemd-celery: 381 382:Usage: `systemctl {start|stop|restart|status} celery.service` 383:Configuration file: /etc/conf.d/celery 384 385Service file: celery.service 386---------------------------------------------------------------------- 387 388This is an example systemd file: 389 390:file:`/etc/systemd/system/celery.service`: 391 392.. code-block:: bash 393 394 [Unit] 395 Description=Celery Service 396 After=network.target 397 398 [Service] 399 Type=forking 400 User=celery 401 Group=celery 402 EnvironmentFile=/etc/conf.d/celery 403 WorkingDirectory=/opt/celery 404 ExecStart=/bin/sh -c '${CELERY_BIN} multi start ${CELERYD_NODES} \ 405 -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \ 406 --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}' 407 ExecStop=/bin/sh -c '${CELERY_BIN} multi stopwait ${CELERYD_NODES} \ 408 --pidfile=${CELERYD_PID_FILE}' 409 ExecReload=/bin/sh -c '${CELERY_BIN} multi restart ${CELERYD_NODES} \ 410 -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \ 411 --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}' 412 413 [Install] 414 WantedBy=multi-user.target 415 416Once you've put that file in :file:`/etc/systemd/system`, you should run 417:command:`systemctl daemon-reload` in order that Systemd acknowledges that file. 418You should also run that command each time you modify it. 419 420To configure user, group, :command:`chdir` change settings: 421``User``, ``Group``, and ``WorkingDirectory`` defined in 422:file:`/etc/systemd/system/celery.service`. 423 424You can also use systemd-tmpfiles in order to create working directories (for logs and pid). 425 426:file: `/etc/tmpfiles.d/celery.conf` 427 428.. code-block:: bash 429 430 d /var/run/celery 0755 celery celery - 431 d /var/log/celery 0755 celery celery - 432 433 434.. _generic-systemd-celery-example: 435 436Example configuration 437~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 438 439This is an example configuration for a Python project: 440 441:file:`/etc/conf.d/celery`: 442 443.. code-block:: bash 444 445 # Name of nodes to start 446 # here we have a single node 447 CELERYD_NODES="w1" 448 # or we could have three nodes: 449 #CELERYD_NODES="w1 w2 w3" 450 451 # Absolute or relative path to the 'celery' command: 452 CELERY_BIN="/usr/local/bin/celery" 453 #CELERY_BIN="/virtualenvs/def/bin/celery" 454 455 # App instance to use 456 # comment out this line if you don't use an app 457 CELERY_APP="proj" 458 # or fully qualified: 459 #CELERY_APP="proj.tasks:app" 460 461 # How to call manage.py 462 CELERYD_MULTI="multi" 463 464 # Extra command-line arguments to the worker 465 CELERYD_OPTS="--time-limit=300 --concurrency=8" 466 467 # - %n will be replaced with the first part of the nodename. 468 # - %I will be replaced with the current child process index 469 # and is important when using the prefork pool to avoid race conditions. 470 CELERYD_PID_FILE="/var/run/celery/%n.pid" 471 CELERYD_LOG_FILE="/var/log/celery/%n%I.log" 472 CELERYD_LOG_LEVEL="INFO" 473 474 # you may wish to add these options for Celery Beat 475 CELERYBEAT_PID_FILE="/var/run/celery/beat.pid" 476 CELERYBEAT_LOG_FILE="/var/log/celery/beat.log" 477 478Service file: celerybeat.service 479---------------------------------------------------------------------- 480 481This is an example systemd file for Celery Beat: 482 483:file:`/etc/systemd/system/celerybeat.service`: 484 485.. code-block:: bash 486 487 [Unit] 488 Description=Celery Beat Service 489 After=network.target 490 491 [Service] 492 Type=simple 493 User=celery 494 Group=celery 495 EnvironmentFile=/etc/conf.d/celery 496 WorkingDirectory=/opt/celery 497 ExecStart=/bin/sh -c '${CELERY_BIN} beat \ 498 -A ${CELERY_APP} --pidfile=${CELERYBEAT_PID_FILE} \ 499 --logfile=${CELERYBEAT_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL}' 500 501 [Install] 502 WantedBy=multi-user.target 503 504 505Running the worker with superuser privileges (root) 506====================================================================== 507 508Running the worker with superuser privileges is a very dangerous practice. 509There should always be a workaround to avoid running as root. Celery may 510run arbitrary code in messages serialized with pickle - this is dangerous, 511especially when run as root. 512 513By default Celery won't run workers as root. The associated error 514message may not be visible in the logs but may be seen if :envvar:`C_FAKEFORK` 515is used. 516 517To force Celery to run workers as root use :envvar:`C_FORCE_ROOT`. 518 519When running as root without :envvar:`C_FORCE_ROOT` the worker will 520appear to start with *"OK"* but exit immediately after with no apparent 521errors. This problem may appear when running the project in a new development 522or production environment (inadvertently) as root. 523 524.. _daemon-supervisord: 525 526:pypi:`supervisor` 527====================================================================== 528 529* `extra/supervisord/`_ 530 531.. _`extra/supervisord/`: 532 https://github.com/celery/celery/tree/master/extra/supervisord/ 533 534.. _daemon-launchd: 535 536``launchd`` (macOS) 537====================================================================== 538 539* `extra/macOS`_ 540 541.. _`extra/macOS`: 542 https://github.com/celery/celery/tree/master/extra/macOS/ 543