1Using subprocesses 2================== 3 4.. py:currentmodule:: anyio 5 6AnyIO allows you to run arbitrary executables in subprocesses, either as a one-shot call or by 7opening a process handle for you that gives you more control over the subprocess. 8 9You can either give the command as a string, in which case it is passed to your default shell 10(equivalent to ``shell=True`` in :func:`subprocess.run`), or as a sequence of strings 11(``shell=False``) in which the executable is the first item in the sequence and the rest are 12arguments passed to it. 13 14.. note:: On Windows and Python 3.7 and earlier, asyncio uses :class:`~asyncio.SelectorEventLoop` 15 by default which does not support subprocesses. It is recommended to upgrade to at least Python 16 3.8 to overcome this limitation. 17 18Running one-shot commands 19------------------------- 20 21To run an external command with one call, use :func:`~run_process`:: 22 23 from anyio import run_process, run 24 25 26 async def main(): 27 result = await run_process('ps') 28 print(result.stdout.decode()) 29 30 run(main) 31 32The snippet above runs the ``ps`` command within a shell (. To run it directly:: 33 34 from anyio import run_process, run 35 36 37 async def main(): 38 result = await run_process(['ps']) 39 print(result.stdout.decode()) 40 41 run(main) 42 43Working with processes 44---------------------- 45 46When you have more complex requirements for your interaction with subprocesses, you can launch one 47with :func:`~open_process`:: 48 49 from anyio import open_process, run 50 from anyio.streams.text import TextReceiveStream 51 52 53 async def main(): 54 async with await open_process(['ps']) as process: 55 async for text in TextReceiveStream(process.stdout): 56 print(text) 57 58 run(main) 59 60See the API documentation of :class:`~.abc.Process` for more information. 61 62.. _RunInProcess: 63 64Running functions in worker processes 65------------------------------------- 66 67When you need to run CPU intensive code, worker processes are better than threads because current 68implementations of Python cannot run Python code in multiple threads at once. 69 70Exceptions to this rule are: 71 72#. Blocking I/O operations 73#. C extension code that explicitly releases the Global Interpreter Lock 74 75If the code you wish to run does not belong in this category, it's best to use worker processes 76instead in order to take advantage of multiple CPU cores. 77This is done by using :func:`.to_process.run_sync`:: 78 79 import time 80 81 from anyio import run, to_process 82 83 84 def cpu_intensive_function(arg1, arg2): 85 time.sleep(1) 86 return arg1 + arg2 87 88 async def main(): 89 result = await to_process.run_sync(cpu_intensive_function, 'Hello, ', 'world!') 90 print(result) 91 92 # This check is important when the application uses run_sync_in_process() 93 if __name__ == '__main__': 94 run(main) 95 96Technical details 97***************** 98 99There are some limitations regarding the arguments and return values passed: 100 101* the arguments must be pickleable (using the highest available protocol) 102* the return value must be pickleable (using the highest available protocol) 103* the target callable must be importable (lambdas and inner functions won't work) 104 105Other considerations: 106 107* Even "cancellable=False" runs can be cancelled before the request has been sent to the worker process 108* If a cancellable call is cancelled during execution on the worker process, the worker process 109 will be killed 110* The worker process imports the parent's ``__main__`` module, so guarding for any import time side 111 effects using ``if __name__ == '__main__':`` is required to avoid infinite recursion 112* ``sys.stdin`` and ``sys.stdout``, ``sys.stderr`` are redirected to ``/dev/null`` so :func:`print` 113 and :func:`input` won't work 114* Worker processes terminate after 5 minutes of inactivity, or when the event loop is finished 115 116 * On asyncio, either :func:`asyncio.run` or :func:`anyio.run` must be used for proper cleanup 117 to happen 118* Multiprocessing-style synchronization primitives are currently not available 119