1Using subprocesses
2==================
3
4.. py:currentmodule:: anyio
5
6AnyIO allows you to run arbitrary executables in subprocesses, either as a one-shot call or by
7opening a process handle for you that gives you more control over the subprocess.
8
9You can either give the command as a string, in which case it is passed to your default shell
10(equivalent to ``shell=True`` in :func:`subprocess.run`), or as a sequence of strings
11(``shell=False``) in which the executable is the first item in the sequence and the rest are
12arguments passed to it.
13
14.. note:: On Windows and Python 3.7 and earlier, asyncio uses :class:`~asyncio.SelectorEventLoop`
15    by default which does not support subprocesses. It is recommended to upgrade to at least Python
16    3.8 to overcome this limitation.
17
18Running one-shot commands
19-------------------------
20
21To run an external command with one call, use :func:`~run_process`::
22
23    from anyio import run_process, run
24
25
26    async def main():
27        result = await run_process('ps')
28        print(result.stdout.decode())
29
30    run(main)
31
32The snippet above runs the ``ps`` command within a shell (. To run it directly::
33
34    from anyio import run_process, run
35
36
37    async def main():
38        result = await run_process(['ps'])
39        print(result.stdout.decode())
40
41    run(main)
42
43Working with processes
44----------------------
45
46When you have more complex requirements for your interaction with subprocesses, you can launch one
47with :func:`~open_process`::
48
49    from anyio import open_process, run
50    from anyio.streams.text import TextReceiveStream
51
52
53    async def main():
54        async with await open_process(['ps']) as process:
55            async for text in TextReceiveStream(process.stdout):
56                print(text)
57
58    run(main)
59
60See the API documentation of :class:`~.abc.Process` for more information.
61
62.. _RunInProcess:
63
64Running functions in worker processes
65-------------------------------------
66
67When you need to run CPU intensive code, worker processes are better than threads because current
68implementations of Python cannot run Python code in multiple threads at once.
69
70Exceptions to this rule are:
71
72#. Blocking I/O operations
73#. C extension code that explicitly releases the Global Interpreter Lock
74
75If the code you wish to run does not belong in this category, it's best to use worker processes
76instead in order to take advantage of multiple CPU cores.
77This is done by using :func:`.to_process.run_sync`::
78
79    import time
80
81    from anyio import run, to_process
82
83
84    def cpu_intensive_function(arg1, arg2):
85        time.sleep(1)
86        return arg1 + arg2
87
88    async def main():
89        result = await to_process.run_sync(cpu_intensive_function, 'Hello, ', 'world!')
90        print(result)
91
92    # This check is important when the application uses run_sync_in_process()
93    if __name__ == '__main__':
94        run(main)
95
96Technical details
97*****************
98
99There are some limitations regarding the arguments and return values passed:
100
101* the arguments must be pickleable (using the highest available protocol)
102* the return value must be pickleable (using the highest available protocol)
103* the target callable must be importable (lambdas and inner functions won't work)
104
105Other considerations:
106
107* Even "cancellable=False" runs can be cancelled before the request has been sent to the worker process
108* If a cancellable call is cancelled during execution on the worker process, the worker process
109  will be killed
110* The worker process imports the parent's ``__main__`` module, so guarding for any import time side
111  effects using ``if __name__ == '__main__':`` is required to avoid infinite recursion
112* ``sys.stdin`` and ``sys.stdout``, ``sys.stderr`` are redirected to ``/dev/null`` so :func:`print`
113  and :func:`input` won't work
114* Worker processes terminate after 5 minutes of inactivity, or when the event loop is finished
115
116  * On asyncio, either :func:`asyncio.run` or :func:`anyio.run` must be used for proper cleanup
117    to happen
118* Multiprocessing-style synchronization primitives are currently not available
119