1================================
2Interaction with remote programs
3================================
4
5Fabric's primary operations, `~fabric.operations.run` and
6`~fabric.operations.sudo`, are capable of sending local input to the remote
7end, in a manner nearly identical to the ``ssh`` program. For example, programs
8which display password prompts (e.g. a database dump utility, or changing a
9user's password) will behave just as if you were interacting with them
10directly.
11
12However, as with ``ssh`` itself, Fabric's implementation of this feature is
13subject to a handful of limitations which are not always intuitive. This
14document discusses such issues in detail.
15
16.. note::
17    Readers unfamiliar with the basics of Unix stdout and stderr pipes, and/or
18    terminal devices, may wish to visit the Wikipedia pages for `Unix pipelines
19    <http://en.wikipedia.org/wiki/Pipe_(Unix)>`_ and `Pseudo terminals
20    <http://en.wikipedia.org/wiki/Pseudo_terminal>`_ respectively.
21
22
23.. _combine_streams:
24
25Combining stdout and stderr
26===========================
27
28The first issue to be aware of is that of the stdout and stderr streams, and
29why they are separated or combined as needed.
30
31Buffering
32---------
33
34Fabric 0.9.x and earlier, and Python itself, buffer output on a line-by-line
35basis: text is not printed to the user until a newline character is found.
36This works fine in most situations but becomes problematic when one needs to
37deal with partial-line output such as prompts.
38
39.. note::
40    Line-buffered output can make programs appear to halt or freeze for no
41    reason, as prompts print out text without a newline, waiting for the user
42    to enter their input and press Return.
43
44Newer Fabric versions buffer both input and output on a character-by-character
45basis in order to make interaction with prompts possible. This has the
46convenient side effect of enabling interaction with complex programs utilizing
47the "curses" libraries or which otherwise redraw the screen (think ``top``).
48
49Crossing the streams
50--------------------
51
52Unfortunately, printing to stderr and stdout simultaneously (as many programs
53do) means that when the two streams are printed independently one byte at a
54time, they can become garbled or meshed together. While this can sometimes be
55mitigated by line-buffering one of the streams and not the other, it's still a
56serious issue.
57
58To solve this problem, Fabric uses a setting in our SSH layer which merges the
59two streams at a low level and causes output to appear more naturally. This
60setting is represented in Fabric as the :ref:`combine-stderr` env var and
61keyword argument, and is ``True`` by default.
62
63Due to this default setting, output will appear correctly, but at the
64cost of an empty ``.stderr`` attribute on the return values of
65`~fabric.operations.run`/`~fabric.operations.sudo`, as all output will appear
66to be stdout.
67
68Conversely, users requiring a distinct stderr stream at the Python level and
69who aren't bothered by garbled user-facing output (or who are hiding stdout and
70stderr from the command in question) may opt to set this to ``False`` as
71needed.
72
73
74.. _pseudottys:
75
76Pseudo-terminals
77================
78
79The other main issue to consider when presenting interactive prompts to users
80is that of echoing the user's own input.
81
82Echoes
83------
84
85Typical terminal applications or bona fide text terminals (e.g. when using a
86Unix system without a running GUI) present programs with a terminal device
87called a tty or pty (for pseudo-terminal). These automatically echo all text
88typed into them back out to the user (via stdout), as interaction without
89seeing what you had just typed would be difficult. Terminal devices are also
90able to conditionally turn off echoing, allowing secure password prompts.
91
92However, it's possible for programs to be run without a tty or pty present at
93all (consider cron jobs, for example) and in this situation, any stdin data
94being fed to the program won't be echoed. This is desirable for programs being
95run without any humans around, and it's also Fabric's old default mode of
96operation.
97
98Fabric's approach
99-----------------
100
101Unfortunately, in the context of executing commands via Fabric, when no pty is
102present to echo a user's stdin, Fabric must echo it for them. This is
103sufficient for many applications, but it presents problems for password
104prompts, which become insecure.
105
106In the interests of security and meeting the principle of least surprise
107(insofar as users are typically expecting things to behave as they would when
108run in a terminal emulator), Fabric 1.0 and greater force a pty by default.
109With a pty enabled, Fabric simply allows the remote end to handle echoing or
110hiding of stdin and does not echo anything itself.
111
112.. note::
113    In addition to allowing normal echo behavior, a pty also means programs
114    that behave differently when attached to a terminal device will then do so.
115    For example, programs that colorize output on terminals but not when run in
116    the background will print colored output. Be wary of this if you inspect
117    the return value of `~fabric.operations.run` or `~fabric.operations.sudo`!
118
119For situations requiring the pty behavior turned off, the :option:`--no-pty`
120command-line argument and :ref:`always-use-pty` env var may be used.
121
122
123Combining the two
124=================
125
126As a final note, keep in mind that use of pseudo-terminals effectively implies
127combining stdout and stderr -- in much the same way as the :ref:`combine_stderr
128<combine_streams>` setting does. This is because a terminal device naturally
129sends both stdout and stderr to the same place -- the user's display -- thus
130making it impossible to differentiate between them.
131
132However, at the Fabric level, the two groups of settings are distinct from one
133another and may be combined in various ways. The default is for both to be set
134to ``True``; the other combinations are as follows:
135
136* ``run("cmd", pty=False, combine_stderr=True)``: will cause Fabric to echo all
137  stdin itself, including passwords, as well as potentially altering ``cmd``'s
138  behavior. Useful if ``cmd`` behaves undesirably when run under a pty and
139  you're not concerned about password prompts.
140* ``run("cmd", pty=False, combine_stderr=False)``: with both settings
141  ``False``, Fabric will echo stdin and won't issue a pty -- and this is highly
142  likely to result in undesired behavior for all but the simplest commands.
143  However, it is also the only way to access a distinct stderr stream, which is
144  occasionally useful.
145* ``run("cmd", pty=True, combine_stderr=False)``: valid, but won't really make
146  much of a difference, as ``pty=True`` will still result in merged streams.
147  May be useful for avoiding any edge case problems in ``combine_stderr`` (none
148  are presently known).
149