1{
2 "cells": [
3  {
4   "cell_type": "markdown",
5   "metadata": {},
6   "source": [
7    "# Capturing C-level stdout/stderr with `wurlitzer`\n",
8    "\n",
9    "Sometimes in Python you are calling some C code.\n",
10    "Sometimes that C code makes calls to `printf`,\n",
11    "or otherwise writes to the stdout/stderr of the process."
12   ]
13  },
14  {
15   "cell_type": "code",
16   "execution_count": 1,
17   "metadata": {
18    "collapsed": false
19   },
20   "outputs": [],
21   "source": [
22    "import ctypes\n",
23    "libc = ctypes.CDLL(None)\n",
24    "\n",
25    "try:\n",
26    "    c_stderr_p = ctypes.c_void_p.in_dll(libc, 'stderr')\n",
27    "except ValueError:\n",
28    "    # libc.stdout is has a funny name on OS X\n",
29    "    c_stderr_p = ctypes.c_void_p.in_dll(libc, '__stderrp')\n",
30    "\n",
31    "\n",
32    "def printf(msg):\n",
33    "    \"\"\"Call C printf\"\"\"\n",
34    "    libc.printf((msg + '\\n').encode('utf8'))\n",
35    "\n",
36    "def printf_err(msg):\n",
37    "    \"\"\"Cal C fprintf on stderr\"\"\"\n",
38    "    libc.fprintf(c_stderr_p, (msg + '\\n').encode('utf8'))"
39   ]
40  },
41  {
42   "cell_type": "markdown",
43   "metadata": {},
44   "source": [
45    "IPython forwards the Python-level `sys.stdout` and `sys.stderr`,\n",
46    "but it leaves the process-level file descriptors that C code will write to untouched.\n",
47    "That means that in a context like this notebook, these functions will print to the terminal, because they are not captured:"
48   ]
49  },
50  {
51   "cell_type": "code",
52   "execution_count": 2,
53   "metadata": {
54    "collapsed": false
55   },
56   "outputs": [],
57   "source": [
58    "printf(\"Hello?\")\n",
59    "printf_err(\"Stderr? Anybody?\")"
60   ]
61  },
62  {
63   "cell_type": "markdown",
64   "metadata": {},
65   "source": [
66    "With wurlitzer, we can capture these C-level functions:"
67   ]
68  },
69  {
70   "cell_type": "code",
71   "execution_count": 3,
72   "metadata": {
73    "collapsed": true
74   },
75   "outputs": [],
76   "source": [
77    "from wurlitzer import pipes, sys_pipes, STDOUT, PIPE"
78   ]
79  },
80  {
81   "cell_type": "code",
82   "execution_count": 4,
83   "metadata": {
84    "collapsed": true
85   },
86   "outputs": [],
87   "source": [
88    "with pipes() as (stdout, stderr):\n",
89    "    printf(\"Hello, stdout!\")\n",
90    "    printf_err(\"Hello, stderr!\")"
91   ]
92  },
93  {
94   "cell_type": "markdown",
95   "metadata": {},
96   "source": [
97    "and redisplay them if we like:"
98   ]
99  },
100  {
101   "cell_type": "code",
102   "execution_count": 5,
103   "metadata": {
104    "collapsed": false
105   },
106   "outputs": [
107    {
108     "name": "stdout",
109     "output_type": "stream",
110     "text": [
111      "Hello, stdout!\n"
112     ]
113    },
114    {
115     "name": "stderr",
116     "output_type": "stream",
117     "text": [
118      "Hello, stderr!\n"
119     ]
120    }
121   ],
122   "source": [
123    "import sys\n",
124    "sys.stdout.write(stdout.read())\n",
125    "sys.stderr.write(stderr.read())"
126   ]
127  },
128  {
129   "cell_type": "markdown",
130   "metadata": {},
131   "source": [
132    "Some tools, such as the IPython kernel for Jupyter,\n",
133    "capture the Python-level `sys.stdout` and `sys.stderr` and forward them somewhere.\n",
134    "In the case of Jupyter, this is over a network socket, so that it ends up in the browser.\n",
135    "\n",
136    "If we know that's going on, we can easily hook up the C outputs to the Python-forwarded ones with a single call:"
137   ]
138  },
139  {
140   "cell_type": "code",
141   "execution_count": 6,
142   "metadata": {
143    "collapsed": false
144   },
145   "outputs": [
146    {
147     "name": "stdout",
148     "output_type": "stream",
149     "text": [
150      "Hello from C, 0!\n",
151      "Hello from C, 1!\n",
152      "Hello from C, 2!\n",
153      "Hello from C, 3!\n",
154      "Hello from C, 4!\n"
155     ]
156    }
157   ],
158   "source": [
159    "import time\n",
160    "\n",
161    "with sys_pipes():\n",
162    "    for i in range(5):\n",
163    "        time.sleep(1)\n",
164    "        printf(\"Hello from C, %i!\" % i)\n"
165   ]
166  },
167  {
168   "cell_type": "markdown",
169   "metadata": {},
170   "source": [
171    "We can also capture the pipes to any writeable streams, such as a `StringIO` object:"
172   ]
173  },
174  {
175   "cell_type": "code",
176   "execution_count": 7,
177   "metadata": {
178    "collapsed": false
179   },
180   "outputs": [
181    {
182     "name": "stdout",
183     "output_type": "stream",
184     "text": [
185      "Hello, stdout!\n",
186      "Hello, stderr!\n",
187      "\n"
188     ]
189    }
190   ],
191   "source": [
192    "import io\n",
193    "\n",
194    "stdout = io.StringIO()\n",
195    "with pipes(stdout=stdout, stderr=STDOUT):\n",
196    "    printf(\"Hello, stdout!\")\n",
197    "    printf_err(\"Hello, stderr!\")\n",
198    "\n",
199    "print(stdout.getvalue())"
200   ]
201  },
202  {
203   "cell_type": "markdown",
204   "metadata": {},
205   "source": [
206    "## IPython extension\n",
207    "\n",
208    "You can also enable wurlitzer as an IPython extension,\n",
209    "so that it always forwards C-level output during execution:"
210   ]
211  },
212  {
213   "cell_type": "code",
214   "execution_count": 8,
215   "metadata": {
216    "collapsed": true
217   },
218   "outputs": [],
219   "source": [
220    "%load_ext wurlitzer"
221   ]
222  },
223  {
224   "cell_type": "code",
225   "execution_count": 9,
226   "metadata": {
227    "collapsed": false
228   },
229   "outputs": [
230    {
231     "name": "stdout",
232     "output_type": "stream",
233     "text": [
234      "Hello from C, 0!\n",
235      "Hello from C, 1!\n",
236      "Hello from C, 2!\n",
237      "Hello from C, 3!\n",
238      "Hello from C, 4!\n"
239     ]
240    }
241   ],
242   "source": [
243    "for i in range(5):\n",
244    "    time.sleep(1)\n",
245    "    printf(\"Hello from C, %i!\" % i)"
246   ]
247  }
248 ],
249 "metadata": {
250  "kernelspec": {
251   "display_name": "Python 3",
252   "language": "python",
253   "name": "python3"
254  },
255  "language_info": {
256   "codemirror_mode": {
257    "name": "ipython",
258    "version": 3
259   },
260   "file_extension": ".py",
261   "mimetype": "text/x-python",
262   "name": "python",
263   "nbconvert_exporter": "python",
264   "pygments_lexer": "ipython3",
265   "version": "3.5.1"
266  }
267 },
268 "nbformat": 4,
269 "nbformat_minor": 1
270}
271