1{ 2 "cells": [ 3 { 4 "cell_type": "markdown", 5 "metadata": {}, 6 "source": [ 7 "# Capturing C-level stdout/stderr with `wurlitzer`\n", 8 "\n", 9 "Sometimes in Python you are calling some C code.\n", 10 "Sometimes that C code makes calls to `printf`,\n", 11 "or otherwise writes to the stdout/stderr of the process." 12 ] 13 }, 14 { 15 "cell_type": "code", 16 "execution_count": 1, 17 "metadata": { 18 "collapsed": false 19 }, 20 "outputs": [], 21 "source": [ 22 "import ctypes\n", 23 "libc = ctypes.CDLL(None)\n", 24 "\n", 25 "try:\n", 26 " c_stderr_p = ctypes.c_void_p.in_dll(libc, 'stderr')\n", 27 "except ValueError:\n", 28 " # libc.stdout is has a funny name on OS X\n", 29 " c_stderr_p = ctypes.c_void_p.in_dll(libc, '__stderrp')\n", 30 "\n", 31 "\n", 32 "def printf(msg):\n", 33 " \"\"\"Call C printf\"\"\"\n", 34 " libc.printf((msg + '\\n').encode('utf8'))\n", 35 "\n", 36 "def printf_err(msg):\n", 37 " \"\"\"Cal C fprintf on stderr\"\"\"\n", 38 " libc.fprintf(c_stderr_p, (msg + '\\n').encode('utf8'))" 39 ] 40 }, 41 { 42 "cell_type": "markdown", 43 "metadata": {}, 44 "source": [ 45 "IPython forwards the Python-level `sys.stdout` and `sys.stderr`,\n", 46 "but it leaves the process-level file descriptors that C code will write to untouched.\n", 47 "That means that in a context like this notebook, these functions will print to the terminal, because they are not captured:" 48 ] 49 }, 50 { 51 "cell_type": "code", 52 "execution_count": 2, 53 "metadata": { 54 "collapsed": false 55 }, 56 "outputs": [], 57 "source": [ 58 "printf(\"Hello?\")\n", 59 "printf_err(\"Stderr? Anybody?\")" 60 ] 61 }, 62 { 63 "cell_type": "markdown", 64 "metadata": {}, 65 "source": [ 66 "With wurlitzer, we can capture these C-level functions:" 67 ] 68 }, 69 { 70 "cell_type": "code", 71 "execution_count": 3, 72 "metadata": { 73 "collapsed": true 74 }, 75 "outputs": [], 76 "source": [ 77 "from wurlitzer import pipes, sys_pipes, STDOUT, PIPE" 78 ] 79 }, 80 { 81 "cell_type": "code", 82 "execution_count": 4, 83 "metadata": { 84 "collapsed": true 85 }, 86 "outputs": [], 87 "source": [ 88 "with pipes() as (stdout, stderr):\n", 89 " printf(\"Hello, stdout!\")\n", 90 " printf_err(\"Hello, stderr!\")" 91 ] 92 }, 93 { 94 "cell_type": "markdown", 95 "metadata": {}, 96 "source": [ 97 "and redisplay them if we like:" 98 ] 99 }, 100 { 101 "cell_type": "code", 102 "execution_count": 5, 103 "metadata": { 104 "collapsed": false 105 }, 106 "outputs": [ 107 { 108 "name": "stdout", 109 "output_type": "stream", 110 "text": [ 111 "Hello, stdout!\n" 112 ] 113 }, 114 { 115 "name": "stderr", 116 "output_type": "stream", 117 "text": [ 118 "Hello, stderr!\n" 119 ] 120 } 121 ], 122 "source": [ 123 "import sys\n", 124 "sys.stdout.write(stdout.read())\n", 125 "sys.stderr.write(stderr.read())" 126 ] 127 }, 128 { 129 "cell_type": "markdown", 130 "metadata": {}, 131 "source": [ 132 "Some tools, such as the IPython kernel for Jupyter,\n", 133 "capture the Python-level `sys.stdout` and `sys.stderr` and forward them somewhere.\n", 134 "In the case of Jupyter, this is over a network socket, so that it ends up in the browser.\n", 135 "\n", 136 "If we know that's going on, we can easily hook up the C outputs to the Python-forwarded ones with a single call:" 137 ] 138 }, 139 { 140 "cell_type": "code", 141 "execution_count": 6, 142 "metadata": { 143 "collapsed": false 144 }, 145 "outputs": [ 146 { 147 "name": "stdout", 148 "output_type": "stream", 149 "text": [ 150 "Hello from C, 0!\n", 151 "Hello from C, 1!\n", 152 "Hello from C, 2!\n", 153 "Hello from C, 3!\n", 154 "Hello from C, 4!\n" 155 ] 156 } 157 ], 158 "source": [ 159 "import time\n", 160 "\n", 161 "with sys_pipes():\n", 162 " for i in range(5):\n", 163 " time.sleep(1)\n", 164 " printf(\"Hello from C, %i!\" % i)\n" 165 ] 166 }, 167 { 168 "cell_type": "markdown", 169 "metadata": {}, 170 "source": [ 171 "We can also capture the pipes to any writeable streams, such as a `StringIO` object:" 172 ] 173 }, 174 { 175 "cell_type": "code", 176 "execution_count": 7, 177 "metadata": { 178 "collapsed": false 179 }, 180 "outputs": [ 181 { 182 "name": "stdout", 183 "output_type": "stream", 184 "text": [ 185 "Hello, stdout!\n", 186 "Hello, stderr!\n", 187 "\n" 188 ] 189 } 190 ], 191 "source": [ 192 "import io\n", 193 "\n", 194 "stdout = io.StringIO()\n", 195 "with pipes(stdout=stdout, stderr=STDOUT):\n", 196 " printf(\"Hello, stdout!\")\n", 197 " printf_err(\"Hello, stderr!\")\n", 198 "\n", 199 "print(stdout.getvalue())" 200 ] 201 }, 202 { 203 "cell_type": "markdown", 204 "metadata": {}, 205 "source": [ 206 "## IPython extension\n", 207 "\n", 208 "You can also enable wurlitzer as an IPython extension,\n", 209 "so that it always forwards C-level output during execution:" 210 ] 211 }, 212 { 213 "cell_type": "code", 214 "execution_count": 8, 215 "metadata": { 216 "collapsed": true 217 }, 218 "outputs": [], 219 "source": [ 220 "%load_ext wurlitzer" 221 ] 222 }, 223 { 224 "cell_type": "code", 225 "execution_count": 9, 226 "metadata": { 227 "collapsed": false 228 }, 229 "outputs": [ 230 { 231 "name": "stdout", 232 "output_type": "stream", 233 "text": [ 234 "Hello from C, 0!\n", 235 "Hello from C, 1!\n", 236 "Hello from C, 2!\n", 237 "Hello from C, 3!\n", 238 "Hello from C, 4!\n" 239 ] 240 } 241 ], 242 "source": [ 243 "for i in range(5):\n", 244 " time.sleep(1)\n", 245 " printf(\"Hello from C, %i!\" % i)" 246 ] 247 } 248 ], 249 "metadata": { 250 "kernelspec": { 251 "display_name": "Python 3", 252 "language": "python", 253 "name": "python3" 254 }, 255 "language_info": { 256 "codemirror_mode": { 257 "name": "ipython", 258 "version": 3 259 }, 260 "file_extension": ".py", 261 "mimetype": "text/x-python", 262 "name": "python", 263 "nbconvert_exporter": "python", 264 "pygments_lexer": "ipython3", 265 "version": "3.5.1" 266 } 267 }, 268 "nbformat": 4, 269 "nbformat_minor": 1 270} 271