1.. preppy documentation master file, created by 2 sphinx-quickstart on Thu Mar 14 21:19:20 2013. 3 You can adapt this file completely to your liking, but it should at least 4 contain the root `toctree` directive. 5 6Preppy - a PREProcessor for PYthon 7================================== 8 9Preppy is ReportLab's templating system. It was developed in late 2000 and has 10been in continual production use since then. It is open source (BSD-license). 11 12The key features are: 13 14* *small*. Preppy is a single Python module. If you want a templating system 'in the box', it's easy to include it in your project 15* *easy to learn*. It takes about one minute to scan all the features 16* *just Python*. We have not invented another language, and if you want to do something - includes, quoting, filters - you just use Python 17* *compiled to bytecode*: a .prep file gets compiled to a Python function in a .pyc file 18* *easy to debug*: preppy generates proper Python exceptions, with the correct line numbers for the .prep file. You can follow tracebacks from Python script to Preppy template and back, through multiple includes 19* *easy to type and read*. We've been using ``{{this}}`` syntax since well before Django was thought of 20* *8-bit safe*: it makes no assumption that you are generating markup and does nothing unexpected with whitespace; you could use it to generate images or binary files if you wanted to. 21 22Why another templating system? 23============================== 24 25Since there are dozens of templating systems out there, it's worth explaining 26why this one exists. 27 28We developed preppy in 2000. At the time, every python developer and his dog 29was busy inventing their own web framework, usually including a templating 30system for web development. 31 32Most of these involved coming up with an unreliable parser, and an incomplete 33'little language' for embedding in markup. We took the view 34that we already had a perfectly good little language - Python - and there was 35no need to invent a new one. Preppy has met all of our needs well, with minimal 36changes. 37 38Our main product is a markup-based system for creating PDF documents, Report 39Markup Language. When we create reports for people, we use a template 40to generate an RML file, in the same way that most people make web pages. We 41need a templating system for our own demos and tutorials, as well as solutions 42we build, which is 'in the box' and easy to understand. We also build web 43applications, and when we want something strict and minimal, we use preppy. 44 45Moving on, most major web frameworks have settled on a templating system. The 46most popular ones, such as Django or Jinja, are very full-featured. They have 47focused on creating a new language for web designers. They are getting fairly 48big and complex and they impose their own learning curve. They also result in 49template programming becoming a different skill to the main application, and 50sometimes with a different philosophy. For example, in the Django world, 51templates are supposed to be coded by designers who lack programming skills, 52and the system is deliberately forgiving of errors. In our experience, this 53laxity leads to more subtle problems a few days later on in development. 54 55Within ReportLab we take the view that a template is code; it is expected to 56be correct, and to raise errors when misspelled or nonexistent variables are 57used. It is far easier to create a robust system when the templates use 58the same language as the rest of the application, and are debugged in exactly 59the same way, as the application code. 60 61Later on, we'll show how many features of high-end templating systems are handled 62with short Python expressions. 63 64 65Quick Start 66=========== 67 68Preppy can be installed from ``easy_install``, ``pip``, by downloading and using ``setup.py install`, by cloning from bitbucket. Or, because it's just a single Python module, you can grab it and put it on your path, or check it into your application. 69 70 71If you're too busy to read the whole page, the next few lines should get you going. First, get preppy on your path through one of the above methods. 72 73Place your template code in a file which, by convention, ends in .prep. Here is an example: 74 75.. code-block:: django 76 77 {{def(name, sex)}} 78 <html><head><title>{{name}}</title></head><body> 79 hello my name is {{name}} and I am 80 {{if sex=="f":}} a gal 81 {{elif sex=="m":}} a guy 82 {{else}} neuter {{endif}} 83 </body></html> 84 85If this is in a file called, say, ``template.prep``, you can invoke it like this:: 86 87 mymodule = preppy.getModule('template.prep') 88 name = 'fred' 89 sex = 'm' 90 html = mymodule.get(name, sex) 91 92``getModule`` returns a Python module object. On first call, or when templates are edited, this will generate a .pyc file, just as a Python module does (assuming you have write access to the disk). This module contains a function, ``get``, which implements your template. The function accepts the parameters which you specified at the top of the prep file, and returns a (non-Unicode, 8-bit) string. 93 94This is preppy's key design idea: we try to make a preppy template as much like a python function as possible. 95 96 97Template syntax 98=============== 99Preppy has a small number of tags, enclosed in pairs of curly braces: 100 101.. describe:: {{def(YOUR_ARGUMENT_LIST}} 102 103This is a special construct which must be placed near the top of a .prep file. If used, it should be the first preppy directive. It allows you to explicitly declare the parameters which will be passed in. You can do most of the things you can do with a Python function: positional arguments, keyword arguments, and ``\*args`` and ``\*\*kwds``. The only difference is that there is no function name. We compile this into a Python function named :func:`get`. 104 105There is an older way of using preppy, which omits this declaration, and lets you pass an arbitrary dictionary of arguments to the template, like most other templating systems. This should be regarded as deprecated and will be omitted in version 2, both for usability reasons, and for technical reasons to do with bytecode generation as we move to Python 3.x. I 106 107We do it this way because, in over a decade working with preppy, we have found that it's very valuable to have an explicit declaration at the top of a .prep file telling you what variables are in the namespace. Otherwise, on large projects, one ends up constantly referring to the calling code in another module. 108 109 110.. describe:: {{expression}} 111 112 113Any Python expression will be evaluated in the current namespace, and thence converted to a string representation. Examples: 114 115.. code-block:: django 116 117 The total is {{2+2}} 118 119 {{range(10)}} 120 121 Dear {{client.name}}, 122 123By default, the expression is converted by Python's ``str()`` function. So the python value ``None`` will appear as the text ``None`` in the output, and any non-ascii characters in a string will trigger an exception. In each application, you have the option to define your own **quoting functions** to use instead, which we discuss below. 124 125 126 127.. describe:: {{eval}} 128 129This is exactly equivalent to ``{{expression}}``, but is useful when you have a long Python expression which spans several lines, or the extra curly braces on the same line as the expression harm readability. For example:: 130 131 {{eval}} 132 a_complex("and", "very", "verbose", function="call") 133 {{endeval}} 134 135 136.. describe:: {{script}}....{{endscript}} 137 138Multiple or single lines of python scripts may be embedded within ``{{script}}...{{endscript}}`` tags. Examples:: 139 140 {{script}}import urllib2{{endscript}} 141 142 {{script}} 143 cur = conn.cursor() 144 cur.execute('select * from some_table') 145 data = cur.fetchall() 146 {{endscript}} 147 148For ``expression``, ``eval``, and ``script``, any newlines in the code text 149will be automatically indented to the proper indentation level for 150the run() module at that insertion point. You may therefore indent your 151code block to match the indentation level of any HTML/XML it is embedded in. 152This is only a concern for triple quoted strings. If this may be an issue, don't 153use triple quoted strings in preppy source. Instead of:: 154 155 x = """ 156 a string 157 """ 158 159use:: 160 161 x = ("\n" 162 "\ta string\n" 163 ) 164 165or something similar. 166 167 168It is generally bad practice to have too much in script tags. If you find yourself 169writing long script sections to fetch and prepare 170data or performing calculations, it is much better to place those things 171in a separate python module, import it within the template, and call 172those functions in one line. 173 174 175 176.. describe:: {{if EXPR}}...{{elif EXPR}}...{{else}}...{{endif}} 177 178The *{{if}}*` statement does exactly what Python's *if* statement does. You may optionally use multiple *elif* clauses and one *else* clause. The final colon after each clause ("*else:*") is optional. 179 180 181 182 183.. describe:: {{for EXPR}}...{{else}}...{{endfor}} 184 185 186This implements a for loop in preppy source. The EXPR should follow 187normal python conventions for python for loops. The resulting python 188code is roughly:: 189 190 for EXPR: 191 interpretation_of(block) 192 else: 193 no break exit 194 195An example:: 196 197 {{for (id, name) in dataset}} 198 <tr><td>{{id}}</td><td>{{name}}</td> 199 {{endfor}} 200 201 202.. describe:: {{while CONDITION}}...{{else}}...{{endwhile}} 203 204This implements a *while* loop in preppy source. The condition should be 205a python expression. The resulting python code is roughly:: 206 207 while CONDITION: 208 interpretation_of(block) 209 else: 210 .... 211 212.. describe:: {{try}}...{{except X}}...{{else}}...{{finally}}...{{endtry}} 213 214The normal python *try* in preppy form. The the else clause is accepted only if 215an except clause is seen. 216 217.. describe:: {{with open('aaa') as x}}...{{endwith}} 218 219As in python the contained block knows about x and handles finalization etc etc. 220 221.. describe:: {{raise Exception}} 222 223This allows raising an exception without using {{script}} {{endscript}}. 224 225.. describe:: {{continue}} 226 227Continues to next iteration without requiring {{script}} {{endscript}}. Only allowed inside a loop. 228 229.. describe:: {{break}} 230 231Breaks from a loop without requiring {{script}} {{endscript}}. Only allowed inside a loop. 232 233.. describe:: {{import module}} or {{import module as x}} or {{from module import x}} etc. 234 235The normal python import statements. 236 237.. describe:: {{assert condition, "....message"}} 238 239The python assert statement. 240 241Module import options 242===================== 243There are two ways to load a preppy module into memory. We refer to these as 'file system semantics' and 'import semantics'. 244 245File system semantics 246--------------------- 247 248The file system method is implemented by :func:`getModule`: 249 250.. function:: getModule(name, directory=".", source_extension=".prep", verbose=0, savefile=None, sourcetext=None, savePy=0, force=0, savePyc=1, importModule=1,_globals=None) 251 252This loads your template, which is a Python module object. 253 254There is no predefined search path or list of template directories; if you want to implement your own template search path, you're free to write a function of your own which wraps :func:`getModule`. 255 256*name* can be a relative or full path. Commonly in web applications we work out the full path to the template directory and do everything with the *name* argument:: 257 258 m = getModule(os.path.join(PROJECT_DIR, 'myapp/templates/template.prep')) 259 260Alternatively, you can pass the module name and directory separately if you prefer:: 261 262 m = getModule('template', directory='TEMPLATE_DIR')) 263 264Finally, you can supply literal source if desired. This is primarily to help us in writing test cases; if you're doing it for real, you are probably either doing something brilliant or stupid ;-) 265 266The resulting module should be treated just like a Python module: import it, keep it around, and call it many times. 267 268Import semantics 269---------------- 270In an attempt to make preppy templates even more like Python code, we have also provided an **import hook**. 271 272.. function:: installImporter() 273 274Let's say you have a template called 'mytemplate.prep', on the current Python path. With the import hook, you can import your template instead of loading it with :func:`getModule`:: 275 276 import preppy 277 preppy.installImporter() 278 ... 279 import mytemplate 280 html = mytemplate.getOutput(namespace) 281 282:func:`installImporter` only needs to be called once in your program. 283 284.. function:: uninstallImporter() 285 286This does what it says. You probably don't need to call it, unless you have a reason to remove import hooks, or you're working on preppy's test suite. 287 288Executing the template 289====================== 290 291We provide two ways to execute a template and generate output. The preferred, new approach is as we demonstrated at the top of the page: you pass the same arguments through that are specified at the top of the .prep file. 292 293 294.. function:: get(arg1, arg2, etc, key1=value1, key2=value2, etc=etc, quoteFunc=str) 295 296For example, if the template starts with this:: 297 298 {{def(name, sex)}} 299 300then you would call it with:: 301 302 output = mymodule.get(name, sex) 303 304The return value is always an 8-bit string (which will become a *bytes* object in Python 3.x). 305 306 307The older approach is to pass in an arbitrary-sized dictionary, as done by most other templating systems (e.g. django, jinja). In this case, you must NOT define a *def* declaration at the top of the module. This uses a function :func:`getOutput` defined as follows: 308 309 310.. function:: getOutput(dictionary, quoteFunc=str) 311 312This would be used as follows: 313 314 namespace = {'name':'fred','age':42, 'sex':'m'} 315 html = template.getOutput(namespace) 316 317In both cases, the *quoteFunc* argument lets you control how non-text variables are displayed. In a typical web project, you will want to supply your own quoting function, to do things like escaping '&' as '&'. This is covered in detail below. 318 319If you prefer a streaming or file-based approach, you can use the :func:`run` function in the old style approach: 320 321.. function:: run(dictionary, __write__=None, quoteFunc=str, outputfile=None,code=__code__) 322 323You may either supply a function callback to *__write__*, which will be called repeatedly with the generated text; or a file-like object to *outputfile*. 324 325 326 327Quoting functions 328----------------- 329By default, preppy will use Python's *str* function to display any expression. 330This causes a problem in the markup world, where output us usually utf-8 encoded. 331The *quoteFunc* argument lets you pass in an alternative function which will be used 332to quote any output. 333 334If you are generating XML or HTML (which most people are), and you have a database field or variable containing one of the characters '<', '>' or '&', then it is easy to generate invalid markup.:: 335 336 <p>{{companyName}}</p> --> <p>Johnson & Johnson</p> 337 338An expression like the one below will fail on the first foreign accent in a name, raising 339a traceback, because Python can't convert this to ASCII:: 340 341 <p>{{client.surname}}</p> 342 343A third use is to identify and remove javascript or SQL snippets, which might 344have been passed in by a hacker. 345 346In general, you should decide on the quoting function you need, and pass it 347in when templates are called. 348 349 350xmlQuote and SafeString 351----------------------- 352We have provided one such function inside preppy.py, *stdQuote*, which is useful for XML and HTML generation. It behaves as follows: 353 354 * 8 bit strings will be xml-escaped. 355 * Any null value will produce no output. This might be useful if you are displaying a lot of numbers in table cells and don't want the word 'None' appearing everywhere. 356 * Anything you DON'T want to be quoted can be wrapped in a special SafeString or SafeUnicode class, and it won't be quoted. 357 * Anything else will be converted to a Unicode string representation (just in case the string representation contains non-ascii characters), and then encoded as utf8, then escaped. 358 359 360 361You would use this as follows:: 362 363 output = mymodule.get(name, sex, quoteFunc=preppy.stdQuote) 364 365If you are using this and you want to output a string which preppy should NOT quote (perhaps because it comes from some utility which already generates correct, escaped HTML), wrap it in the *SafeString* or *SafeUnicode* class, and it will not be escaped:: 366 367 <h1>My markup</h1> 368 {{script}}from preppy import SafeString{{endscript}}}} 369 370 <p>{{SafeString(my_already_escaped_text)}}</p> 371 372Note that SafeString and SafeUnicode are string wrappers designed to work with the *stdQuote* function, and have no useful behaviour in other contexts. 373 374 375 376 377Controlling compilation 378======================= 379 380In normal use, assuming the current user has write access to the file system, preppy will function like Python: edit your template, run your program, and the calls to getModule will trigger a recompile. However, if you want to control this for your own application (for example, in deployment scripts), three functions are provided. 381 382.. function:: compileModule(fn, savePy=0, force=0, verbose=1, importModule=1) 383 384.. function:: compileModules(pattern, savePy=0, force=0, verbose=1) 385 386.. function:: compileDir(dirName, pattern="*.prep", recursive=1, savePy=0, force=0, verbose=1) 387 388The last one works recursively, so is convenient for compiling all .prep files within a project. 389 390 391Command line tools 392================== 393 394preppy can also function as a script to let you control compilation. 395In some web frameworks (including CGI-based ones), the application runs as a restricted user, and it is important to precompile all templates and python 396modules during deployment. 397 398 399 400 401The command line interface lets you test, compile and clean up. **We expect to change this to use the more modern *optparse* module soon**:: 402 403 preppy modulename [arg1=value1, arg2=value2.....] 404 - shorthand for 'preppy run ...', see below. 405 406 preppy run modulename [arg1=value1, arg2=value2.....] 407 - runs the module, optionally with arguments. e.g. 408 preppy.py flintstone.prep name=fred sex=m 409 410 preppy.py compile [-f] [-v] [-p] module1[.prep] module2[.prep] module3 ... 411 - compiles explicit modules 412 413 preppy.py compile [-f] [-v] [-p] dirname1 dirname2 ... 414 - compiles all prep files in directory recursively 415 416 preppy.py clean dirname1 dirname2 ...19 417 - removes any py or pyc files created from past compilations 418 419 420But how do I...? 421================ 422 423People with experience of bigger templating systems typically wonder where their 424beloved features are. The answer is almost always that you can do it with Python. 425 426Here are some of the common 'missing features': 427 428 429Include 430------- 431How do you include other content? With a Python function or method call. 432 433If you want to include totally static content, it's as easy as this:: 434 435 <h1>Appendix</h1> 436 {{open('templates/appendix.html').read()}} 437 438If you want to call other templates, then import them at the top of the module in 439a script tag, and call them inline:: 440 441 {{script}} 442 appendix = preppy.getModule('appendix.prep') 443 {{endscript}} 444 445 <h1>Appendix</h1> 446 {{appendix.get(data, options)}} 447 448Being able to see what is passed in when you are editing the outer template, as well as what is expected in the inner template, turns out to be a huge advantage in maintenance when you are dealing with large systems of nested templates. 449 450 451Automatic escaping 452------------------ 453Many systems can escape all expressions output into HTML as a security measure. Some go further and try to remove Javascript. Preppy solves this by letting you pass in your own quote function. 454 455In systems which do this, they commonly require an extra construct to mark some expressions as 'safe', and not to be escaped. This can be accomplished by having a string subclass, and having your quote function recognise and pass it through. See the *stdQuote* function above 456 457 458 459 460Filters 461------- 462 463Django has a nice syntax for filters - functions which tidy up output:: 464 465 {{number | floatformat}} 466 467Our approach is to have functions with short names, and avoid introducing extra syntax. 468This is very slightly more verbose. For example, if a template had to 469display many values in pounds sterling, we could write a function *fmtPounds* which 470adds the pound sign, formats with commas every thousand and two decimal places. 471These functions can also be set to output an empty string for None or missing values, 472to set a class on the output for negative number display, or whatever you require. 473 474We then display a value like this:: 475 476 <td>{{fmtPounds(total)}}</td> 477 478This approach requires a couple of extra parentheses, but is easy to understand and 479saves us from having to write a ton of filters. It also encourages consistency in 480your formatting. 481 482It is common 483and useful to define these once per application in a helper module and import them. 484For example with our own Report Markup Language (used for PDF generation), we will 485commonly have a large template called 'rmltemplate.prep', and a helper Python module 486'rmlutils.py'. Developers know that this module contains utilities for use in the 487template. 488 489 490 491 492Block inheritance 493----------------- 494We don't support this. It doesn't really fit with the nice metaphor of a template 495working just like a Python function. If anyone can suggest a way of doing it, 496we'll consider it. 497 498Block inheritance is mostly used to let a designer set out the outer structure 499of web pages, with high level `<div>` tags which get filled in later. This 500can be done with a master template and included sub-templates. 501 502 503 504 505 506Support 507======= 508 509We suggest using the ``reportlab-users`` mailing list for any support issues, or raising a bug report on bitbucket. ReportLab's commercial customers can also email our support address; others will be blocked by a whitelist. 510 511 512Future Plans 513============ 514 515It is likely that, in 2013, we will rewrite preppy from the ground up using the compiler support in Python 2.7 and 3.3. We'll be ensuring there is 'only one way to do it'. If you follow the guidelines above, minimal changes should be needed to templates. 516 517We hope there is still a place for a lightweight, minimal, fast templating system in the Python world, and welcome feedback and experiences if you embed this system in your project. 518 519Credits 520======= 521The initial implementation was done by Aaron Watters, to a broad design by Andy Robinson. This worked by actual code generation, creating a python module with the :func:`getOutput` function, which was optionally save to disk. 522 523Robin Becker than optimised this by generating bytecodes directly, and later implemented the newer *function declaration* style. 524 525 526 527 528