1[comment {-*- tcl -*- doctools manpage}] 2[manpage_begin textutil::expander n 1.3.1] 3[see_also {[uri}] 4[see_also http://www.wjduquette.com/expand] 5[see_also regexp] 6[see_also split] 7[see_also string] 8[keywords string] 9[keywords {template processing}] 10[keywords {text expansion}] 11[copyright {William H. Duquette, http://www.wjduquette.com/expand}] 12[moddesc {Text and string utilities, macro processing}] 13[titledesc {Procedures to process templates and expand text.}] 14[category {Documentation tools}] 15[require Tcl 8.2] 16[require textutil::expander [opt 1.3.1]] 17[description] 18 19[para] 20 21The Tcl [cmd subst] command is often used to support a kind of 22template processing. Given a string with embedded variables or 23function calls, [cmd subst] will interpolate the variable and function 24values, returning the new string: 25 26[para] 27 28[example { 29 % set greeting "Howdy" 30 Howdy 31 % proc place {} {return "World"} 32 % subst {$greeting, [place]!} 33 Howdy, World! 34 % 35}] 36 37[para] 38 39By defining a suitable set of Tcl commands, [cmd subst] can be used to 40implement a markup language similar to HTML. 41 42[para] 43 44The [cmd subst] command is efficient, but it has three drawbacks for 45this kind of template processing: 46 47[list_begin itemized] 48 49[item] 50 51There's no way to identify and process the plain text between two 52embedded Tcl commands; that makes it difficult to handle plain text in 53a context-sensitive way. 54 55[item] 56 57Embedded commands are necessarily bracketed by [const [lb]] and 58[const [rb]]; it's convenient to be able to choose different brackets 59in special cases. Someone producing web pages that include a large 60quantity of Tcl code examples might easily prefer to use [const <<] 61and [const >>] as the embedded code delimiters instead. 62 63[item] 64 65There's no easy way to handle incremental input, as one might wish to 66do when reading data from a socket. 67 68[list_end] 69 70[para] 71 72At present, expander solves the first two problems; eventually it will 73solve the third problem as well. 74 75[para] 76 77The following section describes the command API to the expander; this 78is followed by the tutorial sections, see [sectref TUTORIAL]. 79 80[section {EXPANDER API}] 81[para] 82 83The [package textutil::expander] package provides only one command, 84described below. The rest of the section is taken by a description of 85the methods for the expander objects created by this command. 86 87[list_begin definitions] 88 89[call [cmd ::textutil::expander] [arg expanderName]] 90 91The command creates a new expander object with an associated Tcl 92command whose name is [arg expanderName]. This command may be used to 93invoke various operations on the graph. If the [arg expanderName] is 94not fully qualified it is interpreted as relative to the current 95namespace. The command has the following general form: 96 97[example_begin] 98[arg expanderName] option [opt [arg {arg arg ...}]] 99[example_end] 100 101[arg Option] and the [arg arg]s determine the exact behavior of the 102command. 103 104[list_end] 105 106[para] 107 108The following commands are possible for expander objects: 109 110[list_begin definitions] 111 112[call [arg expanderName] [method cappend] [arg text]] 113 114Appends a string to the output in the current context. This command 115should rarely be used by macros or application code. 116 117[call [arg expanderName] [method cget] [arg varname]] 118 119Retrieves the value of variable [arg varname], defined in the current 120context. 121 122[call [arg expanderName] [method cis] [arg cname]] 123 124Determines whether or not the name of the current context is 125 126[arg cname]. 127 128[call [arg expanderName] [method cname]] 129 130Returns the name of the current context. 131 132[call [arg expanderName] [method cpop] [arg cname]] 133 134Pops a context from the context stack, returning all accumulated 135output in that context. The context must be named [arg cname], or an 136error results. 137 138[call [arg expanderName] [method ctopandclear]] 139 140Returns the output currently captured in the topmost context and 141clears that buffer. This is similar to a combination of [method cpop] 142followed by [method cpush], except that internal state (brackets) is 143preserved here. 144 145[call [arg expanderName] [method cpush] [arg cname]] 146 147Pushes a context named [arg cname] onto the context stack. The 148context must be popped by [method cpop] before expansion ends or an 149error results. 150 151[call [arg expanderName] [method cset] [arg varname] [arg value]] 152 153Sets variable [arg varname] to [arg value] in the current context. 154 155[call [arg expanderName] [method cvar] [arg varname]] 156 157Retrieves the internal variable name of context variable 158 159[arg varname]; this allows the variable to be passed to commands like 160[cmd lappend]. 161 162[call [arg expanderName] [method errmode] [arg newErrmode]] 163 164Sets the macro expansion error mode to one of [const nothing], 165[const macro], [const error], or [const fail]; the default value is 166[const fail]. The value determines what the expander does if an 167error is detected during expansion of a macro. 168 169[list_begin definitions] 170[def [const fail]] 171 172The error propagates normally and can be caught or ignored by the 173application. 174 175[def [const error]] 176 177The macro expands into a detailed error message, and expansion 178continues. 179 180[def [const macro]] 181 182The macro expands to itself; that is, it is passed along to the output 183unchanged. 184 185[def [const nothing]] 186 187The macro expands to the empty string, and is effectively ignored. 188 189[list_end] 190[para] 191 192[call [arg expanderName] [method evalcmd] [opt [arg newEvalCmd]]] 193 194Returns the current evaluation command, which defaults to 195 196[cmd {uplevel #0}]. If specified, [arg newEvalCmd] will be saved for 197future use and then returned; it must be a Tcl command expecting one 198additional argument: the macro to evaluate. 199 200[call [arg expanderName] [method expand] [arg string] [opt [arg brackets]]] 201 202Expands the input string, replacing embedded macros with their 203expanded values, and returns the expanded string. 204 205[para] Note that this method pushes a new (empty) context on the stack of 206contexts while it is running, and removes it on return. 207 208[para] 209 210If [arg brackets] is given, it must be a list of two strings; the 211items will be used as the left and right macro expansion bracket 212sequences for this expansion only. 213 214[call [arg expanderName] [method lb] [opt [arg newbracket]]] 215 216Returns the current value of the left macro expansion bracket; this is 217for use as or within a macro, when the bracket needs to be included in 218the output text. If [arg newbracket] is specified, it becomes the new 219bracket, and is returned. 220 221[call [arg expanderName] [method rb] [opt [arg newbracket]]] 222 223Returns the current value of the right macro expansion bracket; this 224is for use as or within a macro, when the bracket needs to be included 225in the output text. If [arg newbracket] is specified, it becomes the 226new bracket, and is returned. 227 228[call [arg expanderName] [method reset]] 229 230Resets all expander settings to their initial values. Unusual results 231are likely if this command is called from within a call to 232 233[method expand]. 234 235[call [arg expanderName] [method setbrackets] [arg {lbrack rbrack}]] 236 237Sets the left and right macro expansion brackets. This command is for 238use as or within a macro, or to permanently change the bracket 239definitions. By default, the brackets are [const [lb]] and 240 241[const [rb]], but any non-empty string can be used; for example, 242[const <] and [const >] or [const (*] and [const *)] or even 243[const Hello,] and [const World!]. 244 245[call [arg expanderName] [method textcmd] [opt [arg newTextCmd]]] 246 247Returns the current command for processing plain text, which defaults 248to the empty string, meaning [emph identity]. If specified, 249 250[arg newTextCmd] will be saved for future use and then returned; it 251must be a Tcl command expecting one additional argument: the text to 252process. The expander object will this command for all plain text it 253encounters, giving the user of the object the ability to process all 254plain text in some standard way before writing it to the output. The 255object expects that the command returns the processed plain text. 256 257[para] 258 259[emph Note] that the combination of "[cmd textcmd] [arg plaintext]" 260is run through the [arg evalcmd] for the actual evaluation. In other 261words, the [arg textcmd] is treated as a special macro implicitly 262surrounding all plain text in the template. 263 264[call [arg expanderName] [method where]] 265 266Returns a three-element list containing the current character 267position, line, and column the expander is at in the processing of the 268current input string. 269 270[list_end] 271 272[section TUTORIAL] 273[subsection Basics] 274 275To begin, create an expander object: 276 277[para] 278[example { 279 % package require textutil::expander 280 1.2 281 % ::textutil::expander myexp 282 ::myexp 283 % 284}] 285[para] 286 287The created [cmd ::myexp] object can be used to expand text strings 288containing embedded Tcl commands. By default, embedded commands are 289delimited by square brackets. Note that expander doesn't attempt to 290interpolate variables, since variables can be referenced by embedded 291commands: 292 293[para] 294[example { 295 % set greeting "Howdy" 296 Howdy 297 % proc place {} {return "World"} 298 % ::myexp expand {[set greeting], [place]!} 299 Howdy, World! 300 % 301}] 302 303[subsection {Embedding Macros}] 304 305An expander macro is simply a Tcl script embedded within a text 306string. Expander evaluates the script in the global context, and 307replaces it with its result string. For example, 308 309[para] 310[example { 311 % set greetings {Howdy Hi "What's up"} 312 Howdy Hi "What's up" 313 % ::myexp expand {There are many ways to say "Hello, World!": 314 [set result {} 315 foreach greeting $greetings { 316 append result "$greeting, World!\\n" 317 } 318 set result] 319 And that's just a small sample!} 320 There are many ways to say "Hello, World!": 321 Howdy, World! 322 Hi, World! 323 What's up, World! 324 325 And that's just a small sample! 326 % 327}] 328 329[subsection {Writing Macro Commands}] 330 331More typically, [emph {macro commands}] are used to create a markup 332language. A macro command is just a Tcl command that returns an 333output string. For example, expand can be used to implement a generic 334document markup language that can be retargeted to HTML or any other 335output format: 336 337[para] 338[example { 339 % proc bold {} {return "<b>"} 340 % proc /bold {} {return "</b>"} 341 % ::myexp expand {Some of this text is in [bold]boldface[/bold]} 342 Some of this text is in <b>boldface</b> 343 % 344}] 345[para] 346 347The above definitions of [cmd bold] and [cmd /bold] returns HTML, but 348such commands can be as complicated as needed; they could, for 349example, decide what to return based on the desired output format. 350 351[subsection {Changing the Expansion Brackets}] 352 353By default, embedded macros are enclosed in square brackets, 354 355[const [lb]] and [const [rb]]. If square brackets need to be 356included in the output, the input can contain the [cmd lb] and 357 358[cmd rb] commands. Alternatively, or if square brackets are 359objectionable for some other reason, the macro expansion brackets can 360be changed to any pair of non-empty strings. 361 362[para] 363 364The [method setbrackets] command changes the brackets permanently. 365For example, you can write pseudo-html by change them to [const <] 366and [const >]: 367 368[para] 369[example { 370 % ::myexp setbrackets < > 371 % ::myexp expand {<bold>This is boldface</bold>} 372 <b>This is boldface</b> 373}] 374[para] 375 376Alternatively, you can change the expansion brackets temporarily by 377passing the desired brackets to the [method expand] command: 378 379[para] 380[example { 381 % ::myexp setbrackets "\\[" "\\]" 382 % ::myexp expand {<bold>This is boldface</bold>} {< >} 383 <b>This is boldface</b> 384 % 385}] 386 387[subsection {Customized Macro Expansion}] 388 389By default, macros are evaluated using the Tcl [cmd {uplevel #0}] 390command, so that the embedded code executes in the global context. 391The application can provide a different evaluation command using 392[method evalcmd]; this allows the application to use a safe 393interpreter, for example, or even to evaluated something other than 394Tcl code. There is one caveat: to be recognized as valid, a macro 395must return 1 when passed to Tcl's "info complete" command. 396 397[para] 398 399For example, the following code "evaluates" each macro by returning 400the macro text itself. 401 402[para] 403[example { 404 proc identity {macro} {return $macro} 405 ::myexp evalcmd identity 406}] 407 408[subsection {Using the Context Stack}] 409 410Often it's desirable to define a pair of macros which operate in some 411way on the plain text between them. Consider a set of macros for 412adding footnotes to a web page: one could have implement something 413like this: 414 415[para] 416[example { 417 Dr. Pangloss, however, thinks that this is the best of all 418 possible worlds.[footnote "See Candide, by Voltaire"] 419}] 420[para] 421 422The [cmd footnote] macro would, presumably, assign a number to this 423footnote and save the text to be formatted later on. However, this 424solution is ugly if the footnote text is long or should contain 425additional markup. Consider the following instead: 426 427[para] 428[example { 429 Dr. Pangloss, however, thinks that this is the best of all 430 possible worlds.[footnote]See [bookTitle "Candide"], by 431 [authorsName "Voltaire"], for more information.[/footnote] 432}] 433[para] 434 435Here the footnote text is contained between [cmd footnote] and 436[cmd /footnote] macros, continues onto a second line, and contains 437several macros of its own. This is both clearer and more flexible; 438however, with the features presented so far there's no easy way to do 439it. That's the purpose of the context stack. 440 441[para] 442 443All macro expansion takes place in a particular context. Here, the 444[cmd footnote] macro pushes a new context onto the context stack. 445Then, all expanded text gets placed in that new context. 446 447[cmd /footnote] retrieves it by popping the context. Here's a 448skeleton implementation of these two macros: 449 450[para] 451[example { 452 proc footnote {} { 453 ::myexp cpush footnote 454 } 455 456 proc /footnote {} { 457 set footnoteText [::myexp cpop footnote] 458 459 # Save the footnote text, and return an appropriate footnote 460 # number and link. 461 } 462}] 463[para] 464 465The [method cpush] command pushes a new context onto the stack; the 466argument is the context's name. It can be any string, but would 467typically be the name of the macro itself. Then, [method cpop] 468verifies that the current context has the expected name, pops it off 469of the stack, and returns the accumulated text. 470 471[para] 472 473Expand provides several other tools related to the context stack. 474Suppose the first macro in a context pair takes arguments or computes 475values which the second macro in the pair needs. After calling 476[method cpush], the first macro can define one or more context 477variables; the second macro can retrieve their values any time before 478calling [method cpop]. For example, suppose the document must specify 479the footnote number explicitly: 480 481[para] 482[example { 483 proc footnote {footnoteNumber} { 484 ::myexp cpush footnote 485 ::myexp csave num $footnoteNumber 486 # Return an appropriate link 487 } 488 489 proc /footnote {} { 490 set footnoteNumber [::myexp cget num] 491 set footnoteText [::myexp cpop footnote] 492 493 # Save the footnote text and its footnoteNumber for future 494 # output. 495 } 496}] 497[para] 498 499At times, it might be desirable to define macros that are valid only 500within a particular context pair; such macros should verify that they 501are only called within the correct context using either [method cis] 502or [method cname]. 503 504[section HISTORY] 505 506[cmd expander] was written by William H. Duquette; it is a repackaging 507of the central algorithm of the expand macro processing tool. 508 509[vset CATEGORY textutil] 510[include ../common-text/feedback.inc] 511[manpage_end] 512