1[library python 2 [version 2.0] 3 [authors [de Guzman, Joel], [Abrahams, David]] 4 [copyright 2002 2003 2004 2005 Joel de Guzman, David Abrahams] 5 [category inter-language support] 6 [purpose 7 Reflects C++ classes and functions into Python 8 ] 9 [license 10 Distributed under the Boost Software License, Version 1.0. 11 (See accompanying file LICENSE_1_0.txt or copy at 12 <ulink url="http://www.boost.org/LICENSE_1_0.txt"> 13 http://www.boost.org/LICENSE_1_0.txt 14 </ulink>) 15 ] 16] 17 18[/ QuickBook Document version 0.9 ] 19 20[def __note__ [$images/note.png]] 21[def __alert__ [$images/alert.png]] 22[def __tip__ [$images/tip.png]] 23[def :-) [$images/smiley.png]] 24[def __jam__ [$images/jam.png]] 25 26[section QuickStart] 27 28The Boost Python Library is a framework for interfacing Python and 29C++. It allows you to quickly and seamlessly expose C++ classes 30functions and objects to Python, and vice-versa, using no special 31tools -- just your C++ compiler. It is designed to wrap C++ interfaces 32non-intrusively, so that you should not have to change the C++ code at 33all in order to wrap it, making Boost.Python ideal for exposing 343rd-party libraries to Python. The library's use of advanced 35metaprogramming techniques simplifies its syntax for users, so that 36wrapping code takes on the look of a kind of declarative interface 37definition language (IDL). 38 39[h2 Hello World] 40 41Following C/C++ tradition, let's start with the "hello, world". A C++ 42Function: 43 44 char const* greet() 45 { 46 return "hello, world"; 47 } 48 49can be exposed to Python by writing a Boost.Python wrapper: 50 51 #include <boost/python.hpp> 52 53 BOOST_PYTHON_MODULE(hello_ext) 54 { 55 using namespace boost::python; 56 def("greet", greet); 57 } 58 59That's it. We're done. We can now build this as a shared library. The 60resulting DLL is now visible to Python. Here's a sample Python session: 61 62[python] 63 64 >>> import hello_ext 65 >>> print hello_ext.greet() 66 hello, world 67 68[c++] 69 70[:['[*Next stop... Building your Hello World module from start to finish...]]] 71 72[endsect] 73[section:hello Building Hello World] 74 75[h2 From Start To Finish] 76 77Now the first thing you'd want to do is to build the Hello World module and 78try it for yourself in Python. In this section, we will outline the steps 79necessary to achieve that. We will use the build tool that comes bundled 80with every boost distribution: [*bjam]. 81 82[note [*Building without bjam] 83 84Besides bjam, there are of course other ways to get your module built. 85What's written here should not be taken as "the one and only way". 86There are of course other build tools apart from [^bjam]. 87 88Take note however that the preferred build tool for Boost.Python is bjam. 89There are so many ways to set up the build incorrectly. Experience shows 90that 90% of the "I can't build Boost.Python" problems come from people 91who had to use a different tool. 92] 93 94We will skip over the details. Our objective will be to simply create 95the hello world module and run it in Python. For a complete reference to 96building Boost.Python, check out: [@../../../building.html 97building.html]. After this brief ['bjam] tutorial, we should have built 98the DLLs and run a python program using the extension. 99 100The tutorial example can be found in the directory: 101[^libs/python/example/tutorial]. There, you can find: 102 103* hello.cpp 104* hello.py 105* Jamroot 106 107The [^hello.cpp] file is our C++ hello world example. The [^Jamroot] is 108a minimalist ['bjam] script that builds the DLLs for us. Finally, 109[^hello.py] is our Python program that uses the extension in 110[^hello.cpp]. 111 112Before anything else, you should have the bjam executable in your boost 113directory or somewhere in your path such that [^bjam] can be executed in 114the command line. Pre-built Boost.Jam executables are available for most 115platforms. The complete list of Bjam executables can be found 116[@http://sourceforge.net/project/showfiles.php?group_id=7586 here]. 117 118[h2 Let's Jam!] 119__jam__ 120 121[@../../../../example/tutorial/Jamroot Here] is our minimalist Jamroot 122file. Simply copy the file and tweak [^use-project boost] to where your 123boost root directory is and your OK. 124 125The comments contained in the Jamrules file above should be sufficient 126to get you going. 127 128[h2 Running bjam] 129 130['bjam] is run using your operating system's command line interpreter. 131 132[:Start it up.] 133 134A file called user-config.jam in your home directory is used to 135configure your tools. In Windows, your home directory can be found by 136typing: 137 138[pre 139ECHO %HOMEDRIVE%%HOMEPATH% 140] 141 142into a command prompt window. Your file should at least have the rules 143for your compiler and your python installation. A specific example of 144this on Windows would be: 145 146[pre 147# MSVC configuration 148using msvc : 8.0 ; 149 150# Python configuration 151using python : 2.4 : C:/dev/tools/Python/ ; 152] 153 154The first rule tells Bjam to use the MSVC 8.0 compiler and associated 155tools. The second rule provides information on Python, its version and 156where it is located. The above assumes that the Python installation is 157in [^C:/dev/tools\/Python/]. If you have one fairly "standard" python 158installation for your platform, you might not need to do this. 159 160Now we are ready... Be sure to [^cd] to [^libs/python/example/tutorial] 161where the tutorial [^"hello.cpp"] and the [^"Jamroot"] is situated. 162 163Finally: 164 165 bjam 166 167It should be building now: 168 169[pre 170cd C:\dev\boost\libs\python\example\tutorial 171bjam 172...patience... 173...found 1101 targets... 174...updating 35 targets... 175] 176 177And so on... Finally: 178 179[pre 180 Creating library /path-to-boost_python.dll/ 181 Creating library /path-to-'''hello_ext'''.exp/ 182'''**passed**''' ... hello.test 183...updated 35 targets... 184] 185 186Or something similar. If all is well, you should now have built the DLLs and 187run the Python program. 188 189[:[*There you go... Have fun!]] 190 191[endsect] 192[section:exposing Exposing Classes] 193 194Now let's expose a C++ class to Python. 195 196Consider a C++ class/struct that we want to expose to Python: 197 198 struct World 199 { 200 void set(std::string msg) { this->msg = msg; } 201 std::string greet() { return msg; } 202 std::string msg; 203 }; 204 205We can expose this to Python by writing a corresponding Boost.Python 206C++ Wrapper: 207 208 #include <boost/python.hpp> 209 using namespace boost::python; 210 211 BOOST_PYTHON_MODULE(hello) 212 { 213 class_<World>("World") 214 .def("greet", &World::greet) 215 .def("set", &World::set) 216 ; 217 } 218 219Here, we wrote a C++ class wrapper that exposes the member functions 220[^greet] and [^set]. Now, after building our module as a shared library, we 221may use our class [^World] in Python. Here's a sample Python session: 222 223[python] 224 225 >>> import hello 226 >>> planet = hello.World() 227 >>> planet.set('howdy') 228 >>> planet.greet() 229 'howdy' 230 231[section Constructors] 232 233Our previous example didn't have any explicit constructors. 234Since [^World] is declared as a plain struct, it has an implicit default 235constructor. Boost.Python exposes the default constructor by default, 236which is why we were able to write 237 238 >>> planet = hello.World() 239 240We may wish to wrap a class with a non-default constructor. Let us 241build on our previous example: 242 243[c++] 244 245 struct World 246 { 247 World(std::string msg): msg(msg) {} // added constructor 248 void set(std::string msg) { this->msg = msg; } 249 std::string greet() { return msg; } 250 std::string msg; 251 }; 252 253This time [^World] has no default constructor; our previous 254wrapping code would fail to compile when the library tried to expose 255it. We have to tell [^class_<World>] about the constructor we want to 256expose instead. 257 258 #include <boost/python.hpp> 259 using namespace boost::python; 260 261 BOOST_PYTHON_MODULE(hello) 262 { 263 class_<World>("World", init<std::string>()) 264 .def("greet", &World::greet) 265 .def("set", &World::set) 266 ; 267 } 268 269[^init<std::string>()] exposes the constructor taking in a 270[^std::string] (in Python, constructors are spelled 271"[^"__init__"]"). 272 273We can expose additional constructors by passing more [^init<...>]s to 274the [^def()] member function. Say for example we have another World 275constructor taking in two doubles: 276 277 class_<World>("World", init<std::string>()) 278 .def(init<double, double>()) 279 .def("greet", &World::greet) 280 .def("set", &World::set) 281 ; 282 283On the other hand, if we do not wish to expose any constructors at 284all, we may use [^no_init] instead: 285 286 class_<Abstract>("Abstract", no_init) 287 288This actually adds an [^__init__] method which always raises a 289Python RuntimeError exception. 290 291[endsect] 292[section Class Data Members] 293 294Data members may also be exposed to Python so that they can be 295accessed as attributes of the corresponding Python class. Each data 296member that we wish to be exposed may be regarded as [*read-only] or 297[*read-write]. Consider this class [^Var]: 298 299 struct Var 300 { 301 Var(std::string name) : name(name), value() {} 302 std::string const name; 303 float value; 304 }; 305 306Our C++ [^Var] class and its data members can be exposed to Python: 307 308 class_<Var>("Var", init<std::string>()) 309 .def_readonly("name", &Var::name) 310 .def_readwrite("value", &Var::value); 311 312Then, in Python, assuming we have placed our Var class inside the namespace 313hello as we did before: 314 315[python] 316 317 >>> x = hello.Var('pi') 318 >>> x.value = 3.14 319 >>> print x.name, 'is around', x.value 320 pi is around 3.14 321 322Note that [^name] is exposed as [*read-only] while [^value] is exposed 323as [*read-write]. 324 325 >>> x.name = 'e' # can't change name 326 Traceback (most recent call last): 327 File "<stdin>", line 1, in ? 328 AttributeError: can't set attribute 329 330[endsect] 331[section Class Properties] 332 333In C++, classes with public data members are usually frowned 334upon. Well designed classes that take advantage of encapsulation hide 335the class' data members. The only way to access the class' data is 336through access (getter/setter) functions. Access functions expose class 337properties. Here's an example: 338 339[c++] 340 341 struct Num 342 { 343 Num(); 344 float get() const; 345 void set(float value); 346 ... 347 }; 348 349However, in Python attribute access is fine; it doesn't neccessarily break 350encapsulation to let users handle attributes directly, because the 351attributes can just be a different syntax for a method call. Wrapping our 352[^Num] class using Boost.Python: 353 354 class_<Num>("Num") 355 .add_property("rovalue", &Num::get) 356 .add_property("value", &Num::get, &Num::set); 357 358And at last, in Python: 359 360[python] 361 362 >>> x = Num() 363 >>> x.value = 3.14 364 >>> x.value, x.rovalue 365 (3.14, 3.14) 366 >>> x.rovalue = 2.17 # error! 367 368Take note that the class property [^rovalue] is exposed as [*read-only] 369since the [^rovalue] setter member function is not passed in: 370 371[c++] 372 373 .add_property("rovalue", &Num::get) 374 375[endsect] 376[section Inheritance] 377 378In the previous examples, we dealt with classes that are not polymorphic. 379This is not often the case. Much of the time, we will be wrapping 380polymorphic classes and class hierarchies related by inheritance. We will 381often have to write Boost.Python wrappers for classes that are derived from 382abstract base classes. 383 384Consider this trivial inheritance structure: 385 386 struct Base { virtual ~Base(); }; 387 struct Derived : Base {}; 388 389And a set of C++ functions operating on [^Base] and [^Derived] object 390instances: 391 392 void b(Base*); 393 void d(Derived*); 394 Base* factory() { return new Derived; } 395 396We've seen how we can wrap the base class [^Base]: 397 398 class_<Base>("Base") 399 /*...*/ 400 ; 401 402Now we can inform Boost.Python of the inheritance relationship between 403[^Derived] and its base class [^Base]. Thus: 404 405 class_<Derived, bases<Base> >("Derived") 406 /*...*/ 407 ; 408 409Doing so, we get some things for free: 410 411# Derived automatically inherits all of Base's Python methods 412 (wrapped C++ member functions) 413# [*If] Base is polymorphic, [^Derived] objects which have been passed to 414 Python via a pointer or reference to [^Base] can be passed where a pointer 415 or reference to [^Derived] is expected. 416 417Now, we will expose the C++ free functions [^b] and [^d] and [^factory]: 418 419 def("b", b); 420 def("d", d); 421 def("factory", factory); 422 423Note that free function [^factory] is being used to generate new 424instances of class [^Derived]. In such cases, we use 425[^return_value_policy<manage_new_object>] to instruct Python to adopt 426the pointer to [^Base] and hold the instance in a new Python [^Base] 427object until the the Python object is destroyed. We will see more of 428Boost.Python [link python.call_policies call policies] later. 429 430 // Tell Python to take ownership of factory's result 431 def("factory", factory, 432 return_value_policy<manage_new_object>()); 433 434[endsect] 435 436[section Class Virtual Functions] 437 438In this section, we will learn how to make functions behave polymorphically 439through virtual functions. Continuing our example, let us add a virtual function 440to our [^Base] class: 441 442 struct Base 443 { 444 virtual ~Base() {} 445 virtual int f() = 0; 446 }; 447 448One of the goals of Boost.Python is to be minimally intrusive on an existing C++ 449design. In principle, it should be possible to expose the interface for a 3rd 450party library without changing it. It is not ideal to add anything to our class 451`Base`. Yet, when you have a virtual function that's going to be overridden in 452Python and called polymorphically *from C++*, we'll need to add some 453scaffoldings to make things work properly. What we'll do is write a class 454wrapper that derives from `Base` that will unintrusively hook into the virtual 455functions so that a Python override may be called: 456 457 struct BaseWrap : Base, wrapper<Base> 458 { 459 int f() 460 { 461 return this->get_override("f")(); 462 } 463 }; 464 465Notice too that in addition to inheriting from `Base`, we also multiply- 466inherited `wrapper<Base>` (See [@../../../v2/wrapper.html Wrapper]). The 467`wrapper` template makes the job of wrapping classes that are meant to 468overridden in Python, easier. 469 470[blurb __alert__ [*MSVC6/7 Workaround] 471 472If you are using Microsoft Visual C++ 6 or 7, you have to write `f` as: 473 474`return call<int>(this->get_override("f").ptr());`.] 475 476BaseWrap's overridden virtual member function `f` in effect calls the 477corresponding method of the Python object through `get_override`. 478 479Finally, exposing `Base`: 480 481 class_<BaseWrap, boost::noncopyable>("Base") 482 .def("f", pure_virtual(&Base::f)) 483 ; 484 485`pure_virtual` signals Boost.Python that the function `f` is a pure virtual 486function. 487 488[note [*member function and methods] 489 490Python, like many object oriented languages uses the term [*methods]. 491Methods correspond roughly to C++'s [*member functions]] 492 493[endsect] 494 495[section Virtual Functions with Default Implementations] 496 497We've seen in the previous section how classes with pure virtual functions are 498wrapped using Boost.Python's [@../../../v2/wrapper.html class wrapper] 499facilities. If we wish to wrap [*non]-pure-virtual functions instead, the 500mechanism is a bit different. 501 502Recall that in the [link python.class_virtual_functions previous section], we 503wrapped a class with a pure virtual function that we then implemented in C++, or 504Python classes derived from it. Our base class: 505 506 struct Base 507 { 508 virtual int f() = 0; 509 }; 510 511had a pure virtual function [^f]. If, however, its member function [^f] was 512not declared as pure virtual: 513 514 struct Base 515 { 516 virtual ~Base() {} 517 virtual int f() { return 0; } 518 }; 519 520We wrap it this way: 521 522 struct BaseWrap : Base, wrapper<Base> 523 { 524 int f() 525 { 526 if (override f = this->get_override("f")) 527 return f(); // *note* 528 return Base::f(); 529 } 530 531 int default_f() { return this->Base::f(); } 532 }; 533 534Notice how we implemented `BaseWrap::f`. Now, we have to check if there is an 535override for `f`. If none, then we call `Base::f()`. 536 537[blurb __alert__ [*MSVC6/7 Workaround] 538 539If you are using Microsoft Visual C++ 6 or 7, you have to rewrite the line 540with the `*note*` as: 541 542`return call<char const*>(f.ptr());`.] 543 544Finally, exposing: 545 546 class_<BaseWrap, boost::noncopyable>("Base") 547 .def("f", &Base::f, &BaseWrap::default_f) 548 ; 549 550Take note that we expose both `&Base::f` and `&BaseWrap::default_f`. 551Boost.Python needs to keep track of 1) the dispatch function [^f] and 2) the 552forwarding function to its default implementation [^default_f]. There's a 553special [^def] function for this purpose. 554 555In Python, the results would be as expected: 556 557[python] 558 559 >>> base = Base() 560 >>> class Derived(Base): 561 ... def f(self): 562 ... return 42 563 ... 564 >>> derived = Derived() 565 566Calling [^base.f()]: 567 568 >>> base.f() 569 0 570 571Calling [^derived.f()]: 572 573 >>> derived.f() 574 42 575 576[endsect] 577[section Class Operators/Special Functions] 578 579[h2 Python Operators] 580 581C is well known for the abundance of operators. C++ extends this to the 582extremes by allowing operator overloading. Boost.Python takes advantage of 583this and makes it easy to wrap C++ operator-powered classes. 584 585Consider a file position class [^FilePos] and a set of operators that take 586on FilePos instances: 587 588[c++] 589 590 class FilePos { /*...*/ }; 591 592 FilePos operator+(FilePos, int); 593 FilePos operator+(int, FilePos); 594 int operator-(FilePos, FilePos); 595 FilePos operator-(FilePos, int); 596 FilePos& operator+=(FilePos&, int); 597 FilePos& operator-=(FilePos&, int); 598 bool operator<(FilePos, FilePos); 599 600The class and the various operators can be mapped to Python rather easily 601and intuitively: 602 603 class_<FilePos>("FilePos") 604 .def(self + int()) // __add__ 605 .def(int() + self) // __radd__ 606 .def(self - self) // __sub__ 607 .def(self - int()) // __sub__ 608 .def(self += int()) // __iadd__ 609 .def(self -= other<int>()) 610 .def(self < self); // __lt__ 611 612The code snippet above is very clear and needs almost no explanation at 613all. It is virtually the same as the operators' signatures. Just take 614note that [^self] refers to FilePos object. Also, not every class [^T] that 615you might need to interact with in an operator expression is (cheaply) 616default-constructible. You can use [^other<T>()] in place of an actual 617[^T] instance when writing "self expressions". 618 619[h2 Special Methods] 620 621Python has a few more ['Special Methods]. Boost.Python supports all of the 622standard special method names supported by real Python class instances. A 623similar set of intuitive interfaces can also be used to wrap C++ functions 624that correspond to these Python ['special functions]. Example: 625 626 class Rational 627 { public: operator double() const; }; 628 629 Rational pow(Rational, Rational); 630 Rational abs(Rational); 631 ostream& operator<<(ostream&,Rational); 632 633 class_<Rational>("Rational") 634 .def(float_(self)) // __float__ 635 .def(pow(self, other<Rational>)) // __pow__ 636 .def(abs(self)) // __abs__ 637 .def(str(self)) // __str__ 638 ; 639 640Need we say more? 641 642[note What is the business of `operator<<`? 643Well, the method `str` requires the `operator<<` to do its work (i.e. 644`operator<<` is used by the method defined by `def(str(self))`.] 645 646[endsect] 647[endsect] [/ Exposing Classes ] 648 649[section Functions] 650 651In this chapter, we'll look at Boost.Python powered functions in closer 652detail. We will see some facilities to make exposing C++ functions to 653Python safe from potential pifalls such as dangling pointers and 654references. We will also see facilities that will make it even easier for 655us to expose C++ functions that take advantage of C++ features such as 656overloading and default arguments. 657 658[:['Read on...]] 659 660But before you do, you might want to fire up Python 2.2 or later and type 661[^>>> import this]. 662 663[pre 664>>> import this 665The Zen of Python, by Tim Peters 666Beautiful is better than ugly. 667Explicit is better than implicit. 668Simple is better than complex. 669Complex is better than complicated. 670Flat is better than nested. 671Sparse is better than dense. 672Readability counts. 673Special cases aren't special enough to break the rules. 674Although practicality beats purity. 675Errors should never pass silently. 676Unless explicitly silenced. 677In the face of ambiguity, refuse the temptation to guess. 678There should be one-- and preferably only one --obvious way to do it 679Although that way may not be obvious at first unless you're Dutch. 680Now is better than never. 681Although never is often better than *right* now. 682If the implementation is hard to explain, it's a bad idea. 683If the implementation is easy to explain, it may be a good idea. 684Namespaces are one honking great idea -- let's do more of those! 685] 686 687[section Call Policies] 688 689In C++, we often deal with arguments and return types such as pointers 690and references. Such primitive types are rather, ummmm, low level and 691they really don't tell us much. At the very least, we don't know the 692owner of the pointer or the referenced object. No wonder languages 693such as Java and Python never deal with such low level entities. In 694C++, it's usually considered a good practice to use smart pointers 695which exactly describe ownership semantics. Still, even good C++ 696interfaces use raw references and pointers sometimes, so Boost.Python 697must deal with them. To do this, it may need your help. Consider the 698following C++ function: 699 700 X& f(Y& y, Z* z); 701 702How should the library wrap this function? A naive approach builds a 703Python X object around result reference. This strategy might or might 704not work out. Here's an example where it didn't 705 706 >>> x = f(y, z) # x refers to some C++ X 707 >>> del y 708 >>> x.some_method() # CRASH! 709 710What's the problem? 711 712Well, what if f() was implemented as shown below: 713 714 X& f(Y& y, Z* z) 715 { 716 y.z = z; 717 return y.x; 718 } 719 720The problem is that the lifetime of result X& is tied to the lifetime 721of y, because the f() returns a reference to a member of the y 722object. This idiom is is not uncommon and perfectly acceptable in the 723context of C++. However, Python users should not be able to crash the 724system just by using our C++ interface. In this case deleting y will 725invalidate the reference to X. We have a dangling reference. 726 727Here's what's happening: 728 729# [^f] is called passing in a reference to [^y] and a pointer to [^z] 730# A reference to [^y.x] is returned 731# [^y] is deleted. [^x] is a dangling reference 732# [^x.some_method()] is called 733# [*BOOM!] 734 735We could copy result into a new object: 736 737[python] 738 739 >>> f(y, z).set(42) # Result disappears 740 >>> y.x.get() # No crash, but still bad 741 3.14 742 743This is not really our intent of our C++ interface. We've broken our 744promise that the Python interface should reflect the C++ interface as 745closely as possible. 746 747Our problems do not end there. Suppose Y is implemented as follows: 748 749[c++] 750 751 struct Y 752 { 753 X x; Z* z; 754 int z_value() { return z->value(); } 755 }; 756 757Notice that the data member [^z] is held by class Y using a raw 758pointer. Now we have a potential dangling pointer problem inside Y: 759 760 >>> x = f(y, z) # y refers to z 761 >>> del z # Kill the z object 762 >>> y.z_value() # CRASH! 763 764For reference, here's the implementation of [^f] again: 765 766 X& f(Y& y, Z* z) 767 { 768 y.z = z; 769 return y.x; 770 } 771 772Here's what's happening: 773 774# [^f] is called passing in a reference to [^y] and a pointer to [^z] 775# A pointer to [^z] is held by [^y] 776# A reference to [^y.x] is returned 777# [^z] is deleted. [^y.z] is a dangling pointer 778# [^y.z_value()] is called 779# [^z->value()] is called 780# [*BOOM!] 781 782[h2 Call Policies] 783 784Call Policies may be used in situations such as the example detailed above. 785In our example, [^return_internal_reference] and [^with_custodian_and_ward] 786are our friends: 787 788 def("f", f, 789 return_internal_reference<1, 790 with_custodian_and_ward<1, 2> >()); 791 792What are the [^1] and [^2] parameters, you ask? 793 794 return_internal_reference<1 795 796Informs Boost.Python that the first argument, in our case [^Y& y], is the 797owner of the returned reference: [^X&]. The "[^1]" simply specifies the 798first argument. In short: "return an internal reference [^X&] owned by the 7991st argument [^Y& y]". 800 801 with_custodian_and_ward<1, 2> 802 803Informs Boost.Python that the lifetime of the argument indicated by ward 804(i.e. the 2nd argument: [^Z* z]) is dependent on the lifetime of the 805argument indicated by custodian (i.e. the 1st argument: [^Y& y]). 806 807It is also important to note that we have defined two policies above. Two 808or more policies can be composed by chaining. Here's the general syntax: 809 810 policy1<args..., 811 policy2<args..., 812 policy3<args...> > > 813 814Here is the list of predefined call policies. A complete reference detailing 815these can be found [@../../../v2/reference.html#models_of_call_policies here]. 816 817* [*with_custodian_and_ward]: Ties lifetimes of the arguments 818* [*with_custodian_and_ward_postcall]: Ties lifetimes of the arguments and results 819* [*return_internal_reference]: Ties lifetime of one argument to that of result 820* [*return_value_policy<T> with T one of:] 821 * [*reference_existing_object]: naive (dangerous) approach 822 * [*copy_const_reference]: Boost.Python v1 approach 823 * [*copy_non_const_reference]: 824 * [*manage_new_object]: Adopt a pointer and hold the instance 825 826[blurb :-) [*Remember the Zen, Luke:] 827 828"Explicit is better than implicit" 829 830"In the face of ambiguity, refuse the temptation to guess" 831] 832 833[endsect] 834[section Overloading] 835 836The following illustrates a scheme for manually wrapping an overloaded 837member functions. Of course, the same technique can be applied to wrapping 838overloaded non-member functions. 839 840We have here our C++ class: 841 842 struct X 843 { 844 bool f(int a) 845 { 846 return true; 847 } 848 849 bool f(int a, double b) 850 { 851 return true; 852 } 853 854 bool f(int a, double b, char c) 855 { 856 return true; 857 } 858 859 int f(int a, int b, int c) 860 { 861 return a + b + c; 862 }; 863 }; 864 865Class X has 4 overloaded functions. We will start by introducing some 866member function pointer variables: 867 868 bool (X::*fx1)(int) = &X::f; 869 bool (X::*fx2)(int, double) = &X::f; 870 bool (X::*fx3)(int, double, char)= &X::f; 871 int (X::*fx4)(int, int, int) = &X::f; 872 873With these in hand, we can proceed to define and wrap this for Python: 874 875 .def("f", fx1) 876 .def("f", fx2) 877 .def("f", fx3) 878 .def("f", fx4) 879 880[endsect] 881[section Default Arguments] 882 883Boost.Python wraps (member) function pointers. Unfortunately, C++ function 884pointers carry no default argument info. Take a function [^f] with default 885arguments: 886 887 int f(int, double = 3.14, char const* = "hello"); 888 889But the type of a pointer to the function [^f] has no information 890about its default arguments: 891 892 int(*g)(int,double,char const*) = f; // defaults lost! 893 894When we pass this function pointer to the [^def] function, there is no way 895to retrieve the default arguments: 896 897 def("f", f); // defaults lost! 898 899Because of this, when wrapping C++ code, we had to resort to manual 900wrapping as outlined in the [link python.overloading previous section], or 901writing thin wrappers: 902 903 // write "thin wrappers" 904 int f1(int x) { return f(x); } 905 int f2(int x, double y) { return f(x,y); } 906 907 /*...*/ 908 909 // in module init 910 def("f", f); // all arguments 911 def("f", f2); // two arguments 912 def("f", f1); // one argument 913 914When you want to wrap functions (or member functions) that either: 915 916* have default arguments, or 917* are overloaded with a common sequence of initial arguments 918 919[h2 BOOST_PYTHON_FUNCTION_OVERLOADS] 920 921Boost.Python now has a way to make it easier. For instance, given a function: 922 923 int foo(int a, char b = 1, unsigned c = 2, double d = 3) 924 { 925 /*...*/ 926 } 927 928The macro invocation: 929 930 BOOST_PYTHON_FUNCTION_OVERLOADS(foo_overloads, foo, 1, 4) 931 932will automatically create the thin wrappers for us. This macro will create 933a class [^foo_overloads] that can be passed on to [^def(...)]. The third 934and fourth macro argument are the minimum arguments and maximum arguments, 935respectively. In our [^foo] function the minimum number of arguments is 1 936and the maximum number of arguments is 4. The [^def(...)] function will 937automatically add all the foo variants for us: 938 939 def("foo", foo, foo_overloads()); 940 941[h2 BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS] 942 943Objects here, objects there, objects here there everywhere. More frequently 944than anything else, we need to expose member functions of our classes to 945Python. Then again, we have the same inconveniences as before when default 946arguments or overloads with a common sequence of initial arguments come 947into play. Another macro is provided to make this a breeze. 948 949Like [^BOOST_PYTHON_FUNCTION_OVERLOADS], 950[^BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS] may be used to automatically create 951the thin wrappers for wrapping member functions. Let's have an example: 952 953 struct george 954 { 955 void 956 wack_em(int a, int b = 0, char c = 'x') 957 { 958 /*...*/ 959 } 960 }; 961 962The macro invocation: 963 964 BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS(george_overloads, wack_em, 1, 3) 965 966will generate a set of thin wrappers for george's [^wack_em] member function 967accepting a minimum of 1 and a maximum of 3 arguments (i.e. the third and 968fourth macro argument). The thin wrappers are all enclosed in a class named 969[^george_overloads] that can then be used as an argument to [^def(...)]: 970 971 .def("wack_em", &george::wack_em, george_overloads()); 972 973See the [@../../../v2/overloads.html#BOOST_PYTHON_FUNCTION_OVERLOADS-spec overloads reference] 974for details. 975 976[h2 init and optional] 977 978A similar facility is provided for class constructors, again, with 979default arguments or a sequence of overloads. Remember [^init<...>]? For example, 980given a class X with a constructor: 981 982 struct X 983 { 984 X(int a, char b = 'D', std::string c = "constructor", double d = 0.0); 985 /*...*/ 986 } 987 988You can easily add this constructor to Boost.Python in one shot: 989 990 .def(init<int, optional<char, std::string, double> >()) 991 992Notice the use of [^init<...>] and [^optional<...>] to signify the default 993(optional arguments). 994 995[endsect] 996[section Auto-Overloading] 997 998It was mentioned in passing in the previous section that 999[^BOOST_PYTHON_FUNCTION_OVERLOADS] and [^BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS] 1000can also be used for overloaded functions and member functions with a 1001common sequence of initial arguments. Here is an example: 1002 1003 void foo() 1004 { 1005 /*...*/ 1006 } 1007 1008 void foo(bool a) 1009 { 1010 /*...*/ 1011 } 1012 1013 void foo(bool a, int b) 1014 { 1015 /*...*/ 1016 } 1017 1018 void foo(bool a, int b, char c) 1019 { 1020 /*...*/ 1021 } 1022 1023Like in the previous section, we can generate thin wrappers for these 1024overloaded functions in one-shot: 1025 1026 BOOST_PYTHON_FUNCTION_OVERLOADS(foo_overloads, foo, 0, 3) 1027 1028Then... 1029 1030 .def("foo", (void(*)(bool, int, char))0, foo_overloads()); 1031 1032Notice though that we have a situation now where we have a minimum of zero 1033(0) arguments and a maximum of 3 arguments. 1034 1035[h2 Manual Wrapping] 1036 1037It is important to emphasize however that [*the overloaded functions must 1038have a common sequence of initial arguments]. Otherwise, our scheme above 1039will not work. If this is not the case, we have to wrap our functions 1040[link python.overloading manually]. 1041 1042Actually, we can mix and match manual wrapping of overloaded functions and 1043automatic wrapping through [^BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS] and 1044its sister, [^BOOST_PYTHON_FUNCTION_OVERLOADS]. Following up on our example 1045presented in the section [link python.overloading on overloading], since the 1046first 4 overload functins have a common sequence of initial arguments, we 1047can use [^BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS] to automatically wrap the 1048first three of the [^def]s and manually wrap just the last. Here's 1049how we'll do this: 1050 1051 BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS(xf_overloads, f, 1, 4) 1052 1053Create a member function pointers as above for both X::f overloads: 1054 1055 bool (X::*fx1)(int, double, char) = &X::f; 1056 int (X::*fx2)(int, int, int) = &X::f; 1057 1058Then... 1059 1060 .def("f", fx1, xf_overloads()); 1061 .def("f", fx2) 1062 1063[endsect] 1064[endsect] [/ Functions ] 1065 1066[section:object Object Interface] 1067 1068Python is dynamically typed, unlike C++ which is statically typed. Python 1069variables may hold an integer, a float, list, dict, tuple, str, long etc., 1070among other things. In the viewpoint of Boost.Python and C++, these 1071Pythonic variables are just instances of class [^object]. We will see in 1072this chapter how to deal with Python objects. 1073 1074As mentioned, one of the goals of Boost.Python is to provide a 1075bidirectional mapping between C++ and Python while maintaining the Python 1076feel. Boost.Python C++ [^object]s are as close as possible to Python. This 1077should minimize the learning curve significantly. 1078 1079[$images/python.png] 1080 1081[section Basic Interface] 1082 1083Class [^object] wraps [^PyObject*]. All the intricacies of dealing with 1084[^PyObject]s such as managing reference counting are handled by the 1085[^object] class. C++ object interoperability is seamless. Boost.Python C++ 1086[^object]s can in fact be explicitly constructed from any C++ object. 1087 1088To illustrate, this Python code snippet: 1089 1090[python] 1091 1092 def f(x, y): 1093 if (y == 'foo'): 1094 x[3:7] = 'bar' 1095 else: 1096 x.items += y(3, x) 1097 return x 1098 1099 def getfunc(): 1100 return f; 1101 1102Can be rewritten in C++ using Boost.Python facilities this way: 1103 1104[c++] 1105 1106 object f(object x, object y) { 1107 if (y == "foo") 1108 x.slice(3,7) = "bar"; 1109 else 1110 x.attr("items") += y(3, x); 1111 return x; 1112 } 1113 object getfunc() { 1114 return object(f); 1115 } 1116 1117Apart from cosmetic differences due to the fact that we are writing the 1118code in C++, the look and feel should be immediately apparent to the Python 1119coder. 1120 1121[endsect] 1122[section Derived Object types] 1123 1124Boost.Python comes with a set of derived [^object] types corresponding to 1125that of Python's: 1126 1127* list 1128* dict 1129* tuple 1130* str 1131* long_ 1132* enum 1133 1134These derived [^object] types act like real Python types. For instance: 1135 1136 str(1) ==> "1" 1137 1138Wherever appropriate, a particular derived [^object] has corresponding 1139Python type's methods. For instance, [^dict] has a [^keys()] method: 1140 1141 d.keys() 1142 1143[^make_tuple] is provided for declaring ['tuple literals]. Example: 1144 1145 make_tuple(123, 'D', "Hello, World", 0.0); 1146 1147In C++, when Boost.Python [^object]s are used as arguments to functions, 1148subtype matching is required. For example, when a function [^f], as 1149declared below, is wrapped, it will only accept instances of Python's 1150[^str] type and subtypes. 1151 1152 void f(str name) 1153 { 1154 object n2 = name.attr("upper")(); // NAME = name.upper() 1155 str NAME = name.upper(); // better 1156 object msg = "%s is bigger than %s" % make_tuple(NAME,name); 1157 } 1158 1159In finer detail: 1160 1161 str NAME = name.upper(); 1162 1163Illustrates that we provide versions of the str type's methods as C++ 1164member functions. 1165 1166 object msg = "%s is bigger than %s" % make_tuple(NAME,name); 1167 1168Demonstrates that you can write the C++ equivalent of [^"format" % x,y,z] 1169in Python, which is useful since there's no easy way to do that in std C++. 1170 1171[blurb 1172 __alert__ [*Beware] the common pitfall of forgetting that the constructors 1173 of most of Python's mutable types make copies, just as in Python. 1174] 1175 1176Python: 1177[python] 1178 1179 >>> d = dict(x.__dict__) # copies x.__dict__ 1180 >>> d['whatever'] = 3 # modifies the copy 1181 1182C++: 1183[c++] 1184 1185 dict d(x.attr("__dict__")); // copies x.__dict__ 1186 d['whatever'] = 3; // modifies the copy 1187 1188[h2 class_<T> as objects] 1189 1190Due to the dynamic nature of Boost.Python objects, any [^class_<T>] may 1191also be one of these types! The following code snippet wraps the class 1192(type) object. 1193 1194We can use this to create wrapped instances. Example: 1195 1196 object vec345 = ( 1197 class_<Vec2>("Vec2", init<double, double>()) 1198 .def_readonly("length", &Point::length) 1199 .def_readonly("angle", &Point::angle) 1200 )(3.0, 4.0); 1201 1202 assert(vec345.attr("length") == 5.0); 1203 1204[endsect] 1205[section Extracting C++ objects] 1206 1207At some point, we will need to get C++ values out of object instances. This 1208can be achieved with the [^extract<T>] function. Consider the following: 1209 1210 double x = o.attr("length"); // compile error 1211 1212In the code above, we got a compiler error because Boost.Python 1213[^object] can't be implicitly converted to [^double]s. Instead, what 1214we wanted to do above can be achieved by writing: 1215 1216 double l = extract<double>(o.attr("length")); 1217 Vec2& v = extract<Vec2&>(o); 1218 assert(l == v.length()); 1219 1220The first line attempts to extract the "length" attribute of the Boost.Python 1221[^object]. The second line attempts to ['extract] the [^Vec2] object from held 1222by the Boost.Python [^object]. 1223 1224Take note that we said "attempt to" above. What if the Boost.Python [^object] 1225does not really hold a [^Vec2] type? This is certainly a possibility considering 1226the dynamic nature of Python [^object]s. To be on the safe side, if the C++ type 1227can't be extracted, an appropriate exception is thrown. To avoid an exception, 1228we need to test for extractibility: 1229 1230 extract<Vec2&> x(o); 1231 if (x.check()) { 1232 Vec2& v = x(); ... 1233 1234__tip__ The astute reader might have noticed that the [^extract<T>] 1235facility in fact solves the mutable copying problem: 1236 1237 dict d = extract<dict>(x.attr("__dict__")); 1238 d["whatever"] = 3; // modifies x.__dict__ ! 1239 1240 1241[endsect] 1242[section Enums] 1243 1244Boost.Python has a nifty facility to capture and wrap C++ enums. While 1245Python has no [^enum] type, we'll often want to expose our C++ enums to 1246Python as an [^int]. Boost.Python's enum facility makes this easy while 1247taking care of the proper conversions from Python's dynamic typing to C++'s 1248strong static typing (in C++, ints cannot be implicitly converted to 1249enums). To illustrate, given a C++ enum: 1250 1251 enum choice { red, blue }; 1252 1253the construct: 1254 1255 enum_<choice>("choice") 1256 .value("red", red) 1257 .value("blue", blue) 1258 ; 1259 1260can be used to expose to Python. The new enum type is created in the 1261current [^scope()], which is usually the current module. The snippet above 1262creates a Python class derived from Python's [^int] type which is 1263associated with the C++ type passed as its first parameter. 1264 1265[note [*what is a scope?] 1266 1267The scope is a class that has an associated global Python object which 1268controls the Python namespace in which new extension classes and wrapped 1269functions will be defined as attributes. Details can be found 1270[@../../../v2/scope.html here].] 1271 1272You can access those values in Python as 1273 1274[python] 1275 1276 >>> my_module.choice.red 1277 my_module.choice.red 1278 1279where my_module is the module where the enum is declared. You can also 1280create a new scope around a class: 1281 1282[c++] 1283 1284 scope in_X = class_<X>("X") 1285 .def( ... ) 1286 .def( ... ) 1287 ; 1288 1289 // Expose X::nested as X.nested 1290 enum_<X::nested>("nested") 1291 .value("red", red) 1292 .value("blue", blue) 1293 ; 1294 1295[def Py_Initialize [@http://www.python.org/doc/current/api/initialization.html#l2h-652 Py_Initialize]] 1296[def Py_Finalize [@http://www.python.org/doc/current/api/initialization.html#l2h-656 Py_Finalize]] 1297[def Py_XINCREF [@http://www.python.org/doc/current/api/countingRefs.html#l2h-65 Py_XINCREF]] 1298[def Py_XDECREF [@http://www.python.org/doc/current/api/countingRefs.html#l2h-67 Py_XDECREF]] 1299[def PyImport_AppendInittab [@http://www.python.org/doc/current/api/importing.html#l2h-137 PyImport_AppendInittab]] 1300[def PyImport_AddModule [@http://www.python.org/doc/current/api/importing.html#l2h-125 PyImport_AddModule]] 1301[def PyModule_New [@http://www.python.org/doc/current/api/moduleObjects.html#l2h-591 PyModule_New]] 1302[def PyModule_GetDict [@http://www.python.org/doc/current/api/moduleObjects.html#l2h-594 PyModule_GetDict]] 1303 1304[endsect] 1305 1306[section:creating_python_object Creating `boost::python::object` from `PyObject*`] 1307 1308When you want a `boost::python::object` to manage a pointer to `PyObject*` pyobj one does: 1309 1310 boost::python::object o(boost::python::handle<>(pyobj)); 1311 1312In this case, the `o` object, manages the `pyobj`, it won’t increase the reference count on construction. 1313 1314Otherwise, to use a borrowed reference: 1315 1316 boost::python::object o(boost::python::handle<>(boost::python::borrowed(pyobj))); 1317 1318In this case, `Py_INCREF` is called, so `pyobj` is not destructed when object o goes out of scope. 1319 1320[endsect] [/ creating_python_object ] 1321 1322[endsect] [/ Object Interface] 1323 1324[section Embedding] 1325 1326By now you should know how to use Boost.Python to call your C++ code from 1327Python. However, sometimes you may need to do the reverse: call Python code 1328from the C++-side. This requires you to ['embed] the Python interpreter 1329into your C++ program. 1330 1331Currently, Boost.Python does not directly support everything you'll need 1332when embedding. Therefore you'll need to use the 1333[@http://www.python.org/doc/current/api/api.html Python/C API] to fill in 1334the gaps. However, Boost.Python already makes embedding a lot easier and, 1335in a future version, it may become unnecessary to touch the Python/C API at 1336all. So stay tuned... :-) 1337 1338[h2 Building embedded programs] 1339 1340To be able to embed python into your programs, you have to link to 1341both Boost.Python's as well as Python's own runtime library. 1342 1343Boost.Python's library comes in two variants. Both are located 1344in Boost's [^/libs/python/build/bin-stage] subdirectory. On Windows, the 1345variants are called [^boost_python.lib] (for release builds) and 1346[^boost_python_debug.lib] (for debugging). If you can't find the libraries, 1347you probably haven't built Boost.Python yet. See 1348[@../../../building.html Building and Testing] on how to do this. 1349 1350Python's library can be found in the [^/libs] subdirectory of 1351your Python directory. On Windows it is called pythonXY.lib where X.Y is 1352your major Python version number. 1353 1354Additionally, Python's [^/include] subdirectory has to be added to your 1355include path. 1356 1357In a Jamfile, all the above boils down to: 1358 1359[pre 1360projectroot c:\projects\embedded_program ; # location of the program 1361 1362# bring in the rules for python 1363SEARCH on python.jam = $(BOOST_BUILD_PATH) ; 1364include python.jam ; 1365 1366exe embedded_program # name of the executable 1367 : #sources 1368 embedded_program.cpp 1369 : # requirements 1370 <find-library>boost_python <library-path>c:\boost\libs\python 1371 $(PYTHON_PROPERTIES) 1372 <library-path>$(PYTHON_LIB_PATH) 1373 <find-library>$(PYTHON_EMBEDDED_LIBRARY) ; 1374] 1375 1376[h2 Getting started] 1377 1378Being able to build is nice, but there is nothing to build yet. Embedding 1379the Python interpreter into one of your C++ programs requires these 4 1380steps: 1381 1382# '''#include''' [^<boost/python.hpp>] 1383 1384# Call Py_Initialize() to start the interpreter and create the [^__main__] module. 1385 1386# Call other Python C API routines to use the interpreter. 1387 1388[/ # Call Py_Finalize() to stop the interpreter and release its resources.] 1389 1390[note [*Note that at this time you must not call Py_Finalize() to stop the 1391interpreter. This may be fixed in a future version of boost.python.] 1392] 1393 1394(Of course, there can be other C++ code between all of these steps.) 1395 1396[:['[*Now that we can embed the interpreter in our programs, lets see how to put it to use...]]] 1397 1398[section Using the interpreter] 1399 1400As you probably already know, objects in Python are reference-counted. 1401Naturally, the [^PyObject]s of the Python C API are also reference-counted. 1402There is a difference however. While the reference-counting is fully 1403automatic in Python, the Python C API requires you to do it 1404[@http://www.python.org/doc/current/c-api/refcounting.html by hand]. This is 1405messy and especially hard to get right in the presence of C++ exceptions. 1406Fortunately Boost.Python provides the [@../../../v2/handle.html handle] and 1407[@../../../v2/object.html object] class templates to automate the process. 1408 1409[h2 Running Python code] 1410 1411Boost.python provides three related functions to run Python code from C++. 1412 1413 object eval(str expression, object globals = object(), object locals = object()) 1414 object exec(str code, object globals = object(), object locals = object()) 1415 object exec_file(str filename, object globals = object(), object locals = object()) 1416 1417eval evaluates the given expression and returns the resulting value. 1418exec executes the given code (typically a set of statements) returning the result, 1419and exec_file executes the code contained in the given file. 1420 1421The [^globals] and [^locals] parameters are Python dictionaries 1422containing the globals and locals of the context in which to run the code. 1423For most intents and purposes you can use the namespace dictionary of the 1424[^__main__] module for both parameters. 1425 1426Boost.python provides a function to import a module: 1427 1428 object import(str name) 1429 1430import imports a python module (potentially loading it into the running process 1431first), and returns it. 1432 1433Let's import the [^__main__] module and run some Python code in its namespace: 1434 1435 object main_module = import("__main__"); 1436 object main_namespace = main_module.attr("__dict__"); 1437 1438 object ignored = exec("hello = file('hello.txt', 'w')\n" 1439 "hello.write('Hello world!')\n" 1440 "hello.close()", 1441 main_namespace); 1442 1443This should create a file called 'hello.txt' in the current directory 1444containing a phrase that is well-known in programming circles. 1445 1446[h2 Manipulating Python objects] 1447 1448Often we'd like to have a class to manipulate Python objects. 1449But we have already seen such a class above, and in the 1450[@python/object.html previous section]: the aptly named [^object] class 1451and its derivatives. We've already seen that they can be constructed from 1452a [^handle]. The following examples should further illustrate this fact: 1453 1454 object main_module = import("__main__"); 1455 object main_namespace = main_module.attr("__dict__"); 1456 object ignored = exec("result = 5 ** 2", main_namespace); 1457 int five_squared = extract<int>(main_namespace["result"]); 1458 1459Here we create a dictionary object for the [^__main__] module's namespace. 1460Then we assign 5 squared to the result variable and read this variable from 1461the dictionary. Another way to achieve the same result is to use eval instead, 1462which returns the result directly: 1463 1464 object result = eval("5 ** 2"); 1465 int five_squared = extract<int>(result); 1466 1467[h2 Exception handling] 1468 1469If an exception occurs in the evaluation of the python expression, 1470[@../../../v2/errors.html#error_already_set-spec error_already_set] is thrown: 1471 1472 try 1473 { 1474 object result = eval("5/0"); 1475 // execution will never get here: 1476 int five_divided_by_zero = extract<int>(result); 1477 } 1478 catch(error_already_set const &) 1479 { 1480 // handle the exception in some way 1481 } 1482 1483The [^error_already_set] exception class doesn't carry any information in itself. 1484To find out more about the Python exception that occurred, you need to use the 1485[@http://www.python.org/doc/api/exceptionHandling.html exception handling functions] 1486of the Python C API in your catch-statement. This can be as simple as calling 1487[@http://www.python.org/doc/api/exceptionHandling.html#l2h-70 PyErr_Print()] to 1488print the exception's traceback to the console, or comparing the type of the 1489exception with those of the [@http://www.python.org/doc/api/standardExceptions.html 1490standard exceptions]: 1491 1492 catch(error_already_set const &) 1493 { 1494 if (PyErr_ExceptionMatches(PyExc_ZeroDivisionError)) 1495 { 1496 // handle ZeroDivisionError specially 1497 } 1498 else 1499 { 1500 // print all other errors to stderr 1501 PyErr_Print(); 1502 } 1503 } 1504 1505(To retrieve even more information from the exception you can use some of the other 1506exception handling functions listed [@http://www.python.org/doc/api/exceptionHandling.html here].) 1507 1508[endsect] 1509[endsect] [/ Embedding] 1510 1511[section Iterators] 1512 1513In C++, and STL in particular, we see iterators everywhere. Python also has 1514iterators, but these are two very different beasts. 1515 1516[*C++ iterators:] 1517 1518* C++ has 5 type categories (random-access, bidirectional, forward, input, output) 1519* There are 2 Operation categories: reposition, access 1520* A pair of iterators is needed to represent a (first/last) range. 1521 1522[*Python Iterators:] 1523 1524* 1 category (forward) 1525* 1 operation category (next()) 1526* Raises StopIteration exception at end 1527 1528The typical Python iteration protocol: [^[*for y in x...]] is as follows: 1529 1530[python] 1531 1532 iter = x.__iter__() # get iterator 1533 try: 1534 while 1: 1535 y = iter.next() # get each item 1536 ... # process y 1537 except StopIteration: pass # iterator exhausted 1538 1539Boost.Python provides some mechanisms to make C++ iterators play along 1540nicely as Python iterators. What we need to do is to produce 1541appropriate `__iter__` function from C++ iterators that is compatible 1542with the Python iteration protocol. For example: 1543 1544[c++] 1545 1546 object get_iterator = iterator<vector<int> >(); 1547 object iter = get_iterator(v); 1548 object first = iter.next(); 1549 1550Or for use in class_<>: 1551 1552 .def("__iter__", iterator<vector<int> >()) 1553 1554[*range] 1555 1556We can create a Python savvy iterator using the range function: 1557 1558* range(start, finish) 1559* range<Policies,Target>(start, finish) 1560 1561Here, start/finish may be one of: 1562 1563* member data pointers 1564* member function pointers 1565* adaptable function object (use Target parameter) 1566 1567[*iterator] 1568 1569* iterator<T, Policies>() 1570 1571Given a container [^T], iterator is a shortcut that simply calls [^range] 1572with &T::begin, &T::end. 1573 1574Let's put this into action... Here's an example from some hypothetical 1575bogon Particle accelerator code: 1576 1577[python] 1578 1579 f = Field() 1580 for x in f.pions: 1581 smash(x) 1582 for y in f.bogons: 1583 count(y) 1584 1585Now, our C++ Wrapper: 1586 1587[c++] 1588 1589 class_<F>("Field") 1590 .property("pions", range(&F::p_begin, &F::p_end)) 1591 .property("bogons", range(&F::b_begin, &F::b_end)); 1592 1593[*stl_input_iterator] 1594 1595So far, we have seen how to expose C++ iterators and ranges to Python. 1596Sometimes we wish to go the other way, though: we'd like to pass a 1597Python sequence to an STL algorithm or use it to initialize an STL 1598container. We need to make a Python iterator look like an STL iterator. 1599For that, we use `stl_input_iterator<>`. Consider how we might 1600implement a function that exposes `std::list<int>::assign()` to 1601Python: 1602 1603[c++] 1604 1605 template<typename T> 1606 void list_assign(std::list<T>& l, object o) { 1607 // Turn a Python sequence into an STL input range 1608 stl_input_iterator<T> begin(o), end; 1609 l.assign(begin, end); 1610 } 1611 1612 // Part of the wrapper for list<int> 1613 class_<std::list<int> >("list_int") 1614 .def("assign", &list_assign<int>) 1615 // ... 1616 ; 1617 1618Now in Python, we can assign any integer sequence to `list_int` objects: 1619 1620[python] 1621 1622 x = list_int(); 1623 x.assign([1,2,3,4,5]) 1624 1625[endsect] 1626[section:exception Exception Translation] 1627 1628All C++ exceptions must be caught at the boundary with Python code. This 1629boundary is the point where C++ meets Python. Boost.Python provides a 1630default exception handler that translates selected standard exceptions, 1631then gives up: 1632 1633 raise RuntimeError, 'unidentifiable C++ Exception' 1634 1635Users may provide custom translation. Here's an example: 1636 1637 struct PodBayDoorException; 1638 void translator(PodBayDoorException const& x) { 1639 PyErr_SetString(PyExc_UserWarning, "I'm sorry Dave..."); 1640 } 1641 BOOST_PYTHON_MODULE(kubrick) { 1642 register_exception_translator< 1643 PodBayDoorException>(translator); 1644 ... 1645 1646[endsect] 1647[section:techniques General Techniques] 1648 1649Here are presented some useful techniques that you can use while wrapping code with Boost.Python. 1650 1651[section Creating Packages] 1652 1653A Python package is a collection of modules that provide to the user a certain 1654functionality. If you're not familiar on how to create packages, a good 1655introduction to them is provided in the 1656[@http://www.python.org/doc/current/tut/node8.html Python Tutorial]. 1657 1658But we are wrapping C++ code, using Boost.Python. How can we provide a nice 1659package interface to our users? To better explain some concepts, let's work 1660with an example. 1661 1662We have a C++ library that works with sounds: reading and writing various 1663formats, applying filters to the sound data, etc. It is named (conveniently) 1664[^sounds]. Our library already has a neat C++ namespace hierarchy, like so: 1665 1666 sounds::core 1667 sounds::io 1668 sounds::filters 1669 1670We would like to present this same hierarchy to the Python user, allowing him 1671to write code like this: 1672 1673 import sounds.filters 1674 sounds.filters.echo(...) # echo is a C++ function 1675 1676The first step is to write the wrapping code. We have to export each module 1677separately with Boost.Python, like this: 1678 1679 /* file core.cpp */ 1680 BOOST_PYTHON_MODULE(core) 1681 { 1682 /* export everything in the sounds::core namespace */ 1683 ... 1684 } 1685 1686 /* file io.cpp */ 1687 BOOST_PYTHON_MODULE(io) 1688 { 1689 /* export everything in the sounds::io namespace */ 1690 ... 1691 } 1692 1693 /* file filters.cpp */ 1694 BOOST_PYTHON_MODULE(filters) 1695 { 1696 /* export everything in the sounds::filters namespace */ 1697 ... 1698 } 1699 1700Compiling these files will generate the following Python extensions: 1701[^core.pyd], [^io.pyd] and [^filters.pyd]. 1702 1703[note The extension [^.pyd] is used for python extension modules, which 1704are just shared libraries. Using the default for your system, like [^.so] for 1705Unix and [^.dll] for Windows, works just as well.] 1706 1707Now, we create this directory structure for our Python package: 1708 1709[pre 1710sounds/ 1711 \_\_init\_\_.py 1712 core.pyd 1713 filters.pyd 1714 io.pyd 1715] 1716 1717The file [^\_\_init\_\_.py] is what tells Python that the directory [^sounds/] is 1718actually a Python package. It can be a empty file, but can also perform some 1719magic, that will be shown later. 1720 1721Now our package is ready. All the user has to do is put [^sounds] into his 1722[@http://www.python.org/doc/current/tut/node8.html#SECTION008110000000000000000 PYTHONPATH] 1723and fire up the interpreter: 1724 1725[python] 1726 1727 >>> import sounds.io 1728 >>> import sounds.filters 1729 >>> sound = sounds.io.open('file.mp3') 1730 >>> new_sound = sounds.filters.echo(sound, 1.0) 1731 1732Nice heh? 1733 1734This is the simplest way to create hierarchies of packages, but it is not very 1735flexible. What if we want to add a ['pure] Python function to the filters 1736package, for instance, one that applies 3 filters in a sound object at once? 1737Sure, you can do this in C++ and export it, but why not do so in Python? You 1738don't have to recompile the extension modules, plus it will be easier to write 1739it. 1740 1741If we want this flexibility, we will have to complicate our package hierarchy a 1742little. First, we will have to change the name of the extension modules: 1743 1744[c++] 1745 1746 /* file core.cpp */ 1747 BOOST_PYTHON_MODULE(_core) 1748 { 1749 ... 1750 /* export everything in the sounds::core namespace */ 1751 } 1752 1753Note that we added an underscore to the module name. The filename will have to 1754be changed to [^_core.pyd] as well, and we do the same to the other extension modules. 1755Now, we change our package hierarchy like so: 1756 1757[pre 1758sounds/ 1759 \_\_init\_\_.py 1760 core/ 1761 \_\_init\_\_.py 1762 _core.pyd 1763 filters/ 1764 \_\_init\_\_.py 1765 _filters.pyd 1766 io/ 1767 \_\_init\_\_.py 1768 _io.pyd 1769] 1770 1771Note that we created a directory for each extension module, and added a 1772\_\_init\_\_.py to each one. But if we leave it that way, the user will have to 1773access the functions in the core module with this syntax: 1774 1775[python] 1776 1777 >>> import sounds.core._core 1778 >>> sounds.core._core.foo(...) 1779 1780which is not what we want. But here enters the [^\_\_init\_\_.py] magic: everything 1781that is brought to the [^\_\_init\_\_.py] namespace can be accessed directly by the 1782user. So, all we have to do is bring the entire namespace from [^_core.pyd] 1783to [^core/\_\_init\_\_.py]. So add this line of code to [^sounds/core/\_\_init\_\_.py]: 1784 1785 from _core import * 1786 1787We do the same for the other packages. Now the user accesses the functions and 1788classes in the extension modules like before: 1789 1790 >>> import sounds.filters 1791 >>> sounds.filters.echo(...) 1792 1793with the additional benefit that we can easily add pure Python functions to 1794any module, in a way that the user can't tell the difference between a C++ 1795function and a Python function. Let's add a ['pure] Python function, 1796[^echo_noise], to the [^filters] package. This function applies both the 1797[^echo] and [^noise] filters in sequence in the given [^sound] object. We 1798create a file named [^sounds/filters/echo_noise.py] and code our function: 1799 1800 import _filters 1801 def echo_noise(sound): 1802 s = _filters.echo(sound) 1803 s = _filters.noise(sound) 1804 return s 1805 1806Next, we add this line to [^sounds/filters/\_\_init\_\_.py]: 1807 1808 from echo_noise import echo_noise 1809 1810And that's it. The user now accesses this function like any other function 1811from the [^filters] package: 1812 1813 >>> import sounds.filters 1814 >>> sounds.filters.echo_noise(...) 1815 1816[endsect] 1817[section Extending Wrapped Objects in Python] 1818 1819Thanks to Python's flexibility, you can easily add new methods to a class, 1820even after it was already created: 1821 1822 >>> class C(object): pass 1823 >>> 1824 >>> # a regular function 1825 >>> def C_str(self): return 'A C instance!' 1826 >>> 1827 >>> # now we turn it in a member function 1828 >>> C.__str__ = C_str 1829 >>> 1830 >>> c = C() 1831 >>> print c 1832 A C instance! 1833 >>> C_str(c) 1834 A C instance! 1835 1836Yes, Python rox. :-) 1837 1838We can do the same with classes that were wrapped with Boost.Python. Suppose 1839we have a class [^point] in C++: 1840 1841[c++] 1842 1843 class point {...}; 1844 1845 BOOST_PYTHON_MODULE(_geom) 1846 { 1847 class_<point>("point")...; 1848 } 1849 1850If we are using the technique from the previous session, 1851[link python.creating_packages Creating Packages], we can code directly 1852into [^geom/\_\_init\_\_.py]: 1853 1854[python] 1855 1856 from _geom import * 1857 1858 # a regular function 1859 def point_str(self): 1860 return str((self.x, self.y)) 1861 1862 # now we turn it into a member function 1863 point.__str__ = point_str 1864 1865[*All] point instances created from C++ will also have this member function! 1866This technique has several advantages: 1867 1868* Cut down compile times to zero for these additional functions 1869* Reduce the memory footprint to virtually zero 1870* Minimize the need to recompile 1871* Rapid prototyping (you can move the code to C++ if required without changing the interface) 1872 1873You can even add a little syntactic sugar with the use of metaclasses. Let's 1874create a special metaclass that "injects" methods in other classes. 1875 1876 # The one Boost.Python uses for all wrapped classes. 1877 # You can use here any class exported by Boost instead of "point" 1878 BoostPythonMetaclass = point.__class__ 1879 1880 class injector(object): 1881 class __metaclass__(BoostPythonMetaclass): 1882 def __init__(self, name, bases, dict): 1883 for b in bases: 1884 if type(b) not in (self, type): 1885 for k,v in dict.items(): 1886 setattr(b,k,v) 1887 return type.__init__(self, name, bases, dict) 1888 1889 # inject some methods in the point foo 1890 class more_point(injector, point): 1891 def __repr__(self): 1892 return 'Point(x=%s, y=%s)' % (self.x, self.y) 1893 def foo(self): 1894 print 'foo!' 1895 1896Now let's see how it got: 1897 1898 >>> print point() 1899 Point(x=10, y=10) 1900 >>> point().foo() 1901 foo! 1902 1903Another useful idea is to replace constructors with factory functions: 1904 1905 _point = point 1906 1907 def point(x=0, y=0): 1908 return _point(x, y) 1909 1910In this simple case there is not much gained, but for constructurs with 1911many overloads and/or arguments this is often a great simplification, again 1912with virtually zero memory footprint and zero compile-time overhead for 1913the keyword support. 1914 1915[endsect] 1916[section Reducing Compiling Time] 1917 1918If you have ever exported a lot of classes, you know that it takes quite a good 1919time to compile the Boost.Python wrappers. Plus the memory consumption can 1920easily become too high. If this is causing you problems, you can split the 1921class_ definitions in multiple files: 1922 1923[c++] 1924 1925 /* file point.cpp */ 1926 #include <point.h> 1927 #include <boost/python.hpp> 1928 1929 void export_point() 1930 { 1931 class_<point>("point")...; 1932 } 1933 1934 /* file triangle.cpp */ 1935 #include <triangle.h> 1936 #include <boost/python.hpp> 1937 1938 void export_triangle() 1939 { 1940 class_<triangle>("triangle")...; 1941 } 1942 1943Now you create a file [^main.cpp], which contains the [^BOOST_PYTHON_MODULE] 1944macro, and call the various export functions inside it. 1945 1946 void export_point(); 1947 void export_triangle(); 1948 1949 BOOST_PYTHON_MODULE(_geom) 1950 { 1951 export_point(); 1952 export_triangle(); 1953 } 1954 1955Compiling and linking together all this files produces the same result as the 1956usual approach: 1957 1958 #include <boost/python.hpp> 1959 #include <point.h> 1960 #include <triangle.h> 1961 1962 BOOST_PYTHON_MODULE(_geom) 1963 { 1964 class_<point>("point")...; 1965 class_<triangle>("triangle")...; 1966 } 1967 1968but the memory is kept under control. 1969 1970This method is recommended too if you are developing the C++ library and 1971exporting it to Python at the same time: changes in a class will only demand 1972the compilation of a single cpp, instead of the entire wrapper code. 1973 1974[note If you're exporting your classes with [@../../../../pyste/index.html Pyste], 1975take a look at the [^--multiple] option, that generates the wrappers in 1976various files as demonstrated here.] 1977 1978[note This method is useful too if you are getting the error message 1979['"fatal error C1204:Compiler limit:internal structure overflow"] when compiling 1980a large source file, as explained in the [@../../../v2/faq.html#c1204 FAQ].] 1981 1982[endsect] 1983[endsect] [/ General Techniques] 1984 1985 1986