1# Copyright (C) 2005-2010, Parrot Foundation. 2 3=head1 PDD 21: Namespaces 4 5=head2 Abstract 6 7Description and implementation of Parrot namespaces. 8 9=head2 Description 10 11=over 4 12 13=item - Namespaces should be stored under first-level namespaces corresponding 14to the HLL language name 15 16=item - Namespaces should be hierarchical 17 18=item - The C<get_namespace> opcode takes a multidimensional hash key or an 19array of name strings 20 21=item - Namespaces follow the semantics of the HLL in which they're defined 22 23=item - exports follow the semantics of the library's language 24 25=item - Two interfaces: typed and untyped 26 27=back 28 29=head2 Definitions 30 31=head3 "HLL" 32 33A High Level Language, such as Perl, Python, or Tcl, in contrast to PIR, which 34is a low-level language. 35 36=head3 "current namespace" 37 38The I<current namespace> at runtime is the namespace associated with the 39currently executing subroutine. PASM assigns each subroutine a namespace when 40compilation of the subroutine begins. Don't change the associated namespace 41of a subroutine unless you're prepared for weird consequences. 42 43(PASM also has its own separate concept of current namespace which is used to 44initialize the runtime current namespace as well as determine where to store 45compiled symbols.) 46 47 48=head2 Implementation 49 50=head3 Namespace Indexing Syntax 51 52Namespaces are denoted in Parrot as simple strings, multidimensional 53hash keys, or arrays of name strings. 54 55A namespace may appear in Parrot source code as the string C<"a"> or the 56key C<["a"]>. 57 58A nested namespace "b" inside the namespace "a" will appear as the key 59C<["a"; "b"]>. 60 61There is no limit to namespace nesting. 62 63=head3 Naming Conventions 64 65Parrot's target languages have a wide variety of namespace models. By 66implementing an API and standard conventions, it should be possible to 67allow interoperability while still allowing each one to choose the best 68internal representation. 69 70 71=over 4 72 73=item True Root Namespace 74 75The true root namespace is hidden from common usage, but it is available 76via the C<get_root_namespace> opcode. For example: 77 78 $P0 = get_root_namespace 79 80This root namespace stringifies to the empty string. 81 82=item HLL Root Namespaces 83 84Each HLL must store public items in a namespace named with the lowercased 85name of the HLL. This is the HLL root namespace. For instance, Tcl's 86user-created namespaces should live in the C<tcl> namespace. This 87eliminates any accidental collisions between languages. 88 89An HLL root namespace must be stored at the first level in Parrot's namespace 90hierarchy. These top-level namespaces should also be specified in a standard 91unicode encoding. The reasons for these restrictions is to allow compilers 92to remain completely ignorant of each other. 93 94Parrot internals are stored in the default HLL root namespace C<parrot>. 95 96=item HLL Implementation Namespaces 97 98Each HLL must store implementation internals (private items) in an HLL 99root namespace named with an underscore and the lowercased name of the 100HLL. For instance, Tcl's implementation internals should live in the 101C<_tcl> namespace. 102 103=item HLL User-Created Namespaces 104 105Each HLL must store all user-created namespaces under the HLL root 106namespace. It is suggested that HLLs use hierarchical namespaces to 107practical extent. A single flat namespace can be made to work, but it 108complicates symbol exportation. 109 110=back 111 112=head3 Namespace PMC API 113 114Most languages leave their symbols plain, which makes lookups quite 115straightforward. Others use sigils or other mangling techniques, complicating 116the problem of interoperability. 117 118Parrot namespaces assist with interoperability by providing two interface 119subsets: the I<untyped interface> and the I<typed interface>. 120 121=head4 Untyped Interface 122 123Each HLL may, when working with its own namespace objects, use the I<untyped 124interface>, which allows direct naming in the native style of the namespace's 125HLL. 126 127This interface consists of the standard Parrot hash interface, with all its 128keys, values, lookups, deletions, etc. Just treat the namespace like a 129hash. (It probably is one, really, deep down.) 130 131The untyped interface also has one method: 132 133=over 4 134 135=item C<get_name> 136 137=begin PIR_FRAGMENT 138 139 $P1 = $P2.'get_name'() 140 141=end PIR_FRAGMENT 142 143Gets the name of the namespace $P2 as an array of strings. For example, 144if $P2 is a Perl 5 namespace "Some::Module", within the Perl 5 HLL, then 145get_name() on $P2 returns an array of "perl5", "Some", "Module". It 146returns the literal namespace names as the HLL stored them, without 147filtering for name mangling. 148 149NOTE: Due to aliasing, this value may be wrong -- i.e. it may disagree with 150the namespace name with which you found the namespace in the first place. 151 152=back 153 154=head4 Typed Interface 155 156When a given namespace's HLL is either different from the current HLL or 157unknown, an HLL should generally use only the language-agnostic namespace 158interface. This interface isolates HLLs from each others' naming quirks. It 159consists of C<add_foo()>, C<find_foo()>, and C<del_foo()> methods, for 160values of "foo" including "sub" (something executable), "namespace" 161(something in which to find more names), and "var" (anything). 162 163NOTE: The job of the typed interface is to bridge I<naming> differences, and 164I<only> naming differences. Therefore: 1) It does not enforce, nor even 165notice, the interface requirements of "sub" or "namespace": e.g. 166execution of C<add_sub("foo", $P0)> does I<not> automatically guarantee 167that $P0 is an invokable subroutine; and 2) it does not prevent 168overwriting one type with another. 169 170=over 4 171 172=item C<add_namespace> 173 174=begin PIR_FRAGMENT 175 176 $P1.'add_namespace'($S2, $P3) 177 178=end PIR_FRAGMENT 179 180Store $P3 as a namespace under the namespace $P1, with the name of $S2. 181 182=item C<add_sub> 183 184=begin PIR_FRAGMENT 185 186 $P1.'add_sub'($S2, $P3) 187 188=end PIR_FRAGMENT 189 190Store $P3 as a subroutine with the name of $S2 in the namespace $P1. 191 192=item C<add_var> 193 194=begin PIR_FRAGMENT 195 196 $P1.'add_var'($S2, $P3) 197 198=end PIR_FRAGMENT 199 200Store $P3 as a variable with the name of $S2 in the namespace $P1. 201 202IMPLEMENTATION NOTE: Perl namespace implementations may choose to implement 203add_var() by checking which parts of the variable interface are 204implemented by $P0 (scalar, array, and/or hash) so it can decide on an 205appropriate sigil. 206 207=item C<del_namespace>, C<del_sub>, C<del_var> 208 209=begin PIR_FRAGMENT 210 211 $P1.'del_namespace'($S2) 212 $P1.'del_sub'($S2) 213 $P1.'del_var'($S2) 214 215=end PIR_FRAGMENT 216 217Delete the sub, namespace, or variable named $S2 from the namespace $P1. 218 219=item C<find_namespace>, C<find_sub>, C<find_var> 220 221=begin PIR_FRAGMENT 222 223 $P1 = $P2.'find_namespace'($S3) 224 $P1 = $P2.'find_sub'($S3) 225 $P1 = $P2.'find_var'($S3) 226 227=end PIR_FRAGMENT 228 229Find the sub, namespace, or variable named $S3 in the namespace $P2. 230 231IMPLEMENTATION NOTE: Perl namespace implementations should implement 232find_var() to check all variable sigils, but the order is not to be counted on 233by users. If you're planning to let Python code see your module, you should 234avoid exporting both C<our $A> and C<our @A>. (Well, you might want to 235consider not exporting variables at all, but that's a style issue.) 236 237=item C<export_to> 238 239=begin PIR_FRAGMENT 240 241 $P1.'export_to'($P2, $P3) 242 243=end PIR_FRAGMENT 244 245Export items from the namespace $P1 into the namespace $P2. The items to 246export are named in $P3, which may be an array of strings, a hash, or null. 247If $P3 is an array of strings, interpretation of items in an array follows 248the conventions of the source (exporting) namespace. 249If $P3 is a hash, the keys correspond to the names in the source namespace, 250and the values correspond to the names in the destination namespace. 251If a hash value is null or an empty string, the name in the hash key is used. 252A null $P3 requests the 'default' set of items. 253Any other type passed into $P3 throws an exception. 254 255The base Parrot namespace export_to() function interprets item names as 256literals -- no wildcards or other special meaning. There is no default list 257of items to export, so $P3 of null and $P3 of an empty array have the same 258behavior. 259 260NOTE: Exportation may entail non-obvious, odd, or even mischievous behavior. 261For example, Perl's pragmata are implemented as exports, and they don't 262actually export anything. 263 264IMPLEMENTATION EXAMPLES: Suppose a Perl program were to import some Tcl module 265with an import pattern of "c*" -- something that might be expressed in Perl 6 266as C<use tcl:Some::Module 'c*'>. This operation would import all the commands 267that start with 'c' from the given Tcl namespace into the current Perl 268namespace. This is so because, regardless of whether 'c*' is a Perl 6 style 269export pattern, it I<is> a valid Tcl export pattern. 270 271{XXX - The ':' for HLL is just proposed. This example will need to be 272updated later.} 273 274IMPLEMENTATION NOTE: Most namespace C<export_to> implementations will restrict 275themselves to using the typed interface on the target namespace. However, 276they may also decide to check the type of the target namespace and, if it 277turns out to be of a compatible type, to use same-language shortcuts. 278 279DESIGN TODO: Figure out a good convention for a default export list in the 280base namespace PMC. Maybe a standard method "expand_export_list()"? 281 282=back 283 284=head3 Compiler PMC API 285 286=head4 Methods 287 288=over 4 289 290=item C<parse_name> 291 292=begin PIR_FRAGMENT 293 294 $P1 = $P2.'parse_name'($S3) 295 296=end PIR_FRAGMENT 297 298Parse the name in $S3 using the rules specific to the compiler $P2, and 299return an array of individual name elements. 300 301For example, a Java compiler would turn 'C<a.b.c>' to C<['a','b','c']>, 302while a Perl compiler would turn 'C<a::b::c>' into the same result. 303Meanwhile, due to Perl's sigil rules, 'C<$a::b::c>' would become 304C<['a','b','$c']>. 305 306=item C<get_namespace> 307 308=begin PIR_FRAGMENT 309 310 $P1 = $P2.'get_namespace'($P3) 311 312=end PIR_FRAGMENT 313 314Ask the compiler $P2 to find its namespace which is named by the 315elements of the array in $P3. If $P3 is a null PMC or an empty array, 316C<get_namespace> retrieves the base namespace for the HLL. It returns a 317namespace PMC on success and a null PMC on failure. 318 319This method allows other HLLs to know one name (the HLL) and then work with 320that HLL's modules without having to know the name it chose for its namespace 321tree. (If you really want to know the name, the get_name() method should work 322on the returned namespace PMC.) 323 324Note that this method is basically a convenience and/or performance hack, as 325it does the equivalent of C<get_root_namespace> followed by 326zero or more calls to <namespace>.get_namespace(). However, any compiler is 327free to cheat if it doesn't get caught, e.g. to use the untyped namespace 328interface if the language doesn't mangle namespace names. 329 330=item C<load_library> 331 332=begin PIR_FRAGMENT 333 334 $P1.'load_library'($P2, $P3) 335 336=end PIR_FRAGMENT 337 338Ask this compiler to load a library/module named by the elements of the array 339in $P2, with optional control information in $P3. 340 341For example, Perl 5's module named "Some::Module" should be loaded using (in 342pseudo Perl 6): C<perl5.load_library(["Some", "Module"], null)>. 343 344The meaning of $P3 is compiler-specific. The only universal legal value is 345Null, which requests a "normal" load. The meaning of "normal" varies, but 346the ideal would be to perform only the minimal actions required. 347 348On failure, an exception is thrown. 349 350=back 351 352=head3 Subroutine PMC API 353 354Some information must be available about subroutines to implement the correct 355behavior about namespaces. 356 357=head4 Methods 358 359=over 4 360 361=item C<get_namespace> 362 363=begin PIR_FRAGMENT 364 365 $P1 = $P2.'get_namespace'() 366 367=end PIR_FRAGMENT 368 369Retrieve the namespace $P1 where the subroutine $P2 was defined. (As 370opposed to the namespace(s) that it may have been exported to.) 371 372=back 373 374=head3 Namespace Opcodes 375 376The namespace opcodes all have 3 variants: one that operates from the 377currently selected namespace (i.e. the namespace of the currently 378executing subroutine), one that operates from the HLL root namespace 379(identified by "hll" in the opcode name), and one that operates from the 380true root namespace (identified by "root" in the name). 381 382=over 4 383 384=item C<set_namespace> 385 386=begin PIR_FRAGMENT_INVALID 387 388 set_namespace ['key'], $P1 389 set_hll_namespace ['key'], $P1 390 set_root_namespace ['key'], $P1 391 392=end PIR_FRAGMENT_INVALID 393 394Add the namespace PMC $P1 under the name denoted by a multidimensional 395hash key. 396 397=begin PIR_FRAGMENT_INVALID 398 399 set_namespace $P1, $P2 400 set_hll_namespace $P1, $P2 401 set_root_namespace $P1, $P2 402 403=end PIR_FRAGMENT_INVALID 404 405Add the namespace PMC $P2 under the name denoted by an array of name 406strings $P1. 407 408=item C<get_namespace> 409 410=begin PIR_FRAGMENT 411 412 $P1 = get_namespace 413 $P1 = get_hll_namespace 414 $P1 = get_root_namespace 415 416=end PIR_FRAGMENT 417 418Retrieve the current namespace, the HLL root namespace, or the true 419root namespace and store it in $P1. 420 421=begin PIR_FRAGMENT_INVALID 422 423 $P1 = get_namespace [key] 424 $P1 = get_hll_namespace [key] 425 $P1 = get_root_namespace [key] 426 427=end PIR_FRAGMENT_INVALID 428 429Retrieve the namespace denoted by a multidimensional hash key and 430store it in C<$P1>. 431 432=begin PIR_FRAGMENT 433 434 $P1 = get_namespace $P2 435 $P1 = get_hll_namespace $P2 436 $P1 = get_root_namespace $P2 437 438=end PIR_FRAGMENT 439 440Retrieve the namespace denoted by the array of names $P2 and store it in 441C<$P1>. 442 443Thus, to get the "Foo::Bar" namespace from the top-level of the HLL if 444the name was known at compile time, you could retrieve the namespace 445with a key: 446 447=begin PIR_FRAGMENT 448 449 $P0 = get_hll_namespace ["Foo"; "Bar"] 450 451=end PIR_FRAGMENT 452 453If the name was not known at compile time, you would retrieve the 454namespace with an array instead: 455 456=begin PIR_FRAGMENT 457 458 $P1 = split "::", "Foo::Bar" 459 $P0 = get_hll_namespace $P1 460 461=end PIR_FRAGMENT 462 463=item C<make_namespace> 464 465=begin PIR_FRAGMENT_INVALID 466 467 $P1 = make_namespace [key] 468 $P1 = make_hll_namespace [key] 469 $P1 = make_root_namespace [key] 470 471=end PIR_FRAGMENT_INVALID 472 473Create and retrieve the namespace denoted by a multidimensional hash key 474and store it in C<$P1>. If the namespace already exists, only retrieve 475it. 476 477=begin PIR_FRAGMENT_INVALID 478 479 $P1 = make_namespace $P2 480 $P1 = make_hll_namespace $P2 481 $P1 = make_root_namespace $P2 482 483=end PIR_FRAGMENT_INVALID 484 485Create and retrieve the namespace denoted by the array of names $P2 and 486store it in C<$P1>. If the namespace already exists, only retrieve it. 487 488=item C<get_global> 489 490=begin PIR_FRAGMENT 491 492 $P1 = get_global $S2 493 $P1 = get_hll_global $S2 494 $P1 = get_root_global $S2 495 496=end PIR_FRAGMENT 497 498Retrieve the symbol named $S2 in the current namespace, HLL root 499namespace, or true root namespace. 500 501=begin PIR_FRAGMENT 502 503 .local pmc key 504 $P1 = get_global [key], $S2 505 $P1 = get_hll_global [key], $S2 506 $P1 = get_root_global [key], $S2 507 508=end PIR_FRAGMENT 509 510Retrieve the symbol named $S2 by a multidimensional hash key relative 511to the current namespace, HLL root namespace, or true root namespace. 512 513=begin PIR_FRAGMENT 514 515 $P1 = get_global $P2, $S3 516 $P1 = get_hll_global $P2, $S3 517 $P1 = get_root_global $P2, $S3 518 519=end PIR_FRAGMENT 520 521Retrieve the symbol named $S3 by the array of names $P2 relative to the 522current namespace, HLL root namespace, or true root namespace. 523 524=item C<set_global> 525 526=begin PIR_FRAGMENT 527 528 set_global $S1, $P2 529 set_hll_global $S1, $P2 530 set_root_global $S1, $P2 531 532=end PIR_FRAGMENT 533 534Store $P2 as the symbol named $S1 in the current namespace, HLL root 535namespace, or true root namespace. 536 537=begin PIR_FRAGMENT 538 539 .local pmc key 540 set_global [key], $S1, $P2 541 set_hll_global [key], $S1, $P2 542 set_root_global [key], $S1, $P2 543 544=end PIR_FRAGMENT 545 546Store $P2 as the symbol named $S1 by a multidimensional hash key, 547relative to the current namespace, HLL root namespace, or true root 548namespace. If the given namespace does not exist it is created. 549 550=begin PIR_FRAGMENT 551 552 set_global $P1, $S2, $P3 553 set_hll_global $P1, $S2, $P3 554 set_root_global $P1, $S2, $P3 555 556=end PIR_FRAGMENT 557 558Store $P3 as the symbol named $S2 by the array of names $P1, relative to 559the current namespace, HLL root namespace, or true root namespace. If 560the given namespace does not exist it is created. 561 562=back 563 564=head3 HLL Namespace Mapping 565 566In order to make this work, Parrot must somehow figure out what type of 567namespace PMC to create. 568 569=head4 Default Namespace 570 571The default namespace PMC will implement Parrot's current behavior. 572 573=head4 Compile-time Creation 574 575This Perl: 576 577 #!/usr/bin/perl 578 package Foo; 579 $x = 5; 580 581should map roughly to this PIR: 582 583=begin PIR_INVALID 584 585 .HLL "Perl5" 586 .loadlib "perl5_group" 587 .namespace [ "Foo" ] 588 .sub main :main 589 $P0 = new 'PerlInt' 590 $P0 = 5 591 set_global "$x", $P0 592 .end 593 594=end PIR_INVALID 595 596In this case, the C<main> sub would be tied to Perl 5 by the C<.HLL> 597directive, so a Perl 5 namespace would be created. 598 599=head4 Run-time Creation 600 601Consider the following Perl 5 program: 602 603 #!/usr/bin/perl 604 $a = 'x'; 605 ${"Foo::$a"} = 5; 606 607The C<Foo::> namespace is created at run-time (without any optimizations). In 608these cases, Parrot should create the namespace based on the HLL of the PIR 609subroutine that calls the store function. 610 611=begin PIR_INVALID 612 613 .HLL "Perl5" 614 .loadlib "perl5_group" 615 .sub main :main 616 # $a = 'x'; 617 $P0 = new 'PerlString' 618 $P0 = "x" 619 set_global "$a", $P0 620 # ${"Foo::$a"} = 5; 621 $P1 = new 'PerlString' 622 $P1 = "Foo::" 623 $P1 .= $P0 624 $S0 = $P1 625 $P2 = split "::", $S0 626 $S0 = pop $P2 627 $S0 = "$" . $S0 628 $P3 = new 'PerlInt' 629 $P3 = 5 630 set_global $P2, $S0, $P3 631 .end 632 633=end PIR_INVALID 634 635In this case, C<set_global> should see that it was called from "main", 636which is in a Perl 5 namespace, so it will create the "Foo" namespace as 637a Perl 5 namespace. 638 639=head2 Language Notes 640 641=head3 Perl 6 642 643=head4 Sigils 644 645Perl 6 may wish to be able to access the namespace as a hash with sigils. 646That is certainly possible, even with subroutines and methods. It's not 647important that a HLL use the typed namespace API, it is only important that it 648provides it for others to use. 649 650So Perl 6 may implement C<get_keyed> and C<set_keyed> VTABLE slots that 651allow the namespace PMC to be used as a hash. The C<find_sub> method would, 652in this case, append a "&" sigil to the front of the sub/method name and 653search in the internal hash. 654 655=head3 Python 656 657=head4 Importing from Python 658 659Since functions and variables overlap in Python's namespaces, when exporting 660to another HLL's namespace, the Python namespace PMC's C<export_to> method 661should use introspection to determine whether C<x> should be added using 662C<add_var> or C<add_sub>. C<$I0 = does $P0, "Sub"> may be enough to decide 663correctly. 664 665=head4 Subroutines and Namespaces 666 667Since Python's subroutines and namespaces are just variables (the namespace 668collides there), the Python PMC's C<find_var> method may return subroutines as 669variables. 670 671 672=head3 Examples 673 674=head4 Aliasing 675 676Perl: 677 678 #!/usr/bin/perl6 679 sub foo {...} 680 %Foo::{"&bar"} = &foo; 681 682PIR: 683 684=begin PIR 685 686 .sub main :main 687 $P0 = get_global "&foo" 688 $P1 = get_namespace ["Foo"] 689 690 # A smart perl6 compiler would emit this, 691 # because it knows that Foo is a perl6 namespace: 692 $P1["&bar"] = $P0 693 694 # But a naive perl6 compiler would emit this: 695 $P1.'add_sub'("bar", $P0) 696 697 .end 698 699 .sub foo 700 #... 701 .end 702 703=end PIR 704 705=head4 Cross-language Exportation 706 707Perl 5: 708 709 #!/usr/bin/perl 710 use tcl:Some::Module 'w*'; # XXX - ':' after HLL may change in Perl 6 711 write("this is a tcl command"); 712 713PIR (without error checking): 714 715=begin PIR 716 717 .sub main :main 718 .local pmc tcl 719 .local pmc ns 720 tcl = compreg "tcl" 721 ns = new 'Array' 722 ns = 2 723 ns[0] = "Some" 724 ns[1] = "Module" 725 null $P0 726 tcl.'load_library'(ns, $P0) 727 $P0 = tcl.'get_namespace'(ns) 728 $P1 = get_namespace 729 $P0.'export_to'($P1, 'w*') 730 "write"("this is a tcl command") 731 .end 732 733=end PIR 734 735=head2 References 736 737None. 738 739=cut 740 741__END__ 742Local Variables: 743 fill-column:78 744End: 745vim: expandtab shiftwidth=4: 746