1<<if: ZXIDBOOK>> 2<<else: >>Installing ZXID (from package or source) 3########################################### 4<<author: Sampo Kellom�ki (sampo@iki.fi)>> 5<<cvsid: $Id: zxid-install.pd,v 1.4 2010-01-08 02:10:09 sampo Exp $>> 6<<class: article!a4paper!!ZXID-COMP 01>> 7<<define: ZXDOC=Compiling and Installing ZXID>> 8 9<<abstract: 10 11ZXID.org Identity Management toolkit implements standalone SAML 2.0 and 12Liberty ID-WSF 2.0 stacks. This document explains how to compile 13ZXID from source code. 14 15>> 16 17<<maketoc: 1>> 18 191 Introduction 20============== 21 22ZXID is distributed in source code form, ready to be compiled 23with C development environment. 24 25Depending on your skill level, you may want to see if a binary package 26is available for your distribution - installing them is similar to 27installing the versions compiled from source, except that you step the 28early steps of compilation. 29 30Once you have compiled ZXID or found a package, you should 31proceed to install ZXID. This will depend heavily on your 32environment and some of the cases are explained in their 33own documents. 34 35* <<link:../mod_auth_saml/mod_auth_saml.html: mod_auth_saml>> Apache 36 module documentation: SSO without programming. 37* <<link:zxid-java.html: javazxid>>: Using ZXID from Java 38* <<link:zxid-perl.html: Net::SAML>>: Using ZXID from Perl 39* <<link:zxid-php.html: php_zxid>>: Using ZXID from PHP 40 41After installation, you should try to run a canned tutorial, 42in section <<see:>>, below, to learn the basics of the technology and to ascertain 43everything works. 44 45When you are ready to deploy your own services, you should check 46configuration instructions in <<link:zxid-conf.html: ZXID Configuration 47Reference>>. 48 491.1 Other documents 50------------------- 51 52<<doc-inc.pd>> 53 542 Installing 55============ 56 57<<fi: >> 58 59If you want to try ZXID out immediately, we recommend compiling the 60library and examples and installing one of the examples as a CGI 61script in an existing web server. See later chapters for more details. 62 63Download from zxid.org or get it with anon git: ~git clone git://zxid.org/zxid~ 64 65 tar xvzf zxid-0.76.tgz 66 cd zxid-0.76 67 # N.B. There is no configure script. The Makefile works for all 68 # supported platforms by provision of correct TARGET option. 69 # N.B2: We distribute some generated files. If they are missing, you need 70 # to regenerate them: make cleaner; make dep ENA_GEN=1 71 # Standard place is /var/zxid. You can change this with, e.g. 72 # make ZXID_PATH=/usr/local/var/zxid 73 make # default Linux. Do `make TARGET=sol8' for Solaris 74 make all # Also builds the optional language bindings, see below 75 make dir # Creates /var/zxid hierarchy (may need to be root) 76 77 make apachezxid # optional 78 make apachezxid_install # optional: install the mod_auth_saml Apache module 79 make samlmod # optional 80 make samlmod_install # optional: install Net::SAML perl module 81 make phpzxid # optional 82 make phpzxid_install # optional: install php_zxid.so PHP extension 83 make javazxid # optional 84 85 cp zxidhlo <webroot>/ 86 # configure your web server to recognize zxid a CGI, e.g. 87 mini_httpd -p 8443 -c 'zxid*' -S -E zxid.pem 88 89 # Edit your /etc/hosts to contain 90 127.0.0.1 localhost sp1.zxidcommon.org sp1.zxidsp.org 91 92 # Point your browser to (zxid_simple() API version) 93 https://sp1.zxidsp.org:8443/zxidhlo?o=E 94 https://sp1.zxidsp.org:8443/zxidhlo.pl?o=E # Perl version 95 https://sp1.zxidsp.org:8443/zxidhlo.php?o=E # PHP version 96 http://sp1.zxidsp.org:8080/zxidservlet/zxidHLO?o=E # Java version 97 98 # Find an IdP to test with and configure it... 99 # see zxididp and zxid-idp.pd for documentation, or use the 100 # free IdP on the net: https://zxidp.org/index-idp.html 101 102 1032.1 Prerequisites 104----------------- 105 106This software depends on the following packages: 107 1081. zlib from zlib.net. Generally whatever comes with your distro is sufficient. 1092. openssl-0.9.8l or openssl-1.0.0c (earlier versions contain a 110 man-in-the-middle security vulnearability) 111 or later. See www.openssl.org. Generally openssl libraries 112 distributed with most Linux distros are sufficient.<<footnote: It is 113 possible to compile without OpenSSL, e.g. for space constrained embedded 114 system, but this has serious security implications.>> 1153. libcurl from http://curl.haxx.se/. I used version 7.15.5, but probably 116 whatever ships with your distribution is fine. libcurl is needed 117 for SOAP bindings and for fetching metadata. It needs to be compiled 118 to support HTTPS.<<footnote: Compilation without libcurl is possible 119 with some loss of functionality.>> 1204. HTTPS capable web server. For most trivial testing CGI support is needed. We 121 recommend mini_httpd(8) available from 122 http://www.acme.com/software/mini_httpd/ 123 124 We also provide Apache HTTP receipe: <<link:apache.html: apache.html>> 125 1265. Perl, PHP, and Java interfaces depend on the respective development 127 tools but should not need any additional modules or tools. 128 1292.1.1 Building from Source: Install Required Packages 130~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 131 132While you may compile the dependency libraries from their respective sources, 133generally you can use the libraries that come with your operating system 134distribution. In this case you need to ensure that "development" versions 135of the packages are installed (the non-development versions do not include 136header files or .a files, which are needed for compiling zxid). 137 138Unfortunately the manner how development packages are installed is 139highly distribution dependent. Here are some commands that have 140been empirically found to work (tested 2011): 141 142 sudo apt-get install libssl-dev # Debian 143 sudo apt-get install libcurl4-openssl-dev 144 sudo apt-get install libapr1-dev 145 sudo apt-get install apache2-dev 146 sudo apt-get install php5-dev 147 sudo apt-get install openjdk-6-jdk 148 sudo apt-get install mini-httpd # But our patched version is better 149 150 sudo yum -y install openssl-devel # Redhat 151 sudo yum -y install libcurl-devel 152 153 1542.1.2 Additional Prerequisites for Complete Rebuild from Scratch 155~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 156 157Following additional packages are needed by developers who wish 158to build from scratch, including the code generation (the standard 159distribution includes the output of the code generation, so most 160people do not need these). 161 162A. gperf from gnu.org (only for build process when generating code) 163B. swig from swig.org (only for build process and only if you 164 want scripting interfaces) 165C. perl from cpan.org (only for build process and only if you 166 want to generate code from .sg) 167D. plaindoc from http://zxid.org/plaindoc/pd.html (only for 168 build process, for code generation from .sg, and for documentation) 169 You can also do git clone git://zxid.org/pd 170 171Although technically not needed to build zxid, you will need an IdP 172to test against. You can use the free IdP at https://zxidp.org/index-idp.html 173- it allows anyone to register a user and any SP to join the Circle 174of Trust. Or you can compile and configure the zxididp, see zxid-idp.pd. 175 176<<ignore: 177* http://symlabs.com/Products/SFIAM.html 178* Lasso: http://lasso.entrouvert.org/ 179* The IDP: http://authentic.labs.libre-entreprise.org/ 180>> 181 1822.1.3 Operating System and Platform Support 183~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 184 185As of January 2011, following platforms are "officially" supported: 186 187* Linux/Unix (default platform, also should work for other Unixes, e.g. FreeBSD, with gnu tools) 188 - Ubuntu / Debian 189 - Amazon Linux (CentOS / Redhat derivative) 190* Solaris 8 (and later) (~make TARGET=sol8~) 191* MacOS X (~make TARGET=macosx~) 192* Windows 2000 using mingw (~make TARGET=mingw~) 193 194Many other platforms have been occasionally tested and found to work, such as 195 196* Windows using Microsoft compilers (MSVC) (~make TARGET=win32cl~) 197* Windows using cygwin (~make TARGET=cygwin~) 198* Linux hosted cross compilation targets: ~xsol8~, ~xmingw~ 199 200As of April 2013 most testing has been on x86_32 and x86_64 201architectures. The code is believed to be 64bit clean. Most testing 202has been in little endian (ix86) architectures, with only occasional 203testing in big endian architectures like PowerPC and 204Sparc. Never-the-less, we are committed to support every modern 205architecture and OS out there. 206 207> N.B. We prefer to support every supported architecture via an 208> explicit ~make TARGET=yourarch~ section in the Makefile. We believe 209> debugging gnu autohell generated ~configure~ scripts is more pain 210> and distraction than it's worth. If you add new platform support, 211> please do so directly in the ~Makefile~. 212 2132.2 Canned Tutorial: Running ZXID as CGI under mini_httpd 214--------------------------------------------------------- 215 216While zxid will run easily under Apache httpd (see <<link:apache.html: 217recipe>>), for sake of simplicity we first illustrate running it with 218mini_httpd(8), a very simple SSL capable web server by Jef Poskanzer. 219 2202.2.1 Getting and installing mini_httpd 221~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 222 223You can download the source for mini_httpd from 224http://www.acme.com/software/mini_httpd/ 225 226You should already have installed OpenSSL, or quite probably OpenSSL 227shipped with your distribution. If it is not located at 228/usr/local/ssl, the you need to edit the mini_httpd ~Makefile~ to 229indicate where it is. At any rate you need to uncomment all lines that 230start by SSL_ in the ~Makefile~. Then say 231 232 make 233 234Now copy the mini_httpd binary somewhere in your path. 235 2362.2.2 Running mini_httpd 237~~~~~~~~~~~~~~~~~~~~~~~~ 238 239After building zxid, cd to zxid directory and run 240 241 mini_httpd -p 8443 -c 'zxid*' -S -E zxid.pem 242 243where 244 245 -p 8443 specifies the port to listen to 246 -c 'zxid*' specifies that URL paths with "zxid" are CGI scripts 247 -S specifies that https is to be used 248 -E zxid.pem specifies the SSL certificate to use 249 250See <<link:apache.html: Apache recipe>> for alternative that 251avoids mini_httpd, but is more complicated otherwise. 252 253> N.B. The zxid.pem certificate and private key combo is shipped with zxid 254> for demonstration purposes. Obviously everybody who downloads zxid 255> has that private key, so there is no real security what-so-ever. For 256> production use, you must generate, or acquire, your own private 257> key-certificate pair (and keep the private key secret). See Certificates 258> chapter for further info. 259 2602.2.3 Accessing ZXID 261~~~~~~~~~~~~~~~~~~~~ 262 263Edit your /etc/hosts file so that the definition of localhost also 264includes sp1.zxidcommon.org and sp1.zxidsp.org domain names, e.g: 265 266 127.0.0.1 localhost sp1.zxidcommon.org sp1.zxidsp.org 267 268Point your browser to 269 270> https://sp1.zxidsp.org:8443/zxidhlo 271 272or if you do not want the common domain cookie check 273 274> https://sp1.zxidsp.org:8443/zxidhlo?o=E 275 276*Dynamic linking problems* 277 278If accessing the URL (while running mini_httpd) you get 279no error message and no content - everything just 280mysteriously fails - you may be hitting a dynamic 281linking problem. If mini_httpd(8) fails to launch CGI 282script it will silently fail. This is unfortunate, but 283I guess that is what the "mini" in the name implies. 284 285To make matters even worse, mini_httpd(8), probably in the 286interest of security, will ignore ~LD_LIBRARY_PATH~ 287variable. Apparently it has its fixed notion of the 288library paths that is set at compile time. 289 290If you suspect this problem, try following: 291 2921. Create shell script called test.sh: 293 294 #!/bin/sh 295 echo Content-Type: text/plain 296 echo 297 echo Test $$ 298 echo lib_path is --$LD_LIBRARY_PATH-- 299 ldd zxidhlo 300 ./zxidhlo -h 2>&1 301 echo Exit value --$?-- 302 3032. Restart mini_httpd(8) like this 304 305 chmod a+x test.sh 306 mini_httpd -p 8443 -c test.sh -S -E zxid.pem -l mini.out 307 3083. Access https://sp1.zxidsp.org:8443/test.sh - you 309 may see something like 310 311 Test 1655 312 lib_path is --/usr/local/lib:/usr/lib-- 313 ./zxidhlo: error while loading shared libraries: libcurl.so.3: cannot 314 open shared object file: No such file or directory 315 316 Now you at least see why it's failing (in this case the 317 directory where ~libcurl~ was installed is not in mini_httpd's 318 notion of LD_LIBRARY_PATH). If the ~zxidhlo~ binary runs 319 fine from comman line, try `ldd zxidhlo' to see where it 320 is finding its libraries. 321 322Easiest dirty fix is to copy the missing libraries to one of the 323hardwired directories of mini_httpd(8) (e.g. /usr/lib). More 324sophisticated fixes include using ldconfig(8), recompiling your 325mini_httpd(8), or statically linking the offending library into zxidhlo 326binary. 327 3282.2.4 Setting up an IdP 329~~~~~~~~~~~~~~~~~~~~~~~ 330 331ZXID now ships with an IdP: ~zxididp~. You may set this up, see 332documentation in zxid-idp.pd, or you may want to use a free 333IdP on the net, such as https://zxidp.org/index-idp.html 334 335For you to test zxidhlo, you will need to acquire an IdP from 336somewhere - any vendor whose product is SAML 2.0 certified 337will do. Possible sources are 338 339* https://zxidp.org/index-idp.html 340* http://symlabs.com/Products/SFIAM.html who have a free download of commercial product 341* Lasso: http://lasso.entrouvert.org/ 342* Their IDP: http://authentic.labs.libre-entreprise.org/ 343 344If you do not want to install an IdP yourself (even for testing), find 345someone who already runs one (e.g. https://zxidp.org/index-idp.html) 346and ask if they would be willing to load the metadata of your zxid 347SP. If you do this, you will need to get externally visible domain 348names. This canned tutorial uses /etc/hosts (see previous step) which 349is only visible on your own machine. 350 351Once you get your IdP up and running, you need to make sure it accepts 352the zxid SP in its Circle of Trust (CoT). This is done by placing 353the metadata of the SP in right place in the IdP product configuration. 354If your IdP supports automatic CoT management, just turn it on 355and chances are you are done.<<footnote: On production IdP you should 356understand the trust implications (i.e. no trust) of flipping automatic 357CoT management on.>> 358 359If not, you can obtain the zxid SP metadata (which is slightly different 360for each install so you can't just copy it from existing install) from 361 362> https://sp1.zxidsp.org:8443/zxidhlo?o=B 363 364This URL is the +well known location+ method metadata URL. It is also 365the SP +Entity ID+ or Provider ID, should the IdP product ask for 366this in its configuration. If the IdP product needs you to 367supply the metadata manually as an xml file, just point your 368web browser to the above URL and save to file, or use curl(1) or wget(1). 369 370zxid SP, by default, has automatic fetching of IdP metadata enabled so 371there is no manual configuration step needed, provided that the IdP 372supports the well known location method. All SAML 2.0 certified IdP 373implementations must support it (but you may still need to enable it 374in configuration). See [SAML2meta] section 4.1 "Publication and 375Resolution via Well-Known Location", p.29, for normative description of this 376method. 377 378However, you will need the Entity ID (Provider ID) of the IdP. This is 379the URL that the IdP uses for well known location method of metadata 380sharing. You may need to dig the IdP documentation or GUI for a while 381to find it. If you already have the IdP metadata as an xml file, open it 382and look for EntityDescriptor/entityID. If you already have the 383file, you can also import it manually by running the following command 384 385 ./zxcot -a </path/to/idp-meta.xml 386 387But the preferred method still is: just let the automatic method 388do its job. See also zxcot(8) documentation, or run ~zxcot -h~ 389 3902.2.5 Your first SSO 391~~~~~~~~~~~~~~~~~~~~ 392 3931. Start at 394 395 > https://sp1.zxidsp.org:8443/zxidhlo 396 397 or 398 399 > https://sp1.zxidsp.org:8443/zxidhlo?o=E 400 401 If you had common domain cookie already in place, and you 402 are already logged in the IdP, the SSO may happen 403 automatically (go to step 3). The automatic experience 404 will be typical when you use SSO regularly for more 405 than one web site (i.e. several SPs). 406 407 However, if you get a screen titled "ZXID SP SSO", 408 you need to paste the IdP's Entity ID to the supplied field 409 and click "Login". If zxid SP already obtained the metadata for the 410 IdP, you may also see a button specific for your IdP (and in this 411 case there is no need to know the Entity ID anymore or paste anything). 412 4132. Next step depends on the IdP product you are using. Usually 414 a login screen will appear asking for user name and password. 415 Supply these and login. You will need an account at the IdP. 416 4173. For more slick IdPs, that's all you need to do and you will 418 land right back at the zxid SP page titled "ZXID SP Management". 419 420 > Congratulations, you have made your first SSO! 421 422 However, some IdPs will pester you with additional questions 423 and you will have to jump through their hoops. A typical 424 question is whether you want to accept a federation. You do. 425 426 Sometimes the federation question does not appear automatically 427 and you need to figure out a way to create a federation 428 in their user interface and how to get them to send you 429 back to the SP. Sometimes the word used is "account linking" 430 instead of federation.<<footnote: Vendor products are constantly 431 improving in this area. From protocol perspective 432 all the additional gyrations are unnecessary. Be sure 433 to provide feedback to the vendor so that simpler, easier 434 to use, products will emerge in future.>> 435 4365 Compilation for Experts 437========================= 438 439 make cleaner 440 make dep ENA_GEN=1 441 make 442 make all ENA_GEN=1 443 4445.1 Build Process 445----------------- 446 447The build process of ZXID relies heavily on code generation techniques 448that are not for the faint of heart. Some of these techniques, like 449xsd2sg.pl were innovated for this project, while others like SWIG and 450gperf(1) are existing software. Here and there some additional 451perl(1) <<footnote: There used to be a sed(1) dependency as well, but 452nowdays sed-zxid.pl perl script handles all seddish stuff.>> scripts 453are run to fix a thing or two. 454 455<<dot: zxid-build: ZXID Build Process 456 457margin=0 458 459sg [label=".sg"]; 460i [label=".i"]; 461phpi [label="phpzxid.i"]; 462hc [label=".h and .c"]; 463pm [label=".pm and glue"]; 464php [label="php glue"]; 465gperf [label=".gperf"]; 466netsaml [label="Net::SAML"]; 467 468sg -> hc [label="xsd2sg.pl"]; 469sg -> gperf [label="xsd2sg.pl"]; 470gperf -> hc [label="gperf"]; 471hc -> hc [label="gen-consts-from-gperf-output.pl"]; 472hc -> libzxid [label="gcc"]; 473hc -> pm [label="swig"]; 474i -> pm [label="swig"]; 475hc -> php [label="swig"]; 476phpi -> php [label="swig"]; 477 478libzxid -> zxid [label="ld"]; 479 480libzxid -> netsaml [label="ld"]; 481pm -> netsaml [label="perl, gcc, ld"]; 482 483libzxid -> php_zxid [label="ld"]; 484php -> php_zxid [label="gcc, ld"]; 485 486>> 487 488Carefully study the ~Makefile~, and this should all start to make 489sense (no pun intended). Please note that there is *no* ~configure~ 490script and GNU Auto-tools ("GNU Autohell") are not used. This is because 491they have been evaluated to be unsuitable for this project (or to 492author's tastes). If the ~Makefile~ does not do what you want, you 493should first study if you can change necessary variables using 494localconf.mk, or if that does not work, then just edit the ~Makefile~ 495itself. 496 497To make your life easier, the build process is +not recursive+ 498and only uses single Makefile, so if you edit it, you can just say 499~make -f MyMakefile~ 500 501You can temporarily customize the ~Makefile~ variables 502by providing a new value on command line, e.g. 503 504 make CC=cc # override the default of CC=gcc 505 506Another philosophical tidbit is that, although some source code is in 507subdirectories, all the build logic is kept in +one+ top level ~Makefile~, 508eliminating all the complexity of hierarchical make. The ~Makefile~ supports 509building outside source tree using +vpath+ feature of gnu make(1). 510 5115.2 Special or embedded compile (reduced functionality) 512------------------------------------------------------- 513 514~libzxid~ contains thousands of functions and any given application is 515unlikely to use them all. Thus the easiest, safest, no loss of 516functionality, way to reduce the footprint is to simply enable 517compiler and linker flags that support dead function 518elimination.<<footnote: Unfortunately the gnu ld does not support dead 519function elimination. You should file this as a bug to them. If they 520tell you to put every one of the 7000-some functions in a separate .c file, 521consider the scalability implications of this. Read the comments in 522pulverize.pl for a full scoop and an approach.>> 523 524> On gcc you should investigate compilation with 525> "-ffunction-sections -fdata-sections" and linking 526> with "-Wl,--gc-sections". 527 528If you need to squeeze zxid into as minimal space as possible, 529some functionality trade-offs are supported. I stress that you 530should only attempt these trade-offs once you are familiar with 531zxid and know what you are doing. The canned install instructions 532and tutorial walk thrus stop working if you omit 533significant functionality. 534 5355.2.1 Compilation without OpenSSL 536~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 537 538Comment out the -DUSE_OPENSSL flag from CFLAGS in ~Makefile~ and 539recompile. 540 541This will cripple zxid from security perspective because it 542will no longer be able to verify or generate digital signatures. 543Unless your environment does not need trust and security, 544or you understand thoroughly how to provide trust and security 545by other means, it is a very bad idea to compile without OpenSSL. 546 547> N.B. Compiling, or not, zxid with OpenSSL does not affect whether your 548> web server will use SSL or TLS. Unless you know what you are doing, 549> you should be using SSL at web server layer. Given that SSL is used at 550> web server layer, the memory footprint savings you would gain from 551> compiling zxid without OpenSSL may be negligible if you use dynamic 552> linking. 553 5545.2.2 Compilation without libcurl 555~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 556 557Comment out the -DUSE_CURL flag from CFLAGS in ~Makefile~ and 558recompile. 559 560Disabling ~libcurl~ does not have adverse security implications: you 561only loose some functionality and depending on your situation you may 562well be able to live without it. 563 5641. Without ~libcurl~, zxid can not act as a SOAP client. This has a few 565 consequences 566 567 a. Artifact profile for SSO is not supported because it needs SOAP 568 to resolve the artifact. In most cases a perfectly 569 viable alternative is to use POST profile for SSO. 570 571 b. SOAP profiles for Single Logout and NameID management (aka 572 defederation) are not supported. You can use the redirect 573 profiles and get mostly the same functionality. 574 575 c. ID-WSF Web Services Client (WSC) and Discovery Client (DIC), 576 as well as XACML client (PEP) functionality is not available. 577 5782. Automatic CoT metadata fetching using well known 579 location method is not supported without ~libcurl~. 580 You can fetch the metadata manually, e.g. using web browser, 581 and place it in /var/zxid/cot directory. 582 583 If you want to manually control your Circle of Trust 584 relationships, you probably want to do this anyway so 585 loss of automatic functionality may be a non-issue.<<footnote: 586 If you compile with libcurl, but still want to disable 587 automatic metadata fetching, investigate the MD_FETCH 588 and related configuration options.>> 589 5903. Web Services Client (WSC) functionality is not supported 591 without ~libcurl~. Effectively this is just another case of 592 "SOAP needed". If you have your own SOAP implementation, 593 you may, at lesser automation, achieve much of the 594 same functionality by calling the encoder and decoder 595 functions manually. 596 5975.2.3 Compiling without zlib (not supported) 598~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 599 600zlib is used mainly in redirect profiles. Since zlib is widely 601available and its foot print is small, we have made no supported 602provision to compile without it. If you hack something together, let 603us know. 604 6055.3 Static Linking and Building Against Dietlibc 606------------------------------------------------ 607 608Static linking has its advantages: (i) you eliminate all library 609dependencies from install process - as well as from error reporting 610and debugging processes; (ii) you avoid the dynamic linking overhead - 611this is significant in CGI use where same executable gets launched 612over and over again. The buffer cache ensures the blocks of dll's 613(.so's) are already in memory, but that does not prevent the system 614calls to do the redundant dynamic linking step (just look at output of 615strace for first about 100 lines on a dynamically linked executable to 616get an idea). With static link, the buffer cache has the blocks and 617there is no dynamic link syscall overhead. 618 619Static linking also has its cons: (i) if any library has a 620bug, such as a critical security vulnearability, then you need to 621rebuild (and reinstall in all places where it was distributed) the 622statically linked binary - it is not enough to upgrade the offending 623library package. But this sword cuts the other way too: you will 624always know what combination of libraries is being used (because you 625built it and you kept records), thus any bugs that are due to 626interactions between versions of libraries are none of your concern. 627 628(ii) Static linking prevents some techniques adopted by modern 629libraries, such as gnu libc. For example, name resolution can not 630dynamically load resolver modules so you probably have to forget about 631YP (Sun Yellow Pages) or NIS (Sun Network Information Service - a 632newer version of YP) support. Your /etc/nsswitch.conf will no longer 633be consulted. Instead you will have a fixed static name resolution 634policy: first check /etc/hosts, then DNS. Heck, some of us wanted that 635to start with - sounds like an improvement. 636 637In bug reduction, minimality is a good thing. glibc has gotten about 638as far as possible from minimality. Luckily, competing libc projects 639have appeared. Felix Leitner's dietlibc (http://www.fefe.de/dietlibc/) 640is perhaps the most promising candidate. 641 642Dietlibc is useful for compiling statically linked binaries 643without glibc dependency. The dependency is to be avoided 644for two reasons: (i) statically linked glibc still needs dynamic 645linking for name resolution, (ii) dietlibc is a different 646code base than glibc. glibc is a fat target while dietlibc is 647used by security conscious minority - thus it is not likely 648to be an attractive target. 649 6505.3.1 dietlibc-0.33 Build 651~~~~~~~~~~~~~~~~~~~~~~~~~ 652 653wget http://www.fefe.de/dietlibc/dietlibc-0.33.tar.bz2 654tar xvf dietlibc-0.33.tar.bz2 655 656N.B. Enabling WANT_DYNAMIC may trigger "no .eh_frame_hdr table will be 657created" linking error. 658 659<<diffout: 660diff -u dietfeatures.h~ dietfeatures.h 661--- dietfeatures.h~ 2012-11-02 11:17:28.000000000 +0000 662+++ dietfeatures.h 2013-04-13 17:47:37.000000000 +0000 663@@ -4,8 +4,8 @@ 664 /* feel free to comment some of these out to reduce code size */ 665 666 #define WANT_FLOATING_POINT_IN_PRINTF 667-#define WANT_FLOATING_POINT_IN_SCANF 668-#define WANT_CHARACTER_CLASSES_IN_SCANF 669+/*#define WANT_FLOATING_POINT_IN_SCANF*/ 670+/*#define WANT_CHARACTER_CLASSES_IN_SCANF*/ 671 #define WANT_NULL_PRINTF 672 /* #define WANT_ERROR_PRINTF */ 673 #define WANT_LONGLONG_PRINTF 674@@ -29,7 +29,7 @@ 675 #define WANT_TLS 676 677 /* make the startcode, etc. dynamic aware ({con,de}structors) */ 678 /* #define WANT_DYNAMIC */ 679 680 /* GDB support in the dynamic linker */ 681 #define WANT_LD_SO_GDB_SUPPORT 682@@ -46,23 +46,23 @@ 683 /* 20040118: enabling this breaks User Mode Linux! It's their fault. */ 684 #define WANT_SYSENTER 685 686-#define WANT_LINKER_WARNINGS 687+/*#define WANT_LINKER_WARNINGS*/ 688 689 /* you need to define this if you want to run your programs with large 690 * file support on kernel 2.2 or 2.0 */ 691-#define WANT_LARGEFILE_BACKCOMPAT 692+/*#define WANT_LARGEFILE_BACKCOMPAT*/ 693 694 /* do you want localtime(3) to read /etc/localtime? 695 * Needed for daylight saving time etc. */ 696-#define WANT_TZFILE_PARSER 697+/*#define WANT_TZFILE_PARSER*/ 698 699 /* do you want the DNS routines to parse and use "domain" and "search" 700 * lines from /etc/resolv.conf? Normally not used on boot floppies and 701 * embedded environments. */ 702-#define WANT_FULL_RESOLV_CONF 703+/*#define WANT_FULL_RESOLV_CONF*/ 704 705 /* do you want IPv6 transport support in the DNS resolver? */ 706-#define WANT_IPV6_DNS 707+/*#define WANT_IPV6_DNS*/ 708 709 /* do you want gethostbyname and friends to consult /etc/hosts? */ 710 #define WANT_ETC_HOSTS 711@@ -72,10 +72,10 @@ 712 #define WANT_INET_ADDR_DNS 713 714 /* do you want math functions high precision rather than fast/small? */ 715-#define WANT_HIGH_PRECISION_MATH 716+/*#define WANT_HIGH_PRECISION_MATH*/ 717 718 /* do you want support for matherr? */ 719-#define WANT_MATHERR 720+/*#define WANT_MATHERR*/ 721 722 /* do you want crypt(3) to use MD5 if the salt starts with "$1$"? */ 723 #define WANT_CRYPT_MD5 724@@ -87,11 +87,11 @@ 725 /* This enables zeroconf DNS aka Rendezvous aka Bonjour. */ 726 /* This code will try zeroconf DNS if you ask for host.local or if you 727 * ask for an unqualified hostname */ 728-#define WANT_PLUGPLAY_DNS 729+/*#define WANT_PLUGPLAY_DNS*/ 730 731 /* This enables LLMNR, the MS variant of zeroconf DNS. This only works 732 * if you also enabled WANT_PLUGPLAY_DNS */ 733-#define WANT_LLMNR 734+/*#define WANT_LLMNR*/ 735 736 /* Uncomment this if you want DNS lookups to fail if /etc/hosts contains 737 * an entry but it's for a different record type */ 738@@ -101,7 +101,7 @@ 739 * check for valgrind, and if detected, turn off optimized SIMD string 740 * routines that cause false positives in valgrind. This enlarges and 741 * slightly slows down your code! */ 742-#define WANT_VALGRIND_SUPPORT 743+/*#define WANT_VALGRIND_SUPPORT*/ 744 745 /* do you want that malloc(0) return a pointer to a "zero-length" object 746 * that is realloc-able; means realloc(..,size) gives a NEW object (like a 747@@ -129,7 +129,7 @@ 748 * and program_invocation_name to be there. This functionality is not 749 * portable and adds useless bloat to libc. Help stomp out code 750 * depending on this! util-linux, I'm looking at you here! */ 751-#define WANT_GNU_STARTUP_BLOAT 752+/*#define WANT_GNU_STARTUP_BLOAT*/ 753 754 /* Include support for ProPolice/SSP, calls guard_setup */ 755 /* ProPolice is part of gcc 4.1 and up, there were patches for earlier 756 757diff -u Makefile~ Makefile 758--- Makefile~ 2006-06-18 14:32:28.000000000 -0400 759+++ Makefile 2006-12-16 22:34:24.000000000 -0500 760@@ -141,6 +141,7 @@ 761 COMMENT = : 762 endif 763 CFLAGS += -Wall -W -Wchar-subscripts -Wmissing-prototypes -Wmissing-declarations -Wno-switch -Wno-unused -Wredundant-decls 764+CFLAGS += -march=i586 765 766 PWD=$(shell pwd) 767 768diff -u libcruft/res_query.c~ libcruft/res_query.c 769--- libcruft/res_query.c~ 2006-12-08 16:24:56.000000000 +0000 770+++ libcruft/res_query.c 2008-06-15 18:39:21.000000000 +0000 771@@ -106,8 +106,10 @@ 772 #ifdef WANT_PLUGPLAY_DNS 773 if (duh[1].fd!=-1) { 774 sendto(pnpfd,packet,size,0,(struct sockaddr*)(&pnpsa4),sizeof(pnpsa4)); 775+#ifdef WANT_IPV6_DNS 776 if (!v4pnp) 777 sendto(pnpfd,packet,size,0,(struct sockaddr*)(&pnpsa6),sizeof(pnpsa6)); 778+#endif 779 } 780 /* if it doesn't work, we don't care */ 781 #endif 782 783diff -u /opt/diet/include/sys/param.h~ /opt/diet/include/sys/param.h 784--- /opt/diet/include/sys/param.h~ 2008-06-15 18:55:43.000000000 +0000 785+++ /opt/diet/include/sys/param.h 2008-06-15 20:12:49.000000000 +0000 786@@ -26,4 +26,6 @@ 787 #define roundup(x, y) ((((x)+((y)-1))/(y))*(y)) 788 #define powerof2(x) ((((x)-1)&(x))==0) 789 790+#define MAXSYMLINKS 20 791+ 792 #endif 793 794diff -u dietlibc-0.32/libugly/gethostent.c~ dietlibc-0.32/libugly/gethostent.c 795--- libugly/gethostent.c~ 2003-06-23 10:48:13.000000000 +0000 796+++ libugly/gethostent.c 2010-08-25 00:12:09.000000000 +0000 797@@ -26,6 +26,8 @@ 798 if (!hostmap) { 799 int hostfd=open(_PATH_HOSTS,O_RDONLY); 800 if (hostfd<0) return 0; 801+ /* *** all this global variable manipulation seems rather at odds with reentrant 802+ * *** nature of gethostent_r(), surely there is race here. 24.8.2010, sampo@iki.fi */ 803 hostlen=lseek(hostfd,0,SEEK_END); 804 hostmap=mmap(0,hostlen,PROT_READ|PROT_WRITE,MAP_PRIVATE,hostfd,0); 805 if ((long)hostmap==(-1)) { close(hostfd); hostmap=0; goto error; } 806@@ -81,8 +83,7 @@ 807 dest+=len; 808 *dest=0; ++dest; 809 } 810- if (*cur=='\n') { ++cur; ++aliasidx; break; } 811- if (cur>=last || !isblank(*cur)) break; 812+ if (cur>=last || *cur=='\n') { ++cur; ++aliasidx; break; } 813 cur++; 814 } 815 pe->h_aliases[aliasidx]=0; 816 817diff -u __v_printf.c.orig __v_printf.c 818--- __v_printf.c.orig 2013-04-16 15:13:12.000000000 +0000 819+++ __v_printf.c 2013-04-16 16:53:49.000000000 +0000 820@@ -7,15 +7,35 @@ 821 #include "dietstdio.h" 822 #include "dietwarning.h" 823 824-#define MAX_WIDTH 10*1024 825- 826 static inline unsigned long skip_to(const char *format) { 827 unsigned long nr; 828 for (nr=0; format[nr] && (format[nr]!='%'); ++nr); 829 return nr; 830 } 831 832-#define A_WRITE(fn,buf,sz) ((fn)->put((void*)(buf),(sz),(fn)->data)) 833+/*#define A_WRITE(fn,buf,sz) ((fn)->put((void*)(buf),(sz),(fn)->data))*/ 834+/* 20130416, sampo@zxidp.org 835+ * Problem: Old A_WRITE() ignores the return value from write(2), thus leaving 836+ * open the possibility that the data was not actually written. 837+ * Proposed fix: deal with error return from write style function 838+ * and keep on trying in blocking fashion until everything is written. */ 839+int __a_write_all(struct arg_printf* fn, const char* buf, unsigned int len) { 840+ int ret, todo=len; 841+ while (todo) { 842+ ret = fn->put((void*)buf, todo, fn->data); 843+ if (ret == -1) { 844+ switch (errno) { 845+ case EAGAIN: /*case EWOULDBLOCK:*/ ret=0; break; 846+ default: 847+ return ret; 848+ } 849+ } 850+ todo -= ret; 851+ buf += ret; 852+ } 853+ return len; 854+} 855+#define A_WRITE(fn,buf,sz) __a_write_all((fn),(buf),(sz)) 856 #define B_WRITE(fn,buf,sz) { if ((unsigned long)(sz) > (((unsigned long)(int)(-1))>>1) || len+(int)(sz)<len) return -1; A_WRITE(fn,buf,sz); } while (0) 857 858 static const char pad_line[2][16]= { " ", "0000000000000000", }; 859@@ -123,7 +143,6 @@ 860 case '9': 861 if(flag_dot) return -1; 862 width=strtoul(format-1,(char**)&s,10); 863- if (width>MAX_WIDTH) return -1; 864 if (ch=='0' && !flag_left) padwith='0'; 865 format=s; 866 goto inn_printf; 867@@ -138,7 +157,6 @@ 868 flag_left=1; 869 tmp=-tmp; 870 } 871- if ((width=(unsigned long)tmp)>MAX_WIDTH) return -1; 872 goto inn_printf; 873 } 874 case '.': 875@@ -152,7 +170,6 @@ 876 preci=tmp<0?0:tmp; 877 format=s; 878 } 879- if (preci>MAX_WIDTH) return -1; 880 goto inn_printf; 881 882 /* print a char or % */ 883 884>> 885 886On gcc-3.4.6 you need 887 888<<diffout: 889diff -u Makefile~ Makefile 890--- Makefile~ 2012-11-02 11:17:28.000000000 +0000 891+++ Makefile 2013-04-13 19:38:26.000000000 +0000 892@@ -141,6 +141,7 @@ 893 COMMENT = : 894 endif 895 CFLAGS += -W -Wall -Wextra -Wchar-subscripts -Wmissing-prototypes -Wmissing-declarations -Wno-switch -Wno-unused -Wredundant-decls 896+CFLAGS += -march=i586 897 898 PWD=$(shell pwd) 899 900@@ -534,7 +535,7 @@ 901 # WANT_SSP 902 # This facepalm brought to you by: Ubuntu! 903 $(OBJDIR)/stackgap.o: lib/stackgap.c dietfeatures.h 904- $(CROSS)$(CC) $(INC) $(CFLAGS) -c lib/stackgap.c -o $@ -D__dietlibc__ -fno-stack-protector 905+ $(CROSS)$(CC) $(INC) $(CFLAGS) -c lib/stackgap.c -o $@ -D__dietlibc__ 906 $(COMMENT) -$(CROSS)strip -x -R .comment -R .note $@ 907 908 # WANT_MALLOC_ZERO 909>> 910 911 patch --dry-run -p0 <the-patch-above-as-needed 912 913After applying patches 914 915 make prefix=/usr/local DEBUG=1 916 make prefix=/usr/local DEBUG=1 dyn 917 sudo make prefix=/usr/local install # works 918 919<<ignore: email: 920Subject dietlibc-0.33: fprintf() does not print long data 921To: dietlibc@fefe.de 922 923I have been trying to use fprintf() to print 924rather long data to a socket, e.g. 925 926 fprintf(stdout, "Content-Type: %s" CRLF "Content-Length: %d" CRLF2 "%.*s", 927 cont_type, ss->len, ss->len, ss->s); 928 929This fails to print the data that should appear in "%.*s" format 930specifier. The fprintf() backends to __v_printf() and write(2). 931 932Examining lib/__v_printf.c I have identified two dubious points that 933are, each alone, liable to break my code: 934 9351. What is the purpose of MAX_WIDTH specification? I propose to 936 not impose any MAX_WIDTH at all. Due to the way __v_printf() is 937 implemented, there does not seem to be any buffer that could be 938 overrun. I did not find in any man page documentation a mention 939 of this MAX_WIDTH, and it does not appear in any header either. 940 Thus I suspect it to be unnecessary implementation dependent 941 restriction. 942 9432. Why does not _A_WRITE() check its return value and why does it 944 not retry write(2) until all data is written? My usual C wisdom 945 is that fprintf(3) blocks until the data is written (at least 946 into kernel). It would appear that __v_printf() as implemented 947 in 0.33 is broken in this regard. Please see my patch for a 948 fix. Basically I add a loop that retries write as many times 949 as it takes. Is there already such write all function somewhere 950 in the library? write_all() would come handy in many other 951 programming tasks. 952 953Cheers, 954--Sampo 955 956 957diff -u __v_printf.c.orig __v_printf.c 958--- __v_printf.c.orig 2013-04-16 15:13:12.000000000 +0000 959+++ __v_printf.c 2013-04-16 16:53:49.000000000 +0000 960@@ -7,15 +7,35 @@ 961 #include "dietstdio.h" 962 #include "dietwarning.h" 963 964-#define MAX_WIDTH 10*1024 965- 966 static inline unsigned long skip_to(const char *format) { 967 unsigned long nr; 968 for (nr=0; format[nr] && (format[nr]!='%'); ++nr); 969 return nr; 970 } 971 972-#define A_WRITE(fn,buf,sz) ((fn)->put((void*)(buf),(sz),(fn)->data)) 973+/*#define A_WRITE(fn,buf,sz) ((fn)->put((void*)(buf),(sz),(fn)->data))*/ 974+/* 20130416, sampo@zxidp.org 975+ * Problem: Old A_WRITE() ignores the return value from write(2), thus leaving 976+ * open the possibility that the data was not actually written. 977+ * Proposed fix: deal with error return from write style function 978+ * and keep on trying in blocking fashion until everything is written. */ 979+int __a_write_all(struct arg_printf* fn, const char* buf, unsigned int len) { 980+ int ret, todo=len; 981+ while (todo) { 982+ ret = fn->put((void*)buf, todo, fn->data); 983+ if (ret == -1) { 984+ switch (errno) { 985+ case EAGAIN: /*case EWOULDBLOCK:*/ ret=0; break; 986+ default: 987+ return ret; 988+ } 989+ } 990+ todo -= ret; 991+ buf += ret; 992+ } 993+ return len; 994+} 995+#define A_WRITE(fn,buf,sz) __a_write_all((fn),(buf),(sz)) 996 #define B_WRITE(fn,buf,sz) { if ((unsigned long)(sz) > (((unsigned long)(int)(-1))>>1) || len+(int)(sz)<len) return -1; A_WRITE(fn,buf,sz); } while (0) 997 998 static const char pad_line[2][16]= { " ", "0000000000000000", }; 999@@ -123,7 +143,6 @@ 1000 case '9': 1001 if(flag_dot) return -1; 1002 width=strtoul(format-1,(char**)&s,10); 1003- if (width>MAX_WIDTH) return -1; 1004 if (ch=='0' && !flag_left) padwith='0'; 1005 format=s; 1006 goto inn_printf; 1007@@ -138,7 +157,6 @@ 1008 flag_left=1; 1009 tmp=-tmp; 1010 } 1011- if ((width=(unsigned long)tmp)>MAX_WIDTH) return -1; 1012 goto inn_printf; 1013 } 1014 case '.': 1015@@ -152,7 +170,6 @@ 1016 preci=tmp<0?0:tmp; 1017 format=s; 1018 } 1019- if (preci>MAX_WIDTH) return -1; 1020 goto inn_printf; 1021 1022 /* print a char or % */ 1023 1024 1025>> 1026 10275.3.2 zlib-1.2.7 linked against dietlibc 1028~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1029 1030<<console: 1031wget http://zlib.net/zlib-1.2.7.tar.bz2 1032tar xvf zlib-1.2.7.tar.bz2 1033cd zlib-1.2.7 1034# diet compile on x86_64 (amd64 as in GQ) 1035cp contrib/amd64/amd64-match.S ./match.S 1036CC='diet gcc' LD='diet ld' CFLAGS="-g -O3 -DASMV -D_BSD_SOURCE -DGNU_SOURCE" ./configure --static --prefix=/usr/local 1037make LOC=-DASMV OBJA=match.o 1038make test 1039sudo make install 1040>> 1041 10425.3.3 OpenSSL-1.0.1e linked against dietlibc 1043~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1044 1045 tar xvf openssl-1.0.1e.tar.gz 1046 1047Add to Configure 1048 1049 "diet-debug-linux-x86_64","diet gcc:-DREF_CHECK -DCONF_DEBUG -DCRYPTO_MDEBUG -m64 -DL_ENDIAN -DTERMIO -O2 -g -Wall::-D_REENTRANT::-ldl:SIXTY_FOUR_BIT_LONG RC4_CHUNK DES_INT DES_UNROLL:${x86_64_asm}:elf:dlfcn:linux-shared:-fPIC:-m64:.so.\$(SHLIB_MAJOR).\$(SHLIB_MINOR):::64", 1050 1051 "diet-debug-linux-pentium", "diet gcc:-DREF_CHECK -DCONF_DEBUG -DL_ENDIAN -DTERMIO -O2 -g -march=pentium -Wall::-D_REENTRANT::-ldl:BN_LLONG ${x86_gcc_des} ${x86_gcc_opts}:${x86_elf_asm}:dlfcn:linux-shared:-fPIC::.so.\$(SHLIB_MAJOR).\$(SHLIB_MINOR)", 1052 1053N.B. -DREF_PRINT causes compilation error. 1054 1055<<console: 1056export PATH=/usr/bin:/bin:/usr/local/bin # /usr/local/bin is needed for diet(1) to be found 1057./Configure --prefix=/usr/local enable-rc5 enable-mdc2 no-dso no-hw no-engine zlib diet-debug-linux-pentium 1058./Configure --prefix=/usr/local enable-rc5 enable-mdc2 no-dso no-hw no-engine zlib diet-debug-linux-x86_64 1059make depend # fails 1060make # ? 1061 1062time make test # segfaults?!? 1063sudo make install 1064>> 1065 10665.3.4 curl-7.30.0 linked against dietlibc and openssl-1.0.1e 1067~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1068 1069<<console: 1070wget http://curl.haxx.se/download/curl-7.30.0.tar.bz2 1071tar xvf curl-7.30.0.tar.bz2 1072cd curl-7.30.0 1073 1074CC='diet gcc' LD='diet ld' CPPFLAGS='-I/usr/local/include' LDFLAGS='-L/usr/local/lib' LIBS='-lssl -lcrypto -lz' ./configure --prefix=/usr/local --with-ssl=/usr/local --without-gnutls --enable-thread --enable-nonblocking --disable-shared 1075 1076make 1077make test 1078sudo make install 1079>> 1080 10815.3.5 zxididp against dietlibc based libraries 1082~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1083 1084 make clean 1085 CC='diet cc' LD='diet cc' make zxididp-static 1086 1087 diet gcc -o zxididp zxididp.o -static -L. -lzxid -pthread -lpthread -L/usr/local/lib -L/usr/local/ssl/lib-x86_64 -lcurl -lssl -lcrypto -lz 1088 1089 10905.4 Choosing Which Standards to Include (default: all) 1091------------------------------------------------------ 1092 1093WARNING: Only regularly tested configuration is to compile all 1094standards in. 1095 1096On space constrained systems you may shed additional weight by only 1097compiling in the IdM standards you actually use. Of course, if you do 1098not use them, the dead function elimination should take care of them, 1099but sometimes you can gain additional savings in space and especially 1100compile time. 1101 1102Another reason could be, in the land of the free, if some modules 1103are covered by a software patent, you may want to compile a binary 1104without the contested functionality.<<footnote: Please do not 1105ask me to add additional baggage to avoid patents. Software 1106patents are a plague and your efforts are best spent in getting 1107them overturned or changing laws so they go away.>> 1108 1109You can tweak the flags, shown in accompanying table, in the ~Makefile~ 1110or by supplying new values in localconf.mk or on commend line. For example 1111 1112 make TARGET=sol8 ENA_SAML2=0 1113 1114would disable SAML 2.0 (and trigger build for Sparc Solaris 8). 1115 1116<<table: Conditional inclusion of standards 1117Makefile flag Standard Comments 1118================ ============= ====================================== 1119ENA_SSO=1 All SSO Must be enabled for any of SSO to work 1120ENA_SAML2=1 SAML 2.0 1121ENA_FF12=1 ID-FF 1.2 Requires ENA_SAML11=1 1122ENA_SAML11=1 SAML 1.1 1123ENA_WSF=1 All WSF Must be enabled for any of WSF to work 1124ENA_WSF2=1 ID-WSF 2.0 1125ENA_WSF11=1 ID-WSF 1.1 1126>> 1127 11285.5 localconf.mk 1129---------------- 1130 1131You can use localconf.mk to remember your own make(1) options, 1132such as ~TARGET~ and different ~ENA~ flags, without editing 1133the distributed ~Makefile~. 1134 1135One useful option to put in localconf.mk is ~ENA_GEN~ which 1136will turn on the dependencies that will trigger generation 1137of the files in zxid/c directory. For example 1138 1139 echo 'ENA_GEN=1' >>localconf.mk 1140 make 1141 1142> WARNING: If you are confused about compilation flags 1143> appearing "out of nowhere", despite not being mentioned 1144> in the Makefile, be sure you have not inadvertently created 1145> a localconf.mk (perhaps you rsynced the sources from another 1146> machine and forgot to remove localconf.mk). A similar 1147> problem can occur if you accidentally copied deps.dep 1148> from another machine. It will reference the dependencies 1149> with the paths of that other machine. Just remove the deps.dep 1150> to solve that issue. 1151 11525.6 Known Compilation Warnings 1153------------------------------ 1154 1155This list was reated around August 2010. Things are likely to have changed since then. 1156 11571. __FUNCTION__ definition incompatible with char*. Ignore. 1158 11592. const char **file incompatible with &((const char*)file 1160 1161 zxsig.c:314: warning: passing arg 1 of `ERR_get_error_line_data' from incompatible pointer type 1162 zxsig.c:314: warning: passing arg 3 of `ERR_get_error_line_data' from incompatible pointer type 1163 1164 This probably is due to const being, or not, defined differently. 1165 1166 Ignore. 1167 11683. Warning about sentinel: Ignore. 1169 1170<<ignore: 1171zxcot.c: In function `zxid_reg_svc': 1172zxcot.c:315: warning: passing arg 1 of `write_all_fd' makes pointer from integer without a cast 1173zxcot.c:322: warning: assignment makes integer from pointer without a cast 1174zxcot.c:323: warning: comparison between pointer and integer 1175zxcot.c:330: warning: passing arg 1 of `write_all_fd' makes pointer from integer without a cast 1176zxcot.c:332: warning: passing arg 1 of `close_file' makes pointer from integer without a cast 1177zxcot.c:340: warning: assignment makes integer from pointer without a cast 1178zxcot.c:341: warning: comparison between pointer and integer 1179zxcot.c:347: warning: passing arg 1 of `write_all_fd' makes pointer from integer without a cast 1180zxcot.c:348: warning: passing arg 1 of `close_file' makes pointer from integer without a cast 1181zxcot.c: In function `zxid_addmd': 1182zxcot.c:390: warning: passing arg 1 of `write_all_fd' makes pointer from integer without a cast 1183zxcot.c:395: warning: assignment makes integer from pointer without a cast 1184zxcot.c:396: warning: comparison between pointer and integer 1185zxcot.c:403: warning: passing arg 1 of `write_all_fd' makes pointer from integer without a cast 1186zxcot.c:405: warning: passing arg 1 of `close_file' makes pointer from integer without a cast 1187>> 1188 11895.7 Tough Compilation Errors 1190---------------------------- 1191 1192ZXID was mostly developed on gcc 3.4.6 and explicitly has not been 1193tested on 4.x series of gcc (due to deprecation of commonly useful 1194features in that gcc series). If you hit compile time problems with 11954.x gcc, try with 3.4.x series gcc first, before reporting bugs. 1196 1197*** elaborate more - until then, mail the author 1198 11995.8 Tough Linking Errors 1200------------------------ 1201 1202* There are some, yet unspecified, compilation or linking 1203 errors that manifest on 64 bit or mixed 32 and 64 bit Linux platforms. 1204 Current suggested workaround is to compile 32 bit only. 1205 1206* Linking error that has been reported a lot on the Google for 1207 other software packages: 1208 1209 /usr/lib/libc_nonshared.a(elf-init.oS): In function `__libc_csu_init': 1210 (.text+0x2a): undefined reference to `__init_array_start' 1211 /usr/bin/ld: zxidjava/libzxidjni.so: hidden symbol `__fini_array_end' isn't defined 1212 /usr/bin/ld: final link failed: Nonrepresentable section on output 1213 1214 This seems to be either glibc version related or linker configuration 1215 related (or compiler version?). See 1216 1217 http://www-gatago.com/gnu/gcc/help/43379636.html 1218 http://lists.debian.org/debian-68k/2006/07/msg00017.html 1219 1220 The verdict seems to be that somewhere in glibc-2.3 development 1221 it was broken by design and nobody cared to provide any viable 1222 fix short of upgrading your binutils. Seems to be Ulrich Drepper 1223 at it again, turning symbols hidden in teeny version number upgrade. 1224 1225 A symptom of this is if this command gives the follwing output 1226 1227 ld --verbose | egrep '__init_array_(start|end)' 1228 PROVIDE_HIDDEN (__init_array_start = .); 1229 PROVIDE_HIDDEN (__init_array_end = .); 1230 1231 You can manually attempt to link with 1232 1233 gcc -nostdlib -o zxidjava/libzxidjni.so -shared --export-all-symbols -Wl,-whole-archive -Wl,--allow-multiple-definition zxidjava/zxid_wrap.o -L. -lzxid -lpthread -L/usr/local/lib -L/usr/local/ssl/lib -lcrypto -lcurl -lz 1234 1235 The trick is to supply the -nostdlib flag. 1236 1237 ZXID project considers this to be glibc or binutils bug and will not 1238 investigate further. Either use the workaround link line or upgrade 1239 binutils, glibc, or both and complain to glibc authors or binutils authors. 1240 12415.9 Overview of the xsd2sg.pl Tool (guru level) 1242----------------------------------------------- 1243 1244The xsd2sg.pl tool, distributed as part of Plaindoc 1245(http://zxid.org/plaindoc/pd.html) system, is at the heart of the 1246ZXID code generation. Unfortunately the tool is still in flux. What 1247follows is general outline valid as of March 2007. 1248 1249xsd2sg.pl can be invoked in three ways (see xsd2sg.pl --help), but the 1250invocation of interest to us is the -S (SG to XSD with Code generation 1251mode). This mode takes as inputs all .sg files and names of 1252decoding root elelment. It will generate one mega parser that is capable 1253of decoding any root element (and since root elements contain other 1254elements, then really all elements). The decoder works by recursive 1255descent parser principle so there will be smaller decoder functions to 1256call. Similarily the encoder is generated by creating many unit 1257encoders for different elements. Then the parent elements will 1258call the child element encoders. 1259 12605.9.1 Process .sg to create %dt hash (throw away XSD output) 1261~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1262 1263xsd2sg.pl first processes the .sg files to XSD. As a side effect it 1264populates %dt multidimentsional hash 1265 1266 $dt{'element'}{elemname}[array-of-sublemens-and-attrs] 1267 $dt{'attribute'}{attrname} = ""; 1268 $dt{'complexType'}{typename}[array-of-sublemens-and-attrs] 1269 $dt{'group'}{groupname}[array-of-sublemens-and-attrs] 1270 $dt{'attributeGroup'}{agname}[array-of-attrs] 1271 1272Where the array-of-subelems-and-attrs can contain tag or reference to fundamental 1273or derived type, e.g. 1274 1275 "tag$sa:Assertion$min$max" 1276 "ref$xs:anyURI$" 1277 "ref$sa:Assertion$" 1278 "base$$0$1$base-type" 1279 "enum$n1$n2$n3$n4$n5" 1280 "any$$min$max$pc$ns" 1281 "anyAttribute$$min$1" 1282 "_d$$0$1$simplecontent" 1283 12845.9.2 The expanding element definitions in %dt 1285~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1286 1287Since the concrete goal is to generate data structures for elements 1288irrespective of how types were used to describe the data, we need to 1289expand any type reference into element definition where it appears. 1290This is done by expand_element() which will in its turn call 1291expand_complex_type(), expand_group() and expand_attribute_group() 1292to do the job. 1293 12945.9.3 Generation Phase 1295~~~~~~~~~~~~~~~~~~~~~~ 1296 1297Once %dt is in expanded form, the get_element() is called for each 1298XML namespace. It will generate the code for encoders and decoders 1299for the elements in the XML namespace. Finally the root decoder 1300is gnerated using gen_element(). Furing generation following 1301template files are used 1302 1303 aux-templ.c - Code generation template for auxiliary functions 1304 dec-templ.c - Code generation template for decoders 1305 enc-templ.c - Code generation template for encoders 1306 getput-templ.c - Code generation template for accessor functions 1307 1308A lot of the generated headers are simply hard coded in the xsd2sg.pl (i.e. no 1309templates). 1310 1311 131211 Integration with Existing Web Sites 1313====================================== 1314 1315Single Sign-On is used to protect some useful resources. ZXID does not 1316have any means of serving these resources, rather a normal web server 1317or application server should do it (e.g. Apache httpd(8) or Tomcat, or 1318our favorite mini_httpd(8) ). ZXID just concentrates on verifying that 1319a user has a valid session, and if not, establishing the session by way 1320of SSO. 1321 132211.1 Brief Overview of Control Flow 1323----------------------------------- 1324 1325The SAML 2.0 specifications mandate a wire protocol, and in order 1326to speak the wire protocol, the SP application typically 1327has to follow certain standard sequence of control flow. 1328 1329<<dia: sp-flow,,:bg: Typical control flow of ZXID SP>> 1330 1331First a user<<footnote: The user is often referred to as "Principal" in 1332more technical jargon. Although the human user and web browser are 1333distinct entities, we do not stress that separation here. Whatever 1334user "does" really will, in protocol, appear as web browser sending 1335requests.>> tries to access a web site that acts in SP role. This 1336triggers following sequence of events 1337 13381. User is redirected to URL in a common domain. This is so that 1339 we can read the Common Domain Cookie that indicates which 1340 IdP the user uses. Alternatively, if you started at 1341 https://sp1.zxidsp.org:8443/zxid?o=E, the CDC check is 1342 by-passed and flow 2b. happens. 1343 13442. After the CDC check, a Authentication Request (AuthnReq) is 1345 generated. The IdP may have been chosen automatically 1346 using CDC (2a), or there may have been some user interface 1347 interaction (not show in the diagram) to choose the IdP. 1348 13493. User is redirected to the IdP. The redirection carries 1350 as a query string a compressed and encoded form of 1351 the SAML 2.0 AuthnReq. 1352 13534. Once the IdP has authenticated the user, or observed 1354 that there already is a valid IdP session (perhaps from 1355 a cookie), the IdP redirects the user back to the SP. 1356 1357 The AuthnResponse may be carried in this redirection 1358 in a number of alternate ways 1359 1360 a. The redirect contains a special token called 1361 +artifact+. The artifact is a reference to the 1362 AuthnResponse and the SP needs to get the 1363 actual AuthnResponse by using a SOAP call (the 1364 4bis step). 1365 1366 b. The "redirect" is actually a HTML page with 1367 a form and little JavaScript that causes the 1368 form to be automatically posted to the SP. 1369 The AuthnResponse is carried as a form field. 1370 13715. After verifying that AuthnResponse indicated a 1372 success, the SP establishes a local session for the 1373 user (perhaps setting a cookie to indicate this). 1374 1375 Depending on how the SP to web site integration 1376 is done the user is taken to the web site in 1377 one of the two ways 1378 1379 a. Redirect to the content. This time the session 1380 is there, therefore the flow passes directly 1381 from check session to the web content. 1382 1383 b. It is also possible to show the content directly 1384 without any intervening redirection. 1385 138611.2 Redirect Approach to Integration 1387------------------------------------- 1388 1389*** TBW 1390 139111.3 Pass-thru Approach to Integration 1392-------------------------------------- 1393 139411.3.1 mod_auth_saml pass-thru 1395~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1396 1397*** TBW, but see mod_auth_saml.pd for some documentation. 1398 139911.3.2 mod_perl pass-thru 1400~~~~~~~~~~~~~~~~~~~~~~~~~ 1401 1402*** TBW, but see zxid-perl.pd for some documentation. 1403 140411.3.3 PHP pass-thru 1405~~~~~~~~~~~~~~~~~~~~ 1406 1407*** TBW, but see zxid-php.pd for some documentation. 1408 140911.4 Proxy Approach to Integration 1410---------------------------------- 1411 1412*** TBD 1413 141420 Testing 1415========== 1416 1417You can skip this section if you are not a developer. However, you may 1418be interested, from quality assurance perspective, in the extent of 1419the testing that we perform. 1420 1421ZXID test suite is driven by zxtest.pl and the test input in t/ subdirectory. 1422The test suite is not distributed in the tar ball, but is available from 1423the git repository. Some tests still rely on detailed manual setup, YMMV. 1424 1425Test coverage of the project is monitored using ~gcov~ and ~lcov~ tools. This 1426requires ZXID to be compiled in a special way: 1427 1428 make clean 1429 make all ENA_PG=1 # Enables -pg flag to gcc, among others 1430 1431Then you need to start the web servers that are used by the tests, for example: 1432 1433 mini_httpd -p 8081 -c 'zx*' & 1434 1435 cd /home/sampo/apache-tomcat-5.5.20/ 1436 killall java; bin/startup.sh 1437 1438Now you are ready to run, in the zxid distribution directory, the test suite 1439 1440 ./zxtest.pl -a -a -dx 1441 1442The test suite can also be run using web interface: http://localhost:8081/zxtest.pl 1443 1444After running the test suite and possibly performing manual tests, you can 1445generate the test coverage report by running. 1446 1447 make lcov 1448 1449You can then view the coverage report by pointing your web 1450browser to lcovhtml/index.html 1451 145220.1 Test Suite 1453--------------- 1454 1455The test suite as of release 0.76, January 2011, covers following items: 1456 1457<<logoutput: Test suite 0.76 1458STATUS SECS GOAL TEST NAME MESSAGES 1459OK 0.052 0.1 HELP1 zxcall -h 1460OK 0.038 0.1 HELP2 zxpasswd -h 1461OK 0.049 0.1 HELP3 zxcot -h 1462OK 0.044 0.1 HELP4 zxdecode -h 1463OK 0.056 0.1 HELP5 zxlogview -h 1464OK 0.058 0.1 SOENC1 EncDec Status 1465OK 0.020 0.1 ATORD1 Attribute sorting 1466OK 0.058 0.1 ATCERT1 Attribute certificate 1467OK 0.034 0.1 CONF1 zxcall -dc dump config 1468OK 0.056 0.1 CONF2 zxidhlo o=d dump config 1469OK 0.021 0.1 CONF3 zxidhlo o=c dump carml 1470OK 0.027 0.1 CONF4 zxidhlo o=B dump metadata 1471OK 0.070 0.1 CONF5 zxididp o=B dump metadata 1472OK 0.100 0.1 META1 Java LEAF Meta 1473OK 0.023 0.1 HLO1 zxidhlo o=M LECP check 1474OK 0.020 0.1 HLO2 zxidhlo o=C CDC 1475OK 0.062 0.1 HLO3 zxidhlo o=E idp select page 1476OK 0.039 0.1 HLO4 zxidhlo o=L start sso failure 1477OK 0.043 0.1 HLO5 zxidhlo o=A artifact failure 1478OK 0.042 0.1 HLO6 zxidhlo o=P POST failure 1479OK 0.021 0.1 HLO7 zxidhlo o=D deleg invite fail 1480OK 0.024 0.1 HLO8 zxidhlo o=F not an idp fail 1481OK 0.021 0.1 IDP1 zxididp o=R fail 1482OK 0.020 0.1 IDP2 zxididp o=F fail 1483OK 0.020 0.1 IDP3 zxididp o=N new user fail 1484OK 0.023 0.1 IDP4 zxididp o=W pwreset fail 1485OK 0.620 0.1 IDP5 zxididp o=S SASL Req 1486OK 0.027 0.1 PW1 zxpasswd list user 1487OK 0.024 0.1 PW2 zxpasswd pw an ok 1488OK 0.023 0.1 PW3 zxpasswd pw an fail 1489OK 0.045 0.1 PW4 zxpasswd create user 1490OK 0.043 0.1 PW5 zxpasswd change pw 1491OK 0.057 0.1 PW6 zxpasswd list user 1492OK 0.054 0.1 COT1 zxcot list 1493OK 0.037 0.1 COT2 zxcot list swap 1494OK 0.042 0.1 COT3 zxcot list s2 1495OK 0.083 0.1 COT4 zxcot get idp meta dry 1496OK 0.046 0.1 COT5 zxcot get sp meta dry 1497OK 0.023 0.1 COT6 zxcot my meta 1498OK 0.039 0.1 COT7 zxcot my meta add 1499OK 0.024 0.1 COT8 zxcot gen epr 1500OK 0.050 0.1 COT9 zxcot gen epr add 1501OK 0.022 0.1 COT10 zxcot my meta 1502OK 0.045 0.1 COT11 zxcot list s2 1503OK 0.019 0.1 LOG1 zxlogview list 1504OK 0.063 0.1 LOG2 zxlogview list 1505OK slow 0.349 0.1 SMIME1 smime key gen ca 1506OK slow 0.239 0.1 SMIME2 smime key gen joe 1507Diff ERR 0.025 0.1 SMIME3 smime ca 1508Diff ERR 0.028 0.1 SMIME4 smime code sig 1509App Err 0.034 0.1 SMIME5 smime code vfy exit=256 1510Diff ERR 0.034 0.1 SMIME6 smime sig 1511Diff ERR 0.031 0.1 SMIME7 smime clear sig 1512OK 0.026 0.1 SMIME8 smime pubenc 1513App Err 0.044 0.1 SMIME8b smime pubencdec exit=256 1514App Err 0.050 0.1 SMIME9 smime sigenc exit=256 1515App Err 0.042 0.1 SMIME10 smime encsig exit=256 1516App Err 0.048 0.1 SMIME11 smime multi sigenc exit=256 1517App Err 0.020 0.1 SMIME12 smime query sig exit=256 1518App Err 0.020 0.1 SMIME13 smime verify exit=256 1519App Err 0.020 0.1 SMIME14 smime query cert exit=256 1520App Err 0.021 0.1 SMIME15 smime verify cert exit=256 1521OK 0.016 0.1 SMIME16 smime mime ent 1522OK 0.015 0.1 SMIME17 smime mime ent b64 1523Diff ERR 0.029 0.1 SMIME18 smime pkcs12 exp 1524Diff ERR 0.021 0.1 SMIME19 smime pkcs12 imp 1525App Err 0.020 0.1 SMIME20 smime query req exit=256 1526OK 0.030 0.1 SMIME21 smime covimp 1527OK 0.071 0.1 SIG1 sig vry shib resp 1528OK 0.037 0.1 SIG2 sig vry shib post 1529OK 0.061 0.1 SIG3 sig vry zxid resp 1530OK 0.056 0.1 SIG4 sig vry zxid post 1531OK 0.030 0.1 SIG5 sig vry sm resp 1532OK 0.031 0.1 SIG6 sig vry sm post 1533App Err 0.030 0.1 SIG7 * sig vry shib resp undecl prefix exit=2816 1534OK 0.024 0.1 SIG8 * sig vry ping resp 1535OK 0.026 0.1 SIG9 sig vry ping post 1536OK 0.040 0.1 SIG10 sig vry hp a7n 1537OK 0.029 0.1 SIG11 sig vry hp post 1538OK 0.028 0.1 SIG12 sig vry hp resp 1539OK 0.028 0.1 SIG13 sig vry hp resp2 1540OK 0.027 0.1 SIG15 sig vry saml artifact response 1541OK 0.026 0.1 SIG16 sig vry saml artifact response 1542OK 0.025 0.1 SIG17 sig vry saml artifact response 1543App Err 0.041 0.1 SIG18b sig vry prstnt-a7n exit=2816 1544OK 0.026 0.1 SIG20b sig vry rsa-a7n 1545OK 0.024 0.1 SIG21b sig vry rsa-a7n2 1546OK 0.026 0.1 SIG22b sig vry rsa-idp-post 1547OK 0.046 0.1 SIG32b sig vry enc-nid-enc-attr 1548OK 0.031 0.1 SIG37 sig vry 1549OK 0.022 0.1 SIG38 sig vry 1550OK 0.023 0.1 SIG39 sig vry 1551OK 0.034 0.1 SIG40 sig vry 1552OK 0.025 0.1 SIG41 sig vry 1553OK 0.187 10 XML1 Decode-Encode SO and WO: ns-bug 1554OK 0.139 10 XML2 Decode-Encode SO and WO: azrq1 1555OK 0.164 10 XML3 Decode-Encode SO and WO: azrs1 1556OK slow 0.014 0.01 XML4 * Decode-Encode RIS malformed 1 1557OK 0.413 10 XML5 Decode-Encode SO and WO: ana7n1 1558OK 0.056 10 XML6 Decode-Encode SO and WO: anrq1 1559OK 0.166 10 XML7 Decode-Encode SO and WO: anrs1 1560OK 0.327 10 XML8 Decode-Encode SO and WO: dirq1 1561OK 0.257 10 XML9 Decode-Encode SO and WO: dirs1 1562OK 0.572 10 XML10 Decode-Encode SO and WO: dirq2 1563OK 0.223 10 XML11 Decode-Encode SO and WO: dia7n1 1564OK 0.261 10 XML12 Decode-Encode SO and WO: epr1 1565OK 0.333 10 XML13 Decode-Encode SO and WO: wsrq1 1566OK 0.281 10 XML14 Decode-Encode SO and WO: wsrs1 1567OK 0.577 10 XML15 Decode-Encode SO and WO: wsrq2 1568OK 0.435 10 XML16 Decode-Encode SO and WO: wsrs2 1569OK 0.046 10 XML17 Decode-Encode SO and WO: as-req 1570OK 0.220 10 XML18 Decode-Encode SO and WO: as-resp 1571OK 0.055 10 XML19 Decode-Encode SO and WO: authnreq 1572OK 0.017 0.1 XML20 Decode-Encode SO and WO: sun-md 1573OK 0.019 0.1 XML21 Decode-Encode SO and WO: provisio 1574OK 0.045 0.1 XML22 Decode-Encode SO and WO: provisio 1575OK 0.041 0.1 XML23 Decode-Encode SO and WO: pds-crea 1576OK 0.021 0.1 XML24 Decode-Encode SO and WO: pds-quer 1577OK 0.032 0.1 XML25 Decode-Encode SO and WO: AdvCliennt 1578OK 0.029 0.1 XML26 Decode-Encode SO and WO: AdvClient 1579OK 0.021 0.1 XML27 Decode-Encode SO and WO: AdvClient 1580OK 0.033 0.1 XML28 Decode-Encode SO and WO: AdvClien 1581OK 0.018 0.1 XML29 Decode-Encode SO and WO: AdvClien 1582OK 0.019 0.1 XML30 Decode-Encode SO and WO: zx a7n 1583OK 0.014 0.1 XML31 Decode-Encode SO and WO: covimp 1584OK 2.736 30 ZXC-AS1 Authentication Service call: SS 1585OK slow 1.069 0.1 ZXC-AS2 Authentication Service call: An 1586OK 5.019 30 ZXC-IM1 Identity Mapping Service call 1587OK 5.059 30 ZXC-IM2 * SAML NID Map call 1588OK 7.840 30 ZXC-IM3 SSOS call 1589OK 1.667 30 ZXC-DI1 Discovery Service call 1590OK slow 2.024 0.03 ZXC-DI2 List EPR cache 1591OK 9.880 30 ZXC-WS1 AS + WSF call: idhrxml 1592OK 16.51 30 ZXC-WS2 AS + WSF call: x-foobar 1593OK slow 9.338 0.1 ZXC-WS3 AS + WSF call leaf (x-recurs) 1594OK slow 2.879 0.1 ZXC-WS4 AS + WSF call EPR not found 1595OK slow 1.086 0.1 ZXC-WS5 AS + WSF call bad pw 1596OK slow 9.408 0.1 ZXC-WS6 AS + WSF call hr-xml bad 1597OK slow 9.677 0.1 ZXC-WS7 AS + WSF call hr-xml create 1598OK slow 9.855 0.1 ZXC-WS8 AS + WSF call hr-xml query 1599OK slow 9.787 0.1 ZXC-WS9 AS + WSF call hr-xml mod 1600OK slow 9.697 0.1 ZXC-WS10 AS + WSF call hr-xml mod 1601OK 0.116 0.5 LOGIN-IDP1 IdP Login screen len=1933 1602OK 0.219 0.5 LOGIN-IDP2 IdP Give password len=685 1603OK 0.121 0.5 LOGIN-IDP3 IdP Local Logout len=2309 1604OK 0.120 0.5 SSOHLO1 IdP selection screen len=3280 1605OK 0.441 0.5 SSOHLO2 Selected IdP len=2649 1606App Err 0.115 0.5 SSOHLO3 Login to IdP 1607OK 0.120 0.5 SSOHLO4 POST to SP len=3280 1608OK slow 0.674 0.5 SSOHLO5 SP SOAP Az len=52742 1609OK 0.113 0.5 SSOHLO6 SP SOAP defed len=3280 1610OK 0.109 0.5 SSOHLO7 SP SOAP defed len=3280 1611OK 0.110 0.5 SSOHLO8 SP SOAP logout len=3280 1612OK 0.110 0.5 SSOHLO9 SP local logout len=3280 1613OK 3.010 10 COVIMP1 Silly tests, to improve cov len=3280 1614=== Test success 135/155 (87.1%) === 1615>> 1616 1617As can be seen from the output, the smime tool has still many unresolved problems. 1618Other test failures are explained in the comments in the zxtest.pl application. 1619Major part of that application is a large table of tests where the test setup 1620is described and any anomalies are commented on. 1621 1622<<img: zxtest-web1: Test output using web interface.>> 1623 1624<<ignore: 1625Some things that should be tested (all are tested as of Jan 2011) 1626 16271. Will generated HTTP redirect sig validate at IdP? 16282. Does IdP issued A7N validate? 16293. Validation of EncryptedAssertion? 16304. Will generated SOAP binding sig validate at IdP? 16315. Does IdP issued SOAP sig validate? 1632 1633Metadata related 1634 16351. IBM metadata (can we parse) 16362. Sun metadata (can we parse) 1637 1638XML related 1639 16401. Fully qualified XML parses? 16412. Unknown ns prefix that refers to known namespace URI 16423. Known ns prefix, referring to wrong URI 16434. Known prefix refers to aliased URI 16445. Use of default namespaces working? 16456. Unknown prefix and URI as long as it is never used 16467. Unknown prefix and URI, used 16478. Known NS (prefix or URI), unknown element 1648>> 1649 165020.2 Test Coverage 1651------------------ 1652 1653The test coverage report looks as follows 1654 1655<<img: zxtest-lcov1: Test coverage report.>> 1656 1657As of R0.73, December 2010, the test coverage was about 63% of the 1658handwritten lines of code that could execute. While this figure is 1659still mediocre, it should be noted that 100% test coverage is 1660inachievable and useless stretch goal. We feel about 80% would 1661be good. 1662 1663There are several reasons why 100% is not achievable: 1664 16651. Defensive coding requires us to insert error checks at 1666 many layers. This means that some of the error checks 1667 will never trigger because the checks in other layers 1668 already checked the situation. It is therefore virutally 1669 impossible to construct a test case that would cause 1670 the code path to be taken. 1671 16722. Similar to above, some checks require extreme inputs, e.g. 1673 to overflow buffers. Such inputs may be very expensive 1674 to generate and extremely unlikely in real practise. 1675 While these situations should be tested for security 1676 vulnearabilities, there is not much harm done if 1677 null pointer exception causes a cgi process to crash 1678 rather than return meaningful error message. The cost 1679 of testing these corner cases can be hard to justify. 1680 16813. We have peppered the code with extensive debug prints. While 1682 crash due to debug print is nasty, it is also avoidable 1683 by turning off the debug flag. Not worth burning too much 1684 midnight oil to concot test cases to test every 1685 corner case in this area. 1686 16874. The test coverage considerations do not include subdirectory c/ 1688 as that contains machine generated code. This is highly 1689 repetitive and there is no point to include it in coverage 1690 score. 1691 1692With the above caveats, it still needs to be admitted that more 1693testing could be done. In particular the biggest gaps appear in 1694areas where new functionality is already supported at encoder-decoder 1695level, but not yet realized in hand written logic. 1696 1697<<if: ZXIDBOOK>> 1698<<else: >> 1699 170096 License 1701========== 1702 1703Copyright (c) 2006-2009 Symlabs (symlabs@symlabs.com), All Rights Reserved. 1704Author: Sampo Kellom�ki (sampo@iki.fi) 1705 1706Copyright (c) 2011 Sampo Kellom�ki (sampo@iki.fi), All Rights Reserved. 1707 1708Licensed under the Apache License, Version 2.0 (the "License"); 1709you may not use this file except in compliance with the License. 1710You may obtain a copy of the License at 1711http://www.apache.org/licenses/LICENSE-2.0 1712 1713Unless required by applicable law or agreed to in writing, software 1714distributed under the License is distributed on an "AS IS" BASIS, 1715WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 1716See the License for the specific language governing permissions and 1717limitations under the License. 1718 1719While the source distribution of ZXID does not contain 1720SSLeay or OpenSSL code, if you use this code you will use OpenSSL 1721library. Please give Eric Young and OpenSSL team credit (as required by 1722their licenses). 1723 1724Binary distribution of this product includes software developed by the 1725OpenSSL Project for use in the OpenSSL Toolkit 1726(http://www.openssl.org/). See LICENSE.openssl for further information. 1727 1728Binary distribution of this product includes cryptographic software 1729written by Eric Young (eay@cryptsoft.com). Binary distribution of 1730this product includes software written by Tim Hudson 1731(tjh@cryptsoft.com). See LICENSE.ssleay for further information. 1732 1733And remember, you, and nobody else but you, are responsible for 1734auditing ZXID and OpenSSL library for security problems, 1735back-doors, and general suitability for your application. 1736 173796.1 Dependency Library Licenses 1738-------------------------------- 1739 1740ZXID strives to maintain IPR hygiene and avoid both 1741non-free and GPL license contamination. All the 1742dependency libraries have BSD style licenses 1743 1744* OpenSSL under BSDish (with "advertising" clause) 1745* libcurl under BSDish 1746* zlib under BSDish 1747* libc available as part of the operating system 1748 1749Please see each library package for the exact details of their 1750licenses. 1751 175296.2 Specification IPR 1753---------------------- 1754 1755ZXID is based on open SAML and Liberty specifications. The parties 1756that have developed these specifications, including Symlabs, have made 1757Royalty Free (RF) licensing commitment. Please ask OASIS and Liberty 1758Alliance for the specifics of their IPR policies and IPR disclosures. 1759 1760Some protocols, such as WS-Trust and WS-Federation enjoy Microsoft's 1761pledge<<footnote: If you have a reference to where this pledge can be 1762found, please let me know so it can be included here.>> that they will 1763not sue you even if you implement these specifications. You should 1764evaluate yourself whether this is good enough for your situation. 1765 176696.3 Further Warranties 1767----------------------- 1768 1769If you need the author or Symlabs to further disclaim IPR interest or 1770make warranties of non-infringement, such declarations are 1771available for a fee. Please contact sales@symlabs.com 1772 1773Legal queries and clarifications will be answered at then-current 1774Symlabs Professional Services rate, please contact sales@symlabs.com. 1775 177697 FAQ 1777====== 1778 177997.1 Compilation Problems 1780------------------------- 1781 178297.1.1 OpenSSL not found: you need to create localconf.mk 1783~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1784 1785ZXID does NOT have a configure script. It ships with a notion 1786of "standard" locations for the three dependency libraries, 1787but if these libraries are not where it expects to find them, 1788then typically you see (n.b. lines were folded for presentation): 1789 1790 make 1791 If you get compilation errors, try: make help 1792 gcc -g -fpic -fmessage-length=0 -Wno-unused-label -Wno-unknown-pragmas 1793 -fno-strict-aliasing -D_REENTRANT -DDEBUG -DUSE_CURL -DUSE_OPENSSL 1794 -DLINUX -I/tmp/zxid-0.20 -I/usr/local/ssl/include -I/usr/local/include 1795 -c -o zxid.o zxid.c 1796 zxid.c:34:23: curl/curl.h: No such file or directory 1797 In file included from zxid.c:38: 1798 zx.h:26:25: openssl/rsa.h: No such file or directory 1799 ... 1800 1801What happened is that OpenSSL for some reason is not in the location 1802where standard OpenSSL distribution would install it (as indicated by 1803-I/usr/local/ssl/include flag that ships with ZXID Makefile). You need 1804to determine where OpenSSL is installed in your case. You can use 1805 1806 find / -name rsa.h -ls 1807 1808to locate candidates. 1809 1810For example, if it turns out that OpenSSL is installed in /opt/ssl, then 1811you need to create a localconf.mk file that indicates this location: 1812 1813 echo OPENSSL_ROOT=/opt/ssl >localconf.mk 1814 1815There are several other make variables you may need to tweak. In the 1816above example, we also notice that libcurl was not found where 1817expected. This would be fixed like this 1818 1819 echo CURL_ROOT=/opt/curl >>localconf.mk 1820 1821Net result? ZXID does not try to guess where the libraries are. It 1822makes you do the foot work of locating the correct libraries (some 1823people have more than one instance installed) and prepare the 1824localconf.mk. This may seem like a lot of work, but in my experience, 1825fixing GNU autohell configure scripts that guess wrong is thousand 1826times more frustrating. The system is dumb by design so you, as a human, 1827do not have to try to second guess it - you are in control. 1828 182997.1.2 Missing gperf 1830~~~~~~~~~~~~~~~~~~~~ 1831 1832 gcc -g -fpic -fmessage-length=0 -Wno-unused-label -Wno-unknown-pragmas -fno-strict-aliasing -D_REENTRANT -DDEBUG -DUSE_CURL -DUSE_OPENSSL -DLINUX -I/c/cvs/zxid_cvs -I/usr/local/ssl/include -I/usr//include -c -o c/zx-a-aux.o c/zx-a-aux.c 1833 c/zx-a-aux.c: In function "zx_NEW_a_Action": 1834 c/zx-a-aux.c:80: error: "zx_a_Action_ELEM" undeclared (first use in this function) 1835 1836This happens because c/zx-const.h was misgenerated (it should not happen at 1837all if you do not supply ENA_GEN=1) and does not include the necessary defines. 1838c/zx-const.h should have more than 1900 lines and look something like 1839 1840 /* generated file, do not edit! zx_ _ATTR */ 1841 #ifndef _zx__ATTR 1842 #define _zx__ATTR 1843 #define zx_use_ATTR 0 1844 #define zx_used_ATTR 1 1845 #define zx_sequence_ATTR 2 1846 ... 1847 #define zx_wantDSEPR_ATTR 347 1848 #define zx_ZX_TOK_NOT_FOUND_ATTR 348 1849 #define zx__ATTR_MAX 349 1850 #endif 1851 /* generated file, do not edit! zx_ _ELEM */ 1852 #ifndef _zx__ELEM 1853 #define _zx__ELEM 1854 #define zx_ds_Y_ELEM 0 1855 #define zx_gl_Y_ELEM 1 1856 #define zx_gl_esrd_ELEM 2 1857 ... 1858 #define zx_wst_OnBehalfOf_ELEM 1629 1859 #define zx_ZX_TOK_NOT_FOUND_ELEM 1630 1860 #define zx__ELEM_MAX 1631 1861 #endif 1862 186397.1.3 make samlmod gives "incompatible types in assignment" 1864~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1865 1866Should not happen with version 0.21 or later. See zxidnoswig.h 1867for explanation of the problem. 1868 186997.1.4 Perl compiled with different compiler than zxid 1870~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1871 1872Perl modules generally want to be compiled with the same C compiler 1873and options as were used to compile perl itself (see perl -V). If this 1874happens to be different than the compiler you have defined in CC 1875variable (gcc by default, near top of Makefile or in localconf.mk), you may 1876get an error like: 1877 1878 cd Net; perl Makefile.PL && make 1879 Warning: -L.. changed to -L/home/sampo/zxid/Net/.. 1880 Writing Makefile for Net::SAML 1881 make[1]: Entering directory `/home/sampo/zxid/Net' 1882 cc -c -I.. -I/apps/openssl/std/include -I/apps/include -D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -pipe -Wdeclaration-after-statement -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -march=i586 -mtune=i686 -fmessage-length=0 -Wall -D_FORTIFY_SOURCE=2 -g -Wall -pipe -DVERSION=\"\" -DXS_VERSION=\"\" -fPIC "-I/usr/lib/perl5/5.8.8/i586-linux-thread-multi/CORE" SAML_wrap.c 1883 /bin/sh: cc: command not found 1884 make[1]: *** [SAML_wrap.o] Error 127 1885 make[1]: Leaving directory `/zxid/Net' 1886 make: *** [samlmod] Error 2 1887 1888*Solutions* 1889 18901. Compile zxid with compiler that was used for perl, e.g. 1891 1892 make CC=the-compiler-that-perl-wants 1893 18942. Recompile perl using the compiler that you want to use for zxid 1895 18963. Tinker with PATH environment variable so that both C compilers 1897 are found. However, using two different compilers is not really supported. 1898 1899In general these types of problems happen when you use perl installed 1900by your distribution, but have later compiled a gcc of your own. It may 1901even be that you never installed the distribution cc - in that case 1902consider installing it and then trying approaches 1 or 3. 1903 1904A similar situation can arise with incompatibility of the compiler and 1905options used for dependency libraries, such as OpenSSL or libcurl, and 1906those used for compiling zxid itself. 1907 190897.1.5 All files under zx missing 1909~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1910 1911You need to symlink zx to zxid source directory, thus 1912 1913 ln -s . zx 1914 1915If you do not have it, then you will get a lot of file inclusion errors for 1916headers that are supposed to be in path starting by zx/ 1917 1918The symlink is there to keep all hand written source files on top 1919level of directory for ease of development, yet allow inclusions to go 1920through ~zx~ subdirectory. When zxid is installed, it goes to 1921/usr/include/zx. Hence the symlink keeps the includes the same whether 1922developing or using installed version. 1923 192497.1.6 Compiler Warnings 1925~~~~~~~~~~~~~~~~~~~~~~~~ 1926 1927If you compile zxid with compiler warnings turned on (CFLAGS += -Wall), 1928you will see quite a number of warnings, most of which are 1929unwarranted. Since the warnings are unwarranted, I ship zxid Makefile 1930with warnings turned off. If this bothers you, feel free to investigate 1931the warnings and report to me any issues you uncover. 1932 1933Following warnings in partuclar are unwarranted: 1934 19351. Any unusued variable warnings, especially in generated code. Most 1936 common of these is ~se~ variable (see enc-templ.c). 19372. "Suggest parenthesis around assignment when used as truth value." I 1938 rely on C language operator precedence. Also, in most cases the 1939 assignment is the only expression in the truth test - there simply 1940 is no opportunity for ambiguity -- and no justified case for gcc to 1941 warn about this. 19423. "Suggest parenthesis around && when used in ||". I rely on C 1943 language operator precedence, hence the suggestion is redundant. 1944 1945Some warnings you may want to worry about 1946 1947A. "int format, long int arg". On 32 bit platforms int and long 1948 are both 32 bits so this warning is not an issue. On 64 bit platforms, 1949 however, there may be cause for worry. 1950 195197.1.7 SWIG and Java Problems 1952~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1953 1954javac -J-Xmx128m -g zxid.java zxidjava/*.java 1955zxidjava/zxidjni.java:159: cannot find symbol 1956symbol : class SWIGTYPE_p_p_void 1957location: class zxidjava.zxidjni 1958 public static zx_str zx_rsa_pub_enc(zx_ctx c, zx_str plain, SWIGTYPE_p_p_void rsa_pkey, int pad) { 1959 ^ 1960zxidjava/zxidjni.java:164: cannot find symbol 1961symbol : class SWIGTYPE_p_p_void 1962location: class zxidjava.zxidjni 1963 public static zx_str zx_rsa_pub_dec(zx_ctx c, zx_str ciphered, SWIGTYPE_p_p_void rsa_pkey, int pad) { 1964 ^ 1965zxidjava/zxidjni.java:169: cannot find symbol 1966symbol : class SWIGTYPE_p_p_void 1967location: class zxidjava.zxidjni 1968 public static zx_str zx_rsa_priv_dec(zx_ctx c, zx_str ciphered, SWIGTYPE_p_p_void rsa_pkey, int pad) { 1969 ^ 1970zxidjava/zxidjni.java:174: cannot find symbol 1971symbol : class SWIGTYPE_p_p_void 1972location: class zxidjava.zxidjni 1973 public static zx_str zx_rsa_priv_enc(zx_ctx c, zx_str plain, SWIGTYPE_p_p_void rsa_pkey, int pad) { 1974 ^ 1975This was due to missing SWIG generated classes. Probably interrupted file transfer. 1976 1977javac -J-Xmx128m -g zxid.java zxidjava/*.java 1978zxid.java:24: cannot find symbol 1979symbol : method new_conf(java.lang.String) 1980location: class zxidjava.zxidjni 1981 cf = zxidjni.new_conf("/var/zxid/"); 1982 ^ 1983zxid.java:27: cannot find symbol 1984symbol : method url_set(zxidjava.zxid_conf,java.lang.String) 1985location: class zxidjava.zxidjni 1986 zxidjni.url_set(cf, url); 1987 ^ 1988zxid.java:28: cannot find symbol 1989 1990jar cf zxidjava.jar *.class 1991jar cf /tmp/zxidjava.jar zxidjava/*.class 1992 1993javac -J-Xmx128m -g zxid.java 1994zxid.java:187: cannot access zxid_conf 1995bad class file: /Library/Java/Extensions/zxidjava.jar(zxid_conf.class) 1996class file contains wrong class: zxidjava.zxid_conf 1997Please remove or make sure it appears in the correct subdirectory of the classpath. 1998 public static int mgmt_screen(zxid_conf cf, zxid_cgi cgi, zxid_ses ses, char op) 1999 ^ 20001 error 2001 2002Underscore in linking error 2003 2004./zxid-java.sh 2005Start... 2006Exception in thread "main" java.lang.NoSuchMethodError: zxidjava.zxidjni.new_conf(Ljava/lang/String;)Lzxidjava/zxid_conf; 2007 at zxid.main(zxid.java:24) 2008 2009This was due to finding some old copies from system paths. 2010 2011java -classpath .:zxidjava -Djava.library.path=zxidjava zxid 2012Start... 2013Exception in thread "main" java.lang.UnsatisfiedLinkError: _zxid_new_conf 2014 at zxidjava.zxidjniJNI._zxid_new_conf(Native Method) 2015 at zxidjava.zxidjni.new_conf(zxidjni.java:586) 2016 at zxid.main(zxid.java:24) 2017 2018 201997.2 Platform Specifics 2020----------------------- 2021 2022If your Unix platform is not mentioned, you should try saying just 2023 2024 make 2025 2026which will compile with Linux options. These options actually 2027are pretty close to pure POSIX compile so you should get very close 2028to working configuration. 2029 203097.2.1 Linux 2031~~~~~~~~~~~~ 2032 2033Native development platform. Just say 2034 2035 make 2036 2037Seems there are some "improvements" that distributions have made. ZXID 2038adopts the policy of expecting dependency modules where the module 2039author meant it to be installed by default - for example OpenSSL by 2040default installs in /usr/local/ssl (naming is historic, but has stuck). 2041Many distros tinker with these paths. This means you need to create 2042a localconf.mk. 2043 2044Redhat used to have an issue with Net::SAML (make samlmod). This has 2045since been fixed, please see zxidnoswig.h for explanation. 2046 2047No doubt, distros will eventually pick up ZXID and provide it as a package. 2048Once that happens they will solve any path issues accoring to their 2049disto policy and that is fine, just do not ask me to comply with any 2050such policy. 2051 205297.2.2 FreeBSD 2053~~~~~~~~~~~~~~ 2054 2055No target available on Makefile, but a port is available from 2056http://www.freshports.org/security/zxid/ 2057 205897.2.3 Solaris (Sparc) 2059~~~~~~~~~~~~~~~~~~~~~~ 2060 2061 make TARGET=sol8 2062 make TARGET=xsol8 # Cross compile for Solaris (e.g. on Linux host) 2063 206497.2.4 MacOS X (PowerPC?) 2065~~~~~~~~~~~~~~~~~~~~~~~~~ 2066 2067 make TARGET=macosx 2068 206997.2.5 Windows Using MinGW 2070~~~~~~~~~~~~~~~~~~~~~~~~~~ 2071 2072 make zxid.dll TARGET=xmingw # Cross compile on Linux host (best supported) 2073 make zxid.dll TARGET=mingw # Native compile for mingw target in Cygwin environment 2074 2075Eitherway, the net result is native Windows DLL that does not have 2076Cygwin library dependencies or GPL encumberation. 2077 2078See Makefile for further mingw notes. 2079 208097.2.6 Windows Using Cygwin 2081~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2082 2083 make TARGET=cygwin 2084 2085Very experimental (as of Oct 2007) native build for Cygwin. 2086 2087Cygwin appears to not have neither flock(2) nor lockf(2). This is 2088strange because at least one of these is implemented on MinGW. 2089Current workaround is to define flock() to be empty macro. This 2090of course means there is no file locking. There are 3 known races 2091where things can go wrong 2092 20931. Audit logs can get garbled. This does not stop ZXID from working, 2094 but may make log analysis more complicated. 20952. Auto-CoT metadata writes can get garbled. This is very unprobable, 2096 but if it happens, the ZXID deployment will not work towards 2097 affected IdP. Nothing to worry about really. 20983. Locking is used to protect against updates of zxid.conf while 2099 zxid is running. Again any corruption is very unlikely. Nothing 2100 to worry about. 2101 2102The results of Cygwin compile may be GPL encumbered due to libraries. 2103 210497.2.7 Windows Using MSVC 2105~~~~~~~~~~~~~~~~~~~~~~~~~ 2106 2107Never been done (as of Oct 2007), but probably this is not very 2108difficult given that MinGW port already has addressed many Windows 2109platform issues. Please send any success reports, and receipes, my way. 2110 211197.5 Deployment Planning 2112------------------------ 2113 2114Here is a rudimentary decision tree for deployment planning 2115 21161. List your applications 2117 a. Any provided by external partner? 2118 b. Non web apps 21192. Document your existing identity stores and approaches to 2120 a. User provisioning (when someone is employed) 2121 b. Application provisioning (when someone starts using app) 2122 c. Authorization: how do you know who is supposed to be 2123 doing what? 2124 d. Deprovisioning: what happens when someone is fired? 2125 e. Login? Per app? Harmonized user names? Enterprise SSO? 21263. Document your goal: federated SAML SSO �ber alles :-) 2127 a. Do you want to run IdP? 2128 b. Could you out-source IdP? 2129 c. Will your partners / customers be running their own IdPs? 2130 d. Will you participate (or run) single CoT or do you 2131 need to consider cross CoT inter-operation (e.g. IdP proxying) 2132 2133To be continued... 2134 2135 213698 Support 2137========== 2138 213998.1 Mailing list and forums 2140---------------------------- 2141 2142Mail the author until we get the list set up. Or volunteer a list :-) 2143 214498.2 Bugs 2145--------- 2146 2147Mail the author until we get bug tracking set up. Or volunteer. 2148 214998.3 Developer access 2150--------------------- 2151 2152We use CVS, but access needs to be manually configured and is not 2153anonymous. If you contribute significantly, I will bother. Others 2154can send patches (good way to show you are worthy of CVS access) 2155to me. I've heard some mixed experiences about open source 2156sites like sourceforge. If you run such site and want to 2157host ZXID Project, please contact me. 2158 2159If you just always want the latest source: get the tar ball from 2160the downloads section. Trust me, this is still so much in flux 2161that only the tar ball snapshots are in any usable state. CVS 2162access just to get latest source would be pointless. 2163 216498.9 Commercial Support 2165----------------------- 2166 2167Following companies provide consultancy and support contracts for 2168ZXID: 2169 2170* symlabs.com 2171* Levelview, Lda. 2172 2173<<zxid-ref.pd>> 2174<<doc-end.pd>> 2175<<notapath: TCP/IP a.k.a xBSD/Unix n/a Perl/mod_perl PHP/mod_php Java/Tomcat>> 2176<<EOF: >> 2177<<fi: >> 2178