1$NetBSD: UPDATING,v 1.78 2002/12/04 12:50:43 lukem Exp $ 2 3This file is intended to be a brief introduction to the build 4process and a reference on what to do if something doesn't work. 5 6For a more detailed description see Makefile. 7 8Recent changes: 9^^^^^^^^^^^^^^^ 10 1120021130: 12 fparseln(3) moved from libutil to libc. 13 If building to DESTDIR=/, reinstall the includes 14 and rebuild libc: 15 make includes 16 make do-lib-libc 17 If using build.sh, "cd tools/compat && make clean" 18 before rebuilding the tools. 19 2020021126: 21 The mk.conf(5) variable SYS_INCLUDE has been deprecated, 22 including the optional "SYS_INCLUDE=symlinks" support. 23 All header files, including <sys/*.h> are copied into 24 /usr/include. 25 2620021121: 27 The C run-time support files crtbegin.o and crtend.o 28 (and their companions crtbeginS.o and crtendS.o) were 29 split up, with new crti.o and crtn.o files resulting. 30 This means that libtool needs to be rebuilt once the 31 new libraries are installed. The process of rebuilding 32 libtool will cause it to automatically notice the new 33 required files, but it *must* be rebuilt in order to 34 do this. 35 36 An out-of-date libtool will result in shared libraries 37 which lack _init() and _fini() routines, which means that 38 their global contructors/destructors will not be invoked. 39 4020021121: 41 A bug related to how ARM ELF objects were tagged has been 42 corrected. 43 44 NetBSD ARM ELF uses the soft-VFP floating point model by 45 default. However, the assembler lacked support for marking 46 objects as using the VFP floating point format, and the 47 compiler was no properly passing the flag indicating "soft-VFP" 48 to the assembler. 49 50 Unfortunately, this means that the linker will now consider 51 old (i.e. not marked "softvfp") NetBSD ARM ELF objects to be 52 incompatible with new (properly marked) objects. 53 54 The problem will only manifest itself if you attempt to compile 55 a new program using the fixed toolchain, and link that program 56 against old libraries which do not have the proper "softvfp" 57 markings. ALL OF YOUR EXISTING BINARIES AND SHARED LIBRARIES 58 WILL CONTINUE TO WORK PROPERLY. 59 60 The only work-around for the problem is to recompile all of 61 the libraries on the system. The easiest way to do this for 62 system libraries is to install a binary snapshot; there are 63 generally available on releng.netbsd.org. Any packages you 64 have installed which supply libraries will have to be recompiled 65 if you wish to link new programs against those libraries. 66 67 If you have questions about this matter, please contact 68 port-arm@netbsd.org. 69 7020021011: 71 Systrace has been improved to support privilege elevation. 72 Updating the kernel requires the userland part of systrace 73 to be rebuild. 74 7520021010: 76 The config(8) grammar was changed to allow options to register 77 dependencies on attributes, as well as other options. Users 78 must update and reinstall usr.sbin/config before building a new 79 kernel. 80 8120021009: 82 A new attribute dependency syntax was introduced to config(8), 83 which is now used by the SCSI configuration description. Users 84 must update and reinstall usr.sbin/config before building a new 85 kernel. 86 8720021003: 88 Several changes have been made to the autoconfiguration 89 framework. Users must update and reinstall usr.sbin/config 90 before building a new kernel. 91 9220021001: 93 The i386mp branch has been merged. To compile a kernel, users 94 will need to add the option 'cpu* at mainbus?' to their configuration 95 file. Multiprocessor kernels will need 96 ioapic* at mainbus? apid ? 97 options MULTIPROCESSOR 98 options COM_MPLOCK 99 10020020922: 101 MKDYNAMICROOT=yes enabled by default, which means that 102 certain shared libraries are installed into /lib, the shared 103 linker is installed into /libexec, and all programs in /bin 104 and /sbin are dynamically linked. 105 If you do not use "make build", you should ensure that 106 you have the libraries and shared linker in the new locations, 107 with: 108 make do-lib-csu do-lib-libc do-lib do-gnu-lib do-ld.elf_so 109 11020020917: 111 USE_NEW_TOOLCHAIN has been replaced with: 112 - TOOLCHAIN_MISSING -- set to "yes" on platforms for which 113 there is no working in-tree toolchain (hppa, ns32k, sh5, 114 x86_64). 115 - EXTERNAL_TOOLCHAIN -- if defined by the user, points to the 116 root ofan external toolchain (e.g. /usr/local/gnu). This 117 enables the cross-build framework even for TOOLCHAIN_MISSING 118 platforms. 119 12020020906: 121 gehenna-devsw has been merged into the trunk. Need to update and 122 reinstall usr.sbin/config before build the kernel. 123 12420020822: 125 Crunched rescue tools (contents of /bin and /sbin, plus others) 126 are now provided in /rescue. 127 128 To ensure that these are built statically linked (no matter 129 what the setting of LDSTATIC is), use a crunchgen(1) built 130 from sources newer than 20020820 (see the next entry). 131 13220020820: 133 crunchgen(1) changed to ensure that the generated program 134 is statically linked. 135 136 Solution: update and reinstall usr.bin/crunch 137 13820020515: 139 sshd user/group has been added. Need to hand add this in, or sshd 140 will not let you log in (with default, or UsePrivlegeSeparation=yes) 141 142 Add the following into /etc/group: 143 144 sshd:*:16: 145 146 and the following to /etc/master.passwd (via vipw): 147 148 sshd:*:16:16::0:0:& pseudo-user:/var/chroot/sshd:/sbin/nologin 149 150 also /var/chroot/sshd directory needs to be present (digged as a part of 151 build process). 152 15320020426: 154 NBUILDJOBS obsoleted in favor of just using -j. 155 15620020426: 157 etc/postinstall added, which performs various checks for 158 configuration file updates and changes, and can fix most of 159 the problems identified. 160 This should make it much easier to upgrade a system's 161 configuration from earlier systems (as far back as NetBSD 1.5). 162 16320020320: 164 <bsd.lib.mk> needs a new install(1) for it's "-a cmd" support. 165 build and install at usr.bin/xinstall before the build. 166 16720020319: 168 raw IPv6 socket now makes strict checking for sa_family and sa_len 169 on send(2) operation. be sure to have sbin/rtsol and usr.sbin/rtsold 170 newer than November 2001 when you upgrade the kernel. 171 17220020311: 173 ssh configuration files were moved from /etc to /etc/ssh. Beware 174 if you restart your machine from remote. Note that sshd.conf needs 175 to be changed (due to the use of "/etc" inside). 176 17720020223: 178 Users of the VAX port will need to rebuild and install gas 179 so it deal with the now present register prefix used in all 180 the VAX assembly files. 181 18220020118: 183 184 ntpd user/group has been added. Need to hand add this in or builds 185 will break as mtree aborts early. 186 187 Add the following into /etc/group: 188 189 ntpd:*:15: 190 191 and the following to /etc/master.passwd (via vipw): 192 193 ntpd:*:15:15::0:0:Ntpd pseudo-user:/var/chroot/ntpd:/sbin/nologin 194 19520011207: 196 197 If you're attempting to build a snapshot on sparc64 and are getting 198 reloc errors from the toolchain groff binary this means your native 199 toolchain has some broken C++ bits. 200 201 To fix: 202 203 Build a new toolchain (i.e. build.sh -t) 204 Use the new toolchain to build and install natively (i.e. /usr/lib) 205 206 gnu/lib/libgcc 207 gnu/lib/libstdc++ 208 209 After this a snapshot will be able to be built. 210 21120011201: 212 In order for a sparc64 build to work you must have a working awk. If 213 you've built and installed a system with the new toolchain up to this 214 point you do not have a working awk as its ability to do floating 215 point is broken. 216 217 To build: 218 219 remake and install gnu/lib/libgcc 220 remake and install gnu/usr.bin/gawk into /usr/bin (make sure it links 221 against the new libgcc.a) 222 22320011128: 224 Kernel config information was changed to use defflag in 225 the various "files" files. Bug fixes to config(8) are 226 required in order for this to work properly. Make sure 227 to build and install in usr.sbin/config before attempting 228 to build a new kernel. 229 23020011030: 231 libc/locale/wcstod.c now needs new lint(1). Update lint(1) 232 before building libc. 233 23420011029: 235 The new document BUILDING.mdoc (view with nroff | more, or 236 see pre-generated .txt and .html versions) describes the build 237 procedure in great detail. BUILDING, and the USE_NEW_TOOLCHAIN 238 build process, are intended in the long run to replace this 239 manual update log. 240 241 Users building a USE_NEW_TOOLCHAIN system should read the 242 BUILDING document for caveats. Generally, BUILDING supersedes 243 UPDATING for these systems, as tool updating is taken care of 244 by the new build system. 245 24620011028: 247 src/etc/Makefile now needs install to be able to handle 248 symlinks that point to nowhere. A bug in install that 249 prevented this was corrected. 250 251 Solution: update and reinstall usr.bin/xinstall 252 Better Solution: Use the new toolchain and it will just work 253 for you. 254 25520011006: 256 /etc/mtree/NetBSD.dist has been updated to take advantage of 257 absolute path support added to mtree(8). Older mtree(8)s don't 258 understand the format. 259 260 Solution: update and reinstall usr.sbin/mtree 261 26220011004: 263 Crunchgen has been updated to work via reach-over makefiles. Updating 264 is suggested before running a snapshot build 265 26620010915: 267 The new "ubcperf" code committed by Chuck Silvers removed 268 a header file, uvm/uvm_vnode.h. There may be stale .depend 269 files that still reference this file. 270 271 Solution: "make cleandir && make dependall" in affected 272 directories. 273 27420010803: 275 grep.info is now built from grep.texi using makeinfo. Since it 276 requires makeinfo v4.0, you need to install new texinfo before 277 building gnu/usr.bin/grep. To install new texinfo, please follow 278 the instruction described in 20010726 entry. 279 28020010803: 281 (i386 only): i386 kernel now uses new instructions like 282 `fxsave' which old gas doesn't understand. To build the 283 kernel successfully, you need to build and install a new toolchain, 284 (i.e., build.sh -t) or (temporarily) comment out "options I686_CPU" 285 from your kernel configuration until you rebuild your userland. 286 See 20011029 above and BUILDING file in this directory for more information. 287 [updated 20020630 since i386 gas moved when USE_NEW_TOOLCHAIN enabled] 288 28920010731: 290 Bootloader update on ELF platforms. DDB in kernels from before 291 this will be unable to read symbol tables provided by newer 292 bootloaders. 293 29420010726: 295 Texinfo was updated to 4.0. To avoid failures when trying to 296 build the included texinfo files, do: 297 298 cd src/gnu/usr.bin/texinfo 299 make MKINFO=no dependall install 300 30120010718: 302 Enabled correct .init/.fini processing in crt0. The way this 303 was done was to change a -I directive to cc(1), which means 304 make(1) will have a stale dependency (it will be checking the 305 timestamp on the wrong "dot_init.h"). 306 307 The symptom you will see is that new programs die with SIGSEGV 308 if you have a stale dependency. 309 310 Solution: "make cleandir" in both lib/csu and libexec/ld.elf_so 311 before starting your build. 312 31320010628: 314 A construct was added to uvm_page.h that uncovered a bug 315 in lint(1). If you get a warning/error about a non-portable 316 bitfield, update your lint(1) before proceeding. 317 31820010226: 319 Added named user/group to system. Need to hand add this in or builds 320 will break as mtree aborts early. 321 322 To work around add by hand: 323 324 named:*:14: 325 326 to /etc/group and add: 327 328 named:*:14:14::0:0:Named pseudo-user:/var/named:/sbin/nologin 329 330 to master.passwd (use vipw for instance if doing by hand). 331 332 Now a make build should progress. 333 33420010219: 335 get/setprogname() added. Any hostprogs that may use this will need 336 to be bootstrapped manually until the host system is current. 337 338 Known problems: sys/arch/macppc/stand/fixcoff 339 usr.sbin/config (adding -DMAKE_BOOTSTRAP to 340 CFLAGS and rebuilding should work) 341 usr.sbin/mdsetimage - Build a static copy if 342 building a snapshot before fully bootstrapped. 343 34420010204: 345 prepare the code to compile with stricter gcc flags. in 346 particular start eliminating redundant declarations. Yacc 347 needs to be installed before make build. 348 34920010114: 350 introduce .if commands(target) in make(1). You need to 351 bring everything up-to-date first, then without installing 352 anything make and install in usr.bin/make, then proceed 353 with make build. 354 35520010101: 356 bsd.subdir.mk committed 20001230 had a bug which caused 357 afterinstall targets to run too soon; update again. 358 35920001230: 360 New share/mk files needed to support .WAIT in SUBDIR variables. 361 If you get make errors, 362 (cd share/mk; make install) 363 Also, PRINTOBJDIR has changed and is now used more heavily. 364 36520001019: 366 The `ca' device driver has been replaced by `ld'; although the 367 major and minor numbers haven't changed, you should update your /dev 368 directory. 369 37020000929: 371 The following make directives are obsoleted. 372 MKCRYPTO_RSA NOCRYPTO_RSA NOCRYPTO_RC5 NOCRYPTO_IDEA 373 By default, RSA is built into libcrypto. IDEA and RC5 will not be 374 built into libcrypto. By using MKCRYPTO_{RC5,IDEA}, you can build 375 additional library libcrypto_{idea,rc5}. 376 377 378Hints for a more successful build: 379^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 380 Build a new kernel first: 381 This makes sure that any new system calls or features 382 expected by the new userland will be present. This 383 helps to avoid critical errors when upgrading. 384 Use object directories: 385 This helps to keep stale object 386 files from polluting the build if a Makefile "forgets" 387 about one. It also makes it easier to clean up after 388 a build. It's also necessary if you want to use the 389 same source tree for multiple machines. 390 To use object directories: 391 a) cd /usr/src ; make cleandir 392 b) Add "OBJMACHINE=yes" to /etc/mk.conf 393 c) Add "MKOBJDIRS=yes" to /etc/mk.conf 394 d) cd /usr/src ; make build 395 Note that running "make obj" in a directory will create 396 in obj.$MACHINE directory. 397 Build to a DESTDIR: 398 This helps to keep old 399 installed files (especially libraries) from interfering 400 with the new build. 401 To build to a DESTDIR, set the DESTDIR environment 402 variable before running make build. It should be set to 403 the pathname of an initially empty directory. 404 Problems: you might need to update critical utilities 405 without using DESTDIR since nothing is executed 406 from what is installed in DESTDIR. 407 (See critical utils, below) 408 Build often: 409 This keeps critical utilities current enough to not choke 410 on any other part of the source tree that depends on up to 411 date functionality. 412 413What to do if things don't work: 414^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 415When things don't work there is usually a few things that commonly 416should be done. 417 1) make includes 418 This should be done automatically by make build. 419 2) cd share/mk && make install 420 Again, automatically done by make build. 421 422Failsafe rebuild of a small part of the tree: 423^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 424To make sure you rebuild something correctly you want to do 425something like the following: 426 1) Make sure the includes and .mk files are up to date. 427 2) Make sure any program used to build the particular 428 utility is up to date. (yacc, lex, etc...) 429 3) cd ...path/to/util... 430 make cleandir 431 rm ...all obj directories... 432 make cleandir # yes, again 433 make obj 434 make depend && make 435 436Failsafe rebuild of the entire tree: 437^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 438If you really want to make sure the source tree is clean and 439ready for a build try the following. Note that sourcing /etc/mk.conf 440(a make(1) Makefile) in this manner is not right, and will not work 441for anyone who uses any make(1) features in /etc/mk.conf. 442 443---cut here--- 444#!/bin/sh 445. /etc/mk.conf 446 447if [ -z $NETBSDSRCDIR ] ; then 448 NETBSDSRCDIR=/usr/src 449fi 450if [ \! -d $NETBSDSRCDIR ] ; then 451 echo Unable to find sources 452 exit 1 453fi 454find $NETBSDSRCDIR -name \*.o -o -name obj.\* -o -name obj -exec rm \{\} \; 455 456if [ -z $BSDOBJDIR ] ; then 457 BSDOBJDIR=/usr/obj 458fi 459if [ -d $BSDOBJDIR ] ; then 460 rm -rf $BSDOBJDIR 461fi 462 463cd $NETBSDSRCDIR && make cleandir 464 465---cut here--- 466 467Critical utilities: 468^^^^^^^^^^^^^^^^^^^ 469 gnu/usr.bin/egcs 470 usr.bin/compile_et 471 usr.bin/make 472 usr.bin/yacc 473 usr.bin/lex 474 usr.bin/xlint 475 usr.sbin/config 476 477Other problems and possible solutions: 478^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 479Symptom:Unreasonable compiler errors. 480Fix: Rebuild gnu/usr.bin/egcs 481 482Symptom:Complaints involving a Makefile. 483Fix: Rebuild usr.bin/make: 484 cd usr.bin/make && make && make install 485 Or, a failsafe method if that doesn't work: 486 cd usr.bin/make && cc *.c */*.c -I . -o make && mv make /usr/bin 487 488Fix: Make sure .mk files are up to date. 489 cd share/mk && make install 490 491Symptom:Kernel `config' fails to configure any kernel, including GENERIC. 492Fix: Rebuild usr.sbin/config 493 494Symptom: 495Fix: Rebuild usr.bin/yacc 496 497Symptom: 498Fix: Rebuild usr.bin/lex 499 500Symptom: 501Fix: rm /usr/lib/libbfd.a 502 503Symptom:Obsolete intermediate files are used during compilation 504Fix: Try the following sequence of commands in the directory in question. 505 make cleandir; rm `make print-objdir`; make cleandir; make obj 506 (If you built the tree without "make obj" in the past, obsolete files 507 may remain. The command tries to clean everything up) 508 509Symptom:.../sysinst/run.c:xx: warning: initialization from incompatible pointer type 510Fix: Rebuild and install usr.bin/menuc 511 512Symptom:mklocale not found during build in share/locale/ctype 513Fix: Build and install usr.bin/mklocale 514 515Symptom:undefined reference to `__assert13' 516Fix: Rebuild and install lib/libc 517 518Symptom:usr.sbin/config fails to build. 519Fix: Try building with -DMAKE_BOOTSTRAP added to CFLAGS in Makefile. 520 521Symptom:undefined reference to `getprogname' or `setprogname' 522Fix: Rebuild and install lib/libc 523 524Symptom:lint does not understand the '-X' option 525Fix: May need to build & install libs with NOLINT=1 before rebuilding lint 526