1$NetBSD: UPDATING,v 1.342 2023/07/19 17:49:40 martin Exp $ 2 3This file (UPDATING) is intended to be a brief reference to recent 4changes that might cause problems in the build process, and a guide for 5what to do if something doesn't work. 6 7For a more detailed description of the recommended way to build NetBSD 8using build.sh, see the BUILDING file. 9 10Note that much of the advice in this UPDATING file was written before 11build.sh existed. Nevertheless, the advice here may be useful for 12working around specific problems with build.sh. 13 14Sections are marked with "^^^^^". After the section on "Recent changes" 15are several sections containing more general information. 16 17See also: BUILDING, build.sh, Makefile. 18 19Recent changes: 20^^^^^^^^^^^^^^^ 21 2220230718: 23 build.sh will use mk.conf in the same directory as build.sh instead 24 of /etc/mk.conf, unless MAKECONF is already defined in the environment. 25 2620230714: 27 Import of gcc 10.5 requires a clean build of at least tools/gcc 28 and external/gpl3/gcc* object directories. An update build 29 without cleaning will result in obscure failures in rare corner 30 cases. 31 3220230604: 33 Building native kernels with the installed make(1) instead of 34 the $TOOLDIR/bin/nbmake-${arch} wrapper will fail if userland 35 has not been updated. Note that this is an unsupported build 36 configuration (but usually works if userland is -current enough). 37 3820230523: 39 ctfmerge was fixed as macOS host tool. This may require a clean tools 40 rebuild to detect the update to tools/compat/configure. 41 4220230515: 43 New openssl build requires removal of the crypto/external/bsd/openssl 44 and tests/crypto/libcrypto subdirectories in the ${OBJDIR} hierachy. 45 Otherwise test results may be wrong due to wrong "golden" output 46 files being installed. 47 4820230322: 49 Fixes for moved /lib/libisns* mean that builds will complain about 50 ./usr/lib/libisns* being missing until src/lib/libisns is forced 51 to (re)install the files, including the symlinks. The simplest 52 way is to clean this subdir before building. 53 5420230112: 55 New binutils require an updated ld.elf_so. If you are doing 56 (unsupported) in-place self builds (with the -E flag to build.sh), 57 make sure to have installed latest ld.elf_so before you rebuild 58 userland. 59 6020221116: 61 The addition to NetBSD's version of UFS2 of support for extended 62 attributes broke backward compatibility with previous releases 63 of NetBSD, so UFS2 has been restored to being compatible with 64 previous NetBSD releases by disabling extended attributes. 65 (Note that ACLs are implemented as extended attributes, so 66 this changes disables ACLs as well.) 67 68 Support for UFS2 with extended attributes is now available in a new 69 UFS variant called UFS2ea. If you have created extended attributes 70 in an original UFS2 file system then "fsck -p" will now fail due to 71 the unexpected presence of extended attributes and "fsck -y" will 72 remove all extended attributes. If you wish to preserve extended 73 attributes rather than delete them, there is a utility to convert 74 a UFS2 file system to UFS2ea and leave extended attributes in place, 75 but this should be used with caution since it will preserve any 76 extended attributes that have been corrupted by the backward 77 incompatibility too. 78 79 If you wish to use a UFS2ea file system as your root file system, 80 then you will need to update your boot loader to a version that 81 supports UFS2ea. 82 83 For more information, see: 84 https://wiki.netbsd.org/features/UFS2ea 85 8620221111: 87 The new libdrm import worsened the conflict issues for the 88 kdump/ktruss ioctl, and i915 now conflicts with base, and has 89 been turned off. This will cause update build issues like: 90 91 kdump-ioctl.c:12175:143: error: 'DRM_IOCTL_I915_DESTROY_HEAP' 92 undeclared here (not in a function); 93 did you mean 'DRM_IOCTL_MODE_DESTROY_DUMB'? 94 95 You'll need to clean usr.bin/ktruss, usr.bin/kdump, and rescue. 96 9720220921: 98 Since the kernel version was bumped to 9.99.100, the bootloader 99 must be updated on x86 or EFI boot platforms in order to load 100 modules from boot.cfg(5) or interactively at the bootloader 101 prompt. 102 103 The kernel can still load modules at runtime, with modload(8) 104 or modules.conf(5), with no bootloader update. This will not 105 affect release branches because it only applies to patch 106 numbers >=100. 107 108 On x86 (i386 or amd64) with BIOS boot, this requires copying a 109 new /usr/mdec/boot to /boot. You can build this in the destdir 110 by running `$TOOLDIR/bin/nbmake-$ARCH dependall install' in 111 sys/arch/i386/stand. 112 113 On platforms with EFI boot (including x86 with EFI boot), this 114 requires copying a new /usr/mdec/boot*.efi to the EFI/BOOT/ 115 directory on the EFI system partition. The boot*.efi file is: 116 117 aarch64 bootaa64.efi 118 amd64 bootx64.efi 119 arm bootarm.efi (32-bit arm) 120 i386 bootx32.efi 121 122 You can build this in the destdir by running 123 `$TOOLDIR/bin/nbmake-$ARCH dependall install' in 124 sys/stand/efiboot/boot*. 125 12620220821: 127 Support for building extsrc/ has been deprecated. 128 EXTSRCSRCDIR and MKEXTSRC have been deprecated. 129 13020220714: 131 Updates to xorg-server and associated drivers may cause builds to 132 fail. Cleaning both DESTDIR and the xorg build tree may be needed. 133 13420220628: 135 Changes in the build of src/games/robots require a clean build 136 of that program. 137 13820211116: 139 Changes in the xorg build require clean obj dirs for external/mit/xorg 140 (or a full clean/non-update build). 141 14220211112: 143 Device tree sources were updated for evbarm. Some device nodes 144 (in particular, ld(4) devices on the ROCKPro64) will be renumbered. 145 14620211014: 147 MKPIE default for aarch64 has changed, a clean build is required. 148 14920210917: 150 Changed MNT_ACLS to default to POSIX1e ACLs instead of NFSv4 151 ACLs, to be compatible with FreeBSD. If you are using NFSv4 ACLs 152 and have "acls" in /etc/fstab you'll need to change it to "nfs4acls". 153 15420210711: 155 Updated GMP sources may require cleaning in tools/gmp and/or in 156 external/lgpl3/gmp/lib, particularly if GCC itself does not build. 157 15820210417: 159 GCC 10 was enabled for several platforms. If builds fail in either 160 tools/gcc or external/gpl/gcc, first try cleaning those objects and 161 removing the $DESTDIR/usr/include/g++ subdirectory. 162 16320201230: 164 lint1/ops.c is no longer autogenerated. If this makes the build 165 fail, clean $OBJDIR/tools/*lint* and $OBJDIR/usr.bin/*xlint*. 166 16720201016: 168 MIPS kernel modules have been disabled until they work. This will 169 turn up in extra files in the DESTDIR, which should be cleaned. 170 17120200925: 172 GNU MPC and MPFR have been updated. At least MPFR needs cleaning 173 in both the tools and external dirs. 174 17520200912: 176 GCC 9 has arrived for Arm and x86 platforms, and will be coming 177 for several more, as will binutils 2.34. Clean as required. 178 17920200907: 180 GCC 9 is coming, and binutils has been updated for MIPS. This 181 probably requires cleaning the tools/binutils. 182 18320200811: 184 GCC updates may require cleaning the objdir. This may occur 185 due to GCC 7.5 update, GCC 8.4 move to gcc.old, or the upcoming 186 GCC 9 upgrade. 187 18820200614: 189 blacklist* has been renamed to blocklist*. postinstall(8) 190 should handle the migration. 191 19220200601: 193 Due to a mistake in LIBISPRIVATE handling, .so libraries were 194 created in the build directories and need cleaning. 195 19620200311: 197 GCC 8 ports will need cleaning in src/tools/gcc and 198 src/external/gpl3/gcc due to GCC 8.4 update. 199 20020191118: 201 More architectures were switched to gcc8: 202 i386, ia64 powerpc64, sparc, sparc64, arm 203 The same comments as in 20191022 apply. 204 20520191112: 206 The LLVM update requires a clean rebuild for all architectures using 207 LLVM during the tools build phase (i386, amd64, aarch64). 208 20920191022: 210 Ports amd64 and aarch64 switched to GCC 8.3 by default. 211 In-place ("expert mode", build.sh -E) builds are not supported 212 when going from a GCC 7 userland to GCC 8. Do a regular 213 build to a different DESTDIR (or preferably: build.sh -U) at least 214 once and install sets, or download comp.{tar.xz,tgz} from the 215 daily builds and install that before doing the next in-place build. 216 21720191001: 218 GCC 8.3 was imported. Builds of src/tools/gcc may fail if 219 old builds with GCC 7 output now uses GCC 8. Clean this 220 directory, and also clean src/external/gpl3/gcc. 221 22220190903: 223 Files with names that coincide with existing files' names on 224 case-insensitive file systems were inadvertently committed, for 225 radeon GPU firmware. We cannot mark these as obsolete for 226 postinstall to fix, so if you updated src since 2019-08-26, and 227 ran build.sh distribution or ran build.sh release, you must 228 manually delete the following files in your DESTDIR (which is 229 usually $OBJDIR/destir.$ARCH), or from / if you have installed 230 them: 231 232 /libdata/firmware/radeon/bonaire_ce.bin 233 /libdata/firmware/radeon/bonaire_mc.bin 234 /libdata/firmware/radeon/bonaire_me.bin 235 /libdata/firmware/radeon/bonaire_mec.bin 236 /libdata/firmware/radeon/bonaire_pfp.bin 237 /libdata/firmware/radeon/bonaire_rlc.bin 238 /libdata/firmware/radeon/bonaire_sdma.bin 239 /libdata/firmware/radeon/bonaire_smc.bin 240 /libdata/firmware/radeon/bonaire_uvd.bin 241 /libdata/firmware/radeon/hainan_ce.bin 242 /libdata/firmware/radeon/hainan_mc.bin 243 /libdata/firmware/radeon/hainan_me.bin 244 /libdata/firmware/radeon/hainan_pfp.bin 245 /libdata/firmware/radeon/hainan_rlc.bin 246 /libdata/firmware/radeon/hainan_smc.bin 247 /libdata/firmware/radeon/hawaii_ce.bin 248 /libdata/firmware/radeon/hawaii_mc.bin 249 /libdata/firmware/radeon/hawaii_me.bin 250 /libdata/firmware/radeon/hawaii_mec.bin 251 /libdata/firmware/radeon/hawaii_pfp.bin 252 /libdata/firmware/radeon/hawaii_rlc.bin 253 /libdata/firmware/radeon/hawaii_sdma.bin 254 /libdata/firmware/radeon/hawaii_smc.bin 255 /libdata/firmware/radeon/kabini_ce.bin 256 /libdata/firmware/radeon/kabini_me.bin 257 /libdata/firmware/radeon/kabini_mec.bin 258 /libdata/firmware/radeon/kabini_pfp.bin 259 /libdata/firmware/radeon/kabini_rlc.bin 260 /libdata/firmware/radeon/kabini_sdma.bin 261 /libdata/firmware/radeon/kaveri_ce.bin 262 /libdata/firmware/radeon/kaveri_me.bin 263 /libdata/firmware/radeon/kaveri_mec.bin 264 /libdata/firmware/radeon/kaveri_mec2.bin 265 /libdata/firmware/radeon/kaveri_pfp.bin 266 /libdata/firmware/radeon/kaveri_rlc.bin 267 /libdata/firmware/radeon/kaveri_sdma.bin 268 /libdata/firmware/radeon/mullins_ce.bin 269 /libdata/firmware/radeon/mullins_me.bin 270 /libdata/firmware/radeon/mullins_mec.bin 271 /libdata/firmware/radeon/mullins_pfp.bin 272 /libdata/firmware/radeon/mullins_rlc.bin 273 /libdata/firmware/radeon/mullins_sdma.bin 274 /libdata/firmware/radeon/oland_ce.bin 275 /libdata/firmware/radeon/oland_mc.bin 276 /libdata/firmware/radeon/oland_me.bin 277 /libdata/firmware/radeon/oland_pfp.bin 278 /libdata/firmware/radeon/oland_rlc.bin 279 /libdata/firmware/radeon/oland_smc.bin 280 /libdata/firmware/radeon/pitcairn_ce.bin 281 /libdata/firmware/radeon/pitcairn_mc.bin 282 /libdata/firmware/radeon/pitcairn_me.bin 283 /libdata/firmware/radeon/pitcairn_pfp.bin 284 /libdata/firmware/radeon/pitcairn_rlc.bin 285 /libdata/firmware/radeon/pitcairn_smc.bin 286 /libdata/firmware/radeon/tahiti_ce.bin 287 /libdata/firmware/radeon/tahiti_mc.bin 288 /libdata/firmware/radeon/tahiti_me.bin 289 /libdata/firmware/radeon/tahiti_pfp.bin 290 /libdata/firmware/radeon/tahiti_rlc.bin 291 /libdata/firmware/radeon/tahiti_smc.bin 292 /libdata/firmware/radeon/verde_ce.bin 293 /libdata/firmware/radeon/verde_mc.bin 294 /libdata/firmware/radeon/verde_me.bin 295 /libdata/firmware/radeon/verde_pfp.bin 296 /libdata/firmware/radeon/verde_rlc.bin 297 /libdata/firmware/radeon/verde_smc.bin 298 299 We will re-import these radeon firmware images another way 300 later. 301 30220190727: 303 The uefi bootloader has gained tftp support and needs a clean 304 build. If you do update builds, manually clean its object 305 directory by something like: 306 cd sys/arch/i386/stand/efiboot && make clean 307 30820190723: 309 The jemalloc allocator in libc is now build without extended 310 debugging (for performance reasons). In update builds make sure 311 to rebuild it completely, by removing all affected object files, 312 including compat builds, something like: 313 cd /usr/obj && find . -type d -name jemalloc|xargs rm -rf 314 31520190207: 316 GCC 7 switched for many ports. Update builds are likely to fail. 317 31820180924: 319 A newer OpenSSL version has been imported. If you are doing 320 update builds, make sure to remove all old obj dirs, like: 321 cd /usr/obj && find . -type d -name openssl | xargs rm -rf 322 32320180717: 324 On aarch64 int64_t and related types have changed from long long 325 to long. This requires recompiling all C++ binaries. 326 32720180713: 328 On amd64 and i386 static binaries are now build position 329 independend. This requires recompilation of all object 330 files used to create the crunched /rescue binary. 331 Clean the rescue directory in your obj directory before 332 doing an update build, otherwise linking will fail. 333 33420180414: 335 Existing binutils was migrated to binutils.old. Manual 336 removal of tools/binutils objects directory may be required 337 to fix tools build failure. 338 33920180311: 340 bdftopcf was updated and may need cleaning in the 341 src/external/mit/xorg/tools/bdftopcf subdirectory if there are 342 link errors. 343 34420180212: 345 between OpenSSL and GCC updates, many things may fail to build. 346 any failure that looks like GCC or openssl is best handled by 347 a clean destdir and objdir. Full cleandir and destdir deletion 348 is recommended if build failures occur. 349 35020171225: 351 removal of the vadvise syscall requires manual removal of all 352 associated files from the libc build object directory (including 353 the .depend files) - a command like: 354 cd $OBJ && find . -type d -name libc | xargs rm -rf 355 For architectures that support multiple "compat" binary targets, 356 you'll need to cleanup both the regular libc directory and the 357 compat ones (the above command will do that). 358 35920171010: 360 a change to the build structure of external/bsd/acpica/bin/iasl 361 means that its objdir (or *.d and .depend at least) might need 362 to be manually removed - or a build done once without -u. 363 36420170822: 365 a new version of GMP has been imported and probably 366 will break parts of builds related to themselves or GCC, both 367 in the tools and the native section. Remove all GCC, GMP, MPFR 368 and MPC objdirs or build once without -u. 369 37020170816: 371 a new version of MPFR and MPC have been imported and probably 372 will break parts of builds related to themselves or GCC, both 373 in the tools and the native section. Remove all GCC, GMP, MPFR 374 and MPC objdirs or build once without -u. 375 37620170402: 377 a new version of dhcpcd has been imported, which does not support 378 update builds from the previous version. Remove your 379 external/bsd/dhcpcd object dir or build once without -u. 380 38120170211: 382 a new terminfo database has been imported. 383 The structure of it has changed slightly from prior versions and 384 an updated tic tool is required. 385 If you build.sh, don't use -u 386 38720170207: 388 various arch dependent libc/exect.S files were removed 389 Either remove the obj directories (lib/libc and compat/amd64/i386/lib 390 if it exists) or do a clean build. 391 (This is a bug in the make system, it should be corrected without 392 human intervention, but isn't.) 393 39420170104: 395 xinput build options have changed. 396 Remove the obj directory (external/mit/xorg/bin/xinput) 397 if you build.sh -u 398 39920170103: 400 a new version of flex has been imported. 401 Remove the file from obj (external/bsd/flex) 402 if you build.sh -u 403 40420161014: 405 a new version of OpenSSL has been imported. 406 Remove the files from obj (crypto/external/bsd/openssl) 407 if you build.sh -u 408 40920161009: 410 a new version of dhcpcd has been imported with slightly changed 411 build infrastructure. When doing a build.sh -u this requires 412 pruning the external/bsd/dhcpcd objdir. 413 41420160914: 415 i386, amd64, shark, ofppc and macppc have joined shark and x68k 416 ports in using xorg-server 1.18. This requires a clean destdir 417 and a clean objdir. 418 41920160527: 420 i386 needs a full cleandir or objdir deletion because PIE has 421 been enabled. (see the amd64 entry two down for further info) 422 42320160418: 424 libedit needs manual removal of all autogenerated files since 425 some of them are not autogenerated anymore. Remember that there 426 might be two copies of libedit if your platform builds "compat". 427 42820160410: 429 amd64 needs full "make cleandir" or deletion of objdir now that 430 PIE has been enabled for amd64. PIE, or position-independent 431 executables, means all code, including executables and not just 432 shared libraries, is position-independent and hence able to be 433 relocated by ASLR, address space layout randomization. 434 435 The change was made in Makefile variables for compiler and 436 linker flags, for which make(1) does not record dependencies, 437 hence it is unable to detect that all .o files need rebuilding. 438 439 Partial rebuilds with some modified source files will likely 440 fail when linking executables, since the linker refuses to mix 441 position-independent code with position-dependent code in 442 position-independent executables: 443 444 .../x86_64--netbsd/bin/ld: foo.o: relocation R_X86_64_32 against `...' cannot be used when making a shared object; recompile with -fPIC 445 44620160401: 447 Ports switching to GCC 5.3 will need a full "make cleandir". 448 Some people have found that cleandir is not sufficient, so if 449 in doubt delete the entire object directory tree for gcc. 450 45120160306: 452 NetBSD's regexp implementation is now part of libnbcompat. 453 Nblex, nbm4, nbpax, nbsed use it and they may crash on 454 non-NetBSD hosts if linked with stale object files produced 455 while they included a non-NetBSD regex.h. 456 In any case, you should "make cleandir" at least these tools 457 before updating your toolchain. 458 45920160125: 460 Dtrace has been enabled by default on some architectures. 461 When doing an update build, make sure to clean the etc/mtree 462 object directory before starting the build - otherwise the 463 needed directories in destdir will not be created. 464 An easy way to do this is: 465 cd src/etc/mtree && $TOOLDIR/bin/nbmake-$arch cleandir 466 46720150818: 468 New acpica requires "make cleandir" in src/external/bsd/acpica 469 again. 470 47120150413: 472 New acpica requires "make cleandir" and reinstalling yacc 473 in /usr/src/external/bsd/byacc and /usr/src/tools/yacc and 474 also "make cleandir" in /usr/src/external/bsd/acpica. 475 47620150404: 477 Lint changes require a full rebuild of the tool, so make 478 sure to build without -u option to build.sh, or manually 479 do a make cleandir in src/tools/lint1 and 480 src/usr.bin/xlint. 481 48220150310: 483 Improvements to openssl for arm mean that update builds of 484 the openssl libraries will fail. A make cleandir in 485 external/bsd/openssl/lib is needed 486 48720141026: 488 A mishap during the import of pppd may cause your corruption 489 in your cvs directory if you happened to do a cvs update 490 during a short period of time. To fix, just remove the 491 directory src/external/bsd/ppp/dist/pppd completely 492 and let cvs restore it on next update. 493 49420140721: 495 The src/external/mit/lua/src directory was accidentally created 496 and then deleted in the CVS repository. If you get errors like 497 498 cvs [update aborted]: cannot open directory 499 /cvsroot/src/external/mit/lua/src: No such file or directory 500 501 then delete your local copy of that directory tree and try again. 502 50320140530: 504 ARM eABI switched to DWARF based exception handling. This requires 505 rebuilding all C++ code. It is strongly advised to do a clean build. 506 50720140131: 508 The new compiler_rt/libc integration moved a few things. It is strongly 509 advised to do a clean build. At least lib/libc, the compat version(s) 510 of libc, libkern, rump and the kernels need to be cleaned. 511 51220131227: 513 1. The new ntpd runs in a restricted mode to prevent amplification 514 attacks. If you need ntpdc to work you need to explicitly enable 515 mode7 in your config file. Make sure you put the necessary restrict 516 statements to avoid being exposed. 517 2. strncat has moved from lib/libc to common/lib/libc; you might need 518 to make clean in libc 519 52020131129: 521 The GMP sources were updated, and builds will likely fail without 522 cleaning their build trees for both tools and in-tree, like below. 523 52420131128: 525 The MPC and MPFR sources were updated, and builds may require their 526 tools and in-tree directories cleaned for successful updates. 527 52820130605: 529 The kernel option FAST_IPSEC no longer exists, it's been renamed 530 to IPSEC (and the older IPSEC version removed). 531 53220130605: 533 Previous freetype installations erroneously installed private 534 header files. If you are building against a non-empty $DESTDIR, 535 please remove ${DESTDIR}//usr/X11R7/include/freetype2/freetype/. 536 53720130531: 538 The xdm update may cause build failure due to xdm.man being 539 in the obj tree. Make sure to ensure any like this: 540 nbmake: nbmake: don't know how to make xdm.man. Stop 541 is fixed by deleting the xdm.man in the obj tree. 542 54320130530: 544 Updates of many xsrc packages will leave old .pc files around. 545 Best to clean out the xsrc objdir entirely before rebuilds. 546 54720130301: 548 The removal of netiso requires manual removal of /usr/include/netiso 549 prior to the build and make cleandir in /usr/src/usr.bin/{ktruss,kdump} 550 55120120726: 552 The update of OpenSSL requires cleaning both the OpenSSL build 553 directory and DESTDIR. *Even non-update builds require cleaning 554 DESTDIR.* Builds done without taking these steps may fail, or in 555 some cases may succeed and install broken OpenSSL libraries that 556 cause third-party software to link incorrectly and/or crash. 557 55820120507: 559 The database schema for makemandb was changed. You will 560 need to update the database using 'makemandb -f' or wait 561 for the next weekly run to fix it. 562 56320120319: 564 sys/conf/Makefile.kern.inc has been modified to adjust the 565 size of db_symtab automatically. You need to update dbsym in 566 your $TOOLDIR to build kernels with options SYMTAB_SPACE. If 567 you don't want this behavior, add AUTO_SYMTAB_SPACE=no to your 568 mk.conf. 569 57020120216: 571 Default for MKCATPAGES changed to NO. Update builds will fail 572 unless DESTDIR is cleaned manually. If you built between 20120207 573 and 20120216, daily and weekly could have created an unreadable 574 /var/db/man.db index for apropos. Running makemandb -f or 575 the next run of weekly will fix it. 576 57720111227: 578 If you built between 20111225 and 20111227 you need to remove 579 /usr/lib/libpam.so.4* and /usr/lib/security/*.so.4, since the 580 bump has been reverted. 581 58220111125: 583 The "rnd" pseudodevice has been added to sys/conf/std, which 584 means it should no longer be explicitly listed in kernel 585 configuration files. The line "pseudo-device rnd" should be 586 removed from any custom kernel config files users may have. 587 58820111119: 589 A problem with the datastructures used by the rndctl(8) 590 utility (pointers in datastructures in an array, making 32->64 591 bit compatibility very painful) has been fixed in a 592 non-backwards-compatible way. If you replace your kernel, 593 replace your rndctl executable too. 594 59520111001: 596 the prop_*_send_syscall() functions from proplib(3) have been 597 changed and their new version is not backward compatible with the old 598 one. So ensure that all consumers of these functions (currently: 599 quota2 code and its tests) are updated together with the new lib. 600 60120110817: 602 sparc has been changed to use GCC 4.5.3, so any objdir or 603 DESTDIR for them should be deleted before updating. 604 60520110806: 606 i386 and amd64 have been changed to use GCC 4.5.3, so any 607 objdir or DESTDIR for them should be deleted before updating. 608 60920110805: 610 The update to GCC 4.5.3 requires a non-trivial portion of 611 the tree to be cleaned. Best to delete both objdir and 612 DESTDIR before running this update. So far, only the 613 sparc64, mips and powerpc platforms have changed. 614 61520110803: 616 The layout of external/public-domain/xz has changed. To do an 617 update build you will have to remove the contents of the OBJDIR 618 for external/public-domain/xz/bin by hand as the xz entry there 619 is now a directory. 620 62120110410: 622 The configuration of src/tools/gcc has changed. To do an 623 update build you have to clean both tools/binutils and 624 tools/gcc by hand. 625 62620110328: 627 Building the Xorg binary was moved into a subdirectory to fix 628 ordering issues with "make all". It may be necessary to remove 629 the OBJDIR for external/mit/xorg/server/xorg-server/hw/xfree86 630 if your update build fails, as the "Xorg" entry there is now a 631 directory. 632 63320110121: 634 Assembler files no longer use -traditional-cpp. This can break 635 the build of individual parts of the tree. This is handled 636 correctly by build.sh. Manual builds have to update /usr/share/mk 637 and re-run config(1) for any kernel configurations as needed. 638 63920101217: 640 The tcpdump(8) program was changed to drop privileges and chroot(2) 641 by default. It may be necessary to manually update passwd(5) and 642 group(5) in order to make the program work with existing setups. 643 64420101125: 645 The latest changes to setenv(3) disallow setting environment 646 variables with names that contain '='. Revision 1.18 of env.c 647 assumed that this was allowed. Installing a new libc with an 648 old copy of /usr/bin/env causes env x=1 printenv | grep x= to 649 break which affects the autoconf tests for dependency finding, 650 so building gcc will end up printing: 651 checking dependency style of gcc... none 652 configure: error: no usable dependency style found 653 Fix it by rebuilding and re-installing env. 654 65520101119: 656 Recent Xorg updates in xsrc/external/mit/ may cause various build 657 or run-time problems. Delete your entire DESTDIR and OBJDIR if you 658 have any build problems with xsrc, or problems with mismatched 659 versions between xorg-server and drivers. 660 66120100604: 662 The update of ATF to 0.9 causes old tests written in shell to fail 663 unless they are rebuilt. If you are building with MKUPDATE=yes, 664 you need to clean the src/external/bsd/atf/tests/ and the src/tests/ 665 trees by hand. 666 66720100522: 668 Recent Xorg updates in xsrc/external/mit/ will cause various build 669 problems. Delete your entire DESTDIR and OBJDIR if you have any 670 build problems with xsrc. 671 67220100522: 673 private section of <ctype.h> was split, and now mklocale(1) 674 include ctype_local.h, so you have to make cleandir in tools/mklocale. 675 67620100520: 677 The location of the xkb compiled descriptions has changed. Please 678 remove usr/X11R7/lib/X11/xkb/compiled from your $DESTDIR. 679 68020100222: 681 The shared objects file extension has been changed from .so to 682 .pico, in order to avoid conflicts with shared libraries names 683 libXX.so. All now stale regular .so files can be removed from 684 your object directories. 685 68620100204: 687 The termcap database has been removed from the sources, 688 but has not been marked obsolete so it is not removed 689 from the system when upgrading. 690 As such, you will need to remove them from your object 691 and destination directories. 692 69320091101: 694 After updating, it may be necessary to make the 'cleandir' 695 target in src/tools/yacc/ and in src/usr.bin/yacc/ before a 696 'build.sh -u tools' or 'build.sh -u distribution'. Ditto 697 src/tools/lex/ and src/usr.bin/lex/. 698 69920091001: 700 On amd64 you must rebuild tools (to get the new binutils) 701 before building a kernel, or the build fails on cpufunc.S. 702 70320091001: 704 An error will create a ./usr/X11R7/lib/X11/xkb/compiled/xkb 705 symlink, failing the build. Delete the link, and the subdir 706 it is in, and retry your build. 707 70820090718: 709 libc build changed so that strchr() provides the extra entry 710 point for index(). Update build of libc.a (and libc_pic.a) 711 may fail because the archive contains the unwanted index.o. 712 (Similarly for strrchr() and rindex().) 713 71420090709: 715 Native Xorg was upgraded again. Builds will probably fail again 716 without a clean objdir, at least for src/external/mit/xorg. 717 71820090616: 719 Native Xorg was upgraded. Builds will need a clean objdir for 720 src/external/mit/xorg. Upgrading a system from sets will not 721 work properly yet as the /usr/X11R7/lib/X11/xkb/symbols/pc 722 subdirectory has been changed into a file, and this needs to 723 be manually "rm -r"'ed before installing xbase.tgz. 724 72520090501: 726 Several new functions were added to string.h/libc, and this 727 can cause autoconf problems during the tool build for people 728 who fail to clean out their tools objects properly. If you 729 note messages about stpcpy, stpncpy or strnlen accompanying a 730 failure during the tool build, clean out all your tools 731 objects and start again. 732 73320090325: 734 The i386 port was switched to i486 default toolchain. This requires 735 cleaning your src/tools directory and $TOOLDIR and rebuilding them. 736 73720090126: 738 The __posix_fadvise50 system call changed assembly stub type. You 739 need to 'rm -f __posix_fadvise50.*' in the libc build directory 740 to avoid using the old assembly stub. 741 74220090202: 743 pkg_install now depends on the pkgdb cache for automatic conflict 744 detection. It is recommended to rebuild the cache with 745 ``pkg_admin rebuild''. 746 74720090110: 748 time_t and dev_t have been bumped to 64 bit quantities. To upgrade: 749 1. Make sure your kernel has COMPAT_50 in it. Build and install. 750 This is needed even in the MODULAR kernel because there is 751 conditionally compiled code in rtsock.c. 752 2. make sure build.sh completes and the binaries in a chroot work 753 before installing. 754 3. If you don't use build.sh and you build directly to root, and 755 your build breaks in the middle, don't despair. Make sure headers 756 are installed properly, and start building libraries first libc 757 and libutil, install them and then continue building all the 758 libraries in src/lib and src/gnu/lib and install them. Once 759 the new libraries are installed, you can restart the build. 760 4. If you compile packages and you notice link time warnings, 761 rebuild the required packages to update their shared libraries. 762 Any package you rebuild will require rebuilding all the packages 763 that depend on it. 764 5. Next time you run pwd_mkdb with the new binary, the file 765 will be upgraded and it will not be backwards compatible. 766 6. The utmpx/wtmpx files (/var/run/utmpx and /var/log/wtmpx, see 767 lastlogx(5)) have been versioned, and there is a heuristic 768 for utmp. You are better off removing the old files after 769 upgrading. The automated clearing of /var/run during 770 boot, and the automated rotating of files in /var/log by 771 newsyslog(8), may mean that you do not have to remove the 772 files manually. 773 7. The optional accounting file (/var/account/acct, see 774 accton(8)) has not been versioned, and will need to be 775 removed. The automatic rotation of the accounting file by 776 /etc/daily limits the bad consequences of failure to remove 777 the file. 778 8. Application software that writes time_t to binary files on 779 disk will break or need attention. Most notably: if you are 780 using PostgreSQL < 8.4, you need to dump your databases, 781 rebuild PostgreSQL with the new time_t, then restore. 782 78320081219: 784 config(1) has been updated, and one of the files it creates - 785 swapnetbsd.o - has changed format. You need to rebuild config 786 (done automatically by build.sh) and then you need to rerun 787 config on all kernel configuration files before rebuilding those 788 kernels. 789 79020081205: 791 If you build with MKX11=no, you should remove /etc/rc.d/xdm and 792 /etc/rc.d/xfs from DESTDIR because those files were moved to the xetc 793 set and will appear as extra files for MKX11=no update builds. 794 79520081122: 796 On i386, various kernel options(4) in GENERIC including 797 file systems have been disabled and moved into kernel modules. 798 Before trying a new GENERIC kernel, you have to prepare the 799 following files as well as a new GENERIC kernel: 800 801 - build and install kernel modules from src/sys/modules 802 803 - install the latest bootloader, which will load a module 804 for the file system from which the kernel is loaded automatically 805 806 If you have to load your kernel from a file system which is not of 807 the same type as the root file system, you have to load the necessary 808 file system module manually on the boot prompt or in the boot.cfg file. 809 81020080827: 811 If you built and installed a libc from sources between 812 2008/08/20 and 2008/08/26 you got a broken strtouq(3) 813 which results in false errors reported by lint(1). 814 Since this breaks the libc build itself, manual help is 815 needed -- lint must be disabled temporarily, e.g.: 816 $ (cd lib/libc && make MKLINT=no dependall install) 817 81820080813: 819 MKDEBUG build was broken because the .depend files did not know 820 about .go files. You need to remove all .depend files and rebuild. 821 82220080802: 823 A regression in binary compatibility for pthread_mutex_t has 824 been fixed. Unfortunately, the price is breaking compatibility 825 for -current. 826 827 Threaded programs (using libpthread) and C++ programs (using 828 libstdc++) compiled after 20070907 and before 20080802 need to 829 be recompiled. 830 831 One way to find affected pkgsrc packages: 832 833 $ grep REQUIRES=/usr/lib/libpthread /var/db/pkg/*/+BUILD_INFO 834 $ grep REQUIRES=/usr/lib/libstdc++ /var/db/pkg/*/+BUILD_INFO 835 83620080731: 837 WAPBL (metadata journaling support) has been added, but at this 838 time isn't backwards compatible with pre-WAPBL aware kernels 839 and userland (fsck_ffs in particular). Please make sure you 840 don't use a journaled filesystem with an older kernel/userland, 841 especially an uncleanly mounted journaled filesystem. WAPBL 842 also requires the super block to be in the UFS2 format. You 843 can use fsck_ffs -c 4 to update the superblock format. 844 84520080721: 846 Assembler warnings are now fatal if $WARNS>0 and $NOGCCERROR 847 isn't defined. 848 84920080531: 850 The ioctl number of DRVSUSPENDDEV command on /dev/drvctl changed 851 from 125 (conflicted with DRVCTLCOMMAND) to 129. The drvctl(8) 852 utility needs to be rebuilt and reinstalled as older binaries 853 won't work correctly. The following sequence of commands: 854 855 $ (cd sys/sys/ && nbmake-$arch includes) 856 $ (cd sbin/drvctl/ && nbmake-$arch clean) 857 $ (cd sbin/drvctl/ && nbmake-$arch all) 858 859 leaves new drvctl utility in sbin/drvctl build directory. 860 86120080503: 862 The <bsd.lib.mk> variable MKPRIVATELIB was renamed to LIBISPRIVATE. 863 86420080521: 865 For a while, unprivileged UPDATE builds would fail to 866 succeed at the checkflist stage, complaining that 867 ${DESTDIR}/stand/<arch>/ did not exist. A fix for this 868 problem was committed to share/mk/bsd.kmodule.mk, revision 1.9. 869 If you already hit this problem, update the .mk file, 870 remove ${DESTDIR}/stand/<arch>, and re-run the build. 871 87220080303: 873 Linker warnings are now fatal if $WARNS>0. 874 87520080126: 876 The posix_fadvise system call has been changed from an assembly 877 stub, to a c file that calls an assembly stub. You need to 878 'rm -f posix_fadvise.* .depend' in the libc build directory to 879 avoid using the old assembly stub. 880 88120071209: 882 The acpiec(4) driver has been split into two attachments. If you 883 get ACPI errors before the attachment, please update your kernel 884 configuration file appropriately or see GENERIC for more details. 885 88620071115: 887 The it(4) driver has been renamed to itesio(4) and the old port 888 argument specified in the kernel configuration file is not valid 889 anymore. The itesio(4) driver now uses the Super I/O address port 890 rather than the EC address port. Please update your kernel 891 configuration file appropriately or see GENERIC for more details. 892 89320071028: 894 The pccons(4) driver has been removed from the NetBSD/shark port. 895 You need to update any custom kernel configuration file you have 896 to remove any references to pccons (which includes removing the 897 now useless XSERVER option) and replace them with the correct 898 entries for the wscons driver. See the GENERIC configuration file 899 for more details. 900 90120070913: 902 A latent bug in dhclient/dhcpd that caused it to be unable to 903 enumerate interfaces was fixed. The bug began to cause 904 problems after 20070911 when the kernel's SIOCGIFCONF 905 implementation was repaired. From 20070529 to 20070911 racoon 906 could not enumerate interfaces. (These are noted because 907 normal kernel/userspace version matching hygiene is not 908 sufficient to avoid this problem.) Ensure that both kernel 909 and userland are from after 20070913. 910 91120070703: 912 nbinstall has been renamed ${MACHINE_GNU_ARCH}-install. It 913 calls the target-specific strip program, and the logic to pass 914 down STRIP from make has been removed. This requires a 915 re-installation of tools. 916 91720070422: 918 The way OS emulations lookup filenames inside the emulation root 919 has been changed. Rather than modify the pathname (and copy back 920 to userspace) namei() and lookup() directly check the emulation 921 root. One side effect is that absolute symlinks inside the emulated 922 root file system will be relative to that file system - unless they 923 start /../ this is useful when the emulated root is a real install 924 that has such links. 925 This might affect symlinks that have been added to reference outside 926 the emulated root. 927 92820070412: 929 The pckbc driver on sgimips IP32 has been removed. Use macekbc 930 instead. See the GENERIC32_IP3x kernel configuration for an 931 example. 932 93320070319: 934 src/lib/libc/Makefile revision 1.129 broke libc and ld.elf_so 935 on many platforms due to incorrect flags settings. If you 936 updated and built after about 20070315, do "nbmake-$arch 937 cleandir" in src/lib/libc and src/libexec/ld.elf_so to force a 938 rebuild of object files that might have been built 939 incorrectly, and ensure that you have at least 940 src/lib/libc/Makefile 1.130. 941 94220070210: 943 src/sys/sys/{sa.h,savar.h} were removed. 944 find ${OBJDIR} \( -name .depend -o -name '*.d' \) -print \ 945 | xargs egrep -l '/sa.h|/savar.h' | xargs rm 946 will allow dependencies on those files to get get rebuilt 947 94820070209: 949 The threading model was changed when the newlock2 branch 950 was merged to NetBSD-current. If you boot with a new 951 kernel (version 4.99.10), then you also need a new pthread 952 library (/usr/lib/libpthread.so.0.7). If you boot with 953 an old kernel, then you need the old pthread library 954 (/usr/lib/libpthread.so.0.6). Provided you keep the kernel and 955 the pthread library in sync, old threaded applications should 956 continue to work with an old or new kernel. Note that named(8) 957 is the only threaded application in the base system. 958 95920061214: 960 Following the move of string_to_flags() and flags_to_string() 961 from the bin/ls/ sources to libutil, users doing UPDATE builds 962 will need to do a "make cleandir" in 963 tools/mtree/, tools/makefs/, tools/binstall/, tools/pax/, 964 bin/pax/, bin/ls/, usr.sbin/mtree/, usr.sbin/makefs/, 965 usr.bin/xinstall/, libexec/ftpd/, rescue/, as well 966 as the installation images in distrib/ 967 in order to excise stale references to the old stat_flags.h header 968 file in the ls sources -- stat_flags.h has been removed. 969 97020061108: 971 The configure script used in the src/tools/gcc compiler has been 972 changed to indicate that our libc has ssp support built-in and 973 does not depend on -lssp and -lssp-nonshared. You'll need to 974 make clean in src/tools/gcc first to rebuild the compiler. 975 97620061009: 977 The sysctl variables net.inet{,6}.tcp{,6}.newreno are no longer 978 available. Use net.inet{,6}.tcp{,6}.congctl.selected instead. 979 98020060814: 981 The vt, vidcconsole, kbd, and rpckbd drivers on acorn32 have been 982 withdrawn. Use vidcvideo and pckbd instead. See the GENERIC 983 kernel configuration for an example. X servers from the last 984 few years should cope. 985 98620060703: 987 MPACPI is no more. We always configure PCI interrupts using ACPI 988 if we have an ACPI kernel. The option MPACPI_SCANPCI has been renamed 989 to ACPI_SCANPCI. Thanks to work from fvdl. 990 99120060627: 992 socket(2) has changed, and its system call has been versioned. 993 For userlands with the old version of socket(2), make sure that 994 your kernel has 'options COMPAT_30' set, or else 'bad system call' 995 errors will result. 996 997Hints for a more successful build: 998^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 999 Use build.sh, but do not use its "expert mode": 1000 This will automatically build the tools in the 1001 correct order, and it will keep the tools and the 1002 new build products from interfering with the running 1003 system. This will allow you to ignore most of the 1004 other advice in this file. 1005 Build a new kernel first: 1006 This makes sure that any new system calls or features 1007 expected by the new userland will be present. This 1008 helps to avoid critical errors when upgrading. 1009 Use object directories: 1010 This helps to keep stale object 1011 files from polluting the build if a Makefile "forgets" 1012 about one. It also makes it easier to clean up after 1013 a build. It's also necessary if you want to use the 1014 same source tree for multiple machines. 1015 To use object directories with build.sh: 1016 a) invoke build.sh with the "-M" or "-O" options. 1017 To use object directories without using build.sh: 1018 a) cd /usr/src ; make cleandir 1019 b) Add "OBJMACHINE=yes" to /etc/mk.conf 1020 c) Add "MKOBJDIRS=yes" to /etc/mk.conf 1021 d) cd /usr/src ; make build 1022 Note that running "make obj" in a directory will create 1023 in obj.$MACHINE directory. 1024 Build to a DESTDIR: 1025 This helps to keep old installed files (especially libraries) 1026 from interfering with the new build. 1027 To build to a DESTDIR with build.sh, use the "-D" option. 1028 To build to a DESTDIR without using build.sh, set the DESTDIR 1029 environment variable before running make build. It should be 1030 set to the pathname of an initially empty directory. 1031 Problems: if you do not use build.sh, you might need to 1032 update critical utilities without using DESTDIR since 1033 nothing is executed from what is installed in DESTDIR. 1034 (See critical utils, below.) 1035 Build often: 1036 This keeps critical utilities current enough to not choke 1037 on any other part of the source tree that depends on up to 1038 date functionality. If you use build.sh, you should not have 1039 this problem. 1040 1041What to do if things don't work: 1042^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1043When things don't work there are usually a few things that commonly 1044should be done. 1045 1) make includes 1046 This should be done automatically by make build. 1047 2) cd share/mk && make install 1048 Again, automatically done by make build. 1049 1050Failsafe rebuild of a small part of the tree: 1051^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1052To make sure you rebuild something correctly you want to do 1053something like the following: 1054 1) Make sure the includes and .mk files are up to date. 1055 2) Make sure any program used to build the particular 1056 utility is up to date. (yacc, lex, etc...) 1057 3) cd ...path/to/util... 1058 make cleandir 1059 rm ...all obj directories... 1060 make cleandir # yes, again 1061 make obj 1062 make depend && make 1063 1064Failsafe rebuild of the entire tree: 1065^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1066If you really want to make sure the source tree is clean and 1067ready for a build try the following. Note that sourcing /etc/mk.conf 1068(a make(1) Makefile) in this manner is not right, and will not work 1069for anyone who uses any make(1) features in /etc/mk.conf. 1070 1071---cut here--- 1072#!/bin/sh 1073. /etc/mk.conf 1074 1075if [ -z $NETBSDSRCDIR ] ; then 1076 NETBSDSRCDIR=/usr/src 1077fi 1078if [ \! -d $NETBSDSRCDIR ] ; then 1079 echo Unable to find sources 1080 exit 1 1081fi 1082find $NETBSDSRCDIR -name \*.o -o -name obj.\* -o -name obj -exec rm \{\} \; 1083 1084if [ -z $BSDOBJDIR ] ; then 1085 BSDOBJDIR=/usr/obj 1086fi 1087if [ -d $BSDOBJDIR ] ; then 1088 rm -rf $BSDOBJDIR 1089fi 1090 1091cd $NETBSDSRCDIR && make cleandir 1092 1093---cut here--- 1094 1095Critical utilities: 1096^^^^^^^^^^^^^^^^^^^ 1097 usr.bin/compile_et 1098 usr.bin/make 1099 usr.bin/yacc 1100 usr.bin/lex 1101 usr.bin/xlint 1102 usr.bin/config 1103 1104Other problems and possible solutions: 1105^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1106Symptom:Complaints involving a Makefile. 1107Fix: Rebuild usr.bin/make: 1108 cd usr.bin/make && make && make install 1109 Or, a failsafe method if that doesn't work: 1110 cd usr.bin/make && cc *.c */*.c -I . -o make && mv make /usr/bin 1111 1112Fix: Make sure .mk files are up to date. 1113 cd share/mk && make install 1114 1115Symptom:Kernel `config' fails to configure any kernel, including GENERIC. 1116Fix: Rebuild usr.bin/config 1117 1118Symptom: 1119Fix: Rebuild usr.bin/yacc 1120 1121Symptom: 1122Fix: Rebuild usr.bin/lex 1123 1124Symptom: 1125Fix: rm /usr/lib/libbfd.a 1126 1127Symptom:Obsolete intermediate files are used during compilation 1128Fix: Try the following sequence of commands in the directory in question. 1129 make cleandir; rm `make print-objdir`; make cleandir; make obj 1130 (If you built the tree without "make obj" in the past, obsolete files 1131 may remain. The command tries to clean everything up) 1132 1133Symptom:.../sysinst/run.c:xx: warning: initialization from incompatible 1134 pointer type 1135Fix: Rebuild and install usr.bin/menuc 1136 1137Symptom:mklocale not found during build in share/locale/ctype 1138Fix: Build and install usr.bin/mklocale 1139 1140Symptom:undefined reference to `__assert13' or `__unsetenv13' 1141Fix: Rebuild and install lib/libc 1142 1143Symptom:usr.bin/config fails to build. 1144Fix: Try building with -DMAKE_BOOTSTRAP added to CFLAGS in Makefile. 1145 1146Symptom:undefined reference to `getprogname' or `setprogname' 1147Fix: Rebuild and install lib/libc 1148 1149Symptom:lint does not understand the '-X' option 1150Fix: May need to build & install libs with NOLINT=1 before rebuilding lint 1151 1152Symptom:Update build fails in src/tools/gcc complaining that a variable 1153 (e.g. CPPFLAGS) has changed since the previous run. 1154Fix: Run "nbmake-${ARCH} clean" in src/tools/gcc or do a clean build. 1155 1156Symptom:cvs [update aborted]: cannot open directory /cvsroot/src/...: 1157 No such file or directory. 1158Cause: If a directory is created by mistake, then it is sometimes 1159 deleted from the CVS repository using administrative commands 1160 that bypass the normal cvs access controls. If your cvs working tree 1161 contains references to a directory that has been deleted on the 1162 server in this way, then "cvs update" reports this error. 1163Fix: Recursively delete the affected directory from your working tree 1164 and try the update again. 1165