1mk-files 2******** 3 4The term ``mk-files`` refers to a collection of ``*.mk`` files. 5 6You need bmake_ or a *recent* NetBSD_ make. 7If in doubt use bmake_. 8 9Introduction 10============ 11 12Many years ago, when building large software projects, I used GNU make 13(or my own patched version of it), and had developed a set of macros 14to simplify developing complex build trees. 15 16Since the early 90's my main development machines, run BSD 17(NetBSD_ to be precise), and the BSD source tree is good example of a 18large software project. It quickly became clear that 19``/usr/share/mk/*.mk`` were a great model, but were quite tightly 20linked to building the BSD tree. 21 22Much as I liked using NetBSD, my customers were more likely to be 23using SunOS, HP-UX etc, so I started on bmake_ and a portable collection 24of mk-files (mk.tar.gz_). NetBSD provided much of the original structure. 25 26Since then I've added a lot of features to NetBSD's make and hence to 27bmake which is kept closely in sync. The mk-files however have 28diverged quite a bit, though ideas are still picked up from NetBSD 29and FreeBSD. 30 31Basics 32------ 33 34The BSD build model is very simple. A directory produces one 35component, which is generally either a library or a program. 36Library makefiles include ``lib.mk`` and programs include ``prog.mk`` 37and they *do the right thing*. 38 39A simple library makefile might look like:: 40 41 LIB = sig 42 43 SRCS = \ 44 sigaction.c \ 45 sigcompat.c \ 46 sighdl.c 47 48 .include <lib.mk> 49 50a simple program makefile:: 51 52 PROG = cat 53 54 SRCS = cat.c 55 56 .include <prog.mk> 57 58in such cases even the ``SRCS`` line is unnecessary as ``prog.mk`` 59will default it to ``${PROG}.c``. 60 61It is the sensible use of defaults and the plethora of macro modifiers 62provided by bmake_ that allow simple makefiles such as the above 63*just work* on many different systems. 64 65 66mk-files 67======== 68 69This section provides a brief description of some of the ``*.mk`` 70files. 71 72sys.mk 73------ 74 75When bmake starts, it looks for ``sys.mk`` and reads it before doing 76anything else. Thus, this is the place to setup the environment for 77everyone else. 78 79In this distribution, sys.mk avoids doing anything platform dependent. 80It is quite short, and includes a number of other files (which may or 81may not exists) 82 83sys.env.mk 84 If it exists, is expected to do things like conditioning the 85 environment. Since it will only be included by the initial 86 instance of bmake, it should ``.export`` anything that 87 sub-makes might need. 88 89examples/sys.clean-env.mk 90 An example of how to clean the environment. 91 See the file for all the details:: 92 93 .if ${MAKE_VERSION} >= 20100606 && ${.MAKE.LEVEL} == 0 94 # we save any env var that starts with these 95 MAKE_SAVE_ENV_PREFIX += SB MK MAKE MACHINE NEED_ CCACHE DISTCC USE_ SSH 96 MAKE_SAVE_ENV_VARS += \ 97 PATH HOME USER LOGNAME \ 98 SRCTOP OBJTOP OBJROOT \ 99 ${_env_vars} 100 101 _env_vars != env | egrep '^(${MAKE_SAVE_ENV_PREFIX:ts|})' | sed 's,=.*,,'; echo 102 _export_list = 103 .for v in ${MAKE_SAVE_ENV_VARS:O:u} 104 .if !empty($v) 105 _export_list += $v 106 $v := ${$v} 107 .endif 108 .endfor 109 # now clobber the environment 110 .unexport-env 111 112 # list of vars that we handle specially below 113 _tricky_env_vars = MAKEOBJDIR 114 # export our selection - sans tricky ones 115 .export ${_export_list:${_tricky_env_vars:${M_ListToSkip}}} 116 117 # this next bit may need tweaking 118 .if defined(MAKEOBJDIR) 119 srctop := ${SRCTOP:U${SB_SRC:U${SB}/src}} 120 objroot := ${OBJROOT:U${SB_OBJROOT:U${SB}/${SB_OBJPREFIX}}} 121 # we'll take care of MACHINE below 122 objtop := ${OBJTOP:U${objroot}${MACHINE}} 123 .if !empty(objtop) 124 # we would normally want something like (/bin/sh): 125 # MAKEOBJDIR="\${.CURDIR:S,${SRCTOP},${OBJROOT}\${MACHINE},}" 126 # the $$ below is how we achieve the same result here. 127 # since everything saved from the environment above 128 # has run through := we need to compensate for ${MACHINE} 129 MAKEOBJDIR = $${.CURDIR:S,${srctop},${objtop:S,${MACHINE},\${MACHINE},},} 130 131 # export these as-is, and do not track... 132 .export-env ${_tricky_env_vars} 133 # now evaluate for ourselves 134 .for v in ${_tricky_env_vars} 135 $v := ${$v} 136 .endfor 137 138 .endif 139 .endif 140 .endif 141 142 143host-target.mk 144 Is used to set macros like ``HOST_TARGET``, ``HOST_OS`` and 145 ``host_os`` which are used to find the next step. 146 147sys/\*.mk 148 Platform specific additions, such as ``Darwin.mk`` or ``SunOS.mk`` 149 set things like ``HOST_LIBEXT = .dylib`` for Darwin or 150 ``SHLIB_FULLVERSION = ${SHLIB_MAJOR}`` for SunOS 5. 151 If there is no OS specific file, ``sys/Generic.mk`` is used. 152 153local.sys.mk 154 Any ``local.*.mk`` file is not part of the distribution. 155 This provides a hook for sites to do extra setup without 156 having to edit the distributed files. 157 158 159The above arrangement makes it easy for the mk files to be part of a 160src tree on an NFS volume and to allow building on multiple platforms. 161 162lib.mk 163------ 164 165This file is used to build a number of different libraries from the 166same SRCS. 167 168lib${LIB}.a 169 An archive lib of ``.o`` files, this is the default 170 171lib${LIB}_p.a 172 A profiled lib of ``.po`` files. 173 Still an archive lib, but all the objects are built with 174 profiling in mind - hence the different extension. 175 It is skipped if ``MKPROFILE`` is "no". 176 177lib${LIB}_pic.a 178 An archive of ``.so`` objects compiled for relocation. 179 On NetBSD this is the input to ``lib${LIB}.${LD_so}``, it is 180 skipped if ``MKPICLIB`` is "no". 181 182lib${LIB}.${LD_so} 183 A shared library. The value of ``LD_so`` is very platform 184 specific. For example:: 185 186 # SunOS 5 and most other ELF systems 187 libsslfd.so.1 188 189 # Darwin 190 libsslfd.1.dylib 191 192 This library will only be built if ``SHLIB_MAJOR`` has 193 a value, and ``MKPIC`` is not set to "no". 194 195There is a lot of platform specific tweaking in ``lib.mk``, largely the 196result of the original distributions trying to avoid interfering with 197the system's ``sys.mk``. 198 199libnames.mk 200----------- 201 202This is included by both ``prog.mk`` and ``lib.mk`` and tries to 203include ``*.libnames.mk`` of which: 204 205local.libnames.mk 206 does not exist unless you create it. It is a handy way for you 207 to customize without touching the distributed files. 208 For example, on a test machine I needed to build openssl but 209 not install it, so put the following in ``local.libnames.mk``:: 210 211 .if ${host_os} == "sunos" 212 LIBCRYPTO = ${OBJTOP}/openssl/lib/crypto/libcrypto${DLIBEXT} 213 LIBSSL = ${OBJTOP}/openssl/lib/ssl/libssl${DLIBEXT} 214 INCLUDES_libcrypto = -I${OBJ_libcrypto} 215 .endif 216 217 The makefile created an openssl dir in ``${OBJ_libcrypto}`` to 218 gather all the headers. dpadd.mk_ did the rest. 219 220sjg.libnames.mk 221 not part of the mk-files distribution. 222 223host.libnames.mk 224 contains logic to find any libs named in ``HOST_LIBS`` in 225 ``HOST_LIBDIRS``. 226 227Each file above gets an opportunity to define things like:: 228 229 LIBSSLFD ?= ${OBJTOP}/ssl/lib/sslfd/libsslfd${DLIBEXT} 230 INCLUDES_libsslfd = -I${SRC_libsslfd}/h -I${OBJ_libslfd} 231 232these are used by dpadd.mk_ and will be explained below. 233 234dpadd.mk 235-------- 236 237This file looks like line noise, and is best considered read-only. 238However it provides some very useful functionality, which simplifies the build. 239 240Makefiles can use the LIB* macros defined via libnames.mk_ or anywhere 241else in various ways:: 242 243 # indicate that we need to include headers from LIBCRYPTO 244 # this would result in ${INCLUDES_libcrypto} being added to CFLAGS. 245 SRC_LIBS += ${LIBCRYPTO} 246 247 # indicate that libsslfd must be built already. 248 # it also has the same effect as SRC_LIBS 249 DPADD += ${LIBSSLFD} 250 251 # indicate that not only must libsslfd be built, 252 # but that we need to link with it. 253 # this is almost exactly equivalent to 254 # DPADD += ${LIBSSLFD} 255 # LDADD += -L${LIBSSLFD:H} -lsslfd 256 # and mostly serves to ensure that DPADD and LDADD are in sync. 257 DPLIBS += ${LIBSSLFD} 258 259Any library (referenced by its full path) in any of the above, is 260added to ``DPMAGIC_LIBS`` with the following results, for each lib *foo*. 261 262SRC_libfoo 263 Is set to indicate where the src for libfoo is. 264 By default it is derived from ``LIBFOO`` by replacing 265 ``${OBJTOP}`` with ``${SRCTOP}``. 266 267OBJ_libfoo 268 Not very exciting, is just the dir where libfoo lives. 269 270INCLUDES_libfoo 271 What to add to ``CFLAGS`` to find the public headers. 272 The default varies. If ``${SRC_libfoo}/h`` exists, it is assumed 273 to be the home of all public headers and thus the default is 274 ``-I${SRC_libfoo}/h`` 275 276 Otherwise we make no assumptions and the default is 277 ``-I${SRC_libfoo} -I${OBJ_libfoo}`` 278 279LDADD_libfoo 280 This only applies to libs reference via ``DPLIBS``. 281 The default is ``-lfoo``, ``LDADD_*`` provides a hook to 282 instantiate other linker flags at the appropriate point 283 without losing the benfits of ``DPLIBS``. 284 285prog.mk 286------- 287 288Compiles the specified SRCS and links them and the nominated libraries 289into a program. Prog makefiles usually need to list the libraries 290that need to be linked. We prefer use of ``DPLIBS`` but the more 291traditional ``DPADD`` and ``LDADD`` work just as well. 292That is:: 293 294 DPLIBS += ${LIBCRYPTO} 295 296is equivalent to:: 297 298 DPADD += ${LIBCRYPTO} 299 LDADD += -lprivate_crypto 300 301obj.mk 302------ 303 304One of the cool aspects of BSD make, is its support for separating 305object files from the src tree. This is also the source of much 306confusion to some. 307 308Traditionally one had to do a separate ``make obj`` pass through the 309tree. If ``MKOBJDIRS`` is "auto", we include auto.obj.mk_. 310 311auto.obj.mk 312----------- 313 314This leverages the ``.OBJDIR`` target introduced some years ago to 315NetBSD make, to automatically create the desired object dir. 316 317subdir.mk 318--------- 319 320This is the traditional means of walking the tree. A makefile sets 321``SUBDIR`` to the list of sub-dirs to visit. 322 323If ``SUBDIR_MUST_EXIST`` is set, missing directories cause an error, 324otherwise a warning is issued. If you don't even want the warning, 325set ``MISSING_DIR=continue``. 326 327Traditionally, ``subdir.mk`` prints clue as it visits each subdir:: 328 329 ===> ssl 330 ===> ssl/lib 331 ===> ssl/lib/sslfd 332 333you can suppress that - or enhance it by setting ``ECHO_DIR``:: 334 335 # suppress subdir noise 336 ECHO_DIR=: 337 # print time stamps 338 ECHO_DIR=echo @ `date "+%s [%Y-%m-%d %T] "` 339 340links.mk 341-------- 342 343Provides rules for processing lists of ``LINKS`` and ``SYMLINKS``. 344Each is expected to be a list of ``link`` and ``target`` pairs 345(``link`` -> ``target``). 346 347The logic is generally in a ``_*_SCRIPT`` which is referenced in a 348``_*_USE`` (``.USE``) target. 349 350The ``_BUILD_*`` forms are identical, but do not use ``${DESTDIR}`` 351and so are useful for creating symlinks during the build phase. 352For example:: 353 354 SYMLINKS += ${.CURDIR}/${MACHINE_ARCH}/include machine 355 header_links: _BUILD_SYMLINKS_USE 356 357 md.o: header_links 358 359would create a symlink called ``machine`` in ``${.OBJDIR}`` pointing to 360``${.CURDIR}/${MACHINE_ARCH}/include`` before compiling ``md.o`` 361 362 363autoconf.mk 364----------- 365 366Deals with running (or generating) GNU autoconf ``configure`` scripts. 367 368dep.mk 369------ 370 371Deals with collecting dependencies. Another useful feature of BSD 372make is the separation of this sort of information into a ``.depend`` 373file. ``MKDEP`` needs to point to a suitable tool (like mkdeps.sh_) 374 375If ``USE_AUTODEP_MK`` is "yes" includes autodep.mk_ 376 377autodep.mk 378---------- 379 380Leverages the ``-MD`` feature of recent GCC to collect dependency 381information as a side effect of compilation. With this GCC puts 382dependency info into a ``.d`` file. 383 384Unfortunately GCC bases the name of the ``.d`` file on the name of the 385input rather than the output file, which causes problems when the same 386source is compiled different ways. The latest GCC supports ``-MF`` to 387name the ``.d`` file and ``-MT`` to control the name to put as the 388dependent. 389 390Recent bmake allows dependencies for the ``.END`` target (run at the 391end if everything was successful), and ``autodep.mk`` uses this to 392post process the ``.d`` files into ``.depend``. 393 394auto.dep.mk 395----------- 396 397A much simpler implementation than autodep.mk_ it uses 398``-MF ${.TARGET:T}.d`` 399to avoid possible conflicts during parallel builds. 400This precludes the use of suffix rules to drive ``make depend``, so 401dep.mk_ handles that if specifically requested. 402 403options.mk 404---------- 405 406Inspired by FreeBSD's ``bsd.own.mk`` more flexible. 407FreeBSD now have similar functionality in ``bsd.mkopt.mk``. 408 409It allows users to express their intent with respect to options 410``MK_*`` by setting ``WITH_*`` or ``WITHOUT_*``. 411 412Note: ``WITHOUT_*`` wins if both are set, and makefiles can set 413``NO_*`` to say they cannot handle that option, or even ``MK_*`` if 414they really need to. 415 416 417own.mk 418------ 419 420Normally included by ``init.mk`` (included by ``lib.mk`` and 421``prog.mk`` etc), sets macros for default ownership etc. 422 423It includes ``${MAKECONF}`` if it is defined and exists. 424 425ldorder.mk 426---------- 427 428Leverages ``bmake`` to compute optimal link order for libraries. 429This works nicely and makes refactoring a breeze - so long as you 430have not (or few) cicular dependencies between libraries. 431 432man.mk 433------ 434 435Deals with man pages. 436 437warnings.mk 438----------- 439 440This provides a means of fine grained control over warnings on a per 441``${MACHINE}`` or even file basis. 442 443A makefile sets ``WARNINGS_SET`` to name a list of warnings 444and individual ``W_*`` macros can be used to tweak them. 445For example:: 446 447 WARNINGS_SET = HIGH 448 W_unused_sparc = -Wno-unused 449 450would add all the warnings in ``${HIGH_WARNINGS}`` to CFLAGS, but 451on sparc, ``-Wno-unused`` would replace ``-Wunused``. 452 453You should never need to edit ``warnings.mk``, it will include 454``warnings-sets.mk`` if it exists and you use that to make any local 455customizations. 456 457rst2htm.mk 458---------- 459 460Logic to simplify generating HTML (and PDF) documents from ReStructuredText. 461 462cython.mk 463--------- 464 465Logic to build Python C interface modules using Cython_ 466 467.. _Cython: http://www.cython.org/ 468 469Meta mode 470========= 471 472The 20110505 and later versions of ``mk-files`` include a number of 473makefiles contributed by Juniper Networks, Inc. 474These allow the latest version of bmake_ to run in `meta mode`_ 475see `dirdeps.mk`_ 476 477.. _`dirdeps.mk`: /help/sjg/dirdeps.htm 478.. _`meta mode`: bmake-meta-mode.htm 479 480Install 481======= 482 483You can use the content of mk.tar.gz_ without installing at all. 484 485The script ``install-mk`` takes care of copying ``*.mk`` into a 486destination directory, and unless told not to, create ``bsd.*.mk`` links 487for ``lib.mk`` etc. 488 489If you just want to create the ``bsd.*.mk`` links in the directory 490where you unpacked the tar file, you can:: 491 492 ./mk/install-mk ./mk 493 494------ 495 496.. _bmake: bmake.htm 497.. _NetBSD: http://www.netbsd.org/ 498.. _mkdeps.sh: http://www.crufty.net/ftp/pub/sjg/mkdeps.sh 499.. _mk.tar.gz: http://www.crufty.net/ftp/pub/sjg/mk.tar.gz 500 501:Author: sjg@crufty.net 502:Revision: $Id: mk-files.txt,v 1.18 2018/12/08 07:27:15 sjg Exp $ 503:Copyright: Crufty.NET 504