1mk-files 2******** 3 4The term ``mk-files`` refers to a collection of ``*.mk`` files. 5 6You need bmake_ or a *recent* NetBSD_ make. 7If in doubt use bmake_. 8 9Introduction 10============ 11 12Many years ago, when building large software projects, I used GNU make 13(or my own patched version of it), and had developed a set of macros 14to simplify developing complex build trees. 15 16Since the early 90's my main development machines, run BSD 17(NetBSD_ to be precise, and more recently FreeBSD), and the BSD source 18tree is good example of a large software project. 19It quickly became clear that ``/usr/share/mk/*.mk`` were a great 20model, but at the time were quite tightly linked to building the BSD tree. 21 22Much as I liked using NetBSD, my customers were more likely to be 23using SunOS, HP-UX etc, so I started on bmake_ and a portable collection 24of mk-files (mk.tar.gz_). NetBSD provided much of the original structure. 25 26Since then I've added a lot of features to NetBSD's make and hence to 27bmake which is kept closely in sync. The mk-files however have 28diverged quite a bit, though ideas are still picked up from NetBSD 29and FreeBSD. 30 31Basics 32------ 33 34The BSD build model is very simple. A directory produces one 35component, which is generally either a library or a program. 36Library makefiles include ``lib.mk`` and programs include ``prog.mk`` 37and they *do the right thing*. 38 39A simple library makefile might look like:: 40 41 LIB = sig 42 43 SRCS = \ 44 sigaction.c \ 45 sigcompat.c \ 46 sighdl.c 47 48 .include <lib.mk> 49 50a simple program makefile:: 51 52 PROG = cat 53 54 SRCS = cat.c 55 56 .include <prog.mk> 57 58in such cases even the ``SRCS`` line is unnecessary as ``prog.mk`` 59will default it to ``${PROG}.c``. 60 61It is the sensible use of defaults and the plethora of macro modifiers 62provided by bmake_ that allow simple makefiles such as the above to 63*just work* on many different systems. 64 65 66mk-files 67======== 68 69This section provides a brief description of some of the ``*.mk`` 70files. 71 72sys.mk 73------ 74 75When bmake starts, it looks for ``sys.mk`` and reads it before doing 76anything else. Thus, this is the place to setup the environment for 77everyone else. 78 79In this distribution, ``sys.mk`` avoids doing anything platform dependent. 80It is quite short, and includes a number of other files (which may or 81may not exists) 82 83sys.env.mk 84 If it exists, is expected to do things like conditioning the 85 environment. Since it will only be included by the initial 86 instance of bmake, it should ``.export`` anything that 87 sub-makes might need. 88 89examples/sys.clean-env.mk 90 An example of how to clean the environment. 91 See the file for all the details:: 92 93 .if ${MAKE_VERSION} >= 20100606 && ${.MAKE.LEVEL} == 0 94 # we save any env var that starts with these 95 MAKE_SAVE_ENV_PREFIX += SB MK MAKE MACHINE NEED_ CCACHE DISTCC USE_ SSH 96 MAKE_SAVE_ENV_VARS += \ 97 PATH HOME USER LOGNAME \ 98 SRCTOP OBJTOP OBJROOT \ 99 ${_env_vars} 100 101 _env_vars != env | egrep '^(${MAKE_SAVE_ENV_PREFIX:ts|})' | sed 's,=.*,,'; echo 102 _export_list = 103 .for v in ${MAKE_SAVE_ENV_VARS:O:u} 104 .if !empty($v) 105 _export_list += $v 106 $v := ${$v} 107 .endif 108 .endfor 109 # now clobber the environment 110 .unexport-env 111 112 # list of vars that we handle specially below 113 _tricky_env_vars = MAKEOBJDIR 114 # export our selection - sans tricky ones 115 .export ${_export_list:${_tricky_env_vars:${M_ListToSkip}}} 116 117 # this next bit may need tweaking 118 .if defined(MAKEOBJDIR) 119 srctop := ${SRCTOP:U${SB_SRC:U${SB}/src}} 120 objroot := ${OBJROOT:U${SB_OBJROOT:U${SB}/${SB_OBJPREFIX}}} 121 # we'll take care of MACHINE below 122 objtop := ${OBJTOP:U${objroot}${MACHINE}} 123 .if !empty(objtop) 124 # we would normally want something like (/bin/sh): 125 # MAKEOBJDIR="\${.CURDIR:S,${SRCTOP},${OBJROOT}\${MACHINE},}" 126 # the $$ below is how we achieve the same result here. 127 # since everything saved from the environment above 128 # has run through := we need to compensate for ${MACHINE} 129 MAKEOBJDIR = $${.CURDIR:S,${srctop},${objtop:S,${MACHINE},\${MACHINE},},} 130 131 # export these as-is, and do not track... 132 .export-env ${_tricky_env_vars} 133 # now evaluate for ourselves 134 .for v in ${_tricky_env_vars} 135 $v := ${$v} 136 .endfor 137 138 .endif 139 .endif 140 .endif 141 142 143host-target.mk 144 Is used to set macros like ``HOST_TARGET``, ``HOST_OS`` and 145 ``host_os`` which are used to find the next step. 146 147sys/\*.mk 148 Platform specific additions, such as ``Darwin.mk`` or ``SunOS.mk`` 149 set things like ``HOST_LIBEXT = .dylib`` for Darwin or 150 ``SHLIB_FULLVERSION = ${SHLIB_MAJOR}`` for SunOS 5. 151 If there is no OS specific file, ``sys/Generic.mk`` is used. 152 153local.sys.mk 154 Any ``local.*.mk`` file is not part of the distribution. 155 This provides a hook for sites to do extra setup without 156 having to edit the distributed files. 157 158 159The above arrangement makes it easy for the mk files to be part of a 160src tree on an NFS volume and to allow building on multiple platforms. 161 162lib.mk 163------ 164 165This file is used to build a number of different libraries from the 166same SRCS. 167 168lib${LIB}.a 169 An archive lib of ``.o`` files, this is the default 170 171lib${LIB}_p.a 172 A profiled lib of ``.po`` files. 173 Still an archive lib, but all the objects are built with 174 profiling in mind - hence the different extension. 175 It is skipped if ``MKPROFILE`` is "no". 176 177lib${LIB}_pic.a 178 An archive of ``.so`` objects compiled for relocation. 179 On NetBSD this is the input to ``lib${LIB}.${LD_so}``, it is 180 skipped if ``MKPICLIB`` is "no". 181 182lib${LIB}.${LD_so} 183 A shared library. The value of ``LD_so`` is very platform 184 specific. For example:: 185 186 # SunOS 5 and most other ELF systems 187 libsslfd.so.1 188 189 # Darwin 190 libsslfd.1.dylib 191 192 This library will only be built if ``SHLIB_MAJOR`` has 193 a value, and ``MKPIC`` is not set to "no". 194 195There is a lot of platform specific tweaking in ``lib.mk``, largely the 196result of the original distributions trying to avoid interfering with 197the system's ``sys.mk``. 198 199libnames.mk 200----------- 201 202This is included by both ``prog.mk`` and ``lib.mk`` and tries to 203include ``*.libnames.mk`` of which: 204 205local.libnames.mk 206 does not exist unless you create it. It is a handy way for you 207 to customize without touching the distributed files. 208 For example, on a test machine I needed to build openssl but 209 not install it, so put the following in ``local.libnames.mk``:: 210 211 .if ${host_os} == "sunos" 212 LIBCRYPTO = ${OBJTOP}/openssl/lib/crypto/libcrypto${DLIBEXT} 213 LIBSSL = ${OBJTOP}/openssl/lib/ssl/libssl${DLIBEXT} 214 INCLUDES_libcrypto = -I${OBJ_libcrypto} 215 .endif 216 217 The makefile created an openssl dir in ``${OBJ_libcrypto}`` to 218 gather all the headers. dpadd.mk_ did the rest. 219 220host.libnames.mk 221 contains logic to find any libs named in ``HOST_LIBS`` in 222 ``HOST_LIBDIRS``. 223 224Each file above gets an opportunity to define things like:: 225 226 LIBSSLFD ?= ${OBJTOP}/ssl/lib/sslfd/libsslfd${DLIBEXT} 227 INCLUDES_libsslfd = -I${SRC_libsslfd}/h -I${OBJ_libslfd} 228 229these are used by dpadd.mk_ and will be explained below. 230 231dpadd.mk 232-------- 233 234This file looks like line noise, and is best considered read-only. 235However it provides some very useful functionality, which simplifies the build. 236 237Makefiles can use the LIB* macros defined via libnames.mk_ or anywhere 238else in various ways:: 239 240 # indicate that we need to include headers from LIBCRYPTO 241 # this would result in ${INCLUDES_libcrypto} being added to CFLAGS. 242 SRC_LIBS += ${LIBCRYPTO} 243 244 # indicate that libsslfd must be built already. 245 # it also has the same effect as SRC_LIBS 246 DPADD += ${LIBSSLFD} 247 248 # indicate that not only must libsslfd be built, 249 # but that we need to link with it. 250 # this is almost exactly equivalent to 251 # DPADD += ${LIBSSLFD} 252 # LDADD += -L${LIBSSLFD:H} -lsslfd 253 # and mostly serves to ensure that DPADD and LDADD are in sync. 254 DPLIBS += ${LIBSSLFD} 255 256Any library (referenced by its full path) in any of the above, is 257added to ``DPMAGIC_LIBS`` with the following results, for each lib *foo*. 258 259SRC_libfoo 260 Is set to indicate where the src for libfoo is. 261 By default it is derived from ``LIBFOO`` by replacing 262 ``${OBJTOP}`` with ``${SRCTOP}``. 263 264OBJ_libfoo 265 Not very exciting, is just the dir where libfoo lives. 266 267INCLUDES_libfoo 268 What to add to ``CFLAGS`` to find the public headers. 269 The default varies. If ``${SRC_libfoo}/h`` exists, it is assumed 270 to be the home of all public headers and thus the default is 271 ``-I${SRC_libfoo}/h`` 272 273 Otherwise we make no assumptions and the default is 274 ``-I${SRC_libfoo} -I${OBJ_libfoo}`` 275 276LDADD_libfoo 277 This only applies to libs reference via ``DPLIBS``. 278 The default is ``-lfoo``, ``LDADD_*`` provides a hook to 279 instantiate other linker flags at the appropriate point 280 without losing the benfits of ``DPLIBS``. 281 282prog.mk 283------- 284 285Compiles the specified SRCS and links them and the nominated libraries 286into a program. Prog makefiles usually need to list the libraries 287that need to be linked. We prefer use of ``DPLIBS`` but the more 288traditional ``DPADD`` and ``LDADD`` work just as well. 289That is:: 290 291 DPLIBS += ${LIBCRYPTO} 292 293is equivalent to:: 294 295 DPADD += ${LIBCRYPTO} 296 LDADD += -lprivate_crypto 297 298obj.mk 299------ 300 301One of the cool aspects of BSD make, is its support for separating 302object files from the src tree. This is also the source of much 303confusion to some. 304 305Traditionally one had to do a separate ``make obj`` pass through the 306tree. If ``MKOBJDIRS`` is "auto", we include auto.obj.mk_. 307 308auto.obj.mk 309----------- 310 311This leverages the ``.OBJDIR`` target introduced some years ago to 312NetBSD make, to automatically create the desired object dir. 313 314subdir.mk 315--------- 316 317This is the traditional means of walking the tree. A makefile sets 318``SUBDIR`` to the list of sub-dirs to visit. 319 320If ``SUBDIR_MUST_EXIST`` is set, missing directories cause an error, 321otherwise a warning is issued. If you don't even want the warning, 322set ``MISSING_DIR=continue``. 323 324Traditionally, ``subdir.mk`` prints clues as it visits each subdir:: 325 326 ===> ssl 327 ===> ssl/lib 328 ===> ssl/lib/sslfd 329 330you can suppress that - or enhance it by setting ``ECHO_DIR``:: 331 332 # suppress subdir noise 333 ECHO_DIR=: 334 # print time stamps 335 ECHO_DIR=echo @ `date "+%s [%Y-%m-%d %T] "` 336 337links.mk 338-------- 339 340Provides rules for processing lists of ``LINKS`` and ``SYMLINKS``. 341Each is expected to be a list of ``link`` and ``target`` pairs 342(``link`` -> ``target``). 343 344The logic is generally in a ``_*_SCRIPT`` which is referenced in a 345``_*_USE`` (``.USE``) target. 346 347The ``_BUILD_*`` forms are identical, but do not use ``${DESTDIR}`` 348and so are useful for creating symlinks during the build phase. 349For example:: 350 351 SYMLINKS += ${.CURDIR}/${MACHINE_ARCH}/include machine 352 header_links: _BUILD_SYMLINKS_USE 353 354 md.o: header_links 355 356would create a symlink called ``machine`` in ``${.OBJDIR}`` pointing to 357``${.CURDIR}/${MACHINE_ARCH}/include`` before compiling ``md.o`` 358 359 360autoconf.mk 361----------- 362 363Deals with running (or generating) GNU autoconf ``configure`` scripts. 364 365dep.mk 366------ 367 368Deals with collecting dependencies. Another useful feature of BSD 369make is the separation of this sort of information into a ``.depend`` 370file. ``MKDEP`` needs to point to a suitable tool (like mkdeps.sh_) 371 372If ``USE_AUTODEP_MK`` is "yes" includes autodep.mk_ 373 374autodep.mk 375---------- 376 377Leverages the ``-MD`` feature of recent GCC to collect dependency 378information as a side effect of compilation. With this GCC puts 379dependency info into a ``.d`` file. 380 381Unfortunately GCC bases the name of the ``.d`` file on the name of the 382input rather than the output file, which causes problems when the same 383source is compiled different ways. The latest GCC supports ``-MF`` to 384name the ``.d`` file and ``-MT`` to control the name to put as the 385dependent. 386 387Recent bmake allows dependencies for the ``.END`` target (run at the 388end if everything was successful), and ``autodep.mk`` uses this to 389post process the ``.d`` files into ``.depend``. 390 391auto.dep.mk 392----------- 393 394A much simpler implementation than autodep.mk_ it uses 395``-MF ${.TARGET:T}.d`` 396to avoid possible conflicts during parallel builds. 397This precludes the use of suffix rules to drive ``make depend``, so 398dep.mk_ handles that if specifically requested. 399 400options.mk 401---------- 402 403Inspired by FreeBSD's ``bsd.own.mk`` more flexible. 404FreeBSD now have similar functionality in ``bsd.mkopt.mk``. 405 406It allows users to express their intent with respect to options 407``MK_*`` by setting ``WITH_*`` or ``WITHOUT_*``. 408 409Note: ``WITHOUT_*`` wins if both are set, and makefiles can set 410``NO_*`` to say they cannot handle that option, or even ``MK_*`` if 411they really need to. 412 413 414own.mk 415------ 416 417Normally included by ``init.mk`` (included by ``lib.mk`` and 418``prog.mk`` etc), sets macros for default ownership etc. 419 420It includes ``${MAKECONF}`` if it is defined and exists. 421 422ldorder.mk 423---------- 424 425Leverages ``bmake`` to compute optimal link order for libraries. 426This works nicely and makes refactoring a breeze - so long as you 427have not (or few) cicular dependencies between libraries. 428 429man.mk 430------ 431 432Deals with man pages. 433 434warnings.mk 435----------- 436 437This provides a means of fine grained control over warnings on a per 438``${MACHINE}`` or even file basis. 439 440A makefile sets ``WARNINGS_SET`` to name a list of warnings 441and individual ``W_*`` macros can be used to tweak them. 442For example:: 443 444 WARNINGS_SET = HIGH 445 W_unused_sparc = -Wno-unused 446 447would add all the warnings in ``${HIGH_WARNINGS}`` to CFLAGS, but 448on sparc, ``-Wno-unused`` would replace ``-Wunused``. 449 450You should never need to edit ``warnings.mk``, it will include 451``warnings-sets.mk`` if it exists and you use that to make any local 452customizations. 453 454rst2htm.mk 455---------- 456 457Logic to simplify generating HTML (and PDF) documents from ReStructuredText. 458 459cython.mk 460--------- 461 462Logic to build Python C interface modules using Cython_ 463 464.. _Cython: http://www.cython.org/ 465 466Meta mode 467========= 468 469The 20110505 and later versions of ``mk-files`` include a number of 470makefiles contributed by Juniper Networks, Inc. 471These allow the latest version of bmake_ to run in `meta mode`_ 472see `dirdeps.mk`_ 473 474.. _`dirdeps.mk`: /help/sjg/dirdeps.htm 475.. _`meta mode`: bmake-meta-mode.htm 476 477Install 478======= 479 480You can use the content of mk.tar.gz_ without installing at all. 481 482The script ``install-mk`` takes care of copying ``*.mk`` into a 483destination directory, and unless told not to, create ``bsd.*.mk`` links 484for ``lib.mk`` etc. 485 486If you just want to create the ``bsd.*.mk`` links in the directory 487where you unpacked the tar file, you can:: 488 489 ./mk/install-mk ./mk 490 491------ 492 493.. _bmake: bmake.htm 494.. _NetBSD: http://www.netbsd.org/ 495.. _mkdeps.sh: http://www.crufty.net/ftp/pub/sjg/mkdeps.sh 496.. _mk.tar.gz: http://www.crufty.net/ftp/pub/sjg/mk.tar.gz 497 498:Author: sjg@crufty.net 499:Revision: $Id: mk-files.txt,v 1.20 2020/08/19 17:51:53 sjg Exp $ 500:Copyright: Crufty.NET 501