Add integrated dports build feature to build-all and install-all targets

When WITH_DPORTS=YES is used build-all and install-all, these targets will
also build the base set of ports and all of their

Add integrated dports build feature to build-all and install-all targets

When WITH_DPORTS=YES is used build-all and install-all, these targets will
also build the base set of ports and all of their dependencies using
configuration data and the "dragonfly/base" dport found in
"/usr/src/dports.base".

This feature is intended to be used by the nrelease build in an upcoming
commit, and may also be used manually.

The /usr/dports tree must exist and contains the dports repo to use as the
basis of the dports build. It will be null-mounted in the chroot and the
"/usr/src/dports.base/dragonfly" directory will be null-mounted under it
to provide the "dragonfly/base" meta-port. The DPORTSBASE make variable
may be used to override the location of the main dports repo.

The build-all and install-all targets use a sophisticated chrooted
environment to do a completely clean dports build with dsynth. It
It will actually "make installworld" plus a clean "make distribution"
inside the chroot environment to create the environment, and the host
system's /etc/resolv.conf will be copied in.

build-all - Augmented when used WITH_DPORTS=YES
install-all - Augmented when used WITH_DPORTS=YES
buildportschroot - Build a fresh chroot environment for dsynth
mountports - Do all null mounts and cp's needed for the env
buildports - Run dsynth build dragonfly/base in the env
installports - Install the built ports on the host system (see note)
umountports - Remove the null mounts

WITH_DPORTS=YES - executes buildportschroot, mountports, buildports,
and umountports when run from build-all.

executes mountports, installports, umountports
when run from install-all.

DPORTSBASE=path - Defaults to /usr/dports, may be used to override
the location where the main dports tree can be
found.

NOTE: Installports may require interactivity as the base system might already
have installed ports. Generally speaking it is best to not revert any
ports you might already have installed, so feel free to tell it not to
install anything.

* Added various make targets and augmented build-all and install-all

* /usr/local/sbin added and the stale /usr/pkg/bin removed from _HOSTPATH.

* Add /usr/src/dports.base infrastructure.

It contains the dsynth.ini and pkg.conf files that will be used for the
local dsynth operations. Some paths are replaced with SED when copied
to the chroot. The dsynth.ini is relative to the chroot.

It also contains the dragonfly/base dport which is the meta-port that
dsynth will be told to build.

All other dports related files and directories are null-mounted from
/usr/dports.

* Add /usr/distfiles.base to mtree/BSD.usr.dist. This directory is also
created by the build if it does not exist since the host system might
not have run a new installworld recently.

This directory will be used during build-all (buildports) to store the
distfiles required to build dragonfly/base, so they only need to be
fetched once.

* Add the "-C ConfigBase" option to dsynth to make running it from the
chrooted environment easier. This overrides the /etc/dsynth directory
and removes /etc/dsynth as a fallback.

* dsynth conditionalizes calling syscap_set() for backwards compatibility
with older host systems that might not have been upgraded to include the
feature yet.

show more ...

Makefile: Rename 'preupgrade' to 'preinstall'

The 'preupgrade' target performs certain pre-installworld operations,
and is executed before 'installworld' automatically. So rename it to
'preinstall'

Makefile: Rename 'preupgrade' to 'preinstall'

The 'preupgrade' target performs certain pre-installworld operations,
and is executed before 'installworld' automatically. So rename it to
'preinstall'.

Update build(7) man page accordingly.

Suggested-by: swildner

show more ...

Makefile: Make 'installworld' depend on 'preupgrade'

The 'preupgrade' target will creates new users/groups, which may be
required by the 'installworld' target, so make 'installworld' depend on
'preu

Makefile: Make 'installworld' depend on 'preupgrade'

The 'preupgrade' target will creates new users/groups, which may be
required by the 'installworld' target, so make 'installworld' depend on
'preupgrade'. This will fix the installation of nvmmctl(8) that
requires the 'nvmm' group.

As another result, no longer need to check for users/groups in the
'installcheck' target.

show more ...

build.7: mention "build-all" and "install-all" targets

Describe "build-all" and "install-all" targets in both Makefile
and build(7).

Submitted by: Pierre-Alain TORET <pierre-alain.toret@protonmail.

build.7: mention "build-all" and "install-all" targets

Describe "build-all" and "install-all" targets in both Makefile
and build(7).

Submitted by: Pierre-Alain TORET <pierre-alain.toret@protonmail.com>
Issue: https://bugs.dragonflybsd.org/issues/3229

show more ...

Makefile: Use internal variable for setting PATH.

Mainly to prevent double override PATH when invoking submakes.

build - Add build-all and install-all targets

* Add a 'smart' build-all and install-all target that spews to a
log file instead of to the screen and. Will automatically
run the build stage -j

build - Add build-all and install-all targets

* Add a 'smart' build-all and install-all target that spews to a
log file instead of to the screen and. Will automatically
run the build stage -j N, and both targets can be specified on
one command line.

make build-all install-all

build-all: runs buildworld and buildkernel -j hw.ncpu
install-all: runs installkernel and installworld -j 1

* The usual warnings and cautions still apply when installing a
new world and kernel.

show more ...

installworld - Generate /etc/os-release

* Generate an /etc/os-release file during an installworld.

Makefile: Require explicit target

Explicit target (e.g., buildworld, buildkernel) is required, otherwise
fail with an error.

Obtained-from: FreeBSD
Bug-report: #3158

Remove groff from base.

There is a dports package available for people who need it. Also,
for some manual pages in dports, man(1) will report that mandoc(1)
cannot render them, and advise to install

Remove groff from base.

There is a dports package available for people who need it. Also,
for some manual pages in dports, man(1) will report that mandoc(1)
cannot render them, and advise to install groff too, which man(1)
will then use.

Also remove vgrind(1) and the old base me(7) macros.

Disable the groff part of manlint for now, also mandiff. I will
decide later if it should be made to work with dports groff.

There are a number of related utilities, like colcrt, checknr etc.
that I have not touched yet. Their fate will be decided later.

There are also some references to these tools left in existing
manual pages.

show more ...

Fix pkg "-f" flag in the "make upgrade" message

initrd: Rework build and install stages

* Rewrite the Makefile to provide only the 'all' and 'install' targets.
Do not separate the handling of the rescue tools and initrd image.
Thus greatly si

initrd: Rework build and install stages

* Rewrite the Makefile to provide only the 'all' and 'install' targets.
Do not separate the handling of the rescue tools and initrd image.
Thus greatly simplify the whole logic.

The 'all' target will build the rescue tools and initrd contents, and
does not require root priviledge, while the 'install' target will
install the rescue tools to the system, create the initrd image and
install it.

* Update the top-level makefiles to call the new 'all' and 'install'.
The 'buildworld' target now doesn't require root priviledge, same as
before the refactoring in 9b724c0dcb2b9548a82d28d97e34375f64668669.

* Only provide the 'initrd' top-level target to keep it simple. This
target will install the rescue tools and initrd image built by the
'buildworld', overwriting the existing ones.

* Clean up the nrelease/Makefile a bit, since 'installworld' will also
install the rescue tools and initrd image in such a case.

* Update the mkinitrd.sh script a bit.

* Update the build(7) man page and clean up a bit.

Reviewed-by: swildner

show more ...

build - refactor rescue and initrd handling

* buildworld now builds the rescue directory tree and initrd
image in /usr/obj.

* installworld will install the rescue and initrd stuff if it
does no

build - refactor rescue and initrd handling

* buildworld now builds the rescue directory tree and initrd
image in /usr/obj.

* installworld will install the rescue and initrd stuff if it
does not exist on the destination. If rescue or initrd stuff
is already installed, it is not overwritten.

* Add 'installrescue' and 'installinitrd' targets. These will
copy the rescue and initrd image built by buildworld.

* The normal 'rescue' and 'initrd' targets are unchanged and
should build (from scratch) and install the rescue and initrd
stuff.

* NOTE: crunchgen is currently hacked to add system paths when
WORLDBUILD is defined, in order to access native binaries
for 'uniq' and a few others, instead of building them in
cross-tools.

In particular, we have to use the native 'vnconfig' binary
as building one in crosstools might not be compatible with
the currently running system. buildworld now uses 'vn' to
create the initrd image.

Possibly needs some work.

show more ...

initrd: Refactor Makefile and introduce quick{rescue,initrd}

Refactor the Makefile by splitting the original targets into smaller
ones.

Introduce targets quick{rescue,initrd} which skip the clean s

initrd: Refactor Makefile and introduce quick{rescue,initrd}

Refactor the Makefile by splitting the original targets into smaller
ones.

Introduce targets quick{rescue,initrd} which skip the clean step.
Such similar behavior was achieved by passing '-DNO_CLEAN' to make.
(suggested-by: swildner)

show more ...

Remove the old 'most' and 'installmost' targets (they are broken too).

initrd: Detach from the world and build on the fly

Why
---
Currently, the files for initrd image creation are built and installed at
/usr/share/initrd by "buildworld" and "installworld", respectivel

initrd: Detach from the world and build on the fly

Why
---
Currently, the files for initrd image creation are built and installed at
/usr/share/initrd by "buildworld" and "installworld", respectively, and then
mkinitrd(8) simply packs them to create the initrd.img.gz.

For normal users, the shipped /usr/share/initrd and mkinitrd(8) don't make
much sense, since they don't need to modify the initrd contents and create
a modified initrd image.

From a developer's perspective, the required steps (buildworld, installworld
and mkinitrd) to create and test a new initrd image can be annoying.

How
---
Detach the build and installation of the initrd contents from the world.
Build the (rescue/initrd) tools on-the-fly when creating the initrd image.

As per ftigeot's suggestion, these statically linked tools can be useful when
dealing with a broken system, so install them under /rescue to be more
intuitive and easier to use (similar to FreeBSD and maybe other BSDs).

What
----
* Move "share/initrd" to be top-level, and detach it from world.

+ Update the patch in bsd.crunchgen.mk accordingly.
+ Add Makefile.inc to simplify subdir's makefiles.
+ Rewrite the Makefile with targets:
- rescue: build and install the rescue tools into /rescue.
- initrd: further packs the etc and rescue staffs to create the initrd
image.

* Move "sbin/mkinitrd/mkinitrd.sh" under the new "initrd" directory and adapt
it to work with the "initrd" make target. Remove the other parts of
mkinitrd(8).

* Export the above "rescue" and "initrd" targets to be top-level.

* Update nrelease framework to use the new "make initrd".

* Update build.7 man page accordingly.

* Print the message about updating the initrd image and 3rd-party packages
after "make upgrade" instead of "make installworld".

* Document this change in UPDATING.

* Other small updates and style cleanups.

TODO
----
* Add initrd.7 man page
* Crunch more useful tools
* Crunch bin/sbin/etc. into a bundle to save space (??)

Reviewed-by: swildner
Thanks-to: dillon, swildner, ftigeot

show more ...

Makefile{,.inc1}: Merge user-driven target descriptions and cleanups

* Merge the descriptions to user-driven targets from Makefile.inc1 to
Makefile and improve the wordings a bit.

* Delete some o

Makefile{,.inc1}: Merge user-driven target descriptions and cleanups

* Merge the descriptions to user-driven targets from Makefile.inc1 to
Makefile and improve the wordings a bit.

* Delete some obsolete information.

* Whitespace cleanups.

show more ...

Add a realquickkernel target, analogous to realquickworld.

It skips the recently added depend step, so it behaves like
quickkernel did before 521f740e8971df6fdb1b63933cb534746e86bfae.

i386 removal, part 2/x: Remove i386 specific kernel config files.

Which are: GENERIC, LINT, SOEKRIS, VKERNEL

Fix lack of buildworld and quickworld messages

The "world" target has a nice start/stop message. However, if somebody
chooses to use "make buildworld" followed by "make installworld" rather
than "m

Fix lack of buildworld and quickworld messages

The "world" target has a nice start/stop message. However, if somebody
chooses to use "make buildworld" followed by "make installworld" rather
than "make world", they get no messages at all. I think this has been an
ongoing oversight.

This commit improves the "world" target messages (e.g. all targets are
ELF so don't need to specify that, and pre/post-world targets are shown.)
More importantly it adds start/complete messages to buildworld, and start
message to installworld. The installworld already had a complete message,
but this was redone to match the format of the other ones.

Similar messages were added to quickworld and realquickworld targets too.

show more ...

build - add installworld-force target and better checks

* Add better checks to installworld. Print an error and exit if the
base system is too old, directing users to run installworld-force.

* A

build - add installworld-force target and better checks

* Add better checks to installworld. Print an error and exit if the
base system is too old, directing users to run installworld-force.

* Add installworld-force. This automates installing the world on a
base system which is too old and thus has chicken-and-egg problems
with binaries and libraries.

This install target will install to a temporary directory, compile up
a static cpdup, and then use it to just blast copy /bin, /sbin, /lib,
/libexec, /usr/bin, /usr/sbin, /usr/lib, and /usr/libexec. It will
then execute a normal make installworld and make upgrade.

show more ...

build - separate out the rescue initrd

* Multiple issues with users crashing at the end of an installworld because
initrd kldloads an out-of-date vn device.

* Not a good idea to update the rescue

build - separate out the rescue initrd

* Multiple issues with users crashing at the end of an installworld because
initrd kldloads an out-of-date vn device.

* Not a good idea to update the rescue image during an installworld anyway.

* Make it a manual target called 'rescue' and remind the user at the end
of the installworld that the target is available, but suggest
rebooting first.

show more ...

bsd.man.mk: add temporary ``mandiff'' target

Show differences in ASCII output between groff(1) and mandoc(1). This
will help generate uptream bug reports. Also useful for spotting local
configurat

bsd.man.mk: add temporary ``mandiff'' target

Show differences in ASCII output between groff(1) and mandoc(1). This
will help generate uptream bug reports. Also useful for spotting local
configuration quirks of both programs.

show more ...

Remove last two references to legacy make

This last two LEGACY MAKE conditions slipped by. The others were removed
in a previous commit.

Give /usr/local/... precedence over /usr/pkg/... in various paths.

Add /usr/local/... to various paths.