1.\" Copyright (c) 1998 Matthew Dillon. Terms and conditions are those of 2.\" the BSD Copyright as specified in the file "/usr/src/COPYRIGHT" in 3.\" the DragonFly source tree. 4.\" 5.\" $FreeBSD: src/share/man/man7/development.7,v 1.4.2.2 2003/05/23 07:48:35 brueffer Exp $ 6.\" $DragonFly: src/share/man/man7/development.7,v 1.12 2008/07/27 22:23:42 thomas Exp $ 7.\" 8.Dd December 15, 2007 9.Dt DEVELOPMENT 7 10.Os 11.Sh NAME 12.Nm development 13.Nd introduction to development with the DragonFly codebase 14.Sh DESCRIPTION 15This manual page describes how an ordinary sysop, 16.Ux admin, or developer 17can, without any special permission, obtain, maintain, and modify the 18.Dx 19codebase as well as how to maintaining a master build which can 20then be exported to other machines in your network. 21This manual page 22is targeted to system operators, programmers, and developers. 23.Pp 24Please note that what is being described here is based on a complete 25.Dx 26environment, not just the 27.Dx 28kernel. 29The methods described 30here are as applicable to production installations as it is to development 31environments. 32You need a good 12-17GB of disk space on one machine to make this work 33conveniently. 34.Sh SETTING UP THE ENVIRONMENT ON THE MASTER SERVER 35Your master server should always run a stable, production version of the 36.Dx 37operating system. 38This does not prevent you from doing -PREVIEW or -DEVELOPMENT 39builds or development. 40The last thing you want to do is to run an 41unstable environment on your master server which could lead to a situation 42where you lose the environment and/or cannot recover from a mistake. 43.Pp 44Create a partition called 45.Pa /DragonFly . 468-12GB is recommended. 47This partition will contain nearly all the development environment, 48including the CVS tree, broken-out source, and possibly even object files. 49You are going to export this partition to your other machines via a 50.Sy read-only 51NFS export so do not mix it with other more security-sensitive 52partitions. 53.Pp 54You have to make a choice in regards to 55.Pa /usr/obj . 56You can put 57.Pa /usr/obj 58in 59.Pa /DragonFly 60or you can make 61.Pa /usr/obj 62its own partition. 63I recommend making it a separate partition for several reasons. 64First, 65as a safety measure since this partition is written to a great deal. 66Second, because you typically do not have to back it up. 67Third, because it makes it far easier to mix and match the development 68environments which are described later in this document. 69I recommend a 70.Pa /usr/obj 71partition of at least 5GB. 72.Pp 73On the master server, use cvsup to automatically pull down and maintain 74the 75.Dx 76CVS archive once a day. 77The first pull will take a long time, 78it is several gigabytes, but once you have it the daily syncs will be quite 79small. 80.Bd -literal -offset 4n 81mkdir /DragonFly/DragonFly-CVS 82rm -rf /home/dcvs 83ln -s /DragonFly/DragonFly-CVS /home/dcvs 84.Ed 85.Pp 86The cron job should look something like this (please randomize the time of 87day!). 88Note that you can use the cvsup file example directly from 89.Pa /usr/share/examples 90without modification by supplying appropriate arguments 91to cvsup. Please use a cvsup mirror, see the 92.Dx 93site. 94.Bd -literal -offset 4n 9533 6 * * * /usr/local/bin/cvsup -gr 20 -L 2 -h cvsup.dragonflybsd.org \\ 96 /usr/share/examples/cvsup/DragonFly-cvs-supfile 97.Ed 98.Pp 99Run the cvsup manually the first time to pull down the archive. 100It could take 101all day depending on how fast your connection is! 102You will run all cvsup and cvs operations as root and you need to set 103up a 104.Pa ~/.cvsrc 105.Pa ( /root/.cvsrc ) 106file, as shown below, for proper cvs operation. 107Using 108.Pa ~/.cvsrc 109to specify cvs defaults is an excellent way 110to "file and forget", but you should never forget that you put them in there. 111.Bd -literal -offset 4n 112# cvs -q 113diff -u 114update -Pd 115checkout -P 116.Ed 117.Pp 118Now use cvs to checkout a -RELEASE source tree and a -DEVELOPMENT source tree, 119as well as docs, to create your initial source environment. 120Keeping the broken-out source in 121.Pa /DragonFly 122allows you to export 123it to other machines via read-only NFS. 124This also means you only need to edit/maintain files in one place and all 125your clients automatically pick up the changes. 126.Bd -literal -offset 4n 127mkdir /DragonFly/DragonFly-RELEASE 128mkdir /DragonFly/DragonFly-DEVELOPMENT 129 130cd /DragonFly/DragonFly-RELEASE 131cvs -d /home/dcvs checkout -r DragonFly_RELEASE_1_10_Slip src 132 133cd /DragonFly/DragonFly-DEVELOPMENT 134cvs -d /home/dcvs checkout src 135cvs -d /home/dcvs checkout doc 136.Ed 137.Pp 138Now create a softlink for 139.Pa /usr/src 140and 141.Pa /usr/src2 . 142On the main server I always point 143.Pa /usr/src 144at -RELEASE and 145.Pa /usr/src2 146at 147-DEVELOPMENT. 148On client machines I usually do not have a 149.Pa /usr/src2 150and I make 151.Pa /usr/src 152point at whatever version of 153.Dx 154the client box is intended to 155run. 156.Bd -literal -offset 4n 157cd /usr 158rm -rf src src2 159ln -s /DragonFly/DragonFly-RELEASE/src src (could be -DEVELOPMENT on a client) 160ln -s /DragonFly/DragonFly-DEVELOPMENT/src src2 (MASTER SERVER ONLY) 161.Ed 162.Pp 163Now you have to make a choice for 164.Pa /usr/obj . 165Well, hopefully you made it already and chose the partition method. 166If you 167chose poorly you probably intend to put it in 168.Pa /DragonFly 169and, if so, this is 170what you want to do: 171.Bd -literal -offset 4n 172(ONLY IF YOU MADE A POOR CHOICE AND PUT /usr/obj in /DragonFly!) 173mkdir /DragonFly/obj 174cd /usr 175rm -rf obj 176ln -s /DragonFly/obj obj 177.Ed 178.Pp 179Alternatively you may chose simply to leave 180.Pa /usr/obj 181in 182.Pa /usr . 183If your 184.Pa /usr 185is large enough this will work, but I do not recommend it for 186safety reasons 187.Pa ( /usr/obj 188is constantly being modified, 189.Pa /usr 190is not). 191.Pp 192Note that exporting 193.Pa /usr/obj 194via read-only NFS to your other boxes will 195allow you to build on your main server and install from your other boxes. 196If you also want to do builds on some or all of the clients you can simply 197have 198.Pa /usr/obj 199be a local directory on those clients. 200You should never export 201.Pa /usr/obj 202read-write, it will lead to all sorts of 203problems and issues down the line and presents a security problem as well. 204It is far easier to do builds on the master server and then only do installs 205on the clients. 206.Sh EXPORTING VIA NFS FROM THE MASTER SERVER 207The master server needs to export 208.Pa /DragonFly 209and 210.Pa /usr/obj 211via NFS so all the 212rest of your machines can get at them. 213I strongly recommend using a read-only export for both security and safety. 214The environment I am describing in this manual page is designed primarily 215around read-only NFS exports. 216Your 217.Pa /etc/exports 218file on the master server should contain the following lines: 219.Bd -literal -offset 4n 220/DragonFly -ro -alldirs -maproot=root: -network YOURLAN -mask YOURLANMASK 221/usr/obj -ro -alldirs -maproot=root: -network YOURLAN -mask YOURLANMASK 222.Ed 223.Pp 224Of course, NFS server operations must also be configured on that machine. 225This is typically done via your 226.Pa /etc/rc.conf : 227.Bd -literal -offset 4n 228nfs_server_enable="YES" 229.Ed 230.Sh THE CLIENT ENVIRONMENT 231All of your client machines can import the development/build environment 232directory simply by NFS mounting 233.Pa /DragonFly 234and 235.Pa /usr/obj 236from the master 237server. 238A typical 239.Pa /etc/fstab 240entry on your client machines will be something like this: 241.Bd -literal -offset 4n 242masterserver:/DragonFly /DragonFly nfs ro,bg 0 0 243masterserver:/usr/obj /usr/obj nfs ro,bg 0 0 244.Ed 245.Pp 246And, of course, you should configure the client for NFS client operations 247via 248.Pa /etc/rc.conf . 249In particular, this will turn on nfsiod which will improve client-side NFS 250performance: 251.Bd -literal -offset 4n 252nfs_client_enable="YES" 253.Ed 254.Pp 255Each client should create softlink for 256.Pa /usr/src 257that point 258into the NFS-mounted environment. 259If a particular client is running -DEVELOPMENT, 260.Pa /usr/src 261should be a softlink to 262.Pa /DragonFly/DragonFly-DEVELOPMENT/src . 263If it is running -RELEASE, 264.Pa /usr/src 265should be a softlink to 266.Pa /DragonFly/DragonFly-RELEASE/src . 267I do not usually create a 268.Pa /usr/src2 269softlink on 270clients, that is used as a convenient shortcut when working on the source 271code on the master server only and could create massive confusion (of the 272human variety) on a client. 273.Bd -literal -offset 4n 274(ON EACH CLIENT) 275cd /usr 276ln -s /DragonFly/DragonFly-XXX/src src 277.Ed 278.Sh BUILDING KERNELS 279Here is how you build a -RELEASE kernel (on your main development box). 280If you want to create a custom kernel, copy 281.Pa GENERIC 282to 283.Pa YOURKERNEL 284and then 285edit it before configuring and building. 286The kernel configuration file lives in 287.Pa /sys/config/KERNELNAME . 288.Bd -literal -offset 4n 289cd /usr/src 290make buildkernel KERNCONF=KERNELNAME 291.Ed 292.Pp 293Building a -DEVELOPMENT kernel 294.Bd -literal -offset 4n 295cd /usr/src2 (on the master server) 296make buildkernel KERNCONF=KERNELNAME 297.Ed 298.Sh INSTALLING KERNELS 299Installing a -RELEASE kernel (typically done on a client. 300Only do this on your main development server if you want to install a new 301kernel for your main development server): 302.Bd -literal -offset 4n 303cd /usr/src 304make installkernel KERNCONF=KERNELNAME 305.Ed 306.Pp 307Installing a -DEVELOPMENT kernel (typically done only on a client) 308.Bd -literal -offset 4n 309(remember /usr/src is pointing to the client's specific environment) 310cd /usr/src 311make installkernel KERNCONF=KERNELNAME 312.Ed 313.Sh BUILDING THE WORLD 314This environment is designed such that you do all builds on the master server, 315and then install from each client. 316You can do builds on a client only if 317.Pa /usr/obj 318is local to that client. 319Building the world is easy: 320.Bd -literal -offset 4n 321cd /usr/src 322make buildworld 323.Ed 324.Pp 325If you are on the master server you are running in a -RELEASE environment, but 326that does not prevent you from building the -DEVELOPMENT world. 327Just cd into the appropriate source directory and you are set. 328Do not 329accidentally install it on your master server though! 330.Bd -literal -offset 4n 331cd /usr/src2 332make buildworld 333.Ed 334.Sh INSTALLING THE WORLD 335You can build on your main development server and install on clients. 336The main development server must export 337.Pa /DragonFly 338and 339.Pa /usr/obj 340via 341read-only NFS to the clients. 342.Pp 343.Em NOTE!!! 344If 345.Pa /usr/obj 346is a softlink on the master server, it must also be the EXACT 347SAME softlink on each client. 348If 349.Pa /usr/obj 350is a directory in 351.Pa /usr 352or a mount point on the master server, 353then it must be (interchangeably) a directory in 354.Pa /usr 355or a mount point on 356each client. 357This is because the 358absolute paths are expected to be the same when building the world as when 359installing it, and you generally build it on your main development box 360and install it from a client. 361If you do not setup 362.Pa /usr/obj 363properly you will not be able to build on 364machine and install on another. 365.Bd -literal -offset 4n 366(ON THE CLIENT) 367(remember 368.Pa /usr/src 369is pointing to the client's specific environment) 370cd /usr/src 371make installworld 372.Ed 373.Pp 374.Sy WARNING! 375If builds work on the master server but installs do not work from the 376clients, for example you try to install and the client complains that 377the install tried to write into the read-only 378.Pa /usr/obj , 379then it is likely 380that the 381.Pa /etc/make.conf 382file on the client does not match the one on the 383master server closely enough and the install is trying to install something 384that was not built. 385.Sh DOING DEVELOPMENT ON A CLIENT (NOT JUST INSTALLING) 386Developers often want to run buildkernel's or buildworld's on client 387boxes simply to life-test the box. 388You do this in the same manner that you buildkernel and buildworld on your 389master server. 390All you have to do is make sure that 391.Pa /usr/obj 392is pointing to local storage. 393If you followed my advise and made 394.Pa /usr/obj 395its own partition on the master 396server, 397then it is typically going to be an NFS mount on the client. 398Simply unmounting 399.Pa /usr/obj 400will leave you with a 401.Pa /usr/obj 402that is a 403subdirectory in 404.Pa /usr 405which is typically local to the client. 406You can then do builds to your heart's content! 407.Sh MAINTAINING A LOCAL BRANCH 408I have described how to maintain two versions of the source tree, a stable 409version in 410.Pa /DragonFly/DragonFly-RELEASE 411and a current version 412in 413.Pa /DragonFly/DragonFly-DEVELOPMENT . 414There is absolutely nothing preventing you 415from breaking out other versions of the source tree 416into 417.Pa /DragonFly/XXX . 418In fact, my 419.Pa /DragonFly 420partition also contains 421.Fx , 422.Ox , 423.Nx , 424and various flavors of Linux. 425You may not necessarily be able to build 426.Pf non- Dx 427operating systems on your master server, but being able 428to collect and manage source distributions from a central server is a very 429useful thing to be able to do and you can certainly export to machines 430which can build those other operating systems. 431.Pp 432Many developers choose to maintain a local branch of 433.Dx 434to test patches or build a custom distribution. 435This can be done with CVS or another source code management system 436(SubVersion, Perforce, BitKeeper) with its own repository. 437Since the main 438.Dx 439tree is based on CVS, the former is convenient. 440.Pp 441First, you need to modify your cvsup environment to avoid it modifying 442the local changes you have committed to the repository. 443It is important to remove the "delete" keyword from your supfile and to 444add the CVSROOT subdirectory to your refuse file. 445For more information, see 446.Xr cvsup 1 . 447.Pp 448The 449.Dx 450version of CVS examines a custom environmental variable, 451CVS_LOCAL_BRANCH_NUM, which specifies an integer to use when doing a cvs 452tag/rtag. 453Set this number to something high (say 1000) to avoid colliding 454with potential future branches of the main repository. 455For example, 456branching a file with version 1.4 produces 1.4.1000. 457Future commits to this branch will produce revisions 1.4.1000.1, 4581.4.1000.2, etc. 459.Pp 460To fork your local branch, do: 461.Bd -literal -offset 4n 462cvs rtag -r DragonFly_RELEASE_1_10_Slip -b LOCAL_DragonFly_RELEASE_1_10 src 463.Ed 464.Pp 465After this, you can check out a copy from your local repository using the 466new tag and begin making changes and committing them. 467For more information on using cvs, see 468.Xr cvs 1 . 469.Pp 470.Sy WARNING! 471The cvsup utility may blow away changes made on a local branch in 472some situations. 473This has been reported to occur when the master CVS repository is 474directly manipulated or an RCS file is changed. 475At this point, cvsup notices that the client and server have entirely 476different RCS files, so it does a full replace instead of trying to 477send just deltas. 478Ideally this situation should never arise, but in the real world it 479happens all the time. 480.Pp 481While this is the only scenario where the problem should crop up, 482there have been some suspicious-sounding reports of 483CVS_LOCAL_BRANCH_NUM lossage that can't be explained by this alone. 484Bottom line is, if you value your local branch then you 485should back it up before every update. 486.Sh UPDATING VIA CVS 487The advantage of using cvsup to maintain an updated copy of the CVS 488repository instead of using it to maintain source trees directly is that you 489can then pick and choose when you bring your source tree (or pieces of your 490source tree) up to date. 491By using a cron job to maintain an updated CVS repository, you can update 492your source tree at any time without any network cost as follows: 493.Bd -literal -offset 4n 494(on the main development server) 495cd /usr/src 496cvs -d /home/dcvs update 497cd /usr/src2 498cvs -d /home/dcvs update 499.Ed 500.Pp 501It is that simple, and since you are exporting the whole lot to your 502clients, your clients have immediately visibility into the updated 503source. 504This is a good time to also remind you that most of the cvs operations 505you do will be done as root, and that certain options are 506required for CVS to operate properly on the 507.Dx 508repository. 509For example, 510.Fl Pd 511is necessary when running "cvs update". 512These options are typically placed in your 513.Pa ~/.cvsrc 514(as already described) 515so you do not have to respecify them every time you run a CVS command. 516Maintaining the CVS repository also gives you far more flexibility 517in regards to breaking out multiple versions of the source tree. 518It is a good idea to give your 519.Pa /DragonFly 520partition a lot of space (I recommend 5218-12GB) precisely for that reason. 522If you can make it 15GB I would do it. 523.Pp 524I generally do not cvs update via a cron job. 525This is because I generally want the source to not change out from under me 526when I am developing code. 527Instead I manually update the source every so often...\& when I feel it is 528a good time. 529My recommendation is to only keep the cvs repository synchronized via cron. 530.Sh SEE ALSO 531.Xr crontab 1 , 532.Xr cvs 1 , 533.Xr cvsup 1 , 534.Xr crontab 5 , 535.Xr exports 5 , 536.Xr kernconf 5 , 537.Xr make.conf 5 , 538.Xr build 7 , 539.Xr committer 7 , 540.Xr firewall 7 , 541.Xr release 7 , 542.Xr tuning 7 , 543.Xr diskless 8 544.Sh HISTORY 545The 546.Nm 547manual page was originally written by 548.An Matthew Dillon Aq dillon@FreeBSD.org 549and first appeared 550in 551.Fx 5.0 , 552December 2002. 553