1.\" 2.\" Copyright (c) 2006, 2007 3.\" The DragonFly Project. All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 9.\" 1. Redistributions of source code must retain the above copyright 10.\" notice, this list of conditions and the following disclaimer. 11.\" 2. Redistributions in binary form must reproduce the above copyright 12.\" notice, this list of conditions and the following disclaimer in 13.\" the documentation and/or other materials provided with the 14.\" distribution. 15.\" 3. Neither the name of The DragonFly Project nor the names of its 16.\" contributors may be used to endorse or promote products derived 17.\" from this software without specific, prior written permission. 18.\" 19.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 20.\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 21.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS 22.\" FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 23.\" COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, 24.\" INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING, 25.\" BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 26.\" LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED 27.\" AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 28.\" OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT 29.\" OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 30.\" SUCH DAMAGE. 31.\" 32.Dd June 20, 2015 33.Dt VKERNEL 7 34.Os 35.Sh NAME 36.Nm vkernel , 37.Nm vcd , 38.Nm vkd , 39.Nm vke 40.Nd virtual kernel architecture 41.Sh SYNOPSIS 42.Cd "platform vkernel64 # for 64 bit vkernels" 43.Cd "device vcd" 44.Cd "device vkd" 45.Cd "device vke" 46.Pp 47.Pa /var/vkernel/boot/kernel/kernel 48.Op Fl hsUvd 49.Op Fl c Ar file 50.Op Fl e Ar name Ns = Ns Li value : Ns Ar name Ns = Ns Li value : Ns ... 51.Op Fl i Ar file 52.Op Fl I Ar interface Ns Op Ar :address1 Ns Oo Ar :address2 Oc Ns Oo Ar /netmask Oc Ns Oo Ar =mac Oc 53.Op Fl l Ar cpulock 54.Op Fl m Ar size 55.Op Fl n Ar numcpus Ns Op Ar :lbits Ns Oo Ar :cbits Oc 56.Op Fl p Ar pidfile 57.Op Fl r Ar file Ns Op Ar :serno 58.Sh DESCRIPTION 59The 60.Nm 61architecture allows for running 62.Dx 63kernels in userland. 64.Pp 65The following options are available: 66.Bl -tag -width ".Fl m Ar size" 67.It Fl c Ar file 68Specify a readonly CD-ROM image 69.Ar file 70to be used by the kernel, with the first 71.Fl c 72option defining 73.Li vcd0 , 74the second one 75.Li vcd1 , 76and so on. 77The first 78.Fl r 79or 80.Fl c 81option specified on the command line will be the boot disk. 82The CD9660 filesystem is assumed when booting from this media. 83.It Fl d 84Disables hardware pagetable for 85.Nm . 86.It Fl e Ar name Ns = Ns Li value : Ns Ar name Ns = Ns Li value : Ns ... 87Specify an environment to be used by the kernel. 88This option can be specified more than once. 89.It Fl h 90Shows a list of available options, each with a short description. 91.It Fl i Ar file 92Specify a memory image 93.Ar file 94to be used by the virtual kernel. 95If no 96.Fl i 97option is given, the kernel will generate a name of the form 98.Pa /var/vkernel/memimg.XXXXXX , 99with the trailing 100.Ql X Ns s 101being replaced by a sequential number, e.g.\& 102.Pa memimg.000001 . 103.It Fl I Ar interface Ns Op Ar :address1 Ns Oo Ar :address2 Oc Ns Oo Ar /netmask Oc Ns Oo Ar =MAC Oc 104Create a virtual network device, with the first 105.Fl I 106option defining 107.Li vke0 , 108the second one 109.Li vke1 , 110and so on. 111.Pp 112The 113.Ar interface 114argument is the name of a 115.Xr tap 4 116device node or the path to a 117.Xr vknetd 8 118socket. 119The 120.Pa /dev/ 121path prefix does not have to be specified and will be automatically prepended 122for a device node. 123Specifying 124.Cm auto 125will pick the first unused 126.Xr tap 4 127device. 128.Pp 129The 130.Ar address1 131and 132.Ar address2 133arguments are the IP addresses of the 134.Xr tap 4 135and 136.Nm vke 137interfaces. 138Optionally, 139.Ar address1 140may be of the form 141.Li bridge Ns Em X 142in which case the 143.Xr tap 4 144interface is added to the specified 145.Xr bridge 4 146interface. 147The 148.Nm vke 149address is not assigned until the interface is brought up in the guest. 150.Pp 151The 152.Ar netmask 153argument applies to all interfaces for which an address is specified. 154.Pp 155The 156.Ar MAC 157argument is the MAC address of the 158.Xr vke 4 159interface. 160If not specified, a pseudo-random one will be generated. 161.Pp 162When running multiple vkernels it is often more convenient to simply 163connect to a 164.Xr vknetd 8 165socket and let vknetd deal with the tap and/or bridge. 166An example of this would be 167.Pa /var/run/vknet:0.0.0.0:10.2.0.2/16 . 168.It Fl l Ar cpulock 169Specify which, if any, real CPUs to lock virtual CPUs to. 170.Ar cpulock 171is one of 172.Cm any , 173.Cm map Ns Op Ns , Ns Ar startCPU , 174or 175.Ar CPU . 176.Pp 177.Cm any 178does not map virtual CPUs to real CPUs. 179This is the default. 180.Pp 181.Cm map Ns Op Ns , Ns Ar startCPU 182maps each virtual CPU to a real CPU starting with real CPU 0 or 183.Ar startCPU 184if specified. 185.Pp 186.Ar CPU 187locks all virtual CPUs to the real CPU specified by 188.Ar CPU . 189.It Fl m Ar size 190Specify the amount of memory to be used by the kernel in bytes, 191.Cm K 192.Pq kilobytes , 193.Cm M 194.Pq megabytes 195or 196.Cm G 197.Pq gigabytes . 198Lowercase versions of 199.Cm K , M , 200and 201.Cm G 202are allowed. 203.It Fl n Ar numcpus Ns Op Ar :lbits Ns Oo Ar :cbits Oc 204.Ar numcpus 205specifies the number of CPUs you wish to emulate. 206Up to 16 CPUs are supported with 2 being the default unless otherwise 207specified. 208.Ar lbits 209specifies the number of bits within APICID(=CPUID) needed for representing 210the logical ID. 211Controls the number of threads/core (0bits - 1 thread, 1bit - 2 threads). 212This parameter is optional (mandatory only if 213.Ar cbits 214is specified). 215.Ar cbits 216specifies the number of bits within APICID(=CPUID) needed for representing 217the core ID. 218Controls the number of core/package (0bits - 1 core, 1bit - 2 cores). 219This parameter is optional. 220.It Fl p Ar pidfile 221Specify a pidfile in which to store the process ID. 222Scripts can use this file to locate the vkernel pid for the purpose of 223shutting down or killing it. 224.Pp 225The vkernel will hold a lock on the pidfile while running. 226Scripts may test for the lock to determine if the pidfile is valid or 227stale so as to avoid accidentally killing a random process. 228Something like '/usr/bin/lockf -ks -t 0 pidfile echo -n' may be used 229to test the lock. 230A non-zero exit code indicates that the pidfile represents a running 231vkernel. 232.Pp 233An error is issued and the vkernel exits if this file cannot be opened for 234writing or if it is already locked by an active vkernel process. 235.It Fl r Ar file Ns Op Ar :serno 236Specify a R/W disk image 237.Ar file 238to be used by the kernel, with the first 239.Fl r 240option defining 241.Li vkd0 , 242the second one 243.Li vkd1 , 244and so on. 245A serial number for the virtual disk can be specified in 246.Ar serno . 247.Pp 248The first 249.Fl r 250or 251.Fl c 252option specified on the command line will be the boot disk. 253.It Fl s 254Boot into single-user mode. 255.It Fl U 256Enable writing to kernel memory and module loading. 257By default, those are disabled for security reasons. 258.It Fl v 259Turn on verbose booting. 260.El 261.Sh DEVICES 262A number of virtual device drivers exist to supplement the virtual kernel. 263.Ss Disk device 264The 265.Nm vkd 266driver allows for up to 16 267.Xr vn 4 268based disk devices. 269The root device will be 270.Li vkd0 271(see 272.Sx EXAMPLES 273for further information on how to prepare a root image). 274.Ss CD-ROM device 275The 276.Nm vcd 277driver allows for up to 16 virtual CD-ROM devices. 278Basically this is a read only 279.Nm vkd 280device with a block size of 2048. 281.Ss Network interface 282The 283.Nm vke 284driver supports up to 16 virtual network interfaces which are associated with 285.Xr tap 4 286devices on the host. 287For each 288.Nm vke 289device, the per-interface read only 290.Xr sysctl 3 291variable 292.Va hw.vke Ns Em X Ns Va .tap_unit 293holds the unit number of the associated 294.Xr tap 4 295device. 296.Pp 297By default, half of the total mbuf clusters available is distributed equally 298among all the vke devices up to 256. 299This can be overridden with the tunable 300.Va hw.vke.max_ringsize . 301Take into account the number passed will be aligned to the lower power of two. 302.Sh SIGNALS 303The virtual kernel only enables 304.Dv SIGQUIT 305and 306.Dv SIGTERM 307while operating in regular console mode. 308Sending 309.Ql \&^\e 310.Pq Dv SIGQUIT 311to the virtual kernel causes the virtual kernel to enter its internal 312.Xr ddb 4 313debugger and re-enable all other terminal signals. 314Sending 315.Dv SIGTERM 316to the virtual kernel triggers a clean shutdown by passing a 317.Dv SIGUSR2 318to the virtual kernel's 319.Xr init 8 320process. 321.Sh DEBUGGING 322It is possible to directly gdb the virtual kernel's process. 323It is recommended that you do a 324.Ql handle SIGSEGV noprint 325to ignore page faults processed by the virtual kernel itself and 326.Ql handle SIGUSR1 noprint 327to ignore signals used for simulating inter-processor interrupts. 328.Sh PROFILING 329To compile a vkernel with profiling support, the 330.Va CONFIGARGS 331variable needs to be used to pass 332.Fl p 333to 334.Xr config 8 . 335.Bd -literal 336cd /usr/src 337make -DNO_MODULES CONFIGARGS=-p buildkernel KERNCONF=VKERNEL64 338.Ed 339.Sh FILES 340.Bl -tag -width ".It Pa /sys/config/VKERNEL64" -compact 341.It Pa /dev/vcdX 342.Nm vcd 343device nodes 344.It Pa /dev/vkdX 345.Nm vkd 346device nodes 347.It Pa /sys/config/VKERNEL64 348.El 349.Pp 350.Nm 351configuration file, for 352.Xr config 8 . 353.Sh CONFIGURATION FILES 354Your virtual kernel is a complete 355.Dx 356system, but you might not want to run all the services a normal kernel runs. 357Here is what a typical virtual kernel's 358.Pa /etc/rc.conf 359file looks like, with some additional possibilities commented out. 360.Bd -literal 361hostname="vkernel" 362network_interfaces="lo0 vke0" 363ifconfig_vke0="DHCP" 364sendmail_enable="NO" 365#syslog_enable="NO" 366blanktime="NO" 367.Ed 368.Sh DISKLESS OPERATION 369To boot a 370.Nm 371from a NFS root, a number of tunables need to be set: 372.Bl -tag -width indent 373.It Va boot.netif.ip 374IP address to be set in the vkernel interface. 375.It Va boot.netif.netmask 376Netmask for the IP to be set. 377.It Va boot.netif.name 378Network interface name inside the vkernel. 379.It Va boot.nfsroot.server 380Host running 381.Xr nfsd 8 . 382.It Va boot.nfsroot.path 383Host path where a world and distribution 384targets are properly installed. 385.El 386.Pp 387See an example on how to boot a diskless 388.Nm 389in the 390.Sx EXAMPLES 391section. 392.Sh EXAMPLES 393A couple of steps are necessary in order to prepare the system to build and 394run a virtual kernel. 395.Ss Setting up the filesystem 396The 397.Nm 398architecture needs a number of files which reside in 399.Pa /var/vkernel . 400Since these files tend to get rather big and the 401.Pa /var 402partition is usually of limited size, we recommend the directory to be 403created in the 404.Pa /home 405partition with a link to it in 406.Pa /var : 407.Bd -literal 408mkdir -p /home/var.vkernel/boot 409ln -s /home/var.vkernel /var/vkernel 410.Ed 411.Pp 412Next, a filesystem image to be used by the virtual kernel has to be 413created and populated (assuming world has been built previously). 414If the image is created on a UFS filesystem you might want to pre-zero it. 415On a HAMMER filesystem you should just truncate-extend to the image size 416as HAMMER does not re-use data blocks already present in the file. 417.Bd -literal 418vnconfig -c -S 2g -T vn0 /var/vkernel/rootimg.01 419disklabel -r -w vn0s0 auto 420disklabel -e vn0s0 # add `a' partition with fstype `4.2BSD' 421newfs /dev/vn0s0a 422mount /dev/vn0s0a /mnt 423cd /usr/src 424make installworld DESTDIR=/mnt 425cd etc 426make distribution DESTDIR=/mnt 427echo '/dev/vkd0s0a / ufs rw 1 1' >/mnt/etc/fstab 428echo 'proc /proc procfs rw 0 0' >>/mnt/etc/fstab 429.Ed 430.Pp 431Edit 432.Pa /mnt/etc/ttys 433and replace the 434.Li console 435entry with the following line and turn off all other gettys. 436.Bd -literal 437console "/usr/libexec/getty Pc" cons25 on secure 438.Ed 439.Pp 440Replace 441.Li \&Pc 442with 443.Li al.Pc 444if you would like to automatically log in as root. 445.Pp 446Then, unmount the disk. 447.Bd -literal 448umount /mnt 449vnconfig -u vn0 450.Ed 451.Ss Compiling the virtual kernel 452In order to compile a virtual kernel use the 453.Li VKERNEL64 454kernel configuration file residing in 455.Pa /sys/config 456(or a configuration file derived thereof): 457.Bd -literal 458cd /usr/src 459make -DNO_MODULES buildkernel KERNCONF=VKERNEL64 460make -DNO_MODULES installkernel KERNCONF=VKERNEL64 DESTDIR=/var/vkernel 461.Ed 462.Ss Enabling virtual kernel operation 463A special 464.Xr sysctl 8 , 465.Va vm.vkernel_enable , 466must be set to enable 467.Nm 468operation: 469.Bd -literal 470sysctl vm.vkernel_enable=1 471.Ed 472.Ss Configuring the network on the host system 473In order to access a network interface of the host system from the 474.Nm , 475you must add the interface to a 476.Xr bridge 4 477device which will then be passed to the 478.Fl I 479option: 480.Bd -literal 481kldload if_bridge.ko 482kldload if_tap.ko 483ifconfig bridge0 create 484ifconfig bridge0 addm re0 # assuming re0 is the host's interface 485ifconfig bridge0 up 486.Ed 487.Ss Running the kernel 488Finally, the virtual kernel can be run: 489.Bd -literal 490cd /var/vkernel 491\&./boot/kernel/kernel -m 64m -r rootimg.01 -I auto:bridge0 492.Ed 493.Pp 494You can issue the 495.Xr reboot 8 , 496.Xr halt 8 , 497or 498.Xr shutdown 8 499commands from inside a virtual kernel. 500After doing a clean shutdown the 501.Xr reboot 8 502command will re-exec the virtual kernel binary while the other two will 503cause the virtual kernel to exit. 504.Ss Diskless operation 505Booting a 506.Nm 507with a 508.Xr vknetd 8 509network configuration: 510.Bd -literal 511\&./boot/kernel/kernel -m 64m -m -i memimg.0000 -I /var/run/vknet 512 -e boot.netif.ip=172.1.0.4 513 -e boot.netif.netmask=255.255.0.0 514 -e boot.netif.name=vke0 515 -e boot.nfsroot.server=172.1.0.1 516 -e boot.nfsroot.path=/home/vkernel/vkdiskless 517.Ed 518.Sh BUILDING THE WORLD UNDER A VKERNEL 519The virtual kernel platform does not have all the header files expected 520by a world build, so the easiest thing to do right now is to specify a 521pc64 (in a 64 bit vkernel) target when building the world under a virtual 522kernel, like this: 523.Bd -literal 524vkernel# make MACHINE_PLATFORM=pc64 buildworld 525vkernel# make MACHINE_PLATFORM=pc64 installworld 526.Ed 527.Sh SEE ALSO 528.Xr vknet 1 , 529.Xr bridge 4 , 530.Xr ifmedia 4 , 531.Xr tap 4 , 532.Xr vn 4 , 533.Xr sysctl.conf 5 , 534.Xr build 7 , 535.Xr config 8 , 536.Xr disklabel 8 , 537.Xr ifconfig 8 , 538.Xr vknetd 8 , 539.Xr vnconfig 8 540.Rs 541.%A Aggelos Economopoulos 542.%D March 2007 543.%T "A Peek at the DragonFly Virtual Kernel" 544.Re 545.Sh HISTORY 546Virtual kernels were introduced in 547.Dx 1.7 . 548.Sh AUTHORS 549.An -nosplit 550.An Matt Dillon 551thought up and implemented the 552.Nm 553architecture and wrote the 554.Nm vkd 555device driver. 556.An Sepherosa Ziehau 557wrote the 558.Nm vke 559device driver. 560This manual page was written by 561.An Sascha Wildner . 562