1.\" 2.\" Copyright (c) 2006, 2007 3.\" The DragonFly Project. All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 9.\" 1. Redistributions of source code must retain the above copyright 10.\" notice, this list of conditions and the following disclaimer. 11.\" 2. Redistributions in binary form must reproduce the above copyright 12.\" notice, this list of conditions and the following disclaimer in 13.\" the documentation and/or other materials provided with the 14.\" distribution. 15.\" 3. Neither the name of The DragonFly Project nor the names of its 16.\" contributors may be used to endorse or promote products derived 17.\" from this software without specific, prior written permission. 18.\" 19.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 20.\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 21.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS 22.\" FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 23.\" COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, 24.\" INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING, 25.\" BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 26.\" LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED 27.\" AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 28.\" OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT 29.\" OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 30.\" SUCH DAMAGE. 31.\" 32.Dd June 20, 2015 33.Dt VKERNEL 7 34.Os 35.Sh NAME 36.Nm vkernel , 37.Nm vcd , 38.Nm vkd , 39.Nm vke 40.Nd virtual kernel architecture 41.Sh SYNOPSIS 42.Cd "platform vkernel64 # for 64 bit vkernels" 43.Cd "device vcd" 44.Cd "device vkd" 45.Cd "device vke" 46.Pp 47.Pa /var/vkernel/boot/kernel/kernel 48.Op Fl hdsUv 49.Op Fl c Ar file 50.Op Fl e Ar name Ns = Ns Li value : Ns Ar name Ns = Ns Li value : Ns ... 51.Op Fl i Ar file 52.Op Fl I Ar interface Ns Op Ar :address1 Ns Oo Ar :address2 Oc Ns Oo Ar /netmask Oc Ns Oo Ar =mac Oc 53.Op Fl l Ar cpulock 54.Op Fl m Ar size 55.Op Fl n Ar numcpus Ns Op Ar :lbits Ns Oo Ar :cbits Oc 56.Op Fl p Ar pidfile 57.Op Fl r Ar file Ns Op Ar :serno 58.Sh DESCRIPTION 59The 60.Nm 61architecture allows for running 62.Dx 63kernels in userland. 64.Pp 65The following options are available: 66.Bl -tag -width ".Fl m Ar size" 67.It Fl c Ar file 68Specify a readonly CD-ROM image 69.Ar file 70to be used by the kernel, with the first 71.Fl c 72option defining 73.Li vcd0 , 74the second one 75.Li vcd1 , 76and so on. 77The first 78.Fl r 79or 80.Fl c 81option specified on the command line will be the boot disk. 82The CD9660 filesystem is assumed when booting from this media. 83.It Fl d 84Disables hardware pagetable for 85.Nm . 86.It Fl e Ar name Ns = Ns Li value : Ns Ar name Ns = Ns Li value : Ns ... 87Specify an environment to be used by the kernel. 88This option can be specified more than once. 89.It Fl h 90Shows a list of available options, each with a short description. 91.It Fl i Ar file 92Specify a memory image 93.Ar file 94to be used by the virtual kernel. 95If no 96.Fl i 97option is given, the kernel will generate a name of the form 98.Pa /var/vkernel/memimg.XXXXXX , 99with the trailing 100.Ql X Ns s 101being replaced by a sequential number, e.g.\& 102.Pa memimg.000001 . 103.It Fl I Ar interface Ns Op Ar :address1 Ns Oo Ar :address2 Oc Ns Oo Ar /netmask Oc Ns Oo Ar =MAC Oc 104Create a virtual network device, with the first 105.Fl I 106option defining 107.Li vke0 , 108the second one 109.Li vke1 , 110and so on. 111.Pp 112The 113.Ar interface 114argument is the name of a 115.Xr tap 4 116device node or the path to a 117.Xr vknetd 8 118socket. 119The 120.Pa /dev/ 121path prefix does not have to be specified and will be automatically prepended 122for a device node. 123Specifying 124.Cm auto 125will pick the first unused 126.Xr tap 4 127device. 128.Pp 129The 130.Ar address1 131and 132.Ar address2 133arguments are the IP addresses of the 134.Xr tap 4 135and 136.Nm vke 137interfaces. 138Optionally, 139.Ar address1 140may be of the form 141.Li bridge Ns Em X 142in which case the 143.Xr tap 4 144interface is added to the specified 145.Xr bridge 4 146interface. 147The 148.Nm vke 149address is not assigned until the interface is brought up in the guest. 150.Pp 151The 152.Ar netmask 153argument applies to all interfaces for which an address is specified. 154.Pp 155The 156.Ar MAC 157argument is the MAC address of the 158.Xr vke 4 159interface. 160If not specified, a pseudo-random one will be generated. 161.Pp 162When running multiple vkernels it is often more convenient to simply 163connect to a 164.Xr vknetd 8 165socket and let vknetd deal with the tap and/or bridge. 166An example of this would be 167.Pa /var/run/vknet:0.0.0.0:10.2.0.2/16 . 168.It Fl l Ar cpulock 169Specify which, if any, real CPUs to lock virtual CPUs to. 170.Ar cpulock 171is one of 172.Cm any , 173.Cm map Ns Op Ns , Ns Ar startCPU , 174or 175.Ar CPU . 176.Pp 177.Cm any 178does not map virtual CPUs to real CPUs. 179This is the default. 180.Pp 181.Cm map Ns Op Ns , Ns Ar startCPU 182maps each virtual CPU to a real CPU starting with real CPU 0 or 183.Ar startCPU 184if specified. 185.Pp 186.Ar CPU 187locks all virtual CPUs to the real CPU specified by 188.Ar CPU . 189.It Fl m Ar size 190Specify the amount of memory to be used by the kernel in bytes, 191.Cm K 192.Pq kilobytes , 193.Cm M 194.Pq megabytes 195or 196.Cm G 197.Pq gigabytes . 198Lowercase versions of 199.Cm K , M , 200and 201.Cm G 202are allowed. 203.It Fl n Ar numcpus Ns Op Ar :lbits Ns Oo Ar :cbits Oc 204.Ar numcpus 205specifies the number of CPUs you wish to emulate. 206Up to 16 CPUs are supported with 2 being the default unless otherwise 207specified. 208.Pp 209.Ar lbits 210specifies the number of bits within APICID(=CPUID) needed for representing 211the logical ID. 212Controls the number of threads/core (0 bits - 1 thread, 1 bit - 2 threads). 213This parameter is optional (mandatory only if 214.Ar cbits 215is specified). 216.Pp 217.Ar cbits 218specifies the number of bits within APICID(=CPUID) needed for representing 219the core ID. 220Controls the number of core/package (0 bits - 1 core, 1 bit - 2 cores). 221This parameter is optional. 222.It Fl p Ar pidfile 223Specify a pidfile in which to store the process ID. 224Scripts can use this file to locate the vkernel pid for the purpose of 225shutting down or killing it. 226.Pp 227The vkernel will hold a lock on the pidfile while running. 228Scripts may test for the lock to determine if the pidfile is valid or 229stale so as to avoid accidentally killing a random process. 230Something like '/usr/bin/lockf -ks -t 0 pidfile echo -n' may be used 231to test the lock. 232A non-zero exit code indicates that the pidfile represents a running 233vkernel. 234.Pp 235An error is issued and the vkernel exits if this file cannot be opened for 236writing or if it is already locked by an active vkernel process. 237.It Fl r Ar file Ns Op Ar :serno 238Specify a R/W disk image 239.Ar file 240to be used by the kernel, with the first 241.Fl r 242option defining 243.Li vkd0 , 244the second one 245.Li vkd1 , 246and so on. 247A serial number for the virtual disk can be specified in 248.Ar serno . 249.Pp 250The first 251.Fl r 252or 253.Fl c 254option specified on the command line will be the boot disk. 255.It Fl s 256Boot into single-user mode. 257.It Fl U 258Enable writing to kernel memory and module loading. 259By default, those are disabled for security reasons. 260.It Fl v 261Turn on verbose booting. 262.El 263.Sh DEVICES 264A number of virtual device drivers exist to supplement the virtual kernel. 265.Ss Disk device 266The 267.Nm vkd 268driver allows for up to 16 269.Xr vn 4 270based disk devices. 271The root device will be 272.Li vkd0 273(see 274.Sx EXAMPLES 275for further information on how to prepare a root image). 276.Ss CD-ROM device 277The 278.Nm vcd 279driver allows for up to 16 virtual CD-ROM devices. 280Basically this is a read only 281.Nm vkd 282device with a block size of 2048. 283.Ss Network interface 284The 285.Nm vke 286driver supports up to 16 virtual network interfaces which are associated with 287.Xr tap 4 288devices on the host. 289For each 290.Nm vke 291device, the per-interface read only 292.Xr sysctl 3 293variable 294.Va hw.vke Ns Em X Ns Va .tap_unit 295holds the unit number of the associated 296.Xr tap 4 297device. 298.Pp 299By default, half of the total mbuf clusters available is distributed equally 300among all the vke devices up to 256. 301This can be overridden with the tunable 302.Va hw.vke.max_ringsize . 303Take into account the number passed will be aligned to the lower power of two. 304.Sh SIGNALS 305The virtual kernel only enables 306.Dv SIGQUIT 307and 308.Dv SIGTERM 309while operating in regular console mode. 310Sending 311.Ql \&^\e 312.Pq Dv SIGQUIT 313to the virtual kernel causes the virtual kernel to enter its internal 314.Xr ddb 4 315debugger and re-enable all other terminal signals. 316Sending 317.Dv SIGTERM 318to the virtual kernel triggers a clean shutdown by passing a 319.Dv SIGUSR2 320to the virtual kernel's 321.Xr init 8 322process. 323.Sh DEBUGGING 324It is possible to directly gdb the virtual kernel's process. 325It is recommended that you do a 326.Ql handle SIGSEGV noprint 327to ignore page faults processed by the virtual kernel itself and 328.Ql handle SIGUSR1 noprint 329to ignore signals used for simulating inter-processor interrupts. 330.Sh PROFILING 331To compile a vkernel with profiling support, the 332.Va CONFIGARGS 333variable needs to be used to pass 334.Fl p 335to 336.Xr config 8 . 337.Bd -literal 338cd /usr/src 339make -DNO_MODULES CONFIGARGS=-p buildkernel KERNCONF=VKERNEL64 340.Ed 341.Sh FILES 342.Bl -tag -width ".It Pa /sys/config/VKERNEL64" -compact 343.It Pa /dev/vcdX 344.Nm vcd 345device nodes 346.It Pa /dev/vkdX 347.Nm vkd 348device nodes 349.It Pa /sys/config/VKERNEL64 350.El 351.Pp 352.Nm 353configuration file, for 354.Xr config 8 . 355.Sh CONFIGURATION FILES 356Your virtual kernel is a complete 357.Dx 358system, but you might not want to run all the services a normal kernel runs. 359Here is what a typical virtual kernel's 360.Pa /etc/rc.conf 361file looks like, with some additional possibilities commented out. 362.Bd -literal 363hostname="vkernel" 364network_interfaces="lo0 vke0" 365ifconfig_vke0="DHCP" 366sendmail_enable="NO" 367#syslog_enable="NO" 368blanktime="NO" 369.Ed 370.Sh DISKLESS OPERATION 371To boot a 372.Nm 373from a NFS root, a number of tunables need to be set: 374.Bl -tag -width indent 375.It Va boot.netif.ip 376IP address to be set in the vkernel interface. 377.It Va boot.netif.netmask 378Netmask for the IP to be set. 379.It Va boot.netif.name 380Network interface name inside the vkernel. 381.It Va boot.nfsroot.server 382Host running 383.Xr nfsd 8 . 384.It Va boot.nfsroot.path 385Host path where a world and distribution 386targets are properly installed. 387.El 388.Pp 389See an example on how to boot a diskless 390.Nm 391in the 392.Sx EXAMPLES 393section. 394.Sh EXAMPLES 395A couple of steps are necessary in order to prepare the system to build and 396run a virtual kernel. 397.Ss Setting up the filesystem 398The 399.Nm 400architecture needs a number of files which reside in 401.Pa /var/vkernel . 402Since these files tend to get rather big and the 403.Pa /var 404partition is usually of limited size, we recommend the directory to be 405created in the 406.Pa /home 407partition with a link to it in 408.Pa /var : 409.Bd -literal 410mkdir -p /home/var.vkernel/boot 411ln -s /home/var.vkernel /var/vkernel 412.Ed 413.Pp 414Next, a filesystem image to be used by the virtual kernel has to be 415created and populated (assuming world has been built previously). 416If the image is created on a UFS filesystem you might want to pre-zero it. 417On a HAMMER filesystem you should just truncate-extend to the image size 418as HAMMER does not re-use data blocks already present in the file. 419.Bd -literal 420vnconfig -c -S 2g -T vn0 /var/vkernel/rootimg.01 421disklabel -r -w vn0s0 auto 422disklabel -e vn0s0 # add `a' partition with fstype `4.2BSD' 423newfs /dev/vn0s0a 424mount /dev/vn0s0a /mnt 425cd /usr/src 426make installworld DESTDIR=/mnt 427cd etc 428make distribution DESTDIR=/mnt 429echo '/dev/vkd0s0a / ufs rw 1 1' >/mnt/etc/fstab 430echo 'proc /proc procfs rw 0 0' >>/mnt/etc/fstab 431.Ed 432.Pp 433Edit 434.Pa /mnt/etc/ttys 435and replace the 436.Li console 437entry with the following line and turn off all other gettys. 438.Bd -literal 439console "/usr/libexec/getty Pc" cons25 on secure 440.Ed 441.Pp 442Replace 443.Li \&Pc 444with 445.Li al.Pc 446if you would like to automatically log in as root. 447.Pp 448Then, unmount the disk. 449.Bd -literal 450umount /mnt 451vnconfig -u vn0 452.Ed 453.Ss Compiling the virtual kernel 454In order to compile a virtual kernel use the 455.Li VKERNEL64 456kernel configuration file residing in 457.Pa /sys/config 458(or a configuration file derived thereof): 459.Bd -literal 460cd /usr/src 461make -DNO_MODULES buildkernel KERNCONF=VKERNEL64 462make -DNO_MODULES installkernel KERNCONF=VKERNEL64 DESTDIR=/var/vkernel 463.Ed 464.Ss Enabling virtual kernel operation 465A special 466.Xr sysctl 8 , 467.Va vm.vkernel_enable , 468must be set to enable 469.Nm 470operation: 471.Bd -literal 472sysctl vm.vkernel_enable=1 473.Ed 474.Ss Configuring the network on the host system 475In order to access a network interface of the host system from the 476.Nm , 477you must add the interface to a 478.Xr bridge 4 479device which will then be passed to the 480.Fl I 481option: 482.Bd -literal 483kldload if_bridge.ko 484kldload if_tap.ko 485ifconfig bridge0 create 486ifconfig bridge0 addm re0 # assuming re0 is the host's interface 487ifconfig bridge0 up 488.Ed 489.Ss Running the kernel 490Finally, the virtual kernel can be run: 491.Bd -literal 492cd /var/vkernel 493\&./boot/kernel/kernel -m 64m -r rootimg.01 -I auto:bridge0 494.Ed 495.Pp 496You can issue the 497.Xr reboot 8 , 498.Xr halt 8 , 499or 500.Xr shutdown 8 501commands from inside a virtual kernel. 502After doing a clean shutdown the 503.Xr reboot 8 504command will re-exec the virtual kernel binary while the other two will 505cause the virtual kernel to exit. 506.Ss Diskless operation 507Booting a 508.Nm 509with a 510.Xr vknetd 8 511network configuration: 512.Bd -literal 513\&./boot/kernel/kernel -m 64m -m -i memimg.0000 -I /var/run/vknet 514 -e boot.netif.ip=172.1.0.4 515 -e boot.netif.netmask=255.255.0.0 516 -e boot.netif.name=vke0 517 -e boot.nfsroot.server=172.1.0.1 518 -e boot.nfsroot.path=/home/vkernel/vkdiskless 519.Ed 520.Sh BUILDING THE WORLD UNDER A VKERNEL 521The virtual kernel platform does not have all the header files expected 522by a world build, so the easiest thing to do right now is to specify a 523pc64 (in a 64 bit vkernel) target when building the world under a virtual 524kernel, like this: 525.Bd -literal 526vkernel# make MACHINE_PLATFORM=pc64 buildworld 527vkernel# make MACHINE_PLATFORM=pc64 installworld 528.Ed 529.Sh SEE ALSO 530.Xr vknet 1 , 531.Xr bridge 4 , 532.Xr ifmedia 4 , 533.Xr tap 4 , 534.Xr vn 4 , 535.Xr sysctl.conf 5 , 536.Xr build 7 , 537.Xr config 8 , 538.Xr disklabel 8 , 539.Xr ifconfig 8 , 540.Xr vknetd 8 , 541.Xr vnconfig 8 542.Rs 543.%A Aggelos Economopoulos 544.%D March 2007 545.%T "A Peek at the DragonFly Virtual Kernel" 546.Re 547.Sh HISTORY 548Virtual kernels were introduced in 549.Dx 1.7 . 550.Sh AUTHORS 551.An -nosplit 552.An Matt Dillon 553thought up and implemented the 554.Nm 555architecture and wrote the 556.Nm vkd 557device driver. 558.An Sepherosa Ziehau 559wrote the 560.Nm vke 561device driver. 562This manual page was written by 563.An Sascha Wildner . 564