1.\" 2.\" Copyright (c) 2006, 2007 3.\" The DragonFly Project. All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 9.\" 1. Redistributions of source code must retain the above copyright 10.\" notice, this list of conditions and the following disclaimer. 11.\" 2. Redistributions in binary form must reproduce the above copyright 12.\" notice, this list of conditions and the following disclaimer in 13.\" the documentation and/or other materials provided with the 14.\" distribution. 15.\" 3. Neither the name of The DragonFly Project nor the names of its 16.\" contributors may be used to endorse or promote products derived 17.\" from this software without specific, prior written permission. 18.\" 19.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 20.\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 21.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS 22.\" FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 23.\" COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, 24.\" INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING, 25.\" BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 26.\" LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED 27.\" AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 28.\" OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT 29.\" OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 30.\" SUCH DAMAGE. 31.\" 32.Dd December 10, 2014 33.Dt VKERNEL 7 34.Os 35.Sh NAME 36.Nm vkernel , 37.Nm vcd , 38.Nm vkd , 39.Nm vke 40.Nd virtual kernel architecture 41.Sh SYNOPSIS 42.Cd "platform vkernel # for 32 bit vkernels" 43.Cd "platform vkernel64 # for 64 bit vkernels" 44.Cd "device vcd" 45.Cd "device vkd" 46.Cd "device vke" 47.Pp 48.Pa /var/vkernel/boot/kernel/kernel 49.Op Fl hsUvd 50.Op Fl c Ar file 51.Op Fl e Ar name Ns = Ns Li value : Ns Ar name Ns = Ns Li value : Ns ... 52.Op Fl i Ar file 53.Op Fl I Ar interface Ns Op Ar :address1 Ns Oo Ar :address2 Oc Ns Oo Ar /netmask Oc Ns Oo Ar =mac Oc 54.Op Fl l Ar cpulock 55.Op Fl m Ar size 56.Op Fl n Ar numcpus Ns Op Ar :lbits Ns Oo Ar :cbits Oc 57.Op Fl p Ar pidfile 58.Op Fl r Ar file Ns Op Ar :serno 59.Sh DESCRIPTION 60The 61.Nm 62architecture allows for running 63.Dx 64kernels in userland. 65.Pp 66The following options are available: 67.Bl -tag -width ".Fl m Ar size" 68.It Fl c Ar file 69Specify a readonly CD-ROM image 70.Ar file 71to be used by the kernel, with the first 72.Fl c 73option defining 74.Li vcd0 , 75the second one 76.Li vcd1 , 77and so on. 78The first 79.Fl r 80or 81.Fl c 82option specified on the command line will be the boot disk. 83The CD9660 filesystem is assumed when booting from this media. 84.It Fl e Ar name Ns = Ns Li value : Ns Ar name Ns = Ns Li value : Ns ... 85Specify an environment to be used by the kernel. 86This option can be specified more than once. 87.It Fl h 88Shows a list of available options, each with a short description. 89.It Fl i Ar file 90Specify a memory image 91.Ar file 92to be used by the virtual kernel. 93If no 94.Fl i 95option is given, the kernel will generate a name of the form 96.Pa /var/vkernel/memimg.XXXXXX , 97with the trailing 98.Ql X Ns s 99being replaced by a sequential number, e.g.\& 100.Pa memimg.000001 . 101.It Fl I Ar interface Ns Op Ar :address1 Ns Oo Ar :address2 Oc Ns Oo Ar /netmask Oc Ns Oo Ar =MAC Oc 102Create a virtual network device, with the first 103.Fl I 104option defining 105.Li vke0 , 106the second one 107.Li vke1 , 108and so on. 109.Pp 110The 111.Ar interface 112argument is the name of a 113.Xr tap 4 114device node or the path to a 115.Xr vknetd 8 116socket. 117The 118.Pa /dev/ 119path prefix does not have to be specified and will be automatically prepended 120for a device node. 121Specifying 122.Cm auto 123will pick the first unused 124.Xr tap 4 125device. 126.Pp 127The 128.Ar address1 129and 130.Ar address2 131arguments are the IP addresses of the 132.Xr tap 4 133and 134.Nm vke 135interfaces. 136Optionally, 137.Ar address1 138may be of the form 139.Li bridge Ns Em X 140in which case the 141.Xr tap 4 142interface is added to the specified 143.Xr bridge 4 144interface. 145The 146.Nm vke 147address is not assigned until the interface is brought up in the guest. 148.Pp 149The 150.Ar netmask 151argument applies to all interfaces for which an address is specified. 152.Pp 153The 154.Ar MAC 155argument is the MAC address of the 156.Xr vke 4 157interface. 158If not specified, a pseudo-random one will be generated. 159.Pp 160When running multiple vkernels it is often more convenient to simply 161connect to a 162.Xr vknetd 8 163socket and let vknetd deal with the tap and/or bridge. 164An example of this would be 165.Pa /var/run/vknet:0.0.0.0:10.2.0.2/16 . 166.It Fl l Ar cpulock 167Specify which, if any, real CPUs to lock virtual CPUs to. 168.Ar cpulock 169is one of 170.Cm any , 171.Cm map Ns Op Ns , Ns Ar startCPU , 172or 173.Ar CPU . 174.Pp 175.Cm any 176does not map virtual CPUs to real CPUs. 177This is the default. 178.Pp 179.Cm map Ns Op Ns , Ns Ar startCPU 180maps each virtual CPU to a real CPU starting with real CPU 0 or 181.Ar startCPU 182if specified. 183.Pp 184.Ar CPU 185locks all virtual CPUs to the real CPU specified by 186.Ar CPU . 187.It Fl m Ar size 188Specify the amount of memory to be used by the kernel in bytes, 189.Cm K 190.Pq kilobytes , 191.Cm M 192.Pq megabytes 193or 194.Cm G 195.Pq gigabytes . 196Lowercase versions of 197.Cm K , M , 198and 199.Cm G 200are allowed. 201.It Fl n Ar numcpus Ns Op Ar :lbits Ns Oo Ar :cbits Oc 202.Ar numcpus 203specifies the number of CPUs you wish to emulate. 204Up to 16 CPUs are supported with 2 being the default unless otherwise 205specified. 206.Ar lbits 207specifies the number of bits within APICID(=CPUID) needed for representing 208the logical ID. 209Controls the number of threads/core (0bits - 1 thread, 1bit - 2 threads). 210This parameter is optional (mandatory only if 211.Ar cbits 212is specified). 213.Ar cbits 214specifies the number of bits within APICID(=CPUID) needed for representing 215the core ID. 216Controls the number of core/package (0bits - 1 core, 1bit - 2 cores). 217This parameter is optional. 218.It Fl p Ar pidfile 219Specify a pidfile in which to store the process ID. 220Scripts can use this file to locate the vkernel pid for the purpose of 221shutting down or killing it. 222.Pp 223The vkernel will hold a lock on the pidfile while running. 224Scripts may test for the lock to determine if the pidfile is valid or 225stale so as to avoid accidentally killing a random process. 226Something like '/usr/bin/lockf -ks -t 0 pidfile echo -n' may be used 227to test the lock. 228A non-zero exit code indicates that the pidfile represents a running 229vkernel. 230.Pp 231An error is issued and the vkernel exits if this file cannot be opened for 232writing or if it is already locked by an active vkernel process. 233.It Fl r Ar file Ns Op Ar :serno 234Specify a R/W disk image 235.Ar file 236to be used by the kernel, with the first 237.Fl r 238option defining 239.Li vkd0 , 240the second one 241.Li vkd1 , 242and so on. 243A serial number for the virtual disk can be specified in 244.Ar serno . 245.Pp 246The first 247.Fl r 248or 249.Fl c 250option specified on the command line will be the boot disk. 251.It Fl s 252Boot into single-user mode. 253.It Fl d 254Disables hardware pagetable for vkernel. 255.It Fl U 256Enable writing to kernel memory and module loading. 257By default, those are disabled for security reasons. 258.It Fl v 259Turn on verbose booting. 260.El 261.Sh DEVICES 262A number of virtual device drivers exist to supplement the virtual kernel. 263.Ss Disk device 264The 265.Nm vkd 266driver allows for up to 16 267.Xr vn 4 268based disk devices. 269The root device will be 270.Li vkd0 271(see 272.Sx EXAMPLES 273for further information on how to prepare a root image). 274.Ss CD-ROM device 275The 276.Nm vcd 277driver allows for up to 16 virtual CD-ROM devices. 278Basically this is a read only 279.Nm vkd 280device with a block size of 2048. 281.Ss Network interface 282The 283.Nm vke 284driver supports up to 16 virtual network interfaces which are associated with 285.Xr tap 4 286devices on the host. 287For each 288.Nm vke 289device, the per-interface read only 290.Xr sysctl 3 291variable 292.Va hw.vke Ns Em X Ns Va .tap_unit 293holds the unit number of the associated 294.Xr tap 4 295device. 296.Pp 297By default, half of the total mbuf clusters available is distributed equally 298among all the vke devices up to 256. 299This can be overridden with the tunable 300.Va hw.vke.max_ringsize . 301Take into account the number passed will be aligned to the lower power of two. 302.Sh SIGNALS 303The virtual kernel only enables 304.Dv SIGQUIT 305and 306.Dv SIGTERM 307while operating in regular console mode. 308Sending 309.Ql \&^\e 310.Pq Dv SIGQUIT 311to the virtual kernel causes the virtual kernel to enter its internal 312.Xr ddb 4 313debugger and re-enable all other terminal signals. 314Sending 315.Dv SIGTERM 316to the virtual kernel triggers a clean shutdown by passing a 317.Dv SIGUSR2 318to the virtual kernel's 319.Xr init 8 320process. 321.Sh DEBUGGING 322It is possible to directly gdb the virtual kernel's process. 323It is recommended that you do a 324.Ql handle SIGSEGV noprint 325to ignore page faults processed by the virtual kernel itself and 326.Ql handle SIGUSR1 noprint 327to ignore signals used for simulating inter-processor interrupts. 328.Sh PROFILING 329To compile a vkernel with profiling support, the 330.Va CONFIGARGS 331variable needs to be used to pass 332.Fl p 333to 334.Xr config 8 . 335.Bd -literal 336cd /usr/src 337make -DNO_MODULES CONFIGARGS=-p buildkernel KERNCONF=VKERNEL 338.Ed 339.Sh FILES 340.Bl -tag -width ".It Pa /sys/config/VKERNEL64" -compact 341.It Pa /dev/vcdX 342.Nm vcd 343device nodes 344.It Pa /dev/vkdX 345.Nm vkd 346device nodes 347.It Pa /sys/config/VKERNEL 348.It Pa /sys/config/VKERNEL64 349.El 350.Pp 351Per architecture 352.Nm 353configuration files, for 354.Xr config 8 . 355.Sh CONFIGURATION FILES 356Your virtual kernel is a complete 357.Dx 358system, but you might not want to run all the services a normal kernel runs. 359Here is what a typical virtual kernel's 360.Pa /etc/rc.conf 361file looks like, with some additional possibilities commented out. 362.Bd -literal 363hostname="vkernel" 364network_interfaces="lo0 vke0" 365ifconfig_vke0="DHCP" 366sendmail_enable="NO" 367#syslog_enable="NO" 368blanktime="NO" 369.Ed 370.Sh DISKLESS OPERATION 371To boot a 372.Nm 373from a NFS root, a number of tunables need to be set: 374.Bl -tag -width indent 375.It Va boot.netif.ip 376IP address to be set in the vkernel interface. 377.It Va boot.netif.netmask 378Netmask for the IP to be set. 379.It Va boot.netif.name 380Network interface name inside the vkernel. 381.It Va boot.nfsroot.server 382Host running 383.Xr nfsd 8 . 384.It Va boot.nfsroot.path 385Host path where a world and distribution 386targets are properly installed. 387.El 388.Pp 389See an example on how to boot a diskless 390.Nm 391in the 392.Sx EXAMPLES 393section. 394.Sh EXAMPLES 395A couple of steps are necessary in order to prepare the system to build and 396run a virtual kernel. 397.Ss Setting up the filesystem 398The 399.Nm 400architecture needs a number of files which reside in 401.Pa /var/vkernel . 402Since these files tend to get rather big and the 403.Pa /var 404partition is usually of limited size, we recommend the directory to be 405created in the 406.Pa /home 407partition with a link to it in 408.Pa /var : 409.Bd -literal 410mkdir -p /home/var.vkernel/boot 411ln -s /home/var.vkernel /var/vkernel 412.Ed 413.Pp 414Next, a filesystem image to be used by the virtual kernel has to be 415created and populated (assuming world has been built previously). 416If the image is created on a UFS filesystem you might want to pre-zero it. 417On a HAMMER filesystem you should just truncate-extend to the image size 418as HAMMER does not re-use data blocks already present in the file. 419.Bd -literal 420vnconfig -c -S 2g -T vn0 /var/vkernel/rootimg.01 421disklabel -r -w vn0s0 auto 422disklabel -e vn0s0 # add `a' partition with fstype `4.2BSD' 423newfs /dev/vn0s0a 424mount /dev/vn0s0a /mnt 425cd /usr/src 426make installworld DESTDIR=/mnt 427cd etc 428make distribution DESTDIR=/mnt 429echo '/dev/vkd0s0a / ufs rw 1 1' >/mnt/etc/fstab 430echo 'proc /proc procfs rw 0 0' >>/mnt/etc/fstab 431.Ed 432.Pp 433Edit 434.Pa /mnt/etc/ttys 435and replace the 436.Li console 437entry with the following line and turn off all other gettys. 438.Bd -literal 439console "/usr/libexec/getty Pc" cons25 on secure 440.Ed 441.Pp 442Replace 443.Li \&Pc 444with 445.Li al.Pc 446if you would like to automatically log in as root. 447.Pp 448Then, unmount the disk. 449.Bd -literal 450umount /mnt 451vnconfig -u vn0 452.Ed 453.Ss Compiling the virtual kernel 454In order to compile a virtual kernel use the 455.Li VKERNEL 456kernel configuration file residing in 457.Pa /sys/config 458(or a configuration file derived thereof): 459.Bd -literal 460cd /usr/src 461make -DNO_MODULES buildkernel KERNCONF=VKERNEL 462make -DNO_MODULES installkernel KERNCONF=VKERNEL DESTDIR=/var/vkernel 463.Ed 464.Ss Enabling virtual kernel operation 465A special 466.Xr sysctl 8 , 467.Va vm.vkernel_enable , 468must be set to enable 469.Nm 470operation: 471.Bd -literal 472sysctl vm.vkernel_enable=1 473.Ed 474.Ss Configuring the network on the host system 475In order to access a network interface of the host system from the 476.Nm , 477you must add the interface to a 478.Xr bridge 4 479device which will then be passed to the 480.Fl I 481option: 482.Bd -literal 483kldload if_bridge.ko 484kldload if_tap.ko 485ifconfig bridge0 create 486ifconfig bridge0 addm re0 # assuming re0 is the host's interface 487ifconfig bridge0 up 488.Ed 489.Ss Running the kernel 490Finally, the virtual kernel can be run: 491.Bd -literal 492cd /var/vkernel 493\&./boot/kernel/kernel -m 64m -r rootimg.01 -I auto:bridge0 494.Ed 495.Pp 496You can issue the 497.Xr reboot 8 , 498.Xr halt 8 , 499or 500.Xr shutdown 8 501commands from inside a virtual kernel. 502After doing a clean shutdown the 503.Xr reboot 8 504command will re-exec the virtual kernel binary while the other two will 505cause the virtual kernel to exit. 506.Ss Diskless operation 507Booting a 508.Nm 509with a 510.Xr vknetd 8 511network configuration: 512.Bd -literal 513\&./boot/kernel/kernel -m 64m -m -i memimg.0000 -I /var/run/vknet 514 -e boot.netif.ip=172.1.0.4 515 -e boot.netif.netmask=255.255.0.0 516 -e boot.netif.name=vke0 517 -e boot.nfsroot.server=172.1.0.1 518 -e boot.nfsroot.path=/home/vkernel/vkdiskless 519.Ed 520.Sh BUILDING THE WORLD UNDER A VKERNEL 521The virtual kernel platform does not have all the header files expected 522by a world build, so the easiest thing to do right now is to specify a 523pc32 (in a 32 bit vkernel) or pc64 (in a 64 bit vkernel) target when 524building the world under a virtual kernel, like this: 525.Bd -literal 526vkernel# make MACHINE_PLATFORM=pc32 buildworld 527vkernel# make MACHINE_PLATFORM=pc32 installworld 528.Ed 529.Sh SEE ALSO 530.Xr vknet 1 , 531.Xr bridge 4 , 532.Xr ifmedia 4 , 533.Xr tap 4 , 534.Xr vn 4 , 535.Xr sysctl.conf 5 , 536.Xr build 7 , 537.Xr config 8 , 538.Xr disklabel 8 , 539.Xr ifconfig 8 , 540.Xr vknetd 8 , 541.Xr vnconfig 8 542.Rs 543.%A Aggelos Economopoulos 544.%D March 2007 545.%T "A Peek at the DragonFly Virtual Kernel" 546.Re 547.Sh HISTORY 548Virtual kernels were introduced in 549.Dx 1.7 . 550.Sh AUTHORS 551.An -nosplit 552.An Matt Dillon 553thought up and implemented the 554.Nm 555architecture and wrote the 556.Nm vkd 557device driver. 558.An Sepherosa Ziehau 559wrote the 560.Nm vke 561device driver. 562This manual page was written by 563.An Sascha Wildner . 564