1.\" 2.\" Copyright (c) 2006, 2007 3.\" The DragonFly Project. All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 9.\" 1. Redistributions of source code must retain the above copyright 10.\" notice, this list of conditions and the following disclaimer. 11.\" 2. Redistributions in binary form must reproduce the above copyright 12.\" notice, this list of conditions and the following disclaimer in 13.\" the documentation and/or other materials provided with the 14.\" distribution. 15.\" 3. Neither the name of The DragonFly Project nor the names of its 16.\" contributors may be used to endorse or promote products derived 17.\" from this software without specific, prior written permission. 18.\" 19.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 20.\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 21.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS 22.\" FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 23.\" COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, 24.\" INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING, 25.\" BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 26.\" LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED 27.\" AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 28.\" OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT 29.\" OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 30.\" SUCH DAMAGE. 31.\" 32.Dd June 20, 2015 33.Dt VKERNEL 7 34.Os 35.Sh NAME 36.Nm vkernel , 37.Nm vcd , 38.Nm vkd , 39.Nm vke 40.Nd virtual kernel architecture 41.Sh SYNOPSIS 42.Cd "platform vkernel64 # for 64 bit vkernels" 43.Cd "device vcd" 44.Cd "device vkd" 45.Cd "device vke" 46.Pp 47.Pa /var/vkernel/boot/kernel/kernel 48.Op Fl hsUvd 49.Op Fl c Ar file 50.Op Fl e Ar name Ns = Ns Li value : Ns Ar name Ns = Ns Li value : Ns ... 51.Op Fl i Ar file 52.Op Fl I Ar interface Ns Op Ar :address1 Ns Oo Ar :address2 Oc Ns Oo Ar /netmask Oc Ns Oo Ar =mac Oc 53.Op Fl l Ar cpulock 54.Op Fl m Ar size 55.Op Fl n Ar numcpus Ns Op Ar :lbits Ns Oo Ar :cbits Oc 56.Op Fl p Ar pidfile 57.Op Fl r Ar file Ns Op Ar :serno 58.Sh DESCRIPTION 59The 60.Nm 61architecture allows for running 62.Dx 63kernels in userland. 64.Pp 65The following options are available: 66.Bl -tag -width ".Fl m Ar size" 67.It Fl c Ar file 68Specify a readonly CD-ROM image 69.Ar file 70to be used by the kernel, with the first 71.Fl c 72option defining 73.Li vcd0 , 74the second one 75.Li vcd1 , 76and so on. 77The first 78.Fl r 79or 80.Fl c 81option specified on the command line will be the boot disk. 82The CD9660 filesystem is assumed when booting from this media. 83.It Fl e Ar name Ns = Ns Li value : Ns Ar name Ns = Ns Li value : Ns ... 84Specify an environment to be used by the kernel. 85This option can be specified more than once. 86.It Fl h 87Shows a list of available options, each with a short description. 88.It Fl i Ar file 89Specify a memory image 90.Ar file 91to be used by the virtual kernel. 92If no 93.Fl i 94option is given, the kernel will generate a name of the form 95.Pa /var/vkernel/memimg.XXXXXX , 96with the trailing 97.Ql X Ns s 98being replaced by a sequential number, e.g.\& 99.Pa memimg.000001 . 100.It Fl I Ar interface Ns Op Ar :address1 Ns Oo Ar :address2 Oc Ns Oo Ar /netmask Oc Ns Oo Ar =MAC Oc 101Create a virtual network device, with the first 102.Fl I 103option defining 104.Li vke0 , 105the second one 106.Li vke1 , 107and so on. 108.Pp 109The 110.Ar interface 111argument is the name of a 112.Xr tap 4 113device node or the path to a 114.Xr vknetd 8 115socket. 116The 117.Pa /dev/ 118path prefix does not have to be specified and will be automatically prepended 119for a device node. 120Specifying 121.Cm auto 122will pick the first unused 123.Xr tap 4 124device. 125.Pp 126The 127.Ar address1 128and 129.Ar address2 130arguments are the IP addresses of the 131.Xr tap 4 132and 133.Nm vke 134interfaces. 135Optionally, 136.Ar address1 137may be of the form 138.Li bridge Ns Em X 139in which case the 140.Xr tap 4 141interface is added to the specified 142.Xr bridge 4 143interface. 144The 145.Nm vke 146address is not assigned until the interface is brought up in the guest. 147.Pp 148The 149.Ar netmask 150argument applies to all interfaces for which an address is specified. 151.Pp 152The 153.Ar MAC 154argument is the MAC address of the 155.Xr vke 4 156interface. 157If not specified, a pseudo-random one will be generated. 158.Pp 159When running multiple vkernels it is often more convenient to simply 160connect to a 161.Xr vknetd 8 162socket and let vknetd deal with the tap and/or bridge. 163An example of this would be 164.Pa /var/run/vknet:0.0.0.0:10.2.0.2/16 . 165.It Fl l Ar cpulock 166Specify which, if any, real CPUs to lock virtual CPUs to. 167.Ar cpulock 168is one of 169.Cm any , 170.Cm map Ns Op Ns , Ns Ar startCPU , 171or 172.Ar CPU . 173.Pp 174.Cm any 175does not map virtual CPUs to real CPUs. 176This is the default. 177.Pp 178.Cm map Ns Op Ns , Ns Ar startCPU 179maps each virtual CPU to a real CPU starting with real CPU 0 or 180.Ar startCPU 181if specified. 182.Pp 183.Ar CPU 184locks all virtual CPUs to the real CPU specified by 185.Ar CPU . 186.It Fl m Ar size 187Specify the amount of memory to be used by the kernel in bytes, 188.Cm K 189.Pq kilobytes , 190.Cm M 191.Pq megabytes 192or 193.Cm G 194.Pq gigabytes . 195Lowercase versions of 196.Cm K , M , 197and 198.Cm G 199are allowed. 200.It Fl n Ar numcpus Ns Op Ar :lbits Ns Oo Ar :cbits Oc 201.Ar numcpus 202specifies the number of CPUs you wish to emulate. 203Up to 16 CPUs are supported with 2 being the default unless otherwise 204specified. 205.Ar lbits 206specifies the number of bits within APICID(=CPUID) needed for representing 207the logical ID. 208Controls the number of threads/core (0bits - 1 thread, 1bit - 2 threads). 209This parameter is optional (mandatory only if 210.Ar cbits 211is specified). 212.Ar cbits 213specifies the number of bits within APICID(=CPUID) needed for representing 214the core ID. 215Controls the number of core/package (0bits - 1 core, 1bit - 2 cores). 216This parameter is optional. 217.It Fl p Ar pidfile 218Specify a pidfile in which to store the process ID. 219Scripts can use this file to locate the vkernel pid for the purpose of 220shutting down or killing it. 221.Pp 222The vkernel will hold a lock on the pidfile while running. 223Scripts may test for the lock to determine if the pidfile is valid or 224stale so as to avoid accidentally killing a random process. 225Something like '/usr/bin/lockf -ks -t 0 pidfile echo -n' may be used 226to test the lock. 227A non-zero exit code indicates that the pidfile represents a running 228vkernel. 229.Pp 230An error is issued and the vkernel exits if this file cannot be opened for 231writing or if it is already locked by an active vkernel process. 232.It Fl r Ar file Ns Op Ar :serno 233Specify a R/W disk image 234.Ar file 235to be used by the kernel, with the first 236.Fl r 237option defining 238.Li vkd0 , 239the second one 240.Li vkd1 , 241and so on. 242A serial number for the virtual disk can be specified in 243.Ar serno . 244.Pp 245The first 246.Fl r 247or 248.Fl c 249option specified on the command line will be the boot disk. 250.It Fl s 251Boot into single-user mode. 252.It Fl d 253Disables hardware pagetable for vkernel. 254.It Fl U 255Enable writing to kernel memory and module loading. 256By default, those are disabled for security reasons. 257.It Fl v 258Turn on verbose booting. 259.El 260.Sh DEVICES 261A number of virtual device drivers exist to supplement the virtual kernel. 262.Ss Disk device 263The 264.Nm vkd 265driver allows for up to 16 266.Xr vn 4 267based disk devices. 268The root device will be 269.Li vkd0 270(see 271.Sx EXAMPLES 272for further information on how to prepare a root image). 273.Ss CD-ROM device 274The 275.Nm vcd 276driver allows for up to 16 virtual CD-ROM devices. 277Basically this is a read only 278.Nm vkd 279device with a block size of 2048. 280.Ss Network interface 281The 282.Nm vke 283driver supports up to 16 virtual network interfaces which are associated with 284.Xr tap 4 285devices on the host. 286For each 287.Nm vke 288device, the per-interface read only 289.Xr sysctl 3 290variable 291.Va hw.vke Ns Em X Ns Va .tap_unit 292holds the unit number of the associated 293.Xr tap 4 294device. 295.Pp 296By default, half of the total mbuf clusters available is distributed equally 297among all the vke devices up to 256. 298This can be overridden with the tunable 299.Va hw.vke.max_ringsize . 300Take into account the number passed will be aligned to the lower power of two. 301.Sh SIGNALS 302The virtual kernel only enables 303.Dv SIGQUIT 304and 305.Dv SIGTERM 306while operating in regular console mode. 307Sending 308.Ql \&^\e 309.Pq Dv SIGQUIT 310to the virtual kernel causes the virtual kernel to enter its internal 311.Xr ddb 4 312debugger and re-enable all other terminal signals. 313Sending 314.Dv SIGTERM 315to the virtual kernel triggers a clean shutdown by passing a 316.Dv SIGUSR2 317to the virtual kernel's 318.Xr init 8 319process. 320.Sh DEBUGGING 321It is possible to directly gdb the virtual kernel's process. 322It is recommended that you do a 323.Ql handle SIGSEGV noprint 324to ignore page faults processed by the virtual kernel itself and 325.Ql handle SIGUSR1 noprint 326to ignore signals used for simulating inter-processor interrupts. 327.Sh PROFILING 328To compile a vkernel with profiling support, the 329.Va CONFIGARGS 330variable needs to be used to pass 331.Fl p 332to 333.Xr config 8 . 334.Bd -literal 335cd /usr/src 336make -DNO_MODULES CONFIGARGS=-p buildkernel KERNCONF=VKERNEL64 337.Ed 338.Sh FILES 339.Bl -tag -width ".It Pa /sys/config/VKERNEL64" -compact 340.It Pa /dev/vcdX 341.Nm vcd 342device nodes 343.It Pa /dev/vkdX 344.Nm vkd 345device nodes 346.It Pa /sys/config/VKERNEL64 347.El 348.Pp 349.Nm 350configuration file, for 351.Xr config 8 . 352.Sh CONFIGURATION FILES 353Your virtual kernel is a complete 354.Dx 355system, but you might not want to run all the services a normal kernel runs. 356Here is what a typical virtual kernel's 357.Pa /etc/rc.conf 358file looks like, with some additional possibilities commented out. 359.Bd -literal 360hostname="vkernel" 361network_interfaces="lo0 vke0" 362ifconfig_vke0="DHCP" 363sendmail_enable="NO" 364#syslog_enable="NO" 365blanktime="NO" 366.Ed 367.Sh DISKLESS OPERATION 368To boot a 369.Nm 370from a NFS root, a number of tunables need to be set: 371.Bl -tag -width indent 372.It Va boot.netif.ip 373IP address to be set in the vkernel interface. 374.It Va boot.netif.netmask 375Netmask for the IP to be set. 376.It Va boot.netif.name 377Network interface name inside the vkernel. 378.It Va boot.nfsroot.server 379Host running 380.Xr nfsd 8 . 381.It Va boot.nfsroot.path 382Host path where a world and distribution 383targets are properly installed. 384.El 385.Pp 386See an example on how to boot a diskless 387.Nm 388in the 389.Sx EXAMPLES 390section. 391.Sh EXAMPLES 392A couple of steps are necessary in order to prepare the system to build and 393run a virtual kernel. 394.Ss Setting up the filesystem 395The 396.Nm 397architecture needs a number of files which reside in 398.Pa /var/vkernel . 399Since these files tend to get rather big and the 400.Pa /var 401partition is usually of limited size, we recommend the directory to be 402created in the 403.Pa /home 404partition with a link to it in 405.Pa /var : 406.Bd -literal 407mkdir -p /home/var.vkernel/boot 408ln -s /home/var.vkernel /var/vkernel 409.Ed 410.Pp 411Next, a filesystem image to be used by the virtual kernel has to be 412created and populated (assuming world has been built previously). 413If the image is created on a UFS filesystem you might want to pre-zero it. 414On a HAMMER filesystem you should just truncate-extend to the image size 415as HAMMER does not re-use data blocks already present in the file. 416.Bd -literal 417vnconfig -c -S 2g -T vn0 /var/vkernel/rootimg.01 418disklabel -r -w vn0s0 auto 419disklabel -e vn0s0 # add `a' partition with fstype `4.2BSD' 420newfs /dev/vn0s0a 421mount /dev/vn0s0a /mnt 422cd /usr/src 423make installworld DESTDIR=/mnt 424cd etc 425make distribution DESTDIR=/mnt 426echo '/dev/vkd0s0a / ufs rw 1 1' >/mnt/etc/fstab 427echo 'proc /proc procfs rw 0 0' >>/mnt/etc/fstab 428.Ed 429.Pp 430Edit 431.Pa /mnt/etc/ttys 432and replace the 433.Li console 434entry with the following line and turn off all other gettys. 435.Bd -literal 436console "/usr/libexec/getty Pc" cons25 on secure 437.Ed 438.Pp 439Replace 440.Li \&Pc 441with 442.Li al.Pc 443if you would like to automatically log in as root. 444.Pp 445Then, unmount the disk. 446.Bd -literal 447umount /mnt 448vnconfig -u vn0 449.Ed 450.Ss Compiling the virtual kernel 451In order to compile a virtual kernel use the 452.Li VKERNEL64 453kernel configuration file residing in 454.Pa /sys/config 455(or a configuration file derived thereof): 456.Bd -literal 457cd /usr/src 458make -DNO_MODULES buildkernel KERNCONF=VKERNEL64 459make -DNO_MODULES installkernel KERNCONF=VKERNEL64 DESTDIR=/var/vkernel 460.Ed 461.Ss Enabling virtual kernel operation 462A special 463.Xr sysctl 8 , 464.Va vm.vkernel_enable , 465must be set to enable 466.Nm 467operation: 468.Bd -literal 469sysctl vm.vkernel_enable=1 470.Ed 471.Ss Configuring the network on the host system 472In order to access a network interface of the host system from the 473.Nm , 474you must add the interface to a 475.Xr bridge 4 476device which will then be passed to the 477.Fl I 478option: 479.Bd -literal 480kldload if_bridge.ko 481kldload if_tap.ko 482ifconfig bridge0 create 483ifconfig bridge0 addm re0 # assuming re0 is the host's interface 484ifconfig bridge0 up 485.Ed 486.Ss Running the kernel 487Finally, the virtual kernel can be run: 488.Bd -literal 489cd /var/vkernel 490\&./boot/kernel/kernel -m 64m -r rootimg.01 -I auto:bridge0 491.Ed 492.Pp 493You can issue the 494.Xr reboot 8 , 495.Xr halt 8 , 496or 497.Xr shutdown 8 498commands from inside a virtual kernel. 499After doing a clean shutdown the 500.Xr reboot 8 501command will re-exec the virtual kernel binary while the other two will 502cause the virtual kernel to exit. 503.Ss Diskless operation 504Booting a 505.Nm 506with a 507.Xr vknetd 8 508network configuration: 509.Bd -literal 510\&./boot/kernel/kernel -m 64m -m -i memimg.0000 -I /var/run/vknet 511 -e boot.netif.ip=172.1.0.4 512 -e boot.netif.netmask=255.255.0.0 513 -e boot.netif.name=vke0 514 -e boot.nfsroot.server=172.1.0.1 515 -e boot.nfsroot.path=/home/vkernel/vkdiskless 516.Ed 517.Sh BUILDING THE WORLD UNDER A VKERNEL 518The virtual kernel platform does not have all the header files expected 519by a world build, so the easiest thing to do right now is to specify a 520pc64 (in a 64 bit vkernel) target when building the world under a virtual 521kernel, like this: 522.Bd -literal 523vkernel# make MACHINE_PLATFORM=pc64 buildworld 524vkernel# make MACHINE_PLATFORM=pc64 installworld 525.Ed 526.Sh SEE ALSO 527.Xr vknet 1 , 528.Xr bridge 4 , 529.Xr ifmedia 4 , 530.Xr tap 4 , 531.Xr vn 4 , 532.Xr sysctl.conf 5 , 533.Xr build 7 , 534.Xr config 8 , 535.Xr disklabel 8 , 536.Xr ifconfig 8 , 537.Xr vknetd 8 , 538.Xr vnconfig 8 539.Rs 540.%A Aggelos Economopoulos 541.%D March 2007 542.%T "A Peek at the DragonFly Virtual Kernel" 543.Re 544.Sh HISTORY 545Virtual kernels were introduced in 546.Dx 1.7 . 547.Sh AUTHORS 548.An -nosplit 549.An Matt Dillon 550thought up and implemented the 551.Nm 552architecture and wrote the 553.Nm vkd 554device driver. 555.An Sepherosa Ziehau 556wrote the 557.Nm vke 558device driver. 559This manual page was written by 560.An Sascha Wildner . 561