1.\" Hey, Emacs, edit this file in -*- nroff-fill -*- mode 2.\"- 3.\" Copyright (c) 1997, 1998 4.\" Nan Yang Computer Services Limited. All rights reserved. 5.\" 6.\" This software is distributed under the so-called ``Berkeley 7.\" License'': 8.\" 9.\" Redistribution and use in source and binary forms, with or without 10.\" modification, are permitted provided that the following conditions 11.\" are met: 12.\" 1. Redistributions of source code must retain the above copyright 13.\" notice, this list of conditions and the following disclaimer. 14.\" 2. Redistributions in binary form must reproduce the above copyright 15.\" notice, this list of conditions and the following disclaimer in the 16.\" documentation and/or other materials provided with the distribution. 17.\" 3. All advertising materials mentioning features or use of this software 18.\" must display the following acknowledgement: 19.\" This product includes software developed by Nan Yang Computer 20.\" Services Limited. 21.\" 4. Neither the name of the Company nor the names of its contributors 22.\" may be used to endorse or promote products derived from this software 23.\" without specific prior written permission. 24.\" 25.\" This software is provided ``as is'', and any express or implied 26.\" warranties, including, but not limited to, the implied warranties of 27.\" merchantability and fitness for a particular purpose are disclaimed. 28.\" In no event shall the company or contributors be liable for any 29.\" direct, indirect, incidental, special, exemplary, or consequential 30.\" damages (including, but not limited to, procurement of substitute 31.\" goods or services; loss of use, data, or profits; or business 32.\" interruption) however caused and on any theory of liability, whether 33.\" in contract, strict liability, or tort (including negligence or 34.\" otherwise) arising in any way out of the use of this software, even if 35.\" advised of the possibility of such damage. 36.\" 37.\" $FreeBSD: src/share/man/man4/vinum.4,v 1.22.2.9 2002/04/22 08:19:35 kuriyama Exp $ 38.\" 39.Dd October 5, 1999 40.Dt vinum 4 41.Os 42.Sh NAME 43.Nm vinum 44.Nd Logical Volume Manager 45.Sh SYNOPSIS 46.Cd "kldload vinum" 47.Cd "kldload Vinum" 48.Sh DESCRIPTION 49.Nm 50is a logical volume manager inspired by, but not derived from, the Veritas 51Volume Manager. It provides the following features: 52.Bl -bullet 53.It 54It provides device-independent logical disks, called \fIvolumes\fP. Volumes are 55not restricted to the size of any disk on the system. 56.It 57The volumes consist of one or more \fIplexes\fP, each of which contain the 58entire address space of a volume. This represents an implementation of RAID-1 59(mirroring). Multiple plexes can also be used for 60.\" XXX What about sparse plexes? Do we want them? 61.if t .sp 62.Bl -bullet 63.It 64Increased read throughput. 65.Nm 66will read data from the least active disk, so if a volume has plexes on multiple 67disks, more data can be read in parallel. 68.Nm 69reads data from only one plex, but it writes data to all plexes. 70.It 71Increased reliability. By storing plexes on different disks, data will remain 72available even if one of the plexes becomes unavailable. In comparison with a 73RAID-5 plex (see below), using multiple plexes requires more storage space, but 74gives better performance, particularly in the case of a drive failure. 75.It 76Additional plexes can be used for on-line data reorganization. By attaching an 77additional plex and subsequently detaching one of the older plexes, data can be 78moved on-line without compromising access. 79.It 80An additional plex can be used to obtain a consistent dump of a file system. By 81attaching an additional plex and detaching at a specific time, the detached plex 82becomes an accurate snapshot of the file system at the time of detachment. 83.\" Make sure to flush! 84.El 85.It 86Each plex consists of one or more logical disk slices, called \fIsubdisks\fP. 87Subdisks are defined as a contiguous block of physical disk storage. A plex may 88consist of any reasonable number of subdisks (in other words, the real limit is 89not the number, but other factors, such as memory and performance, associated 90with maintaining a large number of subdisks). 91.It 92A number of mappings between subdisks and plexes are available: 93.Bl -bullet 94.It 95\fIConcatenated plexes\fP\| consist of one or more subdisks, each of which 96is mapped to a contiguous part of the plex address space. 97.It 98\fIStriped plexes\fP\| consist of two or more subdisks of equal size. The file 99address space is mapped in \fIstripes\fP, integral fractions of the subdisk 100size. Consecutive plex address space is mapped to stripes in each subdisk in 101.if n turn. 102.if t \{\ 103turn. 104.ig 105.\" FIXME 106.br 107.ne 1.5i 108.PS 109move right 2i 110down 111SD0: box 112SD1: box 113SD2: box 114 115"plex 0" at SD0.n+(0,.2) 116"subdisk 0" rjust at SD0.w-(.2,0) 117"subdisk 1" rjust at SD1.w-(.2,0) 118"subdisk 2" rjust at SD2.w-(.2,0) 119.PE 120.. 121.\} 122The subdisks of a striped plex must all be the same size. 123.It 124\fIRAID-5 plexes\fP\| require at least three equal-sized subdisks. They 125resemble striped plexes, except that in each stripe, one subdisk stores parity 126information. This subdisk changes in each stripe: in the first stripe, it is the 127first subdisk, in the second it is the second subdisk, etc. In the event of a 128single disk failure, 129.Nm 130will recover the data based on the information stored on the remaining subdisks. 131This mapping is particularly suited to read-intensive access. The subdisks of a 132RAID-5 plex must all be the same size. 133.\" Make sure to flush! 134.El 135.It 136.Nm Drives 137are the lowest level of the storage hierarchy. They represent disk special 138devices. 139.It 140.Nm 141offers automatic startup. Unlike UNIX file systems, 142.Nm 143volumes contain all the configuration information needed to ensure that they are 144started correctly when the subsystem is enabled. This is also a significant 145advantage over the Veritas\(tm File System. This feature regards the presence 146of the volumes. It does not mean that the volumes will be mounted 147automatically, since the standard startup procedures with 148.Pa /etc/fstab 149perform this function. 150.El 151.Sh KERNEL CONFIGURATION 152.Nm 153is currently supplied as a kernel loadable module (kld), and does not require 154configuration. As with other klds, it is absolutely necessary to match the kld 155to the version of the operating system. Failure to do so will cause 156.Nm 157to issue an error message and terminate. 158.Pp 159It is possible to configure 160.Nm 161in the kernel, but this is not recommended. To do so, add this line to the 162kernel configuration file: 163.Bd -literal -offset indent 164pseudo-device vinum 165.Ed 166.Pp 167.Ss DEBUG OPTIONS 168The current version of 169.Nm , 170both the kernel module and the user program 171.Xr vinum 8 , 172include significant debugging support. It is not recommended to remove 173this support at the moment, but if you do you must remove it from both the 174kernel and the user components. To do this, edit the files 175.Pa /usr/src/sbin/vinum/Makefile 176and 177.Pa /usr/src/sys/modules/vinum/Makefile 178and edit the CFLAGS variable to remove the -DVINUMDEBUG option. If you have 179configured 180.Nm 181into the kernel, either specify the line 182.Bd -literal -offset indent 183options VINUMDEBUG 184.Ed 185.Pp 186in the kernel configuration file or remove the -DVINUMDEBUG option from 187.Pa /usr/src/sbin/vinum/Makefile 188as described above. 189.Pp 190If the VINUMDEBUG variables do not match, 191.Xr vinum 8 192will fail with a message 193explaining the problem and what to do to correct it. 194.Pp 195.Nm 196was previously available in two versions: a freely available version which did 197not contain RAID-5 functionality, and a full version including RAID-5 198functionality, which was available only from Cybernet Systems Inc. The present 199version of 200.Nm 201includes the RAID-5 functionality. 202.Sh RUNNING VINUM 203.Nm 204is part of the base 205.Fx 206system. It does not require installation. 207To start it, start the 208.Nm 209program, which will load the kld if it is not already present. 210Before using 211.Nm , 212it must be configured. See 213.Xr vinum 8 214for information on how to create a 215.Nm 216configuration. 217.Pp 218Normally, you start a configured version of 219.Nm 220at boot time. Set the variable 221.Ar start_vinum 222in 223.Pa /etc/rc.conf 224to 225.Ar YES 226to start 227.Nm 228at boot time. 229.Pp 230If 231.Nm 232is loaded as a kld (the recommended way), the 233.Nm 234.Ar stop 235command will unload it. You can also do this with the 236.Nm kldunload 237command. 238.Pp 239The kld can only be unloaded when idle, in other words when no volumes are 240mounted and no other instances of the 241.Nm 242program are active. Unloading the kld does not harm the data in the volumes. 243.Ss CONFIGURING AND STARTING OBJECTS 244Use the 245.Xr vinum 8 246utility to configure and start 247.Nm 248objects. 249.Sh IOCTL CALLS 250.Pa ioctl 251calls are intended for the use of the 252.Nm 253configuration program only. They are described in the header file 254.Pa /sys/sys/vinumio.h 255.Ss DISK LABELS 256Conventional disk special devices have a 257.Em disk label 258in the second sector of the device. See 259.Xr disklabel 5 260for more details. This disk label describes the layout of the partitions within 261the device. 262.Nm 263does not subdivide volumes, so volumes do not contain a physical disk label. 264For convenience, 265.Nm 266implements the ioctl calls DIOCGDINFO (get disk label), DIOCGPART (get partition 267information), DIOCWDINFO (write partition information) and DIOCSDINFO (set 268partition information). DIOCGDINFO and DIOCGPART refer to an internal 269representation of the disk label which is not present on the volume. As a 270result, the 271.Fl r 272option of 273.Xr disklabel 8 , 274which reads the 275.if t ``raw disk'', 276.if n "raw disk", 277will fail. 278.Pp 279In general, 280.Xr disklabel 8 281serves no useful purpose on a vinum volume. If you run it, it will show you 282three partitions, a, b and c, all the same except for the fstype, for example: 283.br 284.ne 1i 285.Bd -literal -offset 2863 partitions: 287# size offset fstype [fsize bsize bps/cpg] 288 a: 2048 0 4.2BSD 1024 8192 0 # (Cyl. 0 - 0) 289 b: 2048 0 swap # (Cyl. 0 - 0) 290 c: 2048 0 unused 0 0 # (Cyl. 0 - 0) 291.Ed 292.Pp 293.Nm 294ignores the DIOCWDINFO and DIOCSDINFO ioctls, since there is nothing to change. 295As a result, any attempt to modify the disk label will be silently ignored. 296.Sh MAKING FILE SYSTEMS 297Since 298.Nm 299volumes do not contain partitions, the names do not need to conform to the 300standard rules for naming disk partitions. For a physical disk partition, the 301last letter of the device name specifies the partition identifier (a to h). 302.Nm 303volumes need not conform to this convention, but if they do not, 304.Nm newfs 305will complain that it cannot determine the partition. To solve this problem, 306use the 307.Fl v 308flag to 309.Nm newfs . 310For example, if you have a volume 311.Pa concat , 312use the following command to create a ufs file system on it: 313.Pp 314.Bd -literal 315 # newfs -v /dev/vinum/concat 316.Ed 317.Pp 318.Sh OBJECT NAMING 319.Nm 320assigns default names to plexes and subdisks, although they may be overridden. 321We do not recommend overriding the default names. Experience with the 322.if t Veritas\(tm 323.if n Veritas(tm) 324volume manager, which allows arbitary naming of objects, has shown that this 325flexibility does not bring a significant advantage, and it can cause confusion. 326.sp 327Names may contain any non-blank character, but it is recommended to restrict 328them to letters, digits and the underscore characters. The names of volumes, 329plexes and subdisks may be up to 64 characters long, and the names of drives may 330up to 32 characters long. When choosing volume and plex names, bear in mind 331that automatically generated plex and subdisk names are longer than the name 332from which they are derived. 333.Bl -bullet 334.It 335When 336.Xr vinum 8 337creates or deletes objects, it creates a directory 338.Pa /dev/vinum , 339in which it makes device entries for each volume. It also creates the 340subdirectories 341.Pa /dev/vinum/plex 342and 343.Pa /dev/vinum/sd , 344in which it stores device entries for the plexes and subdisks. In addition, it 345creates two more directories, 346.Pa /dev/vinum/vol 347and 348.Pa /dev/vinum/drive , 349in which it stores hierarchical information for volumes and drives. 350.It 351In addition, 352.Nm 353creates three super-devices, 354.Pa /dev/vinum/control , 355.Pa /dev/vinum/Control 356and 357.Pa /dev/vinum/controld . 358.Pa /dev/vinum/control 359is used by 360.Xr vinum 8 361when it has been compiled without the VINUMDEBUG option, 362.Pa /dev/vinum/Control 363is used by 364.Xr vinum 8 365when it has been compiled with the VINUMDEBUG option, 366and 367.Pa /dev/vinum/controld 368is used by the 369.Nm 370daemon. The two control devices for 371.Xr vinum 8 372are used to synchronize the debug status of kernel and user modules. 373.It 374Unlike 375.Nm UNIX 376drives, 377.Nm 378volumes are not subdivided into partitions, and thus do not contain a disk 379label. Unfortunately, this confuses a number of utilities, notably 380.Nm newfs , 381which normally tries to interpret the last letter of a 382.Nm 383volume name as a partition identifier. If you use a volume name which does not 384end in the letters 385.Ar a 386to 387.Ar c , 388you must use the 389.Fl v 390flag to 391.Nm newfs 392in order to tell it to ignore this convention. 393.\" 394.It 395Plexes do not need to be assigned explicit names. By default, a plex name is 396the name of the volume followed by the letters \f(CW.p\fR and the number of the 397plex. For example, the plexes of volume 398.Ar vol3 399are called 400.Ar vol3.p0 , 401.Ar vol3.p1 402and so on. These names can be overridden, but it is not recommended. 403.br 404.It 405Like plexes, subdisks are assigned names automatically, and explicit naming is 406discouraged. A subdisk name is the name of the plex followed by the letters 407\f(CW\&.s\fR and a number identifying the subdisk. For example, the subdisks of 408plex 409.Ar vol3.p0 410are called 411.Ar vol3.p0.s0 , 412.Ar vol3.p0.s1 413and so on. 414.br 415.It 416By contrast, 417.Nm drives 418must be named. This makes it possible to move a drive to a different location 419and still recognize it automatically. Drive names may be up to 32 characters 420long. 421.El 422.Pp 423EXAMPLE 424.Pp 425Assume the 426.Nm 427objects described in the section CONFIGURATION FILE in 428.Xr vinum 8 . 429The directory 430.Ar /dev/vinum 431looks like: 432.Bd -literal -offset indent 433# ls -lR /dev/vinum 434total 5 435crwxr-xr-- 1 root wheel 91, 2 Mar 30 16:08 concat 436crwx------ 1 root wheel 91, 0x40000000 Mar 30 16:08 control 437crwx------ 1 root wheel 91, 0x40000001 Mar 30 16:08 controld 438drwxrwxrwx 2 root wheel 512 Mar 30 16:08 drive 439drwxrwxrwx 2 root wheel 512 Mar 30 16:08 plex 440drwxrwxrwx 2 root wheel 512 Mar 30 16:08 rvol 441drwxrwxrwx 2 root wheel 512 Mar 30 16:08 sd 442crwxr-xr-- 1 root wheel 91, 3 Mar 30 16:08 strcon 443crwxr-xr-- 1 root wheel 91, 1 Mar 30 16:08 stripe 444crwxr-xr-- 1 root wheel 91, 0 Mar 30 16:08 tinyvol 445drwxrwxrwx 7 root wheel 512 Mar 30 16:08 vol 446crwxr-xr-- 1 root wheel 91, 4 Mar 30 16:08 vol5 447 448/dev/vinum/drive: 449total 0 450crw-r----- 1 root operator 4, 15 Oct 21 16:51 drive2 451crw-r----- 1 root operator 4, 31 Oct 21 16:51 drive4 452 453/dev/vinum/plex: 454total 0 455crwxr-xr-- 1 root wheel 91, 0x10000002 Mar 30 16:08 concat.p0 456crwxr-xr-- 1 root wheel 91, 0x10010002 Mar 30 16:08 concat.p1 457crwxr-xr-- 1 root wheel 91, 0x10000003 Mar 30 16:08 strcon.p0 458crwxr-xr-- 1 root wheel 91, 0x10010003 Mar 30 16:08 strcon.p1 459crwxr-xr-- 1 root wheel 91, 0x10000001 Mar 30 16:08 stripe.p0 460crwxr-xr-- 1 root wheel 91, 0x10000000 Mar 30 16:08 tinyvol.p0 461crwxr-xr-- 1 root wheel 91, 0x10000004 Mar 30 16:08 vol5.p0 462crwxr-xr-- 1 root wheel 91, 0x10010004 Mar 30 16:08 vol5.p1 463 464/dev/vinum/sd: 465total 0 466crwxr-xr-- 1 root wheel 91, 0x20000002 Mar 30 16:08 concat.p0.s0 467crwxr-xr-- 1 root wheel 91, 0x20100002 Mar 30 16:08 concat.p0.s1 468crwxr-xr-- 1 root wheel 91, 0x20010002 Mar 30 16:08 concat.p1.s0 469crwxr-xr-- 1 root wheel 91, 0x20000003 Mar 30 16:08 strcon.p0.s0 470crwxr-xr-- 1 root wheel 91, 0x20100003 Mar 30 16:08 strcon.p0.s1 471crwxr-xr-- 1 root wheel 91, 0x20010003 Mar 30 16:08 strcon.p1.s0 472crwxr-xr-- 1 root wheel 91, 0x20110003 Mar 30 16:08 strcon.p1.s1 473crwxr-xr-- 1 root wheel 91, 0x20000001 Mar 30 16:08 stripe.p0.s0 474crwxr-xr-- 1 root wheel 91, 0x20100001 Mar 30 16:08 stripe.p0.s1 475crwxr-xr-- 1 root wheel 91, 0x20000000 Mar 30 16:08 tinyvol.p0.s0 476crwxr-xr-- 1 root wheel 91, 0x20100000 Mar 30 16:08 tinyvol.p0.s1 477crwxr-xr-- 1 root wheel 91, 0x20000004 Mar 30 16:08 vol5.p0.s0 478crwxr-xr-- 1 root wheel 91, 0x20100004 Mar 30 16:08 vol5.p0.s1 479crwxr-xr-- 1 root wheel 91, 0x20010004 Mar 30 16:08 vol5.p1.s0 480crwxr-xr-- 1 root wheel 91, 0x20110004 Mar 30 16:08 vol5.p1.s1 481 482/dev/vinum/vol: 483total 5 484crwxr-xr-- 1 root wheel 91, 2 Mar 30 16:08 concat 485drwxr-xr-x 4 root wheel 512 Mar 30 16:08 concat.plex 486crwxr-xr-- 1 root wheel 91, 3 Mar 30 16:08 strcon 487drwxr-xr-x 4 root wheel 512 Mar 30 16:08 strcon.plex 488crwxr-xr-- 1 root wheel 91, 1 Mar 30 16:08 stripe 489drwxr-xr-x 3 root wheel 512 Mar 30 16:08 stripe.plex 490crwxr-xr-- 1 root wheel 91, 0 Mar 30 16:08 tinyvol 491drwxr-xr-x 3 root wheel 512 Mar 30 16:08 tinyvol.plex 492crwxr-xr-- 1 root wheel 91, 4 Mar 30 16:08 vol5 493drwxr-xr-x 4 root wheel 512 Mar 30 16:08 vol5.plex 494 495/dev/vinum/vol/concat.plex: 496total 2 497crwxr-xr-- 1 root wheel 91, 0x10000002 Mar 30 16:08 concat.p0 498drwxr-xr-x 2 root wheel 512 Mar 30 16:08 concat.p0.sd 499crwxr-xr-- 1 root wheel 91, 0x10010002 Mar 30 16:08 concat.p1 500drwxr-xr-x 2 root wheel 512 Mar 30 16:08 concat.p1.sd 501 502/dev/vinum/vol/concat.plex/concat.p0.sd: 503total 0 504crwxr-xr-- 1 root wheel 91, 0x20000002 Mar 30 16:08 concat.p0.s0 505crwxr-xr-- 1 root wheel 91, 0x20100002 Mar 30 16:08 concat.p0.s1 506 507/dev/vinum/vol/concat.plex/concat.p1.sd: 508total 0 509crwxr-xr-- 1 root wheel 91, 0x20010002 Mar 30 16:08 concat.p1.s0 510 511/dev/vinum/vol/strcon.plex: 512total 2 513crwxr-xr-- 1 root wheel 91, 0x10000003 Mar 30 16:08 strcon.p0 514drwxr-xr-x 2 root wheel 512 Mar 30 16:08 strcon.p0.sd 515crwxr-xr-- 1 root wheel 91, 0x10010003 Mar 30 16:08 strcon.p1 516drwxr-xr-x 2 root wheel 512 Mar 30 16:08 strcon.p1.sd 517 518/dev/vinum/vol/strcon.plex/strcon.p0.sd: 519total 0 520crwxr-xr-- 1 root wheel 91, 0x20000003 Mar 30 16:08 strcon.p0.s0 521crwxr-xr-- 1 root wheel 91, 0x20100003 Mar 30 16:08 strcon.p0.s1 522 523/dev/vinum/vol/strcon.plex/strcon.p1.sd: 524total 0 525crwxr-xr-- 1 root wheel 91, 0x20010003 Mar 30 16:08 strcon.p1.s0 526crwxr-xr-- 1 root wheel 91, 0x20110003 Mar 30 16:08 strcon.p1.s1 527 528/dev/vinum/vol/stripe.plex: 529total 1 530crwxr-xr-- 1 root wheel 91, 0x10000001 Mar 30 16:08 stripe.p0 531drwxr-xr-x 2 root wheel 512 Mar 30 16:08 stripe.p0.sd 532 533/dev/vinum/vol/stripe.plex/stripe.p0.sd: 534total 0 535crwxr-xr-- 1 root wheel 91, 0x20000001 Mar 30 16:08 stripe.p0.s0 536crwxr-xr-- 1 root wheel 91, 0x20100001 Mar 30 16:08 stripe.p0.s1 537 538/dev/vinum/vol/tinyvol.plex: 539total 1 540crwxr-xr-- 1 root wheel 91, 0x10000000 Mar 30 16:08 tinyvol.p0 541drwxr-xr-x 2 root wheel 512 Mar 30 16:08 tinyvol.p0.sd 542 543/dev/vinum/vol/tinyvol.plex/tinyvol.p0.sd: 544total 0 545crwxr-xr-- 1 root wheel 91, 0x20000000 Mar 30 16:08 tinyvol.p0.s0 546crwxr-xr-- 1 root wheel 91, 0x20100000 Mar 30 16:08 tinyvol.p0.s1 547 548/dev/vinum/vol/vol5.plex: 549total 2 550crwxr-xr-- 1 root wheel 91, 0x10000004 Mar 30 16:08 vol5.p0 551drwxr-xr-x 2 root wheel 512 Mar 30 16:08 vol5.p0.sd 552crwxr-xr-- 1 root wheel 91, 0x10010004 Mar 30 16:08 vol5.p1 553drwxr-xr-x 2 root wheel 512 Mar 30 16:08 vol5.p1.sd 554 555/dev/vinum/vol/vol5.plex/vol5.p0.sd: 556total 0 557crwxr-xr-- 1 root wheel 91, 0x20000004 Mar 30 16:08 vol5.p0.s0 558crwxr-xr-- 1 root wheel 91, 0x20100004 Mar 30 16:08 vol5.p0.s1 559 560/dev/vinum/vol/vol5.plex/vol5.p1.sd: 561total 0 562crwxr-xr-- 1 root wheel 91, 0x20010004 Mar 30 16:08 vol5.p1.s0 563crwxr-xr-- 1 root wheel 91, 0x20110004 Mar 30 16:08 vol5.p1.s1 564.Ed 565.Pp 566In the case of unattached plexes and subdisks, the naming is reversed. Subdisks 567are named after the disk on which they are located, and plexes are named after 568the subdisk. 569.\" XXX 570.Nm This mapping is still to be determined. 571.Ss OBJECT STATES 572.Pp 573Each 574.Nm 575object has a \fIstate\fR associated with it. 576.Nm 577uses this state to determine the handling of the object. 578.Pp 579.Ss VOLUME STATES 580Volumes may have the following states: 581.sp 582.Bl -hang -width 14n 583.It Li down 584The volume is completely inaccessible. 585.It Li up 586The volume is up and at least partially functional. Not all plexes may be 587available. 588.El 589.Ss "PLEX STATES" 590Plexes may have the following states: 591.sp 592.ne 1i 593.Bl -hang -width 14n 594.It Li referenced 595A plex entry which has been referenced as part of a volume, but which is 596currently not known. 597.It Li faulty 598A plex which has gone completely down because of I/O errors. 599.It Li down 600A plex which has been taken down by the administrator. 601.It Li initializing 602A plex which is being initialized. 603.sp 604The remaining states represent plexes which are at least partially up. 605.It Li corrupt 606A plex entry which is at least partially up. Not all subdisks are available, 607and an inconsistency has occurred. If no other plex is uncorrupted, the volume 608is no longer consistent. 609.It Li degraded 610A RAID-5 plex entry which is accessible, but one subdisk is down, requiring 611recovery for many I/O requests. 612.It Li flaky 613A plex which is really up, but which has a reborn subdisk which we don't 614completely trust, and which we don't want to read if we can avoid it. 615.It Li up 616A plex entry which is completely up. All subdisks are up. 617.El 618.sp 2v 619.Ss "SUBDISK STATES" 620Subdisks can have the following states: 621.sp 622.ne 1i 623.Bl -hang -width 14n 624.It Li empty 625A subdisk entry which has been created completely. All fields are correct, and 626the disk has been updated, but the on the disk is not valid. 627.It Li referenced 628A subdisk entry which has been referenced as part of a plex, but which is 629currently not known. 630.It Li initializing 631A subdisk entry which has been created completely and which is currently being 632initialized. 633.sp 634The following states represent invalid data. 635.It Li obsolete 636A subdisk entry which has been created completely. All fields are correct, the 637config on disk has been updated, and the data was valid, but since then the 638drive has been taken down, and as a result updates have been missed. 639.It Li stale 640A subdisk entry which has been created completely. All fields are correct, the 641disk has been updated, and the data was valid, but since then the drive has been 642crashed and updates have been lost. 643.sp 644The following states represent valid, inaccessible data. 645.It Li crashed 646A subdisk entry which has been created completely. All fields are correct, the 647disk has been updated, and the data was valid, but since then the drive has gone 648down. No attempt has been made to write to the subdisk since the crash, so the 649data is valid. 650.It Li down 651A subdisk entry which was up, which contained valid data, and which was taken 652down by the administrator. The data is valid. 653.It Li reviving 654The subdisk is currently in the process of being revived. We can write but not 655read. 656.sp 657The following states represent accessible subdisks with valid data. 658.It Li reborn 659A subdisk entry which has been created completely. All fields are correct, the 660disk has been updated, and the data was valid, but since then the drive has gone 661down and up again. No updates were lost, but it is possible that the subdisk 662has been damaged. We won't read from this subdisk if we have a choice. If this 663is the only subdisk which covers this address space in the plex, we set its 664state to up under these circumstances, so this status implies that there is 665another subdisk to fulfil the request. 666.It Li up 667A subdisk entry which has been created completely. All fields are correct, the 668disk has been updated, and the data is valid. 669.El 670.sp 2v 671.Ss "DRIVE STATES" 672Drives can have the following states: 673.sp 674.ne 1i 675.Bl -hang -width 14n 676.It Li referenced 677At least one subdisk refers to the drive, but it is not currently accessible to 678the system. No device name is known. 679.It Li down 680The drive is not accessible. 681.It Li up 682The drive is up and running. 683.El 684.sp 2v 685.Sh BUGS 686.Bl -enum 687.It 688.Nm 689is a new product. Bugs can be expected. The configuration mechanism is not yet 690fully functional. If you have difficulties, please look at the section 691DEBUGGING PROBLEMS WITH VINUM before reporting problems. 692.It 693Kernels with the 694.Nm 695pseudo-device appear to work, but are not supported. If you have trouble with 696this configuration, please first replace the kernel with a non-Vinum 697kernel and test with the kld module. 698.It 699Detection of differences between the version of the kernel and the kld is not 700yet implemented. 701.It 702The RAID-5 functionality is new in 703.Fx 3.3 . 704Some problems have been 705reported with 706.Nm 707in combination with soft updates, but these are not reproducible on all 708systems. If you are planning to use 709.Nm 710in a production environment, please test carefully. 711.El 712.Sh DEBUGGING PROBLEMS WITH VINUM 713Solving problems with 714.Nm 715can be a difficult affair. This section suggests some approaches. 716.Ss Configuration problems 717.Pp 718It is relatively easy (too easy) to run into problems with the 719.Nm 720configuration. If you do, the first thing you should do is stop configuration 721updates: 722.if t .ps -3 723.if t .vs -3 724.Bd -literal 725# \fBvinum setdaemon 4\fP 726.Ed 727.if t .vs 728.if t .ps 729.Pp 730This will stop updates and any further corruption of the on-disk configuration. 731.Pp 732Next, look at the on-disk configuration with the 733.Nm vinum dumpconfig 734command, for example: 735.if t .ps -3 736.if t .vs -3 737.Bd -literal 738# \fBvinum dumpconfig\fP 739Drive 4: Device /dev/da3h 740 Created on crash.lemis.com at Sat May 20 16:32:44 2000 741 Config last updated Sat May 20 16:32:56 2000 742 Size: 601052160 bytes (573 MB) 743volume obj state up 744volume src state up 745volume raid state down 746volume r state down 747volume foo state up 748plex name obj.p0 state corrupt org concat vol obj 749plex name obj.p1 state corrupt org striped 128b vol obj 750plex name src.p0 state corrupt org striped 128b vol src 751plex name src.p1 state up org concat vol src 752plex name raid.p0 state faulty org disorg vol raid 753plex name r.p0 state faulty org disorg vol r 754plex name foo.p0 state up org concat vol foo 755plex name foo.p1 state faulty org concat vol foo 756sd name obj.p0.s0 drive drive2 plex obj.p0 state reborn len 409600b driveoffset 265b plexoffset 0b 757sd name obj.p0.s1 drive drive4 plex obj.p0 state up len 409600b driveoffset 265b plexoffset 409600b 758sd name obj.p1.s0 drive drive1 plex obj.p1 state up len 204800b driveoffset 265b plexoffset 0b 759sd name obj.p1.s1 drive drive2 plex obj.p1 state reborn len 204800b driveoffset 409865b plexoffset 128b 760sd name obj.p1.s2 drive drive3 plex obj.p1 state up len 204800b driveoffset 265b plexoffset 256b 761sd name obj.p1.s3 drive drive4 plex obj.p1 state up len 204800b driveoffset 409865b plexoffset 384b 762.Ed 763.if t .vs 764.if t .ps 765.Pp 766The configuration on all disks should be the same. If this is not the case, 767please save the output to a file and report the problem. There is probably 768little that can be done to recover the on-disk configuration, but if you keep a 769copy of the files used to create the objects, you should be able to re-create 770them. The 771.Cm create 772command does not change the subdisk data, so this will not cause data 773corruption. You may need to use the 774.Cm resetconfig 775command if you have this kind of trouble. 776.Ss Kernel Panics 777.Pp 778In order to analyse a panic which you suspect comes from 779.Nm 780you will need to build a debug kernel. See the online handbook at 781.Pa /usr/share/doc/en/books/developers-handbook/kerneldebug.html 782(if installed) or 783.Pa http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/developers-handbook/kerneldebug.html 784for more details of how to do this. 785.Pp 786Perform the following steps to analyse a 787.Nm 788problem: 789.Bl -enum 790.It 791Copy the files 792.Pa /usr/src/sys/modules/vinum/.gdbinit.crash , 793.Pa /usr/src/sys/modules/vinum/.gdbinit.kernel , 794.Pa /usr/src/sys/modules/vinum/.gdbinit.serial , 795.Pa /usr/src/sys/modules/vinum/.gdbinit.vinum 796and 797.Pa /usr/src/sys/modules/vinum/.gdbinit.vinum.paths 798to the directory in which you will be performing the analysis, typically 799.Pa /var/crash . 800.It 801Make sure that you build the 802.Nm 803module with debugging information. The standard 804.Pa Makefile 805builds a module with debugging symbols by default. If the version of 806.Nm 807in 808.Pa /modules 809does not contain symbols, you will not get an error message, but the stack trace 810will not show the symbols. Check the module before starting 811.Nm gdb : 812.Bd -literal 813$ file /modules/vinum.ko 814/modules/vinum.ko: ELF 32-bit LSB shared object, Intel 80386, 815 version 1 (FreeBSD), not stripped 816.Ed 817.Pp 818If the output shows that 819.Pa /modules/vinum.ko 820is stripped, you will have to find a version which is not. Usually this will be 821either in 822.Pa /usr/obj/sys/modules/vinum/vinum.ko 823(if you have built 824.Nm 825with a 826.Ar make world ) 827or 828.Pa /usr/src/sys/modules/vinum/vinum.ko 829(if you have built 830.Nm 831in this directory). Modify the file 832.Pa .gdbinit.vinum.paths 833accordingly. 834.It 835Either take a dump or use remote serial 836.Cm gdb 837to analyse the problem. To analyse a dump, say 838.Pa /var/crash/vmcore.5 , 839link 840.Pa /var/crash/.gdbinit.crash 841to 842.Pa /var/crash/.gdbinit 843and enter: 844.Bd -literal 845# cd /var/crash 846# gdb -k kernel.debug vmcore.5 847.Ed 848.Pp 849This example assumes that you have installed the correct debug kernel at 850.Pa /var/crash/kernel.debug . 851If not, substitute the correct name of the debug kernel. 852.Pp 853To perform remote serial debugging, 854link 855.Pa /var/crash/.gdbinit.serial 856to 857.Pa /var/crash/.gdbinit 858and enter 859.Bd -literal 860# cd /var/crash 861# gdb -k kernel.debug 862.Ed 863.Pp 864In this case, the 865.Pa .gdbinit 866file performs the functions necessary to establish connection. The remote 867machine must already be in debug mode: enter the kernel debugger and select 868.Nm gdb . 869The serial 870.Pa .gdbinit 871file expects the serial connection to run at 38400 bits per second; if you run 872at a different speed, edit the file accordingly (look for the 873.Ar remotebaud 874specification). 875.Pp 876The following example shows a remote debugging session using the 877.Ar debug 878command of 879.Xr vinum 8 : 880.if t .ps -3 881.if t .vs -3 882.Bd -literal 883GDB 4.16 (i386-unknown-freebsd), Copyright 1996 Free Software Foundation, Inc. 884Debugger (msg=0xf1093174 "vinum debug") at ../../i386/i386/db_interface.c:318 885318 in_Debugger = 0; 886#1 0xf108d9bc in vinumioctl (dev=0x40001900, cmd=0xc008464b, data=0xf6dedee0 "", 887 flag=0x3, p=0xf68b7940) at 888 /usr/src/sys/modules/Vinum/../../dev/Vinum/vinumioctl.c:102 889102 Debugger ("vinum debug"); 890(kgdb) bt 891#0 Debugger (msg=0xf0f661ac "vinum debug") at ../../i386/i386/db_interface.c:318 892#1 0xf0f60a7c in vinumioctl (dev=0x40001900, cmd=0xc008464b, data=0xf6923ed0 "", 893 flag=0x3, p=0xf688e6c0) at 894 /usr/src/sys/modules/vinum/../../dev/vinum/vinumioctl.c:109 895#2 0xf01833b7 in spec_ioctl (ap=0xf6923e0c) at ../../miscfs/specfs/spec_vnops.c:424 896#3 0xf0182cc9 in spec_vnoperate (ap=0xf6923e0c) at ../../miscfs/specfs/spec_vnops.c:129 897#4 0xf01eb3c1 in ufs_vnoperatespec (ap=0xf6923e0c) at ../../ufs/ufs/ufs_vnops.c:2312 898#5 0xf017dbb1 in vn_ioctl (fp=0xf1007ec0, com=0xc008464b, data=0xf6923ed0 "", 899 p=0xf688e6c0) at vnode_if.h:395 900#6 0xf015dce0 in ioctl (p=0xf688e6c0, uap=0xf6923f84) at ../../kern/sys_generic.c:473 901#7 0xf0214c0b in syscall (frame={tf_es = 0x27, tf_ds = 0x27, tf_edi = 0xefbfcff8, 902 tf_esi = 0x1, tf_ebp = 0xefbfcf90, tf_isp = 0xf6923fd4, tf_ebx = 0x2, 903 tf_edx = 0x804b614, tf_ecx = 0x8085d10, tf_eax = 0x36, tf_trapno = 0x7, 904 tf_err = 0x2, tf_eip = 0x8060a34, tf_cs = 0x1f, tf_eflags = 0x286, 905 tf_esp = 0xefbfcf78, tf_ss = 0x27}) at ../../i386/i386/trap.c:1100 906#8 0xf020a1fc in Xint0x80_syscall () 907#9 0x804832d in ?? () 908#10 0x80482ad in ?? () 909#11 0x80480e9 in ?? () 910.Ed 911.if t .vs 912.if t .ps 913.Pp 914When entering from the debugger, it's important that the source of frame 1 915(listed by the 916.Pa .gdbinit 917file at the top of the example) contains the text 918.if t .ps -3 919.if t .vs -3 920.Bd -literal 921Debugger ("vinum debug"); 922.Ed 923.if t .vs 924.if t .ps 925.Pp 926This is an indication that the address specifications are correct. If you get 927some other output, your symbols and the kernel module are out of sync, and the 928trace will be meaningless. 929.El 930.Pp 931For an initial investigation, the most important information is the output of 932the 933.Nm bt 934(backtrace) command above. 935.Ss Reporting problems with Vinum 936.Pp 937If you find any bugs in 938.Nm , 939please report them to Greg Lehey <grog@lemis.com>. Supply the following 940information: 941.Pp 942.Bl -bullet 943.It 944The output of the 945.Nm 946.Cm list 947command. 948.It 949Any messages printed in 950.Pa /var/log/messages . 951All such messages will be identified by the text 952.Nm 953at the beginning. 954.It 955If you have a panic, a stack trace as described above. 956.El 957.Sh AUTHORS 958.An Greg Lehey Aq grog@lemis.com . 959.Sh HISTORY 960.Nm 961first appeared in 962.Fx 3.0 . 963The RAID-5 component of 964.Nm 965was developed by Cybernet Inc. 966.Pa www.cybernet.com 967for its NetMAX product. 968.Sh SEE ALSO 969.Xr disklabel 5 , 970.Xr disklabel 8 , 971.Xr newfs 8 , 972.Xr vinum 8 973