1.\" $NetBSD: file.1,v 1.16 2015/01/02 21:15:32 christos Exp $ 2.\" 3.\" $File: file.man,v 1.111 2014/12/16 23:18:40 christos Exp $ 4.Dd December 16, 2014 5.Dt FILE 1 6.Os 7.Sh NAME 8.Nm file 9.Nd determine file type 10.Sh SYNOPSIS 11.Nm 12.Bk -words 13.Op Fl bcEhiklLNnprsvz0 14.Op Fl Fl apple 15.Op Fl Fl mime-encoding 16.Op Fl Fl mime-type 17.Op Fl e Ar testname 18.Op Fl F Ar separator 19.Op Fl f Ar namefile 20.Op Fl m Ar magicfiles 21.Op Fl P Ar name=value 22.Ar 23.Ek 24.Nm 25.Fl C 26.Op Fl m Ar magicfiles 27.Nm 28.Op Fl Fl help 29.Sh DESCRIPTION 30This manual page documents version 5.22 of the 31.Nm 32command. 33.Pp 34.Nm 35tests each argument in an attempt to classify it. 36There are three sets of tests, performed in this order: 37filesystem tests, magic tests, and language tests. 38The 39.Em first 40test that succeeds causes the file type to be printed. 41.Pp 42The type printed will usually contain one of the words 43.Em text 44(the file contains only 45printing characters and a few common control 46characters and is probably safe to read on an 47.Dv ASCII 48terminal), 49.Em executable 50(the file contains the result of compiling a program 51in a form understandable to some 52.Tn UNIX 53kernel or another), 54or 55.Em data 56meaning anything else (data is usually 57.Dq binary 58or non-printable). 59Exceptions are well-known file formats (core files, tar archives) 60that are known to contain binary data. 61When modifying magic files or the program itself, make sure to 62.Em "preserve these keywords" . 63Users depend on knowing that all the readable files in a directory 64have the word 65.Dq text 66printed. 67Don't do as Berkeley did and change 68.Dq shell commands text 69to 70.Dq shell script . 71.Pp 72The filesystem tests are based on examining the return from a 73.Xr stat 2 74system call. 75The program checks to see if the file is empty, 76or if it's some sort of special file. 77Any known file types appropriate to the system you are running on 78(sockets, symbolic links, or named pipes (FIFOs) on those systems that 79implement them) 80are intuited if they are defined in the system header file 81.In sys/stat.h . 82.Pp 83The magic tests are used to check for files with data in 84particular fixed formats. 85The canonical example of this is a binary executable (compiled program) 86.Dv a.out 87file, whose format is defined in 88.In elf.h , 89.In a.out.h 90and possibly 91.In exec.h 92in the standard include directory. 93These files have a 94.Dq "magic number" 95stored in a particular place 96near the beginning of the file that tells the 97.Tn UNIX 98operating system 99that the file is a binary executable, and which of several types thereof. 100The concept of a 101.Dq "magic" 102has been applied by extension to data files. 103Any file with some invariant identifier at a small fixed 104offset into the file can usually be described in this way. 105The information identifying these files is read from the compiled 106magic file 107.Pa /usr/share/misc/magic.mgc , 108or the files in the directory 109.Pa /usr/share/misc/magic 110if the compiled file does not exist. 111In addition, if 112.Pa $HOME/.magic.mgc 113or 114.Pa $HOME/.magic 115exists, it will be used in preference to the system magic files. 116.Pp 117If a file does not match any of the entries in the magic file, 118it is examined to see if it seems to be a text file. 119ASCII, ISO-8859-x, non-ISO 8-bit extended-ASCII character sets 120(such as those used on Macintosh and IBM PC systems), 121UTF-8-encoded Unicode, UTF-16-encoded Unicode, and EBCDIC 122character sets can be distinguished by the different 123ranges and sequences of bytes that constitute printable text 124in each set. 125If a file passes any of these tests, its character set is reported. 126ASCII, ISO-8859-x, UTF-8, and extended-ASCII files are identified 127as 128.Dq text 129because they will be mostly readable on nearly any terminal; 130UTF-16 and EBCDIC are only 131.Dq character data 132because, while 133they contain text, it is text that will require translation 134before it can be read. 135In addition, 136.Nm 137will attempt to determine other characteristics of text-type files. 138If the lines of a file are terminated by CR, CRLF, or NEL, instead 139of the Unix-standard LF, this will be reported. 140Files that contain embedded escape sequences or overstriking 141will also be identified. 142.Pp 143Once 144.Nm 145has determined the character set used in a text-type file, 146it will 147attempt to determine in what language the file is written. 148The language tests look for particular strings (cf. 149.In names.h ) 150that can appear anywhere in the first few blocks of a file. 151For example, the keyword 152.Em .br 153indicates that the file is most likely a 154.Xr troff 1 155input file, just as the keyword 156.Em struct 157indicates a C program. 158These tests are less reliable than the previous 159two groups, so they are performed last. 160The language test routines also test for some miscellany 161(such as 162.Xr tar 1 163archives). 164.Pp 165Any file that cannot be identified as having been written 166in any of the character sets listed above is simply said to be 167.Dq data . 168.Sh OPTIONS 169.Bl -tag -width indent 170.It Fl Fl apple 171Causes the file command to output the file type and creator code as 172used by older MacOS versions. The code consists of eight letters, 173the first describing the file type, the latter the creator. 174.It Fl b , Fl Fl brief 175Do not prepend filenames to output lines (brief mode). 176.It Fl C , Fl Fl compile 177Write a 178.Pa magic.mgc 179output file that contains a pre-parsed version of the magic file or directory. 180.It Fl c , Fl Fl checking-printout 181Cause a checking printout of the parsed form of the magic file. 182This is usually used in conjunction with the 183.Fl m 184flag to debug a new magic file before installing it. 185.It Fl E 186On filesystem errors (file not found etc), instead of handling the error 187as regular output as POSIX mandates and keep going, issue an error message 188and exit. 189.It Fl e , Fl Fl exclude Ar testname 190Exclude the test named in 191.Ar testname 192from the list of tests made to determine the file type. 193Valid test names are: 194.Bl -tag -width compress 195.It apptype 196.Dv EMX 197application type (only on EMX). 198.It ascii 199Various types of text files (this test will try to guess the text 200encoding, irrespective of the setting of the 201.Sq encoding 202option). 203.It encoding 204Different text encodings for soft magic tests. 205.It tokens 206Ignored for backwards compatibility. 207.It cdf 208Prints details of Compound Document Files. 209.It compress 210Checks for, and looks inside, compressed files. 211.It elf 212Prints ELF file details. 213.It soft 214Consults magic files. 215.It tar 216Examines tar files. 217.El 218.It Fl F , Fl Fl separator Ar separator 219Use the specified string as the separator between the filename and the 220file result returned. 221Defaults to 222.Sq \&: . 223.It Fl f , Fl Fl files-from Ar namefile 224Read the names of the files to be examined from 225.Ar namefile 226(one per line) 227before the argument list. 228Either 229.Ar namefile 230or at least one filename argument must be present; 231to test the standard input, use 232.Sq - 233as a filename argument. 234Please note that 235.Ar namefile 236is unwrapped and the enclosed filenames are processed when this option is 237encountered and before any further options processing is done. 238This allows one to process multiple lists of files with different command line 239arguments on the same 240.Nm 241invocation. 242Thus if you want to set the delimiter, you need to do it before you specify 243the list of files, like: 244.Dq Fl F Ar @ Fl f Ar namefile , 245instead of: 246.Dq Fl f Ar namefile Fl F Ar @ . 247.It Fl h , Fl Fl no-dereference 248option causes symlinks not to be followed 249(on systems that support symbolic links). 250This is the default if the environment variable 251.Dv POSIXLY_CORRECT 252is not defined. 253.It Fl i , Fl Fl mime 254Causes the file command to output mime type strings rather than the more 255traditional human readable ones. 256Thus it may say 257.Sq text/plain; charset=us-ascii 258rather than 259.Dq ASCII text . 260.It Fl Fl mime-type , Fl Fl mime-encoding 261Like 262.Fl i , 263but print only the specified element(s). 264.It Fl k , Fl Fl keep-going 265Don't stop at the first match, keep going. 266Subsequent matches will be 267have the string 268.Sq "\[rs]012\- " 269prepended. 270(If you want a newline, see the 271.Fl r 272option.) 273The magic pattern with the highest strength (see the 274.Fl l 275option) comes first. 276.It Fl l , Fl Fl list 277Shows a list of patterns and their strength sorted descending by 278.Xr magic 4 279strength 280which is used for the matching (see also the 281.Fl k 282option). 283.It Fl L , Fl Fl dereference 284option causes symlinks to be followed, as the like-named option in 285.Xr ls 1 286(on systems that support symbolic links). 287This is the default if the environment variable 288.Ev POSIXLY_CORRECT 289is defined. 290.It Fl m , Fl Fl magic-file Ar magicfiles 291Specify an alternate list of files and directories containing magic. 292This can be a single item, or a colon-separated list. 293If a compiled magic file is found alongside a file or directory, 294it will be used instead. 295.It Fl N , Fl Fl no-pad 296Don't pad filenames so that they align in the output. 297.It Fl n , Fl Fl no-buffer 298Force stdout to be flushed after checking each file. 299This is only useful if checking a list of files. 300It is intended to be used by programs that want filetype output from a pipe. 301.It Fl p , Fl Fl preserve-date 302On systems that support 303.Xr utime 3 304or 305.Xr utimes 2 , 306attempt to preserve the access time of files analyzed, to pretend that 307.Nm 308never read them. 309.It Fl P , Fl Fl parameter Ar name=value 310Set various parameter limits. 311.Bl -column "elf_phnum" "Default" "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" -offset indent 312.It Sy "Name" Ta Sy "Default" Ta Sy "Explanation" 313.It Li indir Ta 15 Ta recursion limit for indirect magic 314.It Li name Ta 30 Ta use count limit for name/use magic 315.It Li elf_notes Ta 256 Ta max ELF notes processed 316.It Li elf_phnum Ta 128 Ta max ELF program sections processed 317.It Li elf_shnum Ta 32768 Ta max ELF sections processed 318.El 319.It Fl r , Fl Fl raw 320Don't translate unprintable characters to \eooo. 321Normally 322.Nm 323translates unprintable characters to their octal representation. 324.It Fl s , Fl Fl special-files 325Normally, 326.Nm 327only attempts to read and determine the type of argument files which 328.Xr stat 2 329reports are ordinary files. 330This prevents problems, because reading special files may have peculiar 331consequences. 332Specifying the 333.Fl s 334option causes 335.Nm 336to also read argument files which are block or character special files. 337This is useful for determining the filesystem types of the data in raw 338disk partitions, which are block special files. 339This option also causes 340.Nm 341to disregard the file size as reported by 342.Xr stat 2 343since on some systems it reports a zero size for raw disk partitions. 344.It Fl v , Fl Fl version 345Print the version of the program and exit. 346.It Fl z , Fl Fl uncompress 347Try to look inside compressed files. 348.It Fl 0 , Fl Fl print0 349Output a null character 350.Sq \e0 351after the end of the filename. 352Nice to 353.Xr cut 1 354the output. 355This does not affect the separator, which is still printed. 356.It Fl -help 357Print a help message and exit. 358.El 359.Sh FILES 360.Bl -tag -width /usr/share/misc/magic.mgc -compact 361.It Pa /usr/share/misc/magic.mgc 362Default compiled list of magic. 363.It Pa /usr/share/misc/magic 364Directory containing default magic files. 365.El 366.Sh ENVIRONMENT 367The environment variable 368.Ev MAGIC 369can be used to set the default magic file name. 370If that variable is set, then 371.Nm 372will not attempt to open 373.Pa $HOME/.magic . 374.Nm 375adds 376.Dq Pa .mgc 377to the value of this variable as appropriate. 378However, 379.Pa file 380has to exist in order for 381.Pa file.mime 382to be considered. 383The environment variable 384.Ev POSIXLY_CORRECT 385controls (on systems that support symbolic links), whether 386.Nm 387will attempt to follow symlinks or not. 388If set, then 389.Nm 390follows symlink, otherwise it does not. 391This is also controlled by the 392.Fl L 393and 394.Fl h 395options. 396.Sh SEE ALSO 397.Xr magic 5 , 398.Xr hexdump 1 , 399.Xr od 1 , 400.Xr strings 1 , 401.Sh STANDARDS CONFORMANCE 402This program is believed to exceed the System V Interface Definition 403of FILE(CMD), as near as one can determine from the vague language 404contained therein. 405Its behavior is mostly compatible with the System V program of the same name. 406This version knows more magic, however, so it will produce 407different (albeit more accurate) output in many cases. 408.\" URL: http://www.opengroup.org/onlinepubs/009695399/utilities/file.html 409.Pp 410The one significant difference 411between this version and System V 412is that this version treats any white space 413as a delimiter, so that spaces in pattern strings must be escaped. 414For example, 415.Bd -literal -offset indent 416\*[Gt]10 string language impress\ (imPRESS data) 417.Ed 418.Pp 419in an existing magic file would have to be changed to 420.Bd -literal -offset indent 421\*[Gt]10 string language\e impress (imPRESS data) 422.Ed 423.Pp 424In addition, in this version, if a pattern string contains a backslash, 425it must be escaped. 426For example 427.Bd -literal -offset indent 4280 string \ebegindata Andrew Toolkit document 429.Ed 430.Pp 431in an existing magic file would have to be changed to 432.Bd -literal -offset indent 4330 string \e\ebegindata Andrew Toolkit document 434.Ed 435.Pp 436SunOS releases 3.2 and later from Sun Microsystems include a 437.Nm 438command derived from the System V one, but with some extensions. 439This version differs from Sun's only in minor ways. 440It includes the extension of the 441.Sq \*[Am] 442operator, used as, 443for example, 444.Bd -literal -offset indent 445\*[Gt]16 long\*[Am]0x7fffffff \*[Gt]0 not stripped 446.Ed 447.Sh MAGIC DIRECTORY 448The magic file entries have been collected from various sources, 449mainly USENET, and contributed by various authors. 450Christos Zoulas (address below) will collect additional 451or corrected magic file entries. 452A consolidation of magic file entries 453will be distributed periodically. 454.Pp 455The order of entries in the magic file is significant. 456Depending on what system you are using, the order that 457they are put together may be incorrect. 458If your old 459.Nm 460command uses a magic file, 461keep the old magic file around for comparison purposes 462(rename it to 463.Pa /usr/share/misc/magic.orig ) . 464.Sh EXAMPLES 465.Bd -literal -offset indent 466$ file file.c file /dev/{wd0a,hda} 467file.c: C program text 468file: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), 469 dynamically linked (uses shared libs), stripped 470/dev/wd0a: block special (0/0) 471/dev/hda: block special (3/0) 472 473$ file -s /dev/wd0{b,d} 474/dev/wd0b: data 475/dev/wd0d: x86 boot sector 476 477$ file -s /dev/hda{,1,2,3,4,5,6,7,8,9,10} 478/dev/hda: x86 boot sector 479/dev/hda1: Linux/i386 ext2 filesystem 480/dev/hda2: x86 boot sector 481/dev/hda3: x86 boot sector, extended partition table 482/dev/hda4: Linux/i386 ext2 filesystem 483/dev/hda5: Linux/i386 swap file 484/dev/hda6: Linux/i386 swap file 485/dev/hda7: Linux/i386 swap file 486/dev/hda8: Linux/i386 swap file 487/dev/hda9: empty 488/dev/hda10: empty 489 490$ file -i file.c file /dev/{wd0a,hda} 491file.c: text/x-c 492file: application/x-executable 493/dev/hda: application/x-not-regular-file 494/dev/wd0a: application/x-not-regular-file 495 496.Ed 497.Sh HISTORY 498There has been a 499.Nm 500command in every 501.Dv UNIX since at least Research Version 4 502(man page dated November, 1973). 503The System V version introduced one significant major change: 504the external list of magic types. 505This slowed the program down slightly but made it a lot more flexible. 506.Pp 507This program, based on the System V version, 508was written by Ian Darwin 509.Aq ian@darwinsys.com 510without looking at anybody else's source code. 511.Pp 512John Gilmore revised the code extensively, making it better than 513the first version. 514Geoff Collyer found several inadequacies 515and provided some magic file entries. 516Contributions by the 517.Sq \*[Am] 518operator by Rob McMahon, 519.Aq cudcv@warwick.ac.uk , 5201989. 521.Pp 522Guy Harris, 523.Aq guy@netapp.com , 524made many changes from 1993 to the present. 5251989. 526.Pp 527Primary development and maintenance from 1990 to the present by 528Christos Zoulas 529.Aq christos@astron.com . 530.Pp 531Altered by Chris Lowth 532.Aq chris@lowth.com , 5332000: handle the 534.Fl i 535option to output mime type strings, using an alternative 536magic file and internal logic. 537.Pp 538Altered by Eric Fischer 539.Aq enf@pobox.com , 540July, 2000, 541to identify character codes and attempt to identify the languages 542of non-ASCII files. 543.Pp 544Altered by Reuben Thomas 545.Aq rrt@sc3d.org , 5462007-2011, to improve MIME support, merge MIME and non-MIME magic, 547support directories as well as files of magic, apply many bug fixes, 548update and fix a lot of magic, improve the build system, improve the 549documentation, and rewrite the Python bindings in pure Python. 550.Pp 551The list of contributors to the 552.Sq magic 553directory (magic files) 554is too long to include here. 555You know who you are; thank you. 556Many contributors are listed in the source files. 557.Sh LEGAL NOTICE 558Copyright (c) Ian F. Darwin, Toronto, Canada, 1986-1999. 559Covered by the standard Berkeley Software Distribution copyright; see the file 560COPYING in the source distribution. 561.Pp 562The files 563.Pa tar.h 564and 565.Pa is_tar.c 566were written by John Gilmore from his public-domain 567.Xr tar 1 568program, and are not covered by the above license. 569.Sh RETURN CODE 570.Nm 571returns 0 on success, and non-zero on error. 572.Sh BUGS 573.Pp 574Please report bugs and send patches to the bug tracker at 575.Pa http://bugs.gw.com/ 576or the mailing list at 577.Aq file@mx.gw.com 578(visit 579.Pa http://mx.gw.com/mailman/listinfo/file 580first to subscribe). 581.Sh TODO 582.Pp 583Fix output so that tests for MIME and APPLE flags are not needed all 584over the place, and actual output is only done in one place. 585This needs a design. 586Suggestion: push possible outputs on to a list, then pick the 587last-pushed (most specific, one hopes) value at the end, or 588use a default if the list is empty. 589This should not slow down evaluation. 590.Pp 591Continue to squash all magic bugs. 592See Debian BTS for a good source. 593.Pp 594Store arbitrarily long strings, for example for %s patterns, so that 595they can be printed out. 596Fixes Debian bug #271672. 597Would require more complex store/load code in apprentice. 598.Pp 599Add syntax for relative offsets after current level (Debian bug #466037). 600.Pp 601Make file -ki work, i.e. give multiple MIME types. 602.Pp 603Add a zip library so we can peek inside Office2007 documents to 604figure out what they are. 605.Pp 606Add an option to print URLs for the sources of the file descriptions. 607.Pp 608Combine script searches and add a way to map executable names to MIME 609types (e.g. have a magic value for !:mime which causes the resulting 610string to be looked up in a table). 611This would avoid adding the same magic repeatedly for each new 612hash-bang interpreter. 613.Pp 614Fix 615.Dq name 616and 617.Dq use 618to check for consistency at compile time (duplicate 619.Dq name , 620.Dq use 621pointing to undefined 622.Dq name 623). 624Make 625.Dq name 626/ 627.Dq use 628more efficient by keeping a sorted list of names. 629Special-case ^ to flip endianness in the parser so that it does not 630have to be escaped, and document it. 631.Sh AVAILABILITY 632You can obtain the original author's latest version by anonymous FTP 633on 634.Pa ftp.astron.com 635in the directory 636.Pa /pub/file/file-X.YZ.tar.gz . 637