1=encoding utf8 2 3=head1 NAME 4 5perllocale - Perl locale handling (internationalization and localization) 6 7=head1 DESCRIPTION 8 9In the beginning there was ASCII, the "American Standard Code for 10Information Interchange", which works quite well for Americans with 11their English alphabet and dollar-denominated currency. But it doesn't 12work so well even for other English speakers, who may use different 13currencies, such as the pound sterling (as the symbol for that currency 14is not in ASCII); and it's hopelessly inadequate for many of the 15thousands of the world's other languages. 16 17To address these deficiencies, the concept of locales was invented 18(formally the ISO C, XPG4, POSIX 1.c "locale system"). And applications 19were and are being written that use the locale mechanism. The process of 20making such an application take account of its users' preferences in 21these kinds of matters is called B<internationalization> (often 22abbreviated as B<i18n>); telling such an application about a particular 23set of preferences is known as B<localization> (B<l10n>). 24 25Perl has been extended to support certain types of locales available in 26the locale system. This is controlled per application by using one 27pragma, one function call, and several environment variables. 28 29Perl supports single-byte locales that are supersets of ASCII, such as 30the ISO 8859 ones, and one multi-byte-type locale, UTF-8 ones, described 31in the next paragraph. Perl doesn't support any other multi-byte 32locales, such as the ones for East Asian languages. 33 34Unfortunately, there are quite a few deficiencies with the design (and 35often, the implementations) of locales. Unicode was invented (see 36L<perlunitut> for an introduction to that) in part to address these 37design deficiencies, and nowadays, there is a series of "UTF-8 38locales", based on Unicode. These are locales whose character set is 39Unicode, encoded in UTF-8. Starting in v5.20, Perl fully supports 40UTF-8 locales, except for sorting and string comparisons like C<lt> and 41C<ge>. Starting in v5.26, Perl can handle these reasonably as well, 42depending on the platform's implementation. However, for earlier 43releases or for better control, use L<Unicode::Collate>. There are 44actually two slightly different types of UTF-8 locales: one for Turkic 45languages and one for everything else. 46 47Starting in Perl v5.30, Perl detects Turkic locales by their 48behaviour, and seamlessly handles both types; previously only the 49non-Turkic one was supported. The name of the locale is ignored, if 50your system has a C<tr_TR.UTF-8> locale and it doesn't behave like a 51Turkic locale, perl will treat it like a non-Turkic locale. 52 53Perl continues to support the old non UTF-8 locales as well. There are 54currently no UTF-8 locales for EBCDIC platforms. 55 56(Unicode is also creating C<CLDR>, the "Common Locale Data Repository", 57L<http://cldr.unicode.org/> which includes more types of information than 58are available in the POSIX locale system. At the time of this writing, 59there was no CPAN module that provides access to this XML-encoded data. 60However, it is possible to compute the POSIX locale data from them, and 61earlier CLDR versions had these already extracted for you as UTF-8 locales 62L<http://unicode.org/Public/cldr/2.0.1/>.) 63 64=head1 WHAT IS A LOCALE 65 66A locale is a set of data that describes various aspects of how various 67communities in the world categorize their world. These categories are 68broken down into the following types (some of which include a brief 69note here): 70 71=over 72 73=item Category C<LC_NUMERIC>: Numeric formatting 74 75This indicates how numbers should be formatted for human readability, 76for example the character used as the decimal point. 77 78=item Category C<LC_MONETARY>: Formatting of monetary amounts 79 80Z<> 81 82=item Category C<LC_TIME>: Date/Time formatting 83 84Z<> 85 86=item Category C<LC_MESSAGES>: Error and other messages 87 88This is used by Perl itself only for accessing operating system error 89messages via L<C<$!>|perlvar/$ERRNO> and L<C<$^E>|perlvar/$EXTENDED_OS_ERROR>. 90 91=item Category C<LC_COLLATE>: Collation 92 93This indicates the ordering of letters for comparison and sorting. 94In Latin alphabets, for example, "b", generally follows "a". 95 96=item Category C<LC_CTYPE>: Character Types 97 98This indicates, for example if a character is an uppercase letter. 99 100=item Other categories 101 102Some platforms have other categories, dealing with such things as 103measurement units and paper sizes. None of these are used directly by 104Perl, but outside operations that Perl interacts with may use 105these. See L</Not within the scope of "use locale"> below. 106 107=back 108 109More details on the categories used by Perl are given below in L</LOCALE 110CATEGORIES>. 111 112Together, these categories go a long way towards being able to customize 113a single program to run in many different locations. But there are 114deficiencies, so keep reading. 115 116=head1 PREPARING TO USE LOCALES 117 118Perl itself (outside the L<POSIX> module) will not use locales unless 119specifically requested to (but 120again note that Perl may interact with code that does use them). Even 121if there is such a request, B<all> of the following must be true 122for it to work properly: 123 124=over 4 125 126=item * 127 128B<Your operating system must support the locale system>. If it does, 129you should find that the C<setlocale()> function is a documented part of 130its C library. 131 132=item * 133 134B<Definitions for locales that you use must be installed>. You, or 135your system administrator, must make sure that this is the case. The 136available locales, the location in which they are kept, and the manner 137in which they are installed all vary from system to system. Some systems 138provide only a few, hard-wired locales and do not allow more to be 139added. Others allow you to add "canned" locales provided by the system 140supplier. Still others allow you or the system administrator to define 141and add arbitrary locales. (You may have to ask your supplier to 142provide canned locales that are not delivered with your operating 143system.) Read your system documentation for further illumination. 144 145=item * 146 147B<Perl must believe that the locale system is supported>. If it does, 148C<perl -V:d_setlocale> will say that the value for C<d_setlocale> is 149C<define>. 150 151=back 152 153If you want a Perl application to process and present your data 154according to a particular locale, the application code should include 155the S<C<use locale>> pragma (see L</The "use locale" pragma>) where 156appropriate, and B<at least one> of the following must be true: 157 158=over 4 159 160=item 1 161 162B<The locale-determining environment variables (see L</"ENVIRONMENT">) 163must be correctly set up> at the time the application is started, either 164by yourself or by whomever set up your system account; or 165 166=item 2 167 168B<The application must set its own locale> using the method described in 169L</The setlocale function>. 170 171=back 172 173=head1 USING LOCALES 174 175=head2 The C<"use locale"> pragma 176 177Starting in Perl 5.28, this pragma may be used in 178L<multi-threaded|threads> applications on systems that have thread-safe 179locale ability. Some caveats apply, see L</Multi-threaded> below. On 180systems without this capability, or in earlier Perls, do NOT use this 181pragma in scripts that have multiple L<threads|threads> active. The 182locale in these cases is not local to a single thread. Another thread 183may change the locale at any time, which could cause at a minimum that a 184given thread is operating in a locale it isn't expecting to be in. On 185some platforms, segfaults can also occur. The locale change need not be 186explicit; some operations cause perl itself to change the locale. You 187are vulnerable simply by having done a S<C<"use locale">>. 188 189By default, Perl itself (outside the L<POSIX> module) 190ignores the current locale. The S<C<use locale>> 191pragma tells Perl to use the current locale for some operations. 192Starting in v5.16, there are optional parameters to this pragma, 193described below, which restrict which operations are affected by it. 194 195The current locale is set at execution time by 196L<setlocale()|/The setlocale function> described below. If that function 197hasn't yet been called in the course of the program's execution, the 198current locale is that which was determined by the L</"ENVIRONMENT"> in 199effect at the start of the program. 200If there is no valid environment, the current locale is whatever the 201system default has been set to. On POSIX systems, it is likely, but 202not necessarily, the "C" locale. On Windows, the default is set via the 203computer's S<C<Control Panel-E<gt>Regional and Language Options>> (or its 204current equivalent). 205 206The operations that are affected by locale are: 207 208=over 4 209 210=item B<Not within the scope of C<"use locale">> 211 212Only certain operations (all originating outside Perl) should be 213affected, as follows: 214 215=over 4 216 217=item * 218 219The current locale is used when going outside of Perl with 220operations like L<system()|perlfunc/system LIST> or 221L<qxE<sol>E<sol>|perlop/qxE<sol>STRINGE<sol>>, if those operations are 222locale-sensitive. 223 224=item * 225 226Also Perl gives access to various C library functions through the 227L<POSIX> module. Some of those functions are always affected by the 228current locale. For example, C<POSIX::strftime()> uses C<LC_TIME>; 229C<POSIX::strtod()> uses C<LC_NUMERIC>; C<POSIX::strcoll()> and 230C<POSIX::strxfrm()> use C<LC_COLLATE>. All such functions 231will behave according to the current underlying locale, even if that 232locale isn't exposed to Perl space. 233 234This applies as well to L<I18N::Langinfo>. 235 236=item * 237 238XS modules for all categories but C<LC_NUMERIC> get the underlying 239locale, and hence any C library functions they call will use that 240underlying locale. For more discussion, see L<perlxs/CAVEATS>. 241 242=back 243 244Note that all C programs (including the perl interpreter, which is 245written in C) always have an underlying locale. That locale is the "C" 246locale unless changed by a call to L<setlocale()|/The setlocale 247function>. When Perl starts up, it changes the underlying locale to the 248one which is indicated by the L</ENVIRONMENT>. When using the L<POSIX> 249module or writing XS code, it is important to keep in mind that the 250underlying locale may be something other than "C", even if the program 251hasn't explicitly changed it. 252 253Z<> 254 255=item B<Lingering effects of C<S<use locale>>> 256 257Certain Perl operations that are set-up within the scope of a 258C<use locale> retain that effect even outside the scope. 259These include: 260 261=over 4 262 263=item * 264 265The output format of a L<write()|perlfunc/write> is determined by an 266earlier format declaration (L<perlfunc/format>), so whether or not the 267output is affected by locale is determined by if the C<format()> is 268within the scope of a C<use locale>, not whether the C<write()> 269is. 270 271=item * 272 273Regular expression patterns can be compiled using 274L<qrE<sol>E<sol>|perlop/qrE<sol>STRINGE<sol>msixpodualn> with actual 275matching deferred to later. Again, it is whether or not the compilation 276was done within the scope of C<use locale> that determines the match 277behavior, not if the matches are done within such a scope or not. 278 279=back 280 281Z<> 282 283=item B<Under C<"use locale";>> 284 285=over 4 286 287=item * 288 289All the above operations 290 291=item * 292 293B<Format declarations> (L<perlfunc/format>) and hence any subsequent 294C<write()>s use C<LC_NUMERIC>. 295 296=item * 297 298B<stringification and output> use C<LC_NUMERIC>. 299These include the results of 300C<print()>, 301C<printf()>, 302C<say()>, 303and 304C<sprintf()>. 305 306=item * 307 308B<The comparison operators> (C<lt>, C<le>, C<cmp>, C<ge>, and C<gt>) use 309C<LC_COLLATE>. C<sort()> is also affected if used without an 310explicit comparison function, because it uses C<cmp> by default. 311 312B<Note:> C<eq> and C<ne> are unaffected by locale: they always 313perform a char-by-char comparison of their scalar operands. What's 314more, if C<cmp> finds that its operands are equal according to the 315collation sequence specified by the current locale, it goes on to 316perform a char-by-char comparison, and only returns I<0> (equal) if the 317operands are char-for-char identical. If you really want to know whether 318two strings--which C<eq> and C<cmp> may consider different--are equal 319as far as collation in the locale is concerned, see the discussion in 320L</Category C<LC_COLLATE>: Collation>. 321 322=item * 323 324B<Regular expressions and case-modification functions> (C<uc()>, C<lc()>, 325C<ucfirst()>, and C<lcfirst()>) use C<LC_CTYPE> 326 327=item * 328 329B<The variables L<C<$!>|perlvar/$ERRNO>> (and its synonyms C<$ERRNO> and 330C<$OS_ERROR>) B<and> L<C<$^E>|perlvar/$EXTENDED_OS_ERROR>> (and its synonym 331C<$EXTENDED_OS_ERROR>) when used as strings use C<LC_MESSAGES>. 332 333=back 334 335=back 336 337The default behavior is restored with the S<C<no locale>> pragma, or 338upon reaching the end of the block enclosing C<use locale>. 339Note that C<use locale> calls may be 340nested, and that what is in effect within an inner scope will revert to 341the outer scope's rules at the end of the inner scope. 342 343The string result of any operation that uses locale 344information is tainted (if your perl supports taint checking), 345as it is possible for a locale to be untrustworthy. 346See L</"SECURITY">. 347 348Starting in Perl v5.16 in a very limited way, and more generally in 349v5.22, you can restrict which category or categories are enabled by this 350particular instance of the pragma by adding parameters to it. For 351example, 352 353 use locale qw(:ctype :numeric); 354 355enables locale awareness within its scope of only those operations 356(listed above) that are affected by C<LC_CTYPE> and C<LC_NUMERIC>. 357 358The possible categories are: C<:collate>, C<:ctype>, C<:messages>, 359C<:monetary>, C<:numeric>, C<:time>, and the pseudo category 360C<:characters> (described below). 361 362Thus you can say 363 364 use locale ':messages'; 365 366and only L<C<$!>|perlvar/$ERRNO> and L<C<$^E>|perlvar/$EXTENDED_OS_ERROR> 367will be locale aware. Everything else is unaffected. 368 369Since Perl doesn't currently do anything with the C<LC_MONETARY> 370category, specifying C<:monetary> does effectively nothing. Some 371systems have other categories, such as C<LC_PAPER>, but Perl 372also doesn't do anything with them, and there is no way to specify 373them in this pragma's arguments. 374 375You can also easily say to use all categories but one, by either, for 376example, 377 378 use locale ':!ctype'; 379 use locale ':not_ctype'; 380 381both of which mean to enable locale awareness of all categories but 382C<LC_CTYPE>. Only one category argument may be specified in a 383S<C<use locale>> if it is of the negated form. 384 385Prior to v5.22 only one form of the pragma with arguments is available: 386 387 use locale ':not_characters'; 388 389(and you have to say C<not_>; you can't use the bang C<!> form). This 390pseudo category is a shorthand for specifying both C<:collate> and 391C<:ctype>. Hence, in the negated form, it is nearly the same thing as 392saying 393 394 use locale qw(:messages :monetary :numeric :time); 395 396We use the term "nearly", because C<:not_characters> also turns on 397S<C<use feature 'unicode_strings'>> within its scope. This form is 398less useful in v5.20 and later, and is described fully in 399L</Unicode and UTF-8>, but briefly, it tells Perl to not use the 400character portions of the locale definition, that is the C<LC_CTYPE> and 401C<LC_COLLATE> categories. Instead it will use the native character set 402(extended by Unicode). When using this parameter, you are responsible 403for getting the external character set translated into the 404native/Unicode one (which it already will be if it is one of the 405increasingly popular UTF-8 locales). There are convenient ways of doing 406this, as described in L</Unicode and UTF-8>. 407 408=head2 The setlocale function 409 410WARNING! Prior to Perl 5.28 or on a system that does not support 411thread-safe locale operations, do NOT use this function in a 412L<thread|threads>. The locale will change in all other threads at the 413same time, and should your thread get paused by the operating system, 414and another started, that thread will not have the locale it is 415expecting. On some platforms, there can be a race leading to segfaults 416if two threads call this function nearly simultaneously. This warning 417does not apply on unthreaded builds, or on perls where 418C<${^SAFE_LOCALES}> exists and is non-zero; namely Perl 5.28 and later 419unthreaded or compiled to be locale-thread-safe. On z/OS systems, this 420function becomes a no-op once any thread is started. Thus, on that 421system, you can set up the locale before creating any threads, and that 422locale will be the one in effect for the entire program. 423 424Otherwise, you can switch locales as often as you wish at run time with 425the C<POSIX::setlocale()> function: 426 427 # Import locale-handling tool set from POSIX module. 428 # This example uses: setlocale -- the function call 429 # LC_CTYPE -- explained below 430 # (Showing the testing for success/failure of operations is 431 # omitted in these examples to avoid distracting from the main 432 # point) 433 434 use POSIX qw(locale_h); 435 use locale; 436 my $old_locale; 437 438 # query and save the old locale 439 $old_locale = setlocale(LC_CTYPE); 440 441 setlocale(LC_CTYPE, "fr_CA.ISO8859-1"); 442 # LC_CTYPE now in locale "French, Canada, codeset ISO 8859-1" 443 444 setlocale(LC_CTYPE, ""); 445 # LC_CTYPE now reset to the default defined by the 446 # LC_ALL/LC_CTYPE/LANG environment variables, or to the system 447 # default. See below for documentation. 448 449 # restore the old locale 450 setlocale(LC_CTYPE, $old_locale); 451 452The first argument of C<setlocale()> gives the B<category>, the second the 453B<locale>. The category tells in what aspect of data processing you 454want to apply locale-specific rules. Category names are discussed in 455L</LOCALE CATEGORIES> and L</"ENVIRONMENT">. The locale is the name of a 456collection of customization information corresponding to a particular 457combination of language, country or territory, and codeset. Read on for 458hints on the naming of locales: not all systems name locales as in the 459example. 460 461If no second argument is provided and the category is something other 462than C<LC_ALL>, the function returns a string naming the current locale 463for the category. You can use this value as the second argument in a 464subsequent call to C<setlocale()>, B<but> on some platforms the string 465is opaque, not something that most people would be able to decipher as 466to what locale it means. 467 468If no second argument is provided and the category is C<LC_ALL>, the 469result is implementation-dependent. It may be a string of 470concatenated locale names (separator also implementation-dependent) 471or a single locale name. Please consult your L<setlocale(3)> man page for 472details. 473 474If a second argument is given and it corresponds to a valid locale, 475the locale for the category is set to that value, and the function 476returns the now-current locale value. You can then use this in yet 477another call to C<setlocale()>. (In some implementations, the return 478value may sometimes differ from the value you gave as the second 479argument--think of it as an alias for the value you gave.) 480 481As the example shows, if the second argument is an empty string, the 482category's locale is returned to the default specified by the 483corresponding environment variables. Generally, this results in a 484return to the default that was in force when Perl started up: changes 485to the environment made by the application after startup may or may not 486be noticed, depending on your system's C library. 487 488Note that when a form of C<use locale> that doesn't include all 489categories is specified, Perl ignores the excluded categories. 490 491If C<setlocale()> fails for some reason (for example, an attempt to set 492to a locale unknown to the system), the locale for the category is not 493changed, and the function returns C<undef>. 494 495Starting in Perl 5.28, on multi-threaded perls compiled on systems that 496implement POSIX 2008 thread-safe locale operations, this function 497doesn't actually call the system C<setlocale>. Instead those 498thread-safe operations are used to emulate the C<setlocale> function, 499but in a thread-safe manner. 500 501You can force the thread-safe locale operations to always be used (if 502available) by recompiling perl with 503 504 -Accflags='-DUSE_THREAD_SAFE_LOCALE' 505 506added to your call to F<Configure>. 507 508For further information about the categories, consult L<setlocale(3)>. 509 510=head2 Multi-threaded operation 511 512Beginning in Perl 5.28, multi-threaded locale operation is supported on 513systems that implement either the POSIX 2008 or Windows-specific 514thread-safe locale operations. Many modern systems, such as various 515Unix variants and Darwin do have this. 516 517You can tell if using locales is safe on your system by looking at the 518read-only boolean variable C<${^SAFE_LOCALES}>. The value is 1 if the 519perl is not threaded, or if it is using thread-safe locale operations. 520 521Thread-safe operations are supported in Windows starting in Visual Studio 5222005, and in systems compatible with POSIX 2008. Some platforms claim 523to support POSIX 2008, but have buggy implementations, so that the hints 524files for compiling to run on them turn off attempting to use 525thread-safety. C<${^SAFE_LOCALES}> will be 0 on them. 526 527Be aware that writing a multi-threaded application will not be portable 528to a platform which lacks the native thread-safe locale support. On 529systems that do have it, you automatically get this behavior for 530threaded perls, without having to do anything. If for some reason, you 531don't want to use this capability (perhaps the POSIX 2008 support is 532buggy on your system), you can manually compile Perl to use the old 533non-thread-safe implementation by passing the argument 534C<-Accflags='-DNO_THREAD_SAFE_LOCALE'> to F<Configure>. 535Except on Windows, this will continue to use certain of the POSIX 2008 536functions in some situations. If these are buggy, you can pass the 537following to F<Configure> instead or additionally: 538C<-Accflags='-DNO_POSIX_2008_LOCALE'>. This will also keep the code 539from using thread-safe locales. 540C<${^SAFE_LOCALES}> will be 0 on systems that turn off the thread-safe 541operations. 542 543Normally on unthreaded builds, the traditional C<setlocale()> is used 544and not the thread-safe locale functions. You can force the use of these 545on systems that have them by adding the 546C<-Accflags='-DUSE_THREAD_SAFE_LOCALE'> to F<Configure>. 547 548The initial program is started up using the locale specified from the 549environment, as currently, described in L</ENVIRONMENT>. All newly 550created threads start with C<LC_ALL> set to C<"C">. Each thread may 551use C<POSIX::setlocale()> to query or switch its locale at any time, 552without affecting any other thread. All locale-dependent operations 553automatically use their thread's locale. 554 555This should be completely transparent to any applications written 556entirely in Perl (minus a few rarely encountered caveats given in the 557L</Multi-threaded> section). Information for XS module writers is given 558in L<perlxs/Locale-aware XS code>. 559 560=head2 Finding locales 561 562For locales available in your system, consult also L<setlocale(3)> to 563see whether it leads to the list of available locales (search for the 564I<SEE ALSO> section). If that fails, try the following command lines: 565 566 locale -a 567 568 nlsinfo 569 570 ls /usr/lib/nls/loc 571 572 ls /usr/lib/locale 573 574 ls /usr/lib/nls 575 576 ls /usr/share/locale 577 578and see whether they list something resembling these 579 580 en_US.ISO8859-1 de_DE.ISO8859-1 ru_RU.ISO8859-5 581 en_US.iso88591 de_DE.iso88591 ru_RU.iso88595 582 en_US de_DE ru_RU 583 en de ru 584 english german russian 585 english.iso88591 german.iso88591 russian.iso88595 586 english.roman8 russian.koi8r 587 588Sadly, even though the calling interface for C<setlocale()> has been 589standardized, names of locales and the directories where the 590configuration resides have not been. The basic form of the name is 591I<language_territory>B<.>I<codeset>, but the latter parts after 592I<language> are not always present. The I<language> and I<country> 593are usually from the standards B<ISO 3166> and B<ISO 639>, the 594two-letter abbreviations for the countries and the languages of the 595world, respectively. The I<codeset> part often mentions some B<ISO 5968859> character set, the Latin codesets. For example, C<ISO 8859-1> 597is the so-called "Western European codeset" that can be used to encode 598most Western European languages adequately. Again, there are several 599ways to write even the name of that one standard. Lamentably. 600 601Two special locales are worth particular mention: "C" and "POSIX". 602Currently these are effectively the same locale: the difference is 603mainly that the first one is defined by the C standard, the second by 604the POSIX standard. They define the B<default locale> in which 605every program starts in the absence of locale information in its 606environment. (The I<default> default locale, if you will.) Its language 607is (American) English and its character codeset ASCII or, rarely, a 608superset thereof (such as the "DEC Multinational Character Set 609(DEC-MCS)"). B<Warning>. The C locale delivered by some vendors 610may not actually exactly match what the C standard calls for. So 611beware. 612 613B<NOTE>: Not all systems have the "POSIX" locale (not all systems are 614POSIX-conformant), so use "C" when you need explicitly to specify this 615default locale. 616 617=head2 LOCALE PROBLEMS 618 619You may encounter the following warning message at Perl startup: 620 621 perl: warning: Setting locale failed. 622 perl: warning: Please check that your locale settings: 623 LC_ALL = "En_US", 624 LANG = (unset) 625 are supported and installed on your system. 626 perl: warning: Falling back to the standard locale ("C"). 627 628This means that your locale settings had C<LC_ALL> set to "En_US" and 629LANG exists but has no value. Perl tried to believe you but could not. 630Instead, Perl gave up and fell back to the "C" locale, the default locale 631that is supposed to work no matter what. (On Windows, it first tries 632falling back to the system default locale.) This usually means your 633locale settings were wrong, they mention locales your system has never 634heard of, or the locale installation in your system has problems (for 635example, some system files are broken or missing). There are quick and 636temporary fixes to these problems, as well as more thorough and lasting 637fixes. 638 639=head2 Testing for broken locales 640 641If you are building Perl from source, the Perl test suite file 642F<lib/locale.t> can be used to test the locales on your system. 643Setting the environment variable C<PERL_DEBUG_FULL_TEST> to 1 644will cause it to output detailed results. For example, on Linux, you 645could say 646 647 PERL_DEBUG_FULL_TEST=1 ./perl -T -Ilib lib/locale.t > locale.log 2>&1 648 649Besides many other tests, it will test every locale it finds on your 650system to see if they conform to the POSIX standard. If any have 651errors, it will include a summary near the end of the output of which 652locales passed all its tests, and which failed, and why. 653 654=head2 Temporarily fixing locale problems 655 656The two quickest fixes are either to render Perl silent about any 657locale inconsistencies or to run Perl under the default locale "C". 658 659Perl's moaning about locale problems can be silenced by setting the 660environment variable C<PERL_BADLANG> to "0" or "". 661This method really just sweeps the problem under the carpet: you tell 662Perl to shut up even when Perl sees that something is wrong. Do not 663be surprised if later something locale-dependent misbehaves. 664 665Perl can be run under the "C" locale by setting the environment 666variable C<LC_ALL> to "C". This method is perhaps a bit more civilized 667than the C<PERL_BADLANG> approach, but setting C<LC_ALL> (or 668other locale variables) may affect other programs as well, not just 669Perl. In particular, external programs run from within Perl will see 670these changes. If you make the new settings permanent (read on), all 671programs you run see the changes. See L</"ENVIRONMENT"> for 672the full list of relevant environment variables and L</"USING LOCALES"> 673for their effects in Perl. Effects in other programs are 674easily deducible. For example, the variable C<LC_COLLATE> may well affect 675your B<sort> program (or whatever the program that arranges "records" 676alphabetically in your system is called). 677 678You can test out changing these variables temporarily, and if the 679new settings seem to help, put those settings into your shell startup 680files. Consult your local documentation for the exact details. For 681Bourne-like shells (B<sh>, B<ksh>, B<bash>, B<zsh>): 682 683 LC_ALL=en_US.ISO8859-1 684 export LC_ALL 685 686This assumes that we saw the locale "en_US.ISO8859-1" using the commands 687discussed above. We decided to try that instead of the above faulty 688locale "En_US"--and in Cshish shells (B<csh>, B<tcsh>) 689 690 setenv LC_ALL en_US.ISO8859-1 691 692or if you have the "env" application you can do (in any shell) 693 694 env LC_ALL=en_US.ISO8859-1 perl ... 695 696If you do not know what shell you have, consult your local 697helpdesk or the equivalent. 698 699=head2 Permanently fixing locale problems 700 701The slower but superior fixes are when you may be able to yourself 702fix the misconfiguration of your own environment variables. The 703mis(sing)configuration of the whole system's locales usually requires 704the help of your friendly system administrator. 705 706First, see earlier in this document about L</Finding locales>. That tells 707how to find which locales are really supported--and more importantly, 708installed--on your system. In our example error message, environment 709variables affecting the locale are listed in the order of decreasing 710importance (and unset variables do not matter). Therefore, having 711LC_ALL set to "En_US" must have been the bad choice, as shown by the 712error message. First try fixing locale settings listed first. 713 714Second, if using the listed commands you see something B<exactly> 715(prefix matches do not count and case usually counts) like "En_US" 716without the quotes, then you should be okay because you are using a 717locale name that should be installed and available in your system. 718In this case, see L</Permanently fixing your system's locale configuration>. 719 720=head2 Permanently fixing your system's locale configuration 721 722This is when you see something like: 723 724 perl: warning: Please check that your locale settings: 725 LC_ALL = "En_US", 726 LANG = (unset) 727 are supported and installed on your system. 728 729but then cannot see that "En_US" listed by the above-mentioned 730commands. You may see things like "en_US.ISO8859-1", but that isn't 731the same. In this case, try running under a locale 732that you can list and which somehow matches what you tried. The 733rules for matching locale names are a bit vague because 734standardization is weak in this area. See again the 735L</Finding locales> about general rules. 736 737=head2 Fixing system locale configuration 738 739Contact a system administrator (preferably your own) and report the exact 740error message you get, and ask them to read this same documentation you 741are now reading. They should be able to check whether there is something 742wrong with the locale configuration of the system. The L</Finding locales> 743section is unfortunately a bit vague about the exact commands and places 744because these things are not that standardized. 745 746=head2 The localeconv function 747 748The C<POSIX::localeconv()> function allows you to get particulars of the 749locale-dependent numeric formatting information specified by the current 750underlying C<LC_NUMERIC> and C<LC_MONETARY> locales (regardless of 751whether called from within the scope of C<S<use locale>> or not). (If 752you just want the name of 753the current locale for a particular category, use C<POSIX::setlocale()> 754with a single parameter--see L</The setlocale function>.) 755 756 use POSIX qw(locale_h); 757 758 # Get a reference to a hash of locale-dependent info 759 $locale_values = localeconv(); 760 761 # Output sorted list of the values 762 for (sort keys %$locale_values) { 763 printf "%-20s = %s\n", $_, $locale_values->{$_} 764 } 765 766C<localeconv()> takes no arguments, and returns B<a reference to> a hash. 767The keys of this hash are variable names for formatting, such as 768C<decimal_point> and C<thousands_sep>. The values are the 769corresponding, er, values. See L<POSIX/localeconv> for a longer 770example listing the categories an implementation might be expected to 771provide; some provide more and others fewer. You don't need an 772explicit C<use locale>, because C<localeconv()> always observes the 773current locale. 774 775Here's a simple-minded example program that rewrites its command-line 776parameters as integers correctly formatted in the current locale: 777 778 use POSIX qw(locale_h); 779 780 # Get some of locale's numeric formatting parameters 781 my ($thousands_sep, $grouping) = 782 @{localeconv()}{'thousands_sep', 'grouping'}; 783 784 # Apply defaults if values are missing 785 $thousands_sep = ',' unless $thousands_sep; 786 787 # grouping and mon_grouping are packed lists 788 # of small integers (characters) telling the 789 # grouping (thousand_seps and mon_thousand_seps 790 # being the group dividers) of numbers and 791 # monetary quantities. The integers' meanings: 792 # 255 means no more grouping, 0 means repeat 793 # the previous grouping, 1-254 means use that 794 # as the current grouping. Grouping goes from 795 # right to left (low to high digits). In the 796 # below we cheat slightly by never using anything 797 # else than the first grouping (whatever that is). 798 if ($grouping) { 799 @grouping = unpack("C*", $grouping); 800 } else { 801 @grouping = (3); 802 } 803 804 # Format command line params for current locale 805 for (@ARGV) { 806 $_ = int; # Chop non-integer part 807 1 while 808 s/(\d)(\d{$grouping[0]}($|$thousands_sep))/$1$thousands_sep$2/; 809 print "$_"; 810 } 811 print "\n"; 812 813Note that if the platform doesn't have C<LC_NUMERIC> and/or 814C<LC_MONETARY> available or enabled, the corresponding elements of the 815hash will be missing. 816 817=head2 I18N::Langinfo 818 819Another interface for querying locale-dependent information is the 820C<I18N::Langinfo::langinfo()> function. 821 822The following example will import the C<langinfo()> function itself and 823three constants to be used as arguments to C<langinfo()>: a constant for 824the abbreviated first day of the week (the numbering starts from 825Sunday = 1) and two more constants for the affirmative and negative 826answers for a yes/no question in the current locale. 827 828 use I18N::Langinfo qw(langinfo ABDAY_1 YESSTR NOSTR); 829 830 my ($abday_1, $yesstr, $nostr) 831 = map { langinfo } qw(ABDAY_1 YESSTR NOSTR); 832 833 print "$abday_1? [$yesstr/$nostr] "; 834 835In other words, in the "C" (or English) locale the above will probably 836print something like: 837 838 Sun? [yes/no] 839 840See L<I18N::Langinfo> for more information. 841 842=head1 LOCALE CATEGORIES 843 844The following subsections describe basic locale categories. Beyond these, 845some combination categories allow manipulation of more than one 846basic category at a time. See L</"ENVIRONMENT"> for a discussion of these. 847 848=head2 Category C<LC_COLLATE>: Collation: Text Comparisons and Sorting 849 850In the scope of a S<C<use locale>> form that includes collation, Perl 851looks to the C<LC_COLLATE> 852environment variable to determine the application's notions on collation 853(ordering) of characters. For example, "b" follows "a" in Latin 854alphabets, but where do "E<aacute>" and "E<aring>" belong? And while 855"color" follows "chocolate" in English, what about in traditional Spanish? 856 857The following collations all make sense and you may meet any of them 858if you C<"use locale">. 859 860 A B C D E a b c d e 861 A a B b C c D d E e 862 a A b B c C d D e E 863 a b c d e A B C D E 864 865Here is a code snippet to tell what "word" 866characters are in the current locale, in that locale's order: 867 868 use locale; 869 print +(sort grep /\w/, map { chr } 0..255), "\n"; 870 871Compare this with the characters that you see and their order if you 872state explicitly that the locale should be ignored: 873 874 no locale; 875 print +(sort grep /\w/, map { chr } 0..255), "\n"; 876 877This machine-native collation (which is what you get unless S<C<use 878locale>> has appeared earlier in the same block) must be used for 879sorting raw binary data, whereas the locale-dependent collation of the 880first example is useful for natural text. 881 882As noted in L</USING LOCALES>, C<cmp> compares according to the current 883collation locale when C<use locale> is in effect, but falls back to a 884char-by-char comparison for strings that the locale says are equal. You 885can use C<POSIX::strcoll()> if you don't want this fall-back: 886 887 use POSIX qw(strcoll); 888 $equal_in_locale = 889 !strcoll("space and case ignored", "SpaceAndCaseIgnored"); 890 891C<$equal_in_locale> will be true if the collation locale specifies a 892dictionary-like ordering that ignores space characters completely and 893which folds case. 894 895Perl uses the platform's C library collation functions C<strcoll()> and 896C<strxfrm()>. That means you get whatever they give. On some 897platforms, these functions work well on UTF-8 locales, giving 898a reasonable default collation for the code points that are important in 899that locale. (And if they aren't working well, the problem may only be 900that the locale definition is deficient, so can be fixed by using a 901better definition file. Unicode's definitions (see L</Freely available 902locale definitions>) provide reasonable UTF-8 locale collation 903definitions.) Starting in Perl v5.26, Perl's use of these functions has 904been made more seamless. This may be sufficient for your needs. For 905more control, and to make sure strings containing any code point (not 906just the ones important in the locale) collate properly, the 907L<Unicode::Collate> module is suggested. 908 909In non-UTF-8 locales (hence single byte), code points above 0xFF are 910technically invalid. But if present, again starting in v5.26, they will 911collate to the same position as the highest valid code point does. This 912generally gives good results, but the collation order may be skewed if 913the valid code point gets special treatment when it forms particular 914sequences with other characters as defined by the locale. 915When two strings collate identically, the code point order is used as a 916tie breaker. 917 918If Perl detects that there are problems with the locale collation order, 919it reverts to using non-locale collation rules for that locale. 920 921If you have a single string that you want to check for "equality in 922locale" against several others, you might think you could gain a little 923efficiency by using C<POSIX::strxfrm()> in conjunction with C<eq>: 924 925 use POSIX qw(strxfrm); 926 $xfrm_string = strxfrm("Mixed-case string"); 927 print "locale collation ignores spaces\n" 928 if $xfrm_string eq strxfrm("Mixed-casestring"); 929 print "locale collation ignores hyphens\n" 930 if $xfrm_string eq strxfrm("Mixedcase string"); 931 print "locale collation ignores case\n" 932 if $xfrm_string eq strxfrm("mixed-case string"); 933 934C<strxfrm()> takes a string and maps it into a transformed string for use 935in char-by-char comparisons against other transformed strings during 936collation. "Under the hood", locale-affected Perl comparison operators 937call C<strxfrm()> for both operands, then do a char-by-char 938comparison of the transformed strings. By calling C<strxfrm()> explicitly 939and using a non locale-affected comparison, the example attempts to save 940a couple of transformations. But in fact, it doesn't save anything: Perl 941magic (see L<perlguts/Magic Variables>) creates the transformed version of a 942string the first time it's needed in a comparison, then keeps this version around 943in case it's needed again. An example rewritten the easy way with 944C<cmp> runs just about as fast. It also copes with null characters 945embedded in strings; if you call C<strxfrm()> directly, it treats the first 946null it finds as a terminator. Don't expect the transformed strings 947it produces to be portable across systems--or even from one revision 948of your operating system to the next. In short, don't call C<strxfrm()> 949directly: let Perl do it for you. 950 951Note: C<use locale> isn't shown in some of these examples because it isn't 952needed: C<strcoll()> and C<strxfrm()> are POSIX functions 953which use the standard system-supplied C<libc> functions that 954always obey the current C<LC_COLLATE> locale. 955 956=head2 Category C<LC_CTYPE>: Character Types 957 958In the scope of a S<C<use locale>> form that includes C<LC_CTYPE>, Perl 959obeys the C<LC_CTYPE> locale 960setting. This controls the application's notion of which characters are 961alphabetic, numeric, punctuation, I<etc>. This affects Perl's C<\w> 962regular expression metanotation, 963which stands for alphanumeric characters--that is, alphabetic, 964numeric, and the platform's native underscore. 965(Consult L<perlre> for more information about 966regular expressions.) Thanks to C<LC_CTYPE>, depending on your locale 967setting, characters like "E<aelig>", "E<eth>", "E<szlig>", and 968"E<oslash>" may be understood as C<\w> characters. 969It also affects things like C<\s>, C<\D>, and the POSIX character 970classes, like C<[[:graph:]]>. (See L<perlrecharclass> for more 971information on all these.) 972 973The C<LC_CTYPE> locale also provides the map used in transliterating 974characters between lower and uppercase. This affects the case-mapping 975functions--C<fc()>, C<lc()>, C<lcfirst()>, C<uc()>, and C<ucfirst()>; 976case-mapping 977interpolation with C<\F>, C<\l>, C<\L>, C<\u>, or C<\U> in double-quoted 978strings and C<s///> substitutions; and case-insensitive regular expression 979pattern matching using the C<i> modifier. 980 981Starting in v5.20, Perl supports UTF-8 locales for C<LC_CTYPE>, but 982otherwise Perl only supports single-byte locales, such as the ISO 8859 983series. This means that wide character locales, for example for Asian 984languages, are not well-supported. Use of these locales may cause core 985dumps. If the platform has the capability for Perl to detect such a 986locale, starting in Perl v5.22, L<Perl will warn, default 987enabled|warnings/Category Hierarchy>, using the C<locale> warning 988category, whenever such a locale is switched into. The UTF-8 locale 989support is actually a 990superset of POSIX locales, because it is really full Unicode behavior 991as if no C<LC_CTYPE> locale were in effect at all (except for tainting; 992see L</SECURITY>). POSIX locales, even UTF-8 ones, 993are lacking certain concepts in Unicode, such as the idea that changing 994the case of a character could expand to be more than one character. 995Perl in a UTF-8 locale, will give you that expansion. Prior to v5.20, 996Perl treated a UTF-8 locale on some platforms like an ISO 8859-1 one, 997with some restrictions, and on other platforms more like the "C" locale. 998For releases v5.16 and v5.18, C<S<use locale 'not_characters>> could be 999used as a workaround for this (see L</Unicode and UTF-8>). 1000 1001Note that there are quite a few things that are unaffected by the 1002current locale. Any literal character is the native character for the 1003given platform. Hence 'A' means the character at code point 65 on ASCII 1004platforms, and 193 on EBCDIC. That may or may not be an 'A' in the 1005current locale, if that locale even has an 'A'. 1006Similarly, all the escape sequences for particular characters, 1007C<\n> for example, always mean the platform's native one. This means, 1008for example, that C<\N> in regular expressions (every character 1009but new-line) works on the platform character set. 1010 1011Starting in v5.22, Perl will by default warn when switching into a 1012locale that redefines any ASCII printable character (plus C<\t> and 1013C<\n>) into a different class than expected. This is likely to 1014happen on modern locales only on EBCDIC platforms, where, for example, 1015a CCSID 0037 locale on a CCSID 1047 machine moves C<"[">, but it can 1016happen on ASCII platforms with the ISO 646 and other 10177-bit locales that are essentially obsolete. Things may still work, 1018depending on what features of Perl are used by the program. For 1019example, in the example from above where C<"|"> becomes a C<\w>, and 1020there are no regular expressions where this matters, the program may 1021still work properly. The warning lists all the characters that 1022it can determine could be adversely affected. 1023 1024B<Note:> A broken or malicious C<LC_CTYPE> locale definition may result 1025in clearly ineligible characters being considered to be alphanumeric by 1026your application. For strict matching of (mundane) ASCII letters and 1027digits--for example, in command strings--locale-aware applications 1028should use C<\w> with the C</a> regular expression modifier. See L</"SECURITY">. 1029 1030=head2 Category C<LC_NUMERIC>: Numeric Formatting 1031 1032After a proper C<POSIX::setlocale()> call, and within the scope 1033of a C<use locale> form that includes numerics, Perl obeys the 1034C<LC_NUMERIC> locale information, which controls an application's idea 1035of how numbers should be formatted for human readability. 1036In most implementations the only effect is to 1037change the character used for the decimal point--perhaps from "." to ",". 1038The functions aren't aware of such niceties as thousands separation and 1039so on. (See L</The localeconv function> if you care about these things.) 1040 1041 use POSIX qw(strtod setlocale LC_NUMERIC); 1042 use locale; 1043 1044 setlocale LC_NUMERIC, ""; 1045 1046 $n = 5/2; # Assign numeric 2.5 to $n 1047 1048 $a = " $n"; # Locale-dependent conversion to string 1049 1050 print "half five is $n\n"; # Locale-dependent output 1051 1052 printf "half five is %g\n", $n; # Locale-dependent output 1053 1054 print "DECIMAL POINT IS COMMA\n" 1055 if $n == (strtod("2,5"))[0]; # Locale-dependent conversion 1056 1057See also L<I18N::Langinfo> and C<RADIXCHAR>. 1058 1059=head2 Category C<LC_MONETARY>: Formatting of monetary amounts 1060 1061The C standard defines the C<LC_MONETARY> category, but not a function 1062that is affected by its contents. (Those with experience of standards 1063committees will recognize that the working group decided to punt on the 1064issue.) Consequently, Perl essentially takes no notice of it. If you 1065really want to use C<LC_MONETARY>, you can query its contents--see 1066L</The localeconv function>--and use the information that it returns in your 1067application's own formatting of currency amounts. However, you may well 1068find that the information, voluminous and complex though it may be, still 1069does not quite meet your requirements: currency formatting is a hard nut 1070to crack. 1071 1072See also L<I18N::Langinfo> and C<CRNCYSTR>. 1073 1074=head2 Category C<LC_TIME>: Respresentation of time 1075 1076Output produced by C<POSIX::strftime()>, which builds a formatted 1077human-readable date/time string, is affected by the current C<LC_TIME> 1078locale. Thus, in a French locale, the output produced by the C<%B> 1079format element (full month name) for the first month of the year would 1080be "janvier". Here's how to get a list of long month names in the 1081current locale: 1082 1083 use POSIX qw(strftime); 1084 for (0..11) { 1085 $long_month_name[$_] = 1086 strftime("%B", 0, 0, 0, 1, $_, 96); 1087 } 1088 1089Note: C<use locale> isn't needed in this example: C<strftime()> is a POSIX 1090function which uses the standard system-supplied C<libc> function that 1091always obeys the current C<LC_TIME> locale. 1092 1093See also L<I18N::Langinfo> and C<ABDAY_1>..C<ABDAY_7>, C<DAY_1>..C<DAY_7>, 1094C<ABMON_1>..C<ABMON_12>, and C<ABMON_1>..C<ABMON_12>. 1095 1096=head2 Other categories 1097 1098The remaining locale categories are not currently used by Perl itself. 1099But again note that things Perl interacts with may use these, including 1100extensions outside the standard Perl distribution, and by the 1101operating system and its utilities. Note especially that the string 1102value of C<$!> and the error messages given by external utilities may 1103be changed by C<LC_MESSAGES>. If you want to have portable error 1104codes, use C<%!>. See L<Errno>. 1105 1106=head1 SECURITY 1107 1108Although the main discussion of Perl security issues can be found in 1109L<perlsec>, a discussion of Perl's locale handling would be incomplete 1110if it did not draw your attention to locale-dependent security issues. 1111Locales--particularly on systems that allow unprivileged users to 1112build their own locales--are untrustworthy. A malicious (or just plain 1113broken) locale can make a locale-aware application give unexpected 1114results. Here are a few possibilities: 1115 1116=over 4 1117 1118=item * 1119 1120Regular expression checks for safe file names or mail addresses using 1121C<\w> may be spoofed by an C<LC_CTYPE> locale that claims that 1122characters such as C<"E<gt>"> and C<"|"> are alphanumeric. 1123 1124=item * 1125 1126String interpolation with case-mapping, as in, say, C<$dest = 1127"C:\U$name.$ext">, may produce dangerous results if a bogus C<LC_CTYPE> 1128case-mapping table is in effect. 1129 1130=item * 1131 1132A sneaky C<LC_COLLATE> locale could result in the names of students with 1133"D" grades appearing ahead of those with "A"s. 1134 1135=item * 1136 1137An application that takes the trouble to use information in 1138C<LC_MONETARY> may format debits as if they were credits and vice versa 1139if that locale has been subverted. Or it might make payments in US 1140dollars instead of Hong Kong dollars. 1141 1142=item * 1143 1144The date and day names in dates formatted by C<strftime()> could be 1145manipulated to advantage by a malicious user able to subvert the 1146C<LC_DATE> locale. ("Look--it says I wasn't in the building on 1147Sunday.") 1148 1149=back 1150 1151Such dangers are not peculiar to the locale system: any aspect of an 1152application's environment which may be modified maliciously presents 1153similar challenges. Similarly, they are not specific to Perl: any 1154programming language that allows you to write programs that take 1155account of their environment exposes you to these issues. 1156 1157Perl cannot protect you from all possibilities shown in the 1158examples--there is no substitute for your own vigilance--but, when 1159C<use locale> is in effect, Perl uses the tainting mechanism (see 1160L<perlsec>) to mark string results that become locale-dependent, and 1161which may be untrustworthy in consequence. 1162 1163Note that it is possible to compile Perl without taint support, 1164in which case all taint features silently do nothing. 1165 1166Here is a summary of the tainting behavior of operators and functions 1167that may be affected by the locale: 1168 1169=over 4 1170 1171=item * 1172 1173B<Comparison operators> (C<lt>, C<le>, C<ge>, C<gt> and C<cmp>): 1174 1175Scalar true/false (or less/equal/greater) result is never tainted. 1176 1177=item * 1178 1179B<Case-mapping interpolation> (with C<\l>, C<\L>, C<\u>, C<\U>, or C<\F>) 1180 1181The result string containing interpolated material is tainted if 1182a C<use locale> form that includes C<LC_CTYPE> is in effect. 1183 1184=item * 1185 1186B<Matching operator> (C<m//>): 1187 1188Scalar true/false result never tainted. 1189 1190All subpatterns, either delivered as a list-context result or as C<$1> 1191I<etc>., are tainted if a C<use locale> form that includes 1192C<LC_CTYPE> is in effect, and the subpattern 1193regular expression contains a locale-dependent construct. These 1194constructs include C<\w> (to match an alphanumeric character), C<\W> 1195(non-alphanumeric character), C<\b> and C<\B> (word-boundary and 1196non-boundardy, which depend on what C<\w> and C<\W> match), C<\s> 1197(whitespace character), C<\S> (non whitespace character), C<\d> and 1198C<\D> (digits and non-digits), and the POSIX character classes, such as 1199C<[:alpha:]> (see L<perlrecharclass/POSIX Character Classes>). 1200 1201Tainting is also likely if the pattern is to be matched 1202case-insensitively (via C</i>). The exception is if all the code points 1203to be matched this way are above 255 and do not have folds under Unicode 1204rules to below 256. Tainting is not done for these because Perl 1205only uses Unicode rules for such code points, and those rules are the 1206same no matter what the current locale. 1207 1208The matched-pattern variables, C<$&>, C<$`> (pre-match), C<$'> 1209(post-match), and C<$+> (last match) also are tainted. 1210 1211=item * 1212 1213B<Substitution operator> (C<s///>): 1214 1215Has the same behavior as the match operator. Also, the left 1216operand of C<=~> becomes tainted when a C<use locale> 1217form that includes C<LC_CTYPE> is in effect, if modified as 1218a result of a substitution based on a regular 1219expression match involving any of the things mentioned in the previous 1220item, or of case-mapping, such as C<\l>, C<\L>,C<\u>, C<\U>, or C<\F>. 1221 1222=item * 1223 1224B<Output formatting functions> (C<printf()> and C<write()>): 1225 1226Results are never tainted because otherwise even output from print, 1227for example C<print(1/7)>, should be tainted if C<use locale> is in 1228effect. 1229 1230=item * 1231 1232B<Case-mapping functions> (C<lc()>, C<lcfirst()>, C<uc()>, C<ucfirst()>): 1233 1234Results are tainted if a C<use locale> form that includes C<LC_CTYPE> is 1235in effect. 1236 1237=item * 1238 1239B<POSIX locale-dependent functions> (C<localeconv()>, C<strcoll()>, 1240C<strftime()>, C<strxfrm()>): 1241 1242Results are never tainted. 1243 1244=back 1245 1246Three examples illustrate locale-dependent tainting. 1247The first program, which ignores its locale, won't run: a value taken 1248directly from the command line may not be used to name an output file 1249when taint checks are enabled. 1250 1251 #/usr/local/bin/perl -T 1252 # Run with taint checking 1253 1254 # Command line sanity check omitted... 1255 $tainted_output_file = shift; 1256 1257 open(F, ">$tainted_output_file") 1258 or warn "Open of $tainted_output_file failed: $!\n"; 1259 1260The program can be made to run by "laundering" the tainted value through 1261a regular expression: the second example--which still ignores locale 1262information--runs, creating the file named on its command line 1263if it can. 1264 1265 #/usr/local/bin/perl -T 1266 1267 $tainted_output_file = shift; 1268 $tainted_output_file =~ m%[\w/]+%; 1269 $untainted_output_file = $&; 1270 1271 open(F, ">$untainted_output_file") 1272 or warn "Open of $untainted_output_file failed: $!\n"; 1273 1274Compare this with a similar but locale-aware program: 1275 1276 #/usr/local/bin/perl -T 1277 1278 $tainted_output_file = shift; 1279 use locale; 1280 $tainted_output_file =~ m%[\w/]+%; 1281 $localized_output_file = $&; 1282 1283 open(F, ">$localized_output_file") 1284 or warn "Open of $localized_output_file failed: $!\n"; 1285 1286This third program fails to run because C<$&> is tainted: it is the result 1287of a match involving C<\w> while C<use locale> is in effect. 1288 1289=head1 ENVIRONMENT 1290 1291=over 12 1292 1293=item PERL_SKIP_LOCALE_INIT 1294 1295This environment variable, available starting in Perl v5.20, if set 1296(to any value), tells Perl to not use the rest of the 1297environment variables to initialize with. Instead, Perl uses whatever 1298the current locale settings are. This is particularly useful in 1299embedded environments, see 1300L<perlembed/Using embedded Perl with POSIX locales>. 1301 1302=item PERL_BADLANG 1303 1304A string that can suppress Perl's warning about failed locale settings 1305at startup. Failure can occur if the locale support in the operating 1306system is lacking (broken) in some way--or if you mistyped the name of 1307a locale when you set up your environment. If this environment 1308variable is absent, or has a value other than "0" or "", Perl will 1309complain about locale setting failures. 1310 1311B<NOTE>: C<PERL_BADLANG> only gives you a way to hide the warning message. 1312The message tells about some problem in your system's locale support, 1313and you should investigate what the problem is. 1314 1315=back 1316 1317The following environment variables are not specific to Perl: They are 1318part of the standardized (ISO C, XPG4, POSIX 1.c) C<setlocale()> method 1319for controlling an application's opinion on data. Windows is non-POSIX, 1320but Perl arranges for the following to work as described anyway. 1321If the locale given by an environment variable is not valid, Perl tries 1322the next lower one in priority. If none are valid, on Windows, the 1323system default locale is then tried. If all else fails, the C<"C"> 1324locale is used. If even that doesn't work, something is badly broken, 1325but Perl tries to forge ahead with whatever the locale settings might 1326be. 1327 1328=over 12 1329 1330=item C<LC_ALL> 1331 1332C<LC_ALL> is the "override-all" locale environment variable. If 1333set, it overrides all the rest of the locale environment variables. 1334 1335=item C<LANGUAGE> 1336 1337B<NOTE>: C<LANGUAGE> is a GNU extension, it affects you only if you 1338are using the GNU libc. This is the case if you are using e.g. Linux. 1339If you are using "commercial" Unixes you are most probably I<not> 1340using GNU libc and you can ignore C<LANGUAGE>. 1341 1342However, in the case you are using C<LANGUAGE>: it affects the 1343language of informational, warning, and error messages output by 1344commands (in other words, it's like C<LC_MESSAGES>) but it has higher 1345priority than C<LC_ALL>. Moreover, it's not a single value but 1346instead a "path" (":"-separated list) of I<languages> (not locales). 1347See the GNU C<gettext> library documentation for more information. 1348 1349=item C<LC_CTYPE> 1350 1351In the absence of C<LC_ALL>, C<LC_CTYPE> chooses the character type 1352locale. In the absence of both C<LC_ALL> and C<LC_CTYPE>, C<LANG> 1353chooses the character type locale. 1354 1355=item C<LC_COLLATE> 1356 1357In the absence of C<LC_ALL>, C<LC_COLLATE> chooses the collation 1358(sorting) locale. In the absence of both C<LC_ALL> and C<LC_COLLATE>, 1359C<LANG> chooses the collation locale. 1360 1361=item C<LC_MONETARY> 1362 1363In the absence of C<LC_ALL>, C<LC_MONETARY> chooses the monetary 1364formatting locale. In the absence of both C<LC_ALL> and C<LC_MONETARY>, 1365C<LANG> chooses the monetary formatting locale. 1366 1367=item C<LC_NUMERIC> 1368 1369In the absence of C<LC_ALL>, C<LC_NUMERIC> chooses the numeric format 1370locale. In the absence of both C<LC_ALL> and C<LC_NUMERIC>, C<LANG> 1371chooses the numeric format. 1372 1373=item C<LC_TIME> 1374 1375In the absence of C<LC_ALL>, C<LC_TIME> chooses the date and time 1376formatting locale. In the absence of both C<LC_ALL> and C<LC_TIME>, 1377C<LANG> chooses the date and time formatting locale. 1378 1379=item C<LANG> 1380 1381C<LANG> is the "catch-all" locale environment variable. If it is set, it 1382is used as the last resort after the overall C<LC_ALL> and the 1383category-specific C<LC_I<foo>>. 1384 1385=back 1386 1387=head2 Examples 1388 1389The C<LC_NUMERIC> controls the numeric output: 1390 1391 use locale; 1392 use POSIX qw(locale_h); # Imports setlocale() and the LC_ constants. 1393 setlocale(LC_NUMERIC, "fr_FR") or die "Pardon"; 1394 printf "%g\n", 1.23; # If the "fr_FR" succeeded, probably shows 1,23. 1395 1396and also how strings are parsed by C<POSIX::strtod()> as numbers: 1397 1398 use locale; 1399 use POSIX qw(locale_h strtod); 1400 setlocale(LC_NUMERIC, "de_DE") or die "Entschuldigung"; 1401 my $x = strtod("2,34") + 5; 1402 print $x, "\n"; # Probably shows 7,34. 1403 1404=head1 NOTES 1405 1406=head2 String C<eval> and C<LC_NUMERIC> 1407 1408A string L<eval|perlfunc/eval EXPR> parses its expression as standard 1409Perl. It is therefore expecting the decimal point to be a dot. If 1410C<LC_NUMERIC> is set to have this be a comma instead, the parsing will 1411be confused, perhaps silently. 1412 1413 use locale; 1414 use POSIX qw(locale_h); 1415 setlocale(LC_NUMERIC, "fr_FR") or die "Pardon"; 1416 my $a = 1.2; 1417 print eval "$a + 1.5"; 1418 print "\n"; 1419 1420prints C<13,5>. This is because in that locale, the comma is the 1421decimal point character. The C<eval> thus expands to: 1422 1423 eval "1,2 + 1.5" 1424 1425and the result is not what you likely expected. No warnings are 1426generated. If you do string C<eval>'s within the scope of 1427S<C<use locale>>, you should instead change the C<eval> line to do 1428something like: 1429 1430 print eval "no locale; $a + 1.5"; 1431 1432This prints C<2.7>. 1433 1434You could also exclude C<LC_NUMERIC>, if you don't need it, by 1435 1436 use locale ':!numeric'; 1437 1438=head2 Backward compatibility 1439 1440Versions of Perl prior to 5.004 B<mostly> ignored locale information, 1441generally behaving as if something similar to the C<"C"> locale were 1442always in force, even if the program environment suggested otherwise 1443(see L</The setlocale function>). By default, Perl still behaves this 1444way for backward compatibility. If you want a Perl application to pay 1445attention to locale information, you B<must> use the S<C<use locale>> 1446pragma (see L</The "use locale" pragma>) or, in the unlikely event 1447that you want to do so for just pattern matching, the 1448C</l> regular expression modifier (see L<perlre/Character set 1449modifiers>) to instruct it to do so. 1450 1451Versions of Perl from 5.002 to 5.003 did use the C<LC_CTYPE> 1452information if available; that is, C<\w> did understand what 1453were the letters according to the locale environment variables. 1454The problem was that the user had no control over the feature: 1455if the C library supported locales, Perl used them. 1456 1457=head2 I18N:Collate obsolete 1458 1459In versions of Perl prior to 5.004, per-locale collation was possible 1460using the C<I18N::Collate> library module. This module is now mildly 1461obsolete and should be avoided in new applications. The C<LC_COLLATE> 1462functionality is now integrated into the Perl core language: One can 1463use locale-specific scalar data completely normally with C<use locale>, 1464so there is no longer any need to juggle with the scalar references of 1465C<I18N::Collate>. 1466 1467=head2 Sort speed and memory use impacts 1468 1469Comparing and sorting by locale is usually slower than the default 1470sorting; slow-downs of two to four times have been observed. It will 1471also consume more memory: once a Perl scalar variable has participated 1472in any string comparison or sorting operation obeying the locale 1473collation rules, it will take 3-15 times more memory than before. (The 1474exact multiplier depends on the string's contents, the operating system 1475and the locale.) These downsides are dictated more by the operating 1476system's implementation of the locale system than by Perl. 1477 1478=head2 Freely available locale definitions 1479 1480The Unicode CLDR project extracts the POSIX portion of many of its 1481locales, available at 1482 1483 https://unicode.org/Public/cldr/2.0.1/ 1484 1485(Newer versions of CLDR require you to compute the POSIX data yourself. 1486See L<http://unicode.org/Public/cldr/latest/>.) 1487 1488There is a large collection of locale definitions at: 1489 1490 http://std.dkuug.dk/i18n/WG15-collection/locales/ 1491 1492You should be aware that it is 1493unsupported, and is not claimed to be fit for any purpose. If your 1494system allows installation of arbitrary locales, you may find the 1495definitions useful as they are, or as a basis for the development of 1496your own locales. 1497 1498=head2 I18n and l10n 1499 1500"Internationalization" is often abbreviated as B<i18n> because its first 1501and last letters are separated by eighteen others. (You may guess why 1502the internalin ... internaliti ... i18n tends to get abbreviated.) In 1503the same way, "localization" is often abbreviated to B<l10n>. 1504 1505=head2 An imperfect standard 1506 1507Internationalization, as defined in the C and POSIX standards, can be 1508criticized as incomplete and ungainly. They also have a tendency, like 1509standards groups, to divide the world into nations, when we all know 1510that the world can equally well be divided into bankers, bikers, gamers, 1511and so on. 1512 1513=head1 Unicode and UTF-8 1514 1515The support of Unicode is new starting from Perl version v5.6, and more fully 1516implemented in versions v5.8 and later. See L<perluniintro>. 1517 1518Starting in Perl v5.20, UTF-8 locales are supported in Perl, except 1519C<LC_COLLATE> is only partially supported; collation support is improved 1520in Perl v5.26 to a level that may be sufficient for your needs 1521(see L</Category C<LC_COLLATE>: Collation: Text Comparisons and Sorting>). 1522 1523If you have Perl v5.16 or v5.18 and can't upgrade, you can use 1524 1525 use locale ':not_characters'; 1526 1527When this form of the pragma is used, only the non-character portions of 1528locales are used by Perl, for example C<LC_NUMERIC>. Perl assumes that 1529you have translated all the characters it is to operate on into Unicode 1530(actually the platform's native character set (ASCII or EBCDIC) plus 1531Unicode). For data in files, this can conveniently be done by also 1532specifying 1533 1534 use open ':locale'; 1535 1536This pragma arranges for all inputs from files to be translated into 1537Unicode from the current locale as specified in the environment (see 1538L</ENVIRONMENT>), and all outputs to files to be translated back 1539into the locale. (See L<open>). On a per-filehandle basis, you can 1540instead use the L<PerlIO::locale> module, or the L<Encode::Locale> 1541module, both available from CPAN. The latter module also has methods to 1542ease the handling of C<ARGV> and environment variables, and can be used 1543on individual strings. If you know that all your locales will be 1544UTF-8, as many are these days, you can use the 1545L<B<-C>|perlrun/-C [numberE<sol>list]> command line switch. 1546 1547This form of the pragma allows essentially seamless handling of locales 1548with Unicode. The collation order will be by Unicode code point order. 1549L<Unicode::Collate> can be used to get Unicode rules collation. 1550 1551All the modules and switches just described can be used in v5.20 with 1552just plain C<use locale>, and, should the input locales not be UTF-8, 1553you'll get the less than ideal behavior, described below, that you get 1554with pre-v5.16 Perls, or when you use the locale pragma without the 1555C<:not_characters> parameter in v5.16 and v5.18. If you are using 1556exclusively UTF-8 locales in v5.20 and higher, the rest of this section 1557does not apply to you. 1558 1559There are two cases, multi-byte and single-byte locales. First 1560multi-byte: 1561 1562The only multi-byte (or wide character) locale that Perl is ever likely 1563to support is UTF-8. This is due to the difficulty of implementation, 1564the fact that high quality UTF-8 locales are now published for every 1565area of the world (L<https://unicode.org/Public/cldr/2.0.1/> for 1566ones that are already set-up, but from an earlier version; 1567L<https://unicode.org/Public/cldr/latest/> for the most up-to-date, but 1568you have to extract the POSIX information yourself), and 1569failing all that, you can use the L<Encode> module to translate to/from 1570your locale. So, you'll have to do one of those things if you're using 1571one of these locales, such as Big5 or Shift JIS. For UTF-8 locales, in 1572Perls (pre v5.20) that don't have full UTF-8 locale support, they may 1573work reasonably well (depending on your C library implementation) 1574simply because both 1575they and Perl store characters that take up multiple bytes the same way. 1576However, some, if not most, C library implementations may not process 1577the characters in the upper half of the Latin-1 range (128 - 255) 1578properly under C<LC_CTYPE>. To see if a character is a particular type 1579under a locale, Perl uses the functions like C<isalnum()>. Your C 1580library may not work for UTF-8 locales with those functions, instead 1581only working under the newer wide library functions like C<iswalnum()>, 1582which Perl does not use. 1583These multi-byte locales are treated like single-byte locales, and will 1584have the restrictions described below. Starting in Perl v5.22 a warning 1585message is raised when Perl detects a multi-byte locale that it doesn't 1586fully support. 1587 1588For single-byte locales, 1589Perl generally takes the tack to use locale rules on code points that can fit 1590in a single byte, and Unicode rules for those that can't (though this 1591isn't uniformly applied, see the note at the end of this section). This 1592prevents many problems in locales that aren't UTF-8. Suppose the locale 1593is ISO8859-7, Greek. The character at 0xD7 there is a capital Chi. But 1594in the ISO8859-1 locale, Latin1, it is a multiplication sign. The POSIX 1595regular expression character class C<[[:alpha:]]> will magically match 15960xD7 in the Greek locale but not in the Latin one. 1597 1598However, there are places where this breaks down. Certain Perl constructs are 1599for Unicode only, such as C<\p{Alpha}>. They assume that 0xD7 always has its 1600Unicode meaning (or the equivalent on EBCDIC platforms). Since Latin1 is a 1601subset of Unicode and 0xD7 is the multiplication sign in both Latin1 and 1602Unicode, C<\p{Alpha}> will never match it, regardless of locale. A similar 1603issue occurs with C<\N{...}>. Prior to v5.20, it is therefore a bad 1604idea to use C<\p{}> or 1605C<\N{}> under plain C<use locale>--I<unless> you can guarantee that the 1606locale will be ISO8859-1. Use POSIX character classes instead. 1607 1608Another problem with this approach is that operations that cross the 1609single byte/multiple byte boundary are not well-defined, and so are 1610disallowed. (This boundary is between the codepoints at 255/256.) 1611For example, lower casing LATIN CAPITAL LETTER Y WITH DIAERESIS (U+0178) 1612should return LATIN SMALL LETTER Y WITH DIAERESIS (U+00FF). But in the 1613Greek locale, for example, there is no character at 0xFF, and Perl 1614has no way of knowing what the character at 0xFF is really supposed to 1615represent. Thus it disallows the operation. In this mode, the 1616lowercase of U+0178 is itself. 1617 1618The same problems ensue if you enable automatic UTF-8-ification of your 1619standard file handles, default C<open()> layer, and C<@ARGV> on non-ISO8859-1, 1620non-UTF-8 locales (by using either the B<-C> command line switch or the 1621C<PERL_UNICODE> environment variable; see 1622L<perlrun|perlrun/-C [numberE<sol>list]>). 1623Things are read in as UTF-8, which would normally imply a Unicode 1624interpretation, but the presence of a locale causes them to be interpreted 1625in that locale instead. For example, a 0xD7 code point in the Unicode 1626input, which should mean the multiplication sign, won't be interpreted by 1627Perl that way under the Greek locale. This is not a problem 1628I<provided> you make certain that all locales will always and only be either 1629an ISO8859-1, or, if you don't have a deficient C library, a UTF-8 locale. 1630 1631Still another problem is that this approach can lead to two code 1632points meaning the same character. Thus in a Greek locale, both U+03A7 1633and U+00D7 are GREEK CAPITAL LETTER CHI. 1634 1635Because of all these problems, starting in v5.22, Perl will raise a 1636warning if a multi-byte (hence Unicode) code point is used when a 1637single-byte locale is in effect. (Although it doesn't check for this if 1638doing so would unreasonably slow execution down.) 1639 1640Vendor locales are notoriously buggy, and it is difficult for Perl to test 1641its locale-handling code because this interacts with code that Perl has no 1642control over; therefore the locale-handling code in Perl may be buggy as 1643well. (However, the Unicode-supplied locales should be better, and 1644there is a feed back mechanism to correct any problems. See 1645L</Freely available locale definitions>.) 1646 1647If you have Perl v5.16, the problems mentioned above go away if you use 1648the C<:not_characters> parameter to the locale pragma (except for vendor 1649bugs in the non-character portions). If you don't have v5.16, and you 1650I<do> have locales that work, using them may be worthwhile for certain 1651specific purposes, as long as you keep in mind the gotchas already 1652mentioned. For example, if the collation for your locales works, it 1653runs faster under locales than under L<Unicode::Collate>; and you gain 1654access to such things as the local currency symbol and the names of the 1655months and days of the week. (But to hammer home the point, in v5.16, 1656you get this access without the downsides of locales by using the 1657C<:not_characters> form of the pragma.) 1658 1659Note: The policy of using locale rules for code points that can fit in a 1660byte, and Unicode rules for those that can't is not uniformly applied. 1661Pre-v5.12, it was somewhat haphazard; in v5.12 it was applied fairly 1662consistently to regular expression matching except for bracketed 1663character classes; in v5.14 it was extended to all regex matches; and in 1664v5.16 to the casing operations such as C<\L> and C<uc()>. For 1665collation, in all releases so far, the system's C<strxfrm()> function is 1666called, and whatever it does is what you get. Starting in v5.26, various 1667bugs are fixed with the way perl uses this function. 1668 1669=head1 BUGS 1670 1671=head2 Collation of strings containing embedded C<NUL> characters 1672 1673C<NUL> characters will sort the same as the lowest collating control 1674character does, or to C<"\001"> in the unlikely event that there are no 1675control characters at all in the locale. In cases where the strings 1676don't contain this non-C<NUL> control, the results will be correct, and 1677in many locales, this control, whatever it might be, will rarely be 1678encountered. But there are cases where a C<NUL> should sort before this 1679control, but doesn't. If two strings do collate identically, the one 1680containing the C<NUL> will sort to earlier. Prior to 5.26, there were 1681more bugs. 1682 1683=head2 Multi-threaded 1684 1685XS code or C-language libraries called from it that use the system 1686L<C<setlocale(3)>> function (except on Windows) likely will not work 1687from a multi-threaded application without changes. See 1688L<perlxs/Locale-aware XS code>. 1689 1690An XS module that is locale-dependent could have been written under the 1691assumption that it will never be called in a multi-threaded environment, 1692and so uses other non-locale constructs that aren't multi-thread-safe. 1693See L<perlxs/Thread-aware system interfaces>. 1694 1695POSIX does not define a way to get the name of the current per-thread 1696locale. Some systems, such as Darwin and NetBSD do implement a 1697function, L<querylocale(3)> to do this. On non-Windows systems without 1698it, such as Linux, there are some additional caveats: 1699 1700=over 1701 1702=item * 1703 1704An embedded perl needs to be started up while the global locale is in 1705effect. See L<perlembed/Using embedded Perl with POSIX locales>. 1706 1707=item * 1708 1709It becomes more important for perl to know about all the possible 1710locale categories on the platform, even if they aren't apparently used 1711in your program. Perl knows all of the Linux ones. If your platform 1712has others, you can submit an issue at 1713L<https://github.com/Perl/perl5/issues> for 1714inclusion of it in the next release. In the meantime, it is possible to 1715edit the Perl source to teach it about the category, and then recompile. 1716Search for instances of, say, C<LC_PAPER> in the source, and use that as 1717a template to add the omitted one. 1718 1719=item * 1720 1721It is possible, though hard to do, to call C<POSIX::setlocale> with a 1722locale that it doesn't recognize as syntactically legal, but actually is 1723legal on that system. This should happen only with embedded perls, or 1724if you hand-craft a locale name yourself. 1725 1726=back 1727 1728=head2 Broken systems 1729 1730In certain systems, the operating system's locale support 1731is broken and cannot be fixed or used by Perl. Such deficiencies can 1732and will result in mysterious hangs and/or Perl core dumps when 1733C<use locale> is in effect. When confronted with such a system, 1734please report in excruciating detail to 1735<L<https://github.com/Perl/perl5/issues>>, and 1736also contact your vendor: bug fixes may exist for these problems 1737in your operating system. Sometimes such bug fixes are called an 1738operating system upgrade. If you have the source for Perl, include in 1739the bug report the output of the test described above in L</Testing 1740for broken locales>. 1741 1742=head1 SEE ALSO 1743 1744L<I18N::Langinfo>, L<perluniintro>, L<perlunicode>, L<open>, 1745L<POSIX/localeconv>, 1746L<POSIX/setlocale>, L<POSIX/strcoll>, L<POSIX/strftime>, 1747L<POSIX/strtod>, L<POSIX/strxfrm>. 1748 1749For special considerations when Perl is embedded in a C program, 1750see L<perlembed/Using embedded Perl with POSIX locales>. 1751 1752=head1 HISTORY 1753 1754Jarkko Hietaniemi's original F<perli18n.pod> heavily hacked by Dominic 1755Dunlop, assisted by the perl5-porters. Prose worked over a bit by 1756Tom Christiansen, and now maintained by Perl 5 porters. 1757