1# 2# $FreeBSD$ 3# 4#======================================================================= 5# File name: DEVANAGA.TXT 6# 7# Contents: Map (external version) from Mac OS Devanagari 8# encoding to Unicode 2.1 and later. 9# 10# Copyright: (c) 1995-2002, 2005 by Apple Computer, Inc., all rights 11# reserved. 12# 13# Contact: charsets@apple.com 14# 15# Changes: 16# 17# c02 2005-Apr-05 Update header comments; add section on 18# roundtrip considerations. Matches internal 19# xml <c1.1> and Text Encoding Converter 2.0. 20# b3,c1 2002-Dec-19 Update URLs. Matches internal utom<b1>. 21# b02 1999-Sep-22 Update contact e-mail address. Matches 22# internal utom<b1>, ufrm<b1>, and Text 23# Encoding Converter version 1.5. 24# n04 1998-Feb-05 First version; matches internal utom<n9>, 25# ufrm<n15>. 26# 27# Standard header: 28# ---------------- 29# 30# Apple, the Apple logo, and Macintosh are trademarks of Apple 31# Computer, Inc., registered in the United States and other countries. 32# Unicode is a trademark of Unicode Inc. For the sake of brevity, 33# throughout this document, "Macintosh" can be used to refer to 34# Macintosh computers and "Unicode" can be used to refer to the 35# Unicode standard. 36# 37# Apple Computer, Inc. ("Apple") makes no warranty or representation, 38# either express or implied, with respect to this document and the 39# included data, its quality, accuracy, or fitness for a particular 40# purpose. In no event will Apple be liable for direct, indirect, 41# special, incidental, or consequential damages resulting from any 42# defect or inaccuracy in this document or the included data. 43# 44# These mapping tables and character lists are subject to change. 45# The latest tables should be available from the following: 46# 47# <http://www.unicode.org/Public/MAPPINGS/VENDORS/APPLE/> 48# 49# For general information about Mac OS encodings and these mapping 50# tables, see the file "README.TXT". 51# 52# Format: 53# ------- 54# 55# Three tab-separated columns; 56# '#' begins a comment which continues to the end of the line. 57# Column #1 is the Mac OS Devanagari code or code sequence 58# (in hex as 0xNN or 0xNN+0xNN) 59# Column #2 is the corresponding Unicode or Unicode sequence 60# (in hex as 0xNNNN or 0xNNNN+0xNNNN). 61# Column #3 is a comment containing the Unicode name or sequence 62# of names. In some cases an additional comment follows the 63# Unicode name(s). 64# 65# The entries are in two sections. The first section is for pairs of 66# Mac OS Devanagari code points that must be mapped in a special way. 67# The second section maps individual code points. 68# 69# Within each section, the entries are in Mac OS Devanagari code order. 70# 71# Control character mappings are not shown in this table, following 72# the conventions of the standard UTC mapping tables. However, the 73# Mac OS Devanagari character set uses the standard control characters 74# at 0x00-0x1F and 0x7F. 75# 76# Notes on Mac OS Devanagari: 77# --------------------------- 78# 79# This is a legacy Mac OS encoding; in the Mac OS X Carbon and Cocoa 80# environments, it is only supported via transcoding to and from 81# Unicode. 82# 83# Mac OS Devanagari is based on IS 13194:1991 (ISCII-91), with the 84# addition of several punctuation and symbol characters. However, 85# Mac OS Devanagari does not support the ATR (attribute) mechanism of 86# ISCII-91. 87# 88# 1. ISCII-91 features in Mac OS Devanagari include: 89# 90# a) Overloading of nukta 91# 92# In addition to using the nukta (0xE9) like a combining dot below, 93# nukta is overloaded to function as a general character modifier. 94# In this role, certain code points followed by 0xE9 are treated as 95# a two-byte code point representing a character which may be 96# rather different than the characters represented by either of 97# the code points alone. For example, the character DEVANAGARI OM 98# (U+0950) is represented in ISCII-91 as candrabindu + nukta. 99# 100# b) Explicit halant and soft halant 101# 102# A double halant (0xE8 + 0xE8) constitutes an "explicit halant", 103# which will always appear as a halant instead of causing formation 104# of a ligature or half-form consonant. 105# 106# Halant followed by nukta (0xE8 + 0xE9) constitutes a "soft 107# halant", which prevents formation of a ligature and instead 108# retains the half-form of the first consonant. 109# 110# c) Invisible consonant 111# 112# The byte 0xD9 (called INV in ISCII-91) is an invisible consonant: 113# It behaves like a consonant but has no visible appearance. It is 114# intended to be used (often in combination with halant) to display 115# dependent forms in isolation, such as the RA forms or consonant 116# half-forms. 117# 118# d) Extensions for Vedic, etc. 119# 120# The byte 0xF0 (called EXT in ISCII-91) followed by any byte in 121# the range 0xA1-0xEE constitutes a two-byte code point which can 122# be used to represent additional characters for Vedic (or other 123# extensions); 0xF0 followed by any other byte value constitutes 124# malformed text. Mac OS Devanagari supports this mechanism, but 125# does not currently map any of these two-byte code points to 126# anything. 127# 128# 2. Mac OS Devanagari additions 129# 130# Mac OS Devanagari adds characters using the code points 131# 0x80-0x8A and 0x90-0x91 (the latter are some Devanagari additions 132# from Unicode). 133# 134# 3. Unused code points 135# 136# The following code points are currently unused, and are not shown 137# here: 0x8B-0x8F, 0x92-0xA0, 0xEB-0xEF, 0xFB-0xFF. In addition, 138# 0xF0 is not shown here, but it has a special function as described 139# above. 140# 141# Unicode mapping issues and notes: 142# --------------------------------- 143# 144# 1. Mapping the byte pairs 145# 146# If one of the following byte values is encountered when mapping 147# Mac OS Devanagari text - 0xA1, 0xA6, 0xA7, 0xAA, 0xDB, 0xDC, 0xDF, 148# 0xE8, or 0xEA - then the next byte (if there is one) should be 149# examined. If the next byte is 0xE9 - or also 0xE8, if the first 150# byte was 0xE8 - then the byte pair should be mapped using the 151# first section of the mapping table below. Otherwise, each byte 152# should be mapped using the second section of the mapping table 153# below. 154# 155# - The Unicode Standard, Version 2.0, specifies how explicit 156# halant and soft halant should be represented in Unicode; 157# these mappings are used below. 158# 159# If the byte value 0xF0 is encountered when mapping Mac OS 160# Devanagari text, then the next byte should be examined. If there 161# is no next byte (e.g. 0xF0 at end of buffer), the mapping 162# process should indicate incomplete character. If there is a next 163# byte but it is not in the range 0xA1-0xEE, the mapping process 164# should indicate malformed text. Otherwise, the mapping process 165# should treat the byte pair as a valid two-byte code point with no 166# mapping (e.g. map it to QUESTION MARK, REPLACEMENT CHARACTER, 167# etc.). 168# 169# 2. Mapping the invisible consonant 170# 171# It has been suggested that INV in ISCII-91 should map to ZERO 172# WIDTH NON-JOINER in Unicode. However, this causes problems with 173# roundtrip fidelity: The ISCII-91 sequences 0xE8+0xE8 and 0xE8+0xD9 174# would map to the same sequence of Unicode characters. We have 175# instead mapped INV to LEFT-TO-RIGHT MARK, which avoids these 176# problems. 177# 178# 3. Additional loose mappings from Unicode 179# 180# These are not preserved in roundtrip mappings. 181# 182# U+0958 0xB3+0xE9 # DEVANAGARI LETTER QA 183# U+0959 0xB4+0xE9 # DEVANAGARI LETTER KHHA 184# U+095A 0xB5+0xE9 # DEVANAGARI LETTER GHHA 185# U+095B 0xBA+0xE9 # DEVANAGARI LETTER ZA 186# U+095C 0xBF+0xE9 # DEVANAGARI LETTER DDDHA 187# U+095D 0xC0+0xE9 # DEVANAGARI LETTER RHA 188# U+095E 0xC9+0xE9 # DEVANAGARI LETTER FA 189# 190# 4. Roundtrip considerations when mapping to decomposed Unicode 191# 192# Both ISCII-91 (hence Mac OS Devanagari) and Unicode provide multiple 193# ways of representing certain Devanagari consonants. For example, 194# DEVANAGARI LETTER NNNA can be represented in Unicode as the single 195# character 0x0929 or as the sequence 0x0928 0x093C; similarly, this 196# consonant can be represented in Mac OS Devanagari as 0xC7 or as the 197# sequence 0xC6 0xE9. This leads to some roundtrip problems. First 198# note that we have the following mappings without such problems: 199# 200# ISCII/ standard decomposition of reverse mapping 201# Mac OS Unicode mapping standard mapping of decomposition 202# ------ ----------------------- ---------------- ---------------- 203# 0xC6 0x0928 ... LETTER NA 0x0928 (same) 0xC6 204# 0xCD 0x092F ... LETTER YA 0x092F (same) 0xCD 205# 0xCF 0x0930 ... LETTER RA 0x0930 (same) 0xCF 206# 0xD2 0x0933 ... LETTER LLA 0x0933 (same) 0xD2 207# 0xE9 0x093C ... SIGN NUKTA 0x093C (same) 0xE9 208# 209# However, those mappings above cause roundtrip problems for the 210# the following mappings if they are decomposed: 211# 212# ISCII/ standard decomposition of reverse mapping 213# Mac OS Unicode mapping standard mapping of decomposition 214# ------ ----------------------- ---------------- ---------------- 215# 0xC7 0x0929 ... LETTER NNNA 0x0928 0x093C 0xC6 0xE9 216# 0xCE 0x095F ... LETTER YYA 0x092F 0x093C 0xCD 0xE9 217# 0xD0 0x0931 ... LETTER RRA 0x0930 0x093C 0xCF 0xE9 218# 0xD3 0x0934 ... LETTER LLLA 0x0933 0x093C 0xD2 0xE9 219# 220# One solution is to use a grouping transcoding hint with the four 221# decompositions above to mark the decomposed sequence for special 222# treatment in transcoding. This yields the following mappings to 223# decomposed Unicode: 224# 225# ISCII/ decomposed 226# Mac OS Unicode mapping 227# ------ ---------------- 228# 0xC7 0xF860 0x0928 0x093C 229# 0xCE 0xF860 0x092F 0x093C 230# 0xD0 0xF860 0x0930 0x093C 231# 0xD3 0xF860 0x0933 0x093C 232# 233# Details of mapping changes in each version: 234# ------------------------------------------- 235# 236################## 237 238# Section 1: Map the following byte pairs as indicated: 239# (ZWNJ means ZERO WIDTH NON-JOINER, ZWJ means ZERO WIDTH JOINER) 240# (Also see note about 0xF0 in comments above) 241 2420xA1+0xE9 0x0950 # DEVANAGARI OM 2430xA6+0xE9 0x090C # DEVANAGARI LETTER VOCALIC L 2440xA7+0xE9 0x0961 # DEVANAGARI LETTER VOCALIC LL 2450xAA+0xE9 0x0960 # DEVANAGARI LETTER VOCALIC RR 2460xDB+0xE9 0x0962 # DEVANAGARI VOWEL SIGN VOCALIC L 2470xDC+0xE9 0x0963 # DEVANAGARI VOWEL SIGN VOCALIC LL 2480xDF+0xE9 0x0944 # DEVANAGARI VOWEL SIGN VOCALIC RR 2490xE8+0xE8 0x094D+0x200C # DEVANAGARI SIGN VIRAMA + ZWNJ # explicit halant 2500xE8+0xE9 0x094D+0x200D # DEVANAGARI SIGN VIRAMA + ZWJ # soft halant 2510xEA+0xE9 0x093D # DEVANAGARI SIGN AVAGRAHA 252 253# Section 2: Map the remaining bytes as follows: 254 2550x20 0x0020 # SPACE 2560x21 0x0021 # EXCLAMATION MARK 2570x22 0x0022 # QUOTATION MARK 2580x23 0x0023 # NUMBER SIGN 2590x24 0x0024 # DOLLAR SIGN 2600x25 0x0025 # PERCENT SIGN 2610x26 0x0026 # AMPERSAND 2620x27 0x0027 # APOSTROPHE 2630x28 0x0028 # LEFT PARENTHESIS 2640x29 0x0029 # RIGHT PARENTHESIS 2650x2A 0x002A # ASTERISK 2660x2B 0x002B # PLUS SIGN 2670x2C 0x002C # COMMA 2680x2D 0x002D # HYPHEN-MINUS 2690x2E 0x002E # FULL STOP 2700x2F 0x002F # SOLIDUS 2710x30 0x0030 # DIGIT ZERO 2720x31 0x0031 # DIGIT ONE 2730x32 0x0032 # DIGIT TWO 2740x33 0x0033 # DIGIT THREE 2750x34 0x0034 # DIGIT FOUR 2760x35 0x0035 # DIGIT FIVE 2770x36 0x0036 # DIGIT SIX 2780x37 0x0037 # DIGIT SEVEN 2790x38 0x0038 # DIGIT EIGHT 2800x39 0x0039 # DIGIT NINE 2810x3A 0x003A # COLON 2820x3B 0x003B # SEMICOLON 2830x3C 0x003C # LESS-THAN SIGN 2840x3D 0x003D # EQUALS SIGN 2850x3E 0x003E # GREATER-THAN SIGN 2860x3F 0x003F # QUESTION MARK 2870x40 0x0040 # COMMERCIAL AT 2880x41 0x0041 # LATIN CAPITAL LETTER A 2890x42 0x0042 # LATIN CAPITAL LETTER B 2900x43 0x0043 # LATIN CAPITAL LETTER C 2910x44 0x0044 # LATIN CAPITAL LETTER D 2920x45 0x0045 # LATIN CAPITAL LETTER E 2930x46 0x0046 # LATIN CAPITAL LETTER F 2940x47 0x0047 # LATIN CAPITAL LETTER G 2950x48 0x0048 # LATIN CAPITAL LETTER H 2960x49 0x0049 # LATIN CAPITAL LETTER I 2970x4A 0x004A # LATIN CAPITAL LETTER J 2980x4B 0x004B # LATIN CAPITAL LETTER K 2990x4C 0x004C # LATIN CAPITAL LETTER L 3000x4D 0x004D # LATIN CAPITAL LETTER M 3010x4E 0x004E # LATIN CAPITAL LETTER N 3020x4F 0x004F # LATIN CAPITAL LETTER O 3030x50 0x0050 # LATIN CAPITAL LETTER P 3040x51 0x0051 # LATIN CAPITAL LETTER Q 3050x52 0x0052 # LATIN CAPITAL LETTER R 3060x53 0x0053 # LATIN CAPITAL LETTER S 3070x54 0x0054 # LATIN CAPITAL LETTER T 3080x55 0x0055 # LATIN CAPITAL LETTER U 3090x56 0x0056 # LATIN CAPITAL LETTER V 3100x57 0x0057 # LATIN CAPITAL LETTER W 3110x58 0x0058 # LATIN CAPITAL LETTER X 3120x59 0x0059 # LATIN CAPITAL LETTER Y 3130x5A 0x005A # LATIN CAPITAL LETTER Z 3140x5B 0x005B # LEFT SQUARE BRACKET 3150x5C 0x005C # REVERSE SOLIDUS 3160x5D 0x005D # RIGHT SQUARE BRACKET 3170x5E 0x005E # CIRCUMFLEX ACCENT 3180x5F 0x005F # LOW LINE 3190x60 0x0060 # GRAVE ACCENT 3200x61 0x0061 # LATIN SMALL LETTER A 3210x62 0x0062 # LATIN SMALL LETTER B 3220x63 0x0063 # LATIN SMALL LETTER C 3230x64 0x0064 # LATIN SMALL LETTER D 3240x65 0x0065 # LATIN SMALL LETTER E 3250x66 0x0066 # LATIN SMALL LETTER F 3260x67 0x0067 # LATIN SMALL LETTER G 3270x68 0x0068 # LATIN SMALL LETTER H 3280x69 0x0069 # LATIN SMALL LETTER I 3290x6A 0x006A # LATIN SMALL LETTER J 3300x6B 0x006B # LATIN SMALL LETTER K 3310x6C 0x006C # LATIN SMALL LETTER L 3320x6D 0x006D # LATIN SMALL LETTER M 3330x6E 0x006E # LATIN SMALL LETTER N 3340x6F 0x006F # LATIN SMALL LETTER O 3350x70 0x0070 # LATIN SMALL LETTER P 3360x71 0x0071 # LATIN SMALL LETTER Q 3370x72 0x0072 # LATIN SMALL LETTER R 3380x73 0x0073 # LATIN SMALL LETTER S 3390x74 0x0074 # LATIN SMALL LETTER T 3400x75 0x0075 # LATIN SMALL LETTER U 3410x76 0x0076 # LATIN SMALL LETTER V 3420x77 0x0077 # LATIN SMALL LETTER W 3430x78 0x0078 # LATIN SMALL LETTER X 3440x79 0x0079 # LATIN SMALL LETTER Y 3450x7A 0x007A # LATIN SMALL LETTER Z 3460x7B 0x007B # LEFT CURLY BRACKET 3470x7C 0x007C # VERTICAL LINE 3480x7D 0x007D # RIGHT CURLY BRACKET 3490x7E 0x007E # TILDE 350# 3510x80 0x00D7 # MULTIPLICATION SIGN 3520x81 0x2212 # MINUS SIGN 3530x82 0x2013 # EN DASH 3540x83 0x2014 # EM DASH 3550x84 0x2018 # LEFT SINGLE QUOTATION MARK 3560x85 0x2019 # RIGHT SINGLE QUOTATION MARK 3570x86 0x2026 # HORIZONTAL ELLIPSIS 3580x87 0x2022 # BULLET 3590x88 0x00A9 # COPYRIGHT SIGN 3600x89 0x00AE # REGISTERED SIGN 3610x8A 0x2122 # TRADE MARK SIGN 362# 3630x90 0x0965 # DEVANAGARI DOUBLE DANDA 3640x91 0x0970 # DEVANAGARI ABBREVIATION SIGN 365# 3660xA1 0x0901 # DEVANAGARI SIGN CANDRABINDU 3670xA2 0x0902 # DEVANAGARI SIGN ANUSVARA 3680xA3 0x0903 # DEVANAGARI SIGN VISARGA 3690xA4 0x0905 # DEVANAGARI LETTER A 3700xA5 0x0906 # DEVANAGARI LETTER AA 3710xA6 0x0907 # DEVANAGARI LETTER I 3720xA7 0x0908 # DEVANAGARI LETTER II 3730xA8 0x0909 # DEVANAGARI LETTER U 3740xA9 0x090A # DEVANAGARI LETTER UU 3750xAA 0x090B # DEVANAGARI LETTER VOCALIC R 3760xAB 0x090E # DEVANAGARI LETTER SHORT E 3770xAC 0x090F # DEVANAGARI LETTER E 3780xAD 0x0910 # DEVANAGARI LETTER AI 3790xAE 0x090D # DEVANAGARI LETTER CANDRA E 3800xAF 0x0912 # DEVANAGARI LETTER SHORT O 3810xB0 0x0913 # DEVANAGARI LETTER O 3820xB1 0x0914 # DEVANAGARI LETTER AU 3830xB2 0x0911 # DEVANAGARI LETTER CANDRA O 3840xB3 0x0915 # DEVANAGARI LETTER KA 3850xB4 0x0916 # DEVANAGARI LETTER KHA 3860xB5 0x0917 # DEVANAGARI LETTER GA 3870xB6 0x0918 # DEVANAGARI LETTER GHA 3880xB7 0x0919 # DEVANAGARI LETTER NGA 3890xB8 0x091A # DEVANAGARI LETTER CA 3900xB9 0x091B # DEVANAGARI LETTER CHA 3910xBA 0x091C # DEVANAGARI LETTER JA 3920xBB 0x091D # DEVANAGARI LETTER JHA 3930xBC 0x091E # DEVANAGARI LETTER NYA 3940xBD 0x091F # DEVANAGARI LETTER TTA 3950xBE 0x0920 # DEVANAGARI LETTER TTHA 3960xBF 0x0921 # DEVANAGARI LETTER DDA 3970xC0 0x0922 # DEVANAGARI LETTER DDHA 3980xC1 0x0923 # DEVANAGARI LETTER NNA 3990xC2 0x0924 # DEVANAGARI LETTER TA 4000xC3 0x0925 # DEVANAGARI LETTER THA 4010xC4 0x0926 # DEVANAGARI LETTER DA 4020xC5 0x0927 # DEVANAGARI LETTER DHA 4030xC6 0x0928 # DEVANAGARI LETTER NA 4040xC7 0x0929 # DEVANAGARI LETTER NNNA 4050xC8 0x092A # DEVANAGARI LETTER PA 4060xC9 0x092B # DEVANAGARI LETTER PHA 4070xCA 0x092C # DEVANAGARI LETTER BA 4080xCB 0x092D # DEVANAGARI LETTER BHA 4090xCC 0x092E # DEVANAGARI LETTER MA 4100xCD 0x092F # DEVANAGARI LETTER YA 4110xCE 0x095F # DEVANAGARI LETTER YYA 4120xCF 0x0930 # DEVANAGARI LETTER RA 4130xD0 0x0931 # DEVANAGARI LETTER RRA 4140xD1 0x0932 # DEVANAGARI LETTER LA 4150xD2 0x0933 # DEVANAGARI LETTER LLA 4160xD3 0x0934 # DEVANAGARI LETTER LLLA 4170xD4 0x0935 # DEVANAGARI LETTER VA 4180xD5 0x0936 # DEVANAGARI LETTER SHA 4190xD6 0x0937 # DEVANAGARI LETTER SSA 4200xD7 0x0938 # DEVANAGARI LETTER SA 4210xD8 0x0939 # DEVANAGARI LETTER HA 4220xD9 0x200E # LEFT-TO-RIGHT MARK # invisible consonant 4230xDA 0x093E # DEVANAGARI VOWEL SIGN AA 4240xDB 0x093F # DEVANAGARI VOWEL SIGN I 4250xDC 0x0940 # DEVANAGARI VOWEL SIGN II 4260xDD 0x0941 # DEVANAGARI VOWEL SIGN U 4270xDE 0x0942 # DEVANAGARI VOWEL SIGN UU 4280xDF 0x0943 # DEVANAGARI VOWEL SIGN VOCALIC R 4290xE0 0x0946 # DEVANAGARI VOWEL SIGN SHORT E 4300xE1 0x0947 # DEVANAGARI VOWEL SIGN E 4310xE2 0x0948 # DEVANAGARI VOWEL SIGN AI 4320xE3 0x0945 # DEVANAGARI VOWEL SIGN CANDRA E 4330xE4 0x094A # DEVANAGARI VOWEL SIGN SHORT O 4340xE5 0x094B # DEVANAGARI VOWEL SIGN O 4350xE6 0x094C # DEVANAGARI VOWEL SIGN AU 4360xE7 0x0949 # DEVANAGARI VOWEL SIGN CANDRA O 4370xE8 0x094D # DEVANAGARI SIGN VIRAMA # halant 4380xE9 0x093C # DEVANAGARI SIGN NUKTA 4390xEA 0x0964 # DEVANAGARI DANDA 440# 4410xF1 0x0966 # DEVANAGARI DIGIT ZERO 4420xF2 0x0967 # DEVANAGARI DIGIT ONE 4430xF3 0x0968 # DEVANAGARI DIGIT TWO 4440xF4 0x0969 # DEVANAGARI DIGIT THREE 4450xF5 0x096A # DEVANAGARI DIGIT FOUR 4460xF6 0x096B # DEVANAGARI DIGIT FIVE 4470xF7 0x096C # DEVANAGARI DIGIT SIX 4480xF8 0x096D # DEVANAGARI DIGIT SEVEN 4490xF9 0x096E # DEVANAGARI DIGIT EIGHT 4500xFA 0x096F # DEVANAGARI DIGIT NINE 451