1 2# Documentation & Support 3http://www.slideshare.net/buganini/bsdconv 4 5http://www.slideshare.net/Buganini/journey-of-bsdconv 6 7API Reference: http://buganini.github.io/bsdconv/ 8 9Use bsdconv-man to show manual page for each module 10 11IRC: irc://irc.freenode.net#bsdconv 12 13# Compilation & Installation 14``` 15make PREFIX=${prefix} # default to /usr/local 16sudo make install PREFIX=${prefix} # default to /usr/local 17sudo ldconfig ${prefix}/lib # Linux 18sudo ldconfig -m ${prefix}/lib # FreeBSD 19``` 20 21# Add codec alias 22``` 23Update modules/{from,inter,to}/alias 24make alias 25``` 26 27# Example 28 29Convert traditional chinese big5 to simplified chinese utf-8 30``` 31bsdconv big5:zhcn:utf-8 in.txt > out.txt 32bsdconv big5:zhcn:utf-8 -i in.txt #inplace 33``` 34 35Convert traditional chinese utf-8 to simplified chinese GB2312 with transliteration 36``` 37bsdconv utf-8:zhcn:cp936,cp936-trans in.txt > out.txt 38``` 39 40Convert simplified chinese to traditional chinese 41 42``` 43bsdconv utf-8:zhtw:zhtw-words:utf-8 44``` 45 46And ignoring whitespaces mixed in words 47``` 48bsdconv utf-8:whitespace-derail:zhtw:zhtw-words:whitespace-rerail:utf-8 49``` 50 51Convert big5 data, traditional chinese to simplified chinese, 52CRLF/CR/LF to CRLF, to big5 data, translate simplified chinese words, which are 53not in big5, to HTML entities, and uppercase the ascii characters. 54``` 55bsdconv big5:zhcn:win:upper:big5,htmlentity in.txt > out.txt 56``` 57 58Counting character width 59``` 60echo -n "aａ" | bsdconv utf-8:width:null 61FULL: 1 62HALF: 1 63 64echo -n "aａˇ" | bsdconv utf-8:width:null 65FULL: 1 66HALF: 1 67AMBI: 1 68``` 69 70Very useful for migrating MySQL DB from Big5 to UTF-8 71``` 72bsdconv htmlentity,big5-5c,big5:utf-8 in.sql > out.sql 73``` 74 75Recover from mis-decoding/encoding (mistreated big5 as iso-8859-1 and converted to utf-8) 76``` 77bsdconv 'utf-8:iso-8859-1|big5:utf-8' 78``` 79 80Decode escaped data (byte/unicode mixed) like %u9644%20 81``` 82bsdconv 'escape,byte:unicode,byte|skip,ascii:utf-8' 83``` 84 85Generate string for fuzzy comparison 86``` 87echo ¼ℌăDžⓐ⁹灣湾ド��鬒æß | bsdconv UTF-8:ZH-FUZZY-TW:KANA-PHONETIC:NFKD-CASEFOLD:UTF-8 88 1⁄4hădža9灣灣do鬒鬒æss 89``` 90 91Translate text to HTML <IMG /> 92``` 93bsdconv big5:nl2br:ascii,html-img in.txt > out.htm 94``` 95 96Use glyph image from http://www.cns11643.gov.tw 97``` 98bsdconv utf-8:ascii,ascii-html-cns11643-img in.txt out.htm 99``` 100 101Maintain inter map: 102``` 103bsdconv bsdconv-keyword,bsdconv:bsdconv-keyword,utf-8 inter/FOO.txt > edit.tmp 104vi edit.tmp 105bsdconv bsdconv-keyword,utf-8:bsdconv-keyword,bsdconv edit.tmp > inter/FOO.txt 106``` 107 108# Windows 109Use mingw with Makefile.win to build it, then copy everythings in build/ to c:\bsdconv\ 110the path of the executable will be c:\bsdconv\bsdconv.exe 111 112If you want to install to directory other than default path 113set BSDCONV_PATH environment variable to your path. 114 115Run setEnvVar.bat as administrator could help you set proper environment variables. 116 117# Bindings 118[Python](https://pypi.python.org/pypi/bsdconv/ "Python") 119 120[Perl](https://github.com/buganini/perl-bsdconv "Perl") 121 122[PHP](https://github.com/buganini/php-bsdconv "PHP") 123 124[Ruby](https://rubygems.org/gems/ruby-bsdconv/ "Ruby") 125 126[Go](https://github.com/buganini/go-bsdconv "Go") 127 128[Java](https://github.com/buganini/jni-bsdconv "Java") 129 130[Haskell](https://github.com/pkmx/hs-bsdconv "Haskell") 131 132[Elasticsearch](https://github.com/buganini/elasticsearch-bsdconv-plugin "Elasticsearch") 133 134[PostgreSQL](https://github.com/buganini/postgres-bsdconv "PostgreSQL") 135 136[MySQL](https://github.com/buganini/mysql-udf-bsdconv "MySQL") 137