• Home
  • History
  • Annotate
Name Date Size #Lines LOC

..03-May-2022-

.gitignoreH A D06-Nov-2020474

.travis.ymlH A D06-Nov-2020771

LICENSEH A D06-Nov-202011.1 KiB

README.mdH A D06-Nov-20202 KiB

go.modH A D06-Nov-202036

utfbom.goH A D06-Nov-20204.5 KiB

utfbom_test.goH A D06-Nov-20206.9 KiB

README.md

1# utfbom [![Godoc](https://godoc.org/github.com/dimchansky/utfbom?status.png)](https://godoc.org/github.com/dimchansky/utfbom) [![License](https://img.shields.io/:license-apache-blue.svg)](https://opensource.org/licenses/Apache-2.0) [![Build Status](https://travis-ci.org/dimchansky/utfbom.svg?branch=master)](https://travis-ci.org/dimchansky/utfbom) [![Go Report Card](https://goreportcard.com/badge/github.com/dimchansky/utfbom)](https://goreportcard.com/report/github.com/dimchansky/utfbom) [![Coverage Status](https://coveralls.io/repos/github/dimchansky/utfbom/badge.svg?branch=master)](https://coveralls.io/github/dimchansky/utfbom?branch=master)
2
3The package utfbom implements the detection of the BOM (Unicode Byte Order Mark) and removing as necessary. It can also return the encoding detected by the BOM.
4
5## Installation
6
7    go get -u github.com/dimchansky/utfbom
8
9## Example
10
11```go
12package main
13
14import (
15	"bytes"
16	"fmt"
17	"io/ioutil"
18
19	"github.com/dimchansky/utfbom"
20)
21
22func main() {
23	trySkip([]byte("\xEF\xBB\xBFhello"))
24	trySkip([]byte("hello"))
25}
26
27func trySkip(byteData []byte) {
28	fmt.Println("Input:", byteData)
29
30	// just skip BOM
31	output, err := ioutil.ReadAll(utfbom.SkipOnly(bytes.NewReader(byteData)))
32	if err != nil {
33		fmt.Println(err)
34		return
35	}
36	fmt.Println("ReadAll with BOM skipping", output)
37
38	// skip BOM and detect encoding
39	sr, enc := utfbom.Skip(bytes.NewReader(byteData))
40	fmt.Printf("Detected encoding: %s\n", enc)
41	output, err = ioutil.ReadAll(sr)
42	if err != nil {
43		fmt.Println(err)
44		return
45	}
46	fmt.Println("ReadAll with BOM detection and skipping", output)
47	fmt.Println()
48}
49```
50
51Output:
52
53```
54$ go run main.go
55Input: [239 187 191 104 101 108 108 111]
56ReadAll with BOM skipping [104 101 108 108 111]
57Detected encoding: UTF8
58ReadAll with BOM detection and skipping [104 101 108 108 111]
59
60Input: [104 101 108 108 111]
61ReadAll with BOM skipping [104 101 108 108 111]
62Detected encoding: Unknown
63ReadAll with BOM detection and skipping [104 101 108 108 111]
64```
65
66
67