1<h2>mxj - to/from maps, XML and JSON</h2> 2Decode/encode XML to/from map[string]interface{} (or JSON) values, and extract/modify values from maps by key or key-path, including wildcards. 3 4mxj supplants the legacy x2j and j2x packages. If you want the old syntax, use mxj/x2j and mxj/j2x packages. 5 6<h4>Related Packages</h4> 7 8https://github.com/clbanning/checkxml provides functions for validating XML data. 9 10<h4>Refactor Decoder - 2015.11.15</h4> 11For over a year I've wanted to refactor the XML-to-map[string]interface{} decoder to make it more performant. I recently took the time to do that, since we were using github.com/clbanning/mxj in a production system that could be deployed on a Raspberry Pi. Now the decoder is comparable to the stdlib JSON-to-map[string]interface{} decoder in terms of its additional processing overhead relative to decoding to a structure value. As shown by: 12 13 BenchmarkNewMapXml-4 100000 18043 ns/op 14 BenchmarkNewStructXml-4 100000 14892 ns/op 15 BenchmarkNewMapJson-4 300000 4633 ns/op 16 BenchmarkNewStructJson-4 300000 3427 ns/op 17 BenchmarkNewMapXmlBooks-4 20000 82850 ns/op 18 BenchmarkNewStructXmlBooks-4 20000 67822 ns/op 19 BenchmarkNewMapJsonBooks-4 100000 17222 ns/op 20 BenchmarkNewStructJsonBooks-4 100000 15309 ns/op 21 22<h4>Notices</h4> 23 24 2018.04.18: mv.Xml/mv.XmlIndent encodes non-map[string]interface{} map values - map[string]string, map[int]uint, etc. 25 2018.03.29: mv.Gob/NewMapGob support gob encoding/decoding of Maps. 26 2018.03.26: Added mxj/x2j-wrapper sub-package for migrating from legacy x2j package. 27 2017.02.22: LeafNode paths can use ".N" syntax rather than "[N]" for list member indexing. 28 2017.02.10: SetFieldSeparator changes field separator for args in UpdateValuesForPath, ValuesFor... methods. 29 2017.02.06: Support XMPP stream processing - HandleXMPPStreamTag(). 30 2016.11.07: Preserve name space prefix syntax in XmlSeq parser - NewMapXmlSeq(), etc. 31 2016.06.25: Support overriding default XML attribute prefix, "-", in Map keys - SetAttrPrefix(). 32 2016.05.26: Support customization of xml.Decoder by exposing CustomDecoder variable. 33 2016.03.19: Escape invalid chars when encoding XML attribute and element values - XMLEscapeChars(). 34 2016.03.02: By default decoding XML with float64 and bool value casting will not cast "NaN", "Inf", and "-Inf". 35 To cast them to float64, first set flag with CastNanInf(true). 36 2016.02.22: New mv.Root(), mv.Elements(), mv.Attributes methods let you examine XML document structure. 37 2016.02.16: Add CoerceKeysToLower() option to handle tags with mixed capitalization. 38 2016.02.12: Seek for first xml.StartElement token; only return error if io.EOF is reached first (handles BOM). 39 2015.12.02: XML decoding/encoding that preserves original structure of document. See NewMapXmlSeq() 40 and mv.XmlSeq() / mv.XmlSeqIndent(). 41 2015-05-20: New: mv.StringIndentNoTypeInfo(). 42 Also, alphabetically sort map[string]interface{} values by key to prettify output for mv.Xml(), 43 mv.XmlIndent(), mv.StringIndent(), mv.StringIndentNoTypeInfo(). 44 2014-11-09: IncludeTagSeqNum() adds "_seq" key with XML doc positional information. 45 (NOTE: PreserveXmlList() is similar and will be here soon.) 46 2014-09-18: inspired by NYTimes fork, added PrependAttrWithHyphen() to allow stripping hyphen from attribute tag. 47 2014-08-02: AnyXml() and AnyXmlIndent() will try to marshal arbitrary values to XML. 48 2014-04-28: ValuesForPath() and NewMap() now accept path with indexed array references. 49 50<h4>Basic Unmarshal XML to map[string]interface{}</h4> 51<pre>type Map map[string]interface{}</pre> 52 53Create a `Map` value, 'mv', from any `map[string]interface{}` value, 'v': 54<pre>mv := Map(v)</pre> 55 56Unmarshal / marshal XML as a `Map` value, 'mv': 57<pre>mv, err := NewMapXml(xmlValue) // unmarshal 58xmlValue, err := mv.Xml() // marshal</pre> 59 60Unmarshal XML from an `io.Reader` as a `Map` value, 'mv': 61<pre>mv, err := NewMapXmlReader(xmlReader) // repeated calls, as with an os.File Reader, will process stream 62mv, raw, err := NewMapXmlReaderRaw(xmlReader) // 'raw' is the raw XML that was decoded</pre> 63 64Marshal `Map` value, 'mv', to an XML Writer (`io.Writer`): 65<pre>err := mv.XmlWriter(xmlWriter) 66raw, err := mv.XmlWriterRaw(xmlWriter) // 'raw' is the raw XML that was written on xmlWriter</pre> 67 68Also, for prettified output: 69<pre>xmlValue, err := mv.XmlIndent(prefix, indent, ...) 70err := mv.XmlIndentWriter(xmlWriter, prefix, indent, ...) 71raw, err := mv.XmlIndentWriterRaw(xmlWriter, prefix, indent, ...)</pre> 72 73Bulk process XML with error handling (note: handlers must return a boolean value): 74<pre>err := HandleXmlReader(xmlReader, mapHandler(Map), errHandler(error)) 75err := HandleXmlReaderRaw(xmlReader, mapHandler(Map, []byte), errHandler(error, []byte))</pre> 76 77Converting XML to JSON: see Examples for `NewMapXml` and `HandleXmlReader`. 78 79There are comparable functions and methods for JSON processing. 80 81Arbitrary structure values can be decoded to / encoded from `Map` values: 82<pre>mv, err := NewMapStruct(structVal) 83err := mv.Struct(structPointer)</pre> 84 85<h4>Extract / modify Map values</h4> 86To work with XML tag values, JSON or Map key values or structure field values, decode the XML, JSON 87or structure to a `Map` value, 'mv', or cast a `map[string]interface{}` value to a `Map` value, 'mv', then: 88<pre>paths := mv.PathsForKey(key) 89path := mv.PathForKeyShortest(key) 90values, err := mv.ValuesForKey(key, subkeys) 91values, err := mv.ValuesForPath(path, subkeys) 92count, err := mv.UpdateValuesForPath(newVal, path, subkeys)</pre> 93 94Get everything at once, irrespective of path depth: 95<pre>leafnodes := mv.LeafNodes() 96leafvalues := mv.LeafValues()</pre> 97 98A new `Map` with whatever keys are desired can be created from the current `Map` and then encoded in XML 99or JSON. (Note: keys can use dot-notation.) 100<pre>newMap, err := mv.NewMap("oldKey_1:newKey_1", "oldKey_2:newKey_2", ..., "oldKey_N:newKey_N") 101newMap, err := mv.NewMap("oldKey1", "oldKey3", "oldKey5") // a subset of 'mv'; see "examples/partial.go" 102newXml, err := newMap.Xml() // for example 103newJson, err := newMap.Json() // ditto</pre> 104 105<h4>Usage</h4> 106 107The package is fairly well [self-documented with examples](http://godoc.org/github.com/clbanning/mxj). 108 109Also, the subdirectory "examples" contains a wide range of examples, several taken from golang-nuts discussions. 110 111<h4>XML parsing conventions</h4> 112 113Using NewMapXml() 114 115 - Attributes are parsed to `map[string]interface{}` values by prefixing a hyphen, `-`, 116 to the attribute label. (Unless overridden by `PrependAttrWithHyphen(false)` or 117 `SetAttrPrefix()`.) 118 - If the element is a simple element and has attributes, the element value 119 is given the key `#text` for its `map[string]interface{}` representation. (See 120 the 'atomFeedString.xml' test data, below.) 121 - XML comments, directives, and process instructions are ignored. 122 - If CoerceKeysToLower() has been called, then the resultant keys will be lower case. 123 124Using NewMapXmlSeq() 125 126 - Attributes are parsed to `map["#attr"]map[<attr_label>]map[string]interface{}`values 127 where the `<attr_label>` value has "#text" and "#seq" keys - the "#text" key holds the 128 value for `<attr_label>`. 129 - All elements, except for the root, have a "#seq" key. 130 - Comments, directives, and process instructions are unmarshalled into the Map using the 131 keys "#comment", "#directive", and "#procinst", respectively. (See documentation for more 132 specifics.) 133 - Name space syntax is preserved: 134 - `<ns:key>something</ns.key>` parses to `map["ns:key"]interface{}{"something"}` 135 - `xmlns:ns="http://myns.com/ns"` parses to `map["xmlns:ns"]interface{}{"http://myns.com/ns"}` 136 137Both 138 139 - By default, "Nan", "Inf", and "-Inf" values are not cast to float64. If you want them 140 to be cast, set a flag to cast them using CastNanInf(true). 141 142<h4>XML encoding conventions</h4> 143 144 - 'nil' `Map` values, which may represent 'null' JSON values, are encoded as `<tag/>`. 145 NOTE: the operation is not symmetric as `<tag/>` elements are decoded as `tag:""` `Map` values, 146 which, then, encode in JSON as `"tag":""` values. 147 - ALSO: there is no guarantee that the encoded XML doc will be the same as the decoded one. (Go 148 randomizes the walk through map[string]interface{} values.) If you plan to re-encode the 149 Map value to XML and want the same sequencing of elements look at NewMapXmlSeq() and 150 mv.XmlSeq() - these try to preserve the element sequencing but with added complexity when 151 working with the Map representation. 152 153<h4>Running "go test"</h4> 154 155Because there are no guarantees on the sequence map elements are retrieved, the tests have been 156written for visual verification in most cases. One advantage is that you can easily use the 157output from running "go test" as examples of calling the various functions and methods. 158 159<h4>Motivation</h4> 160 161I make extensive use of JSON for messaging and typically unmarshal the messages into 162`map[string]interface{}` values. This is easily done using `json.Unmarshal` from the 163standard Go libraries. Unfortunately, many legacy solutions use structured 164XML messages; in those environments the applications would have to be refactored to 165interoperate with my components. 166 167The better solution is to just provide an alternative HTTP handler that receives 168XML messages and parses it into a `map[string]interface{}` value and then reuse 169all the JSON-based code. The Go `xml.Unmarshal()` function does not provide the same 170option of unmarshaling XML messages into `map[string]interface{}` values. So I wrote 171a couple of small functions to fill this gap and released them as the x2j package. 172 173Over the next year and a half additional features were added, and the companion j2x 174package was released to address XML encoding of arbitrary JSON and `map[string]interface{}` 175values. As part of a refactoring of our production system and looking at how we had been 176using the x2j and j2x packages we found that we rarely performed direct XML-to-JSON or 177JSON-to_XML conversion and that working with the XML or JSON as `map[string]interface{}` 178values was the primary value. Thus, everything was refactored into the mxj package. 179 180