1.\" # 2.\" # Copyright (c) 2021, Juniper Networks, Inc. 3.\" # All rights reserved. 4.\" # This SOFTWARE is licensed under the LICENSE provided in the 5.\" # ../Copyright file. By downloading, installing, copying, or 6.\" # using the SOFTWARE, you agree to be bound by the terms of that 7.\" # LICENSE. 8.\" # Phil Shafer, May 2021 9.\" 10.Dd May 13, 2021 11.Dt LIBXO-CSV 7 12.Os 13.Sh NAME 14.Nm --libxo encoder=csv[+options] 15.Nd a CVS encoder for libxo\-based commands 16.Sh DESCRIPTION 17The 18.Nm libxo 19library supports a "pluggable" encoder mechanism, and ships with an 20encoder for CSV (comma separated values) files. The encoder allows 21paths and fields to be selected out of the output contents: 22.Bd -literal -offset indent 23 % df --libxo @csv 24 name,total-blocks,used-blocks,available-blocks,used-percent,mounted-on 25 zroot/ROOT/default,3825984331,29376725,3796607605,1,/ 26 devfs,1,1,0,100,/dev 27 zroot/usr/home,3808301289,11693684,3796607605,0,/usr/home 28 zroot/var/audit,3796607806,201,3796607605,0,/var/audit 29 ... 30 % df --libxo @csv+leafs=name.used-percent 31 name,used-percent 32 zroot/ROOT/default,1 33 devfs,100 34 zroot/usr/home,0 35 zroot/var/audit,0 36 ... 37 % df --libxo @csv+leafs=available-blocks+no-header / 38 3796607605 39.Ed 40 contains software to generate four "built-in" 41formats: text, XML, JSON, and HTML. 42These formats are common and useful, but there are other common and 43useful formats that users will want, and including them all in the 44libxo software would be difficult and cumbersome. 45.Pp 46To allow support for additional encodings, libxo includes a 47"pluggable" extension mechanism for dynamically loading new encoders. 48.Nm libxo -based 49applications can automatically use any installed encoder. 50.Pp 51Use the "encoder=XXX" option to access encoders. The following 52example uses the "cbor" encoder, saving the output into a file: 53.Bd -literal -offset indent 54 df --libxo encoder=cbor > df-output.cbor 55.Ed 56.Pp 57Encoders can support specific options that can be accessed by 58following the encoder name with a colon (':') or a plus sign ('+') and 59one of more options, separated by the same character: 60.Bd -literal -offset indent 61 df --libxo encoder=csv+path=filesystem+leaf=name+no-header 62 df --libxo encoder=csv:path=filesystem:leaf=name:no-header 63.Ed 64.Pp 65These examples instructs libxo to load the "csv" encoder and pass the 66following options: 67.Bd -literal -offset indent 68 path=filesystem 69 leaf=name 70 no-header 71.Ed 72.Pp 73Each of these option is interpreted by the encoder, and all such 74options names and semantics are specific to the particular encoder. 75Refer to the intended encoder for documentation on its options. 76.Pp 77The string "@" can be used in place of the string "encoder=". 78.Bd -literal -offset indent 79 df --libxo @csv:no-header 80.Ed 81.Sh The CSV (Comma Separated Values) Encoder 82.Nm libxo 83ships with a custom encoder for "CSV" files, a common format for 84comma separated values. The output of the CSV encoder can be loaded 85directly into spreadsheets or similar applications. 86.Pp 87A standard for CSV files is provided in RFC 4180, but since the 88format predates that standard by decades, there are many minor 89differences in CSV file consumers and their expectations. The CSV 90encoder has a number of options to tailor output to those 91expectations. 92.Pp 93Consider the following XML: 94.Bd -literal -offset indent 95 % list-items --libxo xml,pretty 96 <top> 97 <data test="value"> 98 <item test2="value2"> 99 <sku test3="value3" key="key">GRO-000-415</sku> 100 <name key="key">gum</name> 101 <sold>1412</sold> 102 <in-stock>54</in-stock> 103 <on-order>10</on-order> 104 </item> 105 <item> 106 <sku test3="value3" key="key">HRD-000-212</sku> 107 <name key="key">rope</name> 108 <sold>85</sold> 109 <in-stock>4</in-stock> 110 <on-order>2</on-order> 111 </item> 112 <item> 113 <sku test3="value3" key="key">HRD-000-517</sku> 114 <name key="key">ladder</name> 115 <sold>0</sold> 116 <in-stock>2</in-stock> 117 <on-order>1</on-order> 118 </item> 119 </data> 120 </top> 121.Ed 122.Pp 123This output is a list of `instances` (named "item"), each containing a 124set of `leafs` ("sku", "name", etc). 125.Pp 126The CSV encoder will emit the leaf values in this output as `fields` 127inside a CSV `record`, which is a line containing a set of 128comma-separated values: 129.Bd -literal -offset indent 130 % list-items --libxo encoder=csv 131 sku,name,sold,in-stock,on-order 132 GRO-000-415,gum,1412,54,10 133 HRD-000-212,rope,85,4,2 134 HRD-000-517,ladder,0,2,1 135.Ed 136.Pp 137Be aware that since the CSV encoder looks for data instances, when 138used with 139.Nm xo , 140the `--instance` option will be needed: 141.Bd -literal -offset indent 142 % xo --libxo encoder=csv --instance foo 'The {:product} is {:status}\n' stereo "in route" 143 product,status 144 stereo,in route 145.Ed 146.Sh The "path" Option 147By default, the CSV encoder will attempt to emit any list instance 148generated by the application. 149In some cases, this may be unacceptable, and a specific list may be 150desired. 151.Pp 152Use the "path" option to limit the processing of output to a specific 153hierarchy. The path should be one or more names of containers or 154lists. 155.Pp 156For example, if the "list-items" application generates other lists, 157the user can give "path=top/data/item" as a path: 158.Bd -literal -offset indent 159 % list-items --libxo encoder=csv:path=top/data/item 160 sku,name,sold,in-stock,on-order 161 GRO-000-415,gum,1412,54,10 162 HRD-000-212,rope,85,4,2 163 HRD-000-517,ladder,0,2,1 164.Ed 165.Pp 166Paths are "relative", meaning they need not be a complete set 167of names to the list. This means that "path=item" may be sufficient 168for the above example. 169.Sh The "leafs" Option 170The CSV encoding requires that all lines of output have the same 171number of fields with the same order. In contrast, XML and JSON allow 172any order (though libxo forces key leafs to appear before other 173leafs). 174.Pp 175To maintain a consistent set of fields inside the CSV file, the same 176set of leafs must be selected from each list item. By default, the 177CSV encoder records the set of leafs that appear in the first list 178instance it processes, and extract only those leafs from future 179instances. If the first instance is missing a leaf that is desired by 180the consumer, the "leaf" option can be used to ensure that an empty 181value is recorded for instances that lack a particular leaf. 182.Pp 183The "leafs" option can also be used to exclude leafs, limiting the 184output to only those leafs provided. 185.Pp 186In addition, the order of the output fields follows the order in which 187the leafs are listed. "leafs=one.two" and "leafs=two.one" give 188distinct output. 189.Pp 190So the "leafs" option can be used to expand, limit, and order the set 191of leafs. 192.Pp 193The value of the leafs option should be one or more leaf names, 194separated by a period ("."): 195.Bd -literal -offset indent 196 % list-items --libxo encoder=csv:leafs=sku.on-order 197 sku,on-order 198 GRO-000-415,10 199 HRD-000-212,2 200 HRD-000-517,1 201 % list-items -libxo encoder=csv:leafs=on-order.sku 202 on-order,sku 203 10,GRO-000-415 204 2,HRD-000-212 205 1,HRD-000-517 206.Ed 207.Pp 208Note that since libxo uses terminology from YANG (:RFC:`7950`), the 209data modeling language for NETCONF (:RFC:`6241`), which uses "leafs" 210as the plural form of "leaf". libxo follows that convention. 211.Sh The "no-header" Option 212CSV files typical begin with a line that defines the fields included 213in that file, in an attempt to make the contents self-defining: 214.Bd -literal -offset indent 215 sku,name,sold,in-stock,on-order 216 GRO-000-415,gum,1412,54,10 217 HRD-000-212,rope,85,4,2 218 HRD-000-517,ladder,0,2,1 219.Ed 220.Pp 221There is no reliable mechanism for determining whether this header 222line is included, so the consumer must make an assumption. 223.Pp 224The csv encoder defaults to producing the header line, but the 225"no-header" option can be included to avoid the header line. 226.Sh The "no-quotes" Option 227RFC 4180 specifies that fields containing spaces should be quoted, but 228many CSV consumers do not handle quotes. The "no-quotes" option 229instruct the CSV encoder to avoid the use of quotes. 230.Sh The "dos" Option 231RFC 4180 defines the end-of-line marker as a carriage return 232followed by a newline. This "CRLF" convention dates from the distant 233past, but its use was anchored in the 1980s by the `DOS` operating 234system. 235.Pp 236The CSV encoder defaults to using the standard Unix end-of-line 237marker, a simple newline. Use the "dos" option to use the `CRLF` 238convention. 239.Sh Option Handling 240The handling of command-line options is complex, since there are three 241hierarchies in use, but the rules are: 242.Bl -bullet 243.It 244commas separate 245.Nm libxo 246options 247.Bd -literal -ofset indent 248 \-\-libxo json,pretty,warn 249.Ed 250.It 251pluses separate encoder options 252.Bd -literal -ofset indent 253\-\-libxo @csv+dos+no-header,warn 254.Ed 255.It 256periods separate tag names 257.Bd -literal -ofset indent 258\-\-libxo @csv+leafs=name.used.free+dos,warn 259.El 260.Sh SEE ALSO 261.Xr libxo 3 , 262.Xr xo_options 7 263.Sh HISTORY 264The 265.Nm libxo 266library first appeared in 267.Fx 11.0 . 268The CSV encoder first appeared in 269.Fx 13.0 . 270.Sh AUTHORS 271.Nm libxo 272was written by 273.An Phil Shafer Aq Mt phil@freebsd.org . 274 275