1Use in C++ {#flatbuffers_guide_use_cpp} 2========== 3 4## Before you get started 5 6Before diving into the FlatBuffers usage in C++, it should be noted that 7the [Tutorial](@ref flatbuffers_guide_tutorial) page has a complete guide 8to general FlatBuffers usage in all of the supported languages (including C++). 9This page is designed to cover the nuances of FlatBuffers usage, specific to 10C++. 11 12#### Prerequisites 13 14This page assumes you have written a FlatBuffers schema and compiled it 15with the Schema Compiler. If you have not, please see 16[Using the schema compiler](@ref flatbuffers_guide_using_schema_compiler) 17and [Writing a schema](@ref flatbuffers_guide_writing_schema). 18 19Assuming you wrote a schema, say `mygame.fbs` (though the extension doesn't 20matter), you've generated a C++ header called `mygame_generated.h` using the 21compiler (e.g. `flatc -c mygame.fbs`), you can now start using this in 22your program by including the header. As noted, this header relies on 23`flatbuffers/flatbuffers.h`, which should be in your include path. 24 25## FlatBuffers C++ library code location 26 27The code for the FlatBuffers C++ library can be found at 28`flatbuffers/include/flatbuffers`. You can browse the library code on the 29[FlatBuffers GitHub page](https://github.com/google/flatbuffers/tree/master/include/flatbuffers). 30 31## Testing the FlatBuffers C++ library 32 33The code to test the C++ library can be found at `flatbuffers/tests`. 34The test code itself is located in 35[test.cpp](https://github.com/google/flatbuffers/blob/master/tests/test.cpp). 36 37This test file is built alongside `flatc`. To review how to build the project, 38please read the [Building](@ref flatbuffers_guide_building) documenation. 39 40To run the tests, execute `flattests` from the root `flatbuffers/` directory. 41For example, on [Linux](https://en.wikipedia.org/wiki/Linux), you would simply 42run: `./flattests`. 43 44## Using the FlatBuffers C++ library 45 46*Note: See [Tutorial](@ref flatbuffers_guide_tutorial) for a more in-depth 47example of how to use FlatBuffers in C++.* 48 49FlatBuffers supports both reading and writing FlatBuffers in C++. 50 51To use FlatBuffers in your code, first generate the C++ classes from your 52schema with the `--cpp` option to `flatc`. Then you can include both FlatBuffers 53and the generated code to read or write FlatBuffers. 54 55For example, here is how you would read a FlatBuffer binary file in C++: 56First, include the library and generated code. Then read the file into 57a `char *` array, which you pass to `GetMonster()`. 58 59~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.cpp} 60 #include "flatbuffers/flatbuffers.h" 61 #include "monster_test_generate.h" 62 #include <iostream> // C++ header file for printing 63 #include <fstream> // C++ header file for file access 64 65 66 std::ifstream infile; 67 infile.open("monsterdata_test.mon", std::ios::binary | std::ios::in); 68 infile.seekg(0,std::ios::end); 69 int length = infile.tellg(); 70 infile.seekg(0,std::ios::beg); 71 char *data = new char[length]; 72 infile.read(data, length); 73 infile.close(); 74 75 auto monster = GetMonster(data); 76~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 77 78`monster` is of type `Monster *`, and points to somewhere *inside* your 79buffer (root object pointers are not the same as `buffer_pointer` !). 80If you look in your generated header, you'll see it has 81convenient accessors for all fields, e.g. `hp()`, `mana()`, etc: 82 83~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.cpp} 84 std::cout << "hp : " << monster->hp() << std::endl; // `80` 85 std::cout << "mana : " << monster->mana() << std::endl; // default value of `150` 86 std::cout << "name : " << monster->name()->c_str() << std::endl; // "MyMonster" 87~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 88 89*Note: That we never stored a `mana` value, so it will return the default.* 90 91The following attributes are supported: 92 93- `shared` (on a field): For string fields, this enables the usage of string 94 pooling (i.e. `CreateSharedString`) as default serialization behavior. 95 96 Specifically, `CreateXxxDirect` functions and `Pack` functions for object 97 based API (see below) will use `CreateSharedString` to create strings. 98 99## Object based API. {#flatbuffers_cpp_object_based_api} 100 101FlatBuffers is all about memory efficiency, which is why its base API is written 102around using as little as possible of it. This does make the API clumsier 103(requiring pre-order construction of all data, and making mutation harder). 104 105For times when efficiency is less important a more convenient object based API 106can be used (through `--gen-object-api`) that is able to unpack & pack a 107FlatBuffer into objects and standard STL containers, allowing for convenient 108construction, access and mutation. 109 110To use: 111 112~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.cpp} 113 // Autogenerated class from table Monster. 114 MonsterT monsterobj; 115 116 // Deserialize from buffer into object. 117 UnPackTo(&monsterobj, flatbuffer); 118 119 // Update object directly like a C++ class instance. 120 cout << monsterobj->name; // This is now a std::string! 121 monsterobj->name = "Bob"; // Change the name. 122 123 // Serialize into new flatbuffer. 124 FlatBufferBuilder fbb; 125 Pack(fbb, &monsterobj); 126~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 127 128The following attributes are specific to the object-based API code generation: 129 130- `native_inline` (on a field): Because FlatBuffer tables and structs are 131 optionally present in a given buffer, they are best represented as pointers 132 (specifically std::unique_ptrs) in the native class since they can be null. 133 This attribute changes the member declaration to use the type directly 134 rather than wrapped in a unique_ptr. 135 136- `native_default`: "value" (on a field): For members that are declared 137 "native_inline", the value specified with this attribute will be included 138 verbatim in the class constructor initializer list for this member. 139 140- `native_custom_alloc`:"custom_allocator" (on a table or struct): When using the 141 object-based API all generated NativeTables that are allocated when unpacking 142 your flatbuffer will use "custom allocator". The allocator is also used by 143 any std::vector that appears in a table defined with `native_custom_alloc`. 144 This can be used to provide allocation from a pool for example, for faster 145 unpacking when using the object-based API. 146 147 Minimal Example: 148 149 schema: 150 151 table mytable(native_custom_alloc:"custom_allocator") { 152 ... 153 } 154 155 with custom_allocator defined before flatbuffers.h is included, as: 156 157 template <typename T> struct custom_allocator : public std::allocator<T> { 158 159 typedef T *pointer; 160 161 template <class U> 162 struct rebind { 163 typedef custom_allocator<U> other; 164 }; 165 166 pointer allocate(const std::size_t n) { 167 return std::allocator<T>::allocate(n); 168 } 169 170 void deallocate(T* ptr, std::size_t n) { 171 return std::allocator<T>::deallocate(ptr,n); 172 } 173 174 custom_allocator() throw() {} 175 template <class U> 176 custom_allocator(const custom_allocator<U>&) throw() {} 177 }; 178 179- `native_type`' "type" (on a struct): In some cases, a more optimal C++ data 180 type exists for a given struct. For example, the following schema: 181 182 struct Vec2 { 183 x: float; 184 y: float; 185 } 186 187 generates the following Object-Based API class: 188 189 struct Vec2T : flatbuffers::NativeTable { 190 float x; 191 float y; 192 }; 193 194 However, it can be useful to instead use a user-defined C++ type since it 195 can provide more functionality, eg. 196 197 struct vector2 { 198 float x = 0, y = 0; 199 vector2 operator+(vector2 rhs) const { ... } 200 vector2 operator-(vector2 rhs) const { ... } 201 float length() const { ... } 202 // etc. 203 }; 204 205 The `native_type` attribute will replace the usage of the generated class 206 with the given type. So, continuing with the example, the generated 207 code would use |vector2| in place of |Vec2T| for all generated code. 208 209 However, becuase the native_type is unknown to flatbuffers, the user must 210 provide the following functions to aide in the serialization process: 211 212 namespace flatbuffers { 213 FlatbufferStruct Pack(const native_type& obj); 214 native_type UnPack(const FlatbufferStruct& obj); 215 } 216 217Finally, the following top-level attribute 218 219- `native_include`: "path" (at file level): Because the `native_type` attribute 220 can be used to introduce types that are unknown to flatbuffers, it may be 221 necessary to include "external" header files in the generated code. This 222 attribute can be used to directly add an #include directive to the top of 223 the generated code that includes the specified path directly. 224 225- `force_align`: this attribute may not be respected in the object API, 226 depending on the aligned of the allocator used with `new`. 227 228# External references. 229 230An additional feature of the object API is the ability to allow you to load 231multiple independent FlatBuffers, and have them refer to eachothers objects 232using hashes which are then represented as typed pointers in the object API. 233 234To make this work have a field in the objects you want to referred to which is 235using the string hashing feature (see `hash` attribute in the 236[schema](@ref flatbuffers_guide_writing_schema) documentation). Then you have 237a similar hash in the field referring to it, along with a `cpp_type` 238attribute specifying the C++ type this will refer to (this can be any C++ 239type, and will get a `*` added). 240 241Then, in JSON or however you create these buffers, make sure they use the 242same string (or hash). 243 244When you call `UnPack` (or `Create`), you'll need a function that maps from 245hash to the object (see `resolver_function_t` for details). 246 247# Using different pointer types. 248 249By default the object tree is built out of `std::unique_ptr`, but you can 250influence this either globally (using the `--cpp-ptr-type` argument to 251`flatc`) or per field (using the `cpp_ptr_type` attribute) to by any smart 252pointer type (`my_ptr<T>`), or by specifying `naked` as the type to get `T *` 253pointers. Unlike the smart pointers, naked pointers do not manage memory for 254you, so you'll have to manage their lifecycles manually. To reference the 255pointer type specified by the `--cpp-ptr-type` argument to `flatc` from a 256flatbuffer field set the `cpp_ptr_type` attribute to `default_ptr_type`. 257 258# Using different string type. 259 260By default the object tree is built out of `std::string`, but you can 261influence this either globally (using the `--cpp-str-type` argument to 262`flatc`) or per field using the `cpp_str_type` attribute. 263 264The type must support T::c_str(), T::length() and T::empty() as member functions. 265 266Further, the type must be constructible from std::string, as by default a 267std::string instance is constructed and then used to initialize the custom 268string type. This behavior impedes efficient and zero-copy construction of 269custom string types; the `--cpp-str-flex-ctor` argument to `flatc` or the 270per field attribute `cpp_str_flex_ctor` can be used to change this behavior, 271so that the custom string type is constructed by passing the pointer and 272length of the FlatBuffers String. The custom string class will require a 273constructor in the following format: custom_str_class(const char *, size_t). 274Please note that the character array is not guaranteed to be NULL terminated, 275you should always use the provided size to determine end of string. 276 277## Reflection (& Resizing) 278 279There is experimental support for reflection in FlatBuffers, allowing you to 280read and write data even if you don't know the exact format of a buffer, and 281even allows you to change sizes of strings and vectors in-place. 282 283The way this works is very elegant; there is actually a FlatBuffer schema that 284describes schemas (!) which you can find in `reflection/reflection.fbs`. 285The compiler, `flatc`, can write out any schemas it has just parsed as a binary 286FlatBuffer, corresponding to this meta-schema. 287 288Loading in one of these binary schemas at runtime allows you traverse any 289FlatBuffer data that corresponds to it without knowing the exact format. You 290can query what fields are present, and then read/write them after. 291 292For convenient field manipulation, you can include the header 293`flatbuffers/reflection.h` which includes both the generated code from the meta 294schema, as well as a lot of helper functions. 295 296And example of usage, for the time being, can be found in 297`test.cpp/ReflectionTest()`. 298 299## Mini Reflection 300 301A more limited form of reflection is available for direct inclusion in 302generated code, which doesn't any (binary) schema access at all. It was designed 303to keep the overhead of reflection as low as possible (on the order of 2-6 304bytes per field added to your executable), but doesn't contain all the 305information the (binary) schema contains. 306 307You add this information to your generated code by specifying `--reflect-types` 308(or instead `--reflect-names` if you also want field / enum names). 309 310You can now use this information, for example to print a FlatBuffer to text: 311 312 auto s = flatbuffers::FlatBufferToString(flatbuf, MonsterTypeTable()); 313 314`MonsterTypeTable()` is declared in the generated code for each type. The 315string produced is very similar to the JSON produced by the `Parser` based 316text generator. 317 318You'll need `flatbuffers/minireflect.h` for this functionality. In there is also 319a convenient visitor/iterator so you can write your own output / functionality 320based on the mini reflection tables without having to know the FlatBuffers or 321reflection encoding. 322 323## Storing maps / dictionaries in a FlatBuffer 324 325FlatBuffers doesn't support maps natively, but there is support to 326emulate their behavior with vectors and binary search, which means you 327can have fast lookups directly from a FlatBuffer without having to unpack 328your data into a `std::map` or similar. 329 330To use it: 331- Designate one of the fields in a table as they "key" field. You do this 332 by setting the `key` attribute on this field, e.g. 333 `name:string (key)`. 334 You may only have one key field, and it must be of string or scalar type. 335- Write out tables of this type as usual, collect their offsets in an 336 array or vector. 337- Instead of `CreateVector`, call `CreateVectorOfSortedTables`, 338 which will first sort all offsets such that the tables they refer to 339 are sorted by the key field, then serialize it. 340- Now when you're accessing the FlatBuffer, you can use `Vector::LookupByKey` 341 instead of just `Vector::Get` to access elements of the vector, e.g.: 342 `myvector->LookupByKey("Fred")`, which returns a pointer to the 343 corresponding table type, or `nullptr` if not found. 344 `LookupByKey` performs a binary search, so should have a similar speed to 345 `std::map`, though may be faster because of better caching. `LookupByKey` 346 only works if the vector has been sorted, it will likely not find elements 347 if it hasn't been sorted. 348 349## Direct memory access 350 351As you can see from the above examples, all elements in a buffer are 352accessed through generated accessors. This is because everything is 353stored in little endian format on all platforms (the accessor 354performs a swap operation on big endian machines), and also because 355the layout of things is generally not known to the user. 356 357For structs, layout is deterministic and guaranteed to be the same 358across platforms (scalars are aligned to their 359own size, and structs themselves to their largest member), and you 360are allowed to access this memory directly by using `sizeof()` and 361`memcpy` on the pointer to a struct, or even an array of structs. 362 363To compute offsets to sub-elements of a struct, make sure they 364are a structs themselves, as then you can use the pointers to 365figure out the offset without having to hardcode it. This is 366handy for use of arrays of structs with calls like `glVertexAttribPointer` 367in OpenGL or similar APIs. 368 369It is important to note is that structs are still little endian on all 370machines, so only use tricks like this if you can guarantee you're not 371shipping on a big endian machine (an `assert(FLATBUFFERS_LITTLEENDIAN)` 372would be wise). 373 374## Access of untrusted buffers 375 376The generated accessor functions access fields over offsets, which is 377very quick. These offsets are not verified at run-time, so a malformed 378buffer could cause a program to crash by accessing random memory. 379 380When you're processing large amounts of data from a source you know (e.g. 381your own generated data on disk), this is acceptable, but when reading 382data from the network that can potentially have been modified by an 383attacker, this is undesirable. 384 385For this reason, you can optionally use a buffer verifier before you 386access the data. This verifier will check all offsets, all sizes of 387fields, and null termination of strings to ensure that when a buffer 388is accessed, all reads will end up inside the buffer. 389 390Each root type will have a verification function generated for it, 391e.g. for `Monster`, you can call: 392 393~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.cpp} 394 bool ok = VerifyMonsterBuffer(Verifier(buf, len)); 395~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 396 397if `ok` is true, the buffer is safe to read. 398 399Besides untrusted data, this function may be useful to call in debug 400mode, as extra insurance against data being corrupted somewhere along 401the way. 402 403While verifying a buffer isn't "free", it is typically faster than 404a full traversal (since any scalar data is not actually touched), 405and since it may cause the buffer to be brought into cache before 406reading, the actual overhead may be even lower than expected. 407 408In specialized cases where a denial of service attack is possible, 409the verifier has two additional constructor arguments that allow 410you to limit the nesting depth and total amount of tables the 411verifier may encounter before declaring the buffer malformed. The default is 412`Verifier(buf, len, 64 /* max depth */, 1000000, /* max tables */)` which 413should be sufficient for most uses. 414 415## Text & schema parsing 416 417Using binary buffers with the generated header provides a super low 418overhead use of FlatBuffer data. There are, however, times when you want 419to use text formats, for example because it interacts better with source 420control, or you want to give your users easy access to data. 421 422Another reason might be that you already have a lot of data in JSON 423format, or a tool that generates JSON, and if you can write a schema for 424it, this will provide you an easy way to use that data directly. 425 426(see the schema documentation for some specifics on the JSON format 427accepted). 428 429There are two ways to use text formats: 430 431#### Using the compiler as a conversion tool 432 433This is the preferred path, as it doesn't require you to add any new 434code to your program, and is maximally efficient since you can ship with 435binary data. The disadvantage is that it is an extra step for your 436users/developers to perform, though you might be able to automate it. 437 438 flatc -b myschema.fbs mydata.json 439 440This will generate the binary file `mydata_wire.bin` which can be loaded 441as before. 442 443#### Making your program capable of loading text directly 444 445This gives you maximum flexibility. You could even opt to support both, 446i.e. check for both files, and regenerate the binary from text when 447required, otherwise just load the binary. 448 449This option is currently only available for C++, or Java through JNI. 450 451As mentioned in the section "Building" above, this technique requires 452you to link a few more files into your program, and you'll want to include 453`flatbuffers/idl.h`. 454 455Load text (either a schema or json) into an in-memory buffer (there is a 456convenient `LoadFile()` utility function in `flatbuffers/util.h` if you 457wish). Construct a parser: 458 459~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.cpp} 460 flatbuffers::Parser parser; 461~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 462 463Now you can parse any number of text files in sequence: 464 465~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.cpp} 466 parser.Parse(text_file.c_str()); 467~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 468 469This works similarly to how the command-line compiler works: a sequence 470of files parsed by the same `Parser` object allow later files to 471reference definitions in earlier files. Typically this means you first 472load a schema file (which populates `Parser` with definitions), followed 473by one or more JSON files. 474 475As optional argument to `Parse`, you may specify a null-terminated list of 476include paths. If not specified, any include statements try to resolve from 477the current directory. 478 479If there were any parsing errors, `Parse` will return `false`, and 480`Parser::err` contains a human readable error string with a line number 481etc, which you should present to the creator of that file. 482 483After each JSON file, the `Parser::fbb` member variable is the 484`FlatBufferBuilder` that contains the binary buffer version of that 485file, that you can access as described above. 486 487`samples/sample_text.cpp` is a code sample showing the above operations. 488 489## Threading 490 491Reading a FlatBuffer does not touch any memory outside the original buffer, 492and is entirely read-only (all const), so is safe to access from multiple 493threads even without synchronisation primitives. 494 495Creating a FlatBuffer is not thread safe. All state related to building 496a FlatBuffer is contained in a FlatBufferBuilder instance, and no memory 497outside of it is touched. To make this thread safe, either do not 498share instances of FlatBufferBuilder between threads (recommended), or 499manually wrap it in synchronisation primites. There's no automatic way to 500accomplish this, by design, as we feel multithreaded construction 501of a single buffer will be rare, and synchronisation overhead would be costly. 502 503## Advanced union features 504 505The C++ implementation currently supports vectors of unions (i.e. you can 506declare a field as `[T]` where `T` is a union type instead of a table type). It 507also supports structs and strings in unions, besides tables. 508 509For an example of these features, see `tests/union_vector`, and 510`UnionVectorTest` in `test.cpp`. 511 512Since these features haven't been ported to other languages yet, if you 513choose to use them, you won't be able to use these buffers in other languages 514(`flatc` will refuse to compile a schema that uses these features). 515 516These features reduce the amount of "table wrapping" that was previously 517needed to use unions. 518 519To use scalars, simply wrap them in a struct. 520 521## Depth limit of nested objects and stack-overflow control 522The parser of Flatbuffers schema or json-files is kind of recursive parser. 523To avoid stack-overflow problem the parser has a built-in limiter of 524recursion depth. Number of nested declarations in a schema or number of 525nested json-objects is limited. By default, this depth limit set to `64`. 526It is possible to override this limit with `FLATBUFFERS_MAX_PARSING_DEPTH` 527definition. This definition can be helpful for testing purposes or embedded 528applications. For details see [build](@ref flatbuffers_guide_building) of 529CMake-based projects. 530 531## Dependence from C-locale {#flatbuffers_locale_cpp} 532The Flatbuffers [grammar](@ref flatbuffers grammar) uses ASCII 533character set for identifiers, alphanumeric literals, reserved words. 534 535Internal implementation of the Flatbuffers depends from functions which 536depend from C-locale: `strtod()` or `strtof()`, for example. 537The library expects the dot `.` symbol as the separator of an integer 538part from the fractional part of a float number. 539Another separator symbols (`,` for example) will break the compatibility 540and may lead to an error while parsing a Flatbuffers schema or a json file. 541 542The Standard C locale is a global resource, there is only one locale for 543the entire application. Some modern compilers and platforms have 544locale-independent or locale-narrow functions `strtof_l`, `strtod_l`, 545`strtoll_l`, `strtoull_l` to resolve this dependency. 546These functions use specified locale rather than the global or per-thread 547locale instead. They are part of POSIX-2008 but not part of the C/C++ 548standard library, therefore, may be missing on some platforms. 549 550The Flatbuffers library try to detect these functions at configuration and 551compile time: 552- `_MSC_VER >= 1900`: check MSVC2012 or higher for MSVC buid 553- `_XOPEN_SOURCE>=700`: check POSIX-2008 for GCC/Clang build 554- `check_cxx_symbol_exists(strtof_l stdlib.h)`: CMake check of `strtod_f` 555 556After detection, the definition `FLATBUFFERS_LOCALE_INDEPENDENT` will be 557set to `0` or `1`. 558 559It is possible to test the compatibility of the Flatbuffers library with 560a specific locale using the environment variable `FLATBUFFERS_TEST_LOCALE`: 561```sh 562>FLATBUFFERS_TEST_LOCALE="" ./flattests 563>FLATBUFFERS_TEST_LOCALE="ru_RU.CP1251" ./flattests 564``` 565 566<br> 567