Need help with daw_json_link?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

beached
214 Stars 9 Forks Other 2.1K Commits 9 Opened issues

Description

Fast JSON serialization and parsing in C++

Services available

!
?

Need anything else?

Contributors list

# 143,557
Shell
cpp17
constex...
Racket
1851 commits

logo image

DAW JSON Link v2

Build Status Macos

Build Status Ubuntu

Build Status Windows - MSVC

Content

Intro

Top

The DAW JSON Link library provides multiple ways to serialization/deserialization JSON documents in C++. The primary one is parsing of JSON directly to your C++ data structures. This allows the known structure of the document to be exploited for greater checking and performance. Alternatively, there is an event passing(SAX) interface that can parse to generic types(double, string, bool,...) or can use the same type restricted parsers as the static parser previously mentioned. A generic DOM(lazy) based parser is provided that can be iterate over the document structure too, again it can use the generic parsers or the type based restricted versions. One can mix the three modes of parsing to form more complicated systems. For serialization, the first static mapping method is required, there is no json value type in the library. The library is, also, non-intrusive into your data structures and does not require member's to be declared/defined within them. This allows keeping the mapping in a separate header file from the data structures themselves.

The library is using the BSL licensed

When the structure of the JSON document is known, parsing is like the following:

c++
MyThing thing = daw::json::from_json( json_string );
or for array documents, where the root of the document is an array, there is a helper method to make it easier and it can be parsed like the following:
c++
std::vector things = daw::json::from_json_array( json_string2 );
If the structure of the JSON document is unknown, one can construct a
json_value
that acts as a container and allows iteration and parsing on demand. It is a lazy parser and will only parse when asked to. The following is an example of opening a
json_value
from JSON data:
c++
json_value val = daw::json::json_value( json_string );

The

from_json
and
to_json
methods allow access most of the parsing needs.

The event based parser(SAX) can be called via

daw::json::json_event_parser
. It takes two arguments, a json document and an event handler. The event handler can opt into events by having the following members: * handleonvalue * handleonarraystart * handleonarrayend * handleonclassstart * handleonclassend * handleonnumber * handleonbool * handleonstring * handleonnull * handleonerror

Code Examples

Mapping of your classes to JSON documents is done by specializing the trait

daw::json::json_data_contract
. A class that is mapped does not need to be mapped again if it is a member of another mapped class. There are two parts to the trait
json_data_contract
, first is a type alias named
type
that maps the JSON members to our class's constructor. This gets around needing private access to the class, assuming that data we would serialize would also be needed to construct the class. For example:
c++
struct Thing {
  int a;
  int b;    
};
The construct for
Thing
requires 2 integers and if we had the following JSON:
json
{
  "a": 42,
  "b": 1234
}
We could do the mapping like the following:
c++
namespace daw::json {
  template<>
  struct json_data_contract {
    static constexpr char const a[] = "a";
    static constexpr char const b[] = "b";
    using type = json_member_list<
      json_number,
      json_number
    >;
  };
}
This says that the JSON class will have at least two members "a", and "b" that will be numbers that are integers. They will be passed to the constructor of
Thing
when
daw::json::from_json( json_doc );
is called, or that another class has a
json_class
member mapping. The above is the C++17 mapping method for the names, it works in future C++ versions too. But, in C++20 and later the names can be inline in the mapping e.g.
json_number
. The above is all that is needed for parsing JSON, for serializing a static member function is needed in the trait. Taking the previous example and extending it we could serialize
Thing
with: ```c++ namespace daw::json { template<> struct jsondatacontract { static constexpr char const a[] = "a"; static constexpr char const b[] = "b"; using type = jsonmemberlist< jsonnumber, jsonnumber >; };

static auto tojsondata( Thing const & v ) { return std::forwardastuple( v.a, v.b ); } } ``

 The ordering of the members returned as a tuple need to match the mapping in the type alias
type
.  This allows for passing the result of accessor methods too, if the data members are not public.
  • Note: The return type of
tojsondata` does not have to return a tuple of references to the existing object members, but can return calculated values too.

The parsers work by constructing each argument in place in the call to the classes constructor. The individual argument parsers can be tuned for the specified circumstances of the data(e.g. floating point and integral numbers). Then with our type trait defining the arguments needed to construct the C++ class and their order we are able to look at each member in the JSON. Now we construct the value with the result of each parser; similar to

T{ parse<0, json_string>( data ), parse<1, json_number>( data ), parse>( data )}
. For each member, the data stream will be moved forward until we find the member we need to parse, storing interested locations for later parsing. This process allows us to parse other classes as members too via the
json_class
mapping type. So that each mapping trait only has to deal with it's specific members and not their details. general parsing flow

Default mapping of types

Top

In unnamed contexts, such as the root value, array elements, some key value types, and variant element lists where the name would be

no_name
, one can use some native C++ data types instead of the the JSON mapping types. This includes, integer, floating point, bool, std::string, std::string_view, and previously mapped classes.

For example, to map an array of string's.

c++
template<>
struct daw::json::json_data_contract {
  using type = json_member_list>;
};

Installing/Using

Top

Including in cmake project

To use dawjsonlink in your cmake projects, adding the following should allow it to pull it in along with the dependencies:

cmake
include( FetchContent )
FetchContent_Declare(
        daw_json_link
        GIT_REPOSITORY https://github.com/beached/daw_json_link
                GIT_TAG release
)
FetchContent_MakeAvailable(daw_json_link)
Then in the targets that need it:
cmake
target_link_libraries( MyTarget daw::json_link )

Installing

On a system with bash, it is similar on other systems too, the following can install for the system

bash
git clone https://github.com/beached/daw_json_link
cd daw_json_link
mkdir build
cd build
cmake ..
cmake --install . 

Testing

The following will build and run the tests.

bash
git clone https://github.com/beached/daw_json_link
cd daw_json_link
mkdir build
cd build
cmake -DDAW_ENABLE_TESTING=On ..
cmake --build . 
ctest .
After the build there the individual examples can be tested too.
city_test_bin
requires the path to the cities JSON file.
bash
./tests/city_test_bin ../test_data/cities.json

Performance considerations

Top

The order of the members in the data structures should generally match that of the JSON data. The parser is faster if it doesn't have to back track for values. Optional values, when missing in the JSON data, can slow down the parsing too. If possible have them sent as null. The parser does not allocate. The parsed to data types may and this allows one to use custom allocators or a mix as their data structures will do the allocation. The defaults for arrays is to use the std::vector and if this isn't desirable, you must supply the type.

Benchmarks

chart desribing kostya benmark results

Escaping/Unescaping of member names

Top

The library, currently, does not unescape/escape member names when serializing, they are expected to be valid and unescaped. This may be a future optional addition, as it does have a cost.

Differences between C++17 and C++20

Top

There are slight differences between C++17 and C++20

Naming of JSON members

namespace daw::json {
  template<>
  struct json_data_contract {
    static constexpr char const member_name[] = "memberName";
    using type = json_member_list>;
  };
}

C++ 20 Naming of JSON members

When compiled within C++20 compiler, in addition to passing a

char const *
as in C++17, the member names can be specified as string literals directly. C++20 compiler support is still really early and here be dragons. There are known issues with g++9.x and it's only tested with g++10. Here be dragons
c++
namespace daw::json {
  template<>
  struct json_data_contract {
    using type = json_member_list>;
  };
}

Using mapped data types

Top

Once a data type has been mapped with a

json_data_contract
, the library provides methods to parse JSON to them
MyClass my_class = from_json( json_str );

Alternatively, if the input is trusted, the less checked version can be faster

c++
MyClass my_class = from_json( json_str );

JSON documents with array root's use the

from_json_array
function to parse
c++
std::vector my_data = from_json_array( json_str );
Alternatively, if the input is trusted, the less checked version can be faster
c++
std::vector my_data = from_json_array, NoCommentSkippingPolicyUnchecked>( json_str );

If you want to work from JSON array data you can get an iterator and use the std algorithms to Iterating over array's in JSON data can be done via the

json_array_iterator
c++
using iterator_t = json_array_iterator;
auto pos = std::find( iterator_t( json_str ), iterator_t( ), MyClass( ... ) );
Alternatively, if the input is trusted you can called the less checked version
c++
using iterator_t = daw::json::json_array_iterator_trusted;
auto pos = std::find( iterator_t( json_str ), iterator_t( ), MyClass( ... ) );

If you want to serialize to JSON

std::string my_json_data = to_json( MyClass{} );

Or serialize a collection of things

c++
std::vector arry = ...;
std::string my_json_data = to_json_array( arry );

Error Handling

Parsing call

Top

Error checking can be modified on a per parse basis. the fromjson/fromjson_array calls can be supplied a Parser Policy. The current policies are

  • NoCommentSkippingPolicyChecked
    - No comments allowed, checks enabled
  • NoCommentSkippingPolicyUnchecked
    - No comments allowed, assumes perfect JSON
  • CppCommentSkippingPolicyChecked
    - C++ style comments
    /* commment */
    and
    // comment until end of line
    , checks enabled
  • CppCommentSkippingPolicyUnchecked
    - C++ style comments
    /* commment */
    and
    // comment until end of line
    , assumes perfect JSON
  • HashCommentSkippingPolicyChecked
    - Hash style comments
    # comment until end of line
    , checks enabled
  • HashCommentSkippingPolicyUnchecked
    - Hash style comments
    # comment until end of line
    , assumes perfect JSON

The unchecked variants can sometimes provide a 5-15% performance increase, but at great risk when the data isn't perfect.

Global

Top

There are two possible ways of handling errors. The default is to throw a

daw::json::json_exception
on an error in the data.
json_exception
has a member function
std::string_view reason( ) const
akin to
std::exception
's
what( )
. Second, calling
std::terminate( );
on an error in data. If you want to disable exceptions in an environment that has them, you can defined
DAW_JSON_DONT_USE_EXCEPTIONS
to disable exception throwing by the library.

Deserializing/Parsing

Top

This can be accomplished by writing a function called jsondatacontract_for with a single argument that is your type. The library is only concerned with it's return value. For example:

#include 

struct TestClass { int i = 0; double d = 0.0; bool b = false; daw::string_view s{}; std::vector y{};

TestClass( int Int, double Double, bool Bool, daw::string_view S, std::vector Y ) : i( Int ) , d( Double ) , b( Bool ) , s( S ) , y( Y ) {} };

namespace daw::json { template<> struct json_data_contract { using type = json_member_list< json_number, json_number, json_bool, json_string, json_array >; }; }

int main( ) { std::string test_001_t_json_data = R"({ "i":5, "d":2.2e4, "b":false, "s":"hello world", "y":[1,2,3,4] })"; std::string json_array_data = R"([{ "i":5, "d":2.2e4, "b":false, "s":"hello world", "y":[1,2,3,4] },{ "i":4, "d":122e4, "b":true, "s":"goodbye world", "y":[4,3,1,4] }])";

TestClass test_class = daw::json::from_json( test_001_t_json_data ); std::vector arry_of_test_class = daw::json::from_json_array( test_001_t_json_data ); }

Both aggregate and user constructors are supported. The description provides the values needed to construct your type and the order. The order specified is the order they are placed into the constructor. There are customization points to provide a way of constructing your type too(TODO discuss customization points) A class like:

#include 

struct AggClass { int a{}; double b{}; };

namespace daw::json { template<> struct json_data_contract { using type = json_member_list< json_number, json_number >; }; }

Works too. Same but C++17 ```c++

include

struct AggClass { int a{}; double b{}; };

namespace daw::json { template<> struct jsondatacontract { static inline constexpr char const a[] = "a"; static inline constexpr char const b[] = "b"; using type = jsonmemberlist< jsonnumber, jsonnumber >; }; } ``

The class descriptions are recursive with their submembers. Using the previous
AggClass` one can include it as a member of another class

// See above for AggClass
struct MyClass {
  AggClass other;
  std::string_view some_name;
};

namespace daw::json { template<> struct json_data_contract { using type = json_member_list< json_class, json_string >; }; }

The above maps a class MyClass that has another class that is described AggClass. Also, you can see that the member names of the C++ class do not have to match that of the mapped JSON names and that strings can use

std::string_view
as the result type. This is an important performance enhancement if you can guarantee the buffer containing the JSON file will exist as long as the class does.

Iterating over JSON arrays. The input iterator

daw::json::json_array_iterator
allows one to iterator over the array of JSON elements. It is technically an input iterator but can be stored and reused like a forward iterator. It does not return a reference but a value. ```c++

include

struct AggClass { int a{}; double b{}; };

namespace daw::json { template<> struct jsondatacontract { using type = jsonmemberlist< jsonnumber<"a", int>, jsonnumber<"b"> >; }; }

int main( ) { std::string jsonarraydata = R"([ {"a":5,"b":2.2}, {"a":5,"b":3.14}, {"a":5,"b":0.122e44}, {"a":5334,"b":34342.2} ])"; using iteratort = daw::json::jsonarrayiterator; auto pos = std::findif( iteratort( jsonarraydata ), iteratort( ), { return element.b > 1000.0; } ); if( pos == iterator_t( ) ) { std::cout << "Not found\n"; } else { std::cout << "Found\n"; } } ```

Member Paths

Parsing can begin at a specific member. An optional member path to

from_json_array
,
from_json_array_unchecked
,
from_json_array
, or
from_json_array_unchecked
can be specified. The format is a dot separated list of member names and optionally an array index such as
member0.member1
or
member0[5].member1
.

Comments

Comments are supported when the parser policy for them is used. Currently there are two forms of comment policies. C++ style

//
and
/* */
. Comments can be placed anywhere there is whitespace allowed
  • Hash style

    { # This is a comment
    "a" #this is also a comment
      : "a's value"
    }
    
  • C++ style

    { // This is a comment
    "a" /*this is also a comment*/: "a's value"
    }
    
    To change the parser policy, you add another argument to
    from_json
    and call like
    from_json( json_data )

Serialization

Top

To enable serialization one must create an additional function in your specialization of

json_data_contract
called
to_json_data( Thing const & );
It will provide a mapping from your type to the arguments provided in the class description. To serialize to a JSON string, one calls
to_json( my_thing );
where value is a registered type or one of the fundamental types like string, bool, and numbers. The result of
to_json_data( Thing const & )
is a tuple who's elements match order in jsondatacontract's type alias
type
. Using the example above lets add that
#include 
#include 

struct AggClass { int a{}; double b{}; };

namespace daw::json { template<> struct json_data_contract { using type = json_member_list< json_number, json_number >;

static inline auto to_json_data( AggClass const &amp; value ) {
  return std::forward_as_tuple( value.a, value.b );
}

}; } //... AggData value = //...; std::string test_001_t_json_data = to_json( value );

// or std::vector values = //...; std::string json_array_data = to_json_array( values );

Alternatively there is an optional

iostreams
interface. In you types
json_data_constract
add a type alias named
opt_into_iostreams
the type it aliases doesn't matter, and include
daw_json_iostream.h
. For example
```c++

include

include

include

struct AggClass { int a{}; double b{}; };

namespace daw::json { template<> struct jsondatacontract { using optintoiostreams = void; using type = jsonmemberlist< jsonnumber<"a", int>, jsonnumber<"b"> >;

static inline auto to_json_data( AggClass const & value ) {
  return std::forward_as_tuple( value.a, value.b );
}

}; } //... AggData value = //...; std::cout << value << '\n';

// or std::vector values = //...; std::cout << values << '\n'; ``` A working example can be found at dawjsoniostream_test.cpp

Build configuration points

There are a few defines that affect how JSON Link operates *

DAW_JSON_DONT_USE_EXCEPTIONS
- Controls if exceptions are allowed. If they are not, an
std::terminate()
on errors will occur *
DAW_ALLOW_SSE42
- Allow experimental SSE3 mode *
DAW_JSON_NO_CONST_EXPR
- This can be used to allow classes without move/copy special members to be constructed from JSON data prior to C++ 20. This mode does not work in a constant expression prior to C++20 when this flag is no longer needed.

Requirements

Top
  • C++ 17 compiler
  • GCC(8/9)/Clang(7/8/9/10) have been tested.
  • MSVC 19.21 has been tested.

For building tests

  • git
  • cmake

Contact

Darrell Wright [email protected]

Limitations

  • When parsing classes, the first member with a mapped name will be used. If you want to parse a class that can have more than one of any member by name, either parse as a
    json_value
    see or as a
    json_key_value
    that is mapped to a
    std::multimap
    or a
    std::vector
    with a pair of key type(
    string
    ) and value type(s). Cookbook Key Values demonstrates these methods. If a
    json_key_value
    is used and the mapped data type does not support duplicate keys, it will insert for each key. This may result in the last item being the value reflected after serializing. If the duplicate member is the tag type in a
    json_tagged_variant
    , it is undefined what the behaviour for parsing is.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.