DAW JSON Link v2

Content

• Intro
  • Default Mapping of Types
• API Documentation - Member mapping classes and methods
• Cookbook Get cooking and putting it all together
  • Arrays
  • Classes
  • Class from Array
  • Dates
  • Enums
  • Graphs
  • Key Values - Map and Dictionary like things
  • Numbers
  • Optional/Nullable Values
  • Parsing Individual Members
  • Parser Policies
  • Strings
  • Unknown JSON and Delayed Parsing - Browsing the JSON Document and delaying of parsing of specified members
  • Variant
  • Automatic Code Generation
• Intro
• Installing/Using
• Performance considerations
  • Benchmarks
• Escaping/Unescaping of member names
• Differences between C++17 and C++20
  • C++ 17 Naming of members
  • C++ 20 Naming of members
• Using data types
• Error Handling
  • Parsing call
  • Global
• Deserializing/Parsing
  • Member Paths
• Serialization
• Build Configuration Points
• Requirements
  • For building tests
• Limitations

Intro

Top

The DAW JSON Link library provides multiple ways to serialization/deserialization JSON documents in C++. The primary one is parsing of JSON directly to your C++ data structures. This allows the known structure of the document to be exploited for greater checking and performance. Alternatively, there is an event passing(SAX) interface that can parse to generic types(double, string, bool,...) or can use the same type restricted parsers as the static parser previously mentioned. A generic DOM(lazy) based parser is provided that can be iterate over the document structure too, again it can use the generic parsers or the type based restricted versions. One can mix the three modes of parsing to form more complicated systems. For serialization, the first static mapping method is required, there is no json value type in the library. The library is, also, non-intrusive into your data structures and does not require member's to be declared/defined within them. This allows keeping the mapping in a separate header file from the data structures themselves.

The library is using the BSL licensed

When the structure of the JSON document is known, parsing is like the following:

c++
MyThing thing = daw::json::from_json( json_string );

c++
std::vector things = daw::json::from_json_array( json_string2 );

json_value

json_value

c++
json_value val = daw::json::json_value( json_string );

The

from_json

to_json

The event based parser(SAX) can be called via

daw::json::json_event_parser

Code Examples

• The Cookbook section has precanned tasks and working code examples
• Tests provide another source of working code samples.
• Some video walkthroughs
  • Making a config parser
  • I Like BigInt's
• Links to other examples
  
  • Parsing a Config File
  • Parsing BigInt/Multiprecision Numbers
    
• Small samples below

Mapping of your classes to JSON documents is done by specializing the trait

daw::json::json_data_contract

json_data_contract

type

c++
struct Thing {
  int a;
  int b;    
};

Thing

json
{
  "a": 42,
  "b": 1234
}

c++
namespace daw::json {
  template<>
  struct json_data_contract {
    static constexpr char const a[] = "a";
    static constexpr char const b[] = "b";
    using type = json_member_list<
      json_number,
      json_number
    >;
  };
}

Thing

daw::json::from_json( json_doc );

json_class

json_number

Thing

static auto tojsondata( Thing const & v ) { return std::forwardastuple( v.a, v.b ); } } ``

 The ordering of the members returned as a tuple need to match the mapping in the type alias

.  This allows for passing the result of accessor methods too, if the data members are not public.

Note: The return type of

The parsers work by constructing each argument in place in the call to the classes constructor. The individual argument parsers can be tuned for the specified circumstances of the data(e.g. floating point and integral numbers). Then with our type trait defining the arguments needed to construct the C++ class and their order we are able to look at each member in the JSON. Now we construct the value with the result of each parser; similar to

T{ parse<0, json_string>( data ), parse<1, json_number>( data ), parse>( data )}

json_class

Default mapping of types

Top

In unnamed contexts, such as the root value, array elements, some key value types, and variant element lists where the name would be

no_name

For example, to map an array of string's.

c++
template<>
struct daw::json::json_data_contract {
  using type = json_member_list>;
};

Installing/Using

Top

Including in cmake project

To use dawjsonlink in your cmake projects, adding the following should allow it to pull it in along with the dependencies:

cmake
include( FetchContent )
FetchContent_Declare(
        daw_json_link
        GIT_REPOSITORY https://github.com/beached/daw_json_link
                GIT_TAG release
)
FetchContent_MakeAvailable(daw_json_link)

cmake
target_link_libraries( MyTarget daw::json_link )

Installing

On a system with bash, it is similar on other systems too, the following can install for the system

bash
git clone https://github.com/beached/daw_json_link
cd daw_json_link
mkdir build
cd build
cmake ..
cmake --install . 

Testing

The following will build and run the tests.

bash
git clone https://github.com/beached/daw_json_link
cd daw_json_link
mkdir build
cd build
cmake -DDAW_ENABLE_TESTING=On ..
cmake --build . 
ctest .

city_test_bin

bash
./tests/city_test_bin ../test_data/cities.json

Performance considerations

Top

The order of the members in the data structures should generally match that of the JSON data. The parser is faster if it doesn't have to back track for values. Optional values, when missing in the JSON data, can slow down the parsing too. If possible have them sent as null. The parser does not allocate. The parsed to data types may and this allows one to use custom allocators or a mix as their data structures will do the allocation. The defaults for arrays is to use the std::vector and if this isn't desirable, you must supply the type.

Benchmarks

• Kostya results using test_dawjsonlink.cpp See Kostya Benchmarks for latest results.

• Parial NativeJson

Escaping/Unescaping of member names

Top

The library, currently, does not unescape/escape member names when serializing, they are expected to be valid and unescaped. This may be a future optional addition, as it does have a cost.

Differences between C++17 and C++20

Top

There are slight differences between C++17 and C++20

Naming of JSON members

namespace daw::json {
  template<>
  struct json_data_contract {
    static constexpr char const member_name[] = "memberName";
    using type = json_member_list>;
  };
}

C++ 20 Naming of JSON members

When compiled within C++20 compiler, in addition to passing a

char const *

c++
namespace daw::json {
  template<>
  struct json_data_contract {
    using type = json_member_list>;
  };
}

Using mapped data types

Top

Once a data type has been mapped with a

json_data_contract

MyClass my_class = from_json( json_str );

Alternatively, if the input is trusted, the less checked version can be faster

c++
MyClass my_class = from_json( json_str );

JSON documents with array root's use the

from_json_array

c++
std::vector my_data = from_json_array( json_str );

c++
std::vector my_data = from_json_array, NoCommentSkippingPolicyUnchecked>( json_str );

If you want to work from JSON array data you can get an iterator and use the std algorithms to Iterating over array's in JSON data can be done via the

json_array_iterator

c++
using iterator_t = json_array_iterator;
auto pos = std::find( iterator_t( json_str ), iterator_t( ), MyClass( ... ) );

c++
using iterator_t = daw::json::json_array_iterator_trusted;
auto pos = std::find( iterator_t( json_str ), iterator_t( ), MyClass( ... ) );

If you want to serialize to JSON

std::string my_json_data = to_json( MyClass{} );

Or serialize a collection of things

c++
std::vector arry = ...;
std::string my_json_data = to_json_array( arry );

Error Handling

Parsing call

Top

Error checking can be modified on a per parse basis. the fromjson/fromjson_array calls can be supplied a Parser Policy. The current policies are

• NoCommentSkippingPolicyChecked

  - No comments allowed, checks enabled

• NoCommentSkippingPolicyUnchecked

  - No comments allowed, assumes perfect JSON

• CppCommentSkippingPolicyChecked

  - C++ style comments

  /* commment */

  and

  // comment until end of line

  , checks enabled

• CppCommentSkippingPolicyUnchecked

  - C++ style comments

  /* commment */

  and

  // comment until end of line

  , assumes perfect JSON

• HashCommentSkippingPolicyChecked

  - Hash style comments

  # comment until end of line

  , checks enabled

• HashCommentSkippingPolicyUnchecked

  - Hash style comments

  # comment until end of line

  , assumes perfect JSON

The unchecked variants can sometimes provide a 5-15% performance increase, but at great risk when the data isn't perfect.

Global

Top

There are two possible ways of handling errors. The default is to throw a

daw::json::json_exception

json_exception

std::string_view reason( ) const

std::exception

what( )

std::terminate( );

DAW_JSON_DONT_USE_EXCEPTIONS

Deserializing/Parsing

Top

This can be accomplished by writing a function called jsondatacontract_for with a single argument that is your type. The library is only concerned with it's return value. For example:

#include 

struct TestClass {
  int i = 0;
  double d = 0.0;
  bool b = false;
  daw::string_view s{};
  std::vector y{};
  TestClass( int Int, double Double, bool Bool, daw::string_view S, std::vector Y ) 
    : i( Int )
    , d( Double )
    , b( Bool )
    , s( S )
    , y( Y ) {}
};
namespace daw::json {
  template<>
  struct json_data_contract {
  using type = json_member_list<
    json_number,
    json_number,
    json_bool,
    json_string,
    json_array
   >;
  };
}
int main( ) {
  std::string test_001_t_json_data = R"({
    "i":5,
    "d":2.2e4,
    "b":false,
    "s":"hello world",
    "y":[1,2,3,4] 
    })";
  std::string json_array_data = R"([{
    "i":5,
    "d":2.2e4,
    "b":false,
    "s":"hello world",
    "y":[1,2,3,4] 
    },{
    "i":4,
    "d":122e4,
    "b":true,
    "s":"goodbye world",
    "y":[4,3,1,4] 
    }])";
  TestClass test_class = daw::json::from_json( test_001_t_json_data );
  std::vector arry_of_test_class = daw::json::from_json_array( test_001_t_json_data );
}

Both aggregate and user constructors are supported. The description provides the values needed to construct your type and the order. The order specified is the order they are placed into the constructor. There are customization points to provide a way of constructing your type too(TODO discuss customization points) A class like:

#include 

struct AggClass {
  int a{};
  double b{};
};
namespace daw::json {
  template<>
  struct json_data_contract {
    using type = json_member_list<
      json_number,
      json_number
    >;
  };
}

Works too. Same but C++17 ```c++

include

struct AggClass { int a{}; double b{}; };

namespace daw::json { template<> struct jsondatacontract { static inline constexpr char const a[] = "a"; static inline constexpr char const b[] = "b"; using type = jsonmemberlist< jsonnumber, jsonnumber >; }; } ``

The class descriptions are recursive with their submembers. Using the previous

// See above for AggClass
struct MyClass {
  AggClass other;
  std::string_view some_name;
};

namespace daw::json {
  template<>
  struct json_data_contract {
    using type = json_member_list<
      json_class,
      json_string
    >;
  };
}

The above maps a class MyClass that has another class that is described AggClass. Also, you can see that the member names of the C++ class do not have to match that of the mapped JSON names and that strings can use

std::string_view

Iterating over JSON arrays. The input iterator

daw::json::json_array_iterator

include

struct AggClass { int a{}; double b{}; };

namespace daw::json { template<> struct jsondatacontract { using type = jsonmemberlist< jsonnumber<"a", int>, jsonnumber<"b"> >; }; }

int main( ) { std::string jsonarraydata = R"([ {"a":5,"b":2.2}, {"a":5,"b":3.14}, {"a":5,"b":0.122e44}, {"a":5334,"b":34342.2} ])"; using iteratort = daw::json::jsonarrayiterator; auto pos = std::findif( iteratort( jsonarraydata ), iteratort( ), { return element.b > 1000.0; } ); if( pos == iterator_t( ) ) { std::cout << "Not found\n"; } else { std::cout << "Found\n"; } } ```

Member Paths

Parsing can begin at a specific member. An optional member path to

from_json_array

from_json_array_unchecked

from_json_array

from_json_array_unchecked

member0.member1

member0[5].member1

Comments

Comments are supported when the parser policy for them is used. Currently there are two forms of comment policies. C++ style

//

/* */

• Hash style

  { # This is a comment
  "a" #this is also a comment
    : "a's value"
  }
  

• C++ style

  { // This is a comment
  "a" /*this is also a comment*/: "a's value"
  }
  

  To change the parser policy, you add another argument to

  from_json

  and call like

  from_json( json_data )

Serialization

Top

To enable serialization one must create an additional function in your specialization of

json_data_contract

to_json_data( Thing const & );

to_json( my_thing );

to_json_data( Thing const & )

type

#include 
#include 

struct AggClass {
  int a{};
  double b{};
};
namespace daw::json {
  template<>
  struct json_data_contract {
    using type = json_member_list<
      json_number,
      json_number
    >;
static inline auto to_json_data( AggClass const &amp; value ) {
  return std::forward_as_tuple( value.a, value.b );
}
  };
}
//...
AggData value = //...;
std::string test_001_t_json_data = to_json( value );
// or
std::vector values = //...;
std::string json_array_data = to_json_array( values );

Alternatively there is an optional

iostreams

json_data_constract

opt_into_iostreams

daw_json_iostream.h

include

include

include

struct AggClass { int a{}; double b{}; };

namespace daw::json { template<> struct jsondatacontract { using optintoiostreams = void; using type = jsonmemberlist< jsonnumber<"a", int>, jsonnumber<"b"> >;

static inline auto to_json_data( AggClass const & value ) {
  return std::forward_as_tuple( value.a, value.b );
}

}; } //... AggData value = //...; std::cout << value << '\n';

// or std::vector values = //...; std::cout << values << '\n'; ``` A working example can be found at dawjsoniostream_test.cpp

Build configuration points

There are a few defines that affect how JSON Link operates *

DAW_JSON_DONT_USE_EXCEPTIONS

std::terminate()

DAW_ALLOW_SSE42

DAW_JSON_NO_CONST_EXPR

Requirements

Top

• C++ 17 compiler
• GCC(8/9)/Clang(7/8/9/10) have been tested.
• MSVC 19.21 has been tested.

For building tests

• git
• cmake

Contact

Darrell Wright [email protected]

Limitations

• When parsing classes, the first member with a mapped name will be used. If you want to parse a class that can have more than one of any member by name, either parse as a

  json_value

  see or as a

  json_key_value

  that is mapped to a

  std::multimap

  or a

  std::vector

  with a pair of key type(

  string

  ) and value type(s). Cookbook Key Values demonstrates these methods. If a

  json_key_value

  is used and the mapped data type does not support duplicate keys, it will insert for each key. This may result in the last item being the value reflected after serializing. If the duplicate member is the tag type in a

  json_tagged_variant

  , it is undefined what the behaviour for parsing is.