Need help with blobify?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

neobrain
216 Stars 12 Forks Boost Software License 1.0 48 Commits 1 Opened issues

Description

C++17 library for all your binary de-/serialization needs

Services available

!
?

Need anything else?

Contributors list

# 29,315
Shell
c-plus-...
3ds
cpp17
42 commits
# 163,222
airdrop
wlan
steam
glsl
5 commits

blobify

blobify is a header-only C++17 library to handle binary de-/serialization in your project. Given a user-defined C++ struct, blobify can encode its data into a compact binary stream and, conversely, load a structure from its binary encoding. Unlike other serialization frameworks, blobify requires no boilerplate code and instead infers the serialized layout from the structure definition alone. Customizations to the default behavior are enabled by specifying properties using an embedded domain specific language.

Common applications of blobify include parsing file formats, network communication, and device drivers.

What is it?

Consider bitmap loading: You'd typically represent bitmap header elements using C++ structs and enums, such as:

enum class Compression : uint32_t {
    None      = 0,
    RLE8      = 1,
    RLE4      = 2,
    // ...
};

struct BMPHeader { uint16_t signature;

uint32_t size_bytes; uint32_t reserved; uint32_t data_offset;

// (other members omitted for simplicity)

Compression compression; };

Parsing a

.bmp
file into a such a
BMPHeader
value is as simple as it gets with blobify:
std::ifstream file("/path/to/bitmap.bmp");
blob::istream_storage storage { file };

auto header = blob::load(storage);

The first two lines open a file and prepare it for input to blobify, and the last line performs the actual read. Not only does this provide a convenient interface, but blobify takes care of the subtle details for you too, such as removing compiler-inserted padding bytes between struct members.

More elaborate usage examples can be found in the examples directory.

Customization via properties

Reliable binary deserialization often requires extensive data validation: Your code might expect version fields to have a certain value, and

enum
-types should often only have one of the enumerated values. Some data might require postprocessing steps, such as fields encoded in different endianness or different units than the one used at runtime. Blobify allow the user to specify such properties and validation requirements using a built-in DSL.

Consider the

signature
member of
BMPHeader
, which identifies a bitmap file as such. If it's not 0x4d42 (an encoding of the string "BM"), a validation error should be triggered. In blobify this can be implemented by defining a
properties
function:
constexpr auto properties(blob::tag) {
    blob::properties_t props { };

// Throw exception if the loaded signature is not 0x4d42
props.member().expected_value = uint16_t { 0x4d42 };

// Throw exception if the loaded compression value is none of None, RLE4, or RLE8
props.member().validate_enum = true;

return props;

}

Crucially, the

properties
function must be
constexpr
and must reside in the same namespace as the definition of
BMPHeader
. If no such function is defined, blobify will apply a set of default properties that describe a compact binary encoding in native endianness without any validation.

Error handling

Coarse error handling can be done by catching blobify's base exception type

blob::exception
:
try {
    auto header = blob::load(storage);
} catch (blob::exception&) {
    std::cerr << "Failed to load BMPHeader" << std::endl;
}

However, especially in data validation you might want to dynamically handle errors depending on the specific error condition. Hence, blobify's exception hierarchy allows to distinguish between different error causes (end-of-file/unexpected value/invalid enum/...) as well as the specific data member that triggered the error.

try {
    auto header = blob::load(storage);
} catch (blob::storage_exhausted_exception&) {
    std::cerr << "Unexpected early end-of-file" << std::endl;
} catch (blob::unexpected_value_exception_for<:signature>&) {
    std::cerr << "Invalid BMP signature" << std::endl;
} catch (blob::invalid_enum_value_exception_for<:compression>&) {
    std::cerr << "Invalid compression value" << std::endl;
}

Usage

Setting up blobify is easiest if your project is built on CMake. Just put a copy of blobify in your project tree (e.g. using a Git submodule) and

add_subdirectory
it from your main CMakeLists.txt. This will register the
blobify
target that you can
target_link_libraries
against.

If your project does not use CMake, setting up blobify is still easy: Just point your compiler to blobify's main include directory as well as the magicget and magicenum header paths and you should be good to go.

Credits

blobify's API significantly benefits from the marvelous work done by Antony Polukhin and Daniil Goncharov on their respective libraries PFR (aka magic_get) and magic_enum.

Support

blobify is an offspring of Mikage, a 3DS emulator for Android devices. The library was created out of the critical need in game console emulation to detect invalid deserialization inputs reliably with minimal boilerplate, since failure to do so may cause unimplemented features or subtle emulation bugs to go unnoticed.

If you want to show your appreciation for blobify, the best way of doing so is to support me on Patreon.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.