SMOL-V: like Vulkan/Khronos SPIR-V, but smaller.
SMOL-V encodes Vulkan/Khronos SPIR-V format programs into a form that is smoller, and is more compressible. Normally no changes to the programs are done; they decode into exactly same program as what was encoded. Optionally, debug information can be removed too.
SPIR-V is a very verbose format, several times larger than same programs expressed in other shader formats (e.g. DX11 bytecode, GLSL, DX9 bytecode etc.). The SSA-form with ever increasing IDs is not very appreciated by regular data compressors either. SMOL-V does several things to improve this:
A somewhat similar utility is spirv-remap from glslang.
See this blog post for more information about how I did SMOL-V.
source/smolv.cppto your C++ project build. It might require C++11 or somesuch. There are Github Actions set up for this project, that build on Windows (VS2017), macOS (Xcode 11.1) and Linux (Ubuntu 16 / gcc 5.4).
smolv::Decodeis the basic functionality. See smolv.h.
Other functions are for development/statistics purposes, to figure out frequencies and distributions of the instructions.
There's a test + compression benchmarking suite in
testing/testmain.cpp, using that needs adding other files under testing/external to the build too (3rd party code: glslang remapper, Zstd, LZ4, miniz).
Code itself: Public Domain or MIT, pick whichever works better for you.
There is 3rd party code under the testing framework (
testing/external); it is not required for using SMOL-V. Most of that code (glslang, LZ4, Zstd, sokol_time.h) is BSD or zlib-licensed, and taken from github repositories of the respective projects. miniz is public domain.
There are SPIR-V binary shader dumps under
tests/spirv-dumpsfor compression testing; these are not required for using SMOL-V. Not sure how to appropriately "license" them (but hey they are kinda useless by themselves out of context), so I'll go with this: "Binary shader dumps under 'tests' folder are only to be used for SMOL-V testing". Details on them:
tests/spirv-dumps/dota2- some shaders from DOTA2, Copyright Valve Corporation, all rights reserved.
tests/spirv-dumps/shadertoy- several most popular shaders from Shadertoy, converted to Vulkan SPIR-V via glslang. Copyrights by their individual authors (filename matches last component of shadertoy URL).
tests/spirv-dumps/talos- some shaders from The Talos Principle, Copyright (c) 2002-2016 Croteam All rights reserved.
tests/spirv-dumps/unity- various Unity shaders, produced through a HLSL -> DX11 bytecode -> HLSLcc -> glslang toolchain.
As of 2020 May 25 29, results on 372 shaders (under
Compressed with : Raw 5188.9KB 100.0% Remapper 5089.0KB 98.1% SmolV 1934.2KB 37.3%
Compressed with zlib: Raw 1301.3KB 25.1% Remapper 1230.5KB 23.7% SmolV 696.6KB 13.4%
Compressed with LZ4 HC: Raw 1448.7KB 27.9% Remapper 1303.9KB 25.1% SmolV 711.3KB 13.7%
Compressed with Zstandard: Raw 983.4KB 19.0% Remapper 870.8KB 16.8% SmolV 541.3KB 10.4%
Compressed with Zstandard 20: Raw 649.6KB 12.5% Remapper 599.1KB 11.5% SmolV 419.7KB 8.1%
Decoding these 372 shaders from SMOL-V back into SPIR-V takes 10.2ms (VS2017, x64 Release, AMD ThreadRipper 1950X 3.4GHz, one thread).