Skip to main content
Be Bitwise
C++Modern C++ Patterns

Capstone: Build a JSON Parser

Time to build a real JSON parser, top to bottom, using the modern C++ features you've collected: std::variant for the value type, std::string_view for zero-copy tokenisation, recursive descent for the grammar. The whole capstone leans on type erasure, move semantics, and the rest of the modern toolkit working together.

JSON Value Type

Start with the types that model a JSON value:

cpp
#include <variant>
#include <string>
#include <vector>
#include <map>
#include <memory>
#include <stdexcept>
#include <iostream>
#include <sstream>
#include <charconv>
struct JsonValue;
using JsonNull = std::monostate;
using JsonArray = std::vector<JsonValue>;
using JsonObject = std::map<std::string, JsonValue>;
struct JsonValue {
using Variant = std::variant<
JsonNull,
bool,
double,
std::string,
std::shared_ptr<JsonArray>,
std::shared_ptr<JsonObject>
>;
Variant data;
JsonValue() : data(JsonNull{}) {}
JsonValue(std::nullptr_t) : data(JsonNull{}) {}
JsonValue(bool b) : data(b) {}
JsonValue(double d) : data(d) {}
JsonValue(int i) : data(static_cast<double>(i)) {}
JsonValue(std::string s) : data(std::move(s)) {}
JsonValue(const char* s) : data(std::string(s)) {}
JsonValue(JsonArray arr)
: data(std::make_shared<JsonArray>(std::move(arr))) {}
JsonValue(JsonObject obj)
: data(std::make_shared<JsonObject>(std::move(obj))) {}
bool is_null() const { return std::holds_alternative<JsonNull>(data); }
bool is_bool() const { return std::holds_alternative<bool>(data); }
bool is_number() const { return std::holds_alternative<double>(data); }
bool is_string() const { return std::holds_alternative<std::string>(data); }
bool is_array() const { return std::holds_alternative<std::shared_ptr<JsonArray>>(data); }
bool is_object() const { return std::holds_alternative<std::shared_ptr<JsonObject>>(data); }
};
Key Concept

Why std::shared_ptr for arrays and objects? Not because of size: a std::vector<JsonValue> is three pointers no matter what it holds, so storing one inline wouldn't make JsonValue infinitely large. The problem is that JsonValue is still incomplete at that point, and the standard only promises that vector, list and forward_list accept an incomplete element type (since C++17). std::map makes no such promise. The pointer sidesteps the question, at a price: copying a JsonValue copies the pointer, so two copies share, and can both modify, the same array or object.

The Tokenizer

The tokenizer chops the input string into a stream of tokens:

cpp
enum class TokenType {
LeftBrace, RightBrace, // { }
LeftBracket, RightBracket, // [ ]
Colon, Comma, // : ,
String, Number,
True, False, Null,
End
};
struct Token {
TokenType type;
std::string_view text;
size_t line;
size_t column;
};
class Tokenizer {
std::string_view source_;
size_t pos_ = 0;
size_t line_ = 1;
size_t col_ = 1;
size_t tok_line_ = 1; // where the current token starts
size_t tok_col_ = 1;
char peek() const {
return pos_ < source_.size() ? source_[pos_] : '\0';
}
char advance() {
char c = source_[pos_++];
if (c == '\n') { line_++; col_ = 1; } else { col_++; }
return c;
}
void skip_whitespace() {
while (pos_ < source_.size() &&
(peek() == ' ' || peek() == '\t' ||
peek() == '\n' || peek() == '\r')) {
advance();
}
}
Token make_token(TokenType type, size_t start) {
return {type, source_.substr(start, pos_ - start), tok_line_, tok_col_};
}
public:
explicit Tokenizer(std::string_view source) : source_(source) {}
Token next() {
skip_whitespace();
if (pos_ >= source_.size()) {
return {TokenType::End, "", line_, col_};
}
size_t start = pos_;
tok_line_ = line_;
tok_col_ = col_;
char c = advance();
switch (c) {
case '{': return make_token(TokenType::LeftBrace, start);
case '}': return make_token(TokenType::RightBrace, start);
case '[': return make_token(TokenType::LeftBracket, start);
case ']': return make_token(TokenType::RightBracket, start);
case ':': return make_token(TokenType::Colon, start);
case ',': return make_token(TokenType::Comma, start);
case '"': return tokenize_string(start);
default:
if (c == '-' || (c >= '0' && c <= '9')) {
return tokenize_number(start);
}
if (c == 't' || c == 'f' || c == 'n') {
return tokenize_keyword(start);
}
throw std::runtime_error("Unexpected character");
}
}
Token tokenize_string(size_t start) {
while (peek() != '"' && pos_ < source_.size()) {
if (peek() == '\\') advance(); // Skip escaped char
advance();
}
if (pos_ >= source_.size()) {
throw std::runtime_error("Unterminated string");
}
advance(); // closing "
return make_token(TokenType::String, start);
}
Token tokenize_number(size_t start) {
while (pos_ < source_.size() &&
((peek() >= '0' && peek() <= '9') ||
peek() == '.' || peek() == 'e' ||
peek() == 'E' || peek() == '+' || peek() == '-')) {
advance();
}
return make_token(TokenType::Number, start);
}
Token tokenize_keyword(size_t start) {
while (pos_ < source_.size() && peek() >= 'a' && peek() <= 'z') {
advance();
}
auto text = source_.substr(start, pos_ - start);
if (text == "true") return {TokenType::True, text, tok_line_, tok_col_};
if (text == "false") return {TokenType::False, text, tok_line_, tok_col_};
if (text == "null") return {TokenType::Null, text, tok_line_, tok_col_};
throw std::runtime_error("Unknown keyword");
}
};

The Parser

A recursive descent parser turns each grammar rule into a function. One non-terminal, one method:

cpp
class Parser {
Tokenizer tokenizer_;
Token current_;
void advance() { current_ = tokenizer_.next(); }
void expect(TokenType type) {
if (current_.type != type) {
throw std::runtime_error("Unexpected token");
}
advance();
}
public:
explicit Parser(std::string_view source)
: tokenizer_(source) {
advance();
}
JsonValue parse() {
auto value = parse_value();
if (current_.type != TokenType::End) {
throw std::runtime_error("Unexpected content after value");
}
return value;
}
JsonValue parse_value() {
switch (current_.type) {
case TokenType::LeftBrace: return parse_object();
case TokenType::LeftBracket: return parse_array();
case TokenType::String: return parse_string();
case TokenType::Number: return parse_number();
case TokenType::True: advance(); return JsonValue(true);
case TokenType::False: advance(); return JsonValue(false);
case TokenType::Null: advance(); return JsonValue(nullptr);
default:
throw std::runtime_error("Expected value");
}
}
JsonValue parse_object() {
expect(TokenType::LeftBrace);
JsonObject obj;
if (current_.type != TokenType::RightBrace) {
do {
if (current_.type != TokenType::String) {
throw std::runtime_error("Expected string key");
}
std::string key = extract_string_value();
expect(TokenType::Colon);
obj[key] = parse_value();
} while (current_.type == TokenType::Comma && (advance(), true));
}
expect(TokenType::RightBrace);
return JsonValue(std::move(obj));
}
JsonValue parse_array() {
expect(TokenType::LeftBracket);
JsonArray arr;
if (current_.type != TokenType::RightBracket) {
do {
arr.push_back(parse_value());
} while (current_.type == TokenType::Comma && (advance(), true));
}
expect(TokenType::RightBracket);
return JsonValue(std::move(arr));
}
JsonValue parse_string() {
std::string value = extract_string_value();
return JsonValue(std::move(value));
}
std::string extract_string_value() {
auto text = current_.text;
advance();
// Remove quotes and handle escapes
std::string result;
for (size_t i = 1; i < text.size() - 1; i++) {
if (text[i] == '\\' && i + 1 < text.size() - 1) {
i++;
switch (text[i]) {
case '"': result += '"'; break;
case '\\': result += '\\'; break;
case 'n': result += '\n'; break;
case 't': result += '\t'; break;
case 'r': result += '\r'; break;
default: result += text[i];
}
} else {
result += text[i];
}
}
return result;
}
JsonValue parse_number() {
auto text = current_.text;
advance();
double value = 0;
auto [ptr, ec] = std::from_chars(text.data(), text.data() + text.size(), value);
// ptr short of the end means from_chars only read a prefix, as in "1-2"
if (ec != std::errc{} || ptr != text.data() + text.size()) {
throw std::runtime_error("Invalid number");
}
return JsonValue(value);
}
};

Trace it on a small document, {"name": "Ada", "tags": [1, true]}. The tokenizer hands over LeftBrace String Colon String Comma String Colon LeftBracket Number Comma True RightBracket RightBrace End, one token per advance(). The parser's calls nest exactly the way the document does, and each box returns its JsonValue to the box above it:

Two caveats about parse_number. The tokenizer grabs any run of digits, signs, dots and es, and from_chars is more forgiving than JSON: it takes 01 and 1., which the JSON grammar rejects. Checking that ptr reached the end of the token at least stops 1-2 being read as 1. And std::from_chars for double needs a recent standard library. libstdc++ has had it since GCC 11, but libc++ (the one Apple's Xcode ships) only gained it in LLVM 20, so an older Mac toolchain refuses to compile this call. std::strtod on a null-terminated copy of the token works everywhere, but it reads the decimal point from the current locale.

The call stack is the parse tree. When parse_array is running, its callers are still waiting further up the C++ stack, which is also why a hostile document nested deeply enough can overflow it.

Pretty Printer

cpp
void pretty_print(const JsonValue& val, int indent = 0) {
std::string pad(indent, ' ');
std::visit([&](const auto& v) {
using T = std::decay_t<decltype(v)>;
if constexpr (std::is_same_v<T, JsonNull>) {
std::cout << "null";
} else if constexpr (std::is_same_v<T, bool>) {
std::cout << (v ? "true" : "false");
} else if constexpr (std::is_same_v<T, double>) {
std::cout << v;
} else if constexpr (std::is_same_v<T, std::string>) {
std::cout << "\"" << v << "\"";
} else if constexpr (std::is_same_v<T, std::shared_ptr<JsonArray>>) {
std::cout << "[\n";
for (size_t i = 0; i < v->size(); i++) {
std::cout << pad << " ";
pretty_print((*v)[i], indent + 2);
if (i + 1 < v->size()) std::cout << ",";
std::cout << "\n";
}
std::cout << pad << "]";
} else if constexpr (std::is_same_v<T, std::shared_ptr<JsonObject>>) {
std::cout << "{\n";
size_t i = 0;
for (const auto& [key, value] : *v) {
std::cout << pad << " \"" << key << "\": ";
pretty_print(value, indent + 2);
if (++i < v->size()) std::cout << ",";
std::cout << "\n";
}
std::cout << pad << "}";
}
}, val.data);
}

Testing

cpp
int main() {
std::string json = R"({
"name": "Be Bitwise",
"version": 2.0,
"features": ["c", "cpp", "rust"],
"config": {
"debug": false,
"max_threads": 8,
"description": null
}
})";
try {
Parser parser(json);
JsonValue value = parser.parse();
pretty_print(value);
std::cout << "\n";
} catch (const std::exception& e) {
std::cerr << "Parse error: " << e.what() << "\n";
}
return 0;
}

The runner on this site compiles C++14, so the parser above (with std::variant, std::string_view and structured bindings) won't build there. This cut-down checker walks the same grammar without building any values, and adds two things the parser lacks: errors that say where they happened, and a cap on nesting depth so a hostile [[[[… can't run the stack out. Predict which of the six inputs pass before you run it.

Try itRaise MAX_DEPTH to 100000000 and grow the deep-nesting input to a few million brackets to see what the limit was protecting against. Then add a test of your own, such as an object key that is not a string.
C++ (GCC 9.2.0)

What You Built

This JSON parser demonstrates:

  • std::variant for type-safe discriminated unions
  • std::visit with if constexpr for type-based dispatch
  • std::string_view for zero-copy tokenisation
  • std::from_chars for locale-independent number parsing
  • Recursive descent parsing - the same technique used in production parsers
  • Move semantics throughout for efficiency

Want to push it further? Add std::from_chars paths for the full number grammar, give JsonValue an operator[] for ergonomic access, write a serialiser that round-trips back to JSON, or tackle Unicode escape sequences (\uXXXX) including surrogate pairs.