Asuro/doc/planning/json.md

JSON Library Plan — core.fmt.json.pas

General Rules

• Always keep doc/src/core/fmt/core.fmt.json.md up-to-date with any changes to the implementation.
• Add any generic utility code that isn't specific to the JSON unit to the appropriate place (e.g. doubleToStr in core.strings.pas, number-related helpers in core.util.pas) and update the corresponding documentation accordingly.

Spec Compliance

Full RFC 8259 compliance:

• All 6 value types: null, true, false, number, string, object, array
• Numbers: full JSON number grammar — integer, fractional, and exponent parts, stored as Double
• Strings: full escape support including \uXXXX (codepoints U+0000-U+007F emitted as bytes; higher codepoints emit ? in v1 since the OS is ASCII-only)
• Whitespace: skip $20, $09, $0A, $0D between tokens

Data Model — Unified with JSON_ERROR

TJSONType = (
    JSON_NULL,
    JSON_BOOL,
    JSON_NUMBER,
    JSON_STRING,
    JSON_ARRAY,
    JSON_OBJECT,
    JSON_ERROR
);

PJSONValue = ^TJSONValue;
TJSONValue = record
    Kind : TJSONType;
    case TJSONType of
        JSON_BOOL:   (boolVal   : boolean);
        JSON_NUMBER: (numberVal : Double);
        JSON_STRING: (stringVal : pchar);
        JSON_ARRAY:  (arrayVal  : PDList);
        JSON_OBJECT: (objectVal : PHashMap);
        JSON_ERROR:  (errPos    : uint32;
                      errMsg    : pchar);
end;

parse() always returns PJSONValue. The caller checks result^.Kind = JSON_ERROR for failure. The error carries the byte offset and a human-readable message.

Public API

{ --- Parsing --- }
function  parse(buf : pchar; len : uint32) : PJSONValue;

{ --- Path accessors --- }
function  getByPath(root : PJSONValue; path : pchar) : PJSONValue;
function  getStringByPath(root : PJSONValue; path : pchar; default : pchar) : pchar;
function  getNumberByPath(root : PJSONValue; path : pchar; default : Double) : Double;
function  getBoolByPath(root : PJSONValue; path : pchar; default : boolean) : boolean;
function  getObjectByPath(root : PJSONValue; path : pchar) : PJSONValue;
function  getArrayByPath(root : PJSONValue; path : pchar) : PJSONValue;

{ --- Direct accessors --- }
function  getType(val : PJSONValue) : TJSONType;
function  getBool(val : PJSONValue) : boolean;
function  getNumber(val : PJSONValue) : Double;
function  getString(val : PJSONValue) : pchar;
function  getArraySize(val : PJSONValue) : uint32;
function  getArrayItem(val : PJSONValue; idx : uint32) : PJSONValue;
function  getObjectItem(val : PJSONValue; key : pchar) : PJSONValue;

{ --- Builders --- }
function  newNull : PJSONValue;
function  newBool(b : boolean) : PJSONValue;
function  newNumber(n : Double) : PJSONValue;
function  newString(s : pchar) : PJSONValue;
function  newArray : PJSONValue;
function  newObject : PJSONValue;

{ --- Mutation --- }
procedure arrayAppend(arr : PJSONValue; item : PJSONValue);
procedure objectSet(obj : PJSONValue; key : pchar; item : PJSONValue);

{ --- Serialization --- }
function  stringify(val : PJSONValue) : pchar;

{ --- Cleanup --- }
procedure freeValue(val : PJSONValue);

{ --- Tests --- }
procedure UnitTest;

Parser Internals

Recursive-descent with a parser state record:

TJSONParser = record
    buf    : pchar;
    len    : uint32;
    pos    : uint32;
    hasErr : boolean;
    errPos : uint32;
    errMsg : pchar;
end;

Internal function	Role
skipWS	Skip $20, $09, $0A, $0D
peek	Return current char without advancing
advance	Return current char and increment pos
expect	Consume expected char or call setError
setError	Record pos + message, set hasErr flag
parseValue	Dispatch on first non-WS char
parseString	"..." with full \ escape + \uXXXX handling
parseNumber	Full JSON number grammar -> Double
parseArray	[ value (, value)* ]
parseObject	{ string : value (, ...) }
parseLiteral	true / false / null

Every sub-parser checks hasErr on entry and bails immediately if set — errors propagate cleanly to the top without exceptions.

Number Parsing (RFC 8259 compliant)

number = [ "-" ] int [ frac ] [ exp ]
int    = "0" / ( digit1-9 *DIGIT )
frac   = "." 1*DIGIT
exp    = ( "e" / "E" ) [ "+" / "-" ] 1*DIGIT

All arithmetic in Double via x87 FPU. Leading zeros rejected per spec (01 is invalid).

Number Serialization (internal doubleToStr)

doubleToStr is generic string utility code and belongs in core.strings.pas (not the JSON unit).

• If no fractional part -> emit as integer (via int64 -> digit extraction)
• Otherwise -> emit integer part, ., then fractional digits (up to 15 significant digits), trim trailing zeros
• Very large/small numbers -> scientific notation (1.23e+10)
• Negative -> leading -
• Zero -> 0

String Escapes (full spec)

Parse:

Sequence	Byte
\"	$22
\\	$5C
\/	$2F
\b	$08
\f	$0C
\n	$0A
\r	$0D
\t	$09
\uXXXX	Emit byte if U+0000-U+007F, ? otherwise

Stringify: reverse mapping — emit \n, \t, etc. for control chars; \\ and \" for those literals.

Path Accessor Internals

Path syntax: "key.nested[0].deep" — dot-separated keys with [N] for array indices.

getByPath walks left to right:

1. Accumulate chars until ., [, or end-of-string -> object key lookup via hashmap.get
2. On [ -> parse digits until ] -> array index lookup via DL_Get
3. Continue until path exhausted or lookup returns nil

freeValue — Recursive Cleanup

Kind	Action
JSON_STRING	kfree(stringVal)
JSON_ARRAY	Iterate DList, freeValue each PJSONValue, then DL_Free
JSON_OBJECT	hashmap.forEach -> freeValue each value, then free the map
JSON_ERROR	kfree(errMsg)
All	kfree the TJSONValue record itself

Test Suite (UnitTest)

Follows the same pattern as core.strings.pas — nested Assert procedure, pass/fail counters, PrintSummary at end, log tag 'JSON'.

Test categories:

Category	What's tested
Literals	null, true, false
Numbers	Integer, negative, fractional, exponent, negative exponent, leading-zero rejection
Strings	Simple, escapes (\n, \t, \\, \"), \uXXXX, empty string
Arrays	Empty [], nested, mixed types
Objects	Empty {}, nested, duplicate keys
Path accessors	Dot navigation, array indexing, combined, missing paths return default
Error handling	Unterminated string, trailing comma, missing colon, unexpected EOF, leading zeros
Builders	Construct values programmatically, verify with accessors
Stringify	Round-trip: parse -> stringify -> parse equivalence for each type
freeValue	Ensure no crashes on deeply nested structures (smoke test)

Dependencies

uses
    memory.heap,
    core.strings,
    core.ds.lists,
    core.ds.hashmap,
    core.util;

Files

src/core/fmt/core.fmt.json.pas     <- implementation + UnitTest
doc/src/core/fmt/core.fmt.json.md  <- documentation (keep up-to-date)