Tolerant YAML

Front matter on agents, skills and doc pages, GitHub workflow files, helm output: the platform reads all of it with one reader of its own, MeshWeaver.Yaml (src/MeshWeaver.Yaml). It has no third-party dependency and one rule:

YAML syntax, JSON semantics. The text is parsed into a System.Text.Json JsonNode tree and then bound with System.Text.Json under the same tolerant options the platform uses for its JSON node files.

The previous third-party reader was strict in every direction at once: an unknown key, a casing mismatch, a number where a string was declared, or a : inside an unquoted value threw. In MarkdownFileParser a throw sent the whole block to a regex rescue that recovers only six single-line fields, so the node imported with its description and every other key silently missing (MarkdownContent.UnboundFrontMatter, #4319, now names what a typed file lost).

The surface

The one failure type is YamlParseException (a FormatException) carrying Line and Column, both 1-based. A tree that parses but does not bind to T throws System.Text.Json's JsonException, exactly as the same JSON would.

From YAML to a JSON tree

This is the YAML 1.2 core schema: yes, no, on, off are strings, and a date such as 2026-01-01 is a string that binds to DateTime/DateTimeOffset like its JSON equivalent.

A tag written before a line break applies to the node on the next line before it is resolved (name: !!str then 007 on the next line is the string 007). In a block scalar, whitespace past the content indentation is content (|- over x and a line of three spaces is "x\n ").

Covered syntax: block mappings and sequences (indented or not under a key, the compact - key: v form, nested - - a), flow collections over several lines with trailing commas, plain scalars folded over several lines, single-quoted ('') and double-quoted scalars with every YAML escape (\n, \t, \xXX, \uXXXX, \UXXXXXXXX, an escaped line break), literal | and folded > block scalars with the -/+ chomping indicators and an explicit indentation digit (chomping keeps only the line breaks that were actually there, so a block at the end of the text without a final break gains none), full-line and trailing # comments, --- and ... document markers, %YAML and %TAG directives (a %TAG handle decides what a tag means), verbatim tags (!<tag:yaml.org,2002:str>), explicit ? key / : value entries in block and flow collections, properties, aliases and one-line flow collections as implicit keys (&k key: v, *a : v, [a, b]: v — a key that is not a scalar becomes its JSON text), anchors &a and aliases *a (an alias is a deep copy), and merge keys <<: *a and <<: [*a, *b] (explicit keys win, then the first source that names a key; every source must be a mapping). A CRLF line ending and a leading byte-order mark are normalised away.

Tags. A core tag — !!str, !!int, !!float, !!bool, !!null, !!map, !!seq, spelled !!x, verbatim or through a %TAG handle — resolves the node AS that type, and text of another type is refused with the tag's position (!!int "12" is 12, !!int true is an error). The non-specific ! makes a plain scalar a string. An application tag (!Ref) carries nothing a JSON tree can use: the node resolves exactly as it would untagged (!Ref 123 is the number 123).

Limits — hostile input fails fast, with a position

The reader parses files that arrive by import and sync, so input it cannot bound is refused as a YamlParseException, never a hang or a crash:

• Nesting deeper than 64 levels (flow or block, properties-only lines included). The reader recurses per level; without the limit, deep nesting is an uncatchable stack overflow. 64 is also System.Text.Json's default MaxDepth, so nothing deeper could be bound anyway.
• Alias and merge-key expansion beyond 100,000 copied nodes per text — every node AND every null slot a copy allocates. Every alias is a deep copy, so anchors that alias earlier anchors grow exponentially ("billion laughs").
• A NUL character anywhere in the text (YAML does not allow one in a stream).
• An escape that is not a Unicode scalar value (\U00110000, a lone \uD800). A \u high surrogate followed by a \u low surrogate, the JSON spelling of one character, is read as that pair.

Every loop in the reader consumes the text it inspects (a whitespace run inside a quoted scalar is read once, not re-scanned from each of its characters), and a duplicate key is replaced in place, so reading time is linear in the text. TolerantYamlTest.GeneratedHostileInputs_… throws several hundred generated hostile texts at both modes — long whitespace runs in every scalar style, wide and deep collections, alias and merge fans, million-character lines, tabs, indicator soup, one-character mutants of a valid document — and requires each to parse or refuse with YamlParseException within its own time budget.

The tolerances

In the reader. Two places where malformed YAML has exactly one sensible reading:

• A : inside a plain value is text. description: new deal: fund pilot reads as the string new deal: fund pilot. A mapping cannot start in the middle of a value line, so the only other reading is a refusal — and the refusal is what used to lose the whole block.
• A duplicate key keeps its last value, as System.Text.Json does with a duplicate JSON property.

Everything else that has no single reading — an unterminated quote or flow collection, an EMPTY flow entry ([a,,b], {,}; a trailing comma, an empty value {k: } and an empty key {: v} are legal), an unknown alias, a tab in indentation, an entry indented between two levels, text after a closing quote — throws YamlParseException.

In the binding (CreateSerializerOptions, the default for Deserialize<T>):

A caller may pass its own JsonSerializerOptions; MarkdownFileParser writes front matter with PascalCase keys in declaration order, which keeps every existing mirrored file byte-identical.

Strict mode — for files strict readers also read

YamlParseOptions.StrictSpec refuses what the grammar refuses where the tolerant reader reads on: the two reader tolerances above, a plain scalar (or plain key) starting with @, a backtick, %, ,, ] or }, a lone - in a flow collection, a leading empty line of a block scalar indented past its content, a # comment not separated from the content by a space, a block sequence on the line of its key or properties, two anchors or two tags on one node, properties on an alias, an implicit flow key spanning lines, continuation lines of a flow collection or quoted scalar not indented past their block, and invalid directives (a directive without a --- document, a malformed or repeated %YAML, a tag handle no %TAG declares). The test-suite section below lists every case where the tolerant reader accepts what strict mode refuses. The platform's own reads are tolerant; strict mode is for guards over files that OTHER readers also consume — SkillFrontMatterGuard (.claude/skills, read by the coding harnesses) and DocumentationFrontMatterGuard (the doc tree, read by GitHub and editors) hold their trees to it, so the platform's tolerance never turns a file another reader would drop into a green guard.

Conformance — the official YAML test suite

The reader is checked against the official YAML test suite (yaml/yaml-test-suite, release data-2022-01-17, MIT licence), vendored under test/MeshWeaver.Hosting.Test/YamlTestSuite/ and run by YamlTestSuiteConformanceTest in both modes. Of its 402 cases, strict mode produces exactly the expected JSON for 274, refuses all 85 error cases it is expected to refuse, and parses the 27 valid cases that have no JSON form (complex keys) without an error — 386 of 402. The remaining 16 are the reasoned exceptions below. The test is a two-way ratchet: a listed case that starts passing fails the test until its line is removed, so a reviewer finding can always be answered against the suite.

Known unsupported (strict mode):

• Tabs — 6CA3, DK95/00, DK95/01, Q5MG, Y79Y/000, Y79Y/003–Y79Y/009. The reader refuses any tab in the leading whitespace of a content line and accepts tabs between tokens; YAML 1.2 allows a tab after the indentation in some positions and forbids it in others, and that finer rule is not implemented.
• End of input without a final line break — JEF9/02, L24T/01. The suite's reference output treats the end of the text as a line break that clip and keep chomping keep; this reader, like libyaml (Ruby's Psych) and YamlDotNet, adds no break that is not in the text.
• Duplicate keys — 2JQS, X38W. Strict mode refuses a repeated key, because a JSON object cannot hold two; the suite treats key uniqueness as a check above the syntax.

Error cases the tolerant reader deliberately accepts (44):

On every VALID case the tolerant reader gives the same answer as strict mode.

Writing

Serialize writes block style with two-space indentation, a sequence under a key at the key's column (Tags: then - a), and the most readable scalar form that reads back as the same value:

• plain when the text reads back as the same string — not when it would resolve to null, a boolean, a number or a time, including the YAML 1.1 forms a 1.1 reader would convert (yes, off, y, 0xFF_FF, -0xFF, 0b101, 017, 1_000, 1,000, 1:20, 1.2.3, .Inf, anything starting like a date such as 2026-01-01) — checked against an independent YAML 1.1 reader (Ruby's Psych);
• single-quoted ('Foo: Bar', 'it''s') when plain would not survive;
• double-quoted with escapes only for control characters;
• a literal block (|, |-) for a multi-line string, with an indentation digit when its first line starts with a space.

Every string written reads back identically through the tolerant AND the strict reader (TolerantYamlTest.Serialize_StringsRoundTripExactly).

How it was checked

Besides the unit tests (test/MeshWeaver.Hosting.Test/TolerantYamlTest.cs, MarkdownFrontMatterTolerantYamlTest.cs), the reader was run differentially against the previous third-party reader over every .yml/.yaml file and every front-matter block in this repository and the plugins repository when it was introduced — 28,405 files. Every file the previous reader accepted produced the same tree (scalar for scalar), every file re-serialized and re-read under strict mode came back equal, and the 70 files the previous reader refused were all the : -inside-a-value shape the tolerant reader now reads.

Related

• Declarative export and import — what the markdown parsers bind
• Authoring Documentation — the front matter a doc page carries