Plain-C token node produced by IoLexer as it scans source text. A token carries its spelling (name/length), an IoTokenType classification, a lineNumber/charNumber for diagnostics, an optional error string, and a forward-link `nextToken` that the lexer uses to splice tokens into a singly-linked chain parallel to the per-frame List tokenStream. Tokens are owned by the lexer and freed via IoToken_free; they are not IoObjects and are not GC-managed — the parser in IoMessage_parser.c consumes them one at a time and translates each into an IoMessage.
Returns true for token types that can begin a message chain: identifiers, mono- and tri-quoted strings, and number literals. IoMessage_parser uses this as the guard for IoMessage_parseName and IoMessage_parseNext so that punctuation tokens never become message names.
Attaches a diagnostic message to the token. The lexer sets this when it detects an unterminated quote, an unmatched bracket, or a stray character — the parser later reads errorToken back via the lexer to build the "compile error" Exception.
Releases the name and error buffers (both io_malloced) and the token itself. Does not recurse into nextToken — the lexer's tokenStream List iterates and frees each element explicitly via IoLexer_clear.
Strict string comparison of the token's spelling against the given C literal. The empty-vs-nonempty guard avoids a strcmp on a synthesized empty-name token produced for unnamed groups like `[...]`.
Convenience form that copies a NUL-terminated C string. Used for synthesized tokens (e.g. the ";" terminator, the "squareBrackets" / "curlyBrackets" group-name tokens) rather than raw source spans.
Copies a non-NUL-terminated span of len bytes into the token's owned name buffer, reallocating as needed and NUL-terminating. This is the hot path called by IoLexer_grabTokenType_ once a span has been matched.
Allocates a zeroed IoToken with name unset and charNumber sentinel -1. Callers populate fields via IoToken_name_length_ / IoToken_type_ as the lexer grabs a matched span.
Wires `self` to `nextToken` in the lexer's forward chain, freeing any previously attached successor. The self-identity guard fires on a lexer bug and aborts the process rather than silently looping.
Prints the token's spelling quoted with single quotes for debug output. Iterates name byte-by-byte rather than using printf("%s") so embedded NULs are rendered literally. Called by IoToken_print and by IoLexer_printTokens.
Rewrites the token's spelling by wrapping it in double quotes. Used when coercing an identifier into a string literal for generated assign messages (see IoMessage_opShuffle's setSlot rewriting).
Maps an IoTokenType enum to a human-readable name for debugging and for the Io-visible Compiler tokensForString result. Kept in sync with the IoTokenType definition in IoToken.h.