2.2.6. Debugging Support

It can be tough to track down the specifics of what a parser is doing (or not doing) because often there’s no directly observable effect. However, Spicy comes with support that helps with debugging.

The simplest way to learn more about what’s going on is to add a hooks with print statements to the grammar. That’s rather disruptive though, and hence there are also special %debug unit hooks which only get compiled into the resulting code if spicy-driver [Missing] is run with both the -d and -g options (the former leads to generating code with debugging support, and the latter enables the hooks):

 export type test = unit {

    a: bytes &length=4 %debug {
         print self.a;
         }

    b: bytes &length=6;

    on b %debug { print self.b; }
};

The second form of debugging support uses HILTI’s debugging streams to instrument the generated parsers for logging activity as they proceed parsing input. For this to be compiled in, spicy-driver must be run with -d. The output can then be enabled by setting the environment variable HILTI_DEBUG to the name of an output stream to select the desired information; see below for the list. HILTI records the output in hlt-debug.log. Here’s an example using the stream spicy, which logs unit fields as they are parsed:

> echo "GET /index.html HTTP/1.0" | HILTI_DEBUG=spicy spicy-driver -d request.spicy
GET /index.html 1.0
> cat hlt-debug.log
00000001 [spicy/main-thread]           RequestLine
00000002 [spicy/main-thread]             method = GET
00000003 [spicy/main-thread]             __anon1 =
00000004 [spicy/main-thread]             uri = /index.html
00000005 [spicy/main-thread]             __anon2 =
00000006 [spicy/main-thread]             version
00000007 [spicy/main-thread]               __anon4 = HTTP/
00000008 [spicy/main-thread]               number = 1.0
00000009 [spicy/main-thread]             __anon3 = \x0a

The following debugging streams are currently available:

spicy

Logs unit fields and variables as they are parsed (or more generally, as they get values assigned). This is often the most helpful output as this shows rather concisely what the parser is doing, and in particular how far it gets in cases where it doesn’t parse something correctly.

spicy-verbose

Logs many internals about the parsing process, including the grammar rules currently being parsed; the current input position; lexer tokens; and look-ahead symbols which might be pending; and more.

This stream is primarily intended for debugging the Spicy compiler itself.

hilti-trace

This is a HILTI-defined debugging level that records every HILTI instruction being executed, along with arguments and return values.

This stream is primarily intended for debugging the Spicy compiler itself.

hilti-flow

This is a HILTI-defined debugging level recording all function calls and returns.

This stream is primarily intended for debugging the Spicy compiler itself, though it may also be helpful to understand the internal control flow when writing a grammar.

Note that multiple streams can be enabled by separating them with colons. Furthermore, when using hilti-build [Missing] with its -d options, the HILTI_DEBUG works in the same way with the generated executable.

Also note that generating code with debugging instrumentation (-d) can be quite a bit slower.