~jonsterling/forester-devel

DRAFT: Improve diagnostics, error handling, etc v1 SUPERSEDED

Kento Okura <kentookura@gmail.com>
a month ago 
Kento Okura: 1
 DRAFT: Improve diagnostics, error handling, etc

 17 files changed, 1233 insertions(+), 225 deletions(-)
Kento Okura <kentookura@gmail.com>
a month ago 
Hi Nick, thanks for your feedback.

I agree on much of what you said. I'd like to say that the branch is in
an unfinished state and I think what I am working towards will
ultimately address your concerns.

- You are right in that the code no longer performs error recovery. I am
  just getting started with the incremental API of menhir, and
  ultimately, we don't want to emit errors via Asai, but rather make the
  syntax tree capable of carrying error nodes. This also means, as you
  point out, that we are moving towards a CST, rather than an AST.

  As far as I understood your implementation, it attempts to fix
  mistaken input by balancing braces, but ultimately, we want the parser
  to be able to recognise the most likely place where an error occurs
  and to insert a special error syntax node there, rather than emitting
  an error. 

  I have found that when using the main branch, some errors are just not
  understandable at all, which is why I am using a stack of positions to
  keep track and report the locations of unclosed delimiters.

- I have spent the last week deep-diving into all the resources than
  Aleksey has generously written. I agree that rust-analyzer is amazing!
  We should learn from the way they do things. In particular, I am aware
  of rowan and am playing around with an ocaml implementation of those
  ideas here:

  https://github.com/kentookura/orochi

- I am interested in using tree-sitter to implement editor
  functionality, but I don't think that incorporating it into the
  forester code base aligns with our goals of keeping dependencies trim.
  If we can use menhir to achieve our goals in the near future, we
  should stick to it, keeping our eyes on the long term plan of
  eventually removing menhir alltogether.

- I don't think that using the `Grammar.messages` file adds
  *complexity*, rather than tediousness. The codebase does not really
  get more complicated, the API just takes a number and returns a
  string, which we can wrap in an Asai diagnostic or something. Of
  course, to make quality error messages we need to deal with that
  rather large `messages` file, but my hope is that we can get a good
  return on investment by adding good messages to the most common parse
  errors. I also don't think that this ties forester to menhir any
  thighter than it currently is, and in any case, it is simple to
  remove. I just added it to see what it's like. If it adds to developer
  churn, I'll gladly rip it back out.

Best, Kento
Export patchset (mbox)
How do I use this?

Copy & paste the following snippet into your terminal to import this patchset into git: 

curl -s https://lists.sr.ht/~jonsterling/forester-devel/patches/55562/mbox | git am -3
Learn more about email & git
Reply to thread View this thread in the archives

[PATCH] DRAFT: Improve diagnostics, error handling, etc Export this patch

Kento Okura <kentookura@gmail.com>
a month ago 
This patch includes various improvements to the reported diagnostics. It
subsumes https://lists.sr.ht/~jonsterling/forester-devel/patches/55546

- FEATURE! The reported range upon encountering an unmatched delimiter,
  or a delimiter that is never closed now includes everthing including
  the last unclosed delimiter:
  ```
  ￫ error[Reporter.Message.Parse_error]
  ￭ ./trees/comment.tree
   1 | \title{title
     ^ This delimiter is never closed
   2 | \p{
   3 |   Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
   4 |   tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
   5 |   quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
   6 |   consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
   7 |   cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
   8 |   proident, sunt in culpa qui officia deserunt mollit anim id est laborum
   9 | }
  10 |
     ^ Did you forget to close it?
  ```
  (The message here should be tweaked, it reads a bit weird)

- feature: use Sc.pp_path instead of Trie.pp_path:

  Instead of
  ```
  | \p/foo{}
  ^ path p.foo could not be resolved
  ```
  we see
  ```
  | \p/foo{}
  ^ path p/foo could not be resolved
  ```

- feature: Add Grammar.messages file for declarative error messages. The
  messages are currently mostly placeholders. In order to write good
  messages, we need to understand what the error state is (the right
  side of the arrow). I need to study the documentation for this more.

- feature: better error reporting during the lexing phase, and a fatal
  bug was fixed: failure to close the verbatim block is now handled
  gracefully, and a helpful message is provided.

- idea: Add Comments and errors to the parse tree. Doesn't get parsed
  yet, haven't mucked around with the lexer and parser.

  The idea here is that the parse tree should be fully lossless: It
  should be possible to perfectly recreate the source file, whitespace
  included, from the parse tree.

  (so my point here is moot: https://todo.sr.ht/~jonsterling/forester/81)

  Futhermore, the result of parsing should be a datastructure that is
  easy to traverse and update (zipper). I am playing around with the
  ideas presented in this video

  https://www.youtube.com/watch?v=n5LDjWIAByM

  in this repository:

  https://github.com/kentookura/orochi

  I am unsure if we should ever produce lexing errors. Instead, the
  input that failed to lex/parse should be contained in an error node.

  We never want encounter error nodes during batch compilation, but
  for IDE features, we obviously want to work with malformed input.
  Depending on the context in which the parser is running, we can
  immediately fail when encountering such a node, or continue. This is
  what the `I.AboutToReduce` state is for.
---
 lib/compiler/Code.ml            |    2 +
 lib/compiler/Expand.ml          |   10 +-
 lib/compiler/Grammar.messages   | 1009 +++++++++++++++++++++++++++++++
 lib/compiler/Grammar.mly        |    1 +
 lib/compiler/Lexer.mll          |   60 +-
 lib/compiler/Parse.ml           |  282 ++++-----
 lib/compiler/Parse.mli          |    5 -
 lib/compiler/Resolver.ml        |    4 +
 lib/compiler/dune               |    6 +
 lib/forest/Forest.ml            |    7 +-
 lib/forest/Legacy_xml_client.ml |    9 +-
 lib/forest/dune                 |    1 +
 lib/frontend/Forester.ml        |   18 +-
 lib/frontend/Import_graph.ml    |    2 +-
 test/dune                       |    4 +-
 test/parse.expected             |   13 -
 test/parse.ml                   |   25 -
 17 files changed, 1233 insertions(+), 225 deletions(-)
 create mode 100644 lib/compiler/Grammar.messages
 delete mode 100644 lib/compiler/Parse.mli
 delete mode 100644 test/parse.expected
 delete mode 100644 test/parse.ml

diff --git a/lib/compiler/Code.ml b/lib/compiler/Code.ml
index af3e6e0..934fa09 100644
--- a/lib/compiler/Code.ml
+++ b/lib/compiler/Code.ml
@@ -37,6 +37,8 @@ type node =
  | Decl_xmlns of string * string
  | Alloc of Trie.path
  | Namespace of Trie.path * t
  | Comment of string
  | Error of string
[@@deriving show, repr]

and t = node Range.located list
diff --git a/lib/compiler/Expand.ml b/lib/compiler/Expand.ml
index 5f9caca..dd1ac36 100644
--- a/lib/compiler/Expand.ml
+++ b/lib/compiler/Expand.ml
@@ -172,6 +172,10 @@ let rec expand : Code.t -> Syn.t = function
    let symbol = Symbol.named path in
    Sc.include_singleton path @@ Term [Range.locate_opt loc (Syn.Sym symbol)];
    expand rest
  | { value = Comment _; loc } :: rest ->
    expand rest
  | { value = Error _; loc } :: rest ->
    expand rest

and expand_method (key, body) =
  key, expand body
@@ -209,7 +213,7 @@ and expand_ident loc path =
          ?loc
          Resolution_error
          "path %a could not be resolved"
          Trie.pp_path
          Sc.pp_path
          path
      | Some (cs, rest) ->
        let rest = match rest with "" -> [] | _ -> [Range.{ value = Syn.Text rest; loc }] in
@@ -220,7 +224,7 @@ and expand_ident loc path =
      ?loc
      Resolution_error
      "path %a could not be resolved"
      Trie.pp_path
      Sc.pp_path
      path
  | Some (Term x, ()), _ ->
    let relocate Range.{ value; _ } = Range.{ value; loc } in
@@ -230,7 +234,7 @@ and expand_ident loc path =
      ?loc
      Resolution_error
      "path %a resolved to xmlns:%s=\"%s\" instead of term"
      Trie.pp_path
      Sc.pp_path
      path
      xmlns
      prefix
diff --git a/lib/compiler/Grammar.messages b/lib/compiler/Grammar.messages
new file mode 100644
index 0000000..0ab89d2
--- /dev/null
+++ b/lib/compiler/Grammar.messages
@@ -0,0 +1,1009 @@
main: TEXT
##
## Ends in an error in state: 0.
##
## main' -> . main [ # ]
##
## The known suffix of the stack is as follows:
##
##

parse error 11. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: SUBTREE XML_ELT_IDENT
##
## Ends in an error in state: 4.
##
## head_node -> SUBTREE . option(squares(wstext)) LBRACE ws_list(locate(head_node)) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## SUBTREE
##

can't put an XML identifier here.

main: SUBTREE LSQUARE XML_ELT_IDENT
##
## Ends in an error in state: 5.
##
## option(squares(wstext)) -> LSQUARE . wstext RSQUARE [ LBRACE ]
##
## The known suffix of the stack is as follows:
## LSQUARE
##

usage: \subtree[addr-XXXX]{

main: SUBTREE LSQUARE TEXT RBRACE
##
## Ends in an error in state: 8.
##
## option(squares(wstext)) -> LSQUARE wstext . RSQUARE [ LBRACE ]
##
## The known suffix of the stack is as follows:
## LSQUARE wstext
##
## WARNING: This example involves spurious reductions.
## This implies that, although the LR(1) items shown above provide an
## accurate view of the past (what has been recognized so far), they
## may provide an INCOMPLETE view of the future (what was expected next).
## In state 10, spurious reduction of production list(ws_or_text) ->
## In state 11, spurious reduction of production list(ws_or_text) -> ws_or_text list(ws_or_text)
## In state 12, spurious reduction of production wstext -> list(ws_or_text)
##

parse error 55. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: DECL_XMLNS LBRACE TEXT XML_ELT_IDENT
##
## Ends in an error in state: 10.
##
## list(ws_or_text) -> ws_or_text . list(ws_or_text) [ RSQUARE RBRACE ]
##
## The known suffix of the stack is as follows:
## ws_or_text
##

parse error 67. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: SUBTREE LSQUARE RSQUARE XML_ELT_IDENT
##
## Ends in an error in state: 13.
##
## head_node -> SUBTREE option(squares(wstext)) . LBRACE ws_list(locate(head_node)) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## SUBTREE option(squares(wstext))
##

expected content

main: SUBTREE LBRACE TEXT
##
## Ends in an error in state: 14.
##
## head_node -> SUBTREE option(squares(wstext)) LBRACE . ws_list(locate(head_node)) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## SUBTREE option(squares(wstext)) LBRACE
##

parse error 91. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: SCOPE XML_ELT_IDENT
##
## Ends in an error in state: 15.
##
## head_node -> SCOPE . arg [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## SCOPE
##

parse error 103. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: SCOPE LBRACE RSQUARE
##
## Ends in an error in state: 17.
##
## arg -> LBRACE . list(locate(textual_node)) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## LBRACE
##

parse error 115. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: PUT XML_ELT_IDENT
##
## Ends in an error in state: 20.
##
## head_node -> PUT . ident arg [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## PUT
##

parse error 127. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: PUT IDENT XML_ELT_IDENT
##
## Ends in an error in state: 22.
##
## head_node -> PUT ident . arg [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## PUT ident
##

parse error 139. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: PATCH XML_ELT_IDENT
##
## Ends in an error in state: 24.
##
## head_node -> PATCH . LBRACE ws_list(locate(head_node)) RBRACE option(squares(bvar)) LBRACE ws_list(method_decl) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## PATCH
##

parse error 151. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: PATCH LBRACE TEXT
##
## Ends in an error in state: 25.
##
## head_node -> PATCH LBRACE . ws_list(locate(head_node)) RBRACE option(squares(bvar)) LBRACE ws_list(method_decl) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## PATCH LBRACE
##

parse error 163. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: OPEN XML_ELT_IDENT
##
## Ends in an error in state: 26.
##
## head_node -> OPEN . ident [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## OPEN
##

parse error 175. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: OBJECT XML_ELT_IDENT
##
## Ends in an error in state: 28.
##
## head_node -> OBJECT . option(squares(bvar)) LBRACE ws_list(method_decl) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## OBJECT
##

parse error 187. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: OBJECT LSQUARE XML_ELT_IDENT
##
## Ends in an error in state: 29.
##
## option(squares(bvar)) -> LSQUARE . bvar RSQUARE [ LBRACE ]
##
## The known suffix of the stack is as follows:
## LSQUARE
##

parse error 199. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: OBJECT LSQUARE TEXT XML_ELT_IDENT
##
## Ends in an error in state: 31.
##
## option(squares(bvar)) -> LSQUARE bvar . RSQUARE [ LBRACE ]
##
## The known suffix of the stack is as follows:
## LSQUARE bvar
##

parse error 211. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: OBJECT LSQUARE TEXT RSQUARE XML_ELT_IDENT
##
## Ends in an error in state: 33.
##
## head_node -> OBJECT option(squares(bvar)) . LBRACE ws_list(method_decl) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## OBJECT option(squares(bvar))
##

parse error 223. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: OBJECT LBRACE XML_ELT_IDENT
##
## Ends in an error in state: 34.
##
## head_node -> OBJECT option(squares(bvar)) LBRACE . ws_list(method_decl) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## OBJECT option(squares(bvar)) LBRACE
##

parse error 235. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: OBJECT LBRACE LSQUARE XML_ELT_IDENT
##
## Ends in an error in state: 36.
##
## method_decl -> LSQUARE . TEXT RSQUARE list(WHITESPACE) arg [ WHITESPACE RBRACE LSQUARE ]
##
## The known suffix of the stack is as follows:
## LSQUARE
##

parse error 247. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: OBJECT LBRACE LSQUARE TEXT XML_ELT_IDENT
##
## Ends in an error in state: 37.
##
## method_decl -> LSQUARE TEXT . RSQUARE list(WHITESPACE) arg [ WHITESPACE RBRACE LSQUARE ]
##
## The known suffix of the stack is as follows:
## LSQUARE TEXT
##

parse error 259. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: OBJECT LBRACE LSQUARE TEXT RSQUARE XML_ELT_IDENT
##
## Ends in an error in state: 38.
##
## method_decl -> LSQUARE TEXT RSQUARE . list(WHITESPACE) arg [ WHITESPACE RBRACE LSQUARE ]
##
## The known suffix of the stack is as follows:
## LSQUARE TEXT RSQUARE
##

parse error 271. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: OBJECT LBRACE LSQUARE TEXT RSQUARE WHITESPACE XML_ELT_IDENT
##
## Ends in an error in state: 39.
##
## list(WHITESPACE) -> WHITESPACE . list(WHITESPACE) [ VERBATIM LBRACE ]
##
## The known suffix of the stack is as follows:
## WHITESPACE
##

parse error 283. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: OBJECT LBRACE WHITESPACE XML_ELT_IDENT
##
## Ends in an error in state: 43.
##
## list(ws_or(method_decl)) -> ws_or(method_decl) . list(ws_or(method_decl)) [ RBRACE ]
##
## The known suffix of the stack is as follows:
## ws_or(method_decl)
##

parse error 295. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: NAMESPACE XML_ELT_IDENT
##
## Ends in an error in state: 49.
##
## head_node -> NAMESPACE . ident LBRACE ws_list(locate(head_node)) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## NAMESPACE
##

parse error 307. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: NAMESPACE IDENT XML_ELT_IDENT
##
## Ends in an error in state: 50.
##
## head_node -> NAMESPACE ident . LBRACE ws_list(locate(head_node)) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## NAMESPACE ident
##

parse error 319. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: NAMESPACE IDENT LBRACE TEXT
##
## Ends in an error in state: 51.
##
## head_node -> NAMESPACE ident LBRACE . ws_list(locate(head_node)) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## NAMESPACE ident LBRACE
##

parse error 331. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: LSQUARE RPAREN
##
## Ends in an error in state: 52.
##
## head_node -> LSQUARE . list(locate(textual_node)) RSQUARE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## LSQUARE
##

parse error 343. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: LPAREN RSQUARE
##
## Ends in an error in state: 53.
##
## head_node -> LPAREN . list(locate(textual_node)) RPAREN [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## LPAREN
##

parse error 355. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: LET XML_ELT_IDENT
##
## Ends in an error in state: 54.
##
## head_node -> LET . ident list(squares(bvar_with_strictness)) arg [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## LET
##

parse error 367. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: LET IDENT XML_ELT_IDENT
##
## Ends in an error in state: 55.
##
## head_node -> LET ident . list(squares(bvar_with_strictness)) arg [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## LET ident
##

parse error 379. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: FUN LSQUARE XML_ELT_IDENT
##
## Ends in an error in state: 56.
##
## list(squares(bvar_with_strictness)) -> LSQUARE . bvar_with_strictness RSQUARE list(squares(bvar_with_strictness)) [ VERBATIM LBRACE ]
##
## The known suffix of the stack is as follows:
## LSQUARE
##

parse error 391. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: FUN LSQUARE TEXT XML_ELT_IDENT
##
## Ends in an error in state: 58.
##
## list(squares(bvar_with_strictness)) -> LSQUARE bvar_with_strictness . RSQUARE list(squares(bvar_with_strictness)) [ VERBATIM LBRACE ]
##
## The known suffix of the stack is as follows:
## LSQUARE bvar_with_strictness
##

parse error 403. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: FUN LSQUARE TEXT RSQUARE XML_ELT_IDENT
##
## Ends in an error in state: 59.
##
## list(squares(bvar_with_strictness)) -> LSQUARE bvar_with_strictness RSQUARE . list(squares(bvar_with_strictness)) [ VERBATIM LBRACE ]
##
## The known suffix of the stack is as follows:
## LSQUARE bvar_with_strictness RSQUARE
##

parse error 415. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: LBRACE RSQUARE
##
## Ends in an error in state: 63.
##
## head_node -> LBRACE . list(locate(textual_node)) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## LBRACE
##

parse error 427. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: IMPORT XML_ELT_IDENT
##
## Ends in an error in state: 64.
##
## head_node -> IMPORT . LBRACE wstext RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## IMPORT
##

parse error 439. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: IMPORT LBRACE XML_ELT_IDENT
##
## Ends in an error in state: 65.
##
## head_node -> IMPORT LBRACE . wstext RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## IMPORT LBRACE
##

parse error 451. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: IMPORT LBRACE TEXT RSQUARE
##
## Ends in an error in state: 66.
##
## head_node -> IMPORT LBRACE wstext . RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## IMPORT LBRACE wstext
##
## WARNING: This example involves spurious reductions.
## This implies that, although the LR(1) items shown above provide an
## accurate view of the past (what has been recognized so far), they
## may provide an INCOMPLETE view of the future (what was expected next).
## In state 10, spurious reduction of production list(ws_or_text) ->
## In state 11, spurious reduction of production list(ws_or_text) -> ws_or_text list(ws_or_text)
## In state 12, spurious reduction of production wstext -> list(ws_or_text)
##

parse error 471. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: HASH_LBRACE RSQUARE
##
## Ends in an error in state: 68.
##
## head_node -> HASH_LBRACE . list(locate(textual_node)) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## HASH_LBRACE
##

parse error 483. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: HASH_HASH_LBRACE RSQUARE
##
## Ends in an error in state: 70.
##
## head_node -> HASH_HASH_LBRACE . list(locate(textual_node)) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## HASH_HASH_LBRACE
##

parse error 495. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: GET XML_ELT_IDENT
##
## Ends in an error in state: 71.
##
## head_node -> GET . ident [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## GET
##

parse error 507. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: FUN XML_ELT_IDENT
##
## Ends in an error in state: 73.
##
## head_node -> FUN . list(squares(bvar_with_strictness)) arg [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## FUN
##

parse error 519. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: EXPORT XML_ELT_IDENT
##
## Ends in an error in state: 76.
##
## head_node -> EXPORT . LBRACE wstext RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## EXPORT
##

parse error 531. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: EXPORT LBRACE XML_ELT_IDENT
##
## Ends in an error in state: 77.
##
## head_node -> EXPORT LBRACE . wstext RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## EXPORT LBRACE
##

parse error 543. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: EXPORT LBRACE TEXT RSQUARE
##
## Ends in an error in state: 78.
##
## head_node -> EXPORT LBRACE wstext . RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## EXPORT LBRACE wstext
##
## WARNING: This example involves spurious reductions.
## This implies that, although the LR(1) items shown above provide an
## accurate view of the past (what has been recognized so far), they
## may provide an INCOMPLETE view of the future (what was expected next).
## In state 10, spurious reduction of production list(ws_or_text) ->
## In state 11, spurious reduction of production list(ws_or_text) -> ws_or_text list(ws_or_text)
## In state 12, spurious reduction of production wstext -> list(ws_or_text)
##

parse error 563. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: DEFAULT XML_ELT_IDENT
##
## Ends in an error in state: 80.
##
## head_node -> DEFAULT . ident arg [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## DEFAULT
##

parse error 575. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: DEFAULT IDENT XML_ELT_IDENT
##
## Ends in an error in state: 81.
##
## head_node -> DEFAULT ident . arg [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## DEFAULT ident
##

parse error 587. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: DEF XML_ELT_IDENT
##
## Ends in an error in state: 83.
##
## head_node -> DEF . ident list(squares(bvar_with_strictness)) arg [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## DEF
##

parse error 599. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: DEF IDENT XML_ELT_IDENT
##
## Ends in an error in state: 84.
##
## head_node -> DEF ident . list(squares(bvar_with_strictness)) arg [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## DEF ident
##

parse error 611. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: DECL_XMLNS XML_ELT_IDENT
##
## Ends in an error in state: 87.
##
## head_node -> DECL_XMLNS . LBRACE wstext RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## DECL_XMLNS
##

parse error 623. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: DECL_XMLNS LBRACE XML_ELT_IDENT
##
## Ends in an error in state: 88.
##
## head_node -> DECL_XMLNS LBRACE . wstext RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## DECL_XMLNS LBRACE
##

parse error 635. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: DECL_XMLNS LBRACE TEXT RSQUARE
##
## Ends in an error in state: 89.
##
## head_node -> DECL_XMLNS LBRACE wstext . RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## DECL_XMLNS LBRACE wstext
##
## WARNING: This example involves spurious reductions.
## This implies that, although the LR(1) items shown above provide an
## accurate view of the past (what has been recognized so far), they
## may provide an INCOMPLETE view of the future (what was expected next).
## In state 10, spurious reduction of production list(ws_or_text) ->
## In state 11, spurious reduction of production list(ws_or_text) -> ws_or_text list(ws_or_text)
## In state 12, spurious reduction of production wstext -> list(ws_or_text)
##

parse error 655. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: CALL XML_ELT_IDENT
##
## Ends in an error in state: 91.
##
## head_node -> CALL . LBRACE ws_list(locate(head_node)) RBRACE LBRACE wstext RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## CALL
##

parse error 667. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: CALL LBRACE TEXT
##
## Ends in an error in state: 92.
##
## head_node -> CALL LBRACE . ws_list(locate(head_node)) RBRACE LBRACE wstext RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## CALL LBRACE
##

parse error 679. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: ALLOC XML_ELT_IDENT
##
## Ends in an error in state: 93.
##
## head_node -> ALLOC . ident [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## ALLOC
##

parse error 691. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: IDENT TEXT
##
## Ends in an error in state: 95.
##
## list(ws_or(locate(head_node))) -> ws_or(locate(head_node)) . list(ws_or(locate(head_node))) [ RBRACE EOF ]
##
## The known suffix of the stack is as follows:
## ws_or(locate(head_node))
##

parse error 703. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: CALL LBRACE IDENT EOF
##
## Ends in an error in state: 99.
##
## head_node -> CALL LBRACE ws_list(locate(head_node)) . RBRACE LBRACE wstext RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## CALL LBRACE ws_list(locate(head_node))
##
## WARNING: This example involves spurious reductions.
## This implies that, although the LR(1) items shown above provide an
## accurate view of the past (what has been recognized so far), they
## may provide an INCOMPLETE view of the future (what was expected next).
## In state 95, spurious reduction of production list(ws_or(locate(head_node))) ->
## In state 96, spurious reduction of production list(ws_or(locate(head_node))) -> ws_or(locate(head_node)) list(ws_or(locate(head_node)))
## In state 104, spurious reduction of production ws_list(locate(head_node)) -> list(ws_or(locate(head_node)))
##

parse error 723. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: CALL LBRACE RBRACE XML_ELT_IDENT
##
## Ends in an error in state: 100.
##
## head_node -> CALL LBRACE ws_list(locate(head_node)) RBRACE . LBRACE wstext RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## CALL LBRACE ws_list(locate(head_node)) RBRACE
##

parse error 735. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: CALL LBRACE RBRACE LBRACE XML_ELT_IDENT
##
## Ends in an error in state: 101.
##
## head_node -> CALL LBRACE ws_list(locate(head_node)) RBRACE LBRACE . wstext RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## CALL LBRACE ws_list(locate(head_node)) RBRACE LBRACE
##

parse error 747. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: CALL LBRACE RBRACE LBRACE TEXT RSQUARE
##
## Ends in an error in state: 102.
##
## head_node -> CALL LBRACE ws_list(locate(head_node)) RBRACE LBRACE wstext . RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## CALL LBRACE ws_list(locate(head_node)) RBRACE LBRACE wstext
##
## WARNING: This example involves spurious reductions.
## This implies that, although the LR(1) items shown above provide an
## accurate view of the past (what has been recognized so far), they
## may provide an INCOMPLETE view of the future (what was expected next).
## In state 10, spurious reduction of production list(ws_or_text) ->
## In state 11, spurious reduction of production list(ws_or_text) -> ws_or_text list(ws_or_text)
## In state 12, spurious reduction of production wstext -> list(ws_or_text)
##

parse error 767. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: HASH_HASH_LBRACE IDENT EOF
##
## Ends in an error in state: 105.
##
## list(locate(textual_node)) -> textual_node . list(locate(textual_node)) [ RSQUARE RPAREN RBRACE ]
##
## The known suffix of the stack is as follows:
## textual_node
##

Did you forget to close it?

main: HASH_HASH_LBRACE IDENT RSQUARE
##
## Ends in an error in state: 108.
##
## head_node -> HASH_HASH_LBRACE list(locate(textual_node)) . RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## HASH_HASH_LBRACE list(locate(textual_node))
##
## WARNING: This example involves spurious reductions.
## This implies that, although the LR(1) items shown above provide an
## accurate view of the past (what has been recognized so far), they
## may provide an INCOMPLETE view of the future (what was expected next).
## In state 105, spurious reduction of production list(locate(textual_node)) ->
## In state 106, spurious reduction of production list(locate(textual_node)) -> textual_node list(locate(textual_node))
##

parse error 798. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: HASH_LBRACE IDENT RSQUARE
##
## Ends in an error in state: 110.
##
## head_node -> HASH_LBRACE list(locate(textual_node)) . RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## HASH_LBRACE list(locate(textual_node))
##
## WARNING: This example involves spurious reductions.
## This implies that, although the LR(1) items shown above provide an
## accurate view of the past (what has been recognized so far), they
## may provide an INCOMPLETE view of the future (what was expected next).
## In state 105, spurious reduction of production list(locate(textual_node)) ->
## In state 106, spurious reduction of production list(locate(textual_node)) -> textual_node list(locate(textual_node))
##

parse error 817. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: LBRACE IDENT RSQUARE
##
## Ends in an error in state: 112.
##
## head_node -> LBRACE list(locate(textual_node)) . RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## LBRACE list(locate(textual_node))
##
## WARNING: This example involves spurious reductions.
## This implies that, although the LR(1) items shown above provide an
## accurate view of the past (what has been recognized so far), they
## may provide an INCOMPLETE view of the future (what was expected next).
## In state 105, spurious reduction of production list(locate(textual_node)) ->
## In state 106, spurious reduction of production list(locate(textual_node)) -> textual_node list(locate(textual_node))
##

parse error 836. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: LPAREN IDENT RSQUARE
##
## Ends in an error in state: 114.
##
## head_node -> LPAREN list(locate(textual_node)) . RPAREN [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## LPAREN list(locate(textual_node))
##
## WARNING: This example involves spurious reductions.
## This implies that, although the LR(1) items shown above provide an
## accurate view of the past (what has been recognized so far), they
## may provide an INCOMPLETE view of the future (what was expected next).
## In state 105, spurious reduction of production list(locate(textual_node)) ->
## In state 106, spurious reduction of production list(locate(textual_node)) -> textual_node list(locate(textual_node))
##

parse error 855. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: LSQUARE IDENT RPAREN
##
## Ends in an error in state: 116.
##
## head_node -> LSQUARE list(locate(textual_node)) . RSQUARE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## LSQUARE list(locate(textual_node))
##
## WARNING: This example involves spurious reductions.
## This implies that, although the LR(1) items shown above provide an
## accurate view of the past (what has been recognized so far), they
## may provide an INCOMPLETE view of the future (what was expected next).
## In state 105, spurious reduction of production list(locate(textual_node)) ->
## In state 106, spurious reduction of production list(locate(textual_node)) -> textual_node list(locate(textual_node))
##

parse error 874. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: NAMESPACE IDENT LBRACE IDENT EOF
##
## Ends in an error in state: 118.
##
## head_node -> NAMESPACE ident LBRACE ws_list(locate(head_node)) . RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## NAMESPACE ident LBRACE ws_list(locate(head_node))
##
## WARNING: This example involves spurious reductions.
## This implies that, although the LR(1) items shown above provide an
## accurate view of the past (what has been recognized so far), they
## may provide an INCOMPLETE view of the future (what was expected next).
## In state 95, spurious reduction of production list(ws_or(locate(head_node))) ->
## In state 96, spurious reduction of production list(ws_or(locate(head_node))) -> ws_or(locate(head_node)) list(ws_or(locate(head_node)))
## In state 104, spurious reduction of production ws_list(locate(head_node)) -> list(ws_or(locate(head_node)))
##

parse error 894. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: PATCH LBRACE IDENT EOF
##
## Ends in an error in state: 120.
##
## head_node -> PATCH LBRACE ws_list(locate(head_node)) . RBRACE option(squares(bvar)) LBRACE ws_list(method_decl) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## PATCH LBRACE ws_list(locate(head_node))
##
## WARNING: This example involves spurious reductions.
## This implies that, although the LR(1) items shown above provide an
## accurate view of the past (what has been recognized so far), they
## may provide an INCOMPLETE view of the future (what was expected next).
## In state 95, spurious reduction of production list(ws_or(locate(head_node))) ->
## In state 96, spurious reduction of production list(ws_or(locate(head_node))) -> ws_or(locate(head_node)) list(ws_or(locate(head_node)))
## In state 104, spurious reduction of production ws_list(locate(head_node)) -> list(ws_or(locate(head_node)))
##

parse error 914. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: PATCH LBRACE RBRACE XML_ELT_IDENT
##
## Ends in an error in state: 121.
##
## head_node -> PATCH LBRACE ws_list(locate(head_node)) RBRACE . option(squares(bvar)) LBRACE ws_list(method_decl) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## PATCH LBRACE ws_list(locate(head_node)) RBRACE
##

parse error 926. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: PATCH LBRACE RBRACE LSQUARE TEXT RSQUARE XML_ELT_IDENT
##
## Ends in an error in state: 122.
##
## head_node -> PATCH LBRACE ws_list(locate(head_node)) RBRACE option(squares(bvar)) . LBRACE ws_list(method_decl) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## PATCH LBRACE ws_list(locate(head_node)) RBRACE option(squares(bvar))
##

parse error 938. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: PATCH LBRACE RBRACE LBRACE XML_ELT_IDENT
##
## Ends in an error in state: 123.
##
## head_node -> PATCH LBRACE ws_list(locate(head_node)) RBRACE option(squares(bvar)) LBRACE . ws_list(method_decl) RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## PATCH LBRACE ws_list(locate(head_node)) RBRACE option(squares(bvar)) LBRACE
##

parse error 950. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: SCOPE LBRACE IDENT RSQUARE
##
## Ends in an error in state: 126.
##
## arg -> LBRACE list(locate(textual_node)) . RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## LBRACE list(locate(textual_node))
##
## WARNING: This example involves spurious reductions.
## This implies that, although the LR(1) items shown above provide an
## accurate view of the past (what has been recognized so far), they
## may provide an INCOMPLETE view of the future (what was expected next).
## In state 105, spurious reduction of production list(locate(textual_node)) ->
## In state 106, spurious reduction of production list(locate(textual_node)) -> textual_node list(locate(textual_node))
##

parse error 969. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. 

main: SUBTREE LBRACE IDENT EOF
##
## Ends in an error in state: 129.
##
## head_node -> SUBTREE option(squares(wstext)) LBRACE ws_list(locate(head_node)) . RBRACE [ XML_ELT_IDENT WHITESPACE VERBATIM TEXT SUBTREE SCOPE RSQUARE RPAREN RBRACE PUT PATCH OPEN OBJECT NAMESPACE LSQUARE LPAREN LET LBRACE IMPORT IDENT HASH_LBRACE HASH_IDENT HASH_HASH_LBRACE GET FUN EXPORT EOF DEFAULT DEF DECL_XMLNS CALL ALLOC ]
##
## The known suffix of the stack is as follows:
## SUBTREE option(squares(wstext)) LBRACE ws_list(locate(head_node))
##
## WARNING: This example involves spurious reductions.
## This implies that, although the LR(1) items shown above provide an
## accurate view of the past (what has been recognized so far), they
## may provide an INCOMPLETE view of the future (what was expected next).
## In state 95, spurious reduction of production list(ws_or(locate(head_node))) ->
## In state 96, spurious reduction of production list(ws_or(locate(head_node))) -> ws_or(locate(head_node)) list(ws_or(locate(head_node)))
## In state 104, spurious reduction of production ws_list(locate(head_node)) -> list(ws_or(locate(head_node)))
##

parse error 989. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. .

main: IDENT RBRACE
##
## Ends in an error in state: 131.
##
## main -> ws_list(locate(head_node)) . EOF [ # ]
##
## The known suffix of the stack is as follows:
## ws_list(locate(head_node))
##
## WARNING: This example involves spurious reductions.
## This implies that, although the LR(1) items shown above provide an
## accurate view of the past (what has been recognized so far), they
## may provide an INCOMPLETE view of the future (what was expected next).
## In state 95, spurious reduction of production list(ws_or(locate(head_node))) ->
## In state 96, spurious reduction of production list(ws_or(locate(head_node))) -> ws_or(locate(head_node)) list(ws_or(locate(head_node)))
## In state 104, spurious reduction of production ws_list(locate(head_node)) -> list(ws_or(locate(head_node)))
##

parse error 1009. Help us improve this message by commenting on the tracking issue: https://todo.sr.ht/~jonsterling/forester. .
diff --git a/lib/compiler/Grammar.mly b/lib/compiler/Grammar.mly
index 612f9b2..512686f 100644
--- a/lib/compiler/Grammar.mly
+++ b/lib/compiler/Grammar.mly
@@ -6,6 +6,7 @@
%token <string> XML_ELT_IDENT
%token <string> DECL_XMLNS
%token <string> TEXT VERBATIM
%token <string> COMMENT
%token <string> WHITESPACE
%token <string> IDENT
%token <string> HASH_IDENT
diff --git a/lib/compiler/Lexer.mll b/lib/compiler/Lexer.mll
index 1818243..451894f 100644
--- a/lib/compiler/Lexer.mll
+++ b/lib/compiler/Lexer.mll
@@ -1,10 +1,14 @@
{
  open Forester_prelude
  let drop_sigil c str = 1 |> List.nth @@ String.split_on_char c str
  let raise_err lexbuf =
  exception Lex_err of (Lexing.lexbuf * string option)

  let raise_err lexbuf ?msg =
    let loc = Asai.Range.of_lexbuf lexbuf in
    Forester_core.Reporter.fatalf ~loc Forester_core.Reporter.Message.Parse_error "unrecognized token `%s`" @@
    String.escaped @@ Lexing.lexeme lexbuf
    Forester_core.Reporter.fatalf
      ~loc
      Forester_core.Reporter.Message.Parse_error
      "unrecognized token `%s`.@ %a" (String.escaped @@ Lexing.lexeme lexbuf) Format.(pp_print_option pp_print_string) msg
}

let digit = ['0'-'9']
@@ -24,6 +28,7 @@ let verbatim_herald_sep = '|'

rule token = parse
  | "\\%" { Grammar.TEXT "%" }
  (* I added comments to the syntax tree, but how to proceed here? *)
  | "%" { comment lexbuf }
  | "##{" { Grammar.HASH_HASH_LBRACE }
  | "#{" { Grammar.HASH_LBRACE }
@@ -79,7 +84,7 @@ rule token = parse
  | wschar+ { Grammar.WHITESPACE (Lexing.lexeme lexbuf) }
  | newline { Lexing.new_line lexbuf; Grammar.WHITESPACE (Lexing.lexeme lexbuf) }
  | eof { Grammar.EOF }
  | _ { raise_err lexbuf }
  | _ { raise_err lexbuf ?msg:None}

and comment = parse
  | newline_followed_by_ws { Lexing.new_line lexbuf; token lexbuf }
@@ -88,13 +93,13 @@ and comment = parse

and custom_verbatim_herald = parse
  | verbatim_herald as herald { eat_verbatim_herald_sep (custom_verbatim herald (Buffer.create 2000)) lexbuf }
  | newline { Lexing.new_line lexbuf; raise_err lexbuf }
  | _ { raise_err lexbuf }
  | newline { Lexing.new_line lexbuf; raise_err lexbuf ?msg:None }
  | _ { raise_err lexbuf ?msg:None }

and eat_verbatim_herald_sep kont = parse
  | verbatim_herald_sep { kont lexbuf }
  | newline { Lexing.new_line lexbuf; raise_err lexbuf }
  | _ { raise_err lexbuf }
  | newline { Lexing.new_line lexbuf; raise_err ~msg:"Did you forget to start the verbatim block with `|`?" lexbuf }
  | _ { raise_err ?msg:None lexbuf }

and custom_verbatim herald buffer = parse
  | newline as c
@@ -106,31 +111,34 @@ and custom_verbatim herald buffer = parse
  | _ as c
    {
      Buffer.add_char buffer c;
      let buff_len = Buffer.length buffer in
      let herald_len = String.length herald in
      let offset = buff_len - herald_len in
      if offset >= 0 && Buffer.sub buffer offset herald_len = herald then
        let text =
          String_util.trim_trailing_whitespace @@
          String_util.trim_newlines @@
          Buffer.sub buffer 0 offset
        in
        Grammar.VERBATIM text
      else
        custom_verbatim herald buffer lexbuf
      try
        let buff_len = Buffer.length buffer in
        let herald_len = String.length herald in
        let offset = buff_len - herald_len in
        if offset >= 0 && Buffer.sub buffer offset herald_len = herald then
          let text =
            String_util.trim_trailing_whitespace @@
            String_util.trim_newlines @@
            Buffer.sub buffer 0 offset
          in
          Grammar.VERBATIM text
        else
          custom_verbatim herald buffer lexbuf
      (* How can we report the location where the verbatim block was opened?*)
      with _ -> raise_err lexbuf ~msg:(Format.sprintf "Did you forget to close the verbatim block with `%s`?" herald);
    }

and xml_qname = parse
  | xml_qname as qname { qname }
  | newline { Lexing.new_line lexbuf; raise_err lexbuf }
  | _ { raise_err lexbuf }
  | newline { Lexing.new_line lexbuf; raise_err ?msg:None lexbuf }
  | _ { raise_err ?msg:None lexbuf }

and xml_base_ident = parse
  | xml_base_ident as x { x }
  | newline { Lexing.new_line lexbuf; raise_err lexbuf }
  | _ { raise_err lexbuf }
  | newline { Lexing.new_line lexbuf; raise_err ?msg:None lexbuf }
  | _ { raise_err ?msg:None lexbuf }

and rangle = parse
  | ">" { () }
  | newline { Lexing.new_line lexbuf; raise_err lexbuf }
  | _ { raise_err lexbuf }
  | newline { Lexing.new_line lexbuf; raise_err ?msg:None lexbuf }
  | _ { raise_err ?msg:None lexbuf }
diff --git a/lib/compiler/Parse.ml b/lib/compiler/Parse.ml
index e5f276c..73d646d 100644
--- a/lib/compiler/Parse.ml
+++ b/lib/compiler/Parse.ml
@@ -1,158 +1,160 @@
open Forester_prelude
open Forester_core
open Lexing

module I = Grammar.MenhirInterpreter

(* debugging helpers *)
let _string_of_token token =
  match token with
  | Grammar.LBRACE -> "LBRACE"
  | Grammar.RBRACE -> "RBRACE"
  | Grammar.LSQUARE -> "LSQUARE"
  | Grammar.RSQUARE -> "RSQUARE"
  | Grammar.LPAREN -> "LPAREN"
  | Grammar.RPAREN -> "RPAREN"
  | Grammar.HASH_LBRACE -> "HASH_LBRACE"
  | Grammar.HASH_HASH_LBRACE -> "HASH_HASH_LBRACE"
  | Grammar.WHITESPACE w -> w
  | Grammar.TEXT s -> s
  | Grammar.EOF -> "EOF"
  | Grammar.IDENT s -> Format.sprintf "IDENT(%s)" s
  | _ -> "<unimplemented>"
(*  NOTE:
    I am unsure if we should ever fail during lexing. I propose: We should
    introduce an `Error` syntax node so that the parse tree is capable of
    representing arbitrary erroneous input. We never want this during batch
    compilation, but always want this for the IDE. Depending on the context in
    which the parser is running, we can immediately fail when encountering
    such a node, or continue. This is what the `I.AboutToReduce` state is for.
  *)

let _char_of_token token =
  match token with
  | Grammar.LBRACE -> '{'
  | Grammar.RBRACE -> '}'
  | Grammar.LSQUARE -> '['
  | Grammar.RSQUARE -> ']'
  | Grammar.LPAREN -> '('
  | Grammar.RPAREN -> ')'
  | Grammar.HASH_LBRACE -> '#'
  | Grammar.HASH_HASH_LBRACE -> '#'
  | _ -> 'x'
let get_parse_error env =
  let open Asai in
  match I.stack env with
  | lazy(Nil) -> Diagnostic.loctext "Did not expect text here."
  | lazy(Cons (I.Element (state, _, start, end_), _)) ->
    try
      let loc = Range.of_lex_range (start, end_) in
      let msg = (Grammar_messages.message (I.number state)) in
      Diagnostic.loctext ~loc msg
    with
      | Not_found -> Diagnostic.loctext "invalid syntax (no specific message for this eror)"

(* drive the parser to the next InputNeeded checkpoint *)
let rec resumes checkpoint =
  match checkpoint with
  | I.InputNeeded env -> I.input_needed env
  | I.Shifting _ | I.AboutToReduce _ -> resumes @@ I.resume checkpoint
  | _ -> assert false
let get_range
    : I.element option -> (position * position) option
  = fun el ->
    match el with
    | Some (I.Element (_, _, start_pos, end_pos)) ->
      Some (start_pos, end_pos)
    | None -> None

(* strategy: whenever we hit an unexpected closing delimiter, we look for a matching opening delimiter in the past
   if we find one, close all intermediate (hanging) delimiters and then continue parsing
   otherwise just continue parsing
   if we hit a premature EOF, try to close all delimiters, and if that fails return the last good parse
   (on each token, we test if the ending here would have produced a valid parse) *)
let try_parse lexbuf =
  let rec fail bracketing last_token last_accept before supplier chkpt =
    match chkpt with
    | I.HandlingError _ ->
      let loc = Asai.Range.of_lexbuf lexbuf in
      Reporter.emitf ~loc Parse_error "syntax error, unexpected `%s`\n" (Lexing.lexeme lexbuf);
      begin
        match last_token with
        | Grammar.RPAREN
        | Grammar.RSQUARE
        | Grammar.RBRACE ->
          begin
            match List.find_index (fun c -> c = last_token) bracketing with
            | Some i ->
              (* try to find a small enclosing scope *)
              let consume = List.to_seq bracketing |> Seq.take (i + 1) in
              let remaining = List.to_seq bracketing |> Seq.drop i |> List.of_seq in
              let continue = Seq.fold_left (fun acc t -> resumes @@ I.offer acc (t, lexbuf.lex_curr_p, lexbuf.lex_curr_p)) before consume in
              run remaining last_token last_accept before supplier continue
            | None ->
              (* ignore this token and move on *)
              run bracketing Grammar.EOF last_accept before supplier before
          end
        | Grammar.EOF ->
          if not @@ List.is_empty bracketing then
            (* have hanging delimiters to close *)
            let continue = List.fold_left (fun acc t -> resumes @@ I.offer acc (t, lexbuf.lex_curr_p, lexbuf.lex_curr_p)) before bracketing in
            run [] last_token last_accept before supplier continue
          else
            (* can't continue, give up and use last_accept *)
            run [] last_token last_accept before supplier last_accept
        | _ ->
          (* ignore this token and move on *)
          run bracketing Grammar.EOF last_accept before supplier before
      end
    | _ -> Reporter.fatal Parse_error "unreachable parser state"
let closed_by c o =
  match (o, c) with
  | (Grammar.LSQUARE, Grammar.RSQUARE)
  | (Grammar.LPAREN, Grammar.RPAREN)
  | (Grammar.LBRACE, Grammar.RBRACE)
  | (Grammar.HASH_LBRACE, Grammar.RBRACE)
  | (Grammar.HASH_HASH_LBRACE, Grammar.RBRACE) ->
    true
  | _ -> false

let is_opening_delim = function
  | Grammar.LSQUARE
  | Grammar.LPAREN
  | Grammar.LBRACE
  | Grammar.HASH_LBRACE
  | Grammar.HASH_HASH_LBRACE ->
    true
  | _ -> false

  and run bracketing last_token last_accept last_input_needed supplier checkpoint =
    match checkpoint with
    | I.InputNeeded _ ->
      (* last_token has been accepted, update bracketing *)
      let bracketing =
        match last_token with
        | Grammar.RPAREN
        | Grammar.RSQUARE
        | Grammar.RBRACE ->
          assert (List.hd bracketing = last_token); List.tl bracketing
        | _ -> bracketing
      in
      (* get new token *)
      let token, start, end_ = supplier () in
      let bracketing =
        match token with
        | Grammar.LPAREN -> Grammar.RPAREN :: bracketing
        | Grammar.LSQUARE -> Grammar.RSQUARE :: bracketing
        | Grammar.LBRACE
        | Grammar.HASH_LBRACE
        | Grammar.HASH_HASH_LBRACE ->
          Grammar.RBRACE :: bracketing
        | _ -> bracketing
      in
      (* check if it's possible to end parsing here, update last_accept *)
      let la =
        if I.acceptable checkpoint Grammar.EOF start then checkpoint
        else last_accept
      in
      run bracketing token la checkpoint supplier @@ I.offer checkpoint (token, start, end_)
    | I.Accepted v -> v
    | I.Rejected
    | I.HandlingError _ ->
      fail bracketing last_token last_accept last_input_needed supplier checkpoint
    | I.Shifting _
    | I.AboutToReduce _ ->
      run bracketing last_token last_accept last_input_needed supplier @@ I.resume checkpoint
  in
  let checkpoint = Grammar.Incremental.main lexbuf.lex_curr_p in
  let supplier = I.lexer_lexbuf_to_supplier Lexer.token lexbuf in
  run [] Grammar.EOF checkpoint checkpoint supplier checkpoint
let is_closing_delim = function
  | Grammar.RSQUARE
  | Grammar.RPAREN
  | Grammar.RBRACE ->
    true
  | _ -> false

let maybe_with_errors (f : unit -> 'a) : ('a, 'a * 'b list) result =
  let errors = ref Bwd.Emp in
  let result =
    let@ () =
      Reporter.map_diagnostic @@
        fun d ->
          errors := Bwd.snoc !errors d;
          d
let incr_parse
    : ?stop_on_err: bool ->
    lexbuf ->
    (Code.t, Reporter.Message.t Asai.Diagnostic.t) Result.t
  = fun ?(stop_on_err = true) lexbuf ->
    (* The idea is simple: push opening delimiters onto the stack, pop them
       once they are closed. Upon encountering an error, the reported range
       includes the last unclosed delimiter.
     *)
    let delim_stack = Stack.create () in
    let initial_checkpoint = (Grammar.Incremental.main lexbuf.lex_curr_p) in
    let rec run
        : _ I.checkpoint ->
        (Code.t, Reporter.Message.t Asai.Diagnostic.t) Result.t
      = fun checkpoint ->
        match checkpoint with
        | I.InputNeeded _env ->
          (* In this phase we push and pop delimiters onto the stack.*)
          let token = Lexer.token lexbuf in
          let start_position = lexbuf.lex_start_p in
          let end_position = lexbuf.lex_curr_p in
          if is_opening_delim token then
            let range = Range.of_lex_range (start_position, end_position) in
            Stack.push (token, range) delim_stack; ;
          if is_closing_delim token then
            begin
              match Stack.top_opt delim_stack with
              | Some (open_delim, _) ->
                if (open_delim |> closed_by token) then
                  Stack.drop delim_stack
              | None -> ()
            end;
          let checkpoint = I.offer checkpoint (token, start_position, end_position) in
          run checkpoint
        | I.Shifting((_, _, _): Code.t I.env * Code.t I.env * bool) ->
          let checkpoint = I.resume checkpoint ~strategy: `Simplified in
          run checkpoint
        | I.AboutToReduce (_, _) ->
          let checkpoint = I.resume checkpoint ~strategy: `Simplified in
          run checkpoint
        | I.HandlingError env ->
          if stop_on_err then
            let err = get_parse_error env in
            let range_of_last_unclosed =
              Option.map snd @@ Stack.top_opt delim_stack
            in
            let loc =
              match Option.map Range.view range_of_last_unclosed with
              | Some (`Range (start_pos, _)) ->
                Range.make
                  (start_pos, Range.of_lex_position @@ Lexing.lexeme_start_p lexbuf)
              | Some (`End_of_file _) -> Range.of_lexbuf lexbuf
              | None -> Range.of_lexbuf lexbuf
            in
            let extra_remarks =
              if Option.is_some range_of_last_unclosed then
                [
                  Asai.Diagnostic.loctext
                    ?loc: range_of_last_unclosed
                    "This delimiter is never closed";
                ]
              else []
            in
            Error
              (
                Asai.Diagnostic.of_loctext
                  ~extra_remarks
                  Error
                  Forester_core.Reporter.Message.Parse_error
                  { loc = Some loc; value = err.value }
              )
          else
            Error
              (
                Asai.Diagnostic.of_text
                  Error
                  Forester_core.Reporter.Message.Parse_error
                  (Asai.Diagnostic.text "")
              )
        | I.Accepted code -> Ok code
        | I.Rejected ->
          assert false
    in
    f ()
  in
  match !errors with
  | Emp -> Result.ok result
  | errs -> Result.error (result, Bwd.prepend errs [])
    run initial_checkpoint

let parse_channel filename ch =
  let@ () = Reporter.tracef "when parsing file `%s`" filename in
  let lexbuf = Lexing.from_channel ch in
  lexbuf.lex_curr_p <- { lexbuf.lex_curr_p with pos_fname = filename };
  let@ () = maybe_with_errors in
  try_parse lexbuf
  try
    incr_parse lexbuf
  with
    | Grammar.Error ->
      Reporter.fatalf ~loc: (Range.of_lexbuf lexbuf) Parse_error "failed to parse"
    | exn -> raise exn

let parse_file filename =
  let ch = open_in filename in
  let@ () = Fun.protect ~finally: (fun _ -> close_in ch) in
  parse_channel filename ch

let parse_string str =
  let@ () = Reporter.tracef "when parsing string" in
  let lexbuf = Lexing.from_string str in
  let@ () = maybe_with_errors in
  try_parse lexbuf
  Fun.protect ~finally: (fun _ -> close_in ch) @@
    fun _ ->
      parse_channel filename ch
diff --git a/lib/compiler/Parse.mli b/lib/compiler/Parse.mli
deleted file mode 100644
index 1a0a374..0000000
--- a/lib/compiler/Parse.mli
@@ -1,5 +0,0 @@
open Forester_core

val parse_file : string -> (Code.t, Code.t * Reporter.Message.t Asai.Diagnostic.t list) result

val parse_string : string -> (Code.t, Code.t * Reporter.Message.t Asai.Diagnostic.t list) result
diff --git a/lib/compiler/Resolver.ml b/lib/compiler/Resolver.ml
index af1ff78..73d1ade 100644
--- a/lib/compiler/Resolver.ml
+++ b/lib/compiler/Resolver.ml
@@ -25,6 +25,10 @@ module Scope = struct
    include_subtree ?modifier (path, subtree)

  let easy_run kont = run kont

  let pp_path ppf path =
    let pp_slash ppf () = Format.fprintf ppf "/" in
    Format.(fprintf ppf "%a" (pp_print_list ~pp_sep: pp_slash pp_print_string) path)
end

module Lang = Yuujinchou.Language
diff --git a/lib/compiler/dune b/lib/compiler/dune
index 1d38bc8..12742b5 100644
--- a/lib/compiler/dune
+++ b/lib/compiler/dune
@@ -5,6 +5,12 @@
 (explain true)
 (flags --inspection --table --dump))

; The target for saving menhir's stdout to Grammar_messages.ml
(rule
 (targets Grammar_messages.ml)
 (deps Grammar.messages Grammar.mly)
 (action  (with-stdout-to %{targets} (run menhir --compile-errors %{deps}))))

(library
 (name Forester_compiler)
 (preprocess
diff --git a/lib/forest/Forest.ml b/lib/forest/Forest.ml
index 2243bd7..bb24309 100644
--- a/lib/forest/Forest.ml
+++ b/lib/forest/Forest.ml
@@ -148,7 +148,12 @@ module Make (Graphs: Forest_graphs.S) : S = struct
    match get_article addr with
    | Some article -> article
    | None ->
      Reporter.fatalf Tree_not_found "Could not find tree %a" pp_iri addr
      Reporter.fatalf
        ~extra_remarks: [Asai.Diagnostic.loctextf "the tree %a is either missing, or failed to evaluate." pp_iri addr]
        Tree_not_found
        "Could not find tree %a"
        pp_iri
        addr

  module Query_engine = Query_engine.Make(Graphs)
  include Query_engine
diff --git a/lib/forest/Legacy_xml_client.ml b/lib/forest/Legacy_xml_client.ml
index 37dd469..9833795 100644
--- a/lib/forest/Legacy_xml_client.ml
+++ b/lib/forest/Legacy_xml_client.ml
@@ -60,9 +60,9 @@ module Make (Params: Params) (F: Forest.S) () : S = struct

  let iri_type iri =
    match Iri.path iri with
    | Absolute ["unstable"; _] -> "machine"
    | Absolute ["hash"; _] -> "hash"
    | Absolute [_] -> "user"
    | Absolute["unstable"; _] -> "machine"
    | Absolute["hash"; _] -> "hash"
    | Absolute[_] -> "user"
    | _ -> failwith "addr_type"

  let route_bare_forester_iri ~path ~is_asset =
@@ -137,7 +137,7 @@ module Make (Params: Params) (F: Forest.S) () : S = struct
    X.prim p []

  let render_img = function
    | T.Inline { format; base64 } ->
    | T.Inline{ format; base64 } ->
      X.img [X.src "data:image/%s;base64,%s" format base64]
    | T.Remote url ->
      X.img [X.src "%s" url]
@@ -379,6 +379,7 @@ module Make (Params: Params) (F: Forest.S) () : S = struct
      ]

  let render_article (article : T.content T.article) : P.node =
    let@ () = Reporter.tracef "when rendering article %a" Format.(pp_print_option Iri.pp) article.frontmatter.iri in
    let xmlns_prefix = Xmlns.{ prefix = X.reserved_prefix; xmlns = X.forester_xmlns } in
    let@ () = Scope.run ~env: article.frontmatter.iri in
    let@ () = Xmlns.run in
diff --git a/lib/forest/dune b/lib/forest/dune
index c1bf87a..9168598 100644
--- a/lib/forest/dune
+++ b/lib/forest/dune
@@ -10,6 +10,7 @@
  ocamlgraph
  yojson
  algaeff
  asai
  pure-html
  iri))

diff --git a/lib/frontend/Forester.ml b/lib/frontend/Forester.ml
index 002a5bf..cdb1232 100644
--- a/lib/frontend/Forester.ml
+++ b/lib/frontend/Forester.ml
@@ -118,11 +118,19 @@ let parse_trees_in_dirs ~dev ?(ignore_malformed = false) dirs =
  let addr = Filename.chop_extension basename in
  let native = EP.native_exn fp in
  let source_path = if dev then Some (Unix.realpath native) else None in
  match Parse.parse_file native with
  | Result.Ok code -> Some Code.{ source_path; addr = Some addr; code }
  | Result.Error _ -> None
  | exception exn ->
    if ignore_malformed then None else raise exn
  try
    match Parse.parse_file native with
    | Ok code -> Some Code.{ source_path; addr = Some addr; code }
    | Error diagnostic ->
      if ignore_malformed then None
      else
        begin
          Reporter.Tty.display diagnostic;
          None
        end
  with
    | exn ->
      if ignore_malformed then None else raise exn

let plant_forest_from_dirs ~env ~host ~dev ~tree_dirs ~asset_dirs : unit =
  let parsed_trees = parse_trees_in_dirs ~dev tree_dirs in
diff --git a/lib/frontend/Import_graph.ml b/lib/frontend/Import_graph.ml
index 42dee1a..c07ce35 100644
--- a/lib/frontend/Import_graph.ml
+++ b/lib/frontend/Import_graph.ml
@@ -34,7 +34,7 @@ let build (trees : Code.tree list) =
    | Object { methods; _ } | Patch { methods; _ } ->
      let@ _, code = List.iter @~ methods in
      analyse_code roots code
    | Text _ | Hash_ident _ | Angle_ident _ | Verbatim _ | Ident _ | Open _ | Put _ | Default _ | Get _ | Decl_xmlns _ | Call _ | Alloc _ -> ()
    | Text _ | Hash_ident _ | Angle_ident _ | Verbatim _ | Ident _ | Open _ | Put _ | Default _ | Get _ | Decl_xmlns _ | Call _ | Alloc _ | Comment _ | Error _ -> ()
  in
  begin
    let@ tree = List.iter @~ trees in
diff --git a/test/dune b/test/dune
index 7db3d0d..f14d778 100644
--- a/test/dune
+++ b/test/dune
@@ -1,5 +1,5 @@
(tests
 (names parse dsl)
(test
 (name dsl)
 (preprocess
  (pps ppx_deriving.show))
 (libraries
diff --git a/test/parse.expected b/test/parse.expected
deleted file mode 100644
index 5cbb180..0000000
--- a/test/parse.expected
@@ -1,13 +0,0 @@
parse_good_result:
[(Code.Ident title); (Code.Group (Base.Braces, [(Code.Text "Good")]));
  (Code.Ident taxon); (Code.Group (Base.Braces, [(Code.Text "Test")]));
  (Code.Ident author); (Code.Group (Base.Braces, [(Code.Text "Testy")]));
  (Code.Ident p);
  (Code.Group (Base.Braces,
     [(Code.Text "\n"); (Code.Text "      "); (Code.Text "This");
       (Code.Text " "); (Code.Text "should"); (Code.Text " ");
       (Code.Text "parse"); (Code.Text " "); (Code.Text "correctly.");
       (Code.Text "\n"); (Code.Text "    ")]
     ))
  ]

diff --git a/test/parse.ml b/test/parse.ml
deleted file mode 100644
index 862167c..0000000
--- a/test/parse.ml
@@ -1,25 +0,0 @@
open Forester_core
open Forester_compiler

let emit _ = () (* ignore *)

let fatal _ = exit 1

let _ =
  Reporter.run ~emit ~fatal @@
    fun () ->
      let good =
        Result.get_ok @@
          Parse.parse_string
            {|
    \title{Good}
    \taxon{Test}
    \author{Testy}

    \p{
      This should parse correctly.
    }
    |}
      in
      Format.printf "parse_good_result:\n%s\n\n" (Code.show good)

-- 
2.46.0
Nick Hu <nick.hu@cs.ox.ac.uk>
a month ago 
Hi Kento,

I just wanted to weigh in here as I've thought about these things
a decent amount in the past when I worked on the LSP server.

It looks like you have simplified the parsing error recovery strategy.
At a glance (although I didn't test your code), it looks like now if you
have a block like

```forester
\p{
  \malformed{ %error here

  The rest of the paragraph node, some \transclude{my-tree-0001} goes
  here etc.
}
```

then the parser can't recover any of the rest of the `paragraph` node,
and moreover you wouldn't e.g. be able to do a LSP hover on that
`transclude` node. This was the original reason I didn't implement this
simple strategy in the first place.

I agree with your point about never letting the lexer emit errors.
I have implemented this on my own local branch (it seems necessary to
have LSP functionality be robust).

I see that you are also trying to use menhir's automata-state-based
error reporting. I remember looking at this, but it seemed like it would
be adding a lot of complexity and really marries forester to menhir.
I know that Jon has been thinking about replacing menhir entirely.
I wonder if, for better tooling, maybe migrating to a tree-sitter based
parser might be a good idea (that way, we can offload the maintenance
burden of error recovery code, get automatic syntax highlighting etc;
plus you have already written one). One caveat is that tree-sitter's
error recovery mechanism is not really customisable (or even
well-documented for that matter), but maybe it's canonical enough to
form the collective expectation of how parse error recovery should
behave anyway (I wasn't able to find any kind of common practice for
this anyway).

Lastly, I see that you are slowly converging towards a concrete syntax
tree over an abstract syntax tree. There are obvious pros and cons to
this, but I think overall you eventually need something like this to do
things like e.g. make a forester formatter/prettifier, or have decent
LSP refactoring support.
Rust-analyzer, which is the best LSP server I've used by far, has
extracted its implementation of concrete syntax trees into the following
crate:
https://lib.rs/crates/rowan
There's a conceptual overview of how it works here:
https://github.com/rust-lang/rust-analyzer/blob/master/docs/dev/syntax.md

I'm not suggesting that we use this, but it might make for interesting
study material if you're considering the design space.

Kind regards,
Nick