LaTeX pipeline: use lualatex, better dvisvgm options v2 PROPOSED

Nick Hu: 1
 LaTeX pipeline: use lualatex, better dvisvgm options

 2 files changed, 4 insertions(+), 2 deletions(-)

Sorry I'm busy lately and I don't check the devel mailing list
regularly, and only found this patch randomly. There are no
notifications like Github issues and patches are not convenient to
test like Github forks.

My forest https://github.com/utensil/forest have a series of notes on
category theory https://utensil.github.io/forest/tt-0001.xml , and
they were affected by the switch to dvilualatex, which has the
problems of

IIRC.

I'll bump my forester to 5.x in Jan, and around that time I might be
able to test the patch. If you are really looking for an example soon
to test locally, you can try copy pasting one of the trees with
tikzcds into your forest along with its imported trees to make it a
minimal example. Or you can try build my forest locally, but I've add
quite some customization so it might be a little time-consuming to
setup all the deps on the first run.

Thanks for your patch, and I would also love to use lualatex, but not
DVI because of https://todo.sr.ht/~jonsterling/forester/82#event-383302
.

Regarding standalone, I use LaTeX in forester not only for diagrams,
but also for rendering real LaTeX (e.g. a few paragraphs with inline
and display math) with minipage etc. It's usually because for every
note, I usually have the LaTeX first (either copied from a source, or
OCRed from a document), use a script I wrote to convert to forester
syntax, then do the heavy edits. It's more efficient than typing out
forester from scratch.

Regards,

Utensil

[PATCH v2] LaTeX pipeline: use lualatex, better dvisvgm options Export this patch

This is a follow-up to the now-reverted
https://lists.sr.ht/~jonsterling/forester-devel/patches/54351.

This patch switches the LaTeX pipeline (again) to use lualatex instead
of latex (I really need this for algorithmic graph drawing in TikZ), and
works around the issues here:
https://todo.sr.ht/~jonsterling/forester/82#event-382726

I didn't ever notice any bad cropping, but I tested by compiling the
following (which I guessed from the conversation above would be
a 'problem case'? I'd appreciate an actual example of where something
'goes wrong' cc ~utensil):

```forester
\tex{
  \usepackage{tikz}
}{
  \begin{minipage}{1.1\linewidth}
  \begin{figure}[H]
  \centering
  Hello \tikz [baseline] \fill [fill=blue!80!black] (0,.75ex) circle[radius=.75ex];
  \end{figure}
  \end{minipage}
}
```

Naievely just changing `dvilualatex` for `lualatex` (without the
additional change to the LaTeX template) makes the resulting SVG
observationally different; in particular, it crops it much tighter,
whereas the non-lualatex version centers the figure.
The change to the template makes the lualatex version match up with the
non-lualatex version.

While I was at it, I also decided to tweak some of the dvisvgm flags:
* woff2 is basically the same as woff but in a different container,
  compressed with brotli, so the SVGs are smaller;
  according to https://caniuse.com/woff2, it's basically been supported
  in web browsers for about the 10 years so it's safe to use.
* the ,autohint option enables font autohinting if dvisvgm has been
  compiled with the ttfautohint library; it gracefully degrades and
  prints a warning if the library is not available.
* from the dvisvgm manual:
        -O, --optimize[=modules]
           Applies several optimizations on the generated SVG tree to reduce the file size.
           The optimizations are performed by running separate optimizer modules specified by
           optional argument modules. It may consist of a single module name or a
           comma-separated list of several module names. The corresponding modules are
           executed one by one in the given order and thus transform the XML tree gradually.

           <snip> <--optimize is equivalent to --optimize=all>

       -R, --relative
           SVG allows you to define graphics paths by a sequence of absolute and/or relative
           path commands, i.e. each command expects either absolute coordinates or coordinates
           relative to the current drawing position. By default, dvisvgm creates paths made up
           of absolute commands. If option --relative is given, relative commands are created
           instead. This slightly reduces the size of the SVG files in
           most cases.

   these flags should make the SVGs smaller and more efficient.
   I tested on this example https://texample.net/tikz/examples/polygon-division/
   and found that in particular the --optimize flag reduced the file
   size from 20KB to 12KB, which is rather significant.

By the way, setting --font-format (so that a font is embedded in the
SVG) makes the output of dvisvgm non-deterministic!!!!
I think the font gets compressed (non-deterministically) and then
embedded in the output.
I didn't find any way to fix this, but the current pipeline is also
affected by this too.
I guess it doesn't necessarily cause any issues, it's just quite nasty.

---
 lib/compiler/LaTeX_template.ml | 2 ++
 lib/frontend/LaTeX_pipeline.ml | 4 ++--
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/lib/compiler/LaTeX_template.ml b/lib/compiler/LaTeX_template.ml
index 083d48c..4bdf2c7 100644
--- a/lib/compiler/LaTeX_template.ml
+++ b/lib/compiler/LaTeX_template.ml
@@ -8,6 +8,8 @@ let pp fmt ~preamble ~body =
   let newline () = Format.fprintf fmt "\n" in
   Format.fprintf fmt {|\documentclass[crop,dvisvgm]{standalone}|};
   newline ();
+  Format.fprintf fmt {|\special{papersize=\the\paperwidth,\the\paperheight}|};
+  newline ();
   Format.fprintf fmt "%s" preamble;
   newline ();
   Format.fprintf fmt {|\begin{document}|};
diff --git a/lib/frontend/LaTeX_pipeline.ml b/lib/frontend/LaTeX_pipeline.ml
index b5d8399..f13c5bc 100644
--- a/lib/frontend/LaTeX_pipeline.ml
+++ b/lib/frontend/LaTeX_pipeline.ml
@@ -29,7 +29,7 @@ let pipe_latex_dvi ~env ~tex_source kont =
     let out_buf = Buffer.create 1000 in
     let stdout = Eio.Flow.buffer_sink out_buf in
     let stderr = Eio_util.null_sink () in
-    let cmd = ["latex"; "-halt-on-error"; "-interaction=nonstopmode"; tex_fn] in
+    let cmd = ["dvilualatex"; "-halt-on-error"; "-interaction=nonstopmode"; tex_fn] in
     try
       Eio.Process.run ~cwd: tmp ~stdout ~stderr mgr cmd
     with
@@ -49,7 +49,7 @@ let pipe_dvi_svg ~env ~dvi_source ~svg_sink =
   let mgr = Eio.Stdenv.process_mgr env in
   let err_buf = Buffer.create 1000 in
   let stderr = Eio.Flow.buffer_sink err_buf in
-  let cmd = ["dvisvgm"; "--exact"; "--clipjoin"; "--font-format=woff"; "--bbox=papersize"; "--zoom=1.5"; "--stdin"; "--stdout"] in
+  let cmd = ["dvisvgm"; "--exact"; "--clipjoin"; "--font-format=woff2,autohint"; "--bbox=papersize"; "--relative"; "--optimize"; "--zoom=1.5"; "--stdin"; "--stdout"] in
   try
     Eio.Process.run ~cwd ~stdin: dvi_source ~stdout: svg_sink ~stderr mgr cmd
   with
-- 
2.47.0

Nick Hu <nick.hu@cs.ox.ac.uk>

a month ago

Whoops, I didn't test this thoroughly enough and the changes to the
LaTeX template are completely busted.

I'm still trying to get `standalone` fixed upstream, but I wonder if
there are any better ways.
Going through PDF does seem to work, but I'm not sure dvisvgm pdf input
is necessarily a good option, e.g. see https://github.com/mgieseki/dvisvgm/issues/265

Another option might be to abandon the `standalone` package; is there
any appetite for that?

I think the other changes to dvisvgm flags in this patch are worth
merging still though.