In one of the threads on cfe-commits I was asked by Tobias to provide
a rationale for adding more HTML-related validation facilities in
Clang's comment parsing.
HTML is an indivisible part of Doxygen syntax. It is impossible to
parse Doxygen without not only merely parsing, but doing semantic
analysis on HTML tags.
For example, paragraph splitting is more complex than just finding an
Somehow (I am not saying that the rules make sense, it is just what it
is), Doxygen interprets this like this:
1. for f():
An unterminated <b> tag started to span multiple paragraphs.
2. for g():
An empty line between table tags made Doxygen add a second paragraph
to a table cell that had its content clearly specified.
Judging just from these two simple examples, it is clear that in order
to parse embedded HTML in Doxygen so that the output actually takes
the HTML markup into account, requires *semantic* analysis of HTML
tags, and transformation of the HTML AST.
It is a non-trivial amount of work to implement this, and I did look
for HTML libraries that could help us in doing so. libtidy  is a
nice one, except that I got the impression that it is "stabilized to
the point of becoming unmaintained" -- there are no releases, code is
available through cvs only, and it was not updated for HTML5. There
is an experimental HTML5 fork of it , which was not updated for
more than 2 years, and probably does not correspond to the current
But even if libtidy did completely support HTML5, its interface is not
suitable for fine-grained parsing and AST manipulation that we need.
The interface accepts only complete HTML docs for parsing, while Clang
deals with fragments. Constructing the HTML AST though libtidy just
to figure out what the tag name is is not going to deliver good
Apart from libtidy, I did not find other *lightweight* libraries (not
HTML rendering engines) that provide low-level manipulation that we
But parsing and doing semantic analysis correctly is only half of the
story. Sanitizing the output is important, otherwise Clang clients
can not use the HTML parts of comments, and have to re-do the parsing
work, now with the intent of sanitizing the output. I think it is
reasonable to state that almost all clients want the output as
I hope this addresses everyone's concerns.