Repair "make man.cgi" which got accidentally broken in the previous
commit to the Makefile. The man.cgi binary now uses roff_escape.o, too.

Make roff_expand() parse left-to-right rather than right-to-left.
Some escape sequences have side effects on global state, implying
that the order of evaluation matters. For example, this fixes the

Make roff_expand() parse left-to-right rather than right-to-left.
Some escape sequences have side effects on global state, implying
that the order of evaluation matters. For example, this fixes the
long-standing bug that "\n+x\n+x\n+x" after ".nr x 0 1" used to
print "321"; now it correctly prints "123".

Right-to-left parsing was convenient because it implicitly handled
nested escape sequences. With correct left-to-right parsing, nesting
now requires an explicit implementation, here solved as follows:
1. Handle nested expanding escape sequences iteratively.
When finding one, expand it, then retry parsing the enclosing escape
sequence from the beginning, which will ultimately succeed as soon
as it no longer contains any nested expanding escape sequences.
2. Handle nested non-expanding escape sequences recursively.
When finding one, the escape sequence parser calls itself to find
the end of the inner sequence, then continues parsing the outer
sequence after that point.

This requires the mandoc_escape() function to operate in two different
modes. The roff(7) parser uses it in a mode where it generates
diagnostics and may return an expansion request instead of a parse
result. All other callers, in particular the formatters, use it
in a simpler mode that never generates diagnostics and always returns
a definite parsing result, but that requires all expanding escape
sequences to already have been expanded earlier. The bulk of the
code is the same for both modes.
Since this required a major rewrite of the function anyway, move
it into its own new file roff_escape.c and out of the file mandoc.c,
which was misnamed in the first place and lacks a clear focus.

As a side benefit, this also fixes a number of assertion failures
that tb@ found with afl(1), for example "\n\\\\*0", "\v\-\\*0",
and "\w\-\\\\\$0*0".

As another side benefit, it also resolves some code duplication
between mandoc_escape() and roff_expand() and centralizes all
handling of escape sequences (except for expansion) in roff_escape.c,
hopefully easing maintenance and feature improvements in the future.

While here, also move end-of-input handling out of the complicated
function roff_expand() and into the simpler function roff_parse_comment(),
making the logic easier to understand.

Since this is a major reorganization of a central component of
mandoc(1), stability of the program might slightly suffer for a few
weeks, but i believe that's not a problem at this point of the
release cycle. The new code already satisfies the regression suite,
but more tweaking and regression testing to further improve the
handling of various escape sequences will likely follow in the near
future.

show more ...

Split tagging into a validation part including prioritization
in tag.{h,c} and {mdoc,man}_validate.c
and into a formatting part including command line argument checking
in term_tag.{h,c}, html.c, and

Split tagging into a validation part including prioritization
in tag.{h,c} and {mdoc,man}_validate.c
and into a formatting part including command line argument checking
in term_tag.{h,c}, html.c, and {mdoc|man}_{term|html}.c.

Immediate functional benefits include:
* Improved prioritization of automatic tags for .Em and .Sy.
* Avoiding bogus automatic tags when .Em, .Fn, or .Sy are explicitly tagged.
* Explicit tagging of .Er and .Fl now works in HTML output.
* Automatic tagging of .IP and .TP now works in HTML output.
But mainly, this patch provides clean earth to build further improvements on.

Technical changes:
* Main program: Write a tag file for ASCII and UTF-8 output only.
* All formatters: There is no more need to delay writing the tags.
* mdoc(7)+man(7) formatters: No more need for elaborate syntax tree inspection.
* HTML formatter: If available, use the "string" attribute as the tag.
* HTML formatter: New function to write permalinks, to reduce code duplication.

Style cleanup in the vicinity while here:
* mdoc(7) terminal formatter: To set up bold font for children,
defer to termp_bold_pre() rather than calling term_fontpush() manually.
* mdoc(7) terminal formatter: Garbage collect some duplicate functions.
* mdoc(7) HTML formatter: Unify <code> handling, delete redundant functions.
* Where possible, use switch statements rather than if cascades.
* Get rid of some more Yoda notation.

The necessity for such changes was first discussed with kn@, but i didn't
bother him with a request to review the resulting -673/+782 line patch.

show more ...

man.cgi also needs to link arch.o; forgotten in rev. 1.116

When the -S option is given to man(1) and the requested manual page
name is not found and the requested architecture is unknown, complain
about the architecture rather than about the manual page name

When the -S option is given to man(1) and the requested manual page
name is not found and the requested architecture is unknown, complain
about the architecture rather than about the manual page name:

$ man -S vax cpu
man: Unknown architecture "vax".
$ man -S sparc64 foobar
man: No entry for foobar in the manual.

Friendlier error message suggested by jmc@, who also OK'ed the patch.

show more ...

Remove -S from install commands

As of usr.bin/xinstall/install.c revision 1.68, -S is a no-op and
install(1) will always create files safely, thus clean the option usage
from the tree.

Diff from La

Remove -S from install commands

As of usr.bin/xinstall/install.c revision 1.68, -S is a no-op and
install(1) will always create files safely, thus clean the option usage
from the tree.

Diff from Lauri Tirkkonen <lotheac at iki dot fi>, thanks.

show more ...

install /usr/share/misc/mandoc.css
users of -T html normally need this file,
so the source tree should not be the only place to get it
pointed out by millert@, OK millert@ deraadt@ tedu@

Major cleanup; may imply minor changes in edge cases of error reporting.

Finally, drop support for the run-time configurable mandocmsg()
callback. It was over-engineered from the start, never used

Major cleanup; may imply minor changes in edge cases of error reporting.

Finally, drop support for the run-time configurable mandocmsg()
callback. It was over-engineered from the start, never used for
anything in a decade, and repeatedly caused maintenance headaches.

Consolidate reporting infrastructure into two files, mandoc.h and
mandoc_msg.c, mopping up the bits and pieces that were scattered
around main.c, read.c, mandoc_parse.h, libmandoc.h, the prototypes
of four parsing-related functions, and both parser structs.

show more ...

Cleanup, no functional change:
Split the top level parser interface out of the utility header
mandoc.h, into a new header mandoc_parse.h, for use in the main
program and in the main parser only.
Move

Cleanup, no functional change:
Split the top level parser interface out of the utility header
mandoc.h, into a new header mandoc_parse.h, for use in the main
program and in the main parser only.
Move enum mandoc_os into roff.h because struct roff_man is the
place where it is stored.
This allows removal of mandoc.h from seven files in low-level
parsers and in formatters.

show more ...

Basic reporting of .Xrs to manual pages that don't exist
in the base system, inspired by mdoclint(1).

We are able to do this because (1) the -mdoc parser, the -Tlint validator,
and the man(1) manual

Basic reporting of .Xrs to manual pages that don't exist
in the base system, inspired by mdoclint(1).

We are able to do this because (1) the -mdoc parser, the -Tlint validator,
and the man(1) manual page lookup code are all in the same program
and (2) the mandoc.db(5) database format allows fast lookup.

Feedback from, previous versions tested by, and OK jmc@.

A few features will be added to this in the tree, step by step.

show more ...

Fix the man.cgi target after the recent file additions;
this didn't break the build because it's not built by default.

Basic implementation of the roff(7) .ta (define tab stops) request.
This is the first feature made possible by the parser reorganization.
Improves the formatting of the SYNOPSIS in many Xenocara GL m

Basic implementation of the roff(7) .ta (define tab stops) request.
This is the first feature made possible by the parser reorganization.
Improves the formatting of the SYNOPSIS in many Xenocara GL manuals.
Also important for ports, as reported by many, including naddy@.

show more ...

Move handling of the roff(7) .ft request from the man(7)
modules to the new roff(7) modules. As a side effect,
mdoc(7) now handles .ft, too. Of course, do not use that.

Start roff formatter modules for HTML and termininal output,
used by both the mdoc and man formatters, with the ultimate
goal of reducing code duplication between the two macro formatters.
Made possi

Start roff formatter modules for HTML and termininal output,
used by both the mdoc and man formatters, with the ultimate
goal of reducing code duplication between the two macro formatters.
Made possible by the parser unification.
Add the first formatting function (for the .br request).

show more ...

Parser unification: use nice ohashes for all three request and macro tables;
no functional change, minus two source files, minus 200 lines of code.

new -mdoc -Tmarkdown output mode; OK millert@ reyk@ tb@;
thanks to reyk@ and to Vsevolod at FreeBSD for suggesting it

Add the style sheet used on man.openbsd.org to the source tree.

It is NOT used during "make build" and NOT installed, but it is
helpful for people trying to read and understand html.c and mdoc_html.

Add the style sheet used on man.openbsd.org to the source tree.

It is NOT used during "make build" and NOT installed, but it is
helpful for people trying to read and understand html.c and mdoc_html.c,
and maybe even for users of mandoc -Thtml.

OK deraadt@

show more ...

Remove the dependency on SQLite without loss of functionality.
Drop the obsolete names_check() now that we deleted MLINKS.
Run "doas makewhatis" after compiling and installing this.

Earlier version

Remove the dependency on SQLite without loss of functionality.
Drop the obsolete names_check() now that we deleted MLINKS.
Run "doas makewhatis" after compiling and installing this.

Earlier version tested by jmc@ and jturner@;
"commit it all" deraadt@ "commit and dodge" krw@

show more ...

Simplify the code and the server setup by deleting the pseudo-manpath
"mandoc" that was used for man.cgi(8) documentation and by assuming
that the apropos(1) and man.cgi(8) manuals are simply install

Simplify the code and the server setup by deleting the pseudo-manpath
"mandoc" that was used for man.cgi(8) documentation and by assuming
that the apropos(1) and man.cgi(8) manuals are simply installed in
the default manpath. Even though man.cgi(8) is not installed by
default when installing OpenBSD, it is easy to copy it into the
default manpath used for man.cgi(8).

Idea found when considering a question asked by wrant dot com.

show more ...

for some time now mandoc has not required MLINKS to function
correctly - logically complete that now by removing MLINKS from base;

authors need only to ensure there is an entry in NAME for any funct

for some time now mandoc has not required MLINKS to function
correctly - logically complete that now by removing MLINKS from base;

authors need only to ensure there is an entry in NAME for any function/
util being added. MLINKS will still work, and remain for perl to ease
upgrades;

ok nicm (curses) bcook (ssl)
ok schwarze, who provided a lot of feedback and assistance
ok tb natano jung

show more ...

use the new function man_validate() here, too

In order to become able to generate syntax tree nodes on the roff(7)
level, validation must be separated from parsing and rewinding.
This first big step moves calling of the mdoc(7) post_*() function

In order to become able to generate syntax tree nodes on the roff(7)
level, validation must be separated from parsing and rewinding.
This first big step moves calling of the mdoc(7) post_*() functions
out of the parser loop into their own mdoc_validate() pass, while
using a new mdoc_state() module to make syntax tree state handling
available to both the parser loop and the validation pass.

show more ...

Reduce the amount of code by moving the three copies of the ohash
callback functions into one common place, preparing for the use of
ohash for some additional purposes. No functional change.

Do not fork and exec gunzip(1), just link with libz instead.
As discussed with deraadt@, that's cleaner and will help tame(2).
Something like this was also suggested earlier by bapt at FreeBSD.
Minus

Do not fork and exec gunzip(1), just link with libz instead.
As discussed with deraadt@, that's cleaner and will help tame(2).
Something like this was also suggested earlier by bapt at FreeBSD.
Minus 50 lines of code, deleting one interface function (mparse_wait),
no functional change intended.

show more ...

Initial, still somewhat experimental implementation to leverage
less(1) -T and :t ctags(1)-like functionality to jump to the
definitions of various terms inside manual pages.
To be polished in the tr

Initial, still somewhat experimental implementation to leverage
less(1) -T and :t ctags(1)-like functionality to jump to the
definitions of various terms inside manual pages.
To be polished in the tree, so bear with me and report issues.

Technically, if less(1) is used as a pager, information is collected
by the mdoc(7) terminal formatter, first stored using the ohash
library, then ultimately written to a temporary file which is passed
to less via -T. No change intended for other output formatters or
when running without a pager.

Based on an idea from Kristaps using feedback from many, in particular
phessler@ nicm@ millert@ halex@ doug@ kspillner@ deraadt@.

show more ...