sh(1): bring sh from freebsd

I used freebsd@19bed3ae2eb256a2c68dabbcb77eff2835d569de.

* Write absolute path in command -vV and type
* Keep ignored SIGINT/SIGQUIT after set in a background job
* pri

sh(1): bring sh from freebsd

I used freebsd@19bed3ae2eb256a2c68dabbcb77eff2835d569de.

* Write absolute path in command -vV and type
* Keep ignored SIGINT/SIGQUIT after set in a background job
* print a newline when ^D quits sh
* Do not ignore INTOFF during a trap
* Fix description of the "\$" sequence for PS1
* Allow more scripts without #!
* Fix double INTON with vfork
* Assert INTOFF rather than applying it in ck*
* Remove remnants to compile out fc completely
* Fix a "may be unused" warning on mips-gcc
* remove duplicate el definition
* fix read builtin on 32-bit systems
* Improve performance of "read" built-in command when using a seekable
* Fix rare memory leak with SIGINT
* Do not use our custom completion function, it is not needed anymore
* Add set -o pipefail
* Send normal output from bind builtin to stdout
* Restore $((x)) error checking after fix for $((-9223372036854775808))
* Make sh(1) support \u in PS1. This removes one fork/exec on
interactive
* Send libedit "ferr" output to fd 2
* Fix an edge case when parsing large numbers which resulted in
inconsistent
* Do not place exported but unset variables into the environment
* Make sh(1) collapse $HOME into "~" in PS1.
* Remove -c string from set builtin documentation
* Fix ${param?} default error message
* Use 126 and 127 exit status for failures opening a script
* Unify EXERROR and EXEXEC
* Use exitstatus instead of exerrno to pass EXEXEC status
* Fix formal overflow in pointer arithmetic
* Describe how to prevent *.core files from being created using ulimit.
* Don't treat % specially in CDPATH
* Don't use padvance() for MAIL/MAILPATH
* Split CNL syntax category to avoid a check on state[level].syntax
* Allow unquoted newlines in word in ${param+word} etc.
* Don't have [ match any [[:class:]]
* Allow overriding VTABSIZE at compile-time.
* Refactor job status printing, preparing for -o pipefail and similar
* Move various structs from jobs.h to jobs.c
* Fix a few speelling errors
* Don't leak wait* implementation details from jobs.c
* Markup and spelling fixes
* Fix mandoc -Tlint warnings in bin/
* General further adoption of SPDX licensing ID tags.
* Remove broken #ifdef NOHACK code (related to sh -c).
* Ignore error when cd writes the directory actually switched to.
* Enable interrupts before executing EXIT trap and doing final flush.
* Call fc -e editor with interrupts enabled.
* Make sure to process SIGINT if SETINTON re-enables processing.
* Allow defining nofork builtins from builtins.def and move always-safe
ones there.
* Fix INTOFF leak when a redirection on a compound command fails.
* Keep output buffer across builtins.
* Ensure memout.bufsize matches allocated buffer, if it exists.
* Simplify output buffering.
* Fix '-' from quoted arithmetic in case/glob pattern range.
* Fix INTOFF leak after a builtin with different locale settings.
* Update TOUR and comments for some code changes, some of them old.
* Simplify handling of newlines in command substitution.
* Simplify setinteractive().
* Fix use after free when resetting an in-use alias.
* Fix unalias -a while an alias is currently in use.
* Reduce size of limits table.
* Remove an XXX comment: it is normal for builtins to use argptr.
* Remove global state from collate_range_cmp().
* Don't scan word twice in ${param=word}.
* Don't scan word twice in ${param#%##%%word}.
* Don't scan word twice in ${param+-word}.
* Remove unused function declaration for arith_lex_reset().
* Remove unused return value of subevalvar_misc().
* Get rid of global variable argbackq.
* Fix executing wrong command with ${unsetvar#$(cmdsubst)}$(cmdsubst).
* Fix crash if a -T trap is taken during command substitution.
* bin: normalize paths using SRCTOP-relative paths or :H when possible
* Remove empty Li
* Do not import IFS's value from the environment.
* Fix a clang warning.
* Path generation was not according to collate
* Back out non-collating [a-z] ranges.
* After removing collation for [a-z] ranges in r302512, do it here too.
* Improve descriptions in 'ulimit -a' output.
* Clarify the explanations for the hostname and FQDN entries.
* Match the descriptions of the \H and \h prompt string sequences to reality.
* Hopefully fix Coverity CID 1008328 (Out-of-bounds write) in /bin/sh.
* Handle empty hostname and $PWD when building prompt
* Write LINENO value to stack string directly.
* Merge the projects/release-pkg branch to head.
* Simplify code by removing variable bracketed_name.
* replace 0 with NULL for pointers.
* Fix use-after-free if a trap replaces itself.
* Fix copying uninitialized field 'special'.
* Fix handling of umtxp resource limit in sh(1)/ulimit(1), limits(1), add
* Avoid out-of-bounds access in setoptionbyindex() for 'set -o nolog'.
* Fix some dead stores.
* Remove a redundant STPUTC check.
* Don't trust that signal descriptions fit within 49 bytes.
* Fix set -v and multiline history after r295825.
* Don't hash alias name when there are no aliases.
* Optimize setprompt(0).
* Remove unnecessary flushouts while reading script.
* Rework code to remove '\0' from shell input.
* Don't allocate a redirtab if there are no redirections.
* Document 'cd -'.
* Constify internal representation in expand.c.
* Remove a global variable from cd.c.
* Use OLDPWD shell variable for 'cd -'.
* Clean a readonly local, even if the variable does not exist outside.
* Simplify some code related to positional parameters.
* Update associated state when restoring locals while leaving a function.
* Reduce size of options table.
* Ensure OPTIND=1 in subshell without forking does not affect outer env.
* Reduce size of builtins table.
* Remove redundant CTLQUOTEMARK checks.
* Reindent expandmeta().
* Perform pathname generation during the first expansion phases.
* Perform IFS splitting during the first expansion phases.
* Split subevalvar() in #/##/%/%% and =/? parts.
* sh/mknodes: Close files and check for errors when writing.
* Fix use-after-free when attempting to modify a read-only variable.
* Remove global state from nodes.c.
* Fix ""$@, which should not use the special case for "$@".
* Avoid copying argv for simple commands.
* Make struct arglist an array instead of a linked list.
* Allow empty << EOF markers.
* Fix read with escaped IFS characters at the end.
* Add set -o nolog.
* Fix out of bounds read when there is no ] after a [:class:].
* Don't create bad parse result when postponing a bad substitution error.
* Avoid negative character values from $'\Uffffffff' etc.
* Backslash-newline should not affect field splitting in read builtin.
* When setting option via long name, don't go via letter.
* libedit has supported multibyte encodings for a while.
* Fix some arithmetic undefined behaviour.

Taken-from: FreeBSD

show more ...

sh - Sync to FreeBSD d038ee76 part 2/2 (c)

* Renumber copyright lines after the advertising clause was taken out.

sh - Sync to FreeBSD d038ee76 part 1/2

* Sync to FreeBSD d038ee76 25 Apr 2015.

Correct BSD License clause numbering from 1-2-4 to 1-2-3.

Apparently everyone's doing it:
http://svnweb.freebsd.org/base?view=revision&revision=251069

Submitted-by: "Eitan Adler" <lists at eitanadl

Correct BSD License clause numbering from 1-2-4 to 1-2-3.

Apparently everyone's doing it:
http://svnweb.freebsd.org/base?view=revision&revision=251069

Submitted-by: "Eitan Adler" <lists at eitanadler.com>

show more ...

Remove advertising header from all userland binaries.

From: Eitan Adler <lists@eitanadler.com>

sh: Sync with FreeBSD:

* Expand assignment-like words specially for export/readonly/local.
* Remove unused variables.
* Reset pendingsigs before checking pending traps, not after.
* Track continued

sh: Sync with FreeBSD:

* Expand assignment-like words specially for export/readonly/local.
* Remove unused variables.
* Reset pendingsigs before checking pending traps, not after.
* Track continued jobs (even if not continued by bg or fg).
* Inline waitproc() into its only caller.
* Fix EINTR race condition in "wait" and "set -T" using sigsuspend().
* use setinputfile instead of open/setinputfd.
* Prefer internal nextopt() to libc getopt().
* Use C99 flexible array instead of accessing array beyond bounds.
* Fix two issues when an alias is redefined.
* Detect and flag write errors on stdout in builtins.
* Prefer strsignal() to accessing sys_siglist directly.
* Don't lose $? when backquoted command ends with semicolon or newline.
* Fix crash when parsing '{ } &'.
* Pass $? to command substitution containing compound/multiple commands.
* Simplify cd-hash interaction.
* Remove mkinit's initialization routine.
* Move some stackmarks to fix high memory usage in some loops.
* Show negated commands (!) in jobs output.
* Prefer our character classification functions to <ctype.h>.
* Expand here documents in the current process.
* Do not test for digit_contig in mksyntax.
* Simplify mksyntax
* Fix a crash with the stackmark code.
* If a SIGINT or SIGQUIT interrupts "wait", return status 128+sig.
* When executing a trap, keep exit status along with evalskip.
* Update $FreeBSD$ to include SVN id.

Obtained-from: FreeBSD

show more ...

sh: Fix swapped INTON/INTOFF.

A possible consequence of this bug was a memory leak if SIGINT arrived
during a 'set' command (listing variables).

Obtained-from: FreeBSD 231001

sh: Use vfork in a few common cases.

This uses vfork() for simple commands and command substitutions containing a
single simple command, invoking an external program under certain conditions
(no red

sh: Use vfork in a few common cases.

This uses vfork() for simple commands and command substitutions containing a
single simple command, invoking an external program under certain conditions
(no redirections or variable assignments, non-interactive shell, no job
control). These restrictions limit the amount of code executed in a vforked
child.

There is a large speedup (for example 35%) in microbenchmarks. The
difference in buildkernel is smaller (for example 0.5%) but still
statistically significant. See
http://lists.freebsd.org/pipermail/freebsd-hackers/2012-January/037581.html
for some numbers.

The use of vfork() can be disabled by setting a variable named
SH_DISABLE_VFORK.

Obtained-from: FreeBSD 230998

show more ...

Initial import of binutils 2.22 on the new vendor branch

Future versions of binutils will also reside on this branch rather
than continuing to create new binutils branches for each new version.

sh: Skip vars with invalid names in "set", "export -p", "readonly -p".

This ensures the output of these commands is valid shell input.

Obtained-from: FreeBSD 223183

sh: Fix duplicate prototypes for builtins.

Have mkbuiltins write the prototypes for the *cmd functions to
builtins.h instead of builtins.c and include builtins.h in more
.c files instead of duplicat

sh: Fix duplicate prototypes for builtins.

Have mkbuiltins write the prototypes for the *cmd functions to
builtins.h instead of builtins.c and include builtins.h in more
.c files instead of duplicating prototypes for *cmd functions in
other headers.

Obtained-from: FreeBSD 223060

show more ...

sh: Save/restore changed variables in optimized command substitution.

In optimized command substitution, save and restore any variables
changed by expansions (${var=value} and $((var=assigned))), in

sh: Save/restore changed variables in optimized command substitution.

In optimized command substitution, save and restore any variables
changed by expansions (${var=value} and $((var=assigned))), instead of
trying to determine if an expansion may cause such changes.

If $! is referenced in optimized command substitution, do not cause jobs
to be remembered longer.

This fixes $(jobs $!) again, simplifies the man page and shortens the
code.

Obtained-from: FreeBSD 223024

show more ...

sh: Minor optimization to output from ulimit/export/readonly.

Obtained-from: FreeBSD SVN rev 221975

sh: Add \u/\U support (in $'...') for UTF-8.

Note that \u/\U are processed using the locale that was active when the
shell started. This is necessary to avoid behaviour that depends on the
parse/exe

sh: Add \u/\U support (in $'...') for UTF-8.

Note that \u/\U are processed using the locale that was active when the
shell started. This is necessary to avoid behaviour that depends on the
parse/execute split (for example when placing braces around an entire
script). Therefore, UTF-8 encoding is implemented manually.

Obtained-from: FreeBSD SVN rev 221669

show more ...

sh: Fix typos in comments.

sh: Optimize variable code by storing the length of the name.

Obtained-from: FreeBSD SVN rev 221668

sh: Track if the current locale's charset is UTF-8 or not.

Obtained-from: FreeBSD SVN rev 221559

sh(1): Deconstify unsetvar()'s argument when passing it to change_env().

Sync libc/stdlib with FreeBSD (ignoring jemalloc, pts, and gdtoa):

-Add a64l(), l64a(), and l64a_r() XSI extentions. These functions
convert between a 32-bit integer and a radix-64 ASCII string.

-

Sync libc/stdlib with FreeBSD (ignoring jemalloc, pts, and gdtoa):

-Add a64l(), l64a(), and l64a_r() XSI extentions. These functions
convert between a 32-bit integer and a radix-64 ASCII string.

-Replace some syscalls with libc version.

-Remove advertising clause.

-alloca() cannot check if the allocation is valid; mention the
consequences.

-Include some verbage about not calling exit() from functions registered
by atexit().

-Use pthread mutexes where possible instead of libc spinlocks.

-Significantly reduce the memory leak as noted in the BUGS section of
setenv(3) by tracking the size of the memory allocated instead of using
strlen() on the current value.

-Prefer setenv() instead of putenv().

-Convert *env() calls to POSIX:
-unsetenv returns an int.
-putenv takes a char * instead of const char *.
-putenv no longer makes a copy of the input string.
-errno is set appropriately. Exceptions involve bad environ
variable and internal initialization code. These both set errno
to EFAULT.

-Make getopt_long() more GNU compatible and sync up with OpenBSD's
version.

-POSIX clearly states that getsubopt() should be declared in <stdlib.h>
not in <unistd.h>

-Use size_t to avoid overflow when sorting arrays larger than 2 GB in
heapsort() and qsort().

-Add new implementations of insque() and remque() which conform to
IEEE Std 1003.1-2001.

-Add qsort_r() for functions that need to be reentrant.

-Improve radixsort()'s preformance when sorting strings with common
prefixes.

-Use 'uint32_t' instead of 'long' when a 32-bit integer is intended in
random(3).

-Rearrange <stdlib.h> in a more logical order based on visibility.

-Move getsubopt()'s prototype to <stdlib.h> (standards).

-Make an internal _getprogname() that is used only inside
libc. For libc, getprogname(3) is a weak symbol in case a
function of the same name is defined in userland.

show more ...

sh: export the default PATH instead of keeping it private

sh specifies a default setting for PATH if none is set in the
environment, but up to now it didn't export this setting. This leads to
stran

sh: export the default PATH instead of keeping it private

sh specifies a default setting for PATH if none is set in the
environment, but up to now it didn't export this setting. This leads to
strange effects, like sh being able to execute binaries in /usr/sbin,
but which(1) not being able to find them. Export PATH to create more
symmetry and harmony.

show more ...

sh: use _PATH_DEFPATH instead of hand-rolling the default path

Use volatile where appropriate.

Partially taken from NetBSD.

Spelling errors.

POSIX demands that set's output (when invoked without arguments) should be
sorted. Sort the variables before printing.

Provide a reasonable error message for `export -p var'.
Use

Spelling errors.

POSIX demands that set's output (when invoked without arguments) should be
sorted. Sort the variables before printing.

Provide a reasonable error message for `export -p var'.
Use argv rather than argptr since getopt() is used here.

Obtained-from: FreeBSD

show more ...

Use -s to flag POSIX's "special built-in" utilities in builtins.def. Add a
new member to struct builtincmd and set it to 1 if -s was specified. This
is done because there are cases where special bu

Use -s to flag POSIX's "special built-in" utilities in builtins.def. Add a
new member to struct builtincmd and set it to 1 if -s was specified. This
is done because there are cases where special builtins must be treated
differently from other builtins.

Implement some of the differences between special built-ins and other builtins
demanded by POSIX.
- A redirection error is only fatal (meaning the execution of a shell script is
terminated) for special built-ins. Previously it was fatal for all shell
builtins, causing problems like the one reported in FreeBSD PR 88845.
- Variable assignments remain in effect for special built-ins.
- Option or operand errors are only fatal for special built-ins.

Add the times builtin. It reports the user and system time for the shell
itself and its children. Instead of calling times() (as implied by POSIX) this
implementation directly calls getrusage() to get the times because this is more
convenient.

Print pointers with %p rather than casting them to long.

Replace home-grown dup2() implementation with actual dup2() calls. This
should slightly reduce the number of system calls in critical portions of
the shell, and select a more efficient path through the fdalloc code.

Implement the PS4 variable which is defined by the POSIX User Portability
Utilities option. Its value is printed at the beginning of the line if tracing
(-x) is active. PS4 defaults to the string "+ " which is compatible with the
old behaviour to always print "+ ".

Don't crash on "<cmd> | { }".

Remove some white space at EOL.

Add the POSIX options -v and -V to the 'command' builtin. Both describe the
type of their argument, if it is a shell function, an alias, a builtin, etc.
-V is more verbose than -v.

Do not assume there is only a space between #define and the macro name
when grepping for JOBS in mkbuiltins

Obtained-from: FreeBSD

show more ...

Fix most compiler warnings. The last warnings at WARNS6 will need to be
fixed in usr.bin/printf.

Remove nested externs.

Rename local variables shadowing globals.

Some changes taken from NetBSD.