sh: Fix job pointer invalidation with trapsasync

Calling dotrap() can do almost anything, including reallocating the
jobtab array. Convert the job pointer to an index before calling
dotrap() and the

sh: Fix job pointer invalidation with trapsasync

Calling dotrap() can do almost anything, including reallocating the
jobtab array. Convert the job pointer to an index before calling
dotrap() and then restore a proper job pointer afterwards.

PR: 290330
Reported by: bdrewery
Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D53793

show more ...

Remove residual blank line at start of Makefile

This is a residual of the $FreeBSD$ removal.

MFC After: 3 days (though I'll just run the command on the branches)
Sponsored by: Netflix

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

sh/tests: Add a second kind of binary scripts without #!

One of the reasons for git commit
e0f5c1387df23c8c4811f5b24a7ef6ecac51a71a was to make "actually portable
executables" work. Add a test that

sh/tests: Add a second kind of binary scripts without #!

One of the reasons for git commit
e0f5c1387df23c8c4811f5b24a7ef6ecac51a71a was to make "actually portable
executables" work. Add a test that is more like those.

MFC after: 1 week

show more ...

sh: Keep ignored SIGINT/SIGQUIT after set in a background job

If job control is not enabled, a background job (... &) ignores SIGINT and
SIGQUIT, but this can be reverted using the trap builtin in t

sh: Keep ignored SIGINT/SIGQUIT after set in a background job

If job control is not enabled, a background job (... &) ignores SIGINT and
SIGQUIT, but this can be reverted using the trap builtin in the same shell
environment.

Using the set builtin to change options would also revert SIGINT and SIGQUIT
to their previous dispositions.

This broke due to r317298. Calling setsignal() reverts the effect of
ignoresig().

Reported by: bdrewery
MFC after: 1 week

show more ...

sh/tests: Add tests for SIGINT in non-jobc background commands

If job control is not enabled, background commands shall ignore SIGINT and
SIGQUIT, and it shall be possible to override that ignore in

sh/tests: Add tests for SIGINT in non-jobc background commands

If job control is not enabled, background commands shall ignore SIGINT and
SIGQUIT, and it shall be possible to override that ignore in the same shell.

MFC after: 1 week

show more ...

sh: Allow more scripts without #!

Austin Group bugs #1226 and #1250 changed the requirements for shell scripts
without #! (POSIX does not specify #!; this is about the shell execution
when execve(2)

sh: Allow more scripts without #!

Austin Group bugs #1226 and #1250 changed the requirements for shell scripts
without #! (POSIX does not specify #!; this is about the shell execution
when execve(2) returns an [ENOEXEC] error).

POSIX says we shall allow execution if the initial part intended to be
parsed by the shell consists of characters and does not contain the NUL
character. This allows concatenating a shell script (ending with exec or
exit) and a binary payload.

In order to reject common binary files such as PNG images, check that there
is a lowercase letter or expansion before the last newline before the NUL
character, in addition to the check for the newline character suggested by
POSIX.

show more ...

sh: Fix double INTON with vfork

The shell maintains a count of the number of times SIGINT processing has
been disabled via INTOFF, so SIGINT processing resumes when all disables
have enabled again (

sh: Fix double INTON with vfork

The shell maintains a count of the number of times SIGINT processing has
been disabled via INTOFF, so SIGINT processing resumes when all disables
have enabled again (INTON).

If an error occurs in a vfork() child, the processing of the error enables
SIGINT processing again, and the INTON in vforkexecshell() causes the count
to become negative.

As a result, a later INTOFF may not actually disable SIGINT processing. This
might cause memory corruption if a SIGINT arrives at an inopportune time. As
of r360452, it causes the shell to abort when it would unsafely allocate or
free memory in certain ways.

Note that various places such as errors in non-special builtins
unconditionally reset the count to 0, so the problem might still not always
be visible.

PR: 246497
Reported by: jbeich
MFC after: 2 weeks

show more ...

sh: Test that executing various binary files is rejected

If executing a file fails with an [ENOEXEC] error, the shell executes the
file as a shell script, except that this execution may instead resu

sh: Test that executing various binary files is rejected

If executing a file fails with an [ENOEXEC] error, the shell executes the
file as a shell script, except that this execution may instead result in an
error message if the file is binary.

Per a recent Austin Group interpretation, we will need to change this to
allow a concatenation of a shell script and a binary payload. See
Austin Group bugs #1226 and #1250.

MFC after: 1 week

show more ...

MFhead@r344527

Merge ^/head r344178 through r344512.

sh: Add set -o pipefail

The pipefail option allows checking the exit status of all commands in a
pipeline more easily, at a limited cost of complexity in sh itself. It works
similarly to the option

sh: Add set -o pipefail

The pipefail option allows checking the exit status of all commands in a
pipeline more easily, at a limited cost of complexity in sh itself. It works
similarly to the option in bash, ksh93 and mksh.

Like ksh93 and unlike bash and mksh, the state of the option is saved when a
pipeline is started. Therefore, even in the case of commands like
A | B &
a later change of the option does not change the exit status, the same way
(A | B) &
works.

Since SIGPIPE is not handled specially, more work in the script is required
for a proper exit status for pipelines containing commands such as head that
may terminate successfully without reading all input. This can be something
like

(
cmd1
r=$?
if [ "$r" -gt 128 ] && [ "$(kill -l "$r")" = PIPE ]; then
exit 0
else
exit "$r"
fi
) | head

PR: 224270
Relnotes: yes

show more ...

sh: Add test for exported but unset variables

PR: 233545

sh: Fix job pointer invalidation with trapsasync

Calling dotrap() can do almost anything, including reallocating the
jobtab array. Convert the job pointer to an index before calling
dotrap() and the

sh: Fix job pointer invalidation with trapsasync

Calling dotrap() can do almost anything, including reallocating the
jobtab array. Convert the job pointer to an index before calling
dotrap() and then restore a proper job pointer afterwards.

PR: 290330
Reported by: bdrewery
Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D53793

show more ...

Remove residual blank line at start of Makefile

This is a residual of the $FreeBSD$ removal.

MFC After: 3 days (though I'll just run the command on the branches)
Sponsored by: Netflix

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

sh/tests: Add a second kind of binary scripts without #!

One of the reasons for git commit
e0f5c1387df23c8c4811f5b24a7ef6ecac51a71a was to make "actually portable
executables" work. Add a test that

sh/tests: Add a second kind of binary scripts without #!

One of the reasons for git commit
e0f5c1387df23c8c4811f5b24a7ef6ecac51a71a was to make "actually portable
executables" work. Add a test that is more like those.

MFC after: 1 week

show more ...

sh: Keep ignored SIGINT/SIGQUIT after set in a background job

If job control is not enabled, a background job (... &) ignores SIGINT and
SIGQUIT, but this can be reverted using the trap builtin in t

sh: Keep ignored SIGINT/SIGQUIT after set in a background job

If job control is not enabled, a background job (... &) ignores SIGINT and
SIGQUIT, but this can be reverted using the trap builtin in the same shell
environment.

Using the set builtin to change options would also revert SIGINT and SIGQUIT
to their previous dispositions.

This broke due to r317298. Calling setsignal() reverts the effect of
ignoresig().

Reported by: bdrewery
MFC after: 1 week

show more ...

sh/tests: Add tests for SIGINT in non-jobc background commands

If job control is not enabled, background commands shall ignore SIGINT and
SIGQUIT, and it shall be possible to override that ignore in

sh/tests: Add tests for SIGINT in non-jobc background commands

If job control is not enabled, background commands shall ignore SIGINT and
SIGQUIT, and it shall be possible to override that ignore in the same shell.

MFC after: 1 week

show more ...

sh: Allow more scripts without #!

Austin Group bugs #1226 and #1250 changed the requirements for shell scripts
without #! (POSIX does not specify #!; this is about the shell execution
when execve(2)

sh: Allow more scripts without #!

Austin Group bugs #1226 and #1250 changed the requirements for shell scripts
without #! (POSIX does not specify #!; this is about the shell execution
when execve(2) returns an [ENOEXEC] error).

POSIX says we shall allow execution if the initial part intended to be
parsed by the shell consists of characters and does not contain the NUL
character. This allows concatenating a shell script (ending with exec or
exit) and a binary payload.

In order to reject common binary files such as PNG images, check that there
is a lowercase letter or expansion before the last newline before the NUL
character, in addition to the check for the newline character suggested by
POSIX.

show more ...

sh: Fix double INTON with vfork

The shell maintains a count of the number of times SIGINT processing has
been disabled via INTOFF, so SIGINT processing resumes when all disables
have enabled again (

sh: Fix double INTON with vfork

The shell maintains a count of the number of times SIGINT processing has
been disabled via INTOFF, so SIGINT processing resumes when all disables
have enabled again (INTON).

If an error occurs in a vfork() child, the processing of the error enables
SIGINT processing again, and the INTON in vforkexecshell() causes the count
to become negative.

As a result, a later INTOFF may not actually disable SIGINT processing. This
might cause memory corruption if a SIGINT arrives at an inopportune time. As
of r360452, it causes the shell to abort when it would unsafely allocate or
free memory in certain ways.

Note that various places such as errors in non-special builtins
unconditionally reset the count to 0, so the problem might still not always
be visible.

PR: 246497
Reported by: jbeich
MFC after: 2 weeks

show more ...

sh: Test that executing various binary files is rejected

If executing a file fails with an [ENOEXEC] error, the shell executes the
file as a shell script, except that this execution may instead resu

sh: Test that executing various binary files is rejected

If executing a file fails with an [ENOEXEC] error, the shell executes the
file as a shell script, except that this execution may instead result in an
error message if the file is binary.

Per a recent Austin Group interpretation, we will need to change this to
allow a concatenation of a shell script and a binary payload. See
Austin Group bugs #1226 and #1250.

MFC after: 1 week

show more ...

MFhead@r344527

Merge ^/head r344178 through r344512.

sh: Add set -o pipefail

The pipefail option allows checking the exit status of all commands in a
pipeline more easily, at a limited cost of complexity in sh itself. It works
similarly to the option

sh: Add set -o pipefail

The pipefail option allows checking the exit status of all commands in a
pipeline more easily, at a limited cost of complexity in sh itself. It works
similarly to the option in bash, ksh93 and mksh.

Like ksh93 and unlike bash and mksh, the state of the option is saved when a
pipeline is started. Therefore, even in the case of commands like
A | B &
a later change of the option does not change the exit status, the same way
(A | B) &
works.

Since SIGPIPE is not handled specially, more work in the script is required
for a proper exit status for pipelines containing commands such as head that
may terminate successfully without reading all input. This can be something
like

(
cmd1
r=$?
if [ "$r" -gt 128 ] && [ "$(kill -l "$r")" = PIPE ]; then
exit 0
else
exit "$r"
fi
) | head

PR: 224270
Relnotes: yes

show more ...