REGRESSION: shellshock patch rejects valid function names

[Eric Blake]

[adding Austin Group]

Yes, it's an application requirement. Regardless, all the versions of bash
we're talking about here reject non-identifiers.

I'm still trying to find that line in the actual POSIX spec.

"The function is named fname; the application shall ensure that it is a
name (see XBD Name) and that it is not the name of a special built-in utility."
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_05

Note that this particular requirement is on "the application" (aka the
shell script defining the function) and NOT on "the implementation" (aka
bash as the shell that is tracking the function definition). There are
a number of places where there are requirements on a strictly conforming
application, where if they violate that requirement, then they have
subjected themselves to unspecified behavior, and that the shell can do
whatever it deems makes the most sense. In other words, I _don't_ read
this sentence of POSIX as requiring that 'bash -o posix' must reject
arbitrary function names, only that a shell script trying to conform to
POSIX must not attempt to define a function with such a name.

I would expect something like "It shall be an error if fname is the name
of a special built-in utility", as _that_ would be a hard requirement on
bash, not just the application. Maybe we have a bug in the POSIX spec
for a missing requirement.

[Stephane Chazelas]

2014-09-29 09:04:00 -0600, Eric Blake:
[...]

Post by Eric Blake

"The function is named fname; the application shall ensure that it is a
name (see XBD Name) and that it is not the name of a special built-in utility."
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_05

Note that this particular requirement is on "the application" (aka the
shell script defining the function) and NOT on "the implementation" (aka
bash as the shell that is tracking the function definition). There are
a number of places where there are requirements on a strictly conforming
application, where if they violate that requirement, then they have
subjected themselves to unspecified behavior, and that the shell can do
whatever it deems makes the most sense. In other words, I _don't_ read
this sentence of POSIX as requiring that 'bash -o posix' must reject
arbitrary function names, only that a shell script trying to conform to
POSIX must not attempt to define a function with such a name.
I would expect something like "It shall be an error if fname is the name
of a special built-in utility", as _that_ would be a hard requirement on
bash, not just the application. Maybe we have a bug in the POSIX spec
for a missing requirement.

I agree the requirement is on the application, and I can't see
why POSIX should force a shell to reject a function whose name
doesn't contain a valid identifier.

() are used for (...), $(...), $((...)) and in "case", but
outside of that should be quoted, so I suppose any shell can do
whatever they want with them. ksh/bash/zsh already use it for
globs or arrays for instance.

The function keyword is also reserved with undefined behavior,
so one shell could very well do:


function with weird name 'blah = blah' [
my function definition
]

Or:

'blah blah'() {
my function
}

(like zsh does).

Or:

()'blah blah'={ my function }

Or:

functions[blah blah]='my function'
(as in zsh, and again unspecified behaviour by POSIX).

None of which are valid POSIX shell syntax, so POSIX shells can
do whatever they want with them.

[Dan Douglas]

Just a few points to add.

Post by Stephane Chazelas
[...]

"The function is named fname; the application shall ensure that it is a
name (see XBD Name) and that it is not the name of a special built-in utility."

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_05

This doesn't normally matter because POSIX requires special builtins to take
precedence over functions during command search, so even if you have such a
function defined it is impossible to call. Bash doesn't use the correct search
order however.

Mksh has the reverse bug. It allows defining the function (wrongly) but then
calls the special builtin anyway (correctly).

Another bug is in ksh93 whose `builtin` allows disabling special builtins
(which according to the manual, shouldn't work).

$ ksh -c 'builtin -d set; function set { echo test; }; set'
test

Bash's "enable" correctly disallows that.

Post by Stephane Chazelas
I agree the requirement is on the application, and I can't see
why POSIX should force a shell to reject a function whose name
doesn't contain a valid identifier.
...

Another thing you can do in bash is bypass its command name check by using a
null zeroth word.

$ { function } { echo test; }; <() }; }
test

Ordinarily } would be uncallable, but apparently since bash only checks the
command name of the first word, calling with e.g. `<() }` or `$() }` works.

[Andreas Schwab]

Post by Dan Douglas
Another thing you can do in bash is bypass its command name check by using a
null zeroth word.
$ { function } { echo test; }; <() }; }
test
Ordinarily } would be uncallable,

Only literal } is special, you can use \} instead, or store it in a
variable.

Andreas.

[Dan Douglas]

Post by Andreas Schwab

Post by Dan Douglas
Another thing you can do in bash is bypass its command name check by using a
null zeroth word.
$ { function } { echo test; }; <() }; }
test
Ordinarily } would be uncallable,

Only literal } is special, you can use \} instead, or store it in a
variable.

Yeah I forgot there are a few ways. `<&0 }` is yet another.

[Chet Ramey]

Post by Dan Douglas
This doesn't normally matter because POSIX requires special builtins to take
precedence over functions during command search, so even if you have such a
function defined it is impossible to call. Bash doesn't use the correct search
order however.

Bash searches for special builtins before shell functions when in Posix
mode.

[David Korn]

I fixed the bug in ksh that allows you delete a special builtin.

Post by Dan Douglas
Just a few points to add.

Post by Stephane Chazelas
[...]

"The function is named fname; the application shall ensure that it

is a

Post by Stephane Chazelas

name (see XBD Name) and that it is not the name of a special built-in

utility."
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_05
This doesn't normally matter because POSIX requires special builtins to take
precedence over functions during command search, so even if you have such a
function defined it is impossible to call. Bash doesn't use the correct search
order however.
Mksh has the reverse bug. It allows defining the function (wrongly) but then
calls the special builtin anyway (correctly).
Another bug is in ksh93 whose `builtin` allows disabling special builtins
(which according to the manual, shouldn't work).
$ ksh -c 'builtin -d set; function set { echo test; }; set'
test
Bash's "enable" correctly disallows that.

Post by Stephane Chazelas
I agree the requirement is on the application, and I can't see
why POSIX should force a shell to reject a function whose name
doesn't contain a valid identifier.
...

Another thing you can do in bash is bypass its command name check by using a
null zeroth word.
$ { function } { echo test; }; <() }; }
test
Ordinarily } would be uncallable, but apparently since bash only checks the
command name of the first word, calling with e.g. `<() }` or `$() }` works.
--
Dan Douglas

[Eric Blake]

Post by David Korn
I fixed the bug in ksh that allows you delete a special builtin.

Thanks; here's another ksh bug:

$ env 'a|b=' bash -c 'set | grep a"."b'
$ env 'a|b=' ksh -c 'set | grep a"."b'
a|b=''

But per the documentation of set, "If no options or arguments are
specified, set shall write the names and values of all shell variables
in the collation sequence of the current locale." And elsewhere,

Environment variable names used by the utilities in the Shell and
Utilities volume of IEEE Std 1003.1-2001 consist solely of uppercase
letters, digits, and the '_' (underscore) from the characters defined in
Portable Character Set and do not begin with a digit. Other characters
may be permitted by an implementation; applications shall tolerate the
presence of such names.

Where this is a problem is that by exposing the name "a|b" through set,
ksh is making the claim that it is a valid shell variable, even though:

$ ksh -c 'unset "a|b"'
ksh: unset: a|b: invalid variable name

So ksh should be fixed to behave like bash to sanitize the environment
and strip out any invalid names before populating the set of shell
variables advertised through 'set' (you can still leave such name/value
pairs in the environment handed to children, unmolested, the way bash
does; it's just that you should not be able to get at or modify those
names from the shell).

https://bugzilla.redhat.com/show_bug.cgi?id=1147645 is also tracking
this issue.

[Eric Blake]

Post by Eric Blake
$ env 'a|b=' bash -c 'set | grep a"."b'
$ env 'a|b=' ksh -c 'set | grep a"."b'
a|b=''
But per the documentation of set, "If no options or arguments are
specified, set shall write the names and values of all shell variables
in the collation sequence of the current locale."

Or, reading further, "The output shall be suitable for reinput to the
shell, setting or resetting, as far as possible, the variables that are
currently set;"

but with the current output of set, trying to invoke 'eval "$(set)"'
will end up executing the command 'a' in the current working directory
rather than setting the "variable" "a|b". While the bash Shell Shock
bug has already established that arbitrary environment variable names
are a far less likely attack vector than arbitrary environment variable
values, I still can't help but wonder if someone can come up with an
exploit where a ksh script that runs with elevated privileges and tries
to eval the output of set will end up running code of the caller's choice.

[Stephane Chazelas]

Post by Eric Blake

Post by Eric Blake
$ env 'a|b=' bash -c 'set | grep a"."b'
$ env 'a|b=' ksh -c 'set | grep a"."b'
a|b=''
But per the documentation of set, "If no options or arguments are
specified, set shall write the names and values of all shell variables
in the collation sequence of the current locale."

Or, reading further, "The output shall be suitable for reinput to the
shell, setting or resetting, as far as possible, the variables that are
currently set;"
but with the current output of set, trying to invoke 'eval "$(set)"'
will end up executing the command 'a' in the current working directory
rather than setting the "variable" "a|b". While the bash Shell Shock
bug has already established that arbitrary environment variable names
are a far less likely attack vector than arbitrary environment variable
values, I still can't help but wonder if someone can come up with an
exploit where a ksh script that runs with elevated privileges and tries
to eval the output of set will end up running code of the caller's choice.

[...]

Same with "export -p":

$ env -i $'a\necho test\na=b' ksh -c 'export -p' | ksh
test

And bash is also vulnerable.

$ env -i $'a\necho test\na=b' bash -c 'export -p'
declare -x OLDPWD
declare -x PWD="/home/stephane"
declare -x SHLVL="1"
declare -x a
echo test
a

(that output doesn't make much sense, suggesting it may also
hide more bugs and vulnerabilities).

[Stephane Chazelas]

2014-09-30 17:06:22 +0100, Stephane Chazelas:
[...]

Post by Stephane Chazelas
$ env -i $'a\necho test\na=b' ksh -c 'export -p' | ksh
test
And bash is also vulnerable.
$ env -i $'a\necho test\na=b' bash -c 'export -p'
declare -x OLDPWD
declare -x PWD="/home/stephane"
declare -x SHLVL="1"
declare -x a
echo test
a
(that output doesn't make much sense, suggesting it may also
hide more bugs and vulnerabilities).

[...]

Sorry, it does make sense. "bash" just outputs:

declare -x var-name

when var-name is not a valid identifier in the current locale.

Both ksh and bash's can be exploited using the LC_XXX with ssh
ForceCommand vector (and the output of "export -p" being
evaluated somehow).

[Stephane Chazelas]

2014-09-29 09:04:00 -0600, Eric Blake:
[...]

Post by Eric Blake
I would expect something like "It shall be an error if fname is the name
of a special built-in utility", as _that_ would be a hard requirement on
bash, not just the application. Maybe we have a bug in the POSIX spec
for a missing requirement.

[...]

Such a requirement would break bash, zsh, mksh, AT&T ksh (for
disciplines), which currently all allow a function called "a.b"

Cheers,
Stephane