Discussion:
[1003.1(2013)/Issue7+TC1 0000842]: meaning of "enclosing loop" with break/continue unclear
Austin Group Bug Tracker
2014-06-02 05:55:13 UTC
Permalink
The following issue has been SUBMITTED.
======================================================================
http://austingroupbugs.net/view.php?id=842
======================================================================
Reported By: rhansen
Assigned To:
======================================================================
Project: 1003.1(2013)/Issue7+TC1
Issue ID: 842
Category: Shell and Utilities
Type: Omission
Severity: Objection
Priority: normal
Status: New
Name: Richard Hansen
Organization: BBN
User Reference:
Section: XCU 2.14 break, continue
Page Number: 2358, 2362
Line Number: 75117-75121, 75244-75250
Interp Status: ---
Final Accepted Text:
======================================================================
Date Submitted: 2014-06-02 05:55 UTC
Last Modified: 2014-06-02 05:55 UTC
======================================================================
Summary: meaning of "enclosing loop" with break/continue
unclear
Description:
The results of running 'break' and 'continue' are specified in terms of the
enclosing for, while, or until loop(s). However, it is unclear what
"enclosing" means. For example, how should the following script behave:

foo() {
for i in 1 2; do
echo " running bar ($i)..."
bar
echo " bar returned $?"
done
}
bar() {
for j in 1 2; do
do_break() {
echo " breaking..."
false
break 2
echo " break returned $?"
}
echo " running do_break ($j)..."
do_break
echo " do_break returned $?"
done
}
echo "running foo..."
foo
echo "foo returned $?"

Should the 'break' command do nothing because it is not a command in a
compound list associated with a loop? Should it only break out of the
inner loop because it is syntactically "inside" the inner loop but not the
outer loop? Should it break out of the outer loop because both loops were
executing at the time break was run?

One could interpret "enclosing" as implying lexical/static scope: a
break/continue command must be one of the commands in the compound list
associated with the loop in order for the loop to qualify as enclosing the
command.

Alternatively, one could interpret "enclosing" as implying dynamic scope:
A break/continue command is enclosed by a loop if the loop is executing
when the command is executed.

The following is a list of how the 'break' in the above script behaves in
some existing implementations:

* bash (POSIX mode), zsh (sh emulation mode), dash, and NetBSD's
/bin/sh: breaks out of the outer loop
* ksh93: does nothing (no error message, exit status is 0)
* mksh: prints an error message but otherwise does nothing (exit
status is 0)

Scripts sourced with the dot command lead to similar questions. Here's an
example script:

cat <<\EOF >/tmp/foo
for i in 1 2; do
echo " running bar ($i)..."
. /tmp/bar
echo " bar returned $?"
done
EOF
cat <<\EOF >/tmp/bar
for j in 1 2; do
echo " running do_break ($j)..."
. /tmp/do_break
echo " do_break returned $?"
done
EOF
cat <<\EOF >/tmp/do_break
echo " breaking..."
false
break 2
echo " break returned $?"
EOF
echo "running foo..."
. /tmp/foo
echo "foo returned $?"

With the above script, the 'break' behaves as follows:

* ksh93, bash (POSIX mode) and now NetBSD's /bin/sh [1]: breaks out
of the outer loop
* zsh (sh emulation mode): errors out
* dash: acts like 'return 0'
* mksh: prints an error message but otherwise does nothing (exit
status is 0)

[1] http://thread.gmane.org/gmane.os.netbsd.bugs/70663

Also, the behavior when break or continue is run outside of a loop is
unclear. The specification for break without arguments says "shall exit
from the smallest enclosing [...] loop, if any", but then goes on to talk
about breaking out of the outermost enclosing loop (because the default is
equivalent to n=1, and 1 is greater than the number of enclosing loops).
The specification for continue doesn't have the "if any" qualifier.

Desired Action:
(I will post proposed wording changes as a bug note later. Given the
implementation diversity, I'm guessing we'll want to go with "unspecified"
for Issue 7 TC2. For Issue 8, I don't know if lexical or dynamic scope is
preferred.)
======================================================================

Issue History
Date Modified Username Field Change
======================================================================
2014-06-02 05:55 rhansen New Issue
2014-06-02 05:55 rhansen Name => Richard Hansen
2014-06-02 05:55 rhansen Organization => BBN
2014-06-02 05:55 rhansen Section => XCU 2.14 break,
continue
2014-06-02 05:55 rhansen Page Number => 2358, 2362
2014-06-02 05:55 rhansen Line Number => 75117-75121,
75244-75250
======================================================================
Austin Group Bug Tracker
2014-06-02 20:42:52 UTC
Permalink
A NOTE has been added to this issue.
======================================================================
http://www.austingroupbugs.net/view.php?id=842
======================================================================
Reported By: rhansen
Assigned To:
======================================================================
Project: 1003.1(2013)/Issue7+TC1
Issue ID: 842
Category: Shell and Utilities
Type: Omission
Severity: Objection
Priority: normal
Status: New
Name: Richard Hansen
Organization: BBN
User Reference:
Section: XCU 2.14 break, continue
Page Number: 2358, 2362
Line Number: 75117-75121, 75244-75250
Interp Status: ---
Final Accepted Text:
======================================================================
Date Submitted: 2014-06-02 05:55 UTC
Last Modified: 2014-06-02 20:42 UTC
======================================================================
Summary: meaning of "enclosing loop" with break/continue
unclear
======================================================================

----------------------------------------------------------------------
(0002256) rhansen (reporter) - 2014-06-02 20:42
http://www.austingroupbugs.net/view.php?id=842#c2256
----------------------------------------------------------------------
Dynamic scope for break/continue can be thought of as an extension to
lexical scope: Any shell script written assuming lexical scoping will
behave as intended if the shell implementation actually uses dynamic
scoping for break/continue. Thus, I think it will be sufficient to simply
specify lexical scoping for Issue 7 TC2 and leave dynamic scoping as an
extension to the standard.

Issue History
Date Modified Username Field Change
======================================================================
2014-06-02 05:55 rhansen New Issue
2014-06-02 05:55 rhansen Name => Richard Hansen
2014-06-02 05:55 rhansen Organization => BBN
2014-06-02 05:55 rhansen Section => XCU 2.14 break,
continue
2014-06-02 05:55 rhansen Page Number => 2358, 2362
2014-06-02 05:55 rhansen Line Number => 75117-75121,
75244-75250
2014-06-02 20:42 rhansen Note Added: 0002256
======================================================================
Per Mildner
2014-06-03 08:44:30 UTC
Permalink
Post by Austin Group Bug Tracker
A NOTE has been added to this issue.
======================================================================
http://www.austingroupbugs.net/view.php?id=842
======================================================================
Reported By: rhansen
======================================================================
Project: 1003.1(2013)/Issue7+TC1
Issue ID: 842
Category: Shell and Utilities
Type: Omission
Severity: Objection
Priority: normal
Status: New
Name: Richard Hansen
Organization: BBN
Section: XCU 2.14 break, continue
Page Number: 2358, 2362
Line Number: 75117-75121, 75244-75250
Interp Status: ---
======================================================================
Date Submitted: 2014-06-02 05:55 UTC
Last Modified: 2014-06-02 20:42 UTC
======================================================================
Summary: meaning of "enclosing loop" with break/continue
unclear
======================================================================
----------------------------------------------------------------------
(0002256) rhansen (reporter) - 2014-06-02 20:42
http://www.austingroupbugs.net/view.php?id=842#c2256
----------------------------------------------------------------------
Dynamic scope for break/continue can be thought of as an extension to
lexical scope: Any shell script written assuming lexical scoping will
behave as intended if the shell implementation actually uses dynamic
scoping for break/continue.
This is not true. The problem is that the standard currently says "If n is greater than the number of enclosing loops, the outermost enclosing loop shall be exited.".

Changing from lexical scope to dynamic scope will change which loop is the outermost enclosing loop.

So, code that use something like "break 1000" as an idiom for getting out of all (lexically) enclosing loops, will no longer work correctly if run on an implementation that uses dynamic scope if there are additional dynamically enclosing loops.

(I used that idiom once...)

Regards,

Per Mildner
Per.Mildner-02ecni/***@public.gmane.org
Richard Hansen
2014-06-03 16:21:51 UTC
Permalink
Post by Austin Group Bug Tracker
----------------------------------------------------------------------
(0002256) rhansen (reporter) - 2014-06-02 20:42
http://www.austingroupbugs.net/view.php?id=842#c2256
----------------------------------------------------------------------
Dynamic scope for break/continue can be thought of as an extension to
lexical scope: Any shell script written assuming lexical scoping will
behave as intended if the shell implementation actually uses dynamic
scoping for break/continue.
This is not true. The problem is that the standard currently says "If
n is greater than the number of enclosing loops, the outermost
enclosing loop shall be exited.".
Changing from lexical scope to dynamic scope will change which loop
is the outermost enclosing loop.
Oops, you're right. I overlooked that case. Thank you for noticing my
error!

I'll have to revise the proposed wording changes in [1]. For Issue 7
TC2 I'm thinking of taking this course:

* keep the proposed text defining an enclosing loop from [1]
* add a new paragraph that says that it is undefined whether dynamic
loops are also considered to be enclosing
* add an application note that says to be careful if n is greater
than the number of lexically enclosing loops, with an example

What do people think about this approach?

For Issue 8 I think we need to pick either lexical or dynamic scope.
This is unfortunate; I was hoping to avoid imposing new requirements on
implementations.

Does anyone have strong opinions about dynamic vs. static for Issue 8?
One NetBSD developer so far has expressed preference for lexical scope
[2] even though their shell currently implements dynamic scope. (If
there's a clear winner I can write up some proposed text before
Thursday's teleconference.)

Thanks,
Richard

[1] http://www.austingroupbugs.net/view.php?id=842#c2257
[2] http://article.gmane.org/gmane.os.netbsd.devel.userlevel/17452
Geoff Clare
2014-06-04 08:31:28 UTC
Permalink
Post by Richard Hansen
I'll have to revise the proposed wording changes in [1]. For Issue 7
* keep the proposed text defining an enclosing loop from [1]
* add a new paragraph that says that it is undefined whether dynamic
s/undefined/unspecified/
Post by Richard Hansen
loops are also considered to be enclosing
* add an application note that says to be careful if n is greater
than the number of lexically enclosing loops, with an example
What do people think about this approach?
Sounds fine to me.
Post by Richard Hansen
For Issue 8 I think we need to pick either lexical or dynamic scope.
I don't see why. The current situation has persisted for at
least 20 years, so I don't have a problem with it continuing.
I imagine that almost all shell script authors just naturally
use break and continue in a way that works the same with either
scope, without being aware of the issue at all.
--
Geoff Clare <g.clare-7882/***@public.gmane.org>
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England
Austin Group Bug Tracker
2014-06-02 20:43:15 UTC
Permalink
A NOTE has been added to this issue.
======================================================================
http://www.austingroupbugs.net/view.php?id=842
======================================================================
Reported By: rhansen
Assigned To:
======================================================================
Project: 1003.1(2013)/Issue7+TC1
Issue ID: 842
Category: Shell and Utilities
Type: Omission
Severity: Objection
Priority: normal
Status: New
Name: Richard Hansen
Organization: BBN
User Reference:
Section: XCU 2.14 break, continue
Page Number: 2358, 2362
Line Number: 75117-75121, 75244-75250
Interp Status: ---
Final Accepted Text:
======================================================================
Date Submitted: 2014-06-02 05:55 UTC
Last Modified: 2014-06-02 20:43 UTC
======================================================================
Summary: meaning of "enclosing loop" with break/continue
unclear
======================================================================

----------------------------------------------------------------------
(0002257) rhansen (reporter) - 2014-06-02 20:43
http://www.austingroupbugs.net/view.php?id=842#c2257
----------------------------------------------------------------------
At page 2358 lines 75117-75121 (XCU 2.14 break description), change:

The break utility shall exit from the smallest enclosing for,
while, or until loop, if any; or from the nth enclosing loop if n
is specified. The value of n is an unsigned decimal integer
greater than or equal to 1. The default shall be equivalent to
n=1. If n is greater than the number of enclosing loops, the
outermost enclosing loop shall be exited. Execution shall
continue with the command immediately following the loop.

to:

If n is specified, the break utility shall exit from the nth
enclosing for, while, or until loop. If n is unspecified, break
shall behave as if n was specified as 1. Execution shall continue
with the command immediately following the exited loop. The value
of n is a decimal integer greater than or equal to 1. If n is
greater than the number of enclosing loops, the outermost
enclosing loop shall be exited. If there is no enclosing loop,
the behavior is unspecified.

A loop shall be considered to enclose a break or continue command
if the command is:
* executing in the same execution environment (see section 2.12)
as the compound-list of the loop's do-group (see section
2.10.2), and
* contained in a compound-list associated with the loop (either
in the compound-list of the loop's do-group or, if the loop is
a while or until loop, in the compound-list following the
while or until reserved word), and
* not in the body of a function whose function definition
command (see section 2.9.5) is contained in a compound-list
associated with the loop.

At page 2362 lines 75244-75250 (XCU 2.14 continue description), change:

The continue utility shall return to the top of the smallest
enclosing for, while, or until loop, or to the top of the nth
enclosing loop, if n is specified. This involves repeating the
condition list of a while or until loop or performing the next
assignment of a for loop, and re-executing the loop if
appropriate.

The value of n is a decimal integer greater than or equal to
1. The default shall be equivalent to n=1. If n is greater than
the number of enclosing loops, the outermost enclosing loop shall
be used.

to:

If n is specified, the continue utility shall return to the top of
the nth enclosing for, while, or until loop. If n is unspecified,
continue shall behave as if n was specified as 1. Returning to
the top of the loop involves repeating the condition list of a
while or until loop or performing the next assignment of a for
loop, and re-executing the loop if appropriate. The value of n is
a decimal integer greater than or equal to 1. If n is greater
than the number of enclosing loops, the outermost enclosing loop
shall be used. If there is no enclosing loop, the behavior is
unspecified.

The meaning of "enclosing" shall be as specified in the
description of the break utility.

Issue History
Date Modified Username Field Change
======================================================================
2014-06-02 05:55 rhansen New Issue
2014-06-02 05:55 rhansen Name => Richard Hansen
2014-06-02 05:55 rhansen Organization => BBN
2014-06-02 05:55 rhansen Section => XCU 2.14 break,
continue
2014-06-02 05:55 rhansen Page Number => 2358, 2362
2014-06-02 05:55 rhansen Line Number => 75117-75121,
75244-75250
2014-06-02 20:42 rhansen Note Added: 0002256
2014-06-02 20:43 rhansen Note Added: 0002257
======================================================================
Austin Group Bug Tracker
2014-06-03 22:23:02 UTC
Permalink
A NOTE has been added to this issue.
======================================================================
http://www.austingroupbugs.net/view.php?id=842
======================================================================
Reported By: rhansen
Assigned To:
======================================================================
Project: 1003.1(2013)/Issue7+TC1
Issue ID: 842
Category: Shell and Utilities
Type: Omission
Severity: Objection
Priority: normal
Status: New
Name: Richard Hansen
Organization: BBN
User Reference:
Section: XCU 2.14 break, continue
Page Number: 2358, 2362
Line Number: 75117-75121, 75244-75250
Interp Status: ---
Final Accepted Text:
======================================================================
Date Submitted: 2014-06-02 05:55 UTC
Last Modified: 2014-06-03 22:23 UTC
======================================================================
Summary: meaning of "enclosing loop" with break/continue
unclear
======================================================================

----------------------------------------------------------------------
(0002258) jilles (reporter) - 2014-06-03 22:23
http://www.austingroupbugs.net/view.php?id=842#c2258
----------------------------------------------------------------------
Here are two more strange cases:

A break or continue could be in a trap handler:
sh -c 'trap break USR1; for i in a b; do kill -USR1 $$; echo $i; done'
Most shells (except mksh) exit from the loop in this example. This can be
used to good effect, although something similar can be done by setting a
variable in the trap handler and checking it in the mainline code.

A break could be in a subshell nested in a loop:
sh -c 'for i in a b; do (break; echo $i) done'
Most shells (except mksh) exit from the subshell in this example. In
FreeBSD sh, this is because the shell attempts to break from the loop, but
the attempt stops at the subshell.

I think the dynamic scope is most compatible with existing shells and
scripts.

Issue History
Date Modified Username Field Change
======================================================================
2014-06-02 05:55 rhansen New Issue
2014-06-02 05:55 rhansen Name => Richard Hansen
2014-06-02 05:55 rhansen Organization => BBN
2014-06-02 05:55 rhansen Section => XCU 2.14 break,
continue
2014-06-02 05:55 rhansen Page Number => 2358, 2362
2014-06-02 05:55 rhansen Line Number => 75117-75121,
75244-75250
2014-06-02 20:42 rhansen Note Added: 0002256
2014-06-02 20:43 rhansen Note Added: 0002257
2014-06-03 22:23 jilles Note Added: 0002258
======================================================================
Austin Group Bug Tracker
2014-06-05 06:24:14 UTC
Permalink
A NOTE has been added to this issue.
======================================================================
http://austingroupbugs.net/view.php?id=842
======================================================================
Reported By: rhansen
Assigned To:
======================================================================
Project: 1003.1(2013)/Issue7+TC1
Issue ID: 842
Category: Shell and Utilities
Type: Omission
Severity: Objection
Priority: normal
Status: New
Name: Richard Hansen
Organization: BBN
User Reference:
Section: XCU 2.14 break, continue
Page Number: 2358, 2362
Line Number: 75117-75121, 75244-75250
Interp Status: ---
Final Accepted Text:
======================================================================
Date Submitted: 2014-06-02 05:55 UTC
Last Modified: 2014-06-05 06:24 UTC
======================================================================
Summary: meaning of "enclosing loop" with break/continue
unclear
======================================================================

----------------------------------------------------------------------
(0002259) rhansen (reporter) - 2014-06-05 06:24
http://austingroupbugs.net/view.php?id=842#c2259
----------------------------------------------------------------------
Thank you for the additional strange cases!
Because no loop lexically encloses the break in the trap, the proposed
wording would cause this to fall under unspecified behavior. However, the
trap is executed in the same environment as the loop so if the shell did
dynamic scope and the trap was invoked while executing the loop then it
should work as expected.
The proposed wording makes this unspecified behavior because the subshell
puts the break in a different shell execution environment.
Post by Austin Group Bug Tracker
I think the dynamic scope is most compatible with existing shells and
scripts.

Agreed, although there are some implementations don't do dynamic scoping.
Also, and I agree with what Geoff said on the mailing list:<blockquote>The
current situation has persisted for at least 20 years, so I don't have a
problem with it continuing. I imagine that almost all shell script authors
just naturally use break and continue in a way that works the same with
either scope, without being aware of the issue at all.</blockquote>

Issue History
Date Modified Username Field Change
======================================================================
2014-06-02 05:55 rhansen New Issue
2014-06-02 05:55 rhansen Name => Richard Hansen
2014-06-02 05:55 rhansen Organization => BBN
2014-06-02 05:55 rhansen Section => XCU 2.14 break,
continue
2014-06-02 05:55 rhansen Page Number => 2358, 2362
2014-06-02 05:55 rhansen Line Number => 75117-75121,
75244-75250
2014-06-02 20:42 rhansen Note Added: 0002256
2014-06-02 20:43 rhansen Note Added: 0002257
2014-06-03 22:23 jilles Note Added: 0002258
2014-06-05 03:49 rhansen Note Edited: 0002256
2014-06-05 03:58 rhansen Note Edited: 0002256
2014-06-05 05:43 rhansen Note Edited: 0002257
2014-06-05 05:59 rhansen Note Edited: 0002257
2014-06-05 06:01 rhansen Note Edited: 0002257
2014-06-05 06:07 rhansen Note Edited: 0002257
2014-06-05 06:24 rhansen Note Added: 0002259
======================================================================
Austin Group Bug Tracker
2014-06-05 16:33:49 UTC
Permalink
The following issue has been RESOLVED.
======================================================================
http://austingroupbugs.net/view.php?id=842
======================================================================
Reported By: rhansen
Assigned To:
======================================================================
Project: 1003.1(2013)/Issue7+TC1
Issue ID: 842
Category: Shell and Utilities
Type: Omission
Severity: Objection
Priority: normal
Status: Resolved
Name: Richard Hansen
Organization: BBN
User Reference:
Section: XCU 2.14 break, continue
Page Number: 2358, 2362
Line Number: 75117-75121, 75244-75250
Interp Status: ---
Final Accepted Text: See
http://austingroupbugs.net/view.php?id=842#c2257.
Resolution: Accepted As Marked
Fixed in Version:
======================================================================
Date Submitted: 2014-06-02 05:55 UTC
Last Modified: 2014-06-05 16:33 UTC
======================================================================
Summary: meaning of "enclosing loop" with break/continue
unclear
======================================================================

Issue History
Date Modified Username Field Change
======================================================================
2014-06-02 05:55 rhansen New Issue
2014-06-02 05:55 rhansen Name => Richard Hansen
2014-06-02 05:55 rhansen Organization => BBN
2014-06-02 05:55 rhansen Section => XCU 2.14 break,
continue
2014-06-02 05:55 rhansen Page Number => 2358, 2362
2014-06-02 05:55 rhansen Line Number => 75117-75121,
75244-75250
2014-06-02 20:42 rhansen Note Added: 0002256
2014-06-02 20:43 rhansen Note Added: 0002257
2014-06-03 22:23 jilles Note Added: 0002258
2014-06-05 03:49 rhansen Note Edited: 0002256
2014-06-05 03:58 rhansen Note Edited: 0002256
2014-06-05 05:43 rhansen Note Edited: 0002257
2014-06-05 05:59 rhansen Note Edited: 0002257
2014-06-05 06:01 rhansen Note Edited: 0002257
2014-06-05 06:07 rhansen Note Edited: 0002257
2014-06-05 06:24 rhansen Note Added: 0002259
2014-06-05 06:30 rhansen Note Edited: 0002259
2014-06-05 16:12 rhansen Note Edited: 0002257
2014-06-05 16:20 rhansen Note Edited: 0002257
2014-06-05 16:33 Don Cragun Interp Status => ---
2014-06-05 16:33 Don Cragun Final Accepted Text => See
http://austingroupbugs.net/view.php?id=842#c2257.
2014-06-05 16:33 Don Cragun Status New => Resolved
2014-06-05 16:33 Don Cragun Resolution Open => Accepted As
Marked
======================================================================
Loading...