rm -f with no file operands fails on old BSD systems

On Mon, Oct 6, 2014 at 7:51 PM, Richard Hansen <rhansen-A08e6c8yq/***@public.gmane.org> wrote:
...

Post by Richard Hansen
Digging around in various CVS/Subversion repositories, it looks like
there are many old (but perhaps not yet museum-worthy) *BSD versions
* NetBSD 4.x and older (5.0 released April 2009). see: [1] [2]
* FreeBSD 3.1.x and older (3.2 released May 1999). see: [3] [4]
* OpenBSD 2.x and older (3.0 released Dec 2001). see [5]

Speaking on behalf of the OpenBSD project: OpenBSD 2.x should be
considered dead, buried, rotted, and worthy of derision. History is
history: useful to study, but if you try to live there you will die of
horrific diseases. Do not hold up progress on the basis that you'll
trip over a 12+ year old release.

Philip Guenther
guenther-***@public.gmane.org

Roger Marquis

2014-10-07 04:45:07 UTC

May not be related but ... is this why the behavior of "-i" and "-f"
flags differ between BSD and Linux, having changed from the traditional
behavior on Linux a couple of years ago?

FreeBSD:

# touch x y
# \cp -i -f x y file is deleted, precedence by arg position

GNU/Linux:

# touch x y
# \cp -i -f x y file is not deleted, precedence by "-i"?
cp: overwrite `y'?

I personally prefer the BSD behavior as cp, mv and rm are often aliased
with "-i" by default/in etc/skel preventing "-f" from working at all.
Either way (precedence by arg order or not) having the same behavior
across platforms would be preferable.

Roger Marquis

...
usage: rm [-f|-i] [-dPRrvW] file ...
Oops!
Your 'rm' program seems unable to run without file operands specified
on the command line, even when the '-f' option is present. This is contrary
to the behaviour of most rm programs out there, and not conforming with
the upcoming POSIX standard: <http://austingroupbugs.net/view.php?id=542>

Geoff Clare

2014-10-07 11:26:51 UTC

Post by Roger Marquis
May not be related but ... is this why the behavior of "-i" and "-f"
flags differ between BSD and Linux, having changed from the traditional
behavior on Linux a couple of years ago?
# touch x y
# \cp -i -f x y file is deleted, precedence by arg position
# touch x y
# \cp -i -f x y file is not deleted, precedence by "-i"?
cp: overwrite `y'?
I personally prefer the BSD behavior as cp, mv and rm are often aliased
with "-i" by default/in etc/skel preventing "-f" from working at all.
Either way (precedence by arg order or not) having the same behavior
across platforms would be preferable.

You seem to be assuming that the -i and -f options of cp, mv and rm
are all similar, but they are not: cp is different. If you check
the behaviour of mv and rm, I expect you will find all implementations
behave the way you expect (as it's required by POSIX that -i cancels
-f and -f cancels -i). For mv and rm this makes sense because the
options have opposite effects. For cp, the -f option is different to
mv and rm, and specifying both options produces behaviour that is
different from the behaviour you get with just -i or just -f. So it
makes no sense for each to cancel the other.

I think the reason for the difference for cp -f stems from the
difference in default behaviour. For mv and rm, with neither -f nor
-i, if a read-only file is going to be unlinked they prompt for
confirmation first. The cp default behaviour is never to prompt: if a
destination file is read-only the default is to report an error.
The -f option tells cp that if it can't open a destination file for
writing then it should unlink it and create it as a new file, and has
no effect on prompting - it is useful with or without -i in effect.

I'm surprised by the FreeBSD behaviour here. If, as you imply, -i and
-f each cancel the other, then it would appear that FreeBSD has no way
to tell cp that you want it to prompt whenever a destination file
already exists and you also want it to unlink and recreate read-only
destination files.

--
Geoff Clare <g.clare-7882/***@public.gmane.org>
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

Joerg Wunsch

2014-10-07 15:37:58 UTC

Post by Geoff Clare
I'm surprised by the FreeBSD behaviour here. If, as you imply, -i and
-f each cancel the other, then it would appear that FreeBSD has no way
to tell cp that you want it to prompt whenever a destination file
already exists and you also want it to unlink and recreate read-only
destination files.

That's indeed the case, yes.

--
cheers, Joerg .-.-. --... ...-- -.. . DL8DTL

http://www.sax.de/~joerg/
Never trust an operating system you don't have sources for. ;-)

Roger Marquis

2014-10-10 01:07:07 UTC

Post by Roger Marquis
# touch x y
# \cp -i -f x y file is deleted, precedence by arg position
# touch x y
# \cp -i -f x y file is not deleted, precedence by "-i"?
cp: overwrite `y'?

You seem to be assuming that the -i and -f options of cp, mv and rm
are all similar, but they are not: cp is different.

They are similar historically, and remain so on FreeBSD, but became
different on GNU/Linux systems a couple of years ago.

Post by Geoff Clare
I think the reason for the difference for cp -f stems from the
difference in default behaviour. For mv and rm, with neither -f nor
-i, if a read-only file is going to be unlinked they prompt for
confirmation first. The cp default behaviour is never to prompt: if a
destination file is read-only the default is to report an error.

That makes no sense. The behavior of these flags was originally designed
to be both logical and consistent across applications. This follows the
principle of least surprise. This is not an area where POSIX should be
breaking existing standards in favor of assumptions that have not been
validated.

Roger Marquis

Bruce Evans

2014-10-10 02:45:48 UTC

Post by Roger Marquis
# touch x y
# \cp -i -f x y file is deleted, precedence by arg position
# touch x y
# \cp -i -f x y file is not deleted, precedence by "-i"?
cp: overwrite `y'?

You seem to be assuming that the -i and -f options of cp, mv and rm
are all similar, but they are not: cp is different.

They are similar historically, and remain so on FreeBSD, but became
different on GNU/Linux systems a couple of years ago.

Actually, in BSD cp, these flags were not designed (they just grew) and
were even more inconsistent than now. 4.4BSD does the following:
- -f just cancels -i.
- -i is ignored unless isatty(fileno(stdin)). When it is not ignored,
it sets an internal flag that gives interactive behaviour. There is
only one internal flag, so -i cancels -f iff isatty(...).
- nothing gives the POSIX behaviour for -f, since the non-interactive
behaviour doesn't give it.

In 1996, FreeBSD fixed -f to give the POSIX behaviour, and also fixed
-i to not depend on isatty(...), but didn't fix the flags canceling
each other, so the POSIX behaviour for -fi is still unsupported.

In 2014, FreeBSD cp still has the flags canceling each other.

Bruce

Geoff Clare

2014-10-10 09:28:27 UTC

Post by Roger Marquis
# touch x y
# \cp -i -f x y file is deleted, precedence by arg position
# touch x y
# \cp -i -f x y file is not deleted, precedence by "-i"?
cp: overwrite `y'?

You seem to be assuming that the -i and -f options of cp, mv and rm
are all similar, but they are not: cp is different.

They are similar historically, and remain so on FreeBSD, but became
different on GNU/Linux systems a couple of years ago.

They were not similar historically. At the time that POSIX.2 was
standardising cp, System V had no -i option at all, and as Bruce
points out, the BSD behaviour for cp -i was rather odd.

You are trying to rewrite history. POSIX.2 did not invent the
unlink-if-you-can't-overwrite semantics for cp -f; it standardised
one type of existing practice. It just happens that the behaviour
of -f that was chosen wasn't the BSD one. Presumably the POSIX.2
developers felt that the other behaviour of -f was the more useful
one.

--
Geoff Clare <g.clare-7882/***@public.gmane.org>
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

Roger Marquis

2014-10-11 17:03:29 UTC

Post by Roger Marquis
# touch x y
# \cp -i -f x y file is deleted, precedence by arg position
# touch x y
# \cp -i -f x y file is not deleted, precedence by "-i"?
cp: overwrite `y'?

They were not similar historically.

To clarify, by historically I meant SunOS, Solaris, FreeBSD, Redhat and
Debian from the mid-90s. Interactive scripts worked on these platforms
(by respecting cp "-f" on the command line where cp was aliases "cp -i")
for well over a decade broke on Linux distributions (and only on Linux
distributions) a couple of years ago.

Post by Geoff Clare
Presumably the POSIX.2 developers felt that the other behaviour of -f was
the more useful one.

That is a presumption I agree with, however, it does not seem to accord
with the current behavior of "-f" with cp on Linux.

Roger Marquis

Andries E. Brouwer

2014-10-11 18:05:48 UTC

Post by Roger Marquis
# touch x y
# \cp -i -f x y file is deleted, precedence by arg position
# touch x y
# \cp -i -f x y file is not deleted, precedence by "-i"?
cp: overwrite `y'?

They were not similar historically.

Post by Geoff Clare
Presumably the POSIX.2 developers felt that the other behaviour of -f was
the more useful one.

That is a presumption I agree with, however, it does not seem to accord
with the current behavior of "-f" with cp on Linux.
Roger Marquis

This is long ago and I may misremember, but I think the Linux changes
were made in order to follow POSIX and/or Austin.

Old source for both cp.c and mv.c in GNU fileutils has

case 'f':
force = 1;
interactive = 0;
break;

case 'i':
force = 0;
interactive = 1;
break;

I have an .../opengroup/austin/austin7/d7text.txt here
(dated 2001-06-15, 12317091 bytes) that says for cp:

10278 NAME
10279 cp - copy files
10280 SYNOPSIS
10281 cp [-fip] source_file target_file
10398 OPTIONS
10402 -f If a file descriptor for a destination file cannot be obtained, as described in step
10403 3.a.ii., attempt to unlink the destination file and proceed.
10406 -i Write a prompt to standard error before copying to any existing destination file. If
10407 the response from the standard input is affirmative, the copy shall be attempted;
10408 otherwise, it shall not.

and for mv:

25127 NAME
25128 mv - move files
25129 SYNOPSIS
25130 mv [-fi] source_file target_file
25190 OPTIONS
25194 -f Do not prompt for confirmation if the destination path exists. Any previous
25195 occurrences of the -i option is ignored.
25196 -i Prompt for confirmation if the destination path exists. Any previous occurrences of
25197 the -f option is ignored.
25198 Specifying more than one of the -f or -i options shall not be considered an error. The last option
25199 specified shall determine the behavior of mv.

Maybe the asymmetry between cp and mv is due to this Austin draft?

Andries

Geoff Clare

2014-10-13 11:44:30 UTC

Post by Andries E. Brouwer
I have an .../opengroup/austin/austin7/d7text.txt here
10278 NAME
10279 cp - copy files
10280 SYNOPSIS
10281 cp [-fip] source_file target_file
10398 OPTIONS
10402 -f If a file descriptor for a destination file cannot be obtained, as described in step
10403 3.a.ii., attempt to unlink the destination file and proceed.
10406 -i Write a prompt to standard error before copying to any existing destination file. If
10407 the response from the standard input is affirmative, the copy shall be attempted;
10408 otherwise, it shall not.
25127 NAME
25128 mv - move files
25129 SYNOPSIS
25130 mv [-fi] source_file target_file
25190 OPTIONS
25194 -f Do not prompt for confirmation if the destination path exists. Any previous
25195 occurrences of the -i option is ignored.
25196 -i Prompt for confirmation if the destination path exists. Any previous occurrences of
25197 the -f option is ignored.
25198 Specifying more than one of the -f or -i options shall not be considered an error. The last option
25199 specified shall determine the behavior of mv.
Maybe the asymmetry between cp and mv is due to this Austin draft?

No, the asymmetry goes all the way back to the first POSIX shell and
utilities standard (POSIX.2-1992).

--
Geoff Clare <g.clare-7882/***@public.gmane.org>
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

Geoff Clare

2014-10-13 11:41:41 UTC

Post by Roger Marquis
# touch x y
# \cp -i -f x y file is deleted, precedence by arg position
# touch x y
# \cp -i -f x y file is not deleted, precedence by "-i"?
cp: overwrite `y'?

They were not similar historically.

Again, you are trying to rewrite history. Have a look at the Solaris
2.4 and 2.5 man pages on Oracle's web site.

Solaris 2.4 cp had no -f option.
Solaris 2.5 cp added the -f option with POSIX semantics:

-f Unlink. If a file descriptor for a destination file cannot be
obtained, attempt to unlink the destination file and proceed.

I can believe that cp -f worked the way you say on SunOS 4, BSD and Linux
systems historically, but the same is not true for Solaris (SunOS 5 and up)
and I doubt if it is true for any of the commercial UNIX systems except the
early BSD-derived ones.

If you remember it working on Solaris, my guess is that you had installed
the GNU version of cp and were using that instead of the native Solaris
version(s).

--
Geoff Clare <g.clare-7882/***@public.gmane.org>
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

Geoff Clare

2014-10-07 09:41:04 UTC

Post by Richard Hansen

$ uname -a
$ /bin/rm -f
usage: rm [-f|-i] [-dPRrvW] file ...
$ echo $?
1

Digging around in various CVS/Subversion repositories, it looks like
there are many old (but perhaps not yet museum-worthy) *BSD versions
* NetBSD 4.x and older (5.0 released April 2009). see: [1] [2]
* FreeBSD 3.1.x and older (3.2 released May 1999). see: [3] [4]
* OpenBSD 2.x and older (3.0 released Dec 2001). see [5]
Given this, I wonder if POSIX bug #542 [6] should be revisited. Perhaps
that bug should change the wording to "unspecified" for Issue 7 TC2, and
we can file a new bug report to adopt the wording currently in #542 for
Issue 8. Thoughts?

Suppose this change had been made in the 2008 revision instead of
being made in TC2. If NetBSD had changed their implementation to
conform to the new 2008 requirement, the change would have been made
in the 5.0 release - the exact same release that it was actually
made in. By making the change in TC2 we are effectively saying "we
should have made this change in 2008 and we a correcting that now",
so I don't think the pre-5.0 NetBSD behaviour should stop us putting
the new requirement into TC2.

A similar argument applies to the other two (but even more so, as
they are older).

--
Geoff Clare <g.clare-7882/***@public.gmane.org>
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

Richard Hansen

2014-10-07 15:25:06 UTC

Post by Richard Hansen

$ uname -a
$ /bin/rm -f
usage: rm [-f|-i] [-dPRrvW] file ...
$ echo $?
1

I had considered that. I'm a bit uneasy with that argument (but not
strongly opposed to keeping things as-is) because that text hasn't
changed since at least Issue 6. Had it changed for Issue 7 in 2008
that'd be one thing, but given that it didn't, an interpretation-less
correction makes it sound like we meant for 'rm -f' to be acceptable
since at least Issue 6. Perhaps that's the case, but I don't think we
would have been as comfortable making this same change back in 2007 for
an Issue 6 TC due to the existence of at least one implementation that
requires a file operand.

I think it's fair for someone to interpret the Issue 6 and 7 wording as
requiring conformant applications to provide at least one file operand,
and that permitting zero file operands is a significant enough
requirements change to warrant a stronger notice.

Post by Geoff Clare
A similar argument applies to the other two (but even more so, as
they are older).

Right. I'm less concerned about FreeBSD and OpenBSD; I mostly mentioned
them for the sake of the Autoconf/Automake bug report (my email was also
sent to bug-automake-mXXj517/***@public.gmane.org).

-Richard

Geoff Clare

2014-10-07 16:07:33 UTC

Post by Richard Hansen