[1003.1(2013)/Issue7+TC1 0000859]: Add posix

Discussion:

[1003.1(2013)/Issue7+TC1 0000859]: Add posix_random family of interfaces

Austin Group Bug Tracker

2014-07-18 13:59:17 UTC

The following issue has been SUBMITTED.
======================================================================
http://austingroupbugs.net/view.php?id=859
======================================================================
Reported By: tedu
Assigned To:
======================================================================
Project: 1003.1(2013)/Issue7+TC1
Issue ID: 859
Category: System Interfaces
Type: Enhancement Request
Severity: Comment
Priority: normal
Status: New
Name: Ted Unangst
Organization: OpenBSD
User Reference:
Section: posix_random
Page Number: 0
Line Number: 0
Interp Status: ---
Final Accepted Text:
======================================================================
Date Submitted: 2014-07-18 13:59 UTC
Last Modified: 2014-07-18 13:59 UTC
======================================================================
Summary: Add posix_random family of interfaces
Description:
Cryptographic software requires a source of unpredictable (pseduo) random
numbers. Various nonstandard system interfaces exist for this purpose, but
they have subtle differences in behavior between platforms. Attempts to
build reliable portable software often fail due to this variation.

The interfaces below are adapted from the arc4random family of interfaces
first introduced in OpenBSD in 1996 and experiences gained since then. They
have been renamed to be more standard like, although the OpenBSD project
would not object to standardizing the existing names.
Desired Action:
SYNOPSIS
#include <stdlib.h>

uint32_t posix_random(void);

void posix_random_buffer(void *buf, size_t nbytes);

uint32_t posix_random_uniform(uint32_t upper_bound);

DESCRIPTION
This family of functions provides higher quality data than those
described in rand(3), random(3), and drand48(3). The generated
numbers
must be unpredictable. In particular, the sequence must not be shared
between processes after fork().

The arc4random() function returns a single 32-bit value.

arc4random_buf() fills the region buf of length nbytes with random
data.

arc4random_uniform() will return a single 32-bit value, uniformly
distributed but less than upper_bound. This is recommended over
constructions like "arc4random() % upper_bound" as it avoids "modulo
bias" when the upper bound is not a power of two.

All of these functions are thread safe.

RETURN VALUES
These functions are always successful, and no return value is reserved
to
indicate an error.

RATIONALE
The standard does not specify a required algorithm, leaving
implementations
some flexibility so long as they meet the interface requirements.

No mechanism is provided to seed or reseed these functions, which
places
an unnecessary burden on application developers. The implementation
is
responsible for ensuring correct operation at all times.
======================================================================

Issue History
Date Modified Username Field Change
======================================================================
2014-07-18 13:59 tedu New Issue
2014-07-18 13:59 tedu Name => Ted Unangst
2014-07-18 13:59 tedu Organization => OpenBSD
2014-07-18 13:59 tedu Section => posix_random
2014-07-18 13:59 tedu Page Number => 0
2014-07-18 13:59 tedu Line Number => 0
======================================================================

Austin Group Bug Tracker

2014-07-18 14:04:10 UTC

Permalink

Austin Group Bug Tracker

2014-07-18 17:35:52 UTC

Permalink

A NOTE has been added to this issue.
======================================================================
http://austingroupbugs.net/view.php?id=859
======================================================================
Reported By: tedu
Assigned To:
======================================================================
Project: 1003.1(2013)/Issue7+TC1
Issue ID: 859
Category: System Interfaces
Type: Enhancement Request
Severity: Comment
Priority: normal
Status: New
Name: Ted Unangst
Organization: OpenBSD
User Reference:
Section: posix_random
Page Number: 0
Line Number: 0
Interp Status: ---
Final Accepted Text:
======================================================================
Date Submitted: 2014-07-18 13:59 UTC
Last Modified: 2014-07-18 17:35 UTC
======================================================================
Summary: Add posix_random family of interfaces
======================================================================

----------------------------------------------------------------------
(0002317) mdempsky (reporter) - 2014-07-18 17:35
http://austingroupbugs.net/view.php?id=859#c2317
----------------------------------------------------------------------
I'm very supportive of this if it can be accepted. A couple notes though:

1. I think for most other posix_FOO() functions, there was often an
existing FOO() function very similar in design/purpose. I'm a little
concerned that naming these just posix_random(), etc. might lead people to
think they're just a new API for the (non-cryptographically secure) rand()
and/or random() functions. So it might be worth considering some
alternative names like "posix_saferandom()", "posix_cryptrandom()",
"posix_securerandom()", etc. (For reference, Windows provides a function
"CryptGenRandom", and Java provides a "SecureRandom" class. In the
cryptographic community the adjective "safe" is getting some use such as in
safecurves.cr.yp.to.)

2. POSIX doesn't seem to use the term "quality" in describing the other RNG
APIs, so it would seem a bit out of place to refer to it here. It might be
better to directly address the requirement that the API provide random
bytes suitable for cryptographic use.

2b. It's theoretically of use to quantify the security level somehow
(e.g., Salsa20 makes some specific claims about conjectured attack costs in
http://cr.yp.to/snuffle/security.pdf), but in the interest of simplicity,
I'm inclined to leave that as a quality-of-implementation issue. Perhaps
just require implementations to document their choice of RNG and the
conjectured security levels?

3. This API doesn't explain that it uses a "sequence" of random values like
the rand(), srand() or drand48() functions do, so saying "In particular,
the sequence must not be shared between processes after fork()." is a bit
out of place too I think. Either we should clarify that these APIs also
use a sequence, or we should try to revise the wording to simply say that
each invocation needs to generate new independent values, even after a
fork(). Not sure how to word that off hand though; I keep wanting to say
"unique", but that might be construed as implying that values aren't
reused. I'll look around for some alternative wording options.

4. If this is added to <stdlib.h>, it needs to be shaded CX, since it's an
extension to an ISO C header.

Issue History
Date Modified Username Field Change
======================================================================
2014-07-18 13:59 tedu New Issue
2014-07-18 13:59 tedu Name => Ted Unangst
2014-07-18 13:59 tedu Organization => OpenBSD
2014-07-18 13:59 tedu Section => posix_random
2014-07-18 13:59 tedu Page Number => 0
2014-07-18 13:59 tedu Line Number => 0
2014-07-18 14:04 tedu Note Added: 0002314
2014-07-18 17:35 mdempsky Note Added: 0002317
======================================================================

Austin Group Bug Tracker

2014-07-18 17:51:46 UTC

Permalink

Austin Group Bug Tracker

2014-07-18 18:22:42 UTC

Permalink

Austin Group Bug Tracker

2014-07-21 02:59:20 UTC

Permalink

A NOTE has been added to this issue.
======================================================================
http://austingroupbugs.net/view.php?id=859
======================================================================
Reported By: tedu
Assigned To:
======================================================================
Project: 1003.1(2013)/Issue7+TC1
Issue ID: 859
Category: System Interfaces
Type: Enhancement Request
Severity: Comment
Priority: normal
Status: New
Name: Ted Unangst
Organization: OpenBSD
User Reference:
Section: posix_random
Page Number: 0
Line Number: 0
Interp Status: ---
Final Accepted Text:
======================================================================
Date Submitted: 2014-07-18 13:59 UTC
Last Modified: 2014-07-21 02:59 UTC
======================================================================
Summary: Add posix_random family of interfaces
======================================================================

----------------------------------------------------------------------
(0002320) dalias (reporter) - 2014-07-21 02:59
http://austingroupbugs.net/view.php?id=859#c2320
----------------------------------------------------------------------
I strongly object to the addition of any interface that terminates the
calling process as a consequence of failure. The only safe way to use such
an interface is to fork each time you want to use it, and this is
prohibitively slow and error-prone (e.g. you also need to exec if the
original process may be multi-threaded, since calling AS-unsafe functions
in the child after fork is UB for a multi-threaded parent) to the point of
making the whole interface useless.

Instead, the interface just need a proper way of reporting failure to the
caller, and should be designed to discourage misuse, i.e. it should not
require you to do awkward things like set errno to 0 before calling then
check errno after return. The functions as proposed clearly have no proper
way to report failure, so I think they should be rejected in their current
form.

posix_random_buffer could easily be fixed by changing the return type from
void to int, and having it return -1 on failure and 0 on success. For the
others, either the return value would need to be returned via a pointer to
the storage (but this makes posix_random essentially a duplicate of
posix_random_buffer with an implicit second argument of 4), or they could
take a pointer to a location to store a failure flag (and not modify the
pointed-to value on success). I like this latter design because it allows
you to use them in complex expressions and loops without checking for
failure on each call, but gives you an easy way to check for error at the
end of the whole computation (and then discard the result) much like what
can be done with floating point exception flags. However I also fear that
naive users may just throw away the flag.

Issue History
Date Modified Username Field Change
======================================================================
2014-07-18 13:59 tedu New Issue
2014-07-18 13:59 tedu Name => Ted Unangst
2014-07-18 13:59 tedu Organization => OpenBSD
2014-07-18 13:59 tedu Section => posix_random
2014-07-18 13:59 tedu Page Number => 0
2014-07-18 13:59 tedu Line Number => 0
2014-07-18 14:04 tedu Note Added: 0002314
2014-07-18 17:35 mdempsky Note Added: 0002317
2014-07-18 17:51 mdempsky Note Added: 0002318
2014-07-18 18:22 tedu Note Added: 0002319
2014-07-21 02:59 dalias Note Added: 0002320
======================================================================

Austin Group Bug Tracker

2014-07-21 04:07:15 UTC

Permalink

A NOTE has been added to this issue.
======================================================================
http://austingroupbugs.net/view.php?id=859
======================================================================
Reported By: tedu
Assigned To:
======================================================================
Project: 1003.1(2013)/Issue7+TC1
Issue ID: 859
Category: System Interfaces
Type: Enhancement Request
Severity: Comment
Priority: normal
Status: New
Name: Ted Unangst
Organization: OpenBSD
User Reference:
Section: posix_random
Page Number: 0
Line Number: 0
Interp Status: ---
Final Accepted Text:
======================================================================
Date Submitted: 2014-07-18 13:59 UTC
Last Modified: 2014-07-21 04:07 UTC
======================================================================
Summary: Add posix_random family of interfaces
======================================================================

----------------------------------------------------------------------
(0002321) mdempsky (reporter) - 2014-07-21 04:07
http://austingroupbugs.net/view.php?id=859#c2321
----------------------------------------------------------------------
In practice, I don't expect these APIs to normally call abort(). I was
just suggesting that in extreme situations, they should fail
loudly/catastrophically rather than silently.

E.g., on most implementations, passing a null pointer to memcpy() will
generally generate a SIGSEGV, which defaults to killing the process. Also,
on systems that use memory overcommit, calling a function that needs
dynamic binding might hit an out-of-memory condition trying to update the
PLT entry and cause the process to die. Those are the sorts of conditions
I had in mind for mentioning abort().

However, looking around more, I see functions like getpid() and getuid() do
simply specify they always succeed, even if existing implementations
provide mechanisms (e.g., ptrace, systrace, seccomp-bpf) that could cause
them to fail.

I think there's benefit to keeping the API simple and failure-free, if
possible. So my preference would be to specify "These functions are always
successful" if the abort() wording is contentious (and in retrospect it
seems superfluous anyway). My second preference would be to mark the
interfaces as optional (i.e., under an option group) so they don't need to
be provided at all by implementations that can't meet the "always
successful" constraint.

Put differently: what sort of error conditions might we want to specify for
these functions?

Issue History
Date Modified Username Field Change
======================================================================
2014-07-18 13:59 tedu New Issue
2014-07-18 13:59 tedu Name => Ted Unangst
2014-07-18 13:59 tedu Organization => OpenBSD
2014-07-18 13:59 tedu Section => posix_random
2014-07-18 13:59 tedu Page Number => 0
2014-07-18 13:59 tedu Line Number => 0
2014-07-18 14:04 tedu Note Added: 0002314
2014-07-18 17:35 mdempsky Note Added: 0002317
2014-07-18 17:51 mdempsky Note Added: 0002318
2014-07-18 18:22 tedu Note Added: 0002319
2014-07-21 02:59 dalias Note Added: 0002320
2014-07-21 04:07 mdempsky Note Added: 0002321
======================================================================

Austin Group Bug Tracker

2014-07-21 04:41:15 UTC

Permalink

A NOTE has been added to this issue.
======================================================================
http://austingroupbugs.net/view.php?id=859
======================================================================
Reported By: tedu
Assigned To:
======================================================================
Project: 1003.1(2013)/Issue7+TC1
Issue ID: 859
Category: System Interfaces
Type: Enhancement Request
Severity: Comment
Priority: normal
Status: New
Name: Ted Unangst
Organization: OpenBSD
User Reference:
Section: posix_random
Page Number: 0
Line Number: 0
Interp Status: ---
Final Accepted Text:
======================================================================
Date Submitted: 2014-07-18 13:59 UTC
Last Modified: 2014-07-21 04:41 UTC
======================================================================
Summary: Add posix_random family of interfaces
======================================================================

----------------------------------------------------------------------
(0002322) dalias (reporter) - 2014-07-21 04:41
http://austingroupbugs.net/view.php?id=859#c2322
----------------------------------------------------------------------
Passing a null pointer to memcpy is a completely different situation: by
invoking undefined behavior, you give the implementation license to abort
(or do whatever else it pleases). As for the situations cited with
overcommit and OOM during lazy binding, I consider these at least extremely
low-quality, and in my opinion non-conforming, implementations.

For the proposed posix_random interfaces, the obvious error condition is
"insufficient entropy available to produce non-predictable output". This
would normally have an underlying cause for the inability to obtain entropy
(e.g. EMFILE or ENFILE attempting to open /dev/urandom or similar). I'm not
sure whether it would be desirable to expose such underlying causes or just
a generic error code indicating that sufficiently non-predictable output
could not be produced.

Of course I would really just prefer that these functions not be permitted
to fail, and that they not be optional. This does not give an
implementation license to crash/abort when it can't satisfy the
requirements; rather, it renders the implementation non-conforming if it
can't satisfy the requirement.

However, I think it should be possible for all implementations to satisfy
the requirements of an always-working posix_random (independent sequences
in every process including after fork), albeit possibly not with ideal
strength properties. For example, at fork time an implementation could
consume enough output from posix_random in the parent to fill the internal
state of the prng in the child before actually forking. This does not
require obtaining any resource that could fail to be available. Another
approach, if reseeding from /dev/urandom or similar is deemed important by
the implementer, is for fork to fail when the resource (e.g. fd) needed to
obtain entropy cannot be allocated.

Issue History
Date Modified Username Field Change
======================================================================
2014-07-18 13:59 tedu New Issue
2014-07-18 13:59 tedu Name => Ted Unangst
2014-07-18 13:59 tedu Organization => OpenBSD
2014-07-18 13:59 tedu Section => posix_random
2014-07-18 13:59 tedu Page Number => 0
2014-07-18 13:59 tedu Line Number => 0
2014-07-18 14:04 tedu Note Added: 0002314
2014-07-18 17:35 mdempsky Note Added: 0002317
2014-07-18 17:51 mdempsky Note Added: 0002318
2014-07-18 18:22 tedu Note Added: 0002319
2014-07-21 02:59 dalias Note Added: 0002320
2014-07-21 04:07 mdempsky Note Added: 0002321
2014-07-21 04:41 dalias Note Added: 0002322
======================================================================