[1003.1(2013)/Issue7+TC1 0000852]: Clarify MAP_FIXED semantics when replacing existing locked mappings

Discussion:

Austin Group Bug Tracker

2014-07-17 16:26:36 UTC

Austin Group Bug Tracker

2014-07-17 18:16:37 UTC

A NOTE has been added to this issue.
======================================================================
http://austingroupbugs.net/view.php?id=852
======================================================================
Reported By: mdempsky
Assigned To:
======================================================================
Project: 1003.1(2013)/Issue7+TC1
Issue ID: 852
Category: System Interfaces
Type: Clarification Requested
Severity: Editorial
Priority: normal
Status: New
Name: Matthew Dempsky
Organization: OpenBSD
User Reference:
Section: mmap
Page Number:
http://pubs.opengroup.org/onlinepubs/9699919799/functions/mmap.html
Line Number:
Interp Status: ---
Final Accepted Text:
======================================================================
Date Submitted: 2014-06-28 22:42 UTC
Last Modified: 2014-07-17 18:16 UTC
======================================================================
Summary: Clarify MAP_FIXED semantics when replacing existing
locked mappings
======================================================================

----------------------------------------------------------------------
(0002312) mdempsky (reporter) - 2014-07-17 18:16
http://austingroupbugs.net/view.php?id=852#c2312
----------------------------------------------------------------------
joerg: Yes, unfortunately like I said, the test program uses the
non-portable RLIMIT_MEMLOCK resource limit which doesn't seem to be
available outside of Linux, OS X, and *BSD. I recall seeing Solaris has
some memory locking limit that can be associated with jails, tasks, or
something, but I'm not familiar enough with Solaris to know how to write a
test program to utilize those (or if that's even possible).

As for why I think illumos has the same behavior, I'm not very familiar
with its VM internals, but here's what I was able to reason about from
browsing src.illumos.org:

In uts/common/os/grow.c, the munmap() system call appears to be implemented
as a call to lwpchan_delete_mapping() and as_unmap(). POSIX explicitly
requires munmap() to unlock wired memory, so I infer that calling both of
these in sequence should ensure the memory is unlocked. (I suspect
as_unmap() alone is actually responsible for the unlocking, but the rest of
my reasoning below doesn't depend on this detail.)

Also in uts/common/os/grow.c, the smmap32() and smmap64() system call entry
points both call into smmap_common(). Considering these two case
separately:

1. If fp == NULL (i.e., MAP_ANON was specified), then smmap_common() will
call lwpchan_delete_mapping() and then call zmap(), which calls
choose_addr(), which if MAP_FIXED was specified uses as_unmap() to remove
any existing mappings.

2. If fp != NULL, then smmap_common() will still (but much later) call
lwpchan_delete_mapping() and then call VOP_MAP(). I haven't reviewed every
vop_map implementation, but at least udf_map(), ufs_map(), and zfs_map()
all unconditionally call choose_addr() (as zmap() did above in case #1).

Therefore when MAP_FIXED is specified, illumos's mmap() appears to always
call the same two functions used to implement munmap()
(lwpchan_delete_mapping() and as_unmap()). So it seems reasonable to
conclude that on illumos the MAP_FIXED flag means the old mappings will be
removed as if by munmap().

Please advise if I've misinterpreted the code.

Issue History
Date Modified Username Field Change
======================================================================
2014-06-28 22:42 mdempsky New Issue
2014-06-28 22:42 mdempsky Name => Matthew Dempsky
2014-06-28 22:42 mdempsky Organization => OpenBSD
2014-06-28 22:42 mdempsky Section => mmap
2014-06-28 22:42 mdempsky Page Number =>
http://pubs.opengroup.org/onlinepubs/9699919799/functions/mmap.html
2014-06-28 22:56 mdempsky Note Added: 0002293
2014-06-29 00:20 mdempsky Note Added: 0002294
2014-07-02 18:02 mdempsky Note Added: 0002301
2014-07-04 17:04 markh Note Added: 0002302
2014-07-05 10:13 mdempsky Note Added: 0002303
2014-07-17 16:26 joerg Note Added: 0002311
2014-07-17 17:02 joerg Note Edited: 0002311
2014-07-17 17:03 joerg Note Edited: 0002311
2014-07-17 18:16 mdempsky Note Added: 0002312
======================================================================

Austin Group Bug Tracker

2014-07-18 07:48:37 UTC

Permalink

A NOTE has been added to this issue.
======================================================================
http://austingroupbugs.net/view.php?id=852
======================================================================
Reported By: mdempsky
Assigned To:
======================================================================
Project: 1003.1(2013)/Issue7+TC1
Issue ID: 852
Category: System Interfaces
Type: Clarification Requested
Severity: Editorial
Priority: normal
Status: New
Name: Matthew Dempsky
Organization: OpenBSD
User Reference:
Section: mmap
Page Number:
http://pubs.opengroup.org/onlinepubs/9699919799/functions/mmap.html
Line Number:
Interp Status: ---
Final Accepted Text:
======================================================================
Date Submitted: 2014-06-28 22:42 UTC
Last Modified: 2014-07-18 07:48 UTC
======================================================================
Summary: Clarify MAP_FIXED semantics when replacing existing
locked mappings
======================================================================

----------------------------------------------------------------------
(0002313) martinr (reporter) - 2014-07-18 07:48
http://austingroupbugs.net/view.php?id=852#c2313
----------------------------------------------------------------------
Solaris mlock(3C) man page says:

If the mapping through which an mlock() has been performed
is removed, an munlock() is implicitly performed. An mun-
lock() is also performed implicitly when a page is deleted
through file removal or truncation.

Test program:
char *p = mmap(NULL, 2 * pagesize, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0); mlock(p, 2 * pagesize);
munmap(p, 2 * pagesize));

Dtrace:
CPU FUNCTION
6 => munmap munmap syscall entry
6 -> munmap
...
6 -> as_unmap unmap the mapping from the address space
...
6 -> segvn_unmap unmap operation of seg_vn driver
6 -> segvn_lockop call for unlock the mapping

Kernel seg_vn segment driver is in use when the mapping was allocated
using
mmap() with MAP_PRIVATE|MAP_ANONYMOUS flags.

Issue History
Date Modified Username Field Change
======================================================================
2014-06-28 22:42 mdempsky New Issue
2014-06-28 22:42 mdempsky Name => Matthew Dempsky
2014-06-28 22:42 mdempsky Organization => OpenBSD
2014-06-28 22:42 mdempsky Section => mmap
2014-06-28 22:42 mdempsky Page Number =>
http://pubs.opengroup.org/onlinepubs/9699919799/functions/mmap.html
2014-06-28 22:56 mdempsky Note Added: 0002293
2014-06-29 00:20 mdempsky Note Added: 0002294
2014-07-02 18:02 mdempsky Note Added: 0002301
2014-07-04 17:04 markh Note Added: 0002302
2014-07-05 10:13 mdempsky Note Added: 0002303
2014-07-17 16:26 joerg Note Added: 0002311
2014-07-17 17:02 joerg Note Edited: 0002311
2014-07-17 17:03 joerg Note Edited: 0002311
2014-07-17 18:16 mdempsky Note Added: 0002312
2014-07-18 07:48 martinr Note Added: 0002313
======================================================================

Austin Group Bug Tracker

2014-07-18 16:03:20 UTC

Permalink

A NOTE has been added to this issue.
======================================================================
http://www.austingroupbugs.net/view.php?id=852
======================================================================
Reported By: mdempsky
Assigned To:
======================================================================
Project: 1003.1(2013)/Issue7+TC1
Issue ID: 852
Category: System Interfaces
Type: Clarification Requested
Severity: Editorial
Priority: normal
Status: New
Name: Matthew Dempsky
Organization: OpenBSD
User Reference:
Section: mmap
Page Number:
http://pubs.opengroup.org/onlinepubs/9699919799/functions/mmap.html
Line Number:
Interp Status: ---
Final Accepted Text:
======================================================================
Date Submitted: 2014-06-28 22:42 UTC
Last Modified: 2014-07-18 16:03 UTC
======================================================================
Summary: Clarify MAP_FIXED semantics when replacing existing
locked mappings
======================================================================

----------------------------------------------------------------------
(0002316) mdempsky (reporter) - 2014-07-18 16:03
http://www.austingroupbugs.net/view.php?id=852#c2316
----------------------------------------------------------------------
martinr: Thanks. Can you repeat your test but replace "munmap(p, 2 *
pagesize);" with "mmap(p, 2 * pagesize, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0);" and confirm that
segvn_lockop() is still called to unlock the original mapping?

Issue History
Date Modified Username Field Change
======================================================================
2014-06-28 22:42 mdempsky New Issue
2014-06-28 22:42 mdempsky Name => Matthew Dempsky
2014-06-28 22:42 mdempsky Organization => OpenBSD
2014-06-28 22:42 mdempsky Section => mmap
2014-06-28 22:42 mdempsky Page Number =>
http://pubs.opengroup.org/onlinepubs/9699919799/functions/mmap.html
2014-06-28 22:56 mdempsky Note Added: 0002293
2014-06-29 00:20 mdempsky Note Added: 0002294
2014-07-02 18:02 mdempsky Note Added: 0002301
2014-07-04 17:04 markh Note Added: 0002302
2014-07-05 10:13 mdempsky Note Added: 0002303
2014-07-17 16:26 joerg Note Added: 0002311
2014-07-17 17:02 joerg Note Edited: 0002311
2014-07-17 17:03 joerg Note Edited: 0002311
2014-07-17 18:16 mdempsky Note Added: 0002312
2014-07-18 07:48 martinr Note Added: 0002313
2014-07-18 16:03 mdempsky Note Added: 0002316
======================================================================