Discussion:
Question about timestamps and periodic update
Tom Ridge
2014-06-27 12:08:28 UTC
Permalink
Dear List,

Reading Base Definitions, Chapter 4, Section 4.8 "File Times Update", the
specification talks about timestamps being "marked for update". It then
says:

"An implementation may update timestamps that are marked for update
immediately, or it may update such timestamps periodically."

My question is: what real-world filesystem implementations actually update
timestamps periodically?

Thanks
Casper.Dik-QHcLZuEGTsvQT0dZR+
2014-06-27 12:20:21 UTC
Permalink
Post by Tom Ridge
Dear List,
Reading Base Definitions, Chapter 4, Section 4.8 "File Times Update", the
specification talks about timestamps being "marked for update". It then
"An implementation may update timestamps that are marked for update
immediately, or it may update such timestamps periodically."
My question is: what real-world filesystem implementations actually update
timestamps periodically?
Depends what you actually mean.

The Solaris UFS file systems, e.g., does not update access times unless
the disk is accessed anyway. If the observer is sitting directly on the
disk, he will see that the time stamp is updated much later.

However, an observer inside the operating system will believe that the
access times have already been changed.

Your question is perhaps about observation from within the POSIX operating
system; then I don't think I know of such file systems other than that
many file systems have the option to not update certain time stamps.

Casper
Schwarz, Konrad
2014-06-27 12:58:25 UTC
Permalink
(Sorry for top-posting)

In addition to what Caspar Dik wrote, I think that “periodically” here means
“from time to time”, i.e., whenever convenient for the operating system.
Conversely, should the operating system crash, the recorded times on disk
might not reflect the true last access times, for example.

From: Tom Ridge [mailto:***@googlemail.com]
Sent: Freitag, 27. Juni 2014 14:08
To: austin-group-***@opengroup.org
Subject: Question about timestamps and periodic update

Dear List,

Reading Base Definitions, Chapter 4, Section 4.8 "File Times Update", the specification talks about timestamps being "marked for update". It then says:

"An implementation may update timestamps that are marked for update immediately, or it may update such timestamps periodically."

My question is: what real-world filesystem implementations actually update timestamps periodically?

Thanks
Tom Ridge
2014-06-27 13:51:46 UTC
Permalink
Thanks for these replies.

As I understand it, POSIX mostly talks about the behaviour observable
within the POSIX operating system. Rarely it talks about the on-disk
behaviour (obvious exceptions are sync and fsync). My understanding of the
"periodic timestamp updates" is that this is describing the behaviour
observable within the POSIX operating system. Given this, I don't
understand why the "periodic timestamp updates" behaviour is mentioned,
given that no operating systems actually do this (and it is hard to
conceive that any would want to). Can someone describe the motivation for
this part of POSIX?

Just to be clear: I realize that the on-disk and in-memory versions of the
file system may differ quite a lot. However, the relationship between the
two does not seem to be really addressed by POSIX. So as far as I can see,
the on-disk behaviour is a separate issue to whether the POSIX "periodic
timestamp update" is observable within the POSIX operating system itself.

Thanks
Post by Schwarz, Konrad
(Sorry for top-posting)
In addition to what Caspar Dik wrote, I think that “periodically” here means
“from time to time”, i.e., whenever convenient for the operating system.
Conversely, should the operating system crash, the recorded times on disk
might not reflect the true last access times, for example.
*Sent:* Freitag, 27. Juni 2014 14:08
*Subject:* Question about timestamps and periodic update
Dear List,
Reading Base Definitions, Chapter 4, Section 4.8 "File Times Update", the
specification talks about timestamps being "marked for update". It then
"An implementation may update timestamps that are marked for update
immediately, or it may update such timestamps periodically."
My question is: what real-world filesystem implementations actually update
timestamps periodically?
Thanks
Geoff Clare
2014-06-27 16:18:08 UTC
Permalink
Post by Tom Ridge
My understanding of the
"periodic timestamp updates" is that this is describing the behaviour
observable within the POSIX operating system. Given this, I don't
understand why the "periodic timestamp updates" behaviour is mentioned,
given that no operating systems actually do this (and it is hard to
conceive that any would want to). Can someone describe the motivation for
this part of POSIX?
I'm confused. Given what Konrad said ("periodically" means from time
to time, or as needed, not that there's a set period of time), don't
all POSIX systems do it (at least for "normal" file systems)?

See for example the ufs_itimes_locked() function in

https://github.com/freebsd/freebsd/blob/master/sys/ufs/ufs/ufs_vnops.c

This looks to see which timestamps have been "marked for update"
(IN_ACCESS, IN_CHANGE or IN_UPDATE set), and sets the corresponding
timestamps in the inode to the current time:

if ((ip->i_flag & (IN_ACCESS | IN_CHANGE | IN_UPDATE)) == 0)
return;

[...]

vfs_timestamp(&ts);
if (ip->i_flag & IN_ACCESS) {
DIP_SET(ip, i_atime, ts.tv_sec);
DIP_SET(ip, i_atimensec, ts.tv_nsec);
}
if (ip->i_flag & IN_UPDATE) {
DIP_SET(ip, i_mtime, ts.tv_sec);
DIP_SET(ip, i_mtimensec, ts.tv_nsec);
}
if (ip->i_flag & IN_CHANGE) {
DIP_SET(ip, i_ctime, ts.tv_sec);
DIP_SET(ip, i_ctimensec, ts.tv_nsec);
DIP_SET(ip, i_modrev, DIP(ip, i_modrev) + 1);
}

out:
ip->i_flag &= ~(IN_ACCESS | IN_CHANGE | IN_UPDATE);

Read and write operations just set the appropriate IN_* flags; the
actual timestamp update is done at some later time by this function
(e.g. on a close() or stat()).
--
Geoff Clare <g.clare-7882/***@public.gmane.org>
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England
Tom Ridge
2014-06-27 17:30:01 UTC
Permalink
ufs_itimes_locked() seems to operate on individual files, and use a
different timestamp on each invocation. If multiple files have "marked for
update", and a periodic update occurs, then (if I read the spec correctly)
***all*** marked-for-update files have to have ***the same timestamp***.

Now, it could be that ufs only updates timestamps on a stat or close or
similar (ie no period ever occurs - this is one possible interpretation of
the "periodic update" scenario). But the problem with this implementation
is that some times that are "marked for update" may need to be written to
disk without a stat, close, or similar. At this point, an actual time value
presumably has to be chosen, and *all* marked-for-update timestamps have to
be updated (whether they are updated on-disk or not). But at this point,
any performance benefit from delaying the timestamp update is presumably
mostly mitigated by having to walk the in-memory list of marked-for-update
files and update them.

Looking at the spec, my issue is with the sentence (Base defns, 4.8) "At
the point in time when an update occurs, any marked timestamps shall be set
to the current time and the update marks shall be cleared."

My guess is that no file system implementations actually perform a
"periodic update" as described here (where *all* marked-for-update
timestamps on *all* files are updated with a single time value).

Does this make sense?
Post by Geoff Clare
Post by Tom Ridge
My understanding of the
"periodic timestamp updates" is that this is describing the behaviour
observable within the POSIX operating system. Given this, I don't
understand why the "periodic timestamp updates" behaviour is mentioned,
given that no operating systems actually do this (and it is hard to
conceive that any would want to). Can someone describe the motivation for
this part of POSIX?
I'm confused. Given what Konrad said ("periodically" means from time
to time, or as needed, not that there's a set period of time), don't
all POSIX systems do it (at least for "normal" file systems)?
See for example the ufs_itimes_locked() function in
https://github.com/freebsd/freebsd/blob/master/sys/ufs/ufs/ufs_vnops.c
This looks to see which timestamps have been "marked for update"
(IN_ACCESS, IN_CHANGE or IN_UPDATE set), and sets the corresponding
if ((ip->i_flag & (IN_ACCESS | IN_CHANGE | IN_UPDATE)) == 0)
return;
[...]
vfs_timestamp(&ts);
if (ip->i_flag & IN_ACCESS) {
DIP_SET(ip, i_atime, ts.tv_sec);
DIP_SET(ip, i_atimensec, ts.tv_nsec);
}
if (ip->i_flag & IN_UPDATE) {
DIP_SET(ip, i_mtime, ts.tv_sec);
DIP_SET(ip, i_mtimensec, ts.tv_nsec);
}
if (ip->i_flag & IN_CHANGE) {
DIP_SET(ip, i_ctime, ts.tv_sec);
DIP_SET(ip, i_ctimensec, ts.tv_nsec);
DIP_SET(ip, i_modrev, DIP(ip, i_modrev) + 1);
}
ip->i_flag &= ~(IN_ACCESS | IN_CHANGE | IN_UPDATE);
Read and write operations just set the appropriate IN_* flags; the
actual timestamp update is done at some later time by this function
(e.g. on a close() or stat()).
--
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England
Philip Guenther
2014-06-27 18:08:21 UTC
Permalink
Post by Tom Ridge
ufs_itimes_locked() seems to operate on individual files, and use a
different timestamp on each invocation. If multiple files have "marked for
update", and a periodic update occurs, then (if I read the spec correctly)
***all*** marked-for-update files have to have ***the same timestamp***.
...
Post by Tom Ridge
Looking at the spec, my issue is with the sentence (Base defns, 4.8) "At
the point in time when an update occurs, any marked timestamps shall be set
to the current time and the update marks shall be cleared."
My guess is that no file system implementations actually perform a
"periodic update" as described here (where *all* marked-for-update
timestamps on *all* files are updated with a single time value).
Does this make sense?
Post by Tom Ridge
An implementation may update timestamps that are marked for update
immediately, or it may
Post by Tom Ridge
Post by Tom Ridge
update such timestamps periodically. At the point in time when an update
occurs, any marked
Post by Tom Ridge
Post by Tom Ridge
timestamps shall be set to the current time and the update marks shall
be cleared. All
Post by Tom Ridge
Post by Tom Ridge
timestamps that are marked for update shall be updated when the file
ceases to be open by any
Post by Tom Ridge
Post by Tom Ridge
process or before a fstat(), fstatat(), fsync(), futimens(), lstat(),
stat(), utime(), utimensat(), or
Post by Tom Ridge
Post by Tom Ridge
utimes() is successfully performed on the file.
I read the phrases "any marked timestamps" and "all timestamps that are
marked" as referring to those *for a given file*, and not for all files
open on the system. That is, if both atime and mtime are marked for update
for a given file, then if the kernel actually updates atime to the current
time then it must *also* update mtime then and not leave it marked for
update, etc.


Philip Guenther
Tom Ridge
2014-06-28 10:53:13 UTC
Permalink
Ah! I think this is the source of my confusion. Thanks.
Post by Tom Ridge
Post by Tom Ridge
ufs_itimes_locked() seems to operate on individual files, and use a
different timestamp on each invocation. If multiple files have "marked for
update", and a periodic update occurs, then (if I read the spec correctly)
***all*** marked-for-update files have to have ***the same timestamp***.
...
Post by Tom Ridge
Looking at the spec, my issue is with the sentence (Base defns, 4.8) "At
the point in time when an update occurs, any marked timestamps shall be set
to the current time and the update marks shall be cleared."
My guess is that no file system implementations actually perform a
"periodic update" as described here (where *all* marked-for-update
timestamps on *all* files are updated with a single time value).
Does this make sense?
Post by Tom Ridge
An implementation may update timestamps that are marked for update
immediately, or it may
Post by Tom Ridge
Post by Tom Ridge
update such timestamps periodically. At the point in time when an
update occurs, any marked
Post by Tom Ridge
Post by Tom Ridge
timestamps shall be set to the current time and the update marks shall
be cleared. All
Post by Tom Ridge
Post by Tom Ridge
timestamps that are marked for update shall be updated when the file
ceases to be open by any
Post by Tom Ridge
Post by Tom Ridge
process or before a fstat(), fstatat(), fsync(), futimens(), lstat(),
stat(), utime(), utimensat(), or
Post by Tom Ridge
Post by Tom Ridge
utimes() is successfully performed on the file.
I read the phrases "any marked timestamps" and "all timestamps that are
marked" as referring to those *for a given file*, and not for all files
open on the system. That is, if both atime and mtime are marked for update
for a given file, then if the kernel actually updates atime to the current
time then it must *also* update mtime then and not leave it marked for
update, etc.
Philip Guenther
Philip Guenther
2014-06-27 16:30:01 UTC
Permalink
Post by Tom Ridge
Reading Base Definitions, Chapter 4, Section 4.8 "File Times Update", the
specification talks about timestamps being "marked for update". It then
"An implementation may update timestamps that are marked for update
immediately, or it may update such timestamps periodically."
My question is: what real-world filesystem implementations actually update
timestamps periodically?
Where the standard requires that a timestamp be "marked for update", BSD
FFS sets flag bits on the in-memory representation of the inode indicating
that the indicated timestamp must be updated before writing to disk or
being observed. So, if you do this:

open a file
write to it
get the current time
wait a bit
fstat it

Then assuming no other access to that inode, BSD FFS will delay the
timestamp update until the fstat and thus show the time of the fstat
instead of the time of the write.


Philip Guenther
Loading...