Differences between revisions 8 and 9
Revision 8 as of 2013-12-16 11:11:25
Size: 11080
Editor: JulianFoad
Comment: Comment on the svn:text-time format.
Revision 9 as of 2013-12-16 11:21:42
Size: 11076
Editor: JulianFoad
Comment: Fix a malformed link and some mark-up.
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
''The content of this page was written by Rick Yorgason: see <''http://svn.haxx.se/dev/archive-2012-02/0492.shtml>. ''The content of this page was written by Rick Yorgason: see <[[http://svn.haxx.se/dev/archive-2012-02/0492.shtml]]>.''
Line 9: Line 9:
 ''[JAF] ''''''According to [[http://svn.apache.org/repos/asf/subversion/trunk/subversion/include/svn_props.h|svn_props.h]], t''he format has a 'Z' on the end''.  ''[JAF] According to [[http://svn.apache.org/repos/asf/subversion/trunk/subversion/include/svn_props.h|svn_props.h]], the format has a 'Z' on the end.''

The content of this page was written by Rick Yorgason: see <http://svn.haxx.se/dev/archive-2012-02/0492.shtml>.


Design: MTime Preservation

Nearly every modern operating system associates a "modified time" with each file on the system, updating it whenever the file is modified. This metadata can be very important, but Subversion currently offers no way to preserve it.

This design proposes adding a new 'svn:text-time' property to store the modified time of each file or directory in UTC time using the format 'YYYY-MM-DDTHH:MM:SS.UUUUUU'. The name of this property was chosen because it is already used in an identical manner by FSVS and svntar.

  • [JAF] According to svn_props.h, the format has a 'Z' on the end.

Rules

  • The svn:text-time property is only relevant when the node is in the repository and at the moment of going into or out of the repository.
    • Conceptually, svn:text-time is only updated when a node is going into the repository, and mtime is only updated when a node is coming out of the repository.

  • When mtime changes locally, svn:text-time is not expected to remain up-to-date; it merely indicates that the svn:text-time property should be updated during the next commit.
  • The presence of svn:text-time on a node indicates that mtime preservation is expected for that node. It may be blank, indicating that it should be updated at the next commit. In this way, auto-props can be used to specify which nodes should preserve mtime.
  • If svn:text-time differs from BASE.svn:text-time, this counts as a local modification, as always.
  • If svn:text-time differs from mtime, this also counts as a local modification, even for mtime-only modifications.
    • Note: This is not true if mtime-usage is set to 'now' or 'commit-time'. See the 'commit' operation for details.

  • Not all operating systems can support the resolution of the svn:text-time format specified. When setting mtime, the time should always be rounded down to the nearest supported resolution first. When comparing mtime against svn:text-time, svn:text-time should always be rounded down first.
  • As svn:text-time is managed by Subversion, it should never conflict, and will always auto-resolve as necessary.

Definitions

mtime
Modified time. In this specification, mtime will always refer to the modified time of a file as stored by the operating system. Under this specification, Subversion will sometimes alter the mtime to equal the value stored in svn:text-time.
stime
System time. The current time according to the operating system at the time of the operation in question; aka now().
ctime
Commit time. The timestamp currently stored by Subversion indicating when the file or directory was committed.

mtime-usage

A new client-side configuration option named 'mtime-usage' should be introduced to specify how Subversion treats a node's mtime during an update operation. It can be one of the following options:

  • text-time-or-now (default; sets mtime=svn:text-time if possible, else sets mtime=stime)
  • text-time-or-commit-time (sets mtime=svn:text-time if possible, else sets mtime=ctime)
  • now (sets mtime=stime; identical to current default behaviour)
  • commit-time (sets mtime=commit-time; identical to use-commit-times)

mtime-usage obsoletes use-commit-times. If mtime-usage is unset and use-commit-times is true, the behaviour is identical to if mtime-usage was set to commit-time.

Operations

  • Any operations not listed are presumed to be unaffected by this proposal.
  • For the sake of being succinct, 'svn:' will be elided from 'svn:text-time' in the following operations.
  • The operations below apply to individual files or directories. For operations that support multiple nodes, the operation is simply repeated for each node.

svn commit

  • If local modifications exist and text-time is defined, text-time = mtime
    • Note: local modifications include mtime-only modifications iff mtime-usage is set to one of the two text-time-or- options. Otherwise every update operation would cause a local modification.

svn update

(The following logic is applied after the file has been updated.)

  • If mtime-usage == now, mtime = stime
  • Else if mtime-usage == commit-time, mtime = ctime
  • Else if text-time is defined:
    • If no local modifications exist, mtime = text-time
    • Else, BASE.text-time = text-time and mtime = stime
  • Else:
    • If text-time was defined locally, delete it.
    • If mtime-usage == text-time-or-now, repeat assuming mtime-usage == now
    • Else if mtime-usage == text-time-or-commit-time, repeat assuming mtime-usage == commit-time

svn checkout

  • Identical to an update from r0

svn merge

  • If no conflicts occur, behaviour is identical to svn update.
  • Else, no changes are made to either mtime or text-time.

svn resolve

  • If (resolved with base, mine-full, or theirs-full) and the file chosen has text-time defined, text-time is copied and mtime=text-time.

svn copy

  • Identical to current behaviour:
    • In WC copies, mtime = stime
    • text-time is preserved.

svn rename

  • Identical to current behaviour:
    • mtime is unchanged.
    • text-time is preserved.

svn revert

  • Following the revert, mtime = now, commit-time, or text-time, as defined by mtime-usage.

Use cases

Control cases

These are cases that Subversion is currently used for, and are expected to continue to work identically.

make

The build system 'make' defines a specific set of output files for a specific set of input files. Whenever the output files are built, their mtime is set to stime. Whenever rebuilding, it checks to see if mtime of the the input files is newer than the mtime of the output files and rebuilds them if that's the case.

Implementation:

  • The output files must not be under version control.
  • The input files must not have text-time set.

Log files

Log files are often opened and resaved, even when nothing is written to them. This is not expected to count as a modification.

Implementation:

  • text-time must not be set on the log files.

Compiler-generated files

When building software, the compiler may be configured to output a compiler-generated file. These often get re-generated with identical output, only modifying the mtime. This is not expected to count as a modification.

Implementation:

  • text-time must not be set on the output files.
  • To reduce the number of unnecessary generations, it should also not be set on the input files.

Motivating use cases

These are some of the reasons that users require this feature.

Image compression

In this scenario, the user stores uncompressed images in one folder and then runs a script to compress the images and dump them in another folder. The compression tool can generate slightly different bitstreams on different computers. Both folders are required to be under version control, and should avoid recompressing images that have not changed.

Implementation:

  • Auto-props should be set up to add text-time to both the input and output image files.
  • Any existing files must have text-time added manually.
  • The script must set the mtime of each output file to the mtime of the matching input file and only recompress when the mtimes differ.

Subversion as a backup tool

In this scenario, Subversion is primarily used to detect modified files and upload them to an off-site server; version control is only used in rare cases, such as a file being backup up after it was corrupted.

Implementation:

  • Auto-props should be set up to add text-time to all files.
  • Any existing file must have text-time added manually.

Finding documents by publication date

Some users maintain their mtime to refer to the publication date of a file. It should be possible to locate a document based on its mtime, even if that document was committed weeks after it was modified.

Implementation:

  • Auto-props should be set up to add text-time to all document files.
  • In an un-mixed environment, matching '*' would be sufficient.
  • Any existing files must have text-time added manually.

Web server

Web servers expose the mtime as the 'Last-Modified-Date' header in http requests, which is used in a variety of ways. Clients use it to determine whether or not the file needs to be re-cached, reducing bandwidth for the server. Search engines use it to determine if the page needs to be reindexed, or when the user requests to find a page matching a certain date range. Users use it when they need to determine if the information on a page is obsolete. Web servers also offer index pages, which show all the files in a directory and allow the user to sort by mtime.

Implementation:

  • Auto-props should be set up to add text-time to all document files.
  • Web sites are often mixed environments, so some file types may have to be excluded to ensure tools like 'make' continue to work.
  • Any existing document files must have text-time added manually.

Auto-props deficiencies

This proposal relies heavily on auto-props, and as a result, betrays some of the deficiencies extant in the auto-props system.

Most of the motivating use cases would benefit from having all users share the same auto-props. TortoiseSVN users can do this today using tsvn:autoprops, but there is no standard way to accomplish this. See Issue 1974: server-side config which 'broadcasts' to clients

Auto-props does not apply to directories. This means that directories cannot preserve mtimes without manually adding the svn:text-time property to all directories. See Issue 1989: make auto-props affect directories too

Auto-props only applies properties to newly imported/added files, so any existing files will have to have svn:text-time added manually when adopting the new feature. One solution would be to extend auto-props to allow the user to specify 'important' rules that should be applied on commit, rather than just on import/add. Another solution would be to add a command to svn that recursively applies the auto-props rules to a selected directory. I am not aware of any filed issues that request these features.

References


The content of this page was written by Rick Yorgason: see <http://svn.haxx.se/dev/archive-2012-02/0492.shtml>.

MtimePreservation (last edited 2013-12-16 11:21:42 by JulianFoad)