Commit Graph

12460 Commits

Author SHA1 Message Date
Junio C Hamano
ce22ea22e8 Merge branch 'js/perf-on-apple' into maint
t/perf needs /usr/bin/time with GNU extension; the invocation of it
is updated to "gtime" on Darwin.

* js/perf-on-apple:
  perf: accommodate for MacOSX
2016-07-11 10:44:15 -07:00
Junio C Hamano
c4cdde45f0 Merge branch 'ak/t7800-wo-readlink' into maint
One among four invocations of readlink(1) in our test suite has
been rewritten so that the test can run on systems without the
command (others are in valgrind test framework and t9802).

* ak/t7800-wo-readlink:
  t7800: readlink may not be available
2016-07-11 10:44:15 -07:00
Junio C Hamano
0c72d6da31 Merge branch 'jk/tzoffset-fix' into maint
The internal code used to show local timezone offset is not
prepared to handle timestamps beyond year 2100, and gave a
bogus offset value to the caller.  Use a more benign looking
+0000 instead and let "git log" going in such a case, instead
of aborting.

* jk/tzoffset-fix:
  local_tzoffset: detect errors from tm_to_time_t
  t0006: test various date formats
  t0006: rename test-date's "show" to "relative"
2016-07-11 10:44:14 -07:00
Junio C Hamano
1a88ca99db Merge branch 'sb/clone-shallow-passthru' into maint
Fix an unintended regression in v2.9 that breaks "clone --depth"
that recurses down to submodules by forcing the submodules to also
be cloned shallowly, which many server instances that host upstream
of the submodules are not prepared for.

* sb/clone-shallow-passthru:
  clone: do not let --depth imply --shallow-submodules
2016-07-11 10:44:12 -07:00
Junio C Hamano
5f30bb4a81 Merge branch 'nd/graph-width-padded' into maint
"log --graph --format=" learned that "%>|(N)" specifies the width
relative to the terminal's left edge, not relative to the area to
draw text that is to the right of the ancestry-graph section.  It
also now accepts negative N that means the column limit is relative
to the right border.

* nd/graph-width-padded:
  pretty.c: support <direction>|(<negative number>) forms
  pretty: pass graph width to pretty formatting for use in '%>|(N)'
2016-07-11 10:44:09 -07:00
Junio C Hamano
369dc4081c Merge branch 'mj/log-show-signature-conf'
"git log" learns log.showSignature configuration variable, and a
command line option "--no-show-signature" to countermand it.

* mj/log-show-signature-conf:
  log: add log.showSignature configuration variable
  log: add "--no-show-signature" command line option
  t4202: refactor test
2016-07-11 10:31:08 -07:00
Junio C Hamano
62e5e83f8d Merge branch 'js/find-commit-subject-ignore-leading-blanks'
A helper function that takes the contents of a commit object and
finds its subject line did not ignore leading blank lines, as is
commonly done by other codepaths.  Make it ignore leading blank
lines to match.

* js/find-commit-subject-ignore-leading-blanks:
  reset --hard: skip blank lines when reporting the commit subject
  sequencer: use skip_blank_lines() to find the commit subject
  commit -C: skip blank lines at the beginning of the message
  commit.c: make find_commit_subject() more robust
  pretty: make the skip_blank_lines() function public
2016-07-11 10:31:08 -07:00
Junio C Hamano
e9a6d71331 Merge branch 'jk/perf-any-version'
Allow t/perf framework to use the features from the most recent
version of Git even when testing an older installed version.

* jk/perf-any-version:
  p4211: explicitly disable renames in no-rename test
  t/perf: fix regression in testing older versions of git
2016-07-11 10:31:06 -07:00
Junio C Hamano
3c5de5c77b Merge branch 'jk/ansi-color'
The output coloring scheme learned two new attributes, italic and
strike, in addition to existing bold, reverse, etc.

* jk/ansi-color:
  color: support strike-through attribute
  color: support "italic" attribute
  color: allow "no-" for negating attributes
  color: refactor parse_attr
  add skip_prefix_mem helper
  doc: refactor description of color format
  color: fix max-size comment
2016-07-11 10:31:05 -07:00
Nguyễn Thái Ngọc Duy
6d308627ca worktree: add "unlock" command
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-08 15:31:04 -07:00
Nguyễn Thái Ngọc Duy
58142c09a4 worktree: add "lock" command
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-08 15:31:04 -07:00
Lars Schneider
d604176d23 git-p4: place temporary refs used for branch import under refs/git-p4-tmp
Git-P4 used to place temporary refs under "git-p4-tmp". Since 3da1f37
Git checks that all refs are placed under "refs". Instruct Git-P4 to
place temporary refs under "refs/git-p4-tmp". There are no backwards
compatibility considerations as these refs are transient.

Use "git show-ref --verify" to check the (non-)existience of the refs
instead of file checks assuming the file-based ref backend.

All refs under "refs" are shared across all worktrees. This is not
desired for temporary Git-P4 refs and will be adressed in a later patch.

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Reviewed-by: Vitor Antunes <vitor.hda@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-08 15:28:16 -07:00
Johannes Schindelin
cbcd2cbd59 rebase -i: we allow extra spaces after fixup!/squash!
This new test case ensures that we handle commit messages that start
with fixup! or squash! followed by more than one space. While we do
not generate such messages when committing with --fixup/--squash, it
is perfectly legal for users to hand-craft their own fixup messages,
and we heed Postel's law by being lenient.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-07 15:40:59 -07:00
Johannes Schindelin
c94e963b53 rebase -i: demonstrate a bug with --autosquash
When rearranging the edit script, we happily mistake the comment
character for a command, and the command for a SHA-1. As a consequence,
when we move fixup! and squash! commits, our logic to skip lines with
already handled SHA-1s mistakenly skips anything but the first
commented-out pick line, too.

The upcoming rebase--helper patches will address this bug, therefore we
do not need to make the current autosquash code even more complex than
it already is, just to fix this bug.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-07 15:40:23 -07:00
Johannes Schindelin
6672b9f962 t3404: add a test for the --gpg-sign option
For the upcoming rebase--helper work (which will accelerate the
interactive rebase noticably), it is important to verify that the
--gpg-sign option is handled properly.

Please note that this patch does this on the cheap, by verifying that
the expected option is printed in the message of the 'edit' operation.

We really should test that the interactive rebase signs the commits
properly, iff GPG is available. This test is left for later.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-07 15:36:56 -07:00
Junio C Hamano
afb516e364 Merge branch 'jc/t2300-setup'
Portability fix for Windows.

* jc/t2300-setup:
  t2300: "git --exec-path" is not usable in $PATH on Windows as-is
2016-07-06 13:38:20 -07:00
Junio C Hamano
4cea655a47 Merge branch 'cb/t7810-test-label-fix'
Test clean-up.

* cb/t7810-test-label-fix:
  t7810: fix duplicated test title
2016-07-06 13:38:18 -07:00
Junio C Hamano
8db528cf5a Merge branch 'sb/t5614-modernize'
Test clean-up.

* sb/t5614-modernize:
  t5614: don't use subshells
2016-07-06 13:38:17 -07:00
Junio C Hamano
3437017fec Merge branch 'js/perf-on-apple'
t/perf needs /usr/bin/time with GNU extension; the invocation of it
is updated to "gtime" on Darwin.

* js/perf-on-apple:
  perf: accommodate for MacOSX
2016-07-06 13:38:16 -07:00
Junio C Hamano
3edaee74fd Merge branch 'ak/t7800-wo-readlink'
One among four invocations of readlink(1) in our test suite has
been rewritten so that the test can run on systems without the
command (others are in valgrind test framework and t9802).

* ak/t7800-wo-readlink:
  t7800: readlink may not be available
2016-07-06 13:38:16 -07:00
Junio C Hamano
f6a729f344 Merge branch 'jk/tzoffset-fix'
The internal code used to show local timezone offset is not
prepared to handle timestamps beyond year 2100, and gave a
bogus offset value to the caller.  Use a more benign looking
+0000 instead and let "git log" going in such a case, instead
of aborting.

* jk/tzoffset-fix:
  local_tzoffset: detect errors from tm_to_time_t
  t0006: test various date formats
  t0006: rename test-date's "show" to "relative"
2016-07-06 13:38:15 -07:00
Junio C Hamano
9f1027d18a Merge branch 'sb/clone-shallow-passthru'
Fix an unintended regression in v2.9 that breaks "clone --depth"
that recurses down to submodules by forcing the submodules to also
be cloned shallowly, which many server instances that host upstream
of the submodules are not prepared for.

* sb/clone-shallow-passthru:
  clone: do not let --depth imply --shallow-submodules
2016-07-06 13:38:13 -07:00
Junio C Hamano
ed0f7bdec9 Merge branch 'jk/gpg-interface-cleanup'
A new run-command API function pipe_command() is introduced to
sanely feed data to the standard input while capturing data from
the standard output and the standard error of an external process,
which is cumbersome to hand-roll correctly without deadlocking.

The codepath to sign data in a prepared buffer with GPG has been
updated to use this API to read from the status-fd to check for
errors (instead of relying on GPG's exit status).

* jk/gpg-interface-cleanup:
  gpg-interface: check gpg signature creation status
  sign_buffer: use pipe_command
  verify_signed_buffer: use pipe_command
  run-command: add pipe_command helper
  verify_signed_buffer: use tempfile object
  verify_signed_buffer: drop pbuf variable
  gpg-interface: use child_process.args
2016-07-06 13:38:12 -07:00
Junio C Hamano
f2140c3890 Merge branch 'nd/graph-width-padded'
"log --graph --format=" learned that "%>|(N)" specifies the width
relative to the terminal's left edge, not relative to the area to
draw text that is to the right of the ancestry-graph section.  It
also now accepts negative N that means the column limit is relative
to the right border.

* nd/graph-width-padded:
  pretty.c: support <direction>|(<negative number>) forms
  pretty: pass graph width to pretty formatting for use in '%>|(N)'
2016-07-06 13:38:12 -07:00
Junio C Hamano
979f030359 Merge branch 'jk/repack-keep-unreachable'
"git repack" learned the "--keep-unreachable" option, which sends
loose unreachable objects to a pack instead of leaving them loose.
This helps heuristics based on the number of loose objects
(e.g. "gc --auto").

* jk/repack-keep-unreachable:
  repack: extend --keep-unreachable to loose objects
  repack: add --keep-unreachable option
  repack: document --unpack-unreachable option
2016-07-06 13:38:11 -07:00
Junio C Hamano
e25a4ded8a Merge branch 'ew/mboxrd-format-am'
Teach format-patch and mailsplit (hence "am") how a line that
happens to begin with "From " in the e-mail message is quoted with
">", so that these lines can be restored to their original shape.

* ew/mboxrd-format-am:
  am: support --patch-format=mboxrd
  mailsplit: support unescaping mboxrd messages
  pretty: support "mboxrd" output format
2016-07-06 13:38:11 -07:00
Junio C Hamano
1e4bf90789 Merge branch 'jk/upload-pack-hook'
"upload-pack" allows a custom "git pack-objects" replacement when
responding to "fetch/clone" via the uploadpack.packObjectsHook.

* jk/upload-pack-hook:
  upload-pack: provide a hook for running pack-objects
  t1308: do not get fooled by symbolic links to the source tree
  config: add a notion of "scope"
  config: return configset value for current_config_ functions
  config: set up config_source for command-line config
  git_config_parse_parameter: refactor cleanup code
  git_config_with_options: drop "found" counting
2016-07-06 13:38:11 -07:00
Junio C Hamano
34bf3bbb30 Merge branch 'nd/test-lib-httpd-show-error-log-in-verbose'
HTTPd tests learned to show the server error log to help diagnosing
a failing tests.

* nd/test-lib-httpd-show-error-log-in-verbose:
  lib-httpd.sh: print error.log on error
2016-07-06 13:38:08 -07:00
Junio C Hamano
b8b6365a8a Merge branch 'jk/string-list-static-init'
Instead of taking advantage of a struct string_list that is
allocated with all NULs happens to be STRING_LIST_INIT_NODUP kind,
initialize them explicitly as such, to document their behaviour
better.

* jk/string-list-static-init:
  use string_list initializer consistently
  blame,shortlog: don't make local option variables static
  interpret-trailers: don't duplicate option strings
  parse_opt_string_list: stop allocating new strings
2016-07-06 13:38:08 -07:00
Junio C Hamano
c8b080af71 Merge branch 'et/add-chmod-x' into maint
"git update-index --add --chmod=+x file" may be usable as an escape
hatch, but not a friendly thing to force for people who do need to
use it regularly.  "git add --chmod=+x file" can be used instead.

* et/add-chmod-x:
  add: add --chmod=+x / --chmod=-x options
2016-07-06 13:06:39 -07:00
Junio C Hamano
7949837520 Merge branch 'sg/reflog-past-root' into maint
"git reflog" stopped upon seeing an entry that denotes a branch
creation event (aka "unborn"), which made it appear as if the
reflog was truncated.

* sg/reflog-past-root:
  reflog: continue walking the reflog past root commits
2016-07-06 13:06:37 -07:00
Armin Kunaschik
c578a09bd6 t7610: test for mktemp before test execution
mktemp is not available on all platforms, so the test
'temporary filenames are used with mergetool.writeToTemp'
fails there.
This patch does not replace mktemp but just disables
the test that otherwise would fail.
mergetool checks itself before executing mktemp and
reports an error.

Signed-off-by: Armin Kunaschik <megabreit@googlemail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-06 12:18:09 -07:00
Torsten Bögershausen
6523728499 convert: unify the "auto" handling of CRLF
Before this change,
$ echo "* text=auto" >.gitattributes
$ echo "* eol=crlf" >>.gitattributes

would have the same effect as
$ echo "* text" >.gitattributes
$ git config core.eol crlf

Since the 'eol' attribute had higher priority than 'text=auto', this may
corrupt binary files and is not what most users expect to happen.

Make the 'eol' attribute to obey 'text=auto' and now
$ echo "* text=auto" >.gitattributes
$ echo "* eol=crlf" >>.gitattributes
behaves the same as
$ echo "* text=auto" >.gitattributes
$ git config core.eol crlf

In other words,
$ echo "* text=auto eol=crlf" >.gitattributes
has the same effect as
$ git config core.autocrlf true

and
$ echo "* text=auto eol=lf" >.gitattributes
has the same effect as
$ git config core.autocrlf input

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-06 11:53:51 -07:00
Nguyễn Thái Ngọc Duy
bc437d1020 fetch: reduce duplicate in ref update status lines with placeholder
In the "remote -> local" line, if either ref is a substring of the
other, the common part in the other string is replaced with "*". For
example

    abc                -> origin/abc
    refs/pull/123/head -> pull/123

become

    abc         -> origin/*
    refs/*/head -> pull/123

Activated with fetch.output=compact.

For the record, this output is not perfect. A single giant ref can
push all refs very far to the right and likely be wrapped around. We
may have a few options:

 - exclude these long lines smarter

 - break the line after "->", exclude it from column width calculation

 - implement a new format, { -> origin/}foo, which makes the problem
   go away at the cost of a bit harder to read

 - reverse all the arrows so we have "* <- looong-ref", again still
   hard to read.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-06 11:48:25 -07:00
Nguyễn Thái Ngọc Duy
6bc91f23a6 fetch: align all "remote -> local" output
We do align "remote -> local" output by allocating 10 columns to
"remote". That produces aligned output only for short refs. An extra
pass is performed to find the longest remote ref name (that does not
produce a line longer than terminal width) to produce better aligned
output.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-06 11:48:25 -07:00
Junio C Hamano
de61cebde7 Merge branch 'jk/common-main-2.8' into jk/common-main
* jk/common-main-2.8:
  mingw: declare main()'s argv as const
  common-main: call git_setup_gettext()
  common-main: call restore_sigpipe_to_default()
  common-main: call sanitize_stdfds()
  common-main: call git_extract_argv0_path()
  add an extra level of indirection to main()
2016-07-06 10:02:57 -07:00
Jeff King
03c39b3458 t/lib-git-daemon: use test_match_signal
When git-daemon exits, we expect it to be with the SIGTERM
we just sent it. If we see anything else, we'll complain.
But our check against exit code "143" is not portable. For
example:

  $ ksh93 t5570-git-daemon.sh
  [...]
  error: git daemon exited with status: 271

We can fix this by using test_match_signal.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-06 07:44:25 -07:00
Jeff King
2472448c88 test_must_fail: use test_match_signal
In 8bf4bec (add "ok=sigpipe" to test_must_fail and use it to
fix flaky tests, 2015-11-27), test_must_fail learned to
recognize "141" as a sigpipe failure. However, testing for
a signal is more complicated than that; we should use
test_match_signal to implement more portable checking.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-06 07:44:25 -07:00
Jeff King
6f5f9d7476 t0005: use test_match_signal as appropriate
The first test already uses this more portable construct
(that was where it was factored from initially), but the
later tests do a raw comparison against 141 to look for
SIGPIPE, which can fail on some shells and platforms.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-06 07:44:25 -07:00
Jeff King
9b67c9942e tests: factor portable signal check out of t0005
In POSIX shells, a program which exits due to a signal
generally has an exit code of 128 plus the signal number.
However, ksh uses 256 plus the signal number.  We've
accounted for that in t0005, but not in other tests.  Let's
pull out the logic so we can use it elsewhere.

It would be nice for debugging if this additionally printed
errors to stderr, like our other test_* helpers. But we're
going to need to use it in other places besides the innards
of a test_expect block. So let's leave it as generic as
possible.

Note that we also leave the magic "3" for Windows out of the
generic helper. This is an artifact of the way we use
raise() to kill ourselves in test-sigchain.c, and will not
necessarily apply to all programs. So it's better to keep it
out of the helper, to reduce the chance of confusing it with
a real call to exit(3).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-06 07:43:29 -07:00
Charles Bailey
b8e47d1acf grep: fix grepping for "intent to add" files
This reverts commit 4d5520053 (grep: make it clear i-t-a entries are
ignored, 2015-12-27) and adds an alternative fix to maintain the -L
--cached behavior.

4d5520053 caused 'git grep' to no longer find matches in new files in
the working tree where the corresponding index entry had the "intent to
add" bit set, despite the fact that these files are tracked.

The content in the index of a file for which the "intent to add" bit is
set is considered indeterminate and not empty. For most grep queries we
want these to behave the same, however for -L --cached (files without a
match) we don't want to respond positively for "intent to add" files as
their contents are indeterminate. This is in contrast to files with
empty contents in the index (no lines implies no matches for any grep
query expression) which should be reported in the output of a grep -L
--cached invocation.

Add tests to cover this case and a few related cases which previously
lacked coverage.

Helped-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Charles Bailey <cbailey32@bloomberg.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01 13:27:41 -07:00
Charles Bailey
89e64100f4 t7810-grep.sh: fix a whitespace inconsistency
Signed-off-by: Charles Bailey <charles@hashpling.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01 13:27:38 -07:00
Charles Bailey
878452b966 t7810-grep.sh: fix duplicated test name
Signed-off-by: Charles Bailey <charles@hashpling.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01 13:26:30 -07:00
Vasco Almeida
415c7dd026 t5541: become resilient to GETTEXT_POISON
Use test_i18n* functions for testing text already marked for
translation.

Signed-off-by: Vasco Almeida <vascomalmeida@sapo.pt>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01 12:47:30 -07:00
Nguyễn Thái Ngọc Duy
b51a9c1479 diffcore-pickaxe: support case insensitive match on non-ascii
Similar to the "grep -F -i" case, we can't use kws on icase search
outside ascii range, so we quote the string and pass it to regcomp as
a basic regexp and let regex engine deal with case sensitivity.

The new test is put in t7812 instead of t4209-log-pickaxe because
lib-gettext.sh might cause problems elsewhere, probably.

Noticed-by: Plamen Totev <plamen.totev@abv.bg>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01 12:44:57 -07:00
Nguyễn Thái Ngọc Duy
18547aacf5 grep/pcre: support utf-8
In the previous change in this function, we add locale support for
single-byte encodings only. It looks like pcre only supports utf-* as
multibyte encodings, the others are left in the cold (which is
fine).

We need to enable PCRE_UTF8 so pcre can find character boundary
correctly. It's needed for case folding (when --ignore-case is used)
or '*', '+' or similar syntax is used.

The "has_non_ascii()" check is to be on the conservative side. If
there's non-ascii in the pattern, the searched content could still be
in utf-8, but we can treat it just like a byte stream and everything
should work. If we force utf-8 based on locale only and pcre validates
utf-8 and the file content is in non-utf8 encoding, things break.

Noticed-by: Plamen Totev <plamen.totev@abv.bg>
Helped-by: Plamen Totev <plamen.totev@abv.bg>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01 12:44:57 -07:00
Nguyễn Thái Ngọc Duy
9d9babb84d grep/pcre: prepare locale-dependent tables for icase matching
The default tables are usually built with C locale and only suitable
for LANG=C or similar.  This should make case insensitive search work
correctly for all single-byte charsets.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01 12:44:57 -07:00
Nguyễn Thái Ngọc Duy
793dc676e0 grep/icase: avoid kwsset when -F is specified
Similar to the previous commit, we can't use kws on icase search
outside ascii range. But we can't simply pass the pattern to
regcomp/pcre like the previous commit because it may contain regex
special characters, so we need to quote the regex first.

To avoid misquote traps that could lead to undefined behavior, we
always stick to basic regex engine in this case. We don't need fancy
features for grepping a literal string anyway.

basic_regex_quote_buf() assumes that if the pattern is in a multibyte
encoding, ascii chars must be unambiguously encoded as single
bytes. This is true at least for UTF-8. For others, let's wait until
people yell up. Chances are nobody uses multibyte, non utf-8 charsets
anymore.

Noticed-by: Plamen Totev <plamen.totev@abv.bg>
Helped-by: René Scharfe <l.s.r@web.de>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01 12:44:30 -07:00
Jeff King
6e8e0991e5 archive-tar: write extended headers for far-future mtime
The ustar format represents timestamps as seconds since the
epoch, but only has room to store 11 octal digits.  To
express anything larger, we need to use an extended header.
This is exactly the same case we fixed for the size field in
the previous commit, and the solution here follows the same
pattern.

This is even mentioned as an issue in f2f0267 (archive-tar:
use xsnprintf for trivial formatting, 2015-09-24), but since
it only affected things far in the future, it wasn't deemed
worth dealing with. But note that my calculations claiming
thousands of years were off there; because our xsnprintf
produces a NUL byte, we only have until the year 2242 to fix
this.

Given that this is just around the corner (geologically
speaking, anyway), and because it's easy to fix, let's just
make it work. Unlike the previous fix for "size", where we
had to write an individual extended header for each file, we
can write one global header (since we have only one mtime
for the whole archive).

There's a slight bit of trickiness there. We may already be
writing a global header with a "comment" field for the
commit sha1. So we need to write our new field into the same
header. To do this, we push the decision of whether to write
such a header down into write_global_extended_header(),
which will now assemble the header as it sees fit, and will
return early if we have nothing to write (in practice, we'll
only have a large mtime if it comes from a commit, but this
makes it also work if you set your system clock ahead such
that time() returns a huge value).

Note that we don't (and never did) handle negative
timestamps (i.e., before 1970). This would probably not be
too hard to support in the same way, but since git does not
support negative timestamps at all, I didn't bother here.

After writing the extended header, we munge the timestamp in
the ustar headers to the maximum-allowable size. This is
wrong, but it's the least-wrong thing we can provide to a
tar implementation that doesn't understand pax headers (it's
also what GNU tar does).

Helped-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01 10:26:01 -07:00
Jeff King
d1657b570a archive-tar: write extended headers for file sizes >= 8GB
The ustar format has a fixed-length field for the size of
each file entry which is supposed to contain up to 11 bytes
of octal-formatted data plus a NUL or space terminator.

These means that the largest size we can represent is
077777777777, or 1 byte short of 8GB. The correct solution
for a larger file, according to POSIX.1-2001, is to add an
extended pax header, similar to how we handle long
filenames. This patch does that, and writes zero for the
size field in the ustar header (the last bit is not
mentioned by POSIX, but it matches how GNU tar behaves with
--format=pax).

This should be a strict improvement over the current
behavior, which is to die in xsnprintf with a "BUG".
However, there's some interesting history here.

Prior to f2f0267 (archive-tar: use xsnprintf for trivial
formatting, 2015-09-24), we silently overflowed the "size"
field. The extra bytes ended up in the "mtime" field of the
header, which was then immediately written itself,
overwriting our extra bytes. What that means depends on how
many bytes we wrote.

If the size was 64GB or greater, then we actually overflowed
digits into the mtime field, meaning our value was
effectively right-shifted by those lost octal digits. And
this patch is again a strict improvement over that.

But if the size was between 8GB and 64GB, then our 12-byte
field held all of the actual digits, and only our NUL
terminator overflowed. According to POSIX, there should be a
NUL or space at the end of the field. However, GNU tar seems
to be lenient here, and will correctly parse a size up 64GB
(minus one) from the field. So sizes in this range might
have just worked, depending on the implementation reading
the tarfile.

This patch is mostly still an improvement there, as the 8GB
limit is specifically mentioned in POSIX as the correct
limit. But it's possible that it could be a regression
(versus the pre-f2f0267 state) if all of the following are
true:

  1. You have a file between 8GB and 64GB.

  2. Your tar implementation _doesn't_ know about pax
     extended headers.

  3. Your tar implementation _does_ parse 12-byte sizes from
     the ustar header without a delimiter.

It's probably not worth worrying about such an obscure set
of conditions, but I'm documenting it here just in case.

Helped-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01 10:25:46 -07:00