perf-test update.
* jh/memihash-opt:
p0004: don't error out if test repo is too small
p0004: don't abort if multi-threaded is too slow
p0004: use test_perf
p0004: avoid using pipes
p0004: simplify calls of test-lazy-init-name-hash
"git pull --rebase --autostash" didn't auto-stash when the local history
fast-forwards to the upstream.
* tb/pull-ff-rebase-autostash:
pull: ff --rebase --autostash works in dirty repo
"git clean -d" used to clean directories that has ignored files,
even though the command should not lose ignored ones without "-x".
"git status --ignored" did not list ignored and untracked files
without "-uall". These have been corrected.
* sl/clean-d-ignored-fix:
clean: teach clean -d to preserve ignored paths
dir: expose cmp_name() and check_contains()
dir: hide untracked contents of untracked dirs
dir: recurse into untracked dirs for ignored files
t7061: status --ignored should search untracked dirs
t7300: clean -d should skip dirs with ignored files
A recent update to t5545-push-options.sh started skipping all the
tests in the script when a web server testing is disabled or
unavailable, not just the ones that require a web server. Non HTTP
tests have been salvaged to always run in this script.
* jc/skip-test-in-the-middle:
t5545: enhance test coverage when no http server is installed
test: allow skipping the remainder
The "run-command" API implementation has been made more robust
against dead-locking in a threaded environment.
* bw/forking-and-threading:
usage.c: drop set_error_handle()
run-command: restrict PATH search to executable files
run-command: expose is_executable function
run-command: block signals between fork and execve
run-command: add note about forking and threading
run-command: handle dup2 and close errors in child
run-command: eliminate calls to error handling functions in child
run-command: don't die in child when duping /dev/null
run-command: prepare child environment before forking
string-list: add string_list_remove function
run-command: use the async-signal-safe execv instead of execvp
run-command: prepare command before forking
t0061: run_command executes scripts without a #! line
t5550: use write_script to generate post-update hook
Introduce the BUG() macro to improve die("BUG: ...").
* jk/bug-to-abort:
usage: add NORETURN to BUG() function definitions
config: complain about --local outside of a git repo
setup_git_env: convert die("BUG") to BUG()
usage.c: add BUG() function
"git checkout --recurse-submodules" did not quite work with a
submodule that itself has submodules.
* sb/checkout-recurse-submodules:
submodule: properly recurse for read-tree and checkout
submodule: avoid auto-discovery in new working tree manipulator code
submodule_move_head: reuse child_process structure for futher commands
The `git stash push` command recently gained the ability to get a
pathspec as its argument to only stash matching files. Calling this
command from a subdirectory does not work, though, as one of the first
things we do is changing to the top level directory without keeping
track of the prefix from which the command is being run.
Fix the shortcoming by storing the prefix previous to the call to
`cd_to_toplevel` and then subsequently using `git rev-parse --prefix` to
correctly resolve the pathspec. Add a test to catch future breakage of
this usecase.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a test for the case when only one parameter is passed to '-m'
(move/rename) option.
For example - if 'git branch -m bbb' is run while checked out on aaa
branch, it should rename the currently checked out branch to bbb.
There was no test for this particular case with only one parameter
for -m option. However, there's one similar test case for -M option.
Add test for making sure HEAD points to the bbb (new branch name). Also
add a test for making sure the reflog that is moved to 'bbb' retains
entries created for the currently checked out branch. Note that since
the topmost entry on reflog for bbb will be about branch creation, we
compare bbb@{1} (instead of bbb@{0}) with aaa@{0} to make sure the
reflog for bbb retains entries from aaa.
Signed-off-by: Sahil Dua <sahildua2305@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add some more simple mailinfo tests including a few that
produce:
fatal: `pos + len' is too far after the end of the buffer
Mark those as 'test_expect_failure'.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use modern style in the test t4005. Remove hard coded sha1 values.
Combine test prep work and the actual test. Rename the first
test to contain the word "setup".
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit a1283866b (t5313: test bounds-checks of
corrupted/malicious pack/idx files, 2016-02-25) added a test
that requires our corrupted pack index to have two objects.
The entry for the first one remains untouched, but we
corrupt the entry for second one. Since the index stores the
entries in sha1-sorted order, this means that the test must
make sure that the sha1 of the object we expect to be
corrupted ("$object") sorts after the other placeholder
object.
That commit used the HEAD commit as the placeholder, but the
script never calls test_tick. That means that the commit
object (and thus its sha1) depends on the timestamp when the
test script is run. This usually works in practice, because
the sha1 of $object starts with "fff". The commit object
will sort after that only 1 in 4096 times, but when it does
the test will fail.
One obvious solution is to add the test_tick call to get a
deterministic commit sha1. But since we're relying on the
sort order for the test to function, let's make that very
explicit by just generating a second blob with a known sha1.
Reported-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since commit 99caeed05 (Let 'git <command> -h' show usage
without a git dir, 2009-11-09), the git wrapper handles "-h"
specially, skipping any repository setup but still calling
the builtin's cmd_foo() function. This means that every
cmd_foo() must be ready to handle this case, but we don't
have any systematic tests. This led to "git am -h" being
broken for some time without anybody noticing.
This patch just tests that "git foo -h" works for every
builtin, where we see a 129 exit code (the normal code for
our usage() helper), and that the word "usage" appears in
the output.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When the tested repo has an index.lock file it should be removed. This
file may be present if e.g. git-status previously crashed in that
repo, and it will make a lot of git commands fail. Let's try harder
and remove the lock.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git pull --rebase --autostash" didn't auto-stash when the local history
fast-forwards to the upstream.
* tb/pull-ff-rebase-autostash:
pull: ff --rebase --autostash works in dirty repo
The implementation of "ref" API around the "packed refs" have been
cleaned up, in preparation for further changes.
* mh/packed-ref-store-prep: (25 commits)
cache_ref_iterator_begin(): avoid priming unneeded directories
ref-filter: limit traversal to prefix
create_ref_entry(): remove `check_name` option
refs_ref_iterator_begin(): handle `GIT_REF_PARANOIA`
read_packed_refs(): report unexpected fopen() failures
read_packed_refs(): do more of the work of reading packed refs
get_packed_ref_cache(): assume "packed-refs" won't change while locked
should_pack_ref(): new function, extracted from `files_pack_refs()`
ref_update_reject_duplicates(): add a sanity check
ref_update_reject_duplicates(): use `size_t` rather than `int`
ref_update_reject_duplicates(): expose function to whole refs module
ref_transaction_prepare(): new optional step for reference updates
ref_transaction_commit(): check for valid `transaction->state`
files_transaction_cleanup(): new helper function
files_ref_store: put the packed files lock directly in this struct
files-backend: move `lock` member to `files_ref_store`
lockfile: add a new method, is_lock_file_locked()
ref_store: take a `msg` parameter when deleting references
refs: use `size_t` indexes when iterating over ref transaction updates
refs_ref_iterator_begin(): don't check prefixes redundantly
...
Make the "indent" heuristics the default in "diff" and diff.indentHeuristics
configuration variable an escape hatch for those who do no want it.
* mb/diff-default-to-indent-heuristics:
add--interactive: drop diff.indentHeuristic handling
diff: enable indent heuristic by default
diff: have the diff-* builtins configure diff before initializing revisions
diff: make the indent heuristic part of diff's basic configuration
A test allowed both "git push" and "git receive-pack" on the other
end write their traces into the same file. This is OK on platforms
that allows atomically appending to a file opened with O_APPEND,
but on other platforms led to a mangled output, causing
intermittent test failures. This has been fixed by disabling
traces from "receive-pack" in the test.
* jk/alternate-ref-optim:
t5400: avoid concurrent writes into a trace file
"git interpret-trailers", when used as GIT_EDITOR for "git commit
-v", looked for and appended to a trailer block at the very end,
i.e. at the end of the "diff" output. The command has been
corrected to pay attention to the cut-mark line "commit -v" adds to
the buffer---the real trailer block should appear just before it.
* bm/interpret-trailers-cut-line-is-eom:
interpret-trailers: honor the cut line
"git clone --config var=val" is a way to populate the
per-repository configuration file of the new repository, but it did
not work well when val is an empty string. This has been fixed.
* jn/clone-add-empty-config-from-command-line:
clone: handle empty config values in -c
"git cherry-pick" and other uses of the sequencer machinery
mishandled a trailer block whose last line is an incomplete line.
This has been fixed so that an additional sign-off etc. are added
after completing the existing incomplete line.
* jt/use-trailer-api-in-commands:
sequencer: add newline before adding footers
The receive-pack program now makes sure that the push certificate
records the same set of push options used for pushing.
* jt/push-options-doc:
receive-pack: verify push options in cert
docs: correct receive.advertisePushOptions default
Make sure our tests would pass when the sources are checked out
with "platform native" line ending convention by default on
Windows. Some "text" files out tests use and the test scripts
themselves that are meant to be run with /bin/sh, ought to be
checked out with eol=LF even on Windows.
* js/eol-on-ourselves:
t4051: mark supporting files as requiring LF-only line endings
Fix the remaining tests that failed with core.autocrlf=true
t3901: move supporting files into t/t3901/
completion: mark bash script as LF-only
git-new-workdir: mark script as LF-only
Fix build with core.autocrlf=true
Tag objects, which are not reachable from any ref, that point at
missing objects were mishandled by "git gc" and friends (they
should silently be ignored instead)
* jk/ignore-broken-tags-when-ignoring-missing-links:
revision.c: ignore broken tags with ignore_missing_links
"pack-objects" can stream a slice of an existing packfile out when
the pack bitmap can tell that the reachable objects are all needed
in the output, without inspecting individual objects. This
strategy however would not work well when "--local" and other
options are in use, and need to be disabled.
* jk/disable-pack-reuse-when-broken:
t5310: fix "; do" style
pack-objects: disable pack reuse for object-selection options
Setting "log.decorate=false" in the configuration file did not take
effect in v2.13, which has been corrected.
* ah/log-decorate-default-to-auto:
builtin/log: honor log.decorate
Update tests to pass under GETTEXT_POISON (a mechanism to ensure
that output strings that should not be translated are not
translated by mistake), and tell TravisCI to run them.
* ab/fix-poison-tests:
travis-ci: add job to run tests with GETTEXT_POISON
travis-ci: setup "prove cache" in "script" step
tests: fix tests broken under GETTEXT_POISON=YesPlease
The Web interface to gmane news archive is long gone, even though
the articles are still accessible via NTTP. Replace the links with
ones to public-inbox.org. Because their message identification is
based on the actual message-id, it is likely that it will be easier
to migrate away from it if/when necessary.
* ab/doc-replace-gmane-links:
doc: replace more gmane links
doc: replace a couple of broken gmane links
The internal implementation of "git grep" has seen some clean-up.
* ab/grep-preparatory-cleanup: (31 commits)
grep: assert that threading is enabled when calling grep_{lock,unlock}
grep: given --threads with NO_PTHREADS=YesPlease, warn
pack-objects: fix buggy warning about threads
pack-objects & index-pack: add test for --threads warning
test-lib: add a PTHREADS prerequisite
grep: move is_fixed() earlier to avoid forward declaration
grep: change internal *pcre* variable & function names to be *pcre1*
grep: change the internal PCRE macro names to be PCRE1
grep: factor test for \0 in grep patterns into a function
grep: remove redundant regflags assignments
grep: catch a missing enum in switch statement
perf: add a comparison test of log --grep regex engines with -F
perf: add a comparison test of log --grep regex engines
perf: add a comparison test of grep regex engines with -F
perf: add a comparison test of grep regex engines
perf: emit progress output when unpacking & building
perf: add a GIT_PERF_MAKE_COMMAND for when *_MAKE_OPTS won't do
grep: add tests to fix blind spots with \0 patterns
grep: prepare for testing binary regexes containing rx metacharacters
grep: add a test helper function for less verbose -f \0 tests
...
The result from "git diff" that compares two blobs, e.g. "git diff
$commit1:$path $commit2:$path", used to be shown with the full
object name as given on the command line, but it is more natural to
use the $path in the output and use it to look up .gitattributes.
* jk/diff-blob:
diff: use blob path for blob/file diffs
diff: use pending "path" if it is available
diff: use the word "path" instead of "name" for blobs
diff: pass whole pending entry in blobinfo
handle_revision_arg: record paths for pending objects
handle_revision_arg: record modes for "a..b" endpoints
t4063: add tests of direct blob diffs
get_sha1_with_context: dynamically allocate oc->path
get_sha1_with_context: always initialize oc->symlink_path
sha1_name: consistently refer to object_context as "oc"
handle_revision_arg: add handle_dotdot() helper
handle_revision_arg: hoist ".." check out of range parsing
handle_revision_arg: stop using "dotdot" as a generic pointer
handle_revision_arg: simplify commit reference lookups
handle_revision_arg: reset "dotdot" consistently
"git clean -d" used to clean directories that has ignored files,
even though the command should not lose ignored ones without "-x".
"git status --ignored" did not list ignored and untracked files
without "-uall". These have been corrected.
* sl/clean-d-ignored-fix:
clean: teach clean -d to preserve ignored paths
dir: expose cmp_name() and check_contains()
dir: hide untracked contents of untracked dirs
dir: recurse into untracked dirs for ignored files
t7061: status --ignored should search untracked dirs
t7300: clean -d should skip dirs with ignored files
Unless --no-validate is passed, send-email will invoke
$repo->repo_path() in its search for a validate hook regardless of
whether a Git repo is actually present. Teach send-email to first check
for repo existence.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
On Windows, certain characters are prohibited in file names, most
prominently the colon. When fopen() is called with such an invalid file
name, the underlying Windows API actually reports a particular error,
but since there is no suitable errno value, this error is translated
to EINVAL. Detect the case and report ENOENT instead.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This added test case simply verifies that users will not be bothered
with bogus complaints à la
warning: unable to access '.git/remotes/D:\repo': Invalid argument
when fetching from a Windows path (in this case, D:\repo).
[j6t: mark the new test as test_expect_failure]
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add support for v2 of the PCRE API. This is a new major version of
PCRE that came out in early 2015[1].
The regular expression syntax is the same, but while the API is
similar, pretty much every function is either renamed or takes
different arguments. Thus using it via entirely new functions makes
sense, as opposed to trying to e.g. have one compile_pcre_pattern()
that would call either PCRE v1 or v2 functions.
Git can now be compiled with either USE_LIBPCRE1=YesPlease or
USE_LIBPCRE2=YesPlease, with USE_LIBPCRE=YesPlease currently being a
synonym for the former. Providing both is a compile-time error.
With earlier patches to enable JIT for PCRE v1 the performance of the
release versions of both libraries is almost exactly the same, with
PCRE v2 being around 1% slower.
However after I reported this to the pcre-dev mailing list[2] I got a
lot of help with the API use from Zoltán Herczeg, he subsequently
optimized some of the JIT functionality in v2 of the library.
Running the p7820-grep-engines.sh performance test against the latest
Subversion trunk of both, with both them and git compiled as -O3, and
the test run against linux.git, gives the following results. Just the
/perl/ tests shown:
$ GIT_PERF_REPEAT_COUNT=30 GIT_PERF_LARGE_REPO=~/g/linux GIT_PERF_MAKE_COMMAND='grep -q LIBPCRE2 Makefile && make -j8 USE_LIBPCRE2=YesPlease CC=~/perl5/installed/bin/gcc NO_R_TO_GCC_LINKER=YesPlease CFLAGS=-O3 LIBPCREDIR=/home/avar/g/pcre2/inst LDFLAGS=-Wl,-rpath,/home/avar/g/pcre2/inst/lib || make -j8 USE_LIBPCRE=YesPlease CC=~/perl5/installed/bin/gcc NO_R_TO_GCC_LINKER=YesPlease CFLAGS=-O3 LIBPCREDIR=/home/avar/g/pcre/inst LDFLAGS=-Wl,-rpath,/home/avar/g/pcre/inst/lib' ./run HEAD~5 HEAD~ HEAD p7820-grep-engines.sh
[...]
Test HEAD~5 HEAD~ HEAD
-----------------------------------------------------------------------------------------------------------------
7820.3: perl grep 'how.to' 0.31(1.10+0.48) 0.21(0.35+0.56) -32.3% 0.21(0.34+0.55) -32.3%
7820.7: perl grep '^how to' 0.56(2.70+0.40) 0.24(0.64+0.52) -57.1% 0.20(0.28+0.60) -64.3%
7820.11: perl grep '[how] to' 0.56(2.66+0.38) 0.29(0.95+0.45) -48.2% 0.23(0.45+0.54) -58.9%
7820.15: perl grep '(e.t[^ ]*|v.ry) rare' 1.02(5.77+0.42) 0.31(1.02+0.54) -69.6% 0.23(0.50+0.54) -77.5%
7820.19: perl grep 'm(ú|u)lt.b(æ|y)te' 0.38(1.57+0.42) 0.27(0.85+0.46) -28.9% 0.21(0.33+0.57) -44.7%
See commit ("perf: add a comparison test of grep regex engines",
2017-04-19) for details on the machine the above test run was executed
on.
Here HEAD~2 is git with PCRE v1 without JIT, HEAD~ is PCRE v1 with
JIT, and HEAD is PCRE v2 (also with JIT). See previous commits of mine
mentioning p7820-grep-engines.sh for more details on the test setup.
For ease of readability, a different run just of HEAD~ (PCRE v1 with
JIT v.s. PCRE v2), again with just the /perl/ tests shown:
[...]
Test HEAD~ HEAD
----------------------------------------------------------------------------------------
7820.3: perl grep 'how.to' 0.21(0.42+0.52) 0.21(0.31+0.58) +0.0%
7820.7: perl grep '^how to' 0.25(0.65+0.50) 0.20(0.31+0.57) -20.0%
7820.11: perl grep '[how] to' 0.30(0.90+0.50) 0.23(0.46+0.53) -23.3%
7820.15: perl grep '(e.t[^ ]*|v.ry) rare' 0.30(1.19+0.38) 0.23(0.51+0.51) -23.3%
7820.19: perl grep 'm(ú|u)lt.b(æ|y)te' 0.27(0.84+0.48) 0.21(0.34+0.57) -22.2%
I.e. the two are either neck-to-neck, but PCRE v2 usually pulls ahead,
when it does it's around 20% faster.
A brief note on thread safety: As noted in pcre2api(3) & pcre2jit(3)
the compiled pattern can be shared between threads, but not some of
the JIT context, however the grep threading support does all pattern &
JIT compilation in separate threads, so this code doesn't need to
concern itself with thread safety.
See commit 63e7e9d8b6 ("git-grep: Learn PCRE", 2011-05-09) for the
initial addition of PCRE v1. This change follows some of the same
patterns it did (and which were discussed on list at the time),
e.g. mocking up types with typedef instead of ifdef-ing them out when
USE_LIBPCRE2 isn't defined. This adds some trivial memory use to the
program, but makes the code look nicer.
1. https://lists.exim.org/lurker/message/20150105.162835.0666407a.en.html
2. https://lists.exim.org/lurker/thread/20170419.172322.833ee099.en.html
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When `git pull --rebase --autostash` in a dirty repository resulted in a
fast-forward, nothing was being autostashed and the pull failed. This
was due to a shortcut to avoid running rebase when we can fast-forward,
but autostash is ignored on that codepath.
Now we will only take the shortcut if autostash is not in effect.
Based on a few tests against the git.git repo, the shortcut does not
seem to give us significant performance benefits, on Linux at least.
Regardless, it is more important to be correct than to be fast.
Signed-off-by: Tyler Brazier <tyler@tylerbrazier.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The closest mapping from the boolean 'submodule.recurse' set to "yes"
to the variety of submodule push modes is "on-demand", so implement that.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In builtin/grep.c we parse the config before evaluating the command line
options. This makes the task of teaching grep to respect the new config
option 'submodule.recurse' very easy by just parsing that option.
As an alternative I had implemented a similar structure to treat
submodules as the fetch/push command have, including
* aligning the meaning of the 'recurse_submodules' to possible submodule
values RECURSE_SUBMODULES_* as defined in submodule.h.
* having a callback to parse the value and
* reacting to the RECURSE_SUBMODULES_DEFAULT state that was the initial
state.
However all this is not needed for a true boolean value, so let's keep
it simple. However this adds another place where "submodule.recurse" is
parsed.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Any command that understands '--recurse-submodules' can have its
default changed to true, by setting the new 'submodule.recurse'
option.
This patch includes read-tree/checkout/reset for working tree
manipulating commands. Later patches will cover other commands.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a later patch we want to introduce a config option to trigger the
submodule recursing by default. As this option should be available and
uniform across all commands that deal with submodules we'd want to test
for this option in the submodule update library.
So instead of calling the whole test set again for
"git -c submodule.recurse foo" instead of "git foo --recurse-submodules",
we'd only want to introduce one basic test that tests if the option is
recognized and respected to not overload the test suite.
Change the test functions by taking only the argument and assemble the
command inside the test function by embedding the arguments into the
command that is "git $arguments --recurse-submodules".
It would be nice to do this for all functions in lib-submodule-update,
but we cannot do that for the non-recursing tests, as there we do not
just pass in a git command but whole functions. (See t3426 for example)
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* ab/grep-preparatory-cleanup: (31 commits)
grep: assert that threading is enabled when calling grep_{lock,unlock}
grep: given --threads with NO_PTHREADS=YesPlease, warn
pack-objects: fix buggy warning about threads
pack-objects & index-pack: add test for --threads warning
test-lib: add a PTHREADS prerequisite
grep: move is_fixed() earlier to avoid forward declaration
grep: change internal *pcre* variable & function names to be *pcre1*
grep: change the internal PCRE macro names to be PCRE1
grep: factor test for \0 in grep patterns into a function
grep: remove redundant regflags assignments
grep: catch a missing enum in switch statement
perf: add a comparison test of log --grep regex engines with -F
perf: add a comparison test of log --grep regex engines
perf: add a comparison test of grep regex engines with -F
perf: add a comparison test of grep regex engines
perf: emit progress output when unpacking & building
perf: add a GIT_PERF_MAKE_COMMAND for when *_MAKE_OPTS won't do
grep: add tests to fix blind spots with \0 patterns
grep: prepare for testing binary regexes containing rx metacharacters
grep: add a test helper function for less verbose -f \0 tests
...
