Commit Graph

19007 Commits

Author SHA1 Message Date
Junio C Hamano
3c0e417827 Merge branch 'ds/fetch-pull-with-sparse-index'
"git fetch" and "git pull" are now declared sparse-index clean.
Also "git ls-files" learns the "--sparse" option to help debugging.

* ds/fetch-pull-with-sparse-index:
  test-read-cache: remove --table, --expand options
  t1091/t3705: remove 'test-tool read-cache --table'
  t1092: replace 'read-cache --table' with 'ls-files --sparse'
  ls-files: add --sparse option
  fetch/pull: use the sparse index
2022-01-10 11:52:50 -08:00
Junio C Hamano
b48c69c3c8 Merge branch 'hn/ref-api-tests-update'
Test updates.

* hn/ref-api-tests-update:
  t7004: use "test-tool ref-store" for reflog inspection
  t7004: create separate tags for different tests
  t5550: require REFFILES
  t5540: require REFFILES
2022-01-10 11:52:50 -08:00
Junio C Hamano
be69d35e48 Merge branch 'ja/perf-use-specified-shell'
Perf tests were run with end-user's shell, but it has been
corrected to use the shell specified by $TEST_SHELL_PATH.

* ja/perf-use-specified-shell:
  t/perf: do not run tests in user's $SHELL
2022-01-10 11:52:50 -08:00
Junio C Hamano
321f7f4fa4 Merge branch 'hn/test-ref-store-show-hash-algo'
Debugging support for refs API.

* hn/test-ref-store-show-hash-algo:
  test-ref-store: print hash algorithm
2022-01-10 11:52:50 -08:00
Junio C Hamano
98ab07ace5 Merge branch 'ws/fast-export-with-revision-options'
Use of certain "git rev-list" options with "git fast-export"
created nonsense results (the worst two of which being "--reverse"
and "--invert-grep --grep=<foo>").  The use of "--first-parent" is
made to behave a bit more sensible than before.

* ws/fast-export-with-revision-options:
  fast-export: fix surprising behavior with --first-parent
2022-01-10 11:52:50 -08:00
Junio C Hamano
09481fec21 Merge branch 'ds/sparse-checkout-malformed-pattern-fix'
Certain sparse-checkout patterns that are valid in non-cone mode
led to segfault in cone mode, which has been corrected.

* ds/sparse-checkout-malformed-pattern-fix:
  sparse-checkout: refuse to add to bad patterns
  sparse-checkout: fix OOM error with mixed patterns
  sparse-checkout: fix segfault on malformed patterns
2022-01-10 11:52:49 -08:00
Junio C Hamano
c91b0b7c72 Merge branch 'rs/pcre2-utf'
"git grep --perl-regexp" failed to match UTF-8 characters with
wildcard when the pattern consists only of ASCII letters, which has
been corrected.

* rs/pcre2-utf:
  grep/pcre2: factor out literal variable
  grep/pcre2: use PCRE2_UTF even with ASCII patterns
2022-01-05 14:01:31 -08:00
Junio C Hamano
5d522cddfd Merge branch 'jc/t4204-do-not-write-git-on-upstream-of-pipe'
Test fixes.

* jc/t4204-do-not-write-git-on-upstream-of-pipe:
  t4204 is not sanitizer clean at all
2022-01-05 14:01:31 -08:00
Junio C Hamano
2043ce828e Merge branch 'rs/log-invert-grep-with-headers'
"git log --invert-grep --author=<name>" used to exclude commits
written by the given author, but now "--invert-grep" only affects
the matches made by the "--grep=<pattern>" option.

* rs/log-invert-grep-with-headers:
  log: let --invert-grep only invert --grep
2022-01-05 14:01:30 -08:00
Junio C Hamano
8292c148df Merge branch 'rs/t4202-invert-grep-test-fix'
Test fix.

* rs/t4202-invert-grep-test-fix:
  t4202: fix patternType setting in --invert-grep test
2022-01-05 14:01:30 -08:00
Junio C Hamano
88a516aca0 Merge branch 'ds/repack-fixlets'
Two fixes around "git repack".

* ds/repack-fixlets:
  repack: make '--quiet' disable progress
  repack: respect kept objects with '--write-midx -b'
2022-01-05 14:01:30 -08:00
Junio C Hamano
bb14cfdfd7 Merge branch 'jc/merge-detached-head-name'
The default merge message prepared by "git merge" records the name
of the current branch; the name can be overridden with a new option
to allow users to pretend a merge is made on a different branch.

* jc/merge-detached-head-name:
  merge: allow to pretend a merge is made into a different branch
2022-01-05 14:01:30 -08:00
Junio C Hamano
2b755b3371 Merge branch 'pw/diff-color-moved-fix'
Correctness and performance update to "diff --color-moved" feature.

* pw/diff-color-moved-fix:
  diff --color-moved: intern strings
  diff: use designated initializers for emitted_diff_symbol
  diff --color-moved-ws=allow-indentation-change: improve hash lookups
  diff --color-moved: stop clearing potential moved blocks
  diff --color-moved: shrink potential moved blocks as we go
  diff --color-moved: unify moved block growth functions
  diff --color-moved: call comparison function directly
  diff --color-moved-ws=allow-indentation-change: simplify and optimize
  diff: simplify allow-indentation-change delta calculation
  diff --color-moved: avoid false short line matches and bad zebra coloring
  diff --color-moved=zebra: fix alternate coloring
  diff --color-moved: rewind when discarding pmb
  diff --color-moved: factor out function
  diff --color-moved: clear all flags on blocks that are too short
  diff --color-moved: add perf tests
2022-01-05 14:01:29 -08:00
Junio C Hamano
ead6767ad7 Merge branch 'xw/am-empty'
"git am" learns "--empty=(stop|drop|keep)" option to tweak what is
done to a piece of e-mail without a patch in it.

* xw/am-empty:
  am: support --allow-empty to record specific empty patches
  am: support --empty=<option> to handle empty patches
  doc: git-format-patch: describe the option --always
2022-01-05 14:01:28 -08:00
Junio C Hamano
da81d473fc Merge branch 'en/keep-cwd'
Many git commands that deal with working tree files try to remove a
directory that becomes empty (i.e. "git switch" from a branch that
has the directory to another branch that does not would attempt
remove all files in the directory and the directory itself).  This
drops users into an unfamiliar situation if the command was run in
a subdirectory that becomes subject to removal due to the command.
The commands have been taught to keep an empty directory if it is
the directory they were started in to avoid surprising users.

* en/keep-cwd:
  t2501: simplify the tests since we can now assume desired behavior
  dir: new flag to remove_dir_recurse() to spare the original_cwd
  dir: avoid incidentally removing the original_cwd in remove_path()
  stash: do not attempt to remove startup_info->original_cwd
  rebase: do not attempt to remove startup_info->original_cwd
  clean: do not attempt to remove startup_info->original_cwd
  symlinks: do not include startup_info->original_cwd in dir removal
  unpack-trees: add special cwd handling
  unpack-trees: refuse to remove startup_info->original_cwd
  setup: introduce startup_info->original_cwd
  t2501: add various tests for removing the current working directory
2022-01-05 14:01:28 -08:00
Junio C Hamano
b58e7bfcd7 Merge branch 'jh/p4-rcs-expansion-in-bytestring'
The RCS keyword substitution in "git p4" used to be done assuming
that the contents are UTF-8 text, which can trigger decoding
errors.  We now treat the contents as a bytestring for robustness
and correctness.

* jh/p4-rcs-expansion-in-bytestring:
  git-p4: resolve RCS keywords in bytes not utf-8
  git-p4: open temporary patch file for write only
  git-p4: add raw option to read_pipelines
  git-p4: pre-compile RCS keyword regexes
  git-p4: use with statements to close files after use in patchRCSKeywords
2022-01-05 14:01:27 -08:00
Junio C Hamano
2dc94da374 Merge branch 'en/sparse-checkout-set'
The "init" and "set" subcommands in "git sparse-checkout" have been
unified for a better user experience and performance.

* en/sparse-checkout-set:
  sparse-checkout: remove stray trailing space
  clone: avoid using deprecated `sparse-checkout init`
  Documentation: clarify/correct a few sparsity related statements
  git-sparse-checkout.txt: update to document init/set/reapply changes
  sparse-checkout: enable reapply to take --[no-]{cone,sparse-index}
  sparse-checkout: enable `set` to initialize sparse-checkout mode
  sparse-checkout: split out code for tweaking settings config
  sparse-checkout: disallow --no-stdin as an argument to set
  sparse-checkout: add sanity-checks on initial sparsity state
  sparse-checkout: break apart functions for sparse_checkout_(set|add)
  sparse-checkout: pass use_stdin as a parameter instead of as a global
2022-01-03 16:24:15 -08:00
Junio C Hamano
4f4b18497a Merge branch 'es/test-chain-lint'
Broken &&-chains in the test scripts have been corrected.

* es/test-chain-lint:
  t6000-t9999: detect and signal failure within loop
  t5000-t5999: detect and signal failure within loop
  t4000-t4999: detect and signal failure within loop
  t0000-t3999: detect and signal failure within loop
  tests: simplify by dropping unnecessary `for` loops
  tests: apply modern idiom for exiting loop upon failure
  tests: apply modern idiom for signaling test failure
  tests: fix broken &&-chains in `{...}` groups
  tests: fix broken &&-chains in `$(...)` command substitutions
  tests: fix broken &&-chains in compound statements
  tests: use test_write_lines() to generate line-oriented output
  tests: simplify construction of large blocks of text
  t9107: use shell parameter expansion to avoid breaking &&-chain
  t6300: make `%(raw:size) --shell` test more robust
  t5516: drop unnecessary subshell and command invocation
  t4202: clarify intent by creating expected content less cleverly
  t1020: avoid aborting entire test script when one test fails
  t1010: fix unnoticed failure on Windows
  t/lib-pager: use sane_unset() to avoid breaking &&-chain
2022-01-03 16:24:15 -08:00
Derrick Stolee
a3eca58445 sparse-checkout: refuse to add to bad patterns
When in cone mode sparse-checkout, it is unclear how 'git
sparse-checkout add <dir1> ...' should behave if the existing
sparse-checkout file does not match the cone mode patterns. Change the
behavior to fail with an error message about the existing patterns.

Also, all cone mode patterns start with a '/' character, so add that
restriction. This is necessary for our example test 'cone mode: warn on
bad pattern', but also requires modifying the example sparse-checkout
file we use to test the warnings related to recognizing cone mode
patterns.

This error checking would cause a failure further down the test script
because of a test that adds non-cone mode patterns without cleaning them
up. Perform that cleanup as part of the test now.

Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-30 14:39:57 -08:00
Derrick Stolee
391c3a1020 sparse-checkout: fix OOM error with mixed patterns
Add a test to t1091-sparse-checkout-builtin.sh that would result in an
infinite loop and out-of-memory error before this change. The issue
relies on having non-cone-mode patterns while trying to modify the
patterns in cone-mode.

The fix is simple, allowing us to break from the loop when the input
path does not contain a slash, as the "dir" pattern we added does not.

This is only a fix to the critical out-of-memory error. A better
response to such a strange state will follow in a later change.

Reported-by: Calbabreaker <calbabreaker@gmail.com>
Helped-by: Taylor Blau <me@ttaylorr.com>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-30 14:39:57 -08:00
Derrick Stolee
a481d4378c sparse-checkout: fix segfault on malformed patterns
Then core.sparseCheckoutCone is enabled, the sparse-checkout patterns are
used to populate two hashsets that accelerate pattern matching. If the user
modifies the sparse-checkout file outside of the 'sparse-checkout' builtin,
then strange patterns can happen, triggering some error checks.

One of these error checks is possible to hit when some special characters
exist in a line. A warning message is correctly written to stderr, but then
there is additional logic that attempts to remove the line from the hashset
and free the data. This leads to a segfault in the 'git sparse-checkout
list' command because it iterates over the contents of the hashset, which is
now invalid.

The fix here is to stop trying to remove from the hashset. In addition,
we disable cone mode sparse-checkout because of the malformed data. This
results in the pattern-matching working with a possibly-slower
algorithm, but using the patterns as they are in the sparse-checkout
file.

This also changes the behavior of commands such as 'git sparse-checkout
list' because the output patterns will be the contents of the
sparse-checkout file instead of the list of directories. This is an
existing behavior for other types of bad patterns.

Add a test that triggers the segfault without the code change.

Reported-by: John Burnett <johnburnett@johnburnett.com>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-30 14:39:57 -08:00
Johannes Altmanninger
9ccab75608 t/perf: do not run tests in user's $SHELL
The environment variable $SHELL is usually set to the user's
interactive shell. Our build and test scripts never use $SHELL because
there are no guarantees about its input language.  Instead, we use
/bin/sh which should be a POSIX shell.

For systems with a broken /bin/sh, we allow to override that path via
SHELL_PATH.  To run tests in yet another shell we allow to override
SHELL_PATH with TEST_SHELL_PATH.

Perf tests run in $SHELL via a wrapper defined in t/perf/perf-lib.sh,
so they break with e.g. SHELL=python.  Use TEST_SHELL_PATH like
in other tests.  TEST_SHELL_PATH is always defined because
t/perf/perf-lib.sh includes t/test-lib.sh, which includes
GIT-BUILD-OPTIONS.

Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Johannes Altmanninger <aclopte@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-25 14:24:58 -08:00
Junio C Hamano
d52da62801 Merge branch 'es/chainlint'
The chainlint test script linter in the test suite has been updated.

* es/chainlint:
  chainlint.sed: stop splitting "(..." into separate lines "(" and "..."
  chainlint.sed: swallow comments consistently
  chainlint.sed: stop throwing away here-doc tags
  chainlint.sed: don't mistake `<< word` in string as here-doc operator
  chainlint.sed: make here-doc "<<-" operator recognition more POSIX-like
  chainlint.sed: drop subshell-closing ">" annotation
  chainlint.sed: drop unnecessary distinction between ?!AMP?! and ?!SEMI?!
  chainlint.sed: tolerate harmless ";" at end of last line in block
  chainlint.sed: improve ?!SEMI?! placement accuracy
  chainlint.sed: improve ?!AMP?! placement accuracy
  t/Makefile: optimize chainlint self-test
  t/chainlint/one-liner: avoid overly intimate chainlint.sed knowledge
  t/chainlint/*.test: generalize self-test commentary
  t/chainlint/*.test: fix invalid test cases due to mixing quote types
  t/chainlint/*.test: don't use invalid shell syntax
2021-12-22 22:48:11 -08:00
Junio C Hamano
62a3a27b91 Merge branch 'jz/apply-quiet-and-allow-empty'
"git apply" has been taught to ignore a message without a patch
with the "--allow-empty" option.  It also learned to honor the
"--quiet" option given from the command line.

* jz/apply-quiet-and-allow-empty:
  git-apply: add --allow-empty flag
  git-apply: add --quiet flag
2021-12-22 22:48:11 -08:00
Junio C Hamano
67b7017593 Merge branch 'ab/common-main-cleanup'
Code clean-up.

* ab/common-main-cleanup:
  common-main.c: call exit(), don't return
2021-12-22 22:48:11 -08:00
Junio C Hamano
dcaf17c75d Merge branch 'ab/fetch-set-upstream-while-detached'
"git fetch --set-upstream" did not check if there is a current
branch, leading to a segfault when it is run on a detached HEAD,
which has been corrected.

* ab/fetch-set-upstream-while-detached:
  pull, fetch: fix segfault in --set-upstream option
2021-12-22 22:48:10 -08:00
Han-Wen Nienhuys
ed8d1d42d0 t7004: use "test-tool ref-store" for reflog inspection
This makes the test work with reftable.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-22 12:13:02 -08:00
Han-Wen Nienhuys
be49349c72 t7004: create separate tags for different tests
Reftable intentionally keeps reflog data for deleted refs.

This breaks tests that delete and recreate "refs/tags/tag_with_reflog" as traces
of the deletion are left in reflog. To resolve this, use a differently named ref
for each test case.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-22 12:13:01 -08:00
Han-Wen Nienhuys
7b6853a826 t5550: require REFFILES
The dumb HTTP protocol exposes ref storage details as part of the protocol,
so it only works with the FILES refstorage backend

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-22 12:00:06 -08:00
Han-Wen Nienhuys
8c21d820e9 t5540: require REFFILES
The dumb HTTP protocol exposes ref storage details as part of the protocol,
so it only works with the FILES refstorage backend

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-22 12:00:04 -08:00
Derrick Stolee
408c51f0b4 test-read-cache: remove --table, --expand options
This commit effectively reverts 2782db3 (test-tool: don't force full
index, 2021-03-30) and e2df6c3 (test-read-cache: print cache entries
with --table, 2021-03-30) to remove the --table and --expand options
from 'test-tool read-cache'. The previous changes already removed these
options from the test suite in favor of 'git ls-files --sparse'.

The initial thought of creating these options was to allow for tests to
see additional information with every cache entry. In particular, the
object type is still not mirrored in 'git ls-files'. Since sparse
directory entries always end with a slash, the object type is not
critical to verify the sparse index is enabled. It was thought that it
would be helpful to have additional information, such as flags, but that
was not needed for the FS Monitor integration and hasn't been needed
since.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-22 11:42:40 -08:00
Derrick Stolee
c2a2940510 t1091/t3705: remove 'test-tool read-cache --table'
Now that 'git ls-files --sparse' exists, we can use it to verify the
state of a sparse index instead of 'test-tool read-cache table'. Replace
these usages within t1091-sparse-checkout-builtin.sh and
t3705-add-sparse-checkout.sh.

The important changes are due to the different output format. In t3705,
we need to use the '--stage' output to get a file mode and OID, but
it also includes a stage value and drops the object type. This leads
to some differences in how we handle looking for specific entries.

In t1091, the test focuses on enabling the sparse index, so we do not
need the --stage flag to demonstrate how the index changes, and instead
can use a diff.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-22 11:42:40 -08:00
Derrick Stolee
3a9a6ac5d9 t1092: replace 'read-cache --table' with 'ls-files --sparse'
Now that 'git ls-files --sparse' exists, we can use it to verify the
state of a sparse index instead of 'test-tool read-cache table'. Replace
these usages within t1092-sparse-checkout-compatibility.sh.

The important changes are due to the different output format. We need to
use the '--stage' output to get a file mode and OID, but it also
includes a stage value and drops the object type. This leads to some
differences in how we handle looking for specific entries.

Some places where we previously looked for no 'tree' entries, we can
instead directly compare the output across the repository with a sparse
index and the one without.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-22 11:42:40 -08:00
Derrick Stolee
78087097b8 ls-files: add --sparse option
Existing callers to 'git ls-files' are expecting file names, not
directories. It is best to expand a sparse index to show all of the
contained files in this case.

However, expert users may want to inspect the contents of the index
itself including which directories are sparse. Add a --sparse option to
allow users to request this information.

During testing, I noticed that options such as --modified did not affect
the output when the files in question were outside the sparse-checkout
definition. Tests are added to document this preexisting behavior and
how it remains unchanged with the sparse index and the --sparse option.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-22 11:42:40 -08:00
Derrick Stolee
5a4e0547e2 fetch/pull: use the sparse index
The 'git fetch' and 'git pull' commands parse the index in order to
determine if submodules exist. Without command_requires_full_index=0,
this will expand a sparse index, causing slow performance even when
there is no new data to fetch.

The .gitmodules file will never be inside a sparse directory entry, and
even if it was, the index_name_pos() method would expand the sparse
index if needed as we search for the path by name. These commands do not
iterate over the index, which is the typical thing we are careful about
when integrating with the sparse index.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-22 11:42:39 -08:00
Han-Wen Nienhuys
57eb368144 test-ref-store: print hash algorithm
This provides a better error message in case SHA256 was inadvertently switched
on through the environment.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-21 22:00:01 -08:00
Junio C Hamano
8d2c37320b Merge branch 'ld/sparse-diff-blame'
Teach diff and blame to work well with sparse index.

* ld/sparse-diff-blame:
  blame: enable and test the sparse index
  diff: enable and test the sparse index
  diff: replace --staged with --cached in t1092 tests
  repo-settings: prepare_repo_settings only in git repos
  test-read-cache: set up repo after git directory
  commit-graph: return if there is no git directory
  git: ensure correct git directory setup with -h
2021-12-21 15:03:17 -08:00
Junio C Hamano
13fa77b689 Merge branch 'ak/protect-any-current-branch'
"git fetch" without the "--update-head-ok" option ought to protect
a checked out branch from getting updated, to prevent the working
tree that checks it out to go out of sync.  The code was written
before the use of "git worktree" got widespread, and only checked
the branch that was checked out in the current worktree, which has
been updated.
(originally called ak/fetch-not-overwrite-any-current-branch)

* ak/protect-any-current-branch:
  branch: protect branches checked out in all worktrees
  receive-pack: protect current branch for bare repository worktree
  receive-pack: clean dead code from update_worktree()
  fetch: protect branches checked out in all worktrees
  worktree: simplify find_shared_symref() memory ownership model
  branch: lowercase error messages
  receive-pack: lowercase error messages
  fetch: lowercase error messages
2021-12-21 15:03:16 -08:00
Junio C Hamano
ee1dc493d1 Merge branch 'fs/ssh-signing-other-keytypes'
The cryptographic signing using ssh keys can specify literal keys
for keytypes whose name do not begin with the "ssh-" prefix by
using the "key::" prefix mechanism (e.g. "key::ecdsa-sha2-nistp256").

* fs/ssh-signing-other-keytypes:
  ssh signing: make sign/amend test more resilient
  ssh signing: support non ssh-* keytypes
2021-12-21 15:03:16 -08:00
Junio C Hamano
d2f0b72759 Merge branch 'fs/ssh-signing-key-lifetime'
Extend the signing of objects with SSH keys and learn to pay
attention to the key validity time range when verifying.

* fs/ssh-signing-key-lifetime:
  ssh signing: verify ssh-keygen in test prereq
  ssh signing: make fmt-merge-msg consider key lifetime
  ssh signing: make verify-tag consider key lifetime
  ssh signing: make git log verify key lifetime
  ssh signing: make verify-commit consider key lifetime
  ssh signing: add key lifetime test prereqs
  ssh signing: use sigc struct to pass payload
  t/fmt-merge-msg: make gpgssh tests more specific
  t/fmt-merge-msg: do not redirect stderr
2021-12-21 15:03:15 -08:00
Junio C Hamano
00cbaf9362 Merge branch 'jk/log-decorate-opts-with-implicit-decorate'
When "git log" implicitly enabled the "decoration" processing
without being explicitly asked with "--decorate" option, it failed
to read and honor the settings given by the "--decorate-refs"
option.

* jk/log-decorate-opts-with-implicit-decorate:
  log: load decorations with --simplify-by-decoration
  log: handle --decorate-refs with userformat "%d"
2021-12-21 15:03:15 -08:00
Junio C Hamano
57f28f4094 Merge branch 'en/rebase-x-wo-git-dir-env'
"git rebase -x" by mistake started exporting the GIT_DIR and
GIT_WORK_TREE environment variables when the command was rewritten
in C, which has been corrected.

* en/rebase-x-wo-git-dir-env:
  sequencer: do not export GIT_DIR and GIT_WORK_TREE for 'exec'
2021-12-21 15:03:15 -08:00
William Sprent
726a228dfb fast-export: fix surprising behavior with --first-parent
The revision traversal machinery typically processes and returns all
children before any parent.  fast-export needs to operate in the
reverse fashion, handling parents before any of their children in
order to build up the history starting from the root commit(s).  This
would be a clear case where we could just use the revision traversal
machinery's "reverse" option to achieve this desired affect.

However, this wasn't what the code did.  It added its own array for
queuing.  The obvious hand-rolled solution would be to just push all
the commits into the array and then traverse afterwards, but it didn't
quite do that either.  It instead attempted to process anything it
could as soon as it could, and once it could, check whether it could
process anything that had been queued.  As far as I can tell, this was
an effort to save a little memory in the case of multiple root commits
since it could process some commits before queueing all of them.  This
involved some helper functions named has_unshown_parent() and
handle_tail().  For typical invocations of fast-export, this
alternative essentially amounted to a hand-rolled method of reversing
the commits -- it was a bunch of work to duplicate the revision
traversal machinery's "reverse" option.

This hand-rolled reversing mechanism is actually somewhat difficult to
reason about.  It takes some time to figure out how it ensures in
normal cases that it will actually process all traversed commits
(rather than just dropping some and not printing anything for them).

And it turns out there are some cases where the code does drop commits
without handling them, and not even printing an error or warning for
the user.  Due to the has_unshown_parent() checks, some commits could
be left in the array at the end of the "while...get_revision()" loop
which would be unprocessed.  This could be triggered for example with
    git fast-export main -- --first-parent
or non-sensical traversal rules such as
    git fast-export main -- --grep=Merge --invert-grep

While most traversals that don't include all parents should likely
trigger errors in fast-export (or at least require being used in
combination with --reference-excluded-parents), the --first-parent
traversal is at least reasonable and it'd be nice if it didn't just drop
commits. It'd also be nice for future readers of the code to have a
simpler "reverse traversal" mechanism. Use the "reverse" option of the
revision traversal machinery to achieve both.

Even for the non-sensical traversal flags like the --grep one above,
this would be an improvement. For example, in that case, the code
previously would have silently truncated history to only those commits
that do not have an ancestor containing "Merge" in their commit message.
After this code change, that case would include all commits without
"Merge" in their commit message -- but any commit that previously had a
"Merge"-mentioning parent would lose that parent
(likely resulting in many new root commits). While the new behavior is
still odd, it is at least understandable given that
--reference-excluded-parents is not the default.

Helped-by: Elijah Newren <newren@gmail.com>
Signed-off-by: William Sprent <williams@unity3d.com>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-21 12:51:09 -08:00
Junio C Hamano
bd2bc94252 merge: allow to pretend a merge is made into a different branch
When a series of patches for a topic-B depends on having topic-A,
the workflow to prepare the topic-B branch would look like this:

    $ git checkout -b topic-B main
    $ git merge --no-ff --no-edit topic-A
    $ git am <mbox-for-topic-B

When topic-A gets updated, recreating the first merge and rebasing
the rest of the topic-B, all on detached HEAD, is a useful
technique.  After updating topic-A with its new round of patches:

    $ git checkout topic-B
    $ prev=$(git rev-parse 'HEAD^{/^Merge branch .topic-A. into}')
    $ git checkout --detach $prev^1
    $ git merge --no-ff --no-edit topic-A
    $ git rebase --onto HEAD $prev @{-1}^0
    $ git checkout -B @{-1}

This will

 (0) check out the current topic-B.
 (1) find the previous merge of topic-A into topic-B.
 (2) detach the HEAD to the parent of the previous merge.
 (3) merge the updated topic-A to it.
 (4) reapply the patches to rebuild the rest of topic-B.
 (5) update topic-B with the result.

without contaminating the reflog of topic-B too much.  topic-B@{1}
is the "logically previous" state before topic-A got updated, for
example.  At (4), comparison (e.g. range-diff) between HEAD and
@{-1} is a meaningful way to sanity check the result, and the same
can be done at (5) by comparing topic-B and topic-B@{1}.

But there is one glitch.  The merge into the detached HEAD done in
the step (3) above gives us "Merge branch 'topic-A' into HEAD", and
does not say "into topic-B".

Teach the "--into-name=<branch>" option to "git merge" and its
underlying "git fmt-merge-message", to pretend as if we were merging
into <branch>, no matter what branch we are actually merging into,
when they prepare the merge message.  The pretend name honors the
usual "into <target>" suppression mechanism, which can be seen in
the tests added here.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-20 14:55:02 -08:00
René Scharfe
dc2c44fbb1 grep/pcre2: use PCRE2_UTF even with ASCII patterns
compile_pcre2_pattern() currently uses the option PCRE2_UTF only for
patterns with non-ASCII characters.  Patterns with ASCII wildcards can
match non-ASCII strings, though.  Without that option PCRE2 mishandles
UTF-8 input, though -- it matches parts of multi-byte characters.  Fix
that by using PCRE2_UTF even for ASCII-only patterns.

This is a remake of the reverted ae39ba431a (grep/pcre2: fix an edge
case concerning ascii patterns and UTF-8 data, 2021-10-15).  The change
to the condition and the test are simplified and more targeted.

Original-patch-by: Hamza Mahfooz <someguy@effective-light.com>
Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-20 12:45:02 -08:00
Derrick Stolee
47ca93d071 repack: make '--quiet' disable progress
While testing some ideas in 'git repack', I ran it with '--quiet' and
discovered that some progress output was still shown. Specifically, the
output for writing the multi-pack-index showed the progress.

The 'show_progress' variable in cmd_repack() is initialized with
isatty(2) and is not modified at all by the '--quiet' flag. The
'--quiet' flag modifies the po_args.quiet option which is translated
into a '--quiet' flag for the 'git pack-objects' child process. However,
'show_progress' is used to directly send progress information to the
multi-pack-index writing logic which does not use a child process.

The fix here is to modify 'show_progress' to be false if po_opts.quiet
is true, and isatty(2) otherwise. This new expectation simplifies a
later condition that checks both.

Update the documentation to make it clear that '-q' will disable all
progress in addition to ensuring the 'git pack-objects' child process
will receive the flag.

Use 'test_terminal' to check that this works to get around the isatty(2)
check.

Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-20 11:59:17 -08:00
Derrick Stolee
e4d0c11c04 repack: respect kept objects with '--write-midx -b'
Historically, we needed a single packfile in order to have reachability
bitmaps. This introduced logic that when 'git repack' had a '-b' option
that we should stop sending the '--honor-pack-keep' option to the 'git
pack-objects' child process, ensuring that we create a packfile
containing all reachable objects.

In the world of multi-pack-index bitmaps, we no longer need to repack
all objects into a single pack to have valid bitmaps. Thus, we should
continue sending the '--honor-pack-keep' flag to 'git pack-objects'.

The fix is very simple: only disable the flag when writing bitmaps but
also _not_ writing the multi-pack-index.

This opens the door to new repacking strategies that might want to keep
some historical set of objects in a stable pack-file while only
repacking more recent objects.

To test, create a new 'test_subcommand_inexact' helper that is more
flexible than 'test_subcommand'. This allows us to look for the
--honor-pack-keep flag without over-indexing on the exact set of
arguments.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-20 11:58:31 -08:00
René Scharfe
e6a9bc0c60 t4202: fix patternType setting in --invert-grep test
Actually use extended regexes as indicated in the comment.

Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-17 16:59:53 -08:00
René Scharfe
794c000267 log: let --invert-grep only invert --grep
The option --invert-grep is documented to filter out commits whose
messages match the --grep filters.  However, it also affects the
header matches (--author, --committer), which is not intended.

Move the handling of that option to grep.c, as only the code there can
distinguish between matches in the header from those in the message
body.  If --invert-grep is given then enable extended expressions (not
the regex type, we just need git grep's --not to work), negate the body
patterns and check if any of them match by piggy-backing on the
collect_hits mechanism of grep_source_1().

Collecting the matches in struct grep_opt is a bit iffy, but with
"last_shown" we have a precedent for writing state information to that
struct.

Reported-by: Dotan Cohen <dotancohen@gmail.com>
Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-17 14:13:08 -08:00
Junio C Hamano
6b2edaa378 t4204 is not sanitizer clean at all
Earlier we marked that this patch-id test is leak-sanitizer clean,
but if we read the test script carefully, it is merely because we
have too many invocations of commands in the "git log" family on the
upstream side of the pipe, hiding breakages from them.

Split the pipeline so that breakages from these commands can be
caught (not limited to aborts due to leak-sanitizer) and unmark
the script as not passing the test with leak-sanitizer in effect.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-16 15:18:31 -08:00