Commit Graph

73280 Commits

Author SHA1 Message Date
Elijah Newren
8259e4154f replay: change rev walking options
Let's force the rev walking options we need after calling
setup_revisions() instead of before.

This might override some user supplied rev walking command line options
though. So let's detect that and warn users by:

  a) setting the desired values, before setup_revisions(),
  b) checking after setup_revisions() whether these values differ from
     the desired values,
  c) if so throwing a warning and setting the desired values again.

We want the command to work from older commits to newer ones by default.
Also we don't want history simplification, as we want to deal with all
the commits in the affected range.

Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Co-authored-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-26 10:10:48 +09:00
Elijah Newren
e787e664da replay: introduce pick_regular_commit()
Let's refactor the code to handle a regular commit (a commit that is
neither a root commit nor a merge commit) into a single function instead
of keeping it inside cmd_replay().

This is good for separation of concerns, and this will help further work
in the future to replay merge commits.

Co-authored-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-26 10:10:48 +09:00
Elijah Newren
a9df61ace3 replay: die() instead of failing assert()
It's not a good idea for regular Git commands to use an assert() to
check for things that could happen but are not supported.

Let's die() with an explanation of the issue instead.

Co-authored-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-26 10:10:48 +09:00
Elijah Newren
d46da6d90b replay: start using parse_options API
Instead of manually parsing arguments, let's start using the parse_options
API. This way this new builtin will look more standard, and in some
upcoming commits will more easily be able to handle more command line
options.

Note that we plan to later use standard revision ranges instead of
hardcoded "<oldbase> <branch>" arguments. When we will use standard
revision ranges, it will be easier to check if there are no spurious
arguments if we keep ARGV[0], so let's call parse_options() with
PARSE_OPT_KEEP_ARGV0 even if we don't need ARGV[0] right now to avoid
some useless code churn.

Co-authored-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-26 10:10:48 +09:00
Elijah Newren
f920b0289b replay: introduce new builtin
For now, this is just a rename from `t/helper/test-fast-rebase.c` into
`builtin/replay.c` with minimal changes to make it build appropriately.

Let's add a stub documentation and a stub test script though.

Subsequent commits will flesh out the capabilities of the new command
and make it a more standard regular builtin.

Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Co-authored-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-26 10:10:48 +09:00
Elijah Newren
b9d0991cc7 t6429: remove switching aspects of fast-rebase
At the time t6429 was written, merge-ort was still under development,
did not have quite as many tests, and certainly was not widely deployed.
Since t6429 was exercising some codepaths just a little differently, we
thought having them also test the "merge_switch_to_result()" bits of
merge-ort was useful even though they weren't intrinsic to the real
point of these tests.

However, the value provided by doing extra testing of the
"merge_switch_to_result()" bits has decreased a bit over time, and it's
actively making it harder to refactor `test-tool fast-rebase` into `git
replay`, which we are going to do in following commits.  Dispense with
these bits.

Co-authored-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-26 10:10:47 +09:00
Patrick Steinhardt
b1df3b3867 commit-graph: disable GIT_COMMIT_GRAPH_PARANOIA by default
In 7a5d604443 (commit: detect commits that exist in commit-graph but not
in the ODB, 2023-10-31), we have introduced a new object existence check
into `repo_parse_commit_internal()` so that we do not parse commits via
the commit-graph that don't have a corresponding object in the object
database. This new check of course comes with a performance penalty,
which the commit put at around 30% for `git rev-list --topo-order`. But
there are in fact scenarios where the performance regression is even
higher. The following benchmark against linux.git with a fully-build
commit-graph:

  Benchmark 1: git.v2.42.1 rev-list --count HEAD
    Time (mean ± σ):     658.0 ms ±   5.2 ms    [User: 613.5 ms, System: 44.4 ms]
    Range (min … max):   650.2 ms … 666.0 ms    10 runs

  Benchmark 2: git.v2.43.0-rc1 rev-list --count HEAD
    Time (mean ± σ):      1.333 s ±  0.019 s    [User: 1.263 s, System: 0.069 s]
    Range (min … max):    1.302 s …  1.361 s    10 runs

  Summary
    git.v2.42.1 rev-list --count HEAD ran
      2.03 ± 0.03 times faster than git.v2.43.0-rc1 rev-list --count HEAD

While it's a noble goal to ensure that results are the same regardless
of whether or not we have a potentially stale commit-graph, taking twice
as much time is a tough sell. Furthermore, we can generally assume that
the commit-graph will be updated by git-gc(1) or git-maintenance(1) as
required so that the case where the commit-graph is stale should not at
all be common.

With that in mind, default-disable GIT_COMMIT_GRAPH_PARANOIA and restore
the behaviour and thus performance previous to the mentioned commit. In
order to not be inconsistent, also disable this behaviour by default in
`lookup_commit_in_graph()`, where the object existence check has been
introduced right at its inception via f559d6d45e (revision: avoid
hitting packfiles when commits are in commit-graph, 2021-08-09).

This results in another speedup in commands that end up calling this
function, even though it's less pronounced compared to the above
benchmark. The following has been executed in linux.git with ~1.2
million references:

  Benchmark 1: GIT_COMMIT_GRAPH_PARANOIA=true git rev-list --all --no-walk=unsorted
    Time (mean ± σ):      2.947 s ±  0.003 s    [User: 2.412 s, System: 0.534 s]
    Range (min … max):    2.943 s …  2.949 s    3 runs

  Benchmark 2: GIT_COMMIT_GRAPH_PARANOIA=false git rev-list --all --no-walk=unsorted
    Time (mean ± σ):      2.724 s ±  0.030 s    [User: 2.207 s, System: 0.514 s]
    Range (min … max):    2.704 s …  2.759 s    3 runs

  Summary
    GIT_COMMIT_GRAPH_PARANOIA=false git rev-list --all --no-walk=unsorted ran
      1.08 ± 0.01 times faster than GIT_COMMIT_GRAPH_PARANOIA=true git rev-list --all --no-walk=unsorted

So whereas 7a5d604443 initially introduced the logic to start doing an
object existence check in `repo_parse_commit_internal()` by default, the
updated logic will now instead cause `lookup_commit_in_graph()` to stop
doing the check by default. This behaviour continues to be tweakable by
the user via the GIT_COMMIT_GRAPH_PARANOIA environment variable.

Note that this requires us to amend some tests to manually turn on the
paranoid checks again. This is because we cause repository corruption by
manually deleting objects which are part of the commit graph already.
These circumstances shouldn't usually happen in repositories.

Reported-by: Jeff King <peff@peff.net>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-26 10:10:00 +09:00
Josh Soref
62b4f7b9c6 doc: refer to internet archive
These pages are no longer reachable from their original locations,
which makes things difficult for readers. Instead, switch to linking to
the Internet Archive for the content.

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-26 10:07:06 +09:00
Josh Soref
28a0c65f5d doc: update links for andre-simon.de
Beyond the fact that it's somewhat traditional to respect sites'
self-identification, it's helpful for links to point to the things
that people expect them to reference. Here that means linking to
specific pages instead of a domain.

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-26 10:07:05 +09:00
Josh Soref
d05b08cd52 doc: switch links to https
These sites offer https versions of their content.
Using the https versions provides some protection for users.

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-26 10:07:05 +09:00
Josh Soref
65175d9ea2 doc: update links to current pages
It's somewhat traditional to respect sites' self-identification.

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-26 10:07:05 +09:00
Josh Brobst
cbf498eb53 builtin/reflog.c: fix dry-run option short name
The documentation for reflog states that the --dry-run option of the
expire and delete subcommands has a corresponding short name, -n.
However, 33d7bdd645 (builtin/reflog.c: use parse-options api for expire,
delete subcommands, 2022-01-06) did not include this short name in the
new options parsing.

Re-add the short name in the new dry-run option definitions.

Signed-off-by: Josh Brobst <josh@brob.st>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-26 09:35:42 +09:00
Junio C Hamano
d44b517137 orphan/unborn: fix use of 'orphan' in end-user facing messages
"orphan branch" is not even grammatical ("orphaned branch" is), and
we have been using "unborn branch" to mean the state where the HEAD
points at a branch that does not yet exist.

Update end-user facing messages to correct them.  There are cases
other random words are used (e.g., "unparented branch") but now we
have a glossary entry, use the term "unborn branch" consistently.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-24 12:11:23 +09:00
Junio C Hamano
49dc156376 orphan/unborn: add to the glossary and use them consistently
To orphan is a verb that denotes the act of getting on an unborn
branch, and a few references to "orphan branch" in our documentation
are misuses of the word.  They caused end-user confusion, which was
made even worse because we did not have the term defined in the
glossary document.  Add entries for "unborn" branch and "orphan"
operation to the glossary, and adjust existing documentation
accordingly.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-24 12:11:23 +09:00
Junio C Hamano
9263c40a0a checkout: refactor die_if_checked_out() caller
There is a bit dense logic to make a call to "die_if_checked_out()"
while trying to check out a branch.  Extract it into a helper
function and give it a bit of comment to describe what is going on.

The most important part of the refactoring is the separation of the
guarding logic before making the call to die_if_checked_out() into
the caller specific part (e.g., the logic that decides that the
caller is trying to check out an existing branch) and the bypass due
to the "--ignore-other-worktrees" option.  The latter will be common
no matter how the current or future callers decides they need this
protection.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-23 15:01:09 +09:00
Jeff Hostetler
16fa3eebc0 t0212: test URL redacting in EVENT format
In the added tests cases, skip testing the `GIT_TRACE2_REDACT=0` case
because we would need to exactly model the full JSON event stream like
we did in the preceding basic tests and I do not think it is worth it.

Furthermore, the Trace2 routines print the same content in normal, perf,
or event format, and in t0210 and t0211 we already tested the basic
functionality, so no need to repeat it here.

In this test, we use the test-helper to unit test each of the event
messages where URLs can appear and confirm that they are redacted in
each event.

Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-23 10:30:33 +09:00
Jeff Hostetler
c73e7f80d3 t0211: test URL redacting in PERF format
This transmogrifies the test case that was just added to t0210, to also
cover the `GIT_TRACE2_PERF` backend.

Just like t0211, we now have to toggle the `TEST_PASSES_SANITIZE_LEAK`
annotation.

Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-23 10:30:33 +09:00
Johannes Schindelin
b7d49ac1ec trace2: redact passwords from https:// URLs by default
It is an unsafe practice to call something like

	git clone https://user:password@example.com/

This not only risks leaking the password "over the shoulder" or into the
readline history of the current Unix shell, it also gets logged via
Trace2 if enabled.

Let's at least avoid logging such secrets via Trace2, much like we avoid
logging secrets in `http.c`. Much like the code in `http.c` is guarded
via `GIT_TRACE_REDACT` (defaulting to `true`), we guard the new code via
`GIT_TRACE2_REDACT` (also defaulting to `true`).

The new tests added in this commit uncover leaks in `builtin/clone.c`
and `remote.c`. Therefore we need to turn off
`TEST_PASSES_SANITIZE_LEAK`. The reasons:

- We observed that `the_repository->remote_status` is not released
  properly.

- We are using `url...insteadOf` and that runs into a code path where an
  allocated URL is replaced with another URL, and the original URL is
  never released.

- `remote_states` contains plenty of `struct remote`s whose refspecs
  seem to be usually allocated by never released.

More investigation is needed here to identify the exact cause and
proper fixes for these leaks/bugs.

Co-authored-by: Jeff Hostetler <jeffhostetler@github.com>
Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-23 10:30:33 +09:00
Jeff Hostetler
abcdb978ea trace2: fix signature of trace2_def_param() macro
Add `struct key_value_info` argument to `trace2_def_param()`.

In dc90208497 (trace2: plumb config kvi, 2023-06-28) a `kvi`
argument was added to `trace2_def_param_fl()` but the macro
was not up updated. Let's fix that.

Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-23 10:30:32 +09:00
Antonin Delpeuch
4f7fd79e57 merge-file: add --diff-algorithm option
Make it possible to use other diff algorithms than the 'myers'
default algorithm, when using the 'git merge-file' command, to help
avoid spurious conflicts by selecting a more recent algorithm such
as 'histogram', for instance when using 'git merge-file' as part of
a custom merge driver.

Signed-off-by: Antonin Delpeuch <antonin@delpeuch.eu>
Reviewed-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-22 14:23:06 +09:00
Junio C Hamano
564d0252ca Git 2.43
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-20 10:28:15 +09:00
Junio C Hamano
c3cc3e1da7 Merge tag 'l10n-2.43.0-rnd2' of https://github.com/git-l10n/git-po
l10n-2.43.0-rnd2

* tag 'l10n-2.43.0-rnd2' of https://github.com/git-l10n/git-po:
  l10n: zh-TW: Git 2.43.0-rc1
  l10n: Update German translation
  l10n: bg.po: Updated Bulgarian translation (5579t)
  l10n: zh_CN: for git 2.43.0-rc1
  l10n: Update Catalan translation
  l10n: po-id for 2.43 (round 1)
  l10n: fr: v2.43.0 rnd 2
  l10n: update uk localization for v2.43
  l10n: sv.po: Update Swedish translation (5579t)
  l10n: tr: v2.43.0
2023-11-20 10:27:33 +09:00
Junio C Hamano
d003a26cca Merge branch 'vd/glossary-dereference-peel'
"To dereference" and "to peel" were sometimes used in in-code
comments and documentation but without description in the glossary.

* vd/glossary-dereference-peel:
  glossary: add definitions for dereference & peel
2023-11-20 09:57:23 +09:00
Junio C Hamano
e9eb93bb2a Merge branch 'tz/send-email-helpfix'
Typoes in "git send-email -h" have been corrected.

* tz/send-email-helpfix:
  send-email: remove stray characters from usage
2023-11-20 09:57:22 +09:00
Jiang Xin
d303432667 Merge branch 'l10n/zh-TW/2023-11-19' of github.com:l10n-tw/git-po
* 'l10n/zh-TW/2023-11-19' of github.com:l10n-tw/git-po:
  l10n: zh-TW: Git 2.43.0-rc1
2023-11-20 07:57:09 +08:00
Arthur Chan
ee41e2d41f fuzz: add new oss-fuzz fuzzer for date.c / date.h
Signed-off-by: Arthur Chan <arthur.chan@adalogics.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-20 08:17:51 +09:00
Johannes Schindelin
50f1abcff6 packfile.c: fix a typo in each_file_in_pack_dir_fn()'s declaration
One parameter is called `file_pach`. On the face of it, this looks as if
it was supposed to talk about a `path` instead of a `pach`.

However, looking at the way this callback is called, it gets fed the
`d_name` from a directory entry, which provides just the file name, not
the full path. Therefore, let's fix this by calling the parameter
`file_name` instead.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-20 08:15:50 +09:00
Yi-Jyun Pan
3d735322df l10n: zh-TW: Git 2.43.0-rc1
Signed-off-by: Yi-Jyun Pan <pan93412@gmail.com>
2023-11-19 23:35:21 +08:00
Jiang Xin
88d26d3fa3 Merge branch 'master' of github.com:alshopov/git-po
* 'master' of github.com:alshopov/git-po:
  l10n: bg.po: Updated Bulgarian translation (5579t)
2023-11-19 20:56:21 +08:00
Ralf Thielow
ed8b3c3078 l10n: Update German translation
Signed-off-by: Ralf Thielow <ralf.thielow@gmail.com>
2023-11-18 14:19:34 +01:00
Alexander Shopov
f42a8bb329 l10n: bg.po: Updated Bulgarian translation (5579t)
Signed-off-by: Alexander Shopov <ash@kambanaria.org>
2023-11-18 13:25:32 +01:00
Teng Long
64294acb07 l10n: zh_CN: for git 2.43.0-rc1
Signed-off-by: Teng Long <dyroneteng@gmail.com>
Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
2023-11-18 12:05:13 +08:00
Jiang Xin
4385297f0d Merge branch '2.43-uk-update' of github.com:arkid15r
* '2.43-uk-update' of github.com:arkid15r/git-ukrainian-l10n:
  l10n: update uk localization for v2.43
2023-11-18 10:51:56 +08:00
Jiang Xin
e3bd57a674 Merge branch 'catalan' of github.com:Softcatala/git-po
* 'catalan' of github.com:Softcatala/git-po:
  l10n: Update Catalan translation
2023-11-18 10:48:39 +08:00
Jiang Xin
ec93805ce3 Merge branch 'tr-l10n' of github.com:bitigchi/git-po
* 'tr-l10n' of github.com:bitigchi/git-po:
  l10n: tr: v2.43.0
2023-11-18 10:45:56 +08:00
Jiang Xin
4dcb316637 Merge branch 'fr_v2.43.0' of github.com:jnavila/git
* 'fr_v2.43.0' of github.com:jnavila/git:
  l10n: fr: v2.43.0 rnd 2
2023-11-18 10:43:22 +08:00
Jiang Xin
4dedb407b1 Merge branch 'po-id' of github.com:bagasme/git-po
* 'po-id' of github.com:bagasme/git-po:
  l10n: po-id for 2.43 (round 1)
2023-11-18 10:42:48 +08:00
Jiang Xin
1af36a14a4 Merge branch 'master' of github.com:nafmo/git-l10n-sv
* 'master' of github.com:nafmo/git-l10n-sv:
  l10n: sv.po: Update Swedish translation (5579t)
2023-11-18 10:42:04 +08:00
Patrick Steinhardt
29a186917b refs: remove delete_refs callback from backends
Now that `refs_delete_refs` is implemented in a generic way via the ref
transaction interfaces there are no callers left that invoke the
`delete_refs` callback anymore. Remove it from all of our backends.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-17 10:12:12 +09:00
Patrick Steinhardt
d6f8e72982 refs: deduplicate code to delete references
Both the files and the packed-refs reference backends now use the same
generic transactions-based code to delete references. Let's pull these
implementations up into `refs_delete_refs()` to deduplicate the code.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-17 10:12:12 +09:00
Patrick Steinhardt
e85e5dd78a refs/files: use transactions to delete references
In the `files_delete_refs()` callback function of the files backend we
implement deletion of references. This is done in two steps:

    1. We lock the packed-refs file and delete all references from it in
       a single transaction.

    2. We delete all loose references via separate calls to
       `refs_delete_ref()`.

These steps essentially duplicate the logic around locking and deletion
order that we already have in the transactional interfaces, where we do
know to lock and evict references from the packed-refs file. Despite the
fact that we duplicate the logic, it's also less efficient than if we
used a single generic transaction:

    - The transactional interface knows to skip locking of the packed
      refs in case they don't contain any of the refs which are about to
      be deleted.

    - We end up creating N+1 separate reference transactions, one for
      the packed-refs file and N for the individual loose references.

Refactor the code to instead delete references via a single transaction.
As we don't assert the expected old object ID this is equivalent to the
previous behaviour, and we already do the same in the packed-refs
backend.

Despite the fact that the result is simpler to reason about, this change
also results in improved performance. The following benchmarks have been
executed in linux.git:

```
$ hyperfine -n '{rev}, packed={packed} refcount={refcount}' \
    -L packed true,false -L refcount 1,1000 -L rev master,pks-ref-store-generic-delete-refs \
    --setup 'git -C /home/pks/Development/git switch --detach {rev} && make -C /home/pks/Development/git -j17' \
    --prepare 'printf "create refs/heads/new-branch-%d HEAD\n" $(seq {refcount}) | git -C /home/pks/Reproduction/linux.git update-ref --stdin && if test {packed} = true; then git pack-refs --all; fi' \
    --warmup=10 \
    '/home/pks/Development/git/bin-wrappers/git -C /home/pks/Reproduction/linux.git branch -d new-branch-{1..{refcount}}'

Benchmark 1: master packed=true refcount=1
  Time (mean ± σ):       7.8 ms ±   1.6 ms    [User: 3.4 ms, System: 4.4 ms]
  Range (min … max):     5.5 ms …  11.0 ms    120 runs

Benchmark 2: master packed=false refcount=1
  Time (mean ± σ):       7.0 ms ±   1.1 ms    [User: 3.2 ms, System: 3.8 ms]
  Range (min … max):     5.7 ms …   9.8 ms    180 runs

Benchmark 3: master packed=true refcount=1000
  Time (mean ± σ):     283.8 ms ±   5.2 ms    [User: 45.7 ms, System: 231.5 ms]
  Range (min … max):   276.7 ms … 291.6 ms    10 runs

Benchmark 4: master packed=false refcount=1000
  Time (mean ± σ):     284.4 ms ±   5.3 ms    [User: 44.2 ms, System: 233.6 ms]
  Range (min … max):   277.1 ms … 293.3 ms    10 runs

Benchmark 5: generic-delete-refs packed=true refcount=1
  Time (mean ± σ):       6.2 ms ±   1.8 ms    [User: 2.3 ms, System: 3.9 ms]
  Range (min … max):     4.1 ms …  12.2 ms    142 runs

Benchmark 6: generic-delete-refs packed=false refcount=1
  Time (mean ± σ):       7.1 ms ±   1.4 ms    [User: 2.8 ms, System: 4.3 ms]
  Range (min … max):     4.2 ms …  10.8 ms    157 runs

Benchmark 7: generic-delete-refs packed=true refcount=1000
  Time (mean ± σ):     198.9 ms ±   1.7 ms    [User: 29.5 ms, System: 165.7 ms]
  Range (min … max):   196.1 ms … 201.4 ms    10 runs

Benchmark 8: generic-delete-refs packed=false refcount=1000
  Time (mean ± σ):     199.7 ms ±   7.8 ms    [User: 32.2 ms, System: 163.2 ms]
  Range (min … max):   193.8 ms … 220.7 ms    10 runs

Summary
  generic-delete-refs packed=true refcount=1 ran
    1.14 ± 0.37 times faster than master packed=false refcount=1
    1.15 ± 0.40 times faster than generic-delete-refs packed=false refcount=1
    1.26 ± 0.44 times faster than master packed=true refcount=1
   32.24 ± 9.17 times faster than generic-delete-refs packed=true refcount=1000
   32.36 ± 9.29 times faster than generic-delete-refs packed=false refcount=1000
   46.00 ± 13.10 times faster than master packed=true refcount=1000
   46.10 ± 13.13 times faster than master packed=false refcount=1000
```

Especially in the case where we have many references we can see a clear
performance speedup of nearly 30%.

This is in contrast to the stated objecive in a27dcf89b6 (refs: make
delete_refs() virtual, 2016-09-04), where the virtual `delete_refs()`
function was introduced with the intent to speed things up rather than
making things slower. So it seems like we have outlived the need for a
virtual function.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-17 10:12:12 +09:00
Patrick Steinhardt
b02c7646f4 t5510: ensure that the packed-refs file needs locking
One of the tests in t5510 asserts that a `git fetch --prune` detects
failures to prune branches. This is done by locking the packed-refs
file, which would then later lead to a locking issue when Git tries to
rewrite the file to prune the branches from it.

Interestingly though, we do not pack the about-to-be-pruned branch into
the packed-refs file, so it never even contained that branch in the
first place. While this is good enough right now because the pruning
will always lock the file regardless of whether it contains the branch
or not, this is a mere implementation detail. In fact, we're about to
rewrite branch deletions to make use of the ref transaction interface,
which knows to skip rewrites of the packed-refs file in the case where
it does not contain the branches in the first place, and this will break
the test.

Prepare the test for that change by packing the refs before trying to
prune them.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-17 10:12:12 +09:00
Todd Zullinger
6ff658cc78 send-email: avoid duplicate specification warnings
A warning is issued for options which are specified more than once
beginning with perl-Getopt-Long >= 2.55.  In addition to causing users
to see warnings, this results in test failures which compare the output.
An example, from t9001-send-email.37:

  | +++ diff -u expect actual
  | --- expect      2023-11-14 10:38:23.854346488 +0000
  | +++ actual      2023-11-14 10:38:23.848346466 +0000
  | @@ -1,2 +1,7 @@
  | +Duplicate specification "no-chain-reply-to" for option "no-chain-reply-to"
  | +Duplicate specification "to-cover|to-cover!" for option "to-cover"
  | +Duplicate specification "cc-cover|cc-cover!" for option "cc-cover"
  | +Duplicate specification "no-thread" for option "no-thread"
  | +Duplicate specification "no-to-cover" for option "no-to-cover"
  |  fatal: longline.patch:35 is longer than 998 characters
  |  warning: no patches were sent
  | error: last command exited with $?=1
  | not ok 37 - reject long lines

Remove the duplicate option specs.  These are primarily the explicit
'--no-' prefix opts which were added in f471494303 (git-send-email.perl:
support no- prefix with older GetOptions, 2015-01-30).  This was done
specifically to support perl-5.8.0 which includes Getopt::Long 2.32[1].

Getopt::Long 2.33 added support for the '--no-' prefix natively by
appending '!' to the option specification string, which was included in
perl-5.8.1 and is not present in perl-5.8.0.  The previous commit bumped
the minimum supported Perl version to 5.8.1 so we no longer need to
provide the '--no-' variants for negatable options manually.

Teach `--git-completion-helper` to output the '--no-' options.  They are
not included in the options hash and would otherwise be lost.

Signed-off-by: Todd Zullinger <tmz@pobox.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-17 07:26:34 +09:00
Todd Zullinger
d13a73e383 perl: bump the required Perl version to 5.8.1 from 5.8.0
The following commit will make use of a Getopt::Long feature which is
only present in Perl >= 5.8.1.  Document that as the minimum version we
support.

Many of our Perl scripts will continue to run with 5.8.0 but this change
allows us to adjust them as needed without breaking any promises to our
users.

The Perl requirement was last changed in d48b284183 (perl: bump the
required Perl version to 5.8 from 5.6.[21], 2010-09-24).  At that time,
5.8.0 was 8 years old.  It is now over 21 years old.

Signed-off-by: Todd Zullinger <tmz@pobox.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-17 07:26:32 +09:00
Victoria Dye
294bfc2441 t/perf: add perf tests for for-each-ref
Add performance tests for 'for-each-ref'. The tests exercise different
combinations of filters/formats/options, as well as the overall performance
of 'git for-each-ref | git cat-file --batch-check' to demonstrate the
performance difference vs. 'git for-each-ref' with "%(*fieldname)" format
specifiers.

All tests are run against a repository with 40k loose refs - 10k commits,
each having a unique:

- branch
- custom ref (refs/custom/special_*)
- annotated tag pointing at the commit
- annotated tag pointing at the other annotated tag (i.e., a nested tag)

After those tests are finished, the refs are packed with 'pack-refs --all'
and the same tests are rerun.

Signed-off-by: Victoria Dye <vdye@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-16 14:03:01 +09:00
Victoria Dye
188782ecb1 ref-filter.c: use peeled tag for '*' format fields
In most builtins ('rev-parse <revision>^{}', 'show-ref --dereference'),
"dereferencing" a tag refers to a recursive peel of the tag object. Unlike
these cases, the dereferencing prefix ('*') in 'for-each-ref' format
specifiers triggers only a single, non-recursive dereference of a given tag
object. For most annotated tags, a single dereference is all that is needed
to access the tag's associated commit or tree; "recursive" and
"non-recursive" dereferencing are functionally equivalent in these cases.
However, nested tags (annotated tags whose target is another annotated tag)
dereferenced once return another tag, where a recursive dereference would
return the commit or tree.

Currently, if a user wants to filter & format refs and include information
about a recursively-dereferenced tag, they can do so with something like
'cat-file --batch-check':

    git for-each-ref --format="%(objectname)^{} %(refname)" <pattern> |
        git cat-file --batch-check="%(objectname) %(rest)"

But the combination of commands is inefficient. So, to improve the
performance of this use case and align the defererencing behavior of
'for-each-ref' with that of other commands, update the ref formatting code
to use the peeled tag (from 'peel_iterated_oid()') to populate '*' fields
rather than the tag's immediate target object (from 'get_tagged_oid()').

Additionally, add a test to 't6300-for-each-ref' to verify new nested tag
behavior and update 't6302-for-each-ref-filter.sh' to print the correct
value for nested dereferenced fields.

Signed-off-by: Victoria Dye <vdye@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-16 14:03:01 +09:00
Victoria Dye
d1dfe6e936 for-each-ref: clean up documentation of --format
Move the description of the `*` prefix from the --format option
documentation to the part of the command documentation that deals with other
object type-specific modifiers. Also reorganize and reword the remaining
--format documentation so that the explanation of the default format doesn't
interrupt the details on format string interpolation.

Signed-off-by: Victoria Dye <vdye@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-16 14:03:00 +09:00
Victoria Dye
bd98f9774e ref-filter.c: filter & format refs in the same callback
Update 'filter_and_format_refs()' to try to perform ref filtering &
formatting in a single ref iteration, without an intermediate 'struct
ref_array'. This can only be done if no operations need to be performed on a
pre-filtered array; specifically, if the refs are

- filtered on reachability,
- sorted, or
- formatted with ahead-behind information

they cannot be filtered & formatted in the same iteration. In that case,
fall back on the current filter-then-sort-then-format flow.

This optimization substantially improves memory usage due to no longer
storing a ref array in memory. In some cases, it also dramatically reduces
runtime (e.g. 'git for-each-ref --no-sort --count=1', which no longer loads
all refs into a 'struct ref_array' to printing only the first ref).

Signed-off-by: Victoria Dye <vdye@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-16 14:03:00 +09:00
Victoria Dye
613d991549 ref-filter.c: refactor to create common helper functions
Factor out parts of 'ref_array_push()', 'ref_filter_handler()', and
'filter_refs()' into new helper functions:

* Extract the code to grow a 'struct ref_array' and append a given 'struct
  ref_array_item *' to it from 'ref_array_push()' into 'ref_array_append()'.
* Extract the code to filter a given ref by refname & object ID then create
  a new 'struct ref_array_item *' from 'filter_one()' into
  'apply_ref_filter()'.
* Extract the code for filter pre-processing, contains cache creation, and
  ref iteration from 'filter_refs()' into 'do_filter_refs()'.

In later patches, these helpers will be used by new ref-filter API
functions. This patch does not result in any user-facing behavior changes or
changes to callers outside of 'ref-filter.c'.

Signed-off-by: Victoria Dye <vdye@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-16 14:03:00 +09:00
Victoria Dye
9c575b0202 ref-filter.c: rename 'ref_filter_handler()' to 'filter_one()'
Rename 'ref_filter_handler()' to 'filter_one()' to more clearly distinguish
it from other ref filtering callbacks that will be added in later patches.
The "*_one()" naming convention is common throughout the codebase for
iteration callbacks.

Signed-off-by: Victoria Dye <vdye@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-16 14:03:00 +09:00