Commit Graph

15358 Commits

Author SHA1 Message Date
Patrick Steinhardt
57db2a094d refs: introduce reftable backend
Due to scalability issues, Shawn Pearce has originally proposed a new
"reftable" format more than six years ago [1]. Initially, this new
format was implemented in JGit with promising results. Around two years
ago, we have then added the "reftable" library to the Git codebase via
a4bbd13be3 (Merge branch 'hn/reftable', 2021-12-15). With this we have
landed all the low-level code to read and write reftables. Notably
missing though was the integration of this low-level code into the Git
code base in the form of a new ref backend that ties all of this
together.

This gap is now finally closed by introducing a new "reftable" backend
into the Git codebase. This new backend promises to bring some notable
improvements to Git repositories:

  - It becomes possible to do truly atomic writes where either all refs
    are committed to disk or none are. This was not possible with the
    "files" backend because ref updates were split across multiple loose
    files.

  - The disk space required to store many refs is reduced, both compared
    to loose refs and packed-refs. This is enabled both by the reftable
    format being a binary format, which is more compact, and by prefix
    compression.

  - We can ignore filesystem-specific behaviour as ref names are not
    encoded via paths anymore. This means there is no need to handle
    case sensitivity on Windows systems or Unicode precomposition on
    macOS.

  - There is no need to rewrite the complete refdb anymore every time a
    ref is being deleted like it was the case for packed-refs. This
    means that ref deletions are now constant time instead of scaling
    linearly with the number of refs.

  - We can ignore file/directory conflicts so that it becomes possible
    to store both "refs/heads/foo" and "refs/heads/foo/bar".

  - Due to this property we can retain reflogs for deleted refs. We have
    previously been deleting reflogs together with their refs to avoid
    file/directory conflicts, which is not necessary anymore.

  - We can properly enumerate all refs. With the "files" backend it is
    not easily possible to distinguish between refs and non-refs because
    they may live side by side in the gitdir.

Not all of these improvements are realized with the current "reftable"
backend implementation. At this point, the new backend is supposed to be
a drop-in replacement for the "files" backend that is used by basically
all Git repositories nowadays. It strives for 1:1 compatibility, which
means that a user can expect the same behaviour regardless of whether
they use the "reftable" backend or the "files" backend for most of the
part.

Most notably, this means we artificially limit the capabilities of the
"reftable" backend to match the limits of the "files" backend. It is not
possible to create refs that would end up with file/directory conflicts,
we do not retain reflogs, we perform stricter-than-necessary checks.
This is done intentionally due to two main reasons:

  - It makes it significantly easier to land the "reftable" backend as
    tests behave the same. It would be tough to argue for each and every
    single test that doesn't pass with the "reftable" backend.

  - It ensures compatibility between repositories that use the "files"
    backend and repositories that use the "reftable" backend. Like this,
    hosters can migrate their repositories to use the "reftable" backend
    without causing issues for clients that use the "files" backend in
    their clones.

It is expected that these artificial limitations may eventually go away
in the long term.

Performance-wise things very much depend on the actual workload. The
following benchmarks compare the "files" and "reftable" backends in the
current version:

  - Creating N refs in separate transactions shows that the "files"
    backend is ~50% faster. This is not surprising given that creating a
    ref only requires us to create a single loose ref. The "reftable"
    backend will also perform auto compaction on updates. In real-world
    workloads we would likely also want to perform pack loose refs,
    which would likely change the picture.

        Benchmark 1: update-ref: create refs sequentially (refformat = files, refcount = 1)
          Time (mean ± σ):       2.1 ms ±   0.3 ms    [User: 0.6 ms, System: 1.7 ms]
          Range (min … max):     1.8 ms …   4.3 ms    133 runs

        Benchmark 2: update-ref: create refs sequentially (refformat = reftable, refcount = 1)
          Time (mean ± σ):       2.7 ms ±   0.1 ms    [User: 0.6 ms, System: 2.2 ms]
          Range (min … max):     2.4 ms …   2.9 ms    132 runs

        Benchmark 3: update-ref: create refs sequentially (refformat = files, refcount = 1000)
          Time (mean ± σ):      1.975 s ±  0.006 s    [User: 0.437 s, System: 1.535 s]
          Range (min … max):    1.969 s …  1.980 s    3 runs

        Benchmark 4: update-ref: create refs sequentially (refformat = reftable, refcount = 1000)
          Time (mean ± σ):      2.611 s ±  0.013 s    [User: 0.782 s, System: 1.825 s]
          Range (min … max):    2.597 s …  2.622 s    3 runs

        Benchmark 5: update-ref: create refs sequentially (refformat = files, refcount = 100000)
          Time (mean ± σ):     198.442 s ±  0.241 s    [User: 43.051 s, System: 155.250 s]
          Range (min … max):   198.189 s … 198.670 s    3 runs

        Benchmark 6: update-ref: create refs sequentially (refformat = reftable, refcount = 100000)
          Time (mean ± σ):     294.509 s ±  4.269 s    [User: 104.046 s, System: 190.326 s]
          Range (min … max):   290.223 s … 298.761 s    3 runs

  - Creating N refs in a single transaction shows that the "files"
    backend is significantly slower once we start to write many refs.
    The "reftable" backend only needs to update two files, whereas the
    "files" backend needs to write one file per ref.

        Benchmark 1: update-ref: create many refs (refformat = files, refcount = 1)
          Time (mean ± σ):       1.9 ms ±   0.1 ms    [User: 0.4 ms, System: 1.4 ms]
          Range (min … max):     1.8 ms …   2.6 ms    151 runs

        Benchmark 2: update-ref: create many refs (refformat = reftable, refcount = 1)
          Time (mean ± σ):       2.5 ms ±   0.1 ms    [User: 0.7 ms, System: 1.7 ms]
          Range (min … max):     2.4 ms …   3.4 ms    148 runs

        Benchmark 3: update-ref: create many refs (refformat = files, refcount = 1000)
          Time (mean ± σ):     152.5 ms ±   5.2 ms    [User: 19.1 ms, System: 133.1 ms]
          Range (min … max):   148.5 ms … 167.8 ms    15 runs

        Benchmark 4: update-ref: create many refs (refformat = reftable, refcount = 1000)
          Time (mean ± σ):      58.0 ms ±   2.5 ms    [User: 28.4 ms, System: 29.4 ms]
          Range (min … max):    56.3 ms …  72.9 ms    40 runs

        Benchmark 5: update-ref: create many refs (refformat = files, refcount = 1000000)
          Time (mean ± σ):     152.752 s ±  0.710 s    [User: 20.315 s, System: 131.310 s]
          Range (min … max):   152.165 s … 153.542 s    3 runs

        Benchmark 6: update-ref: create many refs (refformat = reftable, refcount = 1000000)
          Time (mean ± σ):     51.912 s ±  0.127 s    [User: 26.483 s, System: 25.424 s]
          Range (min … max):   51.769 s … 52.012 s    3 runs

  - Deleting a ref in a fully-packed repository shows that the "files"
    backend scales with the number of refs. The "reftable" backend has
    constant-time deletions.

        Benchmark 1: update-ref: delete ref (refformat = files, refcount = 1)
          Time (mean ± σ):       1.7 ms ±   0.1 ms    [User: 0.4 ms, System: 1.2 ms]
          Range (min … max):     1.6 ms …   2.1 ms    316 runs

        Benchmark 2: update-ref: delete ref (refformat = reftable, refcount = 1)
          Time (mean ± σ):       1.8 ms ±   0.1 ms    [User: 0.4 ms, System: 1.3 ms]
          Range (min … max):     1.7 ms …   2.1 ms    294 runs

        Benchmark 3: update-ref: delete ref (refformat = files, refcount = 1000)
          Time (mean ± σ):       2.0 ms ±   0.1 ms    [User: 0.5 ms, System: 1.4 ms]
          Range (min … max):     1.9 ms …   2.5 ms    287 runs

        Benchmark 4: update-ref: delete ref (refformat = reftable, refcount = 1000)
          Time (mean ± σ):       1.9 ms ±   0.1 ms    [User: 0.5 ms, System: 1.3 ms]
          Range (min … max):     1.8 ms …   2.1 ms    217 runs

        Benchmark 5: update-ref: delete ref (refformat = files, refcount = 1000000)
          Time (mean ± σ):     229.8 ms ±   7.9 ms    [User: 182.6 ms, System: 46.8 ms]
          Range (min … max):   224.6 ms … 245.2 ms    6 runs

        Benchmark 6: update-ref: delete ref (refformat = reftable, refcount = 1000000)
          Time (mean ± σ):       2.0 ms ±   0.0 ms    [User: 0.6 ms, System: 1.3 ms]
          Range (min … max):     2.0 ms …   2.1 ms    3 runs

  - Listing all refs shows no significant advantage for either of the
    backends. The "files" backend is a bit faster, but not by a
    significant margin. When repositories are not packed the "reftable"
    backend outperforms the "files" backend because the "reftable"
    backend performs auto-compaction.

        Benchmark 1: show-ref: print all refs (refformat = files, refcount = 1, packed = true)
          Time (mean ± σ):       1.6 ms ±   0.1 ms    [User: 0.4 ms, System: 1.1 ms]
          Range (min … max):     1.5 ms …   2.0 ms    1729 runs

        Benchmark 2: show-ref: print all refs (refformat = reftable, refcount = 1, packed = true)
          Time (mean ± σ):       1.6 ms ±   0.1 ms    [User: 0.4 ms, System: 1.1 ms]
          Range (min … max):     1.5 ms …   1.8 ms    1816 runs

        Benchmark 3: show-ref: print all refs (refformat = files, refcount = 1000, packed = true)
          Time (mean ± σ):       4.3 ms ±   0.1 ms    [User: 0.9 ms, System: 3.3 ms]
          Range (min … max):     4.1 ms …   4.6 ms    645 runs

        Benchmark 4: show-ref: print all refs (refformat = reftable, refcount = 1000, packed = true)
          Time (mean ± σ):       4.5 ms ±   0.2 ms    [User: 1.0 ms, System: 3.3 ms]
          Range (min … max):     4.2 ms …   5.9 ms    643 runs

        Benchmark 5: show-ref: print all refs (refformat = files, refcount = 1000000, packed = true)
          Time (mean ± σ):      2.537 s ±  0.034 s    [User: 0.488 s, System: 2.048 s]
          Range (min … max):    2.511 s …  2.627 s    10 runs

        Benchmark 6: show-ref: print all refs (refformat = reftable, refcount = 1000000, packed = true)
          Time (mean ± σ):      2.712 s ±  0.017 s    [User: 0.653 s, System: 2.059 s]
          Range (min … max):    2.692 s …  2.752 s    10 runs

        Benchmark 7: show-ref: print all refs (refformat = files, refcount = 1, packed = false)
          Time (mean ± σ):       1.6 ms ±   0.1 ms    [User: 0.4 ms, System: 1.1 ms]
          Range (min … max):     1.5 ms …   1.9 ms    1834 runs

        Benchmark 8: show-ref: print all refs (refformat = reftable, refcount = 1, packed = false)
          Time (mean ± σ):       1.6 ms ±   0.1 ms    [User: 0.4 ms, System: 1.1 ms]
          Range (min … max):     1.4 ms …   2.0 ms    1840 runs

        Benchmark 9: show-ref: print all refs (refformat = files, refcount = 1000, packed = false)
          Time (mean ± σ):      13.8 ms ±   0.2 ms    [User: 2.8 ms, System: 10.8 ms]
          Range (min … max):    13.3 ms …  14.5 ms    208 runs

        Benchmark 10: show-ref: print all refs (refformat = reftable, refcount = 1000, packed = false)
          Time (mean ± σ):       4.5 ms ±   0.2 ms    [User: 1.2 ms, System: 3.3 ms]
          Range (min … max):     4.3 ms …   6.2 ms    624 runs

        Benchmark 11: show-ref: print all refs (refformat = files, refcount = 1000000, packed = false)
          Time (mean ± σ):     12.127 s ±  0.129 s    [User: 2.675 s, System: 9.451 s]
          Range (min … max):   11.965 s … 12.370 s    10 runs

        Benchmark 12: show-ref: print all refs (refformat = reftable, refcount = 1000000, packed = false)
          Time (mean ± σ):      2.799 s ±  0.022 s    [User: 0.735 s, System: 2.063 s]
          Range (min … max):    2.769 s …  2.836 s    10 runs

  - Printing a single ref shows no real difference between the "files"
    and "reftable" backends.

        Benchmark 1: show-ref: print single ref (refformat = files, refcount = 1)
          Time (mean ± σ):       1.5 ms ±   0.1 ms    [User: 0.4 ms, System: 1.0 ms]
          Range (min … max):     1.4 ms …   1.8 ms    1779 runs

        Benchmark 2: show-ref: print single ref (refformat = reftable, refcount = 1)
          Time (mean ± σ):       1.6 ms ±   0.1 ms    [User: 0.4 ms, System: 1.1 ms]
          Range (min … max):     1.4 ms …   2.5 ms    1753 runs

        Benchmark 3: show-ref: print single ref (refformat = files, refcount = 1000)
          Time (mean ± σ):       1.5 ms ±   0.1 ms    [User: 0.3 ms, System: 1.1 ms]
          Range (min … max):     1.4 ms …   1.9 ms    1840 runs

        Benchmark 4: show-ref: print single ref (refformat = reftable, refcount = 1000)
          Time (mean ± σ):       1.6 ms ±   0.1 ms    [User: 0.4 ms, System: 1.1 ms]
          Range (min … max):     1.5 ms …   2.0 ms    1831 runs

        Benchmark 5: show-ref: print single ref (refformat = files, refcount = 1000000)
          Time (mean ± σ):       1.6 ms ±   0.1 ms    [User: 0.4 ms, System: 1.1 ms]
          Range (min … max):     1.5 ms …   2.1 ms    1848 runs

        Benchmark 6: show-ref: print single ref (refformat = reftable, refcount = 1000000)
          Time (mean ± σ):       1.6 ms ±   0.1 ms    [User: 0.4 ms, System: 1.1 ms]
          Range (min … max):     1.5 ms …   2.1 ms    1762 runs

So overall, performance depends on the usecases. Except for many
sequential writes the "reftable" backend is roughly on par or
significantly faster than the "files" backend though. Given that the
"files" backend has received 18 years of optimizations by now this can
be seen as a win. Furthermore, we can expect that the "reftable" backend
will grow faster over time when attention turns more towards
optimizations.

The complete test suite passes, except for those tests explicitly marked
to require the REFFILES prerequisite. Some tests in t0610 are marked as
failing because they depend on still-in-flight bug fixes. Tests can be
run with the new backend by setting the GIT_TEST_DEFAULT_REF_FORMAT
environment variable to "reftable".

There is a single known conceptual incompatibility with the dumb HTTP
transport. As "info/refs" SHOULD NOT contain the HEAD reference, and
because the "HEAD" file is not valid anymore, it is impossible for the
remote client to figure out the default branch without changing the
protocol. This shortcoming needs to be handled in a subsequent patch
series.

As the reftable library has already been introduced a while ago, this
commit message will not go into the details of how exactly the on-disk
format works. Please refer to our preexisting technical documentation at
Documentation/technical/reftable for this.

[1]: https://public-inbox.org/git/CAJo=hJtyof=HRy=2sLP0ng0uZ4=S-DpZ5dR1aF+VHVETKG20OQ@mail.gmail.com/

Original-idea-by: Shawn Pearce <spearce@spearce.org>
Based-on-patch-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-07 08:28:37 -08:00
Junio C Hamano
235986be82 The fourteenth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-06 14:31:50 -08:00
Junio C Hamano
2a540e432f The thirteenth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-02 11:31:51 -08:00
Junio C Hamano
bbc8c05670 Merge branch 'jc/ls-files-doc-update'
The documentation for the --exclude-per-directory option marked it
as deprecated, which confused readers into thinking there may be a
plan to remove it in the future, which was not our intention.

* jc/ls-files-doc-update:
  ls-files: avoid the verb "deprecate" for individual options
2024-02-02 11:31:51 -08:00
Junio C Hamano
9e189a03da Merge branch 'ad/custom-merge-placeholder-for-symbolic-pathnames'
The labels on conflict markers for the common ancestor, our version,
and the other version are available to custom 3-way merge driver
via %S, %X, and %Y placeholders.

* ad/custom-merge-placeholder-for-symbolic-pathnames:
  merge-ll: expose revision names to custom drivers
2024-02-02 11:31:50 -08:00
Junio C Hamano
bc7ee2e5e1 The twelfth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-30 13:34:13 -08:00
Junio C Hamano
e14c0ab176 Merge branch 'rj/advice-disable-how-to-disable'
All conditional "advice" messages show how to turn them off, which
becomes repetitive.  Setting advice.* configuration explicitly on
now omits the instruction part.

* rj/advice-disable-how-to-disable:
  advice: allow disabling the automatic hint in advise_if_enabled()
2024-01-30 13:34:12 -08:00
Junio C Hamano
c5b454771e The eleventh batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-29 16:03:01 -08:00
Junio C Hamano
8282f95928 Merge branch 'ps/not-so-many-refs-are-special'
Define "special ref" as a very narrow set that consists of
FETCH_HEAD and MERGE_HEAD, and clarify everything else that used to
be classified as such are actually just pseudorefs.

* ps/not-so-many-refs-are-special:
  Documentation: add "special refs" to the glossary
  refs: redefine special refs
  refs: convert MERGE_AUTOSTASH to become a normal pseudo-ref
  sequencer: introduce functions to handle autostashes via refs
  refs: convert AUTO_MERGE to become a normal pseudo-ref
  sequencer: delete REBASE_HEAD in correct repo when picking commits
  sequencer: clean up pseudo refs with REF_NO_DEREF
2024-01-29 16:03:00 -08:00
Junio C Hamano
68812df310 Merge branch 'jc/majordomo-to-subspace'
Doc update.

* jc/majordomo-to-subspace:
  Docs: majordomo@vger.kernel.org has been decomissioned
2024-01-29 16:03:00 -08:00
Junio C Hamano
a0003a5490 Merge branch 'nb/rebase-x-shell-docfix'
Doc update.

* nb/rebase-x-shell-docfix:
  rebase: fix documentation about used shell in -x
2024-01-29 16:02:59 -08:00
Junio C Hamano
b50a608ba2 The tenth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-26 08:54:47 -08:00
Junio C Hamano
f3657b3526 Merge branch 'ne/doc-filter-blob-limit-fix'
Docfix.

* ne/doc-filter-blob-limit-fix:
  rev-list-options: fix off-by-one in '--filter=blob:limit=<n>' explainer
2024-01-26 08:54:45 -08:00
Junio C Hamano
bed1524e04 Merge branch 'rj/advice-delete-branch-not-fully-merged'
The error message given when "git branch -d branch" fails due to
commits unique to the branch has been split into an error and a new
conditional advice message.

* rj/advice-delete-branch-not-fully-merged:
  branch: make the advice to force-deleting a conditional one
  advice: fix an unexpected leading space
  advice: sort the advice related lists
2024-01-26 08:54:45 -08:00
Junio C Hamano
0009542cab ls-files: avoid the verb "deprecate" for individual options
When e750951e (ls-files: guide folks to --exclude-standard over
other --exclude* options, 2023-01-13) updated the documentation to
give greater visibility to the `--exclude-standard` option, it marked
the `--exclude-per-directory` option as "deprecated".

While it is technically correct that being deprecated does not
necessarily mean it is planned to be removed later, it seems to
cause confusion to readers, especially when we merely mean

    The option Y can be used to achieve the same thing as the option
    X much simpler. To those of you who aren't familiar with either
    X or Y, we would recommend to use Y when appropriate.

This is especially true for `--exclude-standard` vs the combination
of more granular `--exclude-from` and `--exclude-per-directory`
options.  It is true that one common combination of the granular
options can be obtained by just giving the former, but that does not
necessarily mean a more granular control is not necessary.

State the reason why we recommend readers `--exclude-standard` in
the description of `--exclude-per-directory`, instead of saying that
the option is deprecated.  Also, spell out the recipe to emulate
what `--exclude-standard` does, so that the users can give it minute
tweaks (like "do the same as Git Porcelain, except I do not want to
read the global exclusion file from core.excludes").

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-25 10:55:53 -08:00
Antonin Delpeuch
81effe9468 merge-ll: expose revision names to custom drivers
Custom merge drivers need access to the names of the revisions they
are working on, so that the merge conflict markers they introduce
can refer to those revisions. The placeholders '%S', '%X' and '%Y'
are introduced to this end.

Signed-off-by: Antonin Delpeuch <antonin@delpeuch.eu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-24 13:15:06 -08:00
Junio C Hamano
af3d2c160f Docs: majordomo@vger.kernel.org has been decomissioned
Update the instruction for subscribing to the Git mailing list
we have on a few documentation pages.

Reported-by: Kyle Lippincott <spectral@google.com>
Helped-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-20 10:09:07 -08:00
Junio C Hamano
e02ecfcc53 The ninth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-19 15:04:46 -08:00
Junio C Hamano
22695a38a4 Merge branch 'bk/bisect-doc-fix'
Synopsis fix.

* bk/bisect-doc-fix:
  doc: refer to pathspec instead of path
  doc: use singular form of repeatable path arg
2024-01-19 15:04:46 -08:00
Junio C Hamano
f033388b0f Merge branch 'tb/fetch-all-configuration'
"git fetch" learned to pay attention to "fetch.all" configuration
variable, which pretends as if "--all" was passed from the command
line when no remote parameter was given.

* tb/fetch-all-configuration:
  fetch: add new config option fetch.all
2024-01-19 15:04:45 -08:00
Junio C Hamano
5d1ee0749b Merge branch 'rj/clarify-branch-doc-m'
Doc update.

* rj/clarify-branch-doc-m:
  branch: clarify <oldbranch> term
2024-01-19 15:04:45 -08:00
Patrick Steinhardt
8df4c5d205 Documentation: add "special refs" to the glossary
Add the "special refs" term to our glossary.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-19 11:10:42 -08:00
Nikolay Borisov
f10031fadd rebase: fix documentation about used shell in -x
The shell used when using the -x option is erroneously documented to be
the one pointed to by the $SHELL environmental variable. This was true
when rebase was implemented as a shell script but this is no longer
true.

Signed-off-by: Nikolay Borisov <nik.borisov@suse.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-17 16:14:57 -08:00
Rubén Justo
d919965af1 advice: allow disabling the automatic hint in advise_if_enabled()
Using advise_if_enabled() to display an advice will automatically
include instructions on how to disable the advice, alongside the main
advice:

	hint: use --reapply-cherry-picks to include skipped commits
	hint: Disable this message with "git config advice.skippedCherryPicks false"

To do so, we provide a knob which can be used to disable the advice.

But also to tell us the opposite: to show the advice.

Let's not include the deactivation instructions for an advice if the
user explicitly sets its visibility.

Signed-off-by: Rubén Justo <rjusto@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-16 13:07:00 -08:00
Junio C Hamano
186b115d30 The eighth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-16 10:11:58 -08:00
Junio C Hamano
a57da6bfee Merge branch 'ib/rebase-reschedule-doc'
Doc update.

* ib/rebase-reschedule-doc:
  rebase: clarify --reschedule-failed-exec default
2024-01-16 10:11:58 -08:00
Junio C Hamano
32c6fc3e30 Merge branch 'ps/refstorage-extension'
Introduce a new extension "refstorage" so that we can mark a
repository that uses a non-default ref backend, like reftable.

* ps/refstorage-extension:
  t9500: write "extensions.refstorage" into config
  builtin/clone: introduce `--ref-format=` value flag
  builtin/init: introduce `--ref-format=` value flag
  builtin/rev-parse: introduce `--show-ref-format` flag
  t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar
  setup: introduce GIT_DEFAULT_REF_FORMAT envvar
  setup: introduce "extensions.refStorage" extension
  setup: set repository's formats on init
  setup: start tracking ref storage format
  refs: refactor logic to look up storage backends
  worktree: skip reading HEAD when repairing worktrees
  t: introduce DEFAULT_REPO_FORMAT prereq
2024-01-16 10:11:57 -08:00
Nikolay Edigaryev
8f50984cf4 rev-list-options: fix off-by-one in '--filter=blob:limit=<n>' explainer
'--filter=blob:limit=<n>' was introduced in 25ec7bcac0 (list-objects:
filter objects in traverse_commit_list, 2017-11-21) and later expanded
to bitmaps in 84243da129 (pack-bitmap: implement BLOB_LIMIT filtering,
2020-02-14)

The logic that was introduced in these commits (and that still persists
to this day) omits blobs larger than _or equal_ to n bytes or units.

However, the documentation (Documentation/rev-list-options.txt) states:

>The form '--filter=blob:limit=<n>[kmg]' omits blobs larger than n
bytes or units. n may be zero.

Moreover, the t6113-rev-list-bitmap-filters.sh tests for exactly this
logic, so it seems it is the documentation that needs fixing, not the
code.

This changes the explanation to be similar to
Documentation/git-clone.txt, which is correct.

Signed-off-by: Nikolay Edigaryev <edigaryev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-16 08:53:13 -08:00
Junio C Hamano
d4dbce1db5 The seventh batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-12 16:09:57 -08:00
Junio C Hamano
b3049bbb97 Merge branch 'cp/git-flush-is-an-env-bool'
Unlike other environment variables that took the usual
true/false/yes/no as well as 0/1, GIT_FLUSH only understood 0/1,
which has been corrected.

* cp/git-flush-is-an-env-bool:
  write-or-die: make GIT_FLUSH a Boolean environment variable
2024-01-12 16:09:57 -08:00
Junio C Hamano
566471105c Merge branch 'ms/rebase-insnformat-doc-fix'
Docfix.

* ms/rebase-insnformat-doc-fix:
  Documentation: fix statement about rebase.instructionFormat
2024-01-12 16:09:57 -08:00
Junio C Hamano
0fea6b73f1 Merge branch 'tb/multi-pack-verbatim-reuse'
Streaming spans of packfile data used to be done only from a
single, primary, pack in a repository with multiple packfiles.  It
has been extended to allow reuse from other packfiles, too.

* tb/multi-pack-verbatim-reuse: (26 commits)
  t/perf: add performance tests for multi-pack reuse
  pack-bitmap: enable reuse from all bitmapped packs
  pack-objects: allow setting `pack.allowPackReuse` to "single"
  t/test-lib-functions.sh: implement `test_trace2_data` helper
  pack-objects: add tracing for various packfile metrics
  pack-bitmap: prepare to mark objects from multiple packs for reuse
  pack-revindex: implement `midx_pair_to_pack_pos()`
  pack-revindex: factor out `midx_key_to_pack_pos()` helper
  midx: implement `midx_preferred_pack()`
  git-compat-util.h: implement checked size_t to uint32_t conversion
  pack-objects: include number of packs reused in output
  pack-objects: prepare `write_reused_pack_verbatim()` for multi-pack reuse
  pack-objects: prepare `write_reused_pack()` for multi-pack reuse
  pack-objects: pass `bitmapped_pack`'s to pack-reuse functions
  pack-objects: keep track of `pack_start` for each reuse pack
  pack-objects: parameterize pack-reuse routines over a single pack
  pack-bitmap: return multiple packs via `reuse_partial_packfile_from_bitmap()`
  pack-bitmap: simplify `reuse_partial_packfile_from_bitmap()` signature
  ewah: implement `bitmap_is_empty()`
  pack-bitmap: pass `bitmapped_pack` struct to pack-reuse functions
  ...
2024-01-12 16:09:56 -08:00
Junio C Hamano
3e8558438d Merge branch 'jw/builtin-objectmode-attr'
The builtin_objectmode attribute is populated for each path
without adding anything in .gitattributes files, which would be
useful in magic pathspec, e.g., ":(attr:builtin_objectmode=100755)"
to limit to executables.

* jw/builtin-objectmode-attr:
  attr: add builtin objectmode values support
2024-01-12 16:09:55 -08:00
Junio C Hamano
99bb88a6f6 Merge branch 'js/contributor-docs-updates'
Doc update.

* js/contributor-docs-updates:
  SubmittingPatches: hyphenate non-ASCII
  SubmittingPatches: clarify GitHub artifact format
  SubmittingPatches: clarify GitHub visual
  SubmittingPatches: provide tag naming advice
  SubmittingPatches: update extra tags list
  SubmittingPatches: discourage new trailers
  SubmittingPatches: drop ref to "What's in git.git"
  CodingGuidelines: write punctuation marks
  CodingGuidelines: move period inside parentheses
2024-01-12 16:09:55 -08:00
Rubén Justo
bec9bb4b39 branch: make the advice to force-deleting a conditional one
The error message we show when the user tries to delete a not fully
merged branch describes the error and gives a hint to the user:

	error: the branch 'foo' is not fully merged.
	If you are sure you want to delete it, run 'git branch -D foo'.

Let's move the hint part so that it is displayed using the advice
machinery:

	error: the branch 'foo' is not fully merged
	hint: If you are sure you want to delete it, run 'git branch -D foo'
	hint: Disable this message with "git config advice.forceDeleteBranch false"

Signed-off-by: Rubén Justo <rjusto@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-11 17:15:54 -08:00
Rubén Justo
3196029b5b advice: sort the advice related lists
Let's keep the advice related lists sorted to make them more
digestible.

A multi-line comment has also been changed; that produces the unexpected
'insertion != deletion' in this supposedly 'only sort lines' commit.

Signed-off-by: Rubén Justo <rjusto@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-11 17:15:54 -08:00
Britton Leo Kerin
9cce3be2df doc: refer to pathspec instead of path
Signed-off-by: Britton Leo Kerin <britton.kerin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-10 14:32:27 -08:00
Britton Leo Kerin
3dbb69e1f2 doc: use singular form of repeatable path arg
This is more correct because the <path>... doc syntax already indicates
that the arg is "array-type".  It's how other tools do it.  Finally, the
later document text mentions 'path' arguments, while it doesn't mention
'paths'.

Signed-off-by: Britton Leo Kerin <britton.kergin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-10 14:32:27 -08:00
Junio C Hamano
a54a84b333 The sixth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-08 14:05:24 -08:00
Junio C Hamano
9decd56cc9 Merge branch 'ml/doc-merge-updates'
Doc update.

* ml/doc-merge-updates:
  Documentation/git-merge.txt: use backticks for command wrapping
  Documentation/git-merge.txt: fix reference to synopsis
2024-01-08 14:05:15 -08:00
Tamino Bauknecht
39487a1510 fetch: add new config option fetch.all
Introduce a boolean configuration option fetch.all which allows to
fetch all available remotes by default. The config option can be
overridden by explicitly specifying a remote or by using --no-all.
The behavior for --all is unchanged and calling git-fetch with --all
and a remote will still result in an error.

Additionally, describe the configuration variable in the config
documentation and implement new tests to cover the expected behavior.
Also add --no-all to the command-line documentation of git-fetch.

Signed-off-by: Tamino Bauknecht <dev@tb6.eu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-08 13:36:23 -08:00
Rubén Justo
5aea3955bc branch: clarify <oldbranch> term
Since 52d59cc645 (branch: add a --copy (-c) option to go with --move
(-m), 2017-06-18) <oldbranch> is used in more operations than just -m.

Let's also clarify what we do if <oldbranch> is omitted.

Signed-off-by: Rubén Justo <rjusto@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-08 10:06:05 -08:00
Illia Bobyr
25aec06326 rebase: clarify --reschedule-failed-exec default
Documentation should mention the default behavior.

It is better to explain the persistent nature of the
--reschedule-failed-exec flag from the user standpoint, rather than from
the implementation standpoint.

Signed-off-by: Illia Bobyr <illia.bobyr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-05 09:41:25 -08:00
Chandra Pratap
556e68032f write-or-die: make GIT_FLUSH a Boolean environment variable
Among Git's environment variables, the ones marked as "Boolean"
accept values in a way similar to Boolean configuration variables,
i.e. values like 'yes', 'on', 'true' and positive numbers are
taken as "on" and values like 'no', 'off', 'false' are taken as
"off".

GIT_FLUSH can be used to force Git to use non-buffered I/O when
writing to stdout. It can only accept two values, '1' which causes
Git to flush more often and '0' which makes all output buffered.
Make GIT_FLUSH accept more values besides '0' and '1' by turning it
into a Boolean environment variable, modifying the required logic.
Update the related documentation.

Signed-off-by: Chandra Pratap <chandrapratap3519@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-04 10:32:21 -08:00
Maarten van der Schrieck
9cd30af991 Documentation: fix statement about rebase.instructionFormat
Since commit 62db5247 (rebase -i: generate the script via
rebase--helper, 2017-07-14), the short hash is given in
rebase-todo. Specifying rebase.instructionFormat does not alter this
behavior, contrary to what the documentation implies.

Signed-off-by: Maarten van der Schrieck <maarten@thingsconnected.nl>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-03 11:21:15 -08:00
Junio C Hamano
a26002b628 The fifth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-02 13:51:30 -08:00
Junio C Hamano
601b1571e8 Merge branch 'jc/orphan-unborn'
Doc updates to clarify what an "unborn branch" means.

* jc/orphan-unborn:
  orphan/unborn: fix use of 'orphan' in end-user facing messages
  orphan/unborn: add to the glossary and use them consistently
2024-01-02 13:51:30 -08:00
Patrick Steinhardt
5ed860f51b builtin/clone: introduce --ref-format= value flag
Introduce a new `--ref-format` value flag for git-clone(1) that allows
the user to specify the ref format that is to be used for a newly
initialized repository.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-02 09:24:48 -08:00
Patrick Steinhardt
48fa45f5fb builtin/init: introduce --ref-format= value flag
Introduce a new `--ref-format` value flag for git-init(1) that allows
the user to specify the ref format that is to be used for a newly
initialized repository.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-02 09:24:48 -08:00
Patrick Steinhardt
3c4a5318af builtin/rev-parse: introduce --show-ref-format flag
Introduce a new `--show-ref-format` to git-rev-parse(1) that causes it
to print the ref format used by a repository.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-02 09:24:48 -08:00