If a remote repo has too many tags (or branches), cloning it over the
smart HTTP transport can fail because remote-curl.c puts all the refs
from the remote repo on the fetch-pack command line. This can make the
command line longer than the global OS command line limit, causing
fetch-pack to fail.
This is especially a problem on Windows where the command line limit is
orders of magnitude shorter than Linux. There are already real repos out
there that msysGit cannot clone over smart HTTP due to this problem.
Here is an easy way to trigger this problem:
git init too-many-refs
cd too-many-refs
echo bla > bla.txt
git add .
git commit -m test
sha=$(git rev-parse HEAD)
tag=$(perl -e 'print "bla" x 30')
for i in `seq 50000`; do
echo $sha refs/tags/$tag-$i >> .git/packed-refs
done
Then share this repo over the smart HTTP protocol and try cloning it:
$ git clone http://localhost/.../too-many-refs/.git
Cloning into 'too-many-refs'...
fatal: cannot exec 'fetch-pack': Argument list too long
50k tags is obviously an absurd number, but it is required to
demonstrate the problem on Linux because it has a much more generous
command line limit. On Windows the clone fails with as little as 500
tags in the above loop, which is getting uncomfortably close to the
number of tags you might see in real long lived repos.
This is not just theoretical, msysGit is already failing to clone our
company repo due to this. It's a large repo converted from CVS, nearly
10 years of history.
Four possible solutions were discussed on the Git mailing list (in no
particular order):
1) Call fetch-pack multiple times with smaller batches of refs.
This was dismissed as inefficient and inelegant.
2) Add option --refs-fd=$n to pass a an fd from where to read the refs.
This was rejected because inheriting descriptors other than
stdin/stdout/stderr through exec() is apparently problematic on Windows,
plus it would require changes to the run-command API to open extra
pipes.
3) Add option --refs-from=$tmpfile to pass the refs using a temp file.
This was not favored because of the temp file requirement.
4) Add option --stdin to pass the refs on stdin, one per line.
In the end this option was chosen as the most efficient and most
desirable from scripting perspective.
There was however a small complication when using stdin to pass refs to
fetch-pack. The --stateless-rpc option to fetch-pack also uses stdin for
communication with the remote server.
If we are going to sneak refs on stdin line by line, it would have to be
done very carefully in the presence of --stateless-rpc, because when
reading refs line by line we might read ahead too much data into our
buffer and eat some of the remote protocol data which is also coming on
stdin.
One way to solve this would be to refactor get_remote_heads() in
fetch-pack.c to accept a residual buffer from our stdin line parsing
above, but this function is used in several places so other callers
would be burdened by this residual buffer interface even when most of
them don't need it.
In the end we settled on the following solution:
If --stdin is specified without --stateless-rpc, fetch-pack would read
the refs from stdin one per line, in a script friendly format.
However if --stdin is specified together with --stateless-rpc,
fetch-pack would read the refs from stdin in packetized format
(pkt-line) with a flush packet terminating the list of refs. This way we
can read the exact number of bytes that we need from stdin, and then
get_remote_heads() can continue reading from the same fd without losing
a single byte of remote protocol data.
This way the --stdin option only loses generality and scriptability when
used together with --stateless-rpc, which is not easily scriptable
anyway because it also uses pkt-line when talking to the remote server.
Signed-off-by: Ivan Todoroski <grnch@gmx.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It's actually unlimited recursion if wildcards are active regardless
--max-depth
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Two "^" characters were incorrectly being interpreted as markup for
superscripting. Fix them by writing them as attribute references
"{caret}".
Although a single "^" character in a paragraph cannot be
misinterpreted in this way, also write other "^" characters as
"{caret}" in the interest of good hygiene (unless they are in literal
paragraphs, of course, in which context attribute references are not
recognized).
Spell "{}" consistently, namely *not* quoted as "\{\}". Since the
braces are empty, they cannot be interpreted as an attribute
reference, and either spelling is OK. So arbitrarily choose one
variation and use it consistently.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
Update draft release notes to 1.7.8.4
Update draft release notes to 1.7.7.6
Update draft release notes to 1.7.6.6
thin-pack: try harder to use preferred base objects as base
Since its very first description of -k, the documentation for
git-mailinfo claimed that (in the case without -k) after cleaning up
bracketed strings [blah], it would insert [PATCH].
It doesn't; on the contrary, one of the important jobs of mailinfo is
to remove those strings.
Since we're already there, rewrite the paragraph to give a complete
enumeration of all the transformations. Specifically, it was missing
the whitespace normalization (run of isspace(c) -> ' ') and the
removal of leading ':'.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The description of rerere.enabled left the user in the dark as to who
might create an rr-cache directory. Add a note that simply invoking
rerere does this.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jc/show-sig:
log --show-signature: reword the common two-head merge case
log-tree: show mergetag in log --show-signature output
log-tree.c: small refactor in show_signature()
commit --amend -S: strip existing gpgsig headers
verify_signed_buffer: fix stale comment
gpg-interface: allow use of a custom GPG binary
pretty: %G[?GS] placeholders
test "commit -S" and "log --show-signature"
log: --show-signature
commit: teach --gpg-sign option
Conflicts:
builtin/commit-tree.c
builtin/commit.c
builtin/merge.c
notes-cache.c
pretty.c
The wording seems to suggest that creating the directory is needed and the
setting of rerere.enabled is only for disabling the feature by setting it
to 'false'. But the configuration is meant to be the primary control and
setting it to 'true' will enable it; the rr-cache directory will be
created as necessary and the user does not have to create it.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The FETCH_HEAD refname is supposed to refer to the ref that was fetched
and should be merged. However all fetched refs are written to
.git/FETCH_HEAD in an arbitrary order, and resolve_ref_unsafe simply
takes the first ref as the FETCH_HEAD, which is often the wrong one,
when other branches were also fetched.
The solution is to write the for-merge ref(s) to FETCH_HEAD first.
Then, unless --append is used, the FETCH_HEAD refname behaves as intended.
If the user uses --append, they presumably are doing so in order to
preserve the old FETCH_HEAD.
While we are at it, update an old example in the read-tree documentation
that implied that each entry in FETCH_HEAD only has the object name, which
is not true for quite a while.
[jc: adjusted tests]
Signed-off-by: Joey Hess <joey@kitenet.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* pw/p4-docs-and-tests:
git-p4: document and test submit options
git-p4: test and document --use-client-spec
git-p4: test --keep-path
git-p4: test --max-changes
git-p4: document and test --import-local
git-p4: honor --changesfile option and test
git-p4: document and test clone --branch
git-p4: test cloning with two dirs, clarify doc
git-p4: clone does not use --git-dir
git-p4: introduce asciidoc documentation
rename git-p4 tests
* maint:
docs: describe behavior of relative submodule URLs
fix hang in git fetch if pointed at a 0 length bundle
Documentation: read-tree --prefix works with existing subtrees
Add MYMETA.json to perl/.gitignore
* maint-1.7.7:
docs: describe behavior of relative submodule URLs
Documentation: read-tree --prefix works with existing subtrees
Add MYMETA.json to perl/.gitignore
Since the relative submodule URLs have been introduced in f31a522a2d, they
do not conform to the rules for resolving relative URIs but rather to
those of relative directories.
Document that behavior.
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since 34110cd4 (Make 'unpack_trees()' have a separate source and
destination index) it is no longer true that a subdirectory with
the same prefix must not exist.
Signed-off-by: Clemens Buchacher <drizzd@aon.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
