admin/git
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
45,416 Commits 1 Branch 0 Tags
5da4966f2808371ba94de467d06b4e7e7ab125d8
Commit Graph

45416 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Junio C Hamano
7419a03fdb Merge branch 'jk/http-backend-deadlock-2.2' into jk/http-backend-deadlock-2.3 
* jk/http-backend-deadlock-2.2:
  http-backend: spool ref negotiation requests to buffer
  t5551: factor out tag creation
  http-backend: fix die recursion with custom handler
 2015-05-25 20:44:04 -07:00
Jeff King
6bc0cb5176 http-backend: spool ref negotiation requests to buffer 
When http-backend spawns "upload-pack" to do ref
negotiation, it streams the http request body to
upload-pack, who then streams the http response back to the
client as it reads. In theory, git can go full-duplex; the
client can consume our response while it is still sending
the request.  In practice, however, HTTP is a half-duplex
protocol. Even if our client is ready to read and write
simultaneously, we may have other HTTP infrastructure in the
way, including the webserver that spawns our CGI, or any
intermediate proxies.

In at least one documented case[1], this leads to deadlock
when trying a fetch over http. What happens is basically:

  1. Apache proxies the request to the CGI, http-backend.

  2. http-backend gzip-inflates the data and sends
     the result to upload-pack.

  3. upload-pack acts on the data and generates output over
     the pipe back to Apache. Apache isn't reading because
     it's busy writing (step 1).

This works fine most of the time, because the upload-pack
output ends up in a system pipe buffer, and Apache reads
it as soon as it finishes writing. But if both the request
and the response exceed the system pipe buffer size, then we
deadlock (Apache blocks writing to http-backend,
http-backend blocks writing to upload-pack, and upload-pack
blocks writing to Apache).

We need to break the deadlock by spooling either the input
or the output. In this case, it's ideal to spool the input,
because Apache does not start reading either stdout _or_
stderr until we have consumed all of the input. So until we
do so, we cannot even get an error message out to the
client.

The solution is fairly straight-forward: we read the request
body into an in-memory buffer in http-backend, freeing up
Apache, and then feed the data ourselves to upload-pack. But
there are a few important things to note:

  1. We limit the in-memory buffer to prevent an obvious
     denial-of-service attack. This is a new hard limit on
     requests, but it's unlikely to come into play. The
     default value is 10MB, which covers even the ridiculous
     100,000-ref negotation in the included test (that
     actually caps out just over 5MB). But it's configurable
     on the off chance that you don't mind spending some
     extra memory to make even ridiculous requests work.

  2. We must take care only to buffer when we have to. For
     pushes, the incoming packfile may be of arbitrary
     size, and we should connect the input directly to
     receive-pack. There's no deadlock problem here, though,
     because we do not produce any output until the whole
     packfile has been read.

     For upload-pack's initial ref advertisement, we
     similarly do not need to buffer. Even though we may
     generate a lot of output, there is no request body at
     all (i.e., it is a GET, not a POST).

[1] http://article.gmane.org/gmane.comp.version-control.git/269020

Test-adapted-from: Dennis Kaarsemaker <dennis@kaarsemaker.net>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 20:43:18 -07:00
Michael Haggerty
5cb901a4b0 struct ref_lock: convert old_sha1 member to object_id 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:40 -07:00
Michael Haggerty
4e675d1732 warn_if_dangling_symref(): convert local variable "junk" to object_id 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:39 -07:00
Michael Haggerty
0a0c953217 each_ref_fn_adapter(): remove adapter 
All of the callers of the for_each_ref family of functions have now
been rewritten to work with object_ids, so this adapter is no longer
needed.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:39 -07:00
Michael Haggerty
c38cd1c89d rev_list_insert_ref(): remove unneeded arguments 
Now that the function is not being used as an each_ref_sha1_fn, we can
delete the unused arguments in its signature.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:39 -07:00
Michael Haggerty
b1b49c6eb6 rev_list_insert_ref_oid(): new function, taking an object_oid 
This function can be used with for_each_ref() without having to be
wrapped.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:39 -07:00
Michael Haggerty
6e20a51a80 mark_complete(): remove unneeded arguments 
Now that the function is not being used as an each_ref_sha1_fn, we can
delete the unused arguments in its signature.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:38 -07:00
Michael Haggerty
f8ee4d8522 mark_complete_oid(): new function, taking an object_oid 
This function can be used with for_each_ref() without having to be
wrapped.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:38 -07:00
Michael Haggerty
c50fb6cee6 clear_marks(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:38 -07:00
Michael Haggerty
b4ebaf9eea mark_complete(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:37 -07:00
Michael Haggerty
21758affae send_ref(): convert local variable "peeled" to object_id 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:37 -07:00
Michael Haggerty
363e98bfc2 upload-pack: rewrite functions to take object_id arguments 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:37 -07:00
Michael Haggerty
e45a4949a2 find_symref(): convert local variable "unused" to object_id 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:37 -07:00
Michael Haggerty
7dabd05634 find_symref(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:37 -07:00
Michael Haggerty
1700cb3b05 write_one_ref(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:36 -07:00
Michael Haggerty
f31ba7e116 write_refs_to_temp_dir(): convert local variable sha1 to object_id 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:36 -07:00
Michael Haggerty
7290ef5898 submodule: rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:36 -07:00
Michael Haggerty
580b04ef98 shallow: rewrite functions to take object_id arguments 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:36 -07:00
Michael Haggerty
9c5fe0b846 handle_one_ref(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:35 -07:00
Michael Haggerty
e2b0bcdf4a add_info_ref(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:35 -07:00
Michael Haggerty
a89caf4bd4 handle_one_reflog(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:35 -07:00
Michael Haggerty
00530834fb register_replace_ref(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:35 -07:00
Michael Haggerty
455ade6598 remote: rewrite functions to take object_id arguments 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:34 -07:00
Michael Haggerty
635170f2bb add_one_ref(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:34 -07:00
Michael Haggerty
fd95035fdb string_list_add_one_ref(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:34 -07:00
Michael Haggerty
3d79f65735 add_ref_decoration(): convert local variable original_sha1 to object_id 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:34 -07:00
Michael Haggerty
f124b73023 add_ref_decoration(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:33 -07:00
Michael Haggerty
5f9cf5abd2 show_head_ref(): convert local variable "unused" to object_id 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:33 -07:00
Michael Haggerty
f72f542107 http-backend: rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:33 -07:00
Michael Haggerty
91d6e94ea6 append_similar_ref(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:33 -07:00
Michael Haggerty
6c4461e8d9 builtin/show-ref: rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:33 -07:00
Michael Haggerty
a0cde90ebf show_ref(): convert local variable peeled to object_id 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:32 -07:00
Michael Haggerty
f0a011fa1f builtin/show-ref: rewrite to use object_id 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:32 -07:00
Michael Haggerty
635b99a0c7 fsck: change functions to use object_id 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:32 -07:00
Michael Haggerty
96062b5762 cmd_show_branch(): fix error message 
We need to convert the SHA-1 to hexadecimal before printing it.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:31 -07:00
Michael Haggerty
d1516bf462 builtin/show-branch: rewrite functions to work with object_id 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:31 -07:00
Michael Haggerty
7a456c1eea append_one_rev(): rewrite to work with object_id 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:31 -07:00
Michael Haggerty
2e253a4a12 builtin/show-branch: rewrite functions to take object_id arguments 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:31 -07:00
Michael Haggerty
a00595fbd2 append_matching_ref(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:31 -07:00
Michael Haggerty
d70d7a8a4d show_reference(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:30 -07:00
Michael Haggerty
53dc95b5cf builtin/remote: rewrite functions to take object_id arguments 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:30 -07:00
Michael Haggerty
e26cdf91c1 add_branch_for_removal(): don't set "util" field of string_list entries 
They were never used.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:30 -07:00
Michael Haggerty
45690a57a3 add_branch_for_removal(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:30 -07:00
Michael Haggerty
5bcad1bce2 builtin/reflog: rewrite ref functions to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:30 -07:00
Michael Haggerty
ce2a987329 show_ref_cb(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:29 -07:00
Michael Haggerty
d155254c73 builtin/pack-objects: rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:29 -07:00
Michael Haggerty
73868486f0 name_ref(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:29 -07:00
Michael Haggerty
30a3fd4050 grab_single_ref(): rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:29 -07:00
Michael Haggerty
0e0b7de4c7 builtin/fetch: rewrite to take an object_id argument 
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
 2015-05-25 12:19:29 -07:00