README.md raw

git-lfs-http

A read-only Git LFS HTTP endpoint for git.smesh.lol.

Why it exists

gitweb hands every /<repo>.git/* request to git-http-backend, which understands info/refs, git-upload-pack and git-receive-pack and nothing else. /info/lfs/objects/batch therefore fell through to an empty 404, so an HTTPS client could fetch refs and pack objects but never the LFS content — and the only way to get LFS was the pure-SSH transfer agent over port 2222.

This service answers the two paths git-lfs needs and nothing else:

requestresponse
POST /<repo>.git/info/lfs/objects/batchthe batch JSON: one entry per requested object, with a download action and href when the object is on disk, or a 404 error entry when it is not
GET /<repo>.git/info/lfs/objects/<oid>the object bytes
POST /<repo>.git/info/lfs/objects/<oid>/verify{}

Object store

It reads the tree the SSH side writes, so nothing is copied between them:

/home/git/<name>.git/lfs/objects/<xx>/<yy>/<oid>

That is the layout git-lfs-transfer (the git-shell command on port 2222) creates and reads. An object pushed over SSH is immediately downloadable over HTTPS, and vice versa is not true — see below.

Read-only, on purpose

Uploads are refused with 403. No repository on this host sets http.receivepack, so HTTPS is fetch-only: allowing LFS writes over HTTPS would hand out the one way to write to a repository that the git side deliberately does not have, and LFS objects are exactly the large ones an abusive client would want to push.

Pushing LFS content therefore goes over SSH, which is where the key in /home/git/.ssh/authorized_keys (and the GIT_USER it sets) is what authorises it. A repo that pushes LFS over SSH needs

git config lfs.puresshtransfer true

because git-lfs would otherwise try this HTTP endpoint first and be told 403.

Build

Written for the current compiler (the one in ~/moxie/git.smesh.lol/moxie); it uses no net/net/http because a stdlib listener does not come up in moxie, so the accept loop is raw syscalls — the same approach gitweb takes.

cd lfsd
MOXIEROOT=~/moxie/git.smesh.lol/moxie \
  ~/moxie/git.smesh.lol/moxie/tmp/mxc-init build -o git-lfs-http .

It takes -repos <dir> (default /home/git), -listen <addr> (default 127.0.0.1:3100) and -host <url> (the base for the hrefs it hands out, default https://git.smesh.lol).

Deploy

The binary lives at /usr/local/bin/git-lfs-http and runs as gitweb-lfs.service (user www-data, which can read the object store):

ExecStart=/usr/local/bin/git-lfs-http -repos /home/git -listen 127.0.0.1:3100 -host https://git.smesh.lol

Caddy routes only the LFS paths to it, in the git.smesh.lol block:

@lfs path */info/lfs/*
handle @lfs {
	reverse_proxy 127.0.0.1:3100
}
handle {
	reverse_proxy 127.0.0.1:3000
}

A checkout with content on disk is the whole test:

git clone https://git.smesh.lol/transdb.git   # smudges the corpora
git lfs pull --include="corpora/tatoeba-bg-en/*"
sha256sum corpora/tatoeba-bg-en/Tatoeba.bg-en.en   # must equal the pointer's oid