Skip to content

Latest commit

 

History

History
55 lines (50 loc) · 10.2 KB

README.md

File metadata and controls

55 lines (50 loc) · 10.2 KB

internal/gitlimit

Overview

Shared admission and process control for git subprocesses. It provides a slot pool that bounds concurrent git reads and maintenance, a registry that starts, reaps, and stops git child process groups, and a per-child heap cap on Linux through ulimit -d.

Interface

  • HeavyHeapBytes (256 MiB), ReadSlotCount (8), RepoSlotCount (2), ReadSlotWait (1 s), MaintenanceShare (2) — sizing constants [gitlimit.go:HeavyHeapBytes]
  • MaxArgBytes (131,071), MaxArgListBytes (65,536), ArgListEntryBytes (32), ArgListBytes — the longest argument or environment entry a git child gets, and the budget of a list of request values that git gets as arguments of their own [gitlimit.go:MaxArgBytes] [gitlimit.go:ArgListBytes]
  • ErrBusy, ErrMemoryLimit — no slot in time, and git stopped by its own allocation failure [gitlimit.go:ErrBusy]
  • Slots, NewSlots, (*Slots).Acquire, TryAcquire, Release, InUse — slot pool with a per-key cap [gitlimit.go:Slots]
  • RepoKey — per-repository slot key from a canonical repo ID [gitlimit.go:RepoKey]
  • ReadSlots — process-wide pool shared by JSON git reads and admitted maintenance [gitlimit.go:ReadSlots]
  • TryAdmitMaintenance — non-blocking admission of one maintenance job; returns a release func [gitlimit.go:TryAdmitMaintenance]
  • ErrStopping, Start, Wait, Run, Output, Stopping, StopChildren — tracked git child lifecycle [children.go:Start]
  • LeaseFile, LeaseFD (10), ErrLeaseBusy, OpenLease(ctx, repo), TryExclusive, (*Lease).Attach, Close, StartIn, RunIn, OutputIn, CombinedOutputIn (each takes ctx first) — the shared lease that every git child in a cache repository holds as its descriptor LeaseFD, so that an exclusive lock proves no git that serve started is alive there (#490) [lease.go:LeaseFile] [lease.go:StartIn]
  • NoReplaceObjects — an environment with GIT_NO_REPLACE_OBJECTS=1, for a git child that does not go through Start [children.go:NoReplaceObjects]
  • Name, Clip — a git command line, and one value, for an error message or a log line: a value is cut at 256 bytes and its length is named, and at most 32 arguments are printed [name.go:Name] [name.go:Clip]
  • TailBuffer — the stderr tail that an error quotes: the last 64 KiB, where git prints its fatal message [errtext.go:TailBuffer]
  • HeadBuffer — the output head that a reason quotes: the first 64 KiB, for a command that prints its reason first, with the count of dropped bytes [errtext.go:HeadBuffer]
  • HeapLimitSupported, Command, HeapFailure — heap cap support (Linux: /bin/sh wrapper with ulimit -d; other platforms: no cap) and heap-failure classification [heaplimit_linux.go:Command] [heaplimit_other.go:Command]

Invariants

  • TailBuffer keeps the last 64 KiB written to it and holds at most twice that much memory, also after one large Write; small writes cost time linear in their size; a tail that was cut starts at a UTF-8 character [errtext.go:TailBuffer]
  • HeadBuffer keeps the first 64 KiB written to it and counts the rest in Dropped [errtext.go:HeadBuffer]
  • CombinedOutputIn returns only the last 64 KiB of the child's combined output, kept by a TailBuffer [lease.go:CombinedOutputIn] [lease_test.go:TestCombinedOutputInKeepsTheLast64KiB]
  • NewSlots panics unless n > 0, 0 < perKey <= n, and wait >= 0 [gitlimit.go:NewSlots]
  • Release panics unless the key holds a slot; RepoKey panics on an empty repo ID [gitlimit.go:Release]
  • TryAdmitMaintenance takes the maintenance permit first, releases it if ReadSlots is full, and panics on a second release call [gitlimit.go:TryAdmitMaintenance]
  • Acquire with wait 0 returns ErrBusy at once when no slot is free [gitlimit.go:Acquire]
  • Output panics if the caller already set Stdout [children.go:Output]
  • registry.start panics if SysProcAttr is already set, forces Setpgid, sets GIT_NO_REPLACE_OBJECTS=1 so that git reads every object as itself (any token with git:write can push refs/replace/*), and refuses new children with ErrStopping after stop began [children.go:start]
  • registry.stop sends SIGTERM before SIGKILL so git can clean its lock files [children.go:stop]
  • stop signals only groups it finds registered while it holds the registry lock, and on Linux wait removes a group from the registry before it reaps the leader, so a signal reaches a group whose leader still owns its ID and never one whose ID the kernel recycled (#473) [children.go:wait] [children.go:stop]
  • On Linux a leader that exits after stop began stays an unreaped zombie until stop has sent its final SIGKILL, which still ends a member the leader left behind. awaitExit waits for the exit with waitid(WNOWAIT). A member left behind by a leader that exited before stop began is out of reach: its group left the registry at the exit, and a member that keeps the child's output pipe open keeps Wait blocked until it exits [children.go:wait] [exit_linux.go:awaitExit]
  • Platforms other than Linux use a reap-first wait. macOS has no wait that leaves the child unreaped: Go's os package does not use waitid there because it also returns for a stopped child (golang/go#19314). FreeBSD and NetBSD have wait6, which this package does not use because releases build only linux and darwin. wait reaps first and removes the group right after. stop sends no signal, SIGTERM or SIGKILL, to a group that is not registered or whose leader os reports reaped. A leader that exits during the grace gets no final SIGKILL there, so a member it left behind is not killed. What stays open is the race that os.Process.Signal has on macOS (golang/go#13987): the leader is reaped between the check and the kill [children.go:wait] [children.go:signal] [exit_other.go:awaitExit]
  • A git child that runs in a cache repository starts through StartIn, RunIn, OutputIn or CombinedOutputIn, or attaches a Lease itself, and a plain Start takes no lease. TestEveryGitStartTakesTheLease fails on a git start in the module that does neither and is not listed with a reason, and on a new file that imports os/exec until the file is listed in execImporters. TestAuditSeesEveryWayToBuildGit proves each way it recognizes to build git: exec.Command, an exec.Cmd literal, exec.Command held as a value, any program that is not in nonGitPrograms (a launcher can start git), os.StartProcess and syscall.Exec, a dot import, and a package-level initializer. TestAuditChecksTheAllowLists pins the stale-entry and unlisted-file checks. A binary built with the openstorage_viewcheck tag panics on a plain start that has a Dir or an argument that is a repository path, and on a leased start that runs in another repository [lease_audit_test.go:TestEveryGitStartTakesTheLease] [lease_audit_test.go:TestAuditSeesEveryWayToBuildGit] [lease_audit_test.go:TestAuditChecksTheAllowLists] [leasecheck_viewcheck.go:init]
  • OpenLease creates LeaseFile when it is missing and never removes it. It waits up to 30 s while the lease is held exclusively, then fails with ErrLeaseBusy. The wait ends at once with ctx's error when ctx ends, and with ErrStopping once StopChildren began. TryExclusive never waits [lease.go:OpenLease] [lease.go:TryExclusive] [lease_test.go:TestOpenLeaseStopsWaitingWhenChildrenAreStopping] [lease_test.go:TestOpenLeaseGivesUpAfterTheLeaseWait]
  • Leases are shared: any number coexist, and Attach panics on an exclusive one. The file has mode 0600 [lease.go:Attach] [lease_test.go:TestSharedLeasesCoexist] [lease_test.go:TestAttachRefusesAnExclusiveLease] [lease_test.go:TestLeaseFileIsPrivate]
  • Only a read-only filesystem (EROFS) changes OpenLease: nothing can write a ref lock there, so reads must still run. An existing lease file is opened read-only and locked, which flock allows, so the lease is a real one. With no lease file, the lease holds nothing and Attach adds no descriptor. Any other failure is an error, a permission error included: a directory that serve cannot write can still have a writable refs/, where a git without the lease could take ref locks [lease.go:OpenLease] [lease_test.go:TestReadOnlyFilesystemWithALeaseFileHoldsARealLease] [lease_test.go:TestReadOnlyFilesystemWithoutALeaseFileStillRuns] [lease_test.go:TestReadOnlyFilesystemWithAnUnreadableLeaseFileFails] [lease_test.go:TestOpenLeaseFailsClosedOutsideAReadOnlyFilesystem]
  • A start that fails before the child runs, because the lease cannot be taken or StopChildren began, closes the pipes that the caller made with StdinPipe and StdoutPipe [children.go:failStart] [lease_test.go:TestStartInClosesThePipesOfAStartThatFails]
  • The lock lives as long as the child's process tree: the parent closes its copy after Start, and the child, its hooks and their children keep descriptor LeaseFD. Descriptors 3 to 9 are closed in the child, because Git writes GIT_TRACE=3 to descriptor 3 [lease.go:Attach] [lease_test.go:TestGitTraceToADescriptorBelowTenNeverReachesTheLease]. A start that fails releases it at once [lease.go:StartIn] [lease_test.go:TestLeaseLivesWithTheGitProcessTree] [lease_test.go:TestFailedStartReleasesTheLease]
  • The Linux wrapper keeps a stricter inherited soft limit and exits 125 if it cannot read or set the limit [heaplimit_linux.go:heapLimitWrapper]
  • HeapFailure ignores SIGKILL and matches git diagnostics only at line start [heaplimit_linux.go:HeapFailure]
  • Non-Linux Command asserts maxHeapBytes == 0 [heaplimit_other.go:Command]

Side Effects

  • Starts git children (through /bin/sh when a heap cap is set), each as its own process group [children.go:start]
  • Sends SIGTERM then SIGKILL to tracked groups in StopChildren, polling every 10 ms until every leader has exited or the grace deadline [children.go:stop]
  • Blocks one goroutine per child until its leader exits: in waitid(WNOWAIT) before it reaps on Linux, in Wait elsewhere [exit_linux.go:awaitExit] [exit_other.go:awaitExit]
  • Holds mutexes on slot state and on the child registry; release closes and replaces the freed channel to wake waiters [gitlimit.go:Release]
  • Output buffers child output in memory [children.go:Output]
  • OpenLease and TryExclusive create and lock <repo>/codestorage.gitlease.lock [lease.go:OpenLease]

Verification

  • Run: go test ./internal/gitlimit/... -tags=openstorage_viewcheck -count=1 -race -timeout=30m