Support metalink for updating the centos mirrors

[afbjorklund]

The "metalink" is an XML file with the list of mirrors to use, for the query params:

metalink=https://mirrors.centos.org/metalink?repo=centos-baseos-$stream&arch=$basearch&protocol=https,http

https://fedoraproject.org/wiki/Features/YumMetalinks

Currently each mirror gets a separate entry, which lowers the numbers of hits.

The best would be if they would get automatically rewritten, like the static list...

https://github.com/soulteary/apt-proxy/blob/main/internal/distro/centos.go

[soulteary]

This is a valid enhancement and will remain open.

Metalink support needs more than proxying the XML response: the implementation must preserve query parameters, validate every returned scheme and host, avoid private/link-local/metadata destinations and DNS-rebinding paths, define deterministic mirror-to-cache-key behavior, and test CentOS Stream 9/10 metadata and package downloads. It should be designed alongside the explicit/allowlisted repository mapping work in #57 rather than as arbitrary upstream forwarding.

[soulteary]

Status update after v0.17.0, since this thread referenced #57 as a prerequisite.

The dependency is gone. #57 shipped in v0.17.0: there is now an explicit, allowlist-gated path for fetching a third-party origin (passthrough), with GET/HEAD-only, non-public-address rejection on the allowlist, and no redirect-following. That was the "explicit/allowlisted repository mapping" this issue was waiting to be designed alongside.

Metalink itself is still not implemented, and it is not simply a matter of pointing passthrough at mirrors.centos.org. The remaining work is the part this issue is actually about:

• parse the metalink XML and extract the mirror list, rather than proxying the document through;
• fold those mirrors into the same election path the static list in internal/distro/centos.go already uses, so the elected mirror is the cache key and all clients converge on one entry — which is the "lowers the number of hits" problem you opened with;
• decide refresh semantics: when the list is re-fetched, and how that interacts with SIGHUP / POST /api/mirrors/refresh;
• validate every scheme and host that comes back, and keep the same non-public-address rejection that the allowlist applies;
• test against CentOS Stream 9 and 10 metadata and package downloads.

I have deliberately not estimated the size of the first two points, because I have not been able to fetch a live mirrors.centos.org?metalink=... response to confirm the document's structure, and guessing at it is how a "small change" turns out not to be.

If you have a saved metalink response handy — or can paste a trimmed one for centos-baseos-9-stream — that would remove the main unknown and let this be scoped properly.

Keeping open.