How can we get our CI off the blocklist?

rageltman · 2025-05-07 00:47:42

Hoping this is the right place to ask: we have a CI pipeline which builds some ~2k packages from the AUR and when it fails it resets the specific package pipeline to zero for a full clone. Turns out at some point AUR turned up blocking and we ran afoul of the limiter. Our runs try to clone 100s of repos and sync >1k more and i'm guessing that's not making the auto-blocker very happy.
Is there some way to get off the list or mirror AUR in its entirety to reduce load on the servers from our builds? The source IP WHOIS' back to our org, we've had it for a decade so fairly easy to prove we're not trying to DoS anything but if our usage has the same result then we need to find a better way to build from AUR at this scale.

Lone_Wolf · 2025-05-07 08:28:38

Typically rate limiter bans are for 24 hours .
If they last longer send a message to the aur-general mailing list to contact aur moderators.

Maybe you can use aur json interface to determine which packages have changed and only retrieve those ?

a821 · 2025-05-07 08:55:12

There is a mirror in Github (https://github.com/archlinux/aur), but it stopped syncing some time ago, maybe you could ask to be reestablished?

Edit: I wasn't aware of these issues: 599, 620

Last edited by a821 (2025-05-08 08:36:14)

rageltman · 2025-05-07 11:44:51

@Lone_Wolf - thanks, i saw that instruction in another post and got
```
Your message to the Aur-general mailing-list was rejected for the following
reasons:

The message is not from a list member
```
in response from the list.

The CI pipeline runs every 4h and its been getting blocked non-stop likely because it has a bunch of parallel jobs doing git clones from the same SNAT IP. Is there a way to get whitelisted given that the IP has been ours for a decade and we have no intention of giving it up? Any options for fully mirroring the AUR itself?

Lone_Wolf · 2025-05-07 11:57:42

rageltman wrote:

Your message to the Aur-general mailing-list was rejected for the following
reasons:
The message is not from a list member

You have to subscribe to the list to become a member and be able to send messages to it.

gromit · 2025-05-07 20:36:39

Generally requests like this are best discussed with the DevOps team in the infrastructure repository: https://gitlab.archlinux.org/archlinux/infrastructure/

rageltman · 2025-05-10 10:59:55

Thank you, gitlab issue doesn't seem to be getting any response however.

Is there an individual responsible for the change resulting in these blocks whom i can contact? We need to get our CI fixed and with no clear recourse from upstream our next option is to just bypass IP blocks by setting up a rotating proxy set to vary our srcips... these methods don't work against even lightly resourced attackers but they do very much hamper the efforts of automation. :-(

Last edited by rageltman (2025-05-10 11:00:19)

gromit · 2025-05-10 11:04:06

You created your ticket merely 24 hours ago, this is a volunteer project ... I will reply to your request as soon as I have time for it, except if someone from the other DevOps People beats me to it

Arch Linux

#1 2025-05-07 00:47:42

How can we get our CI off the blocklist?

#2 2025-05-07 08:28:38

Re: How can we get our CI off the blocklist?

#3 2025-05-07 08:55:12

Re: How can we get our CI off the blocklist?

#4 2025-05-07 11:44:51

Re: How can we get our CI off the blocklist?

#5 2025-05-07 11:57:42

Re: How can we get our CI off the blocklist?

#6 2025-05-07 20:36:39

Re: How can we get our CI off the blocklist?

#7 2025-05-10 10:59:55

Re: How can we get our CI off the blocklist?

#8 2025-05-10 11:04:06

Re: How can we get our CI off the blocklist?

Board footer