You are not logged in.

#1 2020-05-01 07:54:45

jjb2016
Member
From: Oxfordshire
Registered: 2016-02-29
Posts: 73

[SOLVED] Limit process network bandwith with systemd / cgroups

Hi all,

I would like to start uploading a full backup of my server to backblaze b2, but this first backup is going to take around 15 days using my full upload bandwidth.  With my wife and i working from home during the day i want to be able to throttle the upload during the day and then set it back to full speed over night, all while the backup process is still running.  I've heard about (but don't know much about) "trickle" and "tc" but I know that resource control can also be done with systemd.  The arch wiki has a good article and example for this:  https://wiki.archlinux.org/index.php/Cg … _systemd_2
This example shows the CPU usage being limited.  I've searched all over the internet, and scanned through the systemd.resource-control man page, but I've not seen any clear information on how network bandwith can be limited.  All examples I've seen show CPU and Memory being limited.  I've seen plenty of articles stating that cgroups can control network bandwith, but no actual how-tos or examples yet.

Anybody got any experience with this?  If anybody out there knows for certain that it CAN'T be done with cgroups, as per the arch wiki page, then can you recomment another way?  I want to be able to adjust the network usage of the backup process while it's running ...

Thanks in advance - Justin

Last edited by jjb2016 (2020-05-10 16:50:52)

Offline

#2 2020-05-03 06:04:34

paulkerry
Member
From: Sheffield, UK
Registered: 2014-10-02
Posts: 462

Re: [SOLVED] Limit process network bandwith with systemd / cgroups

Not sure about the cgroup/trickle/tc options, but you could try using iptables rules using both the "time" ("--timestart"/"--timestop" options) and "hashlimit"/"limit" modules.

man iptables-extensions

and look for the respective "time" and "hashlimit"/"limit" sections.

Offline

#3 2020-05-03 09:29:23

kokoko3k
Member
Registered: 2008-11-14
Posts: 1,978

Re: [SOLVED] Limit process network bandwith with systemd / cgroups

Offline

#4 2020-05-04 11:21:37

jjb2016
Member
From: Oxfordshire
Registered: 2016-02-29
Posts: 73

Re: [SOLVED] Limit process network bandwith with systemd / cgroups

kokoko3k wrote:

Thats an old post with an old solution.  Somebody made a more recent comment there saying that cgroups don't work like that anymore. It seems to be that you have to use cgroups somehow to tag network packets generated by the specific process and then use tc to filter those tagged packets and do some rate limiting on them .... seems straight forward on the surface but the details of exactly how to acheive this seem to be fragmented all over the internet.

Offline

#5 2020-05-04 12:26:22

kokoko3k
Member
Registered: 2008-11-14
Posts: 1,978

Re: [SOLVED] Limit process network bandwith with systemd / cgroups

What exactly are you refferring to?
I posted a google search with multiple 'posts'.

Also, linux doesn't use to break userland.

And now that i think of it, you can limit block device io operations too and achieve the desidered results.

Offline

#6 2020-05-04 13:25:32

jjb2016
Member
From: Oxfordshire
Registered: 2016-02-29
Posts: 73

Re: [SOLVED] Limit process network bandwith with systemd / cgroups

kokoko3k wrote:

What exactly are you refferring to?
I posted a google search with multiple 'posts'.

Also, linux doesn't use to break userland.

And now that i think of it, you can limit block device io operations too and achieve the desidered results.

Apologies - I misread what I saw when I clicked on your link.  I was referring to the first link in that search list.  I've read that one before.

Offline

#7 2020-05-05 18:39:58

jjb2016
Member
From: Oxfordshire
Registered: 2016-02-29
Posts: 73

Re: [SOLVED] Limit process network bandwith with systemd / cgroups

I haven't figured out how to do this with systemd & cgroups yet, but anyway I found a solution.  I'm going to use rclone (rclone.org), which includes a bandwidth limiter.  Actually appears to be a very good cloud sync tool.

Offline

#8 2020-05-10 16:43:21

paulkerry
Member
From: Sheffield, UK
Registered: 2014-10-02
Posts: 462

Re: [SOLVED] Limit process network bandwith with systemd / cgroups

OK. Good to know about rclone as an option.

Remember to prepend "[SOLVED]" to your first post.

Cheers
Paul.

Offline

Board footer

Powered by FluxBB