Arch Linux

W54J04S07T

Member

Registered: 2016-06-27

Posts: 134

New NIC problem [resolved]

I switched from a 1Gb to 10Gb fiber NIC on an old AMD A88X based.server.

Got it back in production and all things pointed to a simple and effective transition
with the NIC showing up at 10Gb/s.

The problem is actual x-fer speeds out of this NIC are around 1.7Gb/s...

My first assumption is that the driver buffers need to be increased by a fair bit
( upwards of 12MB ! ) for a fiber link like this to get moving.  While this is a very simple
thing to do... so I thought...  it isn't at all well documented for ArchLinux distros.

Can anybody advise how to do this on Arch?

Last edited by W54J04S07T (2025-12-18 17:02:36)

-thc

Member

Registered: 2017-03-15

Posts: 1,066

Re: New NIC problem [resolved]

What's your goal here?

Why didn't you mention the card model and/or linux driver?

How exactly did you do the test yielding 1.7 GBit?

W54J04S07T

Member

Registered: 2016-06-27

Posts: 134

Re: New NIC problem [resolved]

What's your goal here?

Why didn't you mention the card model and/or linux driver?

How exactly did you do the test yielding 1.7 GBit?

LOL, my mistake.   The driver is "ixgbe"  and the card is a "gigaplus  X520-2S 10G SFP" card with a
"GESD 10G SFP+ Transceiver - LC MMF300m 10GBase-SR, 850nm MMF"  fiber transceiver. 

As for speed testing...ookla... and a download to another PC.

What I should have made clear, is that I'm trying to find out how to
do the equivalent of the following;

echo 'net.core.wmem_max=12582912' >> /etc/sysctl.conf.

which increases to (in this case ) the size of the write buffer in some distros
and do that in ArchLinux.

Any Ideas?

-thc

Member

Registered: 2017-03-15

Posts: 1,066

Re: New NIC problem [resolved]

What about writing a drop-in file for "/etc/sysctl.d"?

# /etc/sysctl.d/99-wbem_max.conf
net.core.wmem_max=12582912

W54J04S07T

Member

Registered: 2016-06-27

Posts: 134

Re: New NIC problem [resolved]

What about writing a drop-in file for "/etc/sysctl.d"?

# /etc/sysctl.d/99-wbem_max.conf
net.core.wmem_max=12582912

Don't know, so asking...  Can I insert this into 99-sysctl.conf  (currently exists )

Also, for this to work, supposedly I'd need the read and write buffers
expanded...eg.;

net.core.wmem_max=12582912
net.core.rmem_max=12582912

Is that legitimate to insert into 99-sysctl.conf ???

Last edited by W54J04S07T (2025-12-17 19:28:51)

-thc

Member

Registered: 2017-03-15

Posts: 1,066

Re: New NIC problem [resolved]

Strange - in my cases (3 installations) this directory is always empty.

Yes you can append this to an existing "99-sysctl.conf" - but as log as your "99-sysctl.conf" doesn't overwrite those values you can also write something like "50-wbem_max.conf" or so. It's up to you.

W54J04S07T

Member

Registered: 2016-06-27

Posts: 134

Re: New NIC problem [resolved]

Well, the system shows 12MB buffers now, but still no 10Gb/s speed...

I will have to do more investigating.

W54J04S07T

Member

Registered: 2016-06-27

Posts: 134

Re: New NIC problem [resolved]

Problem seems solved. ThankYou for your input.
Sometimes just a few good pointers gets things going.

For the record, the following file

# /etc/sysctl.d/99-sysctl.conf
vm.swappiness=1
net.core.wmem_max=12582912
net.core.rmem_max=12582912
net.ipv4.tcp_wmem= 10240 87380 12582912
net.ipv4.tcp_rmem= 10240 87380 12582912
net.core.netdev_max_backlog = 5000

seems to have done the job with speeds in the 5,600 - 8,000 Mb/S .

I attribute the limitations in those speeds to the speed ( or lack of )
in SATA connected SSDs or NVMe SSDs and a 10year old PCI Gen3 MB.