You are not logged in.

#1 2020-01-24 16:37:21

Nymmie
Member
Registered: 2020-01-22
Posts: 5

Terrible NFS speed/reliability on ZFS pool.

My speed and reliability for network shares over NFS (SMB as well) is terrible. Speed tops out at at about 17-19MB/sec (stalls to 0kb/sec frequently where it stalls out most of the time.) according to windows when dropping files to storage server.

Network Cable Cat6 80ft overall length

Share created via zfs-dkms packages and not traditional way.

System up to date.

'More than a day' is a long time for 1.14TB to transfer on a local network.

Last edited by Nymmie (2020-01-25 18:30:01)

Offline

#2 2020-01-24 17:38:32

seth
Member
Registered: 2012-09-03
Posts: 49,943

Re: Terrible NFS speed/reliability on ZFS pool.

according to windows

?

Share created via zfs-dkms

?

Speed tops out at at about 17-19MB/sec

Sounds like a 100Mbit connection.

You'll have to provide faaaaar more details here - the apparent status quo is "windows says my network is slow, why?"

1. the network layout. What does "windows" (MS windows?) have to do with this? Is arch the server and windows the client or vv. or …?
2. The general connection. ethtool will tell you details on linux.
3. Whether it's actually an NFS thing, eg. if you run an ftp server (eg. vsftpd), can you upload faster on that protocol? What about SMB?
4. The actual NFS specifics (protocol version etcetc)
5. what the ZFS has to do with this. Is it your server FS? Can you upload faster to shares on tmpfs?

The overall distance is irrelevant, the maximum cable length could be, but not at 25m.
Once NFS is indeed determined as issue (and in general), I'd get windows out of the equation (notaby since it seems to index nfs shares whenever you alter them - at least on my last experience I was terribly unimpressed)

Online

#3 2020-01-24 18:09:48

Nymmie
Member
Registered: 2020-01-22
Posts: 5

Re: Terrible NFS speed/reliability on ZFS pool.

according to windows

?

Share created via zfs-dkms

?

Speed tops out at at about 17-19MB/sec

Sounds like a 100Mbit connection.

You'll have to provide faaaaar more details here - the apparent status quo is "windows says my network is slow, why?"

1. the network layout. What does "windows" (MS windows?) have to do with this? Is arch the server and windows the client or vv. or …?

Arch Server / Windows client due to holding Blu-ray Drive

2. The general connection. ethtool will tell you details on linux.

Settings for enp33s0:
        Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                             100baseT/Half 100baseT/Full
                                             1000baseT/Half 1000baseT/Full
        Link partner advertised pause frame use: Symmetric
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: 1000Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: pumbg
        Wake-on: d
        Current message level: 0x00000033 (51)
                               drv probe ifdown ifup
        Link detected: yes

3. Whether it's actually an NFS thing, eg. if you run an ftp server (eg. vsftpd), can you upload faster on that protocol? What about SMB?

No FTP server, SMB same speed and cutout issues

4. The actual NFS specifics (protocol version etcetc)

[root@Templar-Archives ~]# nfsstat
Server rpc stats:
calls      badcalls   badfmt     badauth    badclnt
212790     406        406        0          0

Server nfs v3:
null             getattr          setattr          lookup           access
433       0%     4564      2%     204       0%     10517     4%     124       0%
readlink         read             write            create           mkdir
337       0%     17788     8%     165625   77%     208       0%     43        0%
symlink          mknod            remove           rmdir            rename
0         0%     0         0%     77        0%     9         0%     1         0%
link             readdir          readdirplus      fsstat           fsinfo
0         0%     4366      2%     7750      3%     430       0%     21        0%
pathconf         commit
21        0%     287       0%

5. what the ZFS has to do with this. Is it your server FS? Can you upload faster to shares on tmpfs?

ZFS is /home FS, ZFS created the share 'zfs set sharenfs=on Home/****' opposed to the 'normal' way

The overall distance is irrelevant, the maximum cable length could be, but not at 25m.
Once NFS is indeed determined as issue (and in general), I'd get windows out of the equation (notaby since it seems to index nfs shares whenever you alter them - at least on my last experience I was terribly unimpressed)

Windows is only one with UI So i'd like to keep in my workflow as I also use for gaming

Last edited by Nymmie (2020-01-25 14:05:46)

Offline

#4 2020-01-24 22:42:25

seth
Member
Registered: 2012-09-03
Posts: 49,943

Re: Terrible NFS speed/reliability on ZFS pool.

Please use code tags, not quote tags, for shell IO and file contents (makes it easier to read) - because of the spurious quote tag, you should clean up your previous post anyway (there's an edit link on the lowe right)

due to holding Blu-ray Drive

Does that mean you're uploading from the BR?
In case, try to upload from the HDD

Linux is on 1000MB/full duplex, did you check the windows ethernet status dialog?

No FTP server, SMB same speed and cutout issues

ZFS is /home FS

seth wrote:

Can you upload faster to shares on tmpfs?

You could also just install vdftpd for a test, but since SMB is equally affected, the protocol is probably irrelevant anyway (please edit your initial post and adjust the subject itr.)

Windows is only one with UI So i'd like to keep in my workflow as I also use for gaming

The point is not to give up on windows but to remove the windows system from the equation to isolate the problem.

How fast is your internet connection? > 100MBit?
Can you download faster from the internet to the linux server?

wget -O /dev/null http://speedtest.newark.linode.com/100MB-newark.bin # /dev/null to avoid local FS limits
wget -O ~/testfile.bin http://speedtest.newark.linode.com/100MB-newark.bin # and into your ZFS for comparism

You can use one of "atlanta", "dallas", "frankfurt", "fremont", "london", "mumbai1", "newark", "singapore", "tokyo2", "toronto1" - depending on your location.

Online

#5 2020-01-25 14:02:28

Nymmie
Member
Registered: 2020-01-22
Posts: 5

Re: Terrible NFS speed/reliability on ZFS pool.

Please use code tags, not quote tags, for shell IO and file contents (makes it easier to read) - because of the spurious quote tag, you should clean up your previous post anyway (there's an edit link on the lowe right)

Sure, will do.

Does that mean you're uploading from the BR?
In case, try to upload from the HDD

File Transfer is from HDD not BR

Linux is on 1000MB/full duplex, did you check the windows ethernet status dialog?

Windows shows as Full Duplex 1000MB

How fast is your internet connection? > 100MBit?

Paying for 150mb up/10mb down but get more in the realm of 280mb up/18mb down

Can you download faster from the internet to the linux server?

to /null

[ryan@Templar-Archives Movies]$ wget -O /dev/null http://speedtest.newark.linode.com/100MB-toronto1.bin
Will not apply HSTS. The HSTS database must be a regular and non-world-writable file.
ERROR: could not open HSTS store at '/home/ryan/.wget-hsts'. HSTS will be disabled.
--2020-01-25 08:29:10--  http://speedtest.newark.linode.com/100MB-toronto1.bin
Resolving speedtest.newark.linode.com (speedtest.newark.linode.com)... 2600:3c03::4b, 50.116.57.237
Connecting to speedtest.newark.linode.com (speedtest.newark.linode.com)|2600:3c03::4b|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 104857600 (100M) [application/octet-stream]
Saving to: '/dev/null'

/dev/null                                                                       100%[====================================================================================================================================================================================================>] 100.00M  23.6MB/s    in 4.8s

2020-01-25 08:29:15 (21.0 MB/s) - '/dev/null' saved [104857600/104857600]

to Zpool /home/ryan

[ryan@Templar-Archives Movies]$ wget -O ~/testfile.bin http://speedtest.newark.linode.com/100MB-toronto1.bin
Will not apply HSTS. The HSTS database must be a regular and non-world-writable file.
ERROR: could not open HSTS store at '/home/ryan/.wget-hsts'. HSTS will be disabled.
--2020-01-25 08:43:36--  http://speedtest.newark.linode.com/100MB-toronto1.bin
Resolving speedtest.newark.linode.com (speedtest.newark.linode.com)... 2600:3c03::4b, 50.116.57.237
Connecting to speedtest.newark.linode.com (speedtest.newark.linode.com)|2600:3c03::4b|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 104857600 (100M) [application/octet-stream]
Saving to: '/home/ryan/testfile.bin'

/home/ryan/testfile.bin                                                         100%[====================================================================================================================================================================================================>] 100.00M   566KB/s    in 2m 51s

2020-01-25 08:46:27 (600 KB/s) - '/home/ryan/testfile.bin' saved [104857600/104857600]

ZFS tuning issue? 12 drive raidz2 array ashift=12 all drives identical model

Offline

#6 2020-01-25 15:11:08

seth
Member
Registered: 2012-09-03
Posts: 49,943

Re: Terrible NFS speed/reliability on ZFS pool.

ZFS tuning issue?

I guess it's safe to say that ZFS is the limiting factor here, but I know nothing about that FS (except that Torvalds doesn't like it ;-).
You should alter the subject of your initial post and report it to be moved into the kernel & HW forum.

https://wiki.archlinux.org/index.php/ZFS

To be sure: it would have been http://speedtest.toronto1.linode.com/100MB-toronto1.bin (but it doesn't matter)

Online

#7 2020-01-25 15:35:38

WorMzy
Forum Moderator
From: Scotland
Registered: 2010-06-16
Posts: 11,783
Website

Re: Terrible NFS speed/reliability on ZFS pool.

Mod note: ZFS isn't an Arch-provided package. Moving to AUR Issues.

Please update your topic title to attract the users who know about ZFS.


Sakura:-
Mobo: MSI MAG X570S TORPEDO MAX // Processor: AMD Ryzen 9 5950X @4.9GHz // GFX: AMD Radeon RX 5700 XT // RAM: 32GB (4x 8GB) Corsair DDR4 (@ 3000MHz) // Storage: 1x 3TB HDD, 6x 1TB SSD, 2x 120GB SSD, 1x 275GB M2 SSD

Making lemonade from lemons since 2015.

Online

#8 2020-01-25 15:53:27

progandy
Member
Registered: 2012-05-17
Posts: 5,184

Re: Terrible NFS speed/reliability on ZFS pool.

Since kernel 5.0 ZFS was slowed down due to the removal of some SIMD APIs. The fix for that is available in zfs 0.8.3, but the AUR package zfs-dkms is still on 0.8.2. That might be one of the causes for bad ZFS performance.

https://github.com/zfsonlinux/zfs/commi … 4e17bfee19
https://www.phoronix.com/scan.php?page= … oring-SIMD


| alias CUTF='LANG=en_XX.UTF-8@POSIX ' |

Offline

#9 2020-01-25 18:29:24

Nymmie
Member
Registered: 2020-01-22
Posts: 5

Re: Terrible NFS speed/reliability on ZFS pool.

progandy wrote:

Since kernel 5.0 ZFS was slowed down due to the removal of some SIMD APIs. The fix for that is available in zfs 0.8.3, but the AUR package zfs-dkms is still on 0.8.2. That might be one of the causes for bad ZFS performance.

https://github.com/zfsonlinux/zfs/commi … 4e17bfee19
https://www.phoronix.com/scan.php?page= … oring-SIMD

OK, I'll set aside some time tomorrow to switch to 0.8.3 to test. Unable to do so now.

And topic has now been updated.

Offline

#10 2020-01-28 07:57:34

Nymmie
Member
Registered: 2020-01-22
Posts: 5

Re: Terrible NFS speed/reliability on ZFS pool.

ZFS 0.8.3 did not fix the issue in speed or reliability, though i do get about 2MB/sec faster data transfer to the shares on system.

Offline

Board footer

Powered by FluxBB