You are not logged in.

#76 2011-12-25 00:26:39

steve_v
Member
Registered: 2006-02-11
Posts: 80

Re: system hangs when copying data to usb device

Hrmm, I for one like a usable keyboard @ bootup, besides this doesn't appear to improve matters here.

Just to clarify, can someone run a 'dd if=/dev/zero of=/somewhere/on/usb/drive -bs=1M -count=500'
For me this does NOT cause a problem.
Nor does copying said file back & forth from the usb stick to a sata device.

However, accessing that usb drive with dolphin while the write is in progress hangs up the whole desktop...
Accessing the drive from a console (ls, cat, mc etc.) is fine.

Anyone care to confirm, or am i dealing with yet annother different issue...

Steve.

Offline

#77 2011-12-25 01:24:31

lilsirecho
Veteran
Registered: 2003-10-24
Posts: 5,000

Re: system hangs when copying data to usb device

FWIW:
Perhaps this occurs with USB flash devices which are connected via USB 1.1 reader/writers.

Also, some USB devices reader devices are usb2.0 and are  compatible with true IDE flash devices and may not respond well to devices which are not true IDE and thereby cause the supposed i/o problem.EDIT: I comment on CF devices with this parameter.  I do not know of a similar designator for SD devices.  Perhaps trying CF devices could provide better results?

In my system, true IDE devices can be booted but those which are not true IDE will fail to boot.

Perhaps this isn't relevant for your varied performance problems.

Last edited by lilsirecho (2011-12-25 02:00:03)


Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

Offline

#78 2011-12-27 11:30:53

stqn
Member
Registered: 2010-03-19
Posts: 1,191
Website

Re: system hangs when copying data to usb device

For me disabling transparent huge pages helped (no more freeze, faster copy to usb flash drive.)
https://bbs.archlinux.org/viewtopic.php … 9#p1032819

Offline

#79 2011-12-27 19:11:31

finidine
Member
Registered: 2010-05-24
Posts: 24

Re: system hangs when copying data to usb device

Now trying with this:

echo never >/sys/kernel/mm/transparent_hugepage/defrag
echo 0 > /sys/kernel/mm/transparent_hugepage/khugepaged/defrag

And yes, seems to help a lot. Multiple and very long USB transfers, system keeps responsive.

Offline

#80 2011-12-27 22:53:51

Draje
Member
Registered: 2007-06-21
Posts: 102
Website

Re: system hangs when copying data to usb device

Disabling THP also seems to solve my problem. I no longer see other applications entering D-state when running a file copy.

Offline

#81 2011-12-27 23:28:58

firecat53
Member
From: Lake Stevens, WA, USA
Registered: 2007-05-14
Posts: 1,542
Website

Re: system hangs when copying data to usb device

So, if we disable this hugepage option, what are the possible ramifications for other areas of the system?

Scott

Offline

#82 2011-12-28 10:10:49

Draje
Member
Registered: 2007-06-21
Posts: 102
Website

Re: system hangs when copying data to usb device

firecat53 wrote:

So, if we disable this hugepage option, what are the possible ramifications for other areas of the system?

Scott

Actually, hugepages do not need to be disabled, but only the defragmentation of them. Adding the following to the rc.local will set it on boot.

echo never > /sys/kernel/mm/transparent_hugepage/defrag

Apart from fixing the file copy issue, I'm not sure how else the system is affected. I've noticed only benefits from disabling the defragmentation.

Offline

#83 2011-12-28 11:45:54

unikum
Member
From: Russia
Registered: 2010-09-04
Posts: 151
Website

Re: system hangs when copying data to usb device

Draje, can I take a same effect with sysctl.conf?

Offline

#84 2011-12-28 13:44:47

finidine
Member
Registered: 2010-05-24
Posts: 24

Re: system hangs when copying data to usb device

In fact,

echo madvise > /sys/kernel/mm/transparent_hugepage/defrag

may be enough (to not disable hugepage defrag completely).

Offline

#85 2011-12-29 07:24:23

kokoko3k
Member
Registered: 2008-11-14
Posts: 2,390

Re: system hangs when copying data to usb device

Draje wrote:
firecat53 wrote:

So, if we disable this hugepage option, what are the possible ramifications for other areas of the system?

Scott

Actually, hugepages do not need to be disabled, but only the defragmentation of them. Adding the following to the rc.local will set it on boot.

echo never > /sys/kernel/mm/transparent_hugepage/defrag

Apart from fixing the file copy issue, I'm not sure how else the system is affected. I've noticed only benefits from disabling the defragmentation.

Well, yesterday my netbook refused to enter hibernation state due to segfaults, it worked since months. (and hugepaged defrag appeared in the backtrace)
Re-enabling the defragmentation solved it, now i enable it just before hibernation and disable at wake up.

EDIT
somebody tested madvise instead of never?

Last edited by kokoko3k (2011-12-29 07:26:17)


Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !

Offline

#86 2011-12-29 11:39:32

finidine
Member
Registered: 2010-05-24
Posts: 24

Re: system hangs when copying data to usb device

kokoko3k wrote:

EDIT
somebody tested madvise instead of never?

I'm testing it. Good responsiveness, no crashes / any other issues.

Just for reference, the Ubuntu Oneiric (current) kernel has this default configuration:

transparent_hugepage: madvise
hugepage_defrag: always
khugepaged: enabled (1)

Currently I'm testing this:
transparent_hugepage: always
hugepage_defrag: madvise
khugepaged: enabled (1)

Which, to me, is less restrictive.

Offline

#87 2011-12-29 19:42:00

stqn
Member
Registered: 2010-03-19
Posts: 1,191
Website

Re: system hangs when copying data to usb device

unikum wrote:

Draje, can I take a same effect with sysctl.conf?

sysctl's man page states that it can only affect /proc/sys settings, and indeed when I tried to change /sys/kernel/mm/transparent_hugepage/enabled with sysctl.conf, it didn't work.

Offline

#88 2012-01-02 23:22:50

finidine
Member
Registered: 2010-05-24
Posts: 24

Re: system hangs when copying data to usb device

This is a good reading on the subject:

http://lwn.net/Articles/467328/

Also, note that the default value of /sys/kernel/mm/transparent_hugepage/enabled can be defined at kernel compile time.

Offline

#89 2012-01-15 16:12:52

pyther
Member
Registered: 2008-01-21
Posts: 1,395
Website

Re: system hangs when copying data to usb device

Thank you so much!!! Using madvise and things are working so much better!


Website - Blog - arch-home
Arch User since March 2005

Offline

#90 2012-01-15 17:35:18

kokoko3k
Member
Registered: 2008-11-14
Posts: 2,390

Re: system hangs when copying data to usb device

Any hope this will be the default in arch stock kernel?


Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !

Offline

#91 2012-01-15 17:48:41

karol
Archivist
Registered: 2009-05-06
Posts: 25,440

Re: system hangs when copying data to usb device

I think you can open a feature request.

Offline

#92 2012-01-16 07:37:47

kokoko3k
Member
Registered: 2008-11-14
Posts: 2,390

Re: system hangs when copying data to usb device


Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !

Offline

#93 2012-01-31 01:33:59

toofishes
Developer
From: Chicago, IL
Registered: 2006-06-06
Posts: 602
Website

Re: system hangs when copying data to usb device

We don't patch or change the defaults of the kernel except in very rare cases, and this is not one of them. This bug needs to get resolved upstream, so make sure it is reported on the right channels there.

Offline

#94 2012-01-31 01:44:51

pyther
Member
Registered: 2008-01-21
Posts: 1,395
Website

Re: system hangs when copying data to usb device

There have been some significant changes made to the memory compaction code to avoid the lengthy stalls experienced by some users when writing data to slow devices (USB keys, for example). This problem was described in this article, but the solution has evolved considerably. By making a number of changes to how compaction works, the memory management hackers (and Mel Gorman in particular) were able to avoid disabling synchronous compaction, which had the unfortunate effect of reducing huge page usage. See this commit for a lot of information on how this problem was addressed.

https://lwn.net/Articles/475405/

So the problem might be much improved in 3.3


Website - Blog - arch-home
Arch User since March 2005

Offline

#95 2012-02-09 23:40:53

bwat47
Member
Registered: 2009-10-07
Posts: 638

Re: system hangs when copying data to usb device

I was having this same problem, either cropped up after a recent kernel upgrade or the fact that I switched to kde. copying a ~1.5 gig file to a flash drive caused kde to totally freeze near the end of the transfer for a good 30 seconds. using madvise in rc.local as suggested here fixed it.

Offline

#96 2012-02-13 05:02:08

XD_dued
Member
Registered: 2012-02-13
Posts: 4

Re: system hangs when copying data to usb device

madvise helped a a lot, made the occurances much less and of less duration.

Unfortunately, it didn't completely eliminate the problem.  I will try the never option I guess.

I'm using x86_64 with kde.

Last edited by XD_dued (2012-02-13 05:03:57)

Offline

#97 2012-02-14 23:56:16

XD_dued
Member
Registered: 2012-02-13
Posts: 4

Re: system hangs when copying data to usb device

For me to almost completely fix the issue, I had to disable legacy usb support in bios and use madvise in the rc.local.

Unfortunately, the problem still happened, but only when I forced it to happen (opened libreoffice and firefox at the same time).  And usb transfer is still very slow.

How big would be the disadvantage to try to disable the hugepage defragmentation?

How about to completely disable hugepages?

Offline

#98 2012-02-15 10:01:55

sxe
Member
Registered: 2009-06-04
Posts: 103

Re: system hangs when copying data to usb device

XD_dued wrote:

How big would be the disadvantage to try to disable the hugepage defragmentation?

I figured no disadvantage by doing so. I have disabled it for some month now, works great and no lag at all anymore.

Offline

#99 2012-02-16 23:06:21

achilleas.k
Member
From: Sol 3
Registered: 2011-10-30
Posts: 117

Re: system hangs when copying data to usb device

Disabling hugepage defrag helps. I wont say it fixes the situation because there's still some intermittent freezes of 2-3 seconds. They are especially noticeable in Firefox. But overall it's at least usable now.
Will try with madvise as well.

Offline

#100 2012-02-17 12:22:56

finidine
Member
Registered: 2010-05-24
Posts: 24

Re: system hangs when copying data to usb device

We have to distinguish between disabling transparent hugepages and disabling hugepage defragmentation.

If you disable transparent hugepages completely, that will have some side effects. For example, hibernation will not work.

"madvise" for transparent hugepages and "enable" (default) for hugepage defragmentation seems to be the best workaround settings until this issue is fixed upstream.

Offline

Board footer

Powered by FluxBB