You are not logged in.

#1 2012-06-16 13:08:28

agkbill
Member
Registered: 2011-06-13
Posts: 85

Kernel panic after latest upgrade.

Hi all,

After latest update I consistently get kernel panic.

I can boot OK, login and start work but after a couple of minutes everything freezes and I can not do anything.

It was when I worked from the shell before starting X that I could see that I got kernel panic.
I got it when running rsync and try to copy a file over NFS. I also got it running pacman.

First message is "skb_over_panic:" the a lot of for me more or less cryptic messeges and it ends with "Kernel panic - not syncing: Fatal exception in interupt"

This is the first time I encounter something like this since I started to use Archlinux a couple of years ago.

Any ide of where to start to look?

I tried "mkinicpio -p linux" but it did not help.


Thank you,
Best regards,
/Christer

Offline

#2 2012-06-16 14:47:53

ralvez
Member
From: Canada
Registered: 2005-12-06
Posts: 1,694
Website

Re: Kernel panic after latest upgrade.

I just noticed that in there was a similar bug.
The user had a realtek/r8169 card and reported a similar error.
Here is the link:https://lkml.org/lkml/2012/4/15/64

Perhaps, you can try downgrading your kernel (if nothing else works).

R.

Last edited by ralvez (2012-06-16 14:49:40)

Offline

#3 2012-06-17 09:26:19

agkbill
Member
Registered: 2011-06-13
Posts: 85

Re: Kernel panic after latest upgrade.

I tried downgrading but it is this with dependencies. I could not downgrade kernel because nvidia needed kernel 3.4 and I could not downgrade nvida because it needed kernel 3.4.

I could probably solve it by using -force option in pacman, but I am afraid that I will end up in a dependencies mess.

Hope it will be a kernel update soon that will solve the problem.

Offline

#4 2012-06-17 12:04:24

ralvez
Member
From: Canada
Registered: 2005-12-06
Posts: 1,694
Website

Re: Kernel panic after latest upgrade.

Nope. Do not force the downgrade you are likely to end up with a broken system.
Check in /var/log/pacman.log to see the packages that were upgraded the last time and downgrade them too, so you end up with your system in a similar state as it was **before** the upgrade.

R.

Offline

#5 2012-07-03 04:43:00

durian
Member
Registered: 2011-12-22
Posts: 15

Re: Kernel panic after latest upgrade.

I have the same problem, kernel panics within a minute everytime, same "skb_over_panic" message.

How can I check /var/log/pacman, or in fact do anything else, if the system freezes within no time?

-peter

Offline

#6 2012-07-03 05:34:34

rwd
Member
Registered: 2009-02-08
Posts: 664

Re: Kernel panic after latest upgrade.

boot from cdr, mount harddisk (https://wiki.archlinux.org/index.php/Kernel_Panics)

Offline

#7 2012-07-03 06:47:25

durian
Member
Registered: 2011-12-22
Posts: 15

Re: Kernel panic after latest upgrade.

rwd wrote:

OK, I got that far, and tried to downgrade to kernel 3.3.4, but now nvidia complains it needs linux >=3.4. And downgrading nvidia won't work because it needs kernel < 3.4

Tricky :-)

-peter

Offline

#8 2012-07-03 07:29:10

crystaly
Member
Registered: 2012-06-30
Posts: 3

Re: Kernel panic after latest upgrade.

i got the same kernel panic when causing network traffic. The reason was the ralink sta kernel module. I've blacklisted it and now using the kernel-intern rt2800usb module. Now i'm running fine.

Are you using a ralink card (ethernet or wlan) ?

Offline

#9 2012-07-03 08:31:15

durian
Member
Registered: 2011-12-22
Posts: 15

Re: Kernel panic after latest upgrade.

crystaly wrote:

Are you using a ralink card (ethernet or wlan) ?

Yes, I am, an RT3060 Wireless card. I am using a hand-compiled driver (DPO_RT3562_3592_3062_LinuxSTA_V2.4.1.1).

The build-in ethernet is Realtek. Hm, maybe I should disconnect the wifi and see what happens...

-peter

addition: yep, that made a difference, no panic with ethernet, panic with wifi...

Last edited by durian (2012-07-03 17:08:42)

Offline

#10 2012-07-04 02:04:42

Kiirani
Member
Registered: 2012-07-04
Posts: 6

Re: Kernel panic after latest upgrade.

durian wrote:
crystaly wrote:

Are you using a ralink card (ethernet or wlan) ?

Yes, I am, an RT3060 Wireless card. I am using a hand-compiled driver (DPO_RT3562_3592_3062_LinuxSTA_V2.4.1.1).

Poking head in for a me-too on this one.

Just started getting kernel panics after updating to 3.4.4-2-ARCH. They seemed to be caused by network traffic, hence why I'm here.

I'm running an Asus USB-N13 wifi adapter. It identifies itself as an RT3072
Regarding the suggested option of moving to the rt2800usb driver a few posts back - this driver does not work as advertised for this card, which is why I've been compiling my own from the official sources.

The band-aid solution for now has been to use my package cache to downgrade packages linux, linux-headers, nvidia, and nvidia-utils to the previously installed versions - kernel 3.3.7-1-ARCH works fine.  Nothing on my system other than nvidia had problems with kernel < 3.4, and the latest driver version is not currently a requirement.

Offline

#11 2012-07-04 06:58:20

durian
Member
Registered: 2011-12-22
Posts: 15

Re: Kernel panic after latest upgrade.

Kiirani wrote:

The band-aid solution for now has been to use my package cache to downgrade packages linux, linux-headers, nvidia, and nvidia-utils to the previously installed versions - kernel 3.3.7-1-ARCH works fine.  Nothing on my system other than nvidia had problems with kernel < 3.4, and the latest driver version is not currently a requirement.

Network traffic seems to be the problem on my system as well, it works fine with ethernet.

How did you get around the nvidia/kernel-3.4 catch 22? Force-downgrade?

-peter

Offline

#12 2012-07-04 22:20:33

Kiirani
Member
Registered: 2012-07-04
Posts: 6

Re: Kernel panic after latest upgrade.

durian wrote:
Kiirani wrote:

The band-aid solution for now has been to use my package cache to downgrade packages linux, linux-headers, nvidia, and nvidia-utils to the previously installed versions - kernel 3.3.7-1-ARCH works fine.  Nothing on my system other than nvidia had problems with kernel < 3.4, and the latest driver version is not currently a requirement.

Network traffic seems to be the problem on my system as well, it works fine with ethernet.

How did you get around the nvidia/kernel-3.4 catch 22? Force-downgrade?

-peter

I reinstalled nvidia and nvidia-utils 295.53-1 from my package cache. (This just happens to be the next-latest version sitting in there). Had to grab lib32-nvidia-utils in a matching version from the arch rollback machine.

I didn't really find a catch 22 about it, possibly because I threw the kernel and nvidia into the one downgrade command.

Last edited by Kiirani (2012-07-04 22:20:56)

Offline

#13 2012-07-14 09:39:43

agkbill
Member
Registered: 2011-06-13
Posts: 85

Re: Kernel panic after latest upgrade.

It is only during network traffic using Wifi that I get kernel panic it works fine with cable, ethernet.

I use the rt2870 from AUR.

It have now been two kernel releases since I first experienced the problems.
After updating the system almost daily it would now be a nightmare to downgrade, so I dont think it is an option for me now.

hope a solution will come soon. I believe so smile

Offline

#14 2012-07-15 01:01:18

Kiirani
Member
Registered: 2012-07-04
Posts: 6

Re: Kernel panic after latest upgrade.

agkbill wrote:

After updating the system almost daily it would now be a nightmare to downgrade, so I dont think it is an option for me now.

Remembering that to downgrade your kernel, you don't necessarily have to downgrade your entire system. Most of the packages on your system don't care what kernel version you're running as long as it's not, you know, 2.4... 

You can always pull a working 3.3 era kernel out of the arch rollback machine if you've lost it in your cache.
So far the only packages I've needed to downgrade to fix this problem are linux, linux-headers, nvidia, and nvidia-utils. [and of course, their lib32 equivalents]

Offline

#15 2012-07-16 14:02:25

agkbill
Member
Registered: 2011-06-13
Posts: 85

Re: Kernel panic after latest upgrade.

I manage (almost) to downgrade to 3.3 kernel but after reboot it looks like 3.4 is still loaded.

uname -r

Output: 3.4.4-2-ARCH

I did:

sudo pacman -U /var/cache/pacman/pkg/nvidia-295.59-1-x86_64.pkg.tar.xz /var/cache/pacman/pkg/nvidia-utils-295.59-1-x86_64.pkg.tar.xz /var/cache/pacman/pkg/linux-3.3.7-1-x86_64.pkg.tar.xz

If I run againe I get:

[christer@Arch Archlinux+++]$ sudo pacman -U /var/cache/pacman/pkg/nvidia-295.59-1-x86_64.pkg.tar.xz /var/cache/pacman/pkg/nvidia-utils-295.59-1-x86_64.pkg.tar.xz /var/cache/pacman/pkg/linux-3.3.7-1-x86_64.pkg.tar.xz
Password:
loading packages...
warning: nvidia-295.59-1 is up to date -- reinstalling
warning: nvidia-utils-295.59-1 is up to date -- reinstalling
warning: downgrading package linux (3.4.4-2 => 3.3.7-1)
resolving dependencies...
looking for inter-conflicts...

Targets (3): linux-3.4.4-2  nvidia-295.59-1  nvidia-utils-295.59-1

Total Installed Size:   117.05 MiB
Net Upgrade Size:       0.00 MiB

Proceed with installation? [Y/n] Y
(1/3) checking package integrity                                       [########################################] 100%
(1/3) loading package files                                            [########################################] 100%
(3/3) checking for file conflicts                                      [########################################] 100%
(3/3) checking available disk space                                    [########################################] 100%
(1/3) upgrading linux                                                  [########################################] 100%
>>> Updating module dependencies. Please wait ...
>>> Generating initial ramdisk, using mkinitcpio.  Please wait...
==> Building image from preset: 'default'
  -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
==> Starting build: 3.4.4-2-ARCH
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [pata]
  -> Running build hook: [scsi]
  -> Running build hook: [sata]
  -> Running build hook: [filesystems]
  -> Running build hook: [usbinput]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip initcpio image: /boot/initramfs-linux.img
==> Image generation successful
==> Building image from preset: 'fallback'
  -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-fallback.img -S autodetect
==> Starting build: 3.4.4-2-ARCH
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [pata]
  -> Running build hook: [scsi]
  -> Running build hook: [sata]
  -> Running build hook: [filesystems]
  -> Running build hook: [usbinput]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip initcpio image: /boot/initramfs-linux-fallback.img
==> Image generation successful
(2/3) upgrading nvidia-utils                                           [########################################] 100%
(3/3) upgrading nvidia                                                 [########################################] 100%
[christer@Arch Archlinux+++]$

I also run:

grub-mkconfig -o /boot/grub/grub.cfg

To regenerate grub2 (I dont know it it is nessesary).

But after reboot, still 3.4 kernel.

Any idea what I am missing?

Best regards,
Christer

Offline

#16 2012-07-16 22:57:12

Kiirani
Member
Registered: 2012-07-04
Posts: 6

Re: Kernel panic after latest upgrade.

agkbill wrote:

I manage (almost) to downgrade to 3.3 kernel but after reboot it looks like 3.4 is still loaded.

uname -r

Output: 3.4.4-2-ARCH

I did:

sudo pacman -U /var/cache/pacman/pkg/nvidia-295.59-1-x86_64.pkg.tar.xz /var/cache/pacman/pkg/nvidia-utils-295.59-1-x86_64.pkg.tar.xz /var/cache/pacman/pkg/linux-3.3.7-1-x86_64.pkg.tar.xz

You'll need to get the appropriate linux-headers package as well. Maybe that's the problem?


agkbill wrote:

I also run:

grub-mkconfig -o /boot/grub/grub.cfg

To regenerate grub2 (I dont know it it is nessesary).

Completely unnecessary.

Offline

#17 2012-07-17 20:41:55

agkbill
Member
Registered: 2011-06-13
Posts: 85

Re: Kernel panic after latest upgrade.

Thank you Kiirani, I added the linux-header package as well

sudo pacman -U /var/cache/pacman/pkg/nvidia-295.59-1-x86_64.pkg.tar.xz /var/cache/pacman/pkg/nvidia-utils-295.59-1-x86_64.pkg.tar.xz /var/cache/pacman/pkg/linux-3.3.7-1-x86_64.pkg.tar.xz /var/cache/pacman/pkg/linux-headers-3.3.7-1-x86_64.pkg.tar.xz

But still I get:

[christer@Arch ~]$ uname -r
3.4.4-2-ARCH

Regards

Offline

#18 2012-07-18 00:13:50

Kiirani
Member
Registered: 2012-07-04
Posts: 6

Re: Kernel panic after latest upgrade.

agkbill wrote:

Thank you Kiirani, I added the linux-header package as well

sudo pacman -U /var/cache/pacman/pkg/nvidia-295.59-1-x86_64.pkg.tar.xz /var/cache/pacman/pkg/nvidia-utils-295.59-1-x86_64.pkg.tar.xz /var/cache/pacman/pkg/linux-3.3.7-1-x86_64.pkg.tar.xz /var/cache/pacman/pkg/linux-headers-3.3.7-1-x86_64.pkg.tar.xz

But still I get:

[christer@Arch ~]$ uname -r
3.4.4-2-ARCH

Regards


Have you rebooted to the new kernel?

Offline

#19 2012-07-18 15:50:38

agkbill
Member
Registered: 2011-06-13
Posts: 85

Re: Kernel panic after latest upgrade.

Yes, the 3.4.4-2-ARCH result from "uname -r" is after reboot.

Offline

#20 2012-07-19 16:11:35

SantaSCSI
Member
Registered: 2009-12-04
Posts: 20

Re: Kernel panic after latest upgrade.

I can confirm its the realtek NIC acting like a little b*tch that is causing the locks. My NAS system is having random shutdowns since monday. Every time the last entry in messages.log is:

r8169 0000:03:00.0: eth0: link up
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

When issuing a ping -t from my desktop to the NAS system, it barely stayed online for 1 minute before faceplanting again. When there is no network access it stays online longer, mostly until my ESX system is starting to access its NFS datastores.

Offline

#21 2012-07-19 16:15:39

bangkok_manouel
Member
From: indicates a starting point
Registered: 2005-02-07
Posts: 1,556

Re: Kernel panic after latest upgrade.

please git clone greg's tree and bisect this crap (disclaimer: i haven't read this thread of course, just last post).

Offline

#22 2013-04-27 05:58:23

Kiirani
Member
Registered: 2012-07-04
Posts: 6

Re: Kernel panic after latest upgrade.

So just as a general update, I still encounter this problem on current kernel [3.8.8-2-ARCH] ,  using an ASUS usb N13 wifi card
    Bus 001 Device 002: ID 0b05:1784 ASUSTek Computer, Inc. USB-N13 802.11n Network Adapter (rev. A1) [Ralink RT3072]

Note that this card is not listed as working on the RT2x00 driver site . I haven't tested it on the current kernel's 2800 driver, but it didn't work with the 2800 driver in 3.3 and 3.4.   Perhaps re-checking that is the next step.

Assuming rt2x00 still doesn't support this card, the driver I'm using is the official RT8070 /RT3070 /RT3370 /RT5370 /RT5372 USB driver provided by MediaTek.   This is the offending driver, which functions perfectly for kernels <= 3.3, and causes kernel panics on network traffic* in 3.4 and above.

* No kernel panic with a simple ping, kernel panic usually occurs on HTTP traffic for me. I haven't done extensive enough testing to say whether this is HTTP specifically, TCP specifically, etc...  I'm usually too busy swearing at a screen full of errors.

After first downgrading kernel in July last year, I eventually decided to switch to the linux-lts package and wait until this issue resolves itself.  Long term support is running 3.0,  which these drivers are still compatible with.

There's been no change to this problem in the kernel versions since then, and  it doesn't seem realistic for all users of these cards to continue using long term support kernels forever.   Where do we go from here?
It seems that the bug needs to be tracked down in the 5370sta sources and fixed.   

I'm not  exactly qualified to do this myself so I went looking to see if there's an upstream where bug reports can be filed.

This leads me to the present canonical location for the driver download,  which claims that mediatek is "very active in the linux community".
Available sources are still the 2011 version which is causing the problem.
I can't find any references to a bug tracker or any ability to communicate with upstream.

Has anybody else with this problem had more luck?

Last edited by Kiirani (2013-04-27 06:02:53)

Offline

Board footer

Powered by FluxBB