upgrade breakage (file called /dev/tty[0-9]*)

schuay · 2009-09-25 18:27:04

Today I updated a laptop with the usual pacman -Syu. After the upgrade (with no errors visible in pacman.log) i rebooted into a fscked system. Instead of /dev/tty0, /dev/tty1, ... I had a file called /dev/tty[0-9]* , and when the boot process completed I was left without a gui or a console login.

I have no idea what could have caused this as I updated successfully on several other machines.

Is there a quick fix for this? I tried reinstalling the base group, reinstalling ALL packages, still no go. If possible I'd like to avoid wiping the root partition and installing/configuring all over again - any ideas?

Thanks,
schuay

Last edited by schuay (2009-09-25 19:41:12)

damjan · 2009-09-27 15:39:42

What kernel do you use?

schuay · 2009-09-27 20:20:50

I was using the latest kernel from testing (both standard / fallback). Same as on all my other working machines.

Garns · 2009-09-27 20:46:33

Did the update include the initscripts package? This sounds like it could be a result of this.

Take a look at /etc/inittab which should have several lines like:

c1:2345:respawn:/sbin/agetty -8 38400 tty1 linux
c2:2345:respawn:/sbin/agetty -8 38400 tty2 linux
c3:2345:respawn:/sbin/agetty -8 38400 tty3 linux
c4:2345:respawn:/sbin/agetty -8 38400 tty4 linux
c5:2345:respawn:/sbin/agetty -8 38400 tty5 linux
c6:2345:respawn:/sbin/agetty -8 38400 tty6 linux

damjan · 2009-09-27 22:49:37

schuay wrote:

I was using the latest kernel from testing (both standard / fallback). Same as on all my other working machines.

so that is 2.6.31.1 right?

Because, it seems 2.6.31 has that bug ... for some reason udev doesn't create the /dev/tty[0...] device nodes on 2.6.31.

This happens only sometimes btw. For ex. I run 2.6.31 (self-compiled) with no problems, a friend of mine with the same config as mine has the exactly that problem (he only had to select the HIGHMEM=4G, I have 1GB ram, he has 4GB). We both are runing up-to-date Arch on almost the same hardware.

it seems to be mentioned here, but I haven't seen a patch or a resolution.

Garns wrote:

Did the update include the initscripts package? This sounds like it could be a result of this.

no, this is 2.6.31 related

damjan · 2009-09-27 22:59:26

BTW "/dev/tty[0-9]*" is created by /etc/rc.sysinit since the nodes are missing - lines 363 and the after that in a line like:

for i in /dev/tty[0-9]*; do
...
printf ... > ${i}

which may need to be a bit more robust I think

schuay · 2009-09-28 06:50:20

damjan wrote:

schuay wrote:
I was using the latest kernel from testing (both standard / fallback). Same as on all my other working machines.
so that is 2.6.31.1 right?
Because, it seems 2.6.31 has that bug ... for some reason udev doesn't create the /dev/tty[0...] device nodes on 2.6.31.
This happens only sometimes btw. For ex. I run 2.6.31 (self-compiled) with no problems, a friend of mine with the same config as mine has the exactly that problem (he only had to select the HIGHMEM=4G, I have 1GB ram, he has 4GB). We both are runing up-to-date Arch on almost the same hardware.
it seems to be mentioned here, but I haven't seen a patch or a resolution.

Strange - it only happened on this machine ( and I have 5 other machines running 2.6.31 without problems ) and it was 100% reproducable - i didn't get a single valid boot out of LOTS of attempts. This is the first issue I ran into I didn't know how to fix without a reinstall. I tried everything, even deinstalling all of base and base-devel groups and then reinstalling. The problem only went away after a complete wipe + reinstall of the root partition

Edit: unrelated, but this same laptop also has other problems with linux - the battery display works fine for a while, then suddenly it displays an empty battery. current_charge in /sys/.../BAT1/ shows -1000.

Last edited by schuay (2009-09-28 06:51:36)

Harey · 2009-10-01 16:05:13

Same here. One out of 5 machines had this ugly problem. Had to go back to 2.6.30. Did anyone file a bug report?
Did somebody find a remedy?

Harvey

R00KIE · 2009-10-01 22:21:38

I'm using kernel 2.6.31.1 64bit from testing and everything is working just fine.
It seems this problem is somehow related to the presence of some hardware in the system ...

Here:

00:00.0 Host bridge: ATI Technologies Inc RS690 Host Bridge
00:02.0 PCI bridge: ATI Technologies Inc RS690 PCI to PCI Bridge (PCI Express Graphics Port 0)
00:04.0 PCI bridge: ATI Technologies Inc Device 7914
00:06.0 PCI bridge: ATI Technologies Inc RS690 PCI to PCI Bridge (PCI Express Port 2)
00:07.0 PCI bridge: ATI Technologies Inc RS690 PCI to PCI Bridge (PCI Express Port 3)
00:12.0 SATA controller: ATI Technologies Inc SB600 Non-Raid-5 SATA
00:13.0 USB Controller: ATI Technologies Inc SB600 USB (OHCI0)
00:13.1 USB Controller: ATI Technologies Inc SB600 USB (OHCI1)
00:13.2 USB Controller: ATI Technologies Inc SB600 USB (OHCI2)
00:13.3 USB Controller: ATI Technologies Inc SB600 USB (OHCI3)
00:13.4 USB Controller: ATI Technologies Inc SB600 USB (OHCI4)
00:13.5 USB Controller: ATI Technologies Inc SB600 USB Controller (EHCI)
00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 14)
00:14.1 IDE interface: ATI Technologies Inc SB600 IDE
00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA)
00:14.3 ISA bridge: ATI Technologies Inc SB600 PCI to LPC Bridge
00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge
00:18.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
00:18.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
00:18.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
00:18.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
01:00.0 VGA compatible controller: ATI Technologies Inc Mobility Radeon HD 2400
01:00.1 Audio device: ATI Technologies Inc RV610 audio device [Radeon HD 2400 PRO]
06:00.0 Ethernet controller: Atheros Communications Inc. AR5001 Wireless Network Adapter (rev 01)
07:01.0 FireWire (IEEE 1394): Ricoh Co Ltd R5C832 IEEE 1394 Controller (rev 05)
07:01.1 SD Host controller: Ricoh Co Ltd R5C822 SD/SDIO/MMC/MS/MSPro Host Adapter (rev 22)
07:01.2 System peripheral: Ricoh Co Ltd R5C843 MMC Host Controller (rev 12)
07:01.3 System peripheral: Ricoh Co Ltd R5C592 Memory Stick Bus Host Adapter (rev ff)
07:07.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10)

Edit:
Posted the output of lspci.
By the way, I'm using ext3 in all the partitions.

Last edited by R00KIE (2009-10-02 10:33:25)

schuay · 2009-10-02 07:23:38

Harey wrote:

Same here. One out of 5 machines had this ugly problem. Had to go back to 2.6.30. Did anyone file a bug report?
Did somebody find a remedy?
Harvey

After a complete partition wipe + reinstall, 2.6.31 works. No idea why.
The problem occured on a Belinea o'book 4 notebook - I don't have it here so I can't post the specific hardware components at the moment.

Harey · 2009-10-02 08:48:18

Hi folks,

although this is a pain in the a** I'm somewhat glad not to be the only one

schuay wrote:

After a complete partition wipe + reinstall, 2.6.31 works. No idea why.
The problem occured on a Belinea o'book 4 notebook - I don't have it here so I can't post the specific hardware components at the moment.

I don't have the time to do a complete reinstall right now. I'm out of schedule already... Did you change the filesystem when reinstalling? My machine is ext3. FWIW Lspci gives this output:

00:00.0 Host bridge: Intel Corporation 82945G/GZ/P/PL Memory Controller Hub (rev 02)
00:01.0 PCI bridge: Intel Corporation 82945G/GZ/P/PL PCI Express Root Port (rev 02)
00:1b.0 Audio device: Intel Corporation 82801G (ICH7 Family) High Definition Audio Controller (rev 01)
00:1c.0 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 1 (rev 01)
00:1c.1 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 2 (rev 01)
00:1d.0 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #1 (rev 01)
00:1d.1 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #2 (rev 01)
00:1d.2 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #3 (rev 01)
00:1d.3 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #4 (rev 01)
00:1d.7 USB Controller: Intel Corporation 82801G (ICH7 Family) USB2 EHCI Controller (rev 01)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev e1)
00:1f.0 ISA bridge: Intel Corporation 82801GB/GR (ICH7 Family) LPC Interface Bridge (rev 01)
00:1f.2 IDE interface: Intel Corporation 82801GB/GR/GH (ICH7 Family) SATA IDE Controller (rev 01)
00:1f.3 SMBus: Intel Corporation 82801G (ICH7 Family) SMBus Controller (rev 01)
01:00.0 VGA compatible controller: nVidia Corporation G96 [GeForce 9500 GT] (rev a1)
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet control

I'll be back in this location in about three weeks. I will have another try then...

Harvey

damjan · 2009-10-03 13:42:27

I found the problem in a old /etc/udev/rules.d/udev.rules ... it wasn't owned by any package and the timestamp was from 2007.

I don't understand how and why before 2.6.31 there wasn't a problem?

Tinister · 2009-10-08 15:56:06

Anyone else able to solve this? All of my /etc/udev/udev.rules files are accounted for.

damjan · 2009-10-09 23:10:09

Tinister wrote:

Anyone else able to solve this? All of my /etc/udev/udev.rules files are accounted for.

What are you saying? You deleted /etc/udev/rules.d/udev.rules and it still doesn't create the /dev/tty? device nodes?!?

Maybe also check /lib/udev/rules.d/ for some unsuspected files

Tinister · 2009-10-10 15:03:13

There wasn't even a udev.rules file to delete. Every file in /etc/udev/udev.rules/ is owned by udev.

Same goes for /lib/udev/rules.d/

rollenwiese · 2009-10-12 22:57:08

I also have this problem, but I do not have a notebook. Udev has stopped creating the dev/tty files since updgrading the kernel

Edit: Link http://forum.soft32.com/linux/linux-31- … 95295.html

if you change the inittab refs from tty1,2 to vc/1 vc/2 etc you will be able to boot. it worked for me, I am not an expert and don't know enough to predict unintended consequences of this.

Last edited by rollenwiese (2009-10-12 23:20:22)

schuay · 2009-10-13 06:40:06

rollenwiese wrote:

I also have this problem, but I do not have a notebook. Udev has stopped creating the dev/tty files since updgrading the kernel
Edit: Link http://forum.soft32.com/linux/linux-31- … 95295.html
if you change the inittab refs from tty1,2 to vc/1 vc/2 etc you will be able to boot. it worked for me, I am not an expert and don't know enough to predict unintended consequences of this.

And you're sure it's the kernel upgrade, you didn't upgrade udev (or anything else that might be related) at the same time? Check /var/log/pacman.log

Last edited by schuay (2009-10-13 06:40:41)

rollenwiese · 2009-10-13 22:17:55

Udev was not upgraded. I am currently using udev version 146. Udev was last upgraded per the pacman log on 9/9/09 (to 145-1 to 146-2). I rebooted many time since the 10/11/09 upgrade with no issues.

Relevant pacman log entries:

[2009-10-11 11:46] upgraded kernel26 (2.6.30.6-1 -> 2.6.31.3-1)
[2009-10-11 11:46] upgraded libmysqlclient (5.1.38-1 -> 5.1.39-1)
[2009-10-11 11:46] upgraded mysql-clients (5.1.38-1 -> 5.1.39-1)
[2009-10-11 11:46] upgraded mysql (5.1.38-1 -> 5.1.39-1)
[2009-10-11 11:46] In order to use the new nvidia module, exit Xserver and unload it manually.
[2009-10-11 11:46] upgraded nvidia (185.18.36-1 -> 185.18.36-2)

I have since downgraded to kernel 2.6.30.6-1, and changed my inittab references back from vc/* to tty* - this solved the problem for me.

I noticed with the 2.6.31.3-1 kernel my udev time went WAY up relative to 2.6.30.6-1.

Also want to clarify, that I do not have a udev.rules file. I do have a couple of other udev rules files. I can provide those as well as lspci output upon request.

KERNEL=="tty[0-9]*",        GROUP="tty", MODE="0666", NAME="vc/%n", SYMLINK+="%k" OPTIONS="last_rule"

Last edited by rollenwiese (2009-10-13 22:28:51)

Gen2ly · 2009-10-13 23:34:53

I also fixed my problem by using udev 145 too.

rollenwiese · 2009-10-14 01:18:19

Gen2ly, were you able to use 2.6.31-3 successfully with the 145 version of udev? Thanks,

Harey · 2009-10-27 19:44:41

Okay, I did the following:
I renamed /etc/udev and /lib/udev and reinstalled udev. After that everything was ok, the backup directories could be deleted.

I still don't know what was wrong, though

Harvey

damjan · 2009-10-27 22:44:44

@Harey
rogue rule files somewhere there, probably the same as I mentioned in #12

Harey · 2009-10-28 08:49:29

rogue rule files somewhere there, probably the same as I mentioned in #12

That's why I tried this. But I could swear there are no rules files that I changed on this machine. It's running again, that's important. I can't compare the rulesets because I deleted the old ones. Next time I'll investigate further, I promise

Harvey

rollenwiese · 2009-10-31 23:46:58

Just reporting I was able to resolve this issue also by renaming /etc/udev and /lib/udev, reinstalling udev and the lastest kernel package. just reinstalling alone did not fix had to remove my config files, not sure what the problem was. Thanks!

liquidsunshine · 2009-11-01 18:34:46

I encountered this problem and was able to solve it using this thread, so thanks, everyone!

Also, I did quick check to see what the differences in the /etc/udev/ directory were:

liquidsunshine@ruaidhri:/etc$ diff -r udev udev-bak
Only in udev-bak: lomoco_mouse.conf
Only in udev-bak/rules.d: 10-cups-usb.rules
Only in udev-bak/rules.d: 45-libnjb.rules
Only in udev-bak/rules.d: 60-pcmcia.rules
Only in udev-bak/rules.d: 60-virtualbox.rules
Only in udev-bak/rules.d: 65-libmtp.rules
Only in udev-bak/rules.d: 80-lomoco.rules
Only in udev-bak/rules.d: 90-hal.rules
Only in udev-bak/rules.d: 97-bluetooth-serial.rules
Only in udev-bak/rules.d: 99-fuse.rules
Only in udev-bak/rules.d: device-mapper.rules
Only in udev-bak/rules.d: udev.rules

and the same for /lib/udev/:

liquidsunshine@ruaidhri:/lib$ diff -r udev udev-bak/
Only in udev-bak/: bluetooth_serial
Only in udev-bak/: check-mtp-device
Only in udev-bak/: check-ptp-camera
File udev/devices/console is a character special file while file udev-bak/devices/console is a character special file
File udev/devices/fuse is a character special file while file udev-bak/devices/fuse is a character special file
File udev/devices/kmsg is a character special file while file udev-bak/devices/kmsg is a character special file
File udev/devices/loop0 is a block special file while file udev-bak/devices/loop0 is a block special file
File udev/devices/net/tun is a character special file while file udev-bak/devices/net/tun is a character special file
File udev/devices/null is a character special file while file udev-bak/devices/null is a character special file
File udev/devices/ppp is a character special file while file udev-bak/devices/ppp is a character special file
File udev/devices/zero is a character special file while file udev-bak/devices/zero is a character special file
Only in udev-bak/: devkit-disks-dm-export
Only in udev-bak/: devkit-disks-part-id
Only in udev-bak/: devkit-disks-probe-ata-smart
Only in udev-bak/: lomoco.sh
Only in udev-bak/rules.d: 52-libmtp.rules
Only in udev-bak/rules.d: 53-sane.rules
Only in udev-bak/rules.d: 54-gphoto.rules
Only in udev-bak/rules.d: 55-hpmud.rules
Only in udev-bak/rules.d: 56-hpmud_support.rules
Only in udev-bak/rules.d: 95-devkit-disks.rules
Only in udev-bak/rules.d: 95-devkit-power-battery-recall-dell.rules
Only in udev-bak/rules.d: 95-devkit-power-battery-recall-fujitsu.rules
Only in udev-bak/rules.d: 95-devkit-power-battery-recall-gateway.rules
Only in udev-bak/rules.d: 95-devkit-power-battery-recall-ibm.rules
Only in udev-bak/rules.d: 95-devkit-power-battery-recall-lenovo.rules
Only in udev-bak/rules.d: 95-devkit-power-battery-recall-toshiba.rules
Only in udev-bak/rules.d: 95-devkit-power-csr.rules
Only in udev-bak/rules.d: 95-devkit-power-hid.rules
Only in udev-bak/rules.d: 95-devkit-power-wup.rules
Only in udev-bak/rules.d: 97-bluetooth-hid2hci.rules
Only in udev-bak/rules.d: 97-bluetooth.rules

Unfortunately, I'm not curious enough at this point to do more in-depth investigation, but it looks to me like there were quite a few left-over files lying around from previous udev upgrades...

Arch Linux

#1 2009-09-25 18:27:04

upgrade breakage (file called /dev/tty[0-9]*)

#2 2009-09-27 15:39:42

Re: upgrade breakage (file called /dev/tty[0-9]*)

#3 2009-09-27 20:20:50

Re: upgrade breakage (file called /dev/tty[0-9]*)

#4 2009-09-27 20:46:33

Re: upgrade breakage (file called /dev/tty[0-9]*)

#5 2009-09-27 22:49:37

Re: upgrade breakage (file called /dev/tty[0-9]*)

#6 2009-09-27 22:59:26

Re: upgrade breakage (file called /dev/tty[0-9]*)

#7 2009-09-28 06:50:20

Re: upgrade breakage (file called /dev/tty[0-9]*)

#8 2009-10-01 16:05:13

Re: upgrade breakage (file called /dev/tty[0-9]*)

#9 2009-10-01 22:21:38

Re: upgrade breakage (file called /dev/tty[0-9]*)

#10 2009-10-02 07:23:38

Re: upgrade breakage (file called /dev/tty[0-9]*)

#11 2009-10-02 08:48:18

Re: upgrade breakage (file called /dev/tty[0-9]*)

#12 2009-10-03 13:42:27

Re: upgrade breakage (file called /dev/tty[0-9]*)

#13 2009-10-08 15:56:06

Re: upgrade breakage (file called /dev/tty[0-9]*)

#14 2009-10-09 23:10:09

Re: upgrade breakage (file called /dev/tty[0-9]*)

#15 2009-10-10 15:03:13

Re: upgrade breakage (file called /dev/tty[0-9]*)

#16 2009-10-12 22:57:08

Re: upgrade breakage (file called /dev/tty[0-9]*)

#17 2009-10-13 06:40:06

Re: upgrade breakage (file called /dev/tty[0-9]*)

#18 2009-10-13 22:17:55

Re: upgrade breakage (file called /dev/tty[0-9]*)

#19 2009-10-13 23:34:53

Re: upgrade breakage (file called /dev/tty[0-9]*)

#20 2009-10-14 01:18:19

Re: upgrade breakage (file called /dev/tty[0-9]*)

#21 2009-10-27 19:44:41

Re: upgrade breakage (file called /dev/tty[0-9]*)

#22 2009-10-27 22:44:44

Re: upgrade breakage (file called /dev/tty[0-9]*)

#23 2009-10-28 08:49:29

Re: upgrade breakage (file called /dev/tty[0-9]*)

#24 2009-10-31 23:46:58

Re: upgrade breakage (file called /dev/tty[0-9]*)

#25 2009-11-01 18:34:46

Re: upgrade breakage (file called /dev/tty[0-9]*)

Board footer