You are not logged in.

#1 2012-07-04 11:00:23

usagi
Member
Registered: 2011-11-21
Posts: 54

You knew it was coming: kernel build issues after kmod upgrade

Good morning,

After reading the BIG SCARY NOTE on the mailing list, I just *had* to upgrade my system and kernel. Following a -Syu, i built a kernel from the latest source (3.5.0-rc5+). Naturally, it built just fine. However, when i ran 'make modules_install', it installed the modules into /lib, which i was expecting. Moving on from *that* test, i ran:

 make INSTALL_MOD_PATH=/usr/ modules_install 

(Note that the INSTALL_MOD_PATH is /usr, NOT /usr/lib/modules/. This is because of how the variable is used)

Anyway, that command installed the modules correctly, but shot this error when it tried to depmod (paraphrasing. the important bit here is the path):

 ERROR: Failed to open /usr//usr/lib/modules/98.99.3.5.0-rc5-custom+: no such file or directory

Interestingly enough, running a simple 'depmod 3.5.0-rc5-custom+' works fine, and all is well. the kernel runs fantasticly, right up to the nvidia module.

So, i submit to you:
What is the prepended "98.99." in the file structure about, and why is the kernel's built-in depmod prepending usr twice?

Thanks folks,
Aaron

EDIT: For clarity, the 98.99 isn't actually there in the file structure, just in the depmod command the kernel makefile runs following modules_install.

Last edited by usagi (2012-07-04 11:02:30)

Offline

#2 2012-07-04 13:36:21

falconindy
Developer
From: New York, USA
Registered: 2009-10-22
Posts: 4,097
Website

Re: You knew it was coming: kernel build issues after kmod upgrade

It's a hack workaround (which now fails for the wrong reason) in scripts/depmod.sh (in the kernel tree). In the PKGBUILDs for the kernels, we skip this, since the depmod call at that time serves us no purpose (modules are eventually compressed and then we manually invoke depmod).

In building my own kernel, it didn't seem worth my time to figure out why this broke.

edit: Looking back at it now, you still need to edit depmod.sh to avoid the hack failing, but its sufficient enough to just do:

$ sed -i '/^depmod_hack_needed/ s/true/false/' scripts/depmod.sh

...and then invoking 'make MOD_INSTALL_DIR=/usr modules_install' will work as expected with the depmod call at the end.

Sadly, I believe this hack was added in error and should have been fixed in userland.

Last edited by falconindy (2012-07-04 15:29:25)

Offline

#3 2012-07-05 01:37:08

ngoonee
Forum Fellow
From: Between Thailand and Singapore
Registered: 2009-03-17
Posts: 6,854

Re: You knew it was coming: kernel build issues after kmod upgrade

usagi wrote:

Good morning,

After reading the BIG SCARY NOTE on the mailing list, I just *had* to upgrade my system and kernel.

Did you apply similar changes to your PKGBUILD as those listed in the ML note (there's a link to a patch to testing/linux)? Seems like that would workaround the issue falconindy is talking about.


Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.

Offline

#4 2012-07-06 00:38:17

LessWire
Member
From: Bavaria
Registered: 2010-06-14
Posts: 42

Re: You knew it was coming: kernel build issues after kmod upgrade

I'm just wondering why this change will be done - can anyone tell me the advantage ?

Kernel modules in "/usr/..."?  This looks not like the right place for me and it complicates with some other scripts. "KISS" !??

Offline

#5 2012-07-06 00:39:36

karol
Archivist
Registered: 2009-05-06
Posts: 25,432

Re: You knew it was coming: kernel build issues after kmod upgrade

LessWire wrote:

I'm just wondering why this change will be done - can anyone tell me the advantage ?

Kernel modules in "/usr/..."?  This looks not like the right place for me and it complicates with some other scripts. "KISS" !??

https://wiki.archlinux.org/index.php/De … ki:UsrMove

Offline

#6 2012-07-06 00:48:07

tomegun
Developer
From: France
Registered: 2010-05-28
Posts: 661

Re: You knew it was coming: kernel build issues after kmod upgrade

LessWire wrote:

I'm just wondering why this change will be done - can anyone tell me the advantage ?

Kernel modules in "/usr/..."?  This looks not like the right place for me and it complicates with some other scripts. "KISS" !??

We are merging all of /lib into /usr/lib, and eventually will be making /lib a symlink to /usr/lib for compatibility. In simple terms, this simplifies your system by only having one folder (/usr/lib) rather than two (/lib and /usr/lib). Originally, the split was made so that we could boot a basic system without /usr mounted. This reason no longer applies as we mount /usr from the initramfs.

Offline

#7 2012-07-06 00:53:02

LessWire
Member
From: Bavaria
Registered: 2010-06-14
Posts: 42

Re: You knew it was coming: kernel build issues after kmod upgrade

Thanks carol, i read this. But in this (and some other) case it doesn't make sense in my eyes.
It starts to destroy the "kiss" principle i liked so much so far.

Just my opinion.

Offline

#8 2012-07-06 00:56:20

LessWire
Member
From: Bavaria
Registered: 2010-06-14
Posts: 42

Re: You knew it was coming: kernel build issues after kmod upgrade

@tomegun: thanks for the statement, that's a reason and hopefully a solution!

compatibility is very important.

Last edited by LessWire (2012-07-06 00:57:44)

Offline

#9 2012-07-06 01:00:07

karol
Archivist
Registered: 2009-05-06
Posts: 25,432

Re: You knew it was coming: kernel build issues after kmod upgrade

As Dave Reisner (falconindy) said, it's a bit late for such opinions to matter really: http://mailman.archlinux.org/pipermail/ … 23182.html
The plan wasn't secret and suggestions how to do things differently should have been presented earlier.

You're obviously entitled to your opinion, but like with udev & systemd, things are going to be done this way whether you like it or not.

Offline

#10 2012-07-06 01:14:10

LessWire
Member
From: Bavaria
Registered: 2010-06-14
Posts: 42

Re: You knew it was coming: kernel build issues after kmod upgrade

You are right, everybody must find the distribution which works best for his needs. wink

For me it is since a few years Archlinux (after many years with Debian before) and i see no reason for a change till now. Most important for me is the enduration of the "kiss" principle.

So let's look for new things to come. wink

Offline

#11 2012-07-06 01:21:57

Allan
Developer
From: Brisbane, AU
Registered: 2007-06-09
Posts: 10,428
Website

Re: You knew it was coming: kernel build issues after kmod upgrade

Having just /usr/lib rather than both /lib and /usr/lib seems KISS to me...

Offline

#12 2012-07-06 01:23:36

graysky
Member
From: /run/user/1000
Registered: 2008-12-01
Posts: 8,647
Website

Re: You knew it was coming: kernel build issues after kmod upgrade

Aren't all major distros going this way at some point?  Is this a TGN?


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#13 2012-07-06 01:35:54

ngoonee
Forum Fellow
From: Between Thailand and Singapore
Registered: 2009-03-17
Posts: 6,854

Re: You knew it was coming: kernel build issues after kmod upgrade

graysky wrote:

Aren't all major distros going this way at some point?  Is this a TGN?

Eventually yes, but its always good to leave one thread to collect all the searchers who will be searching for information about this. Especially one with a link to the relevant wiki and comments from multiple developers.

Don't know about major distros going this way, but I don't think that's much of a consideration, this isn't like systemd.


Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.

Offline

#14 2012-07-06 01:37:53

tomegun
Developer
From: France
Registered: 2010-05-28
Posts: 661

Re: You knew it was coming: kernel build issues after kmod upgrade

graysky wrote:

Aren't all major distros going this way at some point?

Some have. Some will. Don't know if all will.

Is this a TGN?

Probably, but I agree with ngoonee: Let it go nowhere for a while longer ;-)

Offline

#15 2012-07-06 01:39:30

LessWire
Member
From: Bavaria
Registered: 2010-06-14
Posts: 42

Re: You knew it was coming: kernel build issues after kmod upgrade

Allan wrote:

Having just /usr/lib rather than both /lib and /usr/lib seems KISS to me...

Yes Allan, but i think only if the "make" procedure of the original kernel would follow this as default and it doesn't.

It's not worth and probably too late for a long discussion, i simply created a symlink and moved my modules directory to "/usr/lib" although it's not a clean solution.

Offline

#16 2012-07-07 11:02:04

sacamano_m82
Member
Registered: 2007-05-29
Posts: 167

Re: You knew it was coming: kernel build issues after kmod upgrade

This breaks "make tar-pkg" in the kernel. I get the same error:

ERROR: could not open directory /home/sujith/dev/wireless-testing/build/tar-install/usr/lib/modules/99.98.3.5.0-rc5-wl: No such file or directory

I understand the /usr move, so is there any clean way to handle this breakage ?

Offline

#17 2012-07-07 11:09:25

tomegun
Developer
From: France
Registered: 2010-05-28
Posts: 661

Re: You knew it was coming: kernel build issues after kmod upgrade

@sacamano_m82: I just do "make && make install && make modules_install" as normal, followed by "mv /lib/modules/* /usr/lib/modules/". Alternatively, just wait for /lib to become a symlink to /usr/lib, and then it won't matter.

Offline

#18 2012-07-07 11:21:23

bangkok_manouel
Member
From: indicates a starting point
Registered: 2005-02-07
Posts: 1,554

Re: You knew it was coming: kernel build issues after kmod upgrade

edit2/ ah no, that should work

make INSTALL_MOD_PATH=/usr modules_install

Last edited by bangkok_manouel (2012-07-07 11:25:06)


All design goals must be phrased in such a way that it is hard to use them as slogans to justify stupidity.

Offline

#19 2012-07-07 11:35:43

sacamano_m82
Member
Registered: 2007-05-29
Posts: 167

Re: You knew it was coming: kernel build issues after kmod upgrade

tomegun wrote:

@sacamano_m82: I just do "make && make install && make modules_install" as normal, followed by "mv /lib/modules/* /usr/lib/modules/". Alternatively, just wait for /lib to become a symlink to /usr/lib, and then it won't matter.

Well, I build a tarball using the kernel's "tar-pkg" option and push it to a bunch of test machines. This allows me to keep my development tree on a single machine and send incremental .ko files to said test machines later. I use the wireless-testing tree, btw (mostly ath9k development). Right now, I am stuck in a situation where years of kernel building on a daily basis has come to a halt. smile

Offline

#20 2012-07-07 12:45:41

falconindy
Developer
From: New York, USA
Registered: 2009-10-22
Posts: 4,097
Website

Re: You knew it was coming: kernel build issues after kmod upgrade

sacamano_m82 wrote:

This breaks "make tar-pkg" in the kernel. I get the same error:

ERROR: could not open directory /home/sujith/dev/wireless-testing/build/tar-install/usr/lib/modules/99.98.3.5.0-rc5-wl: No such file or directory

I understand the /usr move, so is there any clean way to handle this breakage ?

Sure, see my fix in the first reply to the OP.

Offline

#21 2012-07-07 13:11:03

sacamano_m82
Member
Registered: 2007-05-29
Posts: 167

Re: You knew it was coming: kernel build issues after kmod upgrade

falconindy wrote:

Sure, see my fix in the first reply to the OP.

That fix works only when "modules_install" is invoked from the root Makefile. The script 'buildtar' is broken - http://tinyurl.com/bpbkrxx
(/lib is assumed to be default installation path). And this script is part of the kernel ! I am sure that more scripts/build-setups out there in the wild would break too..

Fixing 'buildtar' doesn't solve the issue, though. scripts/depmod.sh appends an extra "usr" to INSTALL_MOD_PATH (when invoked using make tar-pkg).
A couple of hacks later, I can finally get linux-foo.tar smile

But, I am using the testing repo after all, so the onus is on me to workaround this issue, I guess, until /lib becomes a tiny footnote.

Offline

#22 2012-07-07 13:42:02

falconindy
Developer
From: New York, USA
Registered: 2009-10-22
Posts: 4,097
Website

Re: You knew it was coming: kernel build issues after kmod upgrade

sacamano_m82 wrote:
falconindy wrote:

Sure, see my fix in the first reply to the OP.

That fix works only when "modules_install" is invoked from the root Makefile. The script 'buildtar' is broken - http://tinyurl.com/bpbkrxx

No... the error is being generated from the same depmod.sh script which is called in the temporary root that is created by Kbuild when you call tar-pkg. The same fix of removing the hack in this script will fix your tarball creation.

Last edited by falconindy (2012-07-07 13:42:21)

Offline

#23 2012-07-07 14:07:31

sacamano_m82
Member
Registered: 2007-05-29
Posts: 167

Re: You knew it was coming: kernel build issues after kmod upgrade

falconindy wrote:

No... the error is being generated from the same depmod.sh script which is called in the temporary root that is created by Kbuild when you call tar-pkg. The same fix of removing the hack in this script will fix your tarball creation.

The sequence goes something like this:

In a clean, vanilla tree, make tar-pkg borks with this message (the same one the OP reported):

ERROR: could not open directory /home/sujith/dev/wireless-testing/build/tar-install/usr/usr/lib/modules/99.98.3.5.0-rc5-wl: No such file or directory

After adding the fix you posted:

ERROR: could not open directory /home/sujith/dev/wireless-testing/build/tar-install/usr/lib/modules/3.5.0-rc5-wl: No such file or directory

After fixing the 'buildtar' script to use '/usr' as the install-prefix (note the extra 'usr'):

ERROR: could not open directory /home/sujith/dev/wireless-testing/build/tar-install/usr/usr/lib/modules/3.5.0-rc5-wl: No such file or directory

After a dirty hack using dirname to strip the extra 'usr' (at this point, I just wanted to move on :-) ):

Tarball successfully created in /home/sujith/dev/wireless-testing/build/linux-3.5.0-rc5-wl.tar

Hope that clears things a bit !

Offline

#24 2012-07-07 14:22:21

falconindy
Developer
From: New York, USA
Registered: 2009-10-22
Posts: 4,097
Website

Re: You knew it was coming: kernel build issues after kmod upgrade

Ah the buildtar script is indeed separately broken.

Offline

#25 2012-07-14 06:18:21

sacamano_m82
Member
Registered: 2007-05-29
Posts: 167

Re: You knew it was coming: kernel build issues after kmod upgrade

Just a quick update: I updated all of my machines (about 8 systems, using the testing repo) and didn't encounter any nasty glitch. The /lib migration went surprisingly smooth, I didn't end up shooting myself in the foot or anything. The ARCH developers have done a good job, indeed.

Offline

Board footer

Powered by FluxBB