filesystem Update Error /bin exists in filesystem

instantaphex · 2013-07-04 05:48:29

ah ok. So here is the output of

$ pacman -Sl catalyst | grep installed | awk ' { print $2 }' | pacman -Ql - | grep ' /s\?bin/\| /usr/sbin/'
catalyst-utils /usr/sbin/
catalyst-utils /usr/sbin/amdnotifyui
catalyst-utils /usr/sbin/atieventsd
catalyst-utils /usr/sbin/atigetsysteminfo.sh

EDIT: Btw I really appreciate you taking the time to respond. Looks like you've been quite the vanguard of n00b helping in this particular thread. Much appreciated.

Last edited by instantaphex (2013-07-04 05:50:13)

Scimmia · 2013-07-04 05:52:56

So catalyst-utils is a problem that needs to be dealt with. If it hasn't already been fixed by the repo maintainer, you'll need to rebuild it yourself. You can run pacman -Sy catalyst-utils and see if it upgrades. If it does, run that command again to check and see if it's been fixed.

Yes, I'm recommending pacman -Sy, but only in this instance and only because a full system upgrade is going to happen immediately afterwards.

instantaphex · 2013-07-04 05:58:38

Ok so I updated (it updated but with some weird disclaimer that fglrx had some weird new but and it included a workaround, which I implemented, another topic for another thread perhaps). I then ran:

pacman -Sl catalyst | grep installed | awk ' { print $2 }' | pacman -Ql - | grep ' /s\?bin/\| /usr/sbin/'

There was no output this time. Same with substituting catalyst for xorg113 in the above. Good to go?

Scimmia · 2013-07-04 05:59:47

Good to go, move on to step 2.

instantaphex · 2013-07-04 06:03:20

Awesome. I don't use IgnorePkg or group. On to step 3:

$ find /bin /sbin /usr/sbin -exec pacman -Qo -- {} + >/dev/null
$

Looks like I'm good to go there as well.

Step 4. I don't use autofs.

I suppose I'll go onto step 5. Keeping my fingers crossed that my system doesn't get hosed...

instantaphex · 2013-07-04 06:23:19

Well here I am post step 5 and 1 reboot later. Hats off to you good sir. I really appreciate you taking the time to help.

zero-giulio · 2013-07-05 11:13:21

Ok, I can't follow official instruction to update Arch.

Long story...
Basically, there should be something wrong on my system because pacman can't tell me who owns files on /sbin /usr/sbin etc directories
(pacman -Qo does not work)

If a following the instruction, after the
pacman -Syu -ignore filesystem,bash
I still have /sbin, /usr/sbin and /bin full of files, but I can't discover which packages need to be fixed.
They are not AUR (all my AUR packages were already uninstalled, I will reinstall them after the update, they are not very important), and I do not use non-official repos.

I'm looking for a different way to move binaries into /usr/bin.
I know this is not the recommended way, but actually I have no choice (other than reinstall Arch from zero, but I have no enough time in this period).

I'm looking for something like: copy file, remove directory, manually set symlinks.

I discovered Manjaro developed a script for such task.
I guess it should be the one I have copied/pasted below.

Do you think something like this can work still in Arch?
Can I run such script (with little editing) without destroying everything?

err() {
    ALL_OFF="\e[1;0m"
    BOLD="\e[1;1m"
    RED="${BOLD}\e[1;31m"
	local mesg=$1; shift
	printf "${RED}==>${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2
}

msg() {
    ALL_OFF="\e[1;0m"
    BOLD="\e[1;1m"
    GREEN="${BOLD}\e[1;32m"
	local mesg=$1; shift
	printf "${GREEN}==>${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2
}



post_upgrade() {
	# Update filesystem to /usr/bin
	if [ "$(vercmp $2 20130606-1)" -lt 0 ] && [ ! -L "/bin" ] && [ ! -L "/sbin" ] && [ ! -L "/usr/sbin" ]; then
		msg "Binaries move to /usr/bin ..."
		msg "Please be patient."
		local files="$(find /bin/ /sbin/ /usr/sbin/ | tr "\n" " ")"

		# List unowned files
		for file in $files
		do
			local filename="${file##*/}"
			if [ "${filename}" == "" ]; then continue; fi

			pacman -Qo "${file}" &>/dev/null
			if [ $? -ne 0 ]; then msg "Moving unowned file '${file}' to '/usr/bin'"; fi
		done

		# Update database
		local pkgs="$(pacman -Qqo /bin /sbin /usr/sbin | sort -u | tr "\n" " ")"
		for pkg in ${pkgs}
		do
			if [ "${pkg}" == "filesystem" ]; then continue; fi
			local path="/var/lib/pacman/local/$(pacman -Q ${pkg} | sed 's/ /-/g')/files"
			if [ ! -f "${path}" ]; then continue; fi

			sed -i -e 's|^bin/|usr/bin/|' -e 's|^sbin/|usr/bin/|' -e 's|^usr/sbin/|usr/bin/|' -e 's|^bin$|usr/bin|' -e 's|^sbin$|usr/bin|' -e 's|^usr/sbin$|usr/bin|' "${path}"
		done

		# Move files
		for file in $files
		do
			local filename="${file##*/}"
			if [ "${filename}" == "" ]; then continue; fi

			# Check if file is a symlink
			if [ -L "${file}" ]; then
				local filelink="$(readlink "${file}")"

				# Check if destination file exists. Otherwise move the symlink.
				if [ -e "/usr/bin/${filename}" ]; then
					# Remove link
					unlink "${file}"
				elif [[ "${file}" =~ ^"/bin/".*|^"/sbin/".* ]] && [[ "${filelink}" =~ ^"../usr/bin/".* ]]; then
					# Move link to /usr/bin and update relative path
					filelink="$(echo "${filelink}" | sed 's|^\.\./usr/bin/||')"
					ln -s "${filelink}" "/usr/bin/${filename}"
					unlink "${file}"
				elif [[ "${file}" =~ ^"/bin/".*|^"/sbin/".* ]] && [[ "${filelink}" =~ ^"../".* ]]; then
					# Move link to /usr/bin and update relative path
					filelink="$(echo "${filelink}" | sed 's|^\.\./|\.\./\.\./|')"
					ln -s "${filelink}" "/usr/bin/${filename}"
					unlink "${file}"
				elif [[ "${file}" =~ ^"/usr/sbin/".* ]] && [[ "${filelink}" =~ ^"../bin/".* ]]; then
					# Move link to /usr/bin and update relative path
					filelink="$(echo "${filelink}" | sed 's|^\.\./bin/||')"
					ln -s "${filelink}" "/usr/bin/${filename}"
					unlink "${file}"
				else
					# Move link as it is
					ln -s "${filelink}" "/usr/bin/${filename}"
					unlink "${file}"
				fi
			else
				if [ -L "/usr/bin/${filename}" ]; then
					unlink "/usr/bin/${filename}"
					mv "${file}" "/usr/bin/"
				elif [ -e "/usr/bin/${filename}" ]; then
					err "'/usr/bin/${filename}' already exists! Moving file to '/usr/bin_duplicates/${filename}'"
					if [ ! -d "/usr/bin_duplicates" ]; then mkdir "/usr/bin_duplicates"; fi
					mv "${file}" "/usr/bin_duplicates/"
				else
					mv "${file}" "/usr/bin/"
				fi
			fi
		done

		# Fix /usr/bin/init symlink
		ln -sf "../lib/systemd/systemd" "/usr/bin/init"

		# filesystem should own /bin, /sbin and /usr/sbin
		local filesystem_path="/var/lib/pacman/local/$(pacman -Q filesystem | sed 's/ /-/g')/files"
		sed -i 's|^\%FILES\%|\%FILES\%\nbin\nsbin\nusr/sbin|' "${filesystem_path}"

		# Remove directories and create symlinks
		rm -fr /bin
		rm -fr /sbin
		rm -fr /usr/sbin
		ln -s usr/bin /bin
		ln -s usr/bin /sbin
		ln -s bin /usr/sbin

		msg "Now update your system."
	fi

	# Remove obsolete version file of manjaro-system
	if [ -f /var/lib/manjaro-system/version ]; then
		rm /var/lib/manjaro-system/version
	fi
}

Last edited by zero-giulio (2013-07-05 12:21:01)

Trilby · 2013-07-05 12:29:15

zero-guilio, we can't help you unless you tell us what these files are. Are these things you put there? What is the *actual output* or errors from the commands in the instructions.

As for needing to copy or move files - assuming that these really are complete untracked files that you have put there - then I don't know why you'd need a script, just cp, or mv. Or much better yet, if you don't recognize these files: rm. But before really recommending any of these, you need to show actual output.

Stop trying to creatively do things different from the directions, this is what got you into trouble a couple weeks ago. We can help, but only if you do things the recommended way.

I just read through that script ... that seems to have dozens of commands to do next to nothing. And the few things it does are probably not a good idea.

Stop looking for short cuts. Doing this upgrade properly should only take a few minutes, but only if you don't cause other problems by thinking you know better than the devs who wrote the news item.

Last edited by Trilby (2013-07-05 12:33:13)

zero-giulio · 2013-07-05 13:16:29

I can't give you a list of files which remain in /sbin /bin /usr/sbin directory as I should try to update again.
Actually I'm sticky at 23 May (I restore a clonezilla backup).

But surely the files which remain in /sbin /bin /usr/sbin are not created by me, and they are not from AUR nor non-official repos.
They are pretty important files, for example "depmod" (the one I remember, because without it the command mkinitcpio -p linux fails).

The story is not short, and please do not expect that in a few minutes somebody can update my system, because it's not so simple as you expect.

The point is that after having restored the backup of root partition pacman has something wrong (for same reason).
There are a number of program which "are functioning", but which for pacman "are not installed".

Example: firefox.
I have firefox. It works. But "pacman -Q firefox" tells me that firefox was not installed.
And, as firefox, a number of other packages (almost all).

When I restore a clonezilla backup, the system is perfectly functioning, but pacman do not "see" a number of packages.

This is a problem, because when I run a complete update, there are a number of packages which should be updated but are not present in the list generated automatically by pacman.
Example: "pacman -Su" does not include firefox, even if firefox is present and should be updated.

This is clearly a problem, as a number of packages with files in /sbin /bin and /usr/sbin were not updated.

The problem is that I can't manually update the corresponding packages because "pacman -Qo" does not work (it can't tell me which package own that particularly file; example: which package I need to update in order to fix depmod?).

I can't reproduce exactly the problem because I should try another update which will again broke my system.
But, believe me, the number of files which remain in those directory are big and they are all important files (as depmod, for example) which cannot be removed (they are not old and useless stuff).
And if a try a pacman -Qo on that files I obtain nothing.

I do not know if I explain correctly my problem.
The problem is that the system "works", in the sense that everything work ("all program" run without problem).
But for pacman only a "subset" of those are installed.

This is the reason why I'm looking for another way to update the system.
The script of my previous post should be a viable alternative.
It's not the official Arch way, I know, but it's used in the official Manjaro update (which should be a distro 100% compatible, more or less).
It can't be a very dangerous way to update, even if it is not the official one.

Could you spend a few minutes to read such script and to tell me if I can use it?

Even better, which lines should I edit?

Last edited by zero-giulio (2013-07-05 13:35:22)

Lone_Wolf · 2013-07-05 13:53:41

zero-giulio ,

your description suggest strongly there's something VERY WRONG with the pacman databases in that backup.

moving files won't solve that.
did you include the full path to the file in pacman -Qo ?

what is the output of pacman -Qi firefox ?

Trilby · 2013-07-05 13:54:10

As I said above, I read the script - the whole thing is ridiculous. the only *potentially* functional part of that script is one command `mv` - but that to would be a bad idea. I know this will not take just a few minutes on your system - because first you have to undo all the things you've done over the past weeks that were not recommended.

When you restore your backup, do you also restore /var/?

Last edited by Trilby (2013-07-05 13:55:30)

WorMzy · 2013-07-05 14:04:20

It sounds like you've messed up your pacman database, and/or have been installing programs with the traditional "./configure; make; make install", with --prefix=/usr passed to the configure script.

If it's the latter, you're on your own. I suggest you "make uninstall" where possible, and remove untracked files created by the "make install" process manually where it is not.

If it's the former, then there are things you can do: https://wiki.archlinux.org/index.php/Pa … l_database

Last edited by WorMzy (2013-07-05 14:05:32)

zero-giulio · 2013-07-05 14:24:15

@Trilby:
No, maybe the problem is that.
/var (as /home and /boot) was not touched.
Only / was restored.

@WorMzy:
I never installed with the configure/make/etc commands, so this is not the point.
I also thing that the problem is the local pacman database. I'll try to restore it as described in your link, thanks.

@Lone_Wolf:
I never try "pacman -Qi", so I do not know the corresponding output on firefox.
I can try this evening, but actually firefox is installed: after having restored the partition, the first thing I did was to run a bunch of "pacman -Sf <something>" (basically one for each application in my start menu, plus java, linux, lxde and other "famous and well-know" things) in order to let pacman know that such packages are present.
(pacman -S, without the -force option, did not work because, as you expect, "the file are already present in filesystem")
I should find some other packages which are present but not for pacman.
Surely such packages still exist on my system, but I should spend some time searching them.
I'll eventually let you know the output of "pacman -Qi" on such packages.

Last edited by zero-giulio (2013-07-05 14:25:31)

Trilby · 2013-07-05 14:47:05

zero-giulio wrote:

@Trilby:
No, maybe the problem is that.

*headdesk*

Yeah, maybe. (edit: this was sarcasm, this was definitely the problem as it was the last time you did this).

Stop trying to take short cuts. Restore your *entire system* to the backup pre-silliness from a couple weeks ago, then *follow the directions* in the news.

Remember this: https://bbs.archlinux.org/viewtopic.php … 8#p1288128

You've been given the proper directions *many* times. Why you think you can creatively come up with better solutions is beyond me. You've created your own problems, ask for help, you are given the exact information you need, but you choose to completely ignore it and do something else. Why come back asking for more help?

Last edited by Trilby (2013-07-05 14:52:25)

zero-giulio · 2013-07-05 15:47:26

Believe me, man, I'm not so creative as you think...

This was not the first time I restored a clonezilla backup in such way (i.e.: restoring only the root), so the problem on pacman db was already present when I tried for the first time to update the system with the last new (the /usr/bin move...).

This is the reason why the "official guide" on the homepage did not work and my update failed.
I never thought to come up with solution better than official ones.
Simply, I searched for something else because the official ones does not work in my case.

Now that the problem was discovered (the pacman database), I can try to solve it (with the link of WorMzy or by restoring both / and /var as you suggest) first, and then re-try with the official guidelines.

As for the:

@zero (RE: reinstall), you can definitely leave your home partition untouched - and should in fact keep it not mounted. The boot partition you could also leave alone, though you'll likely need to touch up your bootload config after the reinstall.
The var partition, however, will definitely need to be mounted for pacman to run. In fact, the var partition is exactly where you current problem exists: that's where pacman's database is, and where the package cache is held. Reinstalling without access to the var partition could actually get you in the exact perdicament you now find yourself in: none of the installed files would be tracked by pacman.

I misunderstood your answer.
I thought it was relative only to the hypothesis of a new installation of Arch from zero (which actually did not took place).
I didn't realize that it apply also to the case of a backup restoration.

My fault, of course, but not in bad faith.

Last edited by zero-giulio (2013-07-05 15:50:51)

idet2 · 2013-07-10 16:37:28

Hi! When I tried to install a package from AUR I got the same message.

My problem is that this is a production server and the first commands outputs many-many packages

# pacman -Qqo /bin /sbin /usr/sbin | pacman -Qm -
error: cannot determine ownership of directory '/bin'
error: cannot determine ownership of directory '/sbin'
error: cannot determine ownership of directory '/usr/sbin'
cloog-ppl 0.15.10-2
dbus-core 1.4.16-1
dcron 4.5-2
gcc-gcj 4.6.2-2
gen-init-cpio 2.6.36-1
gnupg2 2.0.18-1
hd2u 1.0.3-3
initscripts 2011.06.4-1
kernel26 2.6.36.1-3
libfetch 2.33-3
libmysqlclient 5.5.14-2
libusb 1.0.8-1
md5deep 3.7-1
module-init-tools 3.16-1
mysql 5.5.14-2
mysql-clients 5.5.14-2
pdftk 1.44-4
perl-unicode-normalize 1.16-1
perl-unicode-string 2.09-6
php-suhosin 0.9.32.1-1
procps 3.2.8-3
sqlite3 3.7.7.1-1
sysvinit 2.88-2
tcp_wrappers 7.6-12
udev 171-2

Does this mean that I have to uninstall all of the above and reinstall them after the upgrade?

I am very concern because this might mean that the server will be off for a few days until everything is back to normal.

What will happen to all the processes that are running?

All the best

Trilby · 2013-07-10 16:43:33

idet - a majority of those packages have been covered in this thread. But basically, yes, you should uninstall them, then reinstall them after the upgrade completes. I don't know why you think this would take a few days - as long as you do it right, it should take a few minutes, maybe tens of minutes if you really are going to reinstall all those aur packages (many of them you probably shouldn't).

But removing a package will not stop any currently running processes - just the next time they are launched (or when you reboot) the new version will be used.

Are you really still using kernel26? That's not even in the AUR, which suggests that you just had it installed when it was in the repos. If you haven't updated since then (years?) then you may want to reconsider using arch on that machine. Edit: that's not even the LTS pacakge ... the last time that was in the repos was 2 and half years ago. Has it been that long since your last update? That was right about the time you registered to the forums ... have you ever updated since you installed? If you haven't, there are *many* required manual intervention steps between then and now in the news. The general concensus has been with a system that out of date, a reinstallation might be the only sane way to handle it - and this was even before the latest filesystem update.

Last edited by Trilby (2013-07-10 17:03:49)

WonderWoofy · 2013-07-10 18:23:11

It would seem that idet2 has been partially updating his/her system for years. I am surprised that this has not lead to some really serious breakage. I see in that list that udev is still present, as is dbus-core. There are certainly others (gen-init-cpio?!), but just to exemplify the situation, those two packages not only odn't exist in the repos any longer, but actually have been replaced by other packages which provide their functionality. So there is no way that idet2 could be regularly running an -Syu.

Idet2, it would seem that you have some work to do. It is probably advisable to read through some of this thread. It repeats itself, so you don't have to read the whole thing. But you might run into trouble since there is the very real possibility that you have not gone through some of the harrier news items yet.

Scimmia · 2013-07-10 19:08:22

udev 171 puts the last full update at somewhere around the middle of 2011. Honestly, idet, updating is going to be much more difficult and take longer than just reinstalling.

idet2 · 2013-07-11 06:28:05

Thanks for all your replies!

Truth is that I am partially and manually updating my system for all this time.
That's the reason why in my first post I wrote that this will take for days! Because I knew that they might appear major problems!
I probably have to consider reinstallation from scratch as suggested...

Thanks a lot for your help!

jasonwryan · 2013-07-11 10:05:26

Unstickying. What will forever be known as Scimmia's borked bin thread is officially DONE.

geo909 · 2013-07-11 14:40:46

Hi all,

Another lost soul here due to a user error. I was wondering if
anybody could help. I read around but I'm lost and don't want to
try anything until I'm sure I won't make things worse.

I read the front page and started following the instructions. First,

 pacman -Qqo /bin /sbin /usr/sbin | pacman -Qm -

gave nothing. Initially it was showing the old grub but I installed
grub2 and it was fine, no problems.

I am not using any unofficial repositories so I skipped the
corresponding line on the front page. I'm also not using testing.

Then:

 $ find /bin /sbin /usr/sbin -exec pacman -Qo -- {} + >/dev/null

gave two files in sbin: startMobilePartner and usbmod. These were both (I
think) from an old time when I had manually installed the drivers of a usb
modem. This has been obsolete to me since a very long time ago, so I chose
to ignore it (sigh) at a lazy and naive moment. I'm assuming this is what
causes me problems now.

Then I did

# pacman -Syu --ignore filesystem,bash
# pacman -S bash

No problems at all.

But now, when I do pacman -Su I get the usual "sbin exists in filesystem"
error:

jorge@flamingo:/sbin$ sudo pacman -Su
:: Starting full system upgrade...
resolving dependencies...
looking for inter-conflicts...

Packages (1): filesystem-2013.05-2

Total Installed Size:   0.01 MiB
Net Upgrade Size:       -0.30 MiB

:: Proceed with installation? [Y/n] 
(1/1) checking keys in keyring                     [#########################] 100%
(1/1) checking package integrity                   [#########################] 100%
(1/1) loading package files                        [#########################] 100%
(1/1) checking for file conflicts                  [#########################] 100%
error: failed to commit transaction (conflicting files)
filesystem: /sbin exists in filesystem
Errors occurred, no packages were upgraded.

I backed up and deleted the files /sbin/startMobilePartner and /sbin/usbmod but I still get that
error. I'm afraid to do anything manually to /sbin as I read people had various problems trying
various things. I'm also afraid to reboot. I prefer to stop doing anything unless I hear from
somebody who knows..

So this is my situation, I'm hoping that there's still hope for a relatively easy fix. Any ideas?

Many thanks in advance.

Scimmia · 2013-07-11 14:45:53

If /sbin is empty, delete the directory. If it's not, tell us what's left.

Your instincts on rebooting were good. Do not reboot until you finish the upgrade

geo909 · 2013-07-11 15:12:34

Scimmia wrote:

If /sbin is empty, delete the directory. If it's not, tell us what's left.
Your instincts on rebooting were good. Do not reboot until you finish the upgrade

Thanks for the feedback!

Inside sbin there used to be two files called usbmod and startMobilePartner (I'm very sure
that both of those have to do with some drivers that I had manually installed a long time ago
and I don't use anymore). Now I backed up both of these and deleted them from sbin, so it's
empty, but because I manually deleted the files. Should I go on and delete the directory?

Last edited by geo909 (2013-07-11 15:14:44)

Scimmia · 2013-07-11 15:18:59

Yes, delete the empty directory.

Arch Linux

#776 2013-07-04 05:48:29

Re: filesystem Update Error /bin exists in filesystem

#777 2013-07-04 05:52:56

Re: filesystem Update Error /bin exists in filesystem

#778 2013-07-04 05:58:38

Re: filesystem Update Error /bin exists in filesystem

#779 2013-07-04 05:59:47

Re: filesystem Update Error /bin exists in filesystem

#780 2013-07-04 06:03:20

Re: filesystem Update Error /bin exists in filesystem

#781 2013-07-04 06:23:19

Re: filesystem Update Error /bin exists in filesystem

#782 2013-07-05 11:13:21

Re: filesystem Update Error /bin exists in filesystem

#783 2013-07-05 12:29:15

Re: filesystem Update Error /bin exists in filesystem

#784 2013-07-05 13:16:29

Re: filesystem Update Error /bin exists in filesystem

#785 2013-07-05 13:53:41

Re: filesystem Update Error /bin exists in filesystem

#786 2013-07-05 13:54:10

Re: filesystem Update Error /bin exists in filesystem

#787 2013-07-05 14:04:20

Re: filesystem Update Error /bin exists in filesystem

#788 2013-07-05 14:24:15

Re: filesystem Update Error /bin exists in filesystem

#789 2013-07-05 14:47:05

Re: filesystem Update Error /bin exists in filesystem

#790 2013-07-05 15:47:26

Re: filesystem Update Error /bin exists in filesystem

#791 2013-07-10 16:37:28

Re: filesystem Update Error /bin exists in filesystem

#792 2013-07-10 16:43:33

Re: filesystem Update Error /bin exists in filesystem

#793 2013-07-10 18:23:11

Re: filesystem Update Error /bin exists in filesystem

#794 2013-07-10 19:08:22

Re: filesystem Update Error /bin exists in filesystem

#795 2013-07-11 06:28:05

Re: filesystem Update Error /bin exists in filesystem

#796 2013-07-11 10:05:26

Re: filesystem Update Error /bin exists in filesystem

#797 2013-07-11 14:40:46

Re: filesystem Update Error /bin exists in filesystem

#798 2013-07-11 14:45:53

Re: filesystem Update Error /bin exists in filesystem

#799 2013-07-11 15:12:34

Re: filesystem Update Error /bin exists in filesystem

#800 2013-07-11 15:18:59

Re: filesystem Update Error /bin exists in filesystem

Board footer