[SOLVED] Arch repo has invalid symlinks in it

junn · 2017-08-24 10:46:58

Hi,

im rsyncing the arch repo onto my machine but this has stopped working a few days ago. I guess it is a broken symlink some where in the repo:

rsync -rtlH --delete-after --delay-updates --copy-links --safe-links "rsync://mirror.23media.de/archlinux/" "/data/arch"

symlink has no referent: "/community/os/i686/spyder3-3.1.4-2-any.pkg.tar.xz" (in archlinux)
symlink has no referent: "/community/os/i686/spyder3-3.1.4-2-any.pkg.tar.xz.sig" (in archlinux)
symlink has no referent: "/community/os/x86_64/spyder3-3.1.4-2-any.pkg.tar.xz" (in archlinux)
symlink has no referent: "/community/os/x86_64/spyder3-3.1.4-2-any.pkg.tar.xz.sig" (in archlinux)

The `--copy-links` argument is mandatory as the system im feeding the files into do not support symlinks.

Any ideas how to fix or at least workaround these problems (ignoring not existent files)?

Last edited by junn (2017-09-04 14:08:04)

c00ter · 2017-08-24 16:08:48

Do you think this news header from the archlinux.org homepage may be impacting your operation?: https://www.archlinux.org/news/deprecation-of-abs/

regards

Slithery · 2017-08-24 16:16:32

c00ter wrote:

Do you think this news header from the archlinux.org homepage may be impacting your operation?: https://www.archlinux.org/news/deprecation-of-abs/
regards

Nope. He's not trying to access the old ABS rsync endpoint, he's rsyncing with a remote repo.

Just for reference this is the mirror sync script I'm currently using with no issues, the target is a linux filesystem in my case though.

#!/bin/sh
# Sync local Arch Linux repository

mirror="rsync://mirrors.uk2.net/archlinux/"
local="/mnt/data/repo/"

rsync \
  --no-motd \
  --recursive \
  --links \
  --safe-links \
  --hard-links \
  --times \
  --delete-after \
  --delay-updates \
  --delete-excluded \
  --exclude="**i686*" \
  $mirror \
  $local

Last edited by Slithery (2017-08-24 16:16:51)

Muflone · 2017-08-24 20:58:16

Hi

I can confirm there were some issues with the db-update script for the community repository about the Spyder packages. It confused itself when I removed the spyder pkgname and it removed the whole spyder pkgbase.

I've fixed the links manually in the main repo at least 10 days ago, so I don't know why you still receive such broken links.
Maybe your repository is not fully updated or for some reason you downloaded a broken link.

Please first try to change your mirror.

junn · 2017-08-25 11:44:20

@slithery and @Muflone

I think the problem lies in "--copy-links" (or actually: it makes it visible).

from the man page:

--copy-links
When symlinks are encountered, the item that they point to (the referent) is copied, rather than the symlink. [...]
--safe-links
This tells rsync to ignore any symbolic links which point outside the copied tree. All absolute symlinks are also ignored. [...]

So "--safe-links" checks for symlinks referring outside of the dest dir but does not check for invalid symlinks. "--copy-links" checks (obviously) for invalid links (as it needs to copy the symlinked file). I think that's the reason why it's working for slithery but isn't for me.

i tried these mirrors:

>>> rsyncing from `rsync://mirror.f4st.host/archlinux/`...
symlink has no referent: "/community/os/i686/spyder3-3.1.4-2-any.pkg.tar.xz.sig" (in archlinux)
symlink has no referent: "/community/os/i686/spyder3-3.1.4-2-any.pkg.tar.xz" (in archlinux)
symlink has no referent: "/community/os/x86_64/spyder3-3.1.4-2-any.pkg.tar.xz.sig" (in archlinux)
symlink has no referent: "/community/os/x86_64/spyder3-3.1.4-2-any.pkg.tar.xz" (in archlinux)
IO error encountered -- skipping file deletion
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1650) [generator=3.1.2]
>>> rsync failed
>>> rsyncing from `rsync://pkg.adfinis-sygroup.ch/archlinux/`...
symlink has no referent: "/community/os/i686/spyder3-3.1.4-2-any.pkg.tar.xz.sig" (in archlinux)
symlink has no referent: "/community/os/i686/spyder3-3.1.4-2-any.pkg.tar.xz" (in archlinux)
symlink has no referent: "/community/os/x86_64/spyder3-3.1.4-2-any.pkg.tar.xz.sig" (in archlinux)
symlink has no referent: "/community/os/x86_64/spyder3-3.1.4-2-any.pkg.tar.xz" (in archlinux)
IO error encountered -- skipping file deletion
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1650) [generator=3.1.2]
>>> rsync failed
>>> rsyncing from `rsync://mirror.23media.de/archlinux/`...
symlink has no referent: "/community/os/i686/spyder3-3.1.4-2-any.pkg.tar.xz" (in archlinux)
symlink has no referent: "/community/os/i686/spyder3-3.1.4-2-any.pkg.tar.xz.sig" (in archlinux)
symlink has no referent: "/community/os/x86_64/spyder3-3.1.4-2-any.pkg.tar.xz" (in archlinux)
symlink has no referent: "/community/os/x86_64/spyder3-3.1.4-2-any.pkg.tar.xz.sig" (in archlinux)
IO error encountered -- skipping file deletion
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1650) [generator=3.1.2]
>>> rsync failed

junn · 2017-08-25 17:50:57

i used slithery script for that (- the exclude).

they are all broken/point to nonexistent files:

3808175 4 lrwxrwxrwx 1 root root 54 Jul 9 19:03 ./community/os/i686/spyder3-3.1.4-2-any.pkg.tar.xz -> ../../../pool/community/spyder3-3.1.4-2-any.pkg.tar.xz
3808176 4 lrwxrwxrwx 1 root root 58 Jul 9 19:03 ./community/os/i686/spyder3-3.1.4-2-any.pkg.tar.xz.sig -> ../../../pool/community/spyder3-3.1.4-2-any.pkg.tar.xz.sig
3820139 4 lrwxrwxrwx 1 root root 54 Jul 9 19:03 ./community/os/x86_64/spyder3-3.1.4-2-any.pkg.tar.xz -> ../../../pool/community/spyder3-3.1.4-2-any.pkg.tar.xz
3820140 4 lrwxrwxrwx 1 root root 58 Jul 9 19:03 ./community/os/x86_64/spyder3-3.1.4-2-any.pkg.tar.xz.sig -> ../../../pool/community/spyder3-3.1.4-2-any.pkg.tar.xz.sig

Last edited by junn (2017-08-25 17:51:47)

Muflone · 2017-09-03 22:25:54

For the records, I've fixed the repository broken links.
If you can please check it yourself if the issue was solved.

Last edited by Muflone (2017-09-03 22:26:18)

junn · 2017-09-04 14:08:34

Muflone wrote:

For the records, I've fixed the repository broken links.
If you can please check it yourself if the issue was solved.

Yes, thank you.

Arch Linux

#1 2017-08-24 10:46:58

[SOLVED] Arch repo has invalid symlinks in it

#2 2017-08-24 16:08:48

Re: [SOLVED] Arch repo has invalid symlinks in it

#3 2017-08-24 16:16:32

Re: [SOLVED] Arch repo has invalid symlinks in it

#4 2017-08-24 20:58:16

Re: [SOLVED] Arch repo has invalid symlinks in it

#5 2017-08-25 11:44:20

Re: [SOLVED] Arch repo has invalid symlinks in it

#6 2017-08-25 17:50:57

Re: [SOLVED] Arch repo has invalid symlinks in it

#7 2017-09-03 22:25:54

Re: [SOLVED] Arch repo has invalid symlinks in it

#8 2017-09-04 14:08:34

Re: [SOLVED] Arch repo has invalid symlinks in it

Board footer