[solved] How Freq is "ibiblio" updated? --> Issue on other mirror site

dboat · 2010-08-30 06:42:52

I have built a private local mirror that rsync with the captioned server. When using it, I find that pacman -S still need to access to other mirror quite frequently. There was once I suspected that I had not completed the rsync successfully. Then I picked several packages that were downloaded from other mirrors, check with my Local Mirror, and find that an older version do exist right on my harddisk !

Is it something supposed to be? Or, it rarely happens but I am just so unlucky that I encounter it during my first setup, and it will be fixed soon?

Last edited by dboat (2010-09-01 15:33:00)

karol · 2010-08-30 12:47:32

Those are up-to-date mirrors: https://www.archlinux.de/?page=MirrorStatus Pick some of them.
Because of http://mailman.archlinux.org/pipermail/ … 17431.html some mirrors may have stopped syncing.

wonder · 2010-08-30 13:03:00

http://wiki.archlinux.org/index.php/Dev … NewMirrors

use tier 1 mirrors for that purpose. kernel.org from example is syncing on 10 minutes intervals

Inxsible · 2010-08-30 14:40:21

wonder wrote:

http://wiki.archlinux.org/index.php/Dev … NewMirrors
use tier 1 mirrors for that purpose. kernel.org from example is syncing on 10 minutes intervals

But isn't that "wrong", The Arch instructions to mirrors clearly states that you should not sync more frequently than once an hour.

wonder · 2010-08-30 15:01:27

no. we allowed tier 1 to sync more often because only they are doing it. in the past all mirrors were syncing directly from us but that changed.

btw i'm not suggesting that you should do the same. you should do it once an hour using some mirror

Last edited by wonder (2010-08-30 15:02:51)

dboat · 2010-08-30 18:19:22

Thanks a lot for all your helps.
The link provided by wonder on #3 is particularly useful.

BTW, I was really confused about the policy/set-up of the mirror hierarchy.
Even now, I am not sure whether it is preferred to keep this link mainly among developers or to everyone.
Suggest to have some official statement in the wiki of Local Mirror to help newbies like me, if appropriate. ;p

dboat · 2010-08-31 06:24:48

After an exercise of 7+hr of rsync, my local mirror added ~5G & removed ~4G of files. However, when I do pacman -Syu from another vm, it still reports "File unavailable".

One example is --> kernel26-2.6.35.4-1-i686.pkg.tar.xz
I can locate and find it in /home/mirror/files/core/os/i686 and it shows there

lrwxrwxrwx 1 mirror mirror 58 Aug 30 00:32 kernel26-2.6.35.4-1-i686.pkg.tar.xz -> ../../../pool/packages/kernel26-2.6.35.4-1-i686.pkg.tar.xz

so I have only rsync a link that points to a folder "pool" which does not exist on my harddisk ^_^!!
The tier 1 mirror I used is : rit.edu (USA) - rsync://mirror.rit.edu/archlinux/

What should I do?
rsync with another tier 1 mirror?
figure out the folder structure at rit.edu, adopt it, and add other rsync command to cater this part?

Sorry for my poor English cause it may sound rude. I do believe those hosts of different mirror sites have their problems & limitations to face and handle. I also believe there is some practices that is making the whole machine running, just unknown to newbies like me. Please give me some hints to work it out.

wonder · 2010-08-31 06:43:28

so you used that script from the local mirror wiki. that _doesnt_ sync the pool directory. that's important this days

dboat · 2010-08-31 07:53:04

Yes, I use the script from the local mirror wiki and it _doesnt_ sync the pool directory.
I amend the mirrorsync.sh as below that it only sync (core extra community) to my harddisk. I only have this 3 directories now under /home/mirror/files. I also cannot find pool as a sub-directory anywhere under it.

SYNC_REPO=(core extra community)

does this amendment make the different?

Or, should I create the pool directory (somewhere) manually and rsync again?
I dare not do blind test myself on this part.

----------------------------------------------------------------------------------------------------------
The 2nd amendment I made on mirrorsync.sh was inserting --exclude=os/x86_64 after --delete-after in the two rsync command line.

The 3rd and the last is to change the mirror from ibiblio to rit

#SYNC_SERVER=distro.ibiblio.org::distros/archlinux
SYNC_SERVER=mirror.rit.edu::archlinux

Sorry I don't actually understand the meaning of :: here
I just try to follow the original format as close as possible and map similar thing to their position.
Did I make any mistake?

Last edited by dboat (2010-08-31 08:21:48)

Pierre · 2010-08-31 08:12:34

The script from the wiki is broken and shouldn't be used by anybody. Also most users wont need a local mirror anyway.

Btw: kernel.org syncs every hour like most tier 1 mirrors. The delay you see on the status page is probably not what people expect; maybe I should just remove that value.

And last but not least one shouldn't sync from a tier 1 if not needed; they probably have enough traffic from the other mirrors. It's best to use a fast mirror nearby.

dboat · 2010-08-31 10:19:53

Would like to clarify if the key concern is not to setup a local mirror, or just not to sync from a tier 1 mirror?

My ideal setup is my pc do the rsync while I am sleeping, and I have a full set of yesterday's image next morning. I think a tier 2 mirror can serve my need, am I correct?

Btw, I don't have a mirror in my city.

Pierre · 2010-08-31 10:40:48

Yes, don't choose a mirror by ltier 1 or 2 but by location and speed. (Have a look at reflector)

I still don't think you need a local mirror for your setup. If you want to download packages in the background just run "sudo pacman -Syuw --noconfirm" at night. This will download all packages into the cache without installing them. You can then run "pacman -Su" in the morning. This should save you and the mirrors a lot of traffic.

dboat · 2010-08-31 10:55:30

Well, let me explain my story then.

I am a newbie here, and I am one-year-old in the Linux world.
None of my friend use Linux, I am on my own except Internet.
My limited Linux experience tells me that, it is easy to do something (e.g. install a package) but could be a crisis to undo it.
My tactic is to use different vm's. Sometime even start with identical vm's, make changes step-by-step, trace & monitor good and bad results and learn from between ... ... So, I have a lot of vm's. One reason I love ArchLinux is it allows me to do it fast. It's lightweight also allows me to run more vm's simultaneously on my hosting computer. Sorry I abuse it.

I create/install vm so frequently and I have many vm's to be -Syu as well. I thought it may be a good idea to have a private local mirror, so I don't need to visit other mirrors again and again for the same set of files. So, ...

Btw, so, you mean the mirror hierarchy has a 4th layer under tier 2?

dboat · 2010-09-01 05:02:01

Just want to report that rsync mirrors on the Unofficial mirror list can serve my need
I thought it would be an alternative that all parties can accept ...
then, I find that content on the local mirror wiki had been replaced ... ...
Very Sorry for those inconvenience caused.

Btw, if I had noticed the "Network Shared Pacman Cache" earlier, I might not even think about building a local mirror myself. That is, as described in the previous wiki, painful.
Anyway, sorry again.

.:B:. · 2010-09-01 14:27:20

Please use [Solved] instead of [Closed] to mark your topic as solved. Only when a topic is effectively closed (thus preventing people from posting in it) the forum software will prefix the topic title with Closed:.

It's kinda confusing .

karol · 2010-09-01 14:59:24

.:B:. wrote:

Please use [Solved] instead of [Closed] to mark your topic as solved. Only when a topic is effectively closed (thus preventing people from posting in it) the forum software will prefix the topic title with Closed:.
It's kinda confusing .

+1
My first thought was: "How did this topic got closed?"

Network Shared Pacman Cache should fit most people, that's why the local mirror wiki article wasn't really needed anymore.

Arch Linux

#1 2010-08-30 06:42:52

[solved] How Freq is "ibiblio" updated? --> Issue on other mirror site

#2 2010-08-30 12:47:32

Re: [solved] How Freq is "ibiblio" updated? --> Issue on other mirror site

#3 2010-08-30 13:03:00

Re: [solved] How Freq is "ibiblio" updated? --> Issue on other mirror site

#4 2010-08-30 14:40:21

Re: [solved] How Freq is "ibiblio" updated? --> Issue on other mirror site

#5 2010-08-30 15:01:27

Re: [solved] How Freq is "ibiblio" updated? --> Issue on other mirror site

#6 2010-08-30 18:19:22

Re: [solved] How Freq is "ibiblio" updated? --> Issue on other mirror site

#7 2010-08-31 06:24:48

Re: [solved] How Freq is "ibiblio" updated? --> Issue on other mirror site

#8 2010-08-31 06:43:28

Re: [solved] How Freq is "ibiblio" updated? --> Issue on other mirror site

#9 2010-08-31 07:53:04

Re: [solved] How Freq is "ibiblio" updated? --> Issue on other mirror site

#10 2010-08-31 08:12:34

Re: [solved] How Freq is "ibiblio" updated? --> Issue on other mirror site

#11 2010-08-31 10:19:53

Re: [solved] How Freq is "ibiblio" updated? --> Issue on other mirror site

#12 2010-08-31 10:40:48

Re: [solved] How Freq is "ibiblio" updated? --> Issue on other mirror site

#13 2010-08-31 10:55:30

Re: [solved] How Freq is "ibiblio" updated? --> Issue on other mirror site

#14 2010-09-01 05:02:01

Re: [solved] How Freq is "ibiblio" updated? --> Issue on other mirror site

#15 2010-09-01 14:27:20

Re: [solved] How Freq is "ibiblio" updated? --> Issue on other mirror site

#16 2010-09-01 14:59:24

Re: [solved] How Freq is "ibiblio" updated? --> Issue on other mirror site

Board footer