Removing the "Local Mirror" article?

stefanwilkens · 2010-09-11 22:09:50

you may be confusion proper mirrors with private mirrors here guys, T-u-N-i-X seems to maintain a proper mirror located in turkey.. listed in our mirrorlist. Not a private mirror. Am I correct to deduce this T-u-N-i-X ?

If this is the case and you require a proper script example, perhaps you should contact the devs through the mailing list (wich they more frequently use than these forums)

T-u-N-i-X · 2010-09-11 22:20:18

stefanwilkens wrote:

you may be confusion proper mirrors with private mirrors here guys, T-u-N-i-X seems to maintain a proper mirror located in turkey.. listed in our mirrorlist. Not a private mirror. Am I correct to deduce this T-u-N-i-X ?
If this is the case and you require a proper script example, perhaps you should contact the devs through the mailing list (wich they more frequently use than these forums)

Right. But I want this not just for myself but for future mirror admins. We should ease their job without forcing them to make a research or write a script for Arch Linux. We can provide a standardized script. And I made my objection here because this topic seemed to be the start and end points of the article removing process. The decision was made here as far as I see.

Last edited by T-u-N-i-X (2010-09-11 22:21:34)

stefanwilkens · 2010-09-11 22:46:23

T-u-N-i-X wrote:

stefanwilkens wrote:
you may be confusion proper mirrors with private mirrors here guys, T-u-N-i-X seems to maintain a proper mirror located in turkey.. listed in our mirrorlist. Not a private mirror. Am I correct to deduce this T-u-N-i-X ?
If this is the case and you require a proper script example, perhaps you should contact the devs through the mailing list (wich they more frequently use than these forums)
Right. But I want this not just for myself but for future mirror admins. We should ease their job without forcing them to make a research or write a script for Arch Linux. We can provide a standardized script. And I made my objection here because this topic seemed to be the start and end points of the article removing process. The decision was made here as far as I see.

It was, but the article that was removed referred to private mirrors.

Mirror admins for authorized mirrors should refer to this wiki article:
http://wiki.archlinux.org/index.php/Dev … inistrator

I have raised this issue on the bug tracker in order to attract developer attention, I agree that you make a fair point Perhaps you yourself can start a discussion about this on the mailing lists?

the related bug report / feature request is listed here:
https://bugs.archlinux.org/task/20819

Last edited by stefanwilkens (2010-09-11 22:48:43)

T-u-N-i-X · 2010-09-11 23:20:25

stefanwilkens wrote:

T-u-N-i-X wrote:
stefanwilkens wrote:
you may be confusion proper mirrors with private mirrors here guys, T-u-N-i-X seems to maintain a proper mirror located in turkey.. listed in our mirrorlist. Not a private mirror. Am I correct to deduce this T-u-N-i-X ?
If this is the case and you require a proper script example, perhaps you should contact the devs through the mailing list (wich they more frequently use than these forums)
Right. But I want this not just for myself but for future mirror admins. We should ease their job without forcing them to make a research or write a script for Arch Linux. We can provide a standardized script. And I made my objection here because this topic seemed to be the start and end points of the article removing process. The decision was made here as far as I see.
It was, but the article that was removed referred to private mirrors.
Mirror admins for authorized mirrors should refer to this wiki article:
http://wiki.archlinux.org/index.php/Dev … inistrator
I have raised this issue on the bug tracker in order to attract developer attention, I agree that you make a fair point Perhaps you yourself can start a discussion about this on the mailing lists?
the related bug report / feature request is listed here:
https://bugs.archlinux.org/task/20819

thank you! i'll write my ideas on the mailing list.. (arch general discussion)

crouse · 2010-09-19 05:27:39

Not to be a naysayer here, but I've ran a private mirror for years, and there are indeed reasons for some people to run them. My doing so has saved other mirrors ALOT of bandwidth. Just because 99.9% of the people might not need/want to, doesn't mean there aren't perfectly legitimate reasons to do so. Removing information because 99.9% of the people might not need it, doesn't seem "reasonable" to me. Perhaps mark it "OUTDATED", put the other options at the top, removing it, NO. Were do you stop that trend ? 99.9% of the people probably don't use Ranger either, so do we remove that page ????

Someone went to the trouble of donating their time/energy/etc to writing/editing the wiki page to allow the information to be readily available for people, and it is fairly presumptuous of other people to deem their contributions invalid. I submit that there are perfectly valid reasons private repos might be needed/used. I am against removing information from the wiki.

Pierre · 2010-09-19 08:34:19

I don't get why people are so upset. The previous version of the article was quite bad; so if you want to keep an article about local mirroring just improve the article and fix the known problems.

T-u-N-i-X · 2010-09-19 10:33:15

Pierre wrote:

I don't get why people are so upset. The previous version of the article was quite bad; so if you want to keep an article about local mirroring just improve the article and fix the known problems.

I guess our English is so bad that you guys cannot understand. It's not about the quality of a document or removal of it. It's about removing some important and precious information that some people might need in the future. Some guy from the forums just sent me an e-mail because he never saved the mirroring script. He thought he could always find it on the wiki but now he can't and wanted me to send mine to him.

You're telling us to improve the article and fix the problems but I can't write to the article in the wiki because it's locked. I've sent an e-mail to the mailing list and noone even responded.

Last edited by T-u-N-i-X (2010-09-19 10:34:53)

Pierre · 2010-09-19 10:49:37

You are right: I don't understand it. The old information is still there. It's a wiki with a complete history. And I don't see that the article is locked. I have also linked to a better sync script above. Just make sure to not put a specific mirror into the article.

T-u-N-i-X · 2010-09-19 10:57:44

I'm talking about this article: http://wiki.archlinux.org/index.php/Dev … NewMirrors

and it's locked. Why do you force people to look for the history? If I was a new mirror admin, I wouldn't think that the information would be in the history. It's there or not.

Your scripts might be better but they lack of documentation. And the older script works for years for me, it logs everything etc..

Pierre · 2010-09-19 11:07:47

Sorry, but this thread is not about the article n question. It's about http://wiki.archlinux.org/index.php/Local_Mirror. The DeveloperWiki-Article is not meant for users.

pyther · 2010-09-19 14:13:36

1. I agree that in some circumstances local mirrors are warranted
2. How hard is it to run rsync and look at the man page for rsync? rsync is the only command that is needed to create a mirror.

Last edited by pyther (2010-09-19 16:14:54)

pyther · 2010-09-19 19:24:47

After some discussion on the Mailing List I went and ahead and update the wiki article. I've included some basic information on setting up a local mirror. I believe it provides enough information to help someone set up local mirror while still holding them accountable for a certain amount of knowledge.

@Pierre I would appreciate it if you can review the modifications

http://wiki.archlinux.org/index.php/Local_Mirror

T-u-N-i-X · 2010-09-20 07:13:20

@pyther: can we add script samples to Local Mirror page? I can add references for both the old/new scripts so the problem will be over.

pyther · 2010-09-21 00:25:12

I personally would prefer if you didn't. The moment we start providing scripts, we will have a certain user base that will copy and paste and blindly follow the wiki. These people are very unlikely to need a local mirror. Therefore, I have provided enough pointers, so that someone who legitimately has a reason for a local mirror can set one up without to much difficulty. While a copy-and-paster would figure it is too much work and give up and maybe try one of the alternatives.

Maybe I'm missing something but what would the script do? Create a lock file? Redirect rsync output to a log file?

T-u-N-i-X · 2010-09-21 05:58:04

pyther wrote:

I personally would prefer if you didn't. The moment we start providing scripts, we will have a certain user base that will copy and paste and blindly follow the wiki. These people are very unlikely to need a local mirror. Therefore, I have provided enough pointers, so that someone who legitimately has a reason for a local mirror can set one up without to much difficulty. While a copy-and-paster would figure it is too much work and give up and maybe try one of the alternatives.
Maybe I'm missing something but what would the script do? Create a lock file? Redirect rsync output to a log file?

Yes, it does all of those. But if you just need a part of the mirror (say only i686 packages or just the core and testing) you can also do that easily by just changing an array. You can of course write all of these in a script by just looking to the manual but why lose time? Forcing the users for alternatives is against KISS and I think we should avoid doing that.

pyther · 2010-09-21 12:11:14

That script really isn't valid and it will become much more difficult to exclude core/extra/community/testing/etc.... because all the packages for these repos are now stored in /pool. Symlinks are created from pool to core, extra, testing, etc...

See: http://mailman.archlinux.org/pipermail/ … 16341.html

And this is why Pierre's quote holds true...

Pierre wrote:

I think this article should be deleted as it is no longer valid...

hokasch · 2010-09-21 12:17:24

But if you just need a part of the mirror (say only i686 packages or just the core and testing) you can also do that easily by just changing an array.

An easy way without the need for scripts is already explained on the current wiki page (here).

pyther · 2010-09-21 13:05:34

I updated the wiki article to include some information about the pool directory. I also updated the example exclude file. It should be able to exclude i686 or x86_64 packages. However, there won't be any easy to to exclude specific repos.

fukawi2 · 2010-09-21 22:41:39

pyther wrote:

That script really isn't valid and it will become much more difficult to exclude core/extra/community/testing/etc.... because all the packages for these repos are now stored in /pool. Symlinks are created from pool to core, extra, testing, etc...

pyther wrote:

However, there won't be any easy to to exclude specific repos.

Sure there is... Use --copy-links

Last edited by fukawi2 (2010-09-21 22:42:14)

pyther · 2010-09-21 22:47:02

Hmm interesting. That is good to know. Maybe someone can add that to the wiki.

fukawi2 · 2010-09-22 01:21:18

Done:
http://wiki.archlinux.org/index.php/Loc … ple_Script

pyther · 2010-09-22 01:32:46

We need to think about how we want to go about this. That script is good for only syncing core, extra, and community, but if you want to sync all the repos such as testing, community-testing, etc... it would be better to use my method. This is because with your method we'd be downloading packages twice (once for testing and once for extra).

fukawi2 · 2010-09-22 03:37:43

pyther wrote:

This is because with your method we'd be downloading packages twice (once for testing and once for extra).

Well they are 2 different packages, so probably not.... I'd imagine a common usage scenario would include a local mirror serving multiple machines; some of them using [testing] for testing new packages before deploying to production from core/extra/community.

Pierre · 2010-09-22 04:14:07

We should underline that this should not be used by public mirrors.

pyther · 2010-09-22 13:45:06

fukawi2 wrote:

pyther wrote:
This is because with your method we'd be downloading packages twice (once for testing and once for extra).
Well they are 2 different packages, so probably not.... I'd imagine a common usage scenario would include a local mirror serving multiple machines; some of them using [testing] for testing new packages before deploying to production from core/extra/community.

Say we get a new package into testing called kernel-2.40.0-1

With your method the following will happen
*Download package (into testing) when it enters testing
*Download package (into core) when it enters core
(2 Downloads)

Where as my method does the following
*Download package (into pool/packages) and creates a symlink in testing
*When it gets moved to core it deletes the symlink in testing and recreates it in extra
(1 Download)

Last edited by pyther (2010-09-22 13:45:47)

Arch Linux

#26 2010-09-11 22:09:50

Re: Removing the "Local Mirror" article?

#27 2010-09-11 22:20:18

Re: Removing the "Local Mirror" article?

#28 2010-09-11 22:46:23

Re: Removing the "Local Mirror" article?

#29 2010-09-11 23:20:25

Re: Removing the "Local Mirror" article?

#30 2010-09-19 05:27:39

Re: Removing the "Local Mirror" article?

#31 2010-09-19 08:34:19

Re: Removing the "Local Mirror" article?

#32 2010-09-19 10:33:15

Re: Removing the "Local Mirror" article?

#33 2010-09-19 10:49:37

Re: Removing the "Local Mirror" article?

#34 2010-09-19 10:57:44

Re: Removing the "Local Mirror" article?

#35 2010-09-19 11:07:47

Re: Removing the "Local Mirror" article?

#36 2010-09-19 14:13:36

Re: Removing the "Local Mirror" article?

#37 2010-09-19 19:24:47

Re: Removing the "Local Mirror" article?

#38 2010-09-20 07:13:20

Re: Removing the "Local Mirror" article?

#39 2010-09-21 00:25:12

Re: Removing the "Local Mirror" article?

#40 2010-09-21 05:58:04

Re: Removing the "Local Mirror" article?

#41 2010-09-21 12:11:14

Re: Removing the "Local Mirror" article?

#42 2010-09-21 12:17:24

Re: Removing the "Local Mirror" article?

#43 2010-09-21 13:05:34

Re: Removing the "Local Mirror" article?

#44 2010-09-21 22:41:39

Re: Removing the "Local Mirror" article?

#45 2010-09-21 22:47:02

Re: Removing the "Local Mirror" article?

#46 2010-09-22 01:21:18

Re: Removing the "Local Mirror" article?

#47 2010-09-22 01:32:46

Re: Removing the "Local Mirror" article?

#48 2010-09-22 03:37:43

Re: Removing the "Local Mirror" article?

#49 2010-09-22 04:14:07

Re: Removing the "Local Mirror" article?

#50 2010-09-22 13:45:06

Re: Removing the "Local Mirror" article?

Board footer