You are not logged in.

#1 2007-03-28 08:20:05

hacosta
Member
From: Mexico
Registered: 2006-10-22
Posts: 423

mirror list proposal

i was thinking, all those files in pacman.d/ aren't really necessary. how about using just one file named mirrors (or whatever) in this format
Server = ftp://ftp-linux.cc.gatech.edu/pub/linux … archlinux/
Server = ftp://ftp.archlinux.org/
.
.
.
so on.. and pacman would know that for current, the complete path is Server+current/os/i686..  i don't see the need for having all those files, (you need to edit 5 files just to change the mirror) and if some server only mirrors current for instance, and you want to use it, this is easily fixable in your pacman.conf
so what do you guys think?

Offline

#2 2007-03-28 15:42:44

phrakture
Arch Overlord
From: behind you
Registered: 2003-10-29
Posts: 7,879
Website

Re: mirror list proposal

Yeah, dan and I have talked about this.  If you're interested, please join the pacman-dev mailing list where we will be discussing this shortly.

Offline

#3 2007-03-28 17:40:37

codemac
Member
From: Cliche Tech Place
Registered: 2005-05-13
Posts: 790
Website

Re: mirror list proposal

Honestly, I don't like it.  It's only useful for mirrors that are exactly rsyncing whatever the main archlinux.org has.  What if I want to download all my packages into a different directory structure, and create my own db?

Being explicit about where the packages are for each repos is better for hackery than just having implicit locations.

Offline

#4 2007-03-28 18:18:46

hacosta
Member
From: Mexico
Registered: 2006-10-22
Posts: 423

Re: mirror list proposal

then you would still be able to use pacman.conf to override the list

Offline

#5 2007-03-28 18:55:33

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: mirror list proposal

Oh, great that someone mentioned it. I was whining on irc that I found it so annoying to edit the same mirrors 5 times (current, extra, community, testing, and unstable). Why would anyone want to use one list for current, and another for extra?
I really like how it's done in debian. See http://www.debian.org/doc/manuals/apt-h … co.en.html
Maybe we could have something just slighty inspired from it? smile

Or maybe the current syntax could be extended to something like that:
[current extra community testing unstable]
Server = ftp://mir1.archlinuxfr.org/archlinux/$branch/os/i686

Oh well, I don't know, but anything that would allow to edit the mirror of the standard repos in one operation.
And it should obviously still allows to add unofficial repos.


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#6 2007-03-28 19:55:07

codemac
Member
From: Cliche Tech Place
Registered: 2005-05-13
Posts: 790
Website

Re: mirror list proposal

I like shining's syntax there.  Have the [tag] not only be descriptive of multiple repos, but let it be a variable in the url as well.

@hacosta: Wouldn't pacman.conf be using the same syntax as the Include files... and then wouldn't this mean that you couldn't override it in pacman.conf because it would be using this ambiguous syntax?

Offline

#7 2007-03-28 22:32:35

Zer0
Member
From: Windsor, ON, Canada
Registered: 2006-08-25
Posts: 299

Re: mirror list proposal

1+ [tag] idea

Offline

#8 2007-03-28 22:55:32

hacosta
Member
From: Mexico
Registered: 2006-10-22
Posts: 423

Re: mirror list proposal

@codemac, now i understand.. i don't think it's such a big deal to use os/$arch.. but i get your point, anyways the tag idea is nice

Offline

#9 2007-03-29 04:27:40

gentoofu
Member
Registered: 2006-10-05
Posts: 34

Re: mirror list proposal

The tag idea should be optional if implemented. I really like how pacman.conf/d is implemented right now. I just wish there'd be more mirrors available, especially dedicated to one repository each instead of serving the whole shebang.

Offline

#10 2007-03-29 07:55:21

lloeki
Member
From: France
Registered: 2007-02-20
Posts: 456
Website

Re: mirror list proposal

personnally, I just modify in one place, and use some scripty with sed to propagate.

but personally on that case, if it were to be implemented, I like the apt syntax because it's clear, concise, and the 'tags' are on the same line as the repo url, so they're immediately associated and the urls can be reconstructed in microseconds by simple read.

plus, to me, it seems nonsense to have things like:

[stable community]
blah blah
[stable community testing]
bleh bleh

or it may just as well end up like:

[foo]
server = quux
[bar]
server = quuux
[baz]
server = quuuux

where

quux foo
quuux bar
quuuuux baz

reads simply

and, well, what's the point of doing something different just for the sake of it? watch out for the bikeshed effect.

well, I see one advantage to the [a b c] method: if all the mirrors support the exact same set of tags, then it's useful, because it's a snap to add or remove 'testing'. but if some have different tags, it becomes increasingly painful, exponentially WRT the number of different tags (worst case, as potential combinations grow up).

and after all, we only get to edit mirrors once in a while, so is it really a benefit? I think the main issue to consider here is readability and manageability.

Last edited by lloeki (2007-03-29 07:57:07)


To know recursion, you must first know recursion.

Offline

#11 2007-03-29 12:11:07

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: mirror list proposal

lloeki wrote:

well, I see one advantage to the [a b c] method: if all the mirrors support the exact same set of tags, then it's useful, because it's a snap to add or remove 'testing'. but if some have different tags, it becomes increasingly painful, exponentially WRT the number of different tags (worst case, as potential combinations grow up).

I don't see anything exponential anywhere. In the best case, it will be much easier, in the worst case, it'll be just like it is right now (a different list for each repo).

and after all, we only get to edit mirrors once in a while, so is it really a benefit? I think the main issue to consider here is readability and manageability.

It'll be a big benefit to me. I don't have huge problems with arch, just minor annoying details like this one smile
I agree readability and manageability are important, both for the users, and the developers.
And that's generally contradictory, so we just need to find the best compromise big_smile

The debian way is the one I prefer as an user.
Then I tried to figure out another way closer to arch current one, for easing up the developers' task, but I've no idea if it's a good one, it's just a random idea.


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#12 2007-03-29 13:39:29

vanel86
Member
From: Trieste, Italy
Registered: 2007-03-29
Posts: 4

Re: mirror list proposal

Why the mirrors official file should get this odd partial mirror status?, the standard file should list only the full ones, leaving the partial one to the custom repository status. The mirrors will have a fixed standard folder structure and pacman will scan and found everything where it is supposed to be. The mirrors file however should include the geographical location of the mirrors, so pacman could go straight to the more nearby mirror.

Last edited by vanel86 (2007-03-29 13:42:49)

Offline

#13 2007-03-29 16:43:57

mOLOk
Member
From: Milan, Italy
Registered: 2005-08-09
Posts: 20
Website

Re: mirror list proposal

Maybe would make sense to use rc.conf syntax, e.g:

repos=(current extra unstable community !testing)

( "!" means that testing is disabled)

Offline

#14 2007-03-29 17:37:19

T-u-N-i-X
Member
From: İstanbul
Registered: 2006-03-14
Posts: 435
Website

Re: mirror list proposal

XML would be just fine. smile

<current>
  <server>ftp://ftp.archlinux.org/current/os/i686/</server>

and etc..


Quis custodiet ipsos custodiet?

Offline

#15 2007-03-29 18:05:45

Mr Green
Forum Fellow
From: U.K.
Registered: 2003-12-21
Posts: 5,863

Re: mirror list proposal

I' m with mOLOk and tag idea sounds cool too


Mr Green

Offline

#16 2007-03-29 18:56:02

phrakture
Arch Overlord
From: behind you
Registered: 2003-10-29
Posts: 7,879
Website

Re: mirror list proposal

T-u-N-i-X wrote:

XML would be just fine. smile

<current>
  <server>ftp://ftp.archlinux.org/current/os/i686/</server>

and etc..

Never.

Offline

#17 2007-03-29 23:04:34

joaospinto
Member
Registered: 2007-02-13
Posts: 40

Re: mirror list proposal

Kiss

Offline

#18 2007-03-29 23:40:19

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: mirror list proposal

hmm, well, I was bored and wanted to know if I still knew some C, and also, my idea seemed quite easy to implement, so I went ahead, and wrote really crap and ugly code, as always big_smile
It was fun though, now I've just theses lines in my pacman.conf :

[testing current extra community unstable]
Server = ftp://mir1.archlinuxfr.org/archlinux/$repo/os/i686

and here is the crappy patch :

Index: lib/libalpm/alpm.c
===================================================================
RCS file: /home/cvs-pacman/pacman-lib/lib/libalpm/alpm.c,v
retrieving revision 1.127
diff -u -r1.127 alpm.c
--- lib/libalpm/alpm.c    11 Mar 2007 21:10:03 -0000    1.127
+++ lib/libalpm/alpm.c    29 Mar 2007 23:27:11 -0000
@@ -871,6 +871,68 @@
     return(_alpm_fetch_pkgurl(url));
 }
 
+
+/** Create a list of mirrors
+ * @param url a generic url for a mirror, like this : "ftp://ftp.archlinux.org/$repo/os/i686"
+ * @param repolist a list of repository, separated by espace, like this : "current extra"
+ * return the list of mirrors generated
+ */
+alpm_list_t *create_mirror_list(char *url, char *repolist) {
+    char *repo = NULL;
+    char *s1 = NULL;
+    char *s2 = NULL;
+    char var[] = "$repo";
+    int length;
+
+    alpm_list_t *list = alpm_list_new();
+    repo = malloc(strlen(repolist)+1);
+    strcpy(repo, repolist);
+    length = strlen(repo)+strlen(url)+2;
+
+    if (strchr(repo, ' ') == NULL) {
+        char *mirror = malloc(length);
+        *mirror = '\0';
+        mirror = strcat(mirror, repo);
+        mirror = strcat(mirror, "=");
+        mirror = strcat(mirror, url);
+
+        list = alpm_list_add(list, mirror);
+        return list;
+    }
+
+    repo = strtok(repo, " ");
+    s1 = strstr(url, var);
+    if (s1 != NULL) {
+        *s1 = '\0';
+        s1 += strlen(var);
+        s2 = malloc(strlen(s1)+1);
+        s2 = strcpy(s2, s1);
+        s1 = malloc(strlen(url)+1);
+        s1 = strcpy(s1, url);
+    }
+    while (repo != NULL) {
+        if (s1 != NULL) {
+            *url = '\0';
+            url = strcat(url, s1);
+            url = strcat(url, repo);
+            url = strcat(url, s2);
+        }
+
+        char *mirror = malloc(length);
+        *mirror = '\0';
+        mirror = strcat(mirror, repo);
+        mirror = strcat(mirror, "=");
+        mirror = strcat(mirror, url);
+
+        list = alpm_list_add(list, mirror);
+
+        repo = strtok(NULL, " ");
+    }
+
+    return list;
+
+}
+
 /** Parses a configuration file.
  * @param file path to the config file.
  * @param callback a function to be called upon new database creation
@@ -899,9 +961,6 @@
         if(!strcmp(section, "local")) {
             RET_ERR(PM_ERR_CONF_LOCAL, -1);
         }
-        if(strcmp(section, "options")) {
-            db = _alpm_db_register(section, callback);
-        }
     }
 
     while(fgets(line, PATH_MAX, fp)) {
@@ -923,13 +982,6 @@
             if(!strcmp(section, "local")) {
                 RET_ERR(PM_ERR_CONF_LOCAL, -1);
             }
-            if(strcmp(section, "options")) {
-                db = _alpm_db_register(section, callback);
-                if(db == NULL) {
-                    /* pm_errno is set by alpm_db_register */
-                    return(-1);
-                }
-            }
         } else {
             /* directive */
             ptr = line;
@@ -1048,11 +1100,23 @@
                     }
                 } else {
                     if(!strcmp(key, "SERVER")) {
-                        /* add to the list */
-                        if(alpm_db_setserver(db, ptr) != 0) {
-                            /* pm_errno is set by alpm_db_setserver */
-                            return(-1);
+                        alpm_list_t *mirrorlist = create_mirror_list(ptr, section);
+
+                        for (; mirrorlist != NULL; mirrorlist = mirrorlist->next) {
+                            char *url = mirrorlist->data;
+                            char *repo = strsep(&url, "=");
+                            db = _alpm_db_register(repo, callback);
+                            if(db == NULL) {
+                                /* pm_errno is set by alpm_db_register */
+                                return(-1);
+                            }
+                            /* add to the list */
+                            if(alpm_db_setserver(db, url) != 0) {
+                                /* pm_errno is set by alpm_db_setserver */
+                                return(-1);
+                            }
                         }
+
                     } else {
                         RET_ERR(PM_ERR_CONF_BAD_SYNTAX, -1);
                     }

pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#19 2007-04-20 08:22:37

Romashka
Forum Fellow
Registered: 2005-12-07
Posts: 1,054

Re: mirror list proposal


to live is to die

Offline

Board footer

Powered by FluxBB