reposearch: a tool for finding binary repos

Xyne · 2009-03-19 15:44:14

Info page: http://xyne.archlinux.ca/info/reposearch

After following this thread I got the idea to create a database for third-party binary repositories. I started by grabbing all of the binary repos listed on the Unofficial User Repositories wiki page.

The current repo list can be found at http://xyne.archlinux.ca/interface/reposearch.php

I then wrote a simple php page to handle queries and then reposearch to query the list from the command line. See the info page to learn about search options.

Example output:

$reposearch -s xyne
##### any #####
[xyne-any]
# The home of Powerpill and Xyne's other contributions.
# More info including a package list can be found at http://xyne.archlinux.ca/repositories
Server = http://xyne.archlinux.ca/repos/any/

$reposearch -s game
##### x86_64 #####
[arch-games]
# The Arch Linux Gaming repository project
Server = http://twilightlair.net/files/arch/games/x86_64
Server = http://arch-games.iskrembilen.com/x86_64
Server = http://arch-games.ontheplayground.co.uk/x86_64

$reposearch -s game -a all
##### i686 #####
[arch-games]
# The Arch Linux Gaming repository project
Server = http://twilightlair.net/files/arch/games/i686
Server = http://arch-games.iskrembilen.com/i686
Server = http://arch-games.ontheplayground.co.uk/i686

##### i686 #####
[esclinux]
# Mostly games, interactive fiction and abc notation stuffs already on AUR
Server = http://download.tuxfamily.org/esclinuxcd/ressources/repo/i686/

##### x86_64 #####
[arch-games]
# The Arch Linux Gaming repository project
Server = http://twilightlair.net/files/arch/games/x86_64
Server = http://arch-games.iskrembilen.com/x86_64
Server = http://arch-games.ontheplayground.co.uk/x86_64

Hopefully the usefulness of this tool will grow with the number or repos.

Aside from the usual questions and feedback, I would really appreciate it if you could post repo information in this thread so that I can keep the list up-to-date and comprehensive. Both modification of current repo data and the addition of new repos are welcome. Please post info in the following form to make it easier for me:

##### <architecture> #####
[<repo>]
# descriptions and other information
# multiple lines if necessary
Server = http://www.example.com/repo/
Server = http://www.anotherexample.com/mirror/repo/

WARNING!
As mentioned in the help message, you use the third-party repos that you find through repo-search entirely at your own risk. Don't install packages from repos that you don't trust. They can contain malware and other creatively wicked things.

rson451 · 2009-03-19 15:50:01

You owe me 5 dollars.

haxit · 2009-03-19 15:55:46

Great idea! Xyne, you are a champ! Where do you come up with all these ideas
People can trust my repo, right.....

Xyne · 2009-03-19 16:21:53

rson451 wrote:

You owe me 5 dollars.

http://bbs.archlinux.org/viewtopic.php? … 17#p518617
I helped you, so you owe me 50% of whatever you pulled in (don't worry, we can haggle a bit).

Anyway, your words were "5 dollars says Xyne does this in the next 2 days", not "...says you do...", so I'm in the clear either way.

@haxit
thanks

Hopefully this will catch on and people will start submitting repos in this thread (and on the wiki).

rson451 · 2009-03-19 16:51:01

That "you" was a collective "you" directed at whoever read the thread Good work sir.

cinan · 2009-03-19 20:44:24

Progress bar during searching should be great.

Xyne · 2009-03-20 11:22:29

I'm not really sure how useful the progress bar is considering that the data sent and received is nearly negligible. Most of the time taken is by the server processing the request and there's no reasonable way to have the server report on its own progress during that time. It really shouldn't be taking much time at all btw.

Still, I've added both "-p" and "-#" options to use curls progress output (along with "--curl-args" for additional curl arguments).

Last edited by Xyne (2009-03-20 12:12:02)

archlinuxsagi · 2009-03-20 11:48:30

I am just wondering.. with so many applications created by Xyne for Archlinux. Isn't he a Arch developer or TU?

Xyne · 2009-03-20 12:13:13

archlinuxsagi wrote:

I am just wondering.. with so many applications created by Xyne for Archlinux. Isn't he a Arch developer or TU?

hehe, I'd just sent in a TU application before reading this post

cinan · 2009-03-20 13:04:44

Does reposearch have some configs? Where?

Xyne · 2009-03-20 13:09:27

reposearch -h

cinan · 2009-03-20 14:16:31

Xyne wrote:

I'm not really sure how useful the progress bar is considering that the data sent and received is nearly negligible. Most of the time taken is by the server processing the request and there's no reasonable way to have the server report on its own progress during that time. It really shouldn't be taking much time at all btw.
Still, I've added both "-p" and "-#" options to use curls progress output (along with "--curl-args" for additional curl arguments).

I thought someting like: "Checking 1/20 repository"

Army · 2009-03-20 22:07:12

I vote for Xyne to become a TU and later dev (in case you want to)! I have your complete repo installed on my computer and many of them (about half or so) are really helpful for me, the rest is about to become helpful

Xyne · 2009-03-20 22:30:13

cinan wrote:

I thought someting like: "Checking 1/20 repository"

Maybe the OP is unclear. The package "reposearch" doesn't actually contain the repo list. The repo list is on my site and reposearch sends queries to the site to get a subset of the list. There is no way to intermittently report progress due to the way that apache and php work.

How long are your searches taking? What search patterns are you using? Queries should be really fast.

Army wrote:

I vote for Xyne to become a TU and later dev (in case you want to)! I have your complete repo installed on my computer and many of them (about half or so) are really helpful for me, the rest is about to become helpful

I'd definitely like to become a dev, but until I learn a "serious" programming language such as C or C++, I don't think I can. I appreciate your support though.
I'll learn them eventually, but Haskell is next on my list so it may take a while.

Dieter@be · 2009-03-21 12:46:23

taurball? what's wrong with the word "source tarball" ?

Daenyth · 2009-03-21 15:25:39

I for one welcome our new taurball overlords

Dusty · 2009-03-21 15:30:31

Is this why you needed that database?

Dusty

Xyne · 2009-03-21 16:41:59

Dieter@be wrote:

taurball? what's wrong with the word "source tarball" ?

Why say "a melted chocolate, toasted marshmallow and graham cracker sandwich" when you can say "smore"?
A tarball that you upload to the AUR = taurball... you instantly understand that the speaker means a PKGBUILD and local source files.

Dusty wrote:

Is this why you needed that database?
Dusty

Yep... although I haven't used it yet. I decided that configuring MySQL without shell access (i.e. write php -> upload -> run -> check -> repeat) was too much of a bitch and that a flat file would do for now considering the size.

I'll get around to converting it later (I'm working on a complete refactoring of powerpill right now).

Last edited by Xyne (2009-03-21 16:57:48)

Arch Linux

#1 2009-03-19 15:44:14

reposearch: a tool for finding binary repos

#2 2009-03-19 15:50:01

Re: reposearch: a tool for finding binary repos

#3 2009-03-19 15:55:46

Re: reposearch: a tool for finding binary repos

#4 2009-03-19 16:21:53

Re: reposearch: a tool for finding binary repos

#5 2009-03-19 16:51:01

Re: reposearch: a tool for finding binary repos

#6 2009-03-19 20:44:24

Re: reposearch: a tool for finding binary repos

#7 2009-03-20 11:22:29

Re: reposearch: a tool for finding binary repos

#8 2009-03-20 11:48:30

Re: reposearch: a tool for finding binary repos

#9 2009-03-20 12:13:13

Re: reposearch: a tool for finding binary repos

#10 2009-03-20 13:04:44

Re: reposearch: a tool for finding binary repos

#11 2009-03-20 13:09:27

Re: reposearch: a tool for finding binary repos

#12 2009-03-20 14:16:31

Re: reposearch: a tool for finding binary repos

#13 2009-03-20 22:07:12

Re: reposearch: a tool for finding binary repos

#14 2009-03-20 22:30:13

Re: reposearch: a tool for finding binary repos

#15 2009-03-21 12:46:23

Re: reposearch: a tool for finding binary repos

#16 2009-03-21 15:25:39

Re: reposearch: a tool for finding binary repos

#17 2009-03-21 15:30:31

Re: reposearch: a tool for finding binary repos

#18 2009-03-21 16:41:59

Re: reposearch: a tool for finding binary repos

Board footer