Installing Arch without net (from backed up /var/cache)

s26c.sayan · 2007-03-12 14:04:23

Hi!!

I am running an arch installation on my machine, in which I have installed many apps that I need. Now I wish to install arch along with all those packages I am using now on another machine which doesn't have internet connection. I have tried copying the contents of my /var/cache/pacman/pkg folder from my existing machine to the other one. But it has not helped much.
I cannot use pacman -S to install all packages effectively, as every now and then it begins to seek internet connection!

pacman -A to install each of the packages individually from /var/cache lands me into 'dependency hell'!!

So, I think I need something like a 'local CD based repository'. I tried browsing the wiki pages for ways to do so, but none of the pages which came up seems to address my issue well enough.

To sum up, I need a method of creating a CD from the packages in /var/cache/pacman which will be able to function as a local repository which can be added to pacman.conf.

(In debian /Ubuntu I used the utility called apt-move to do this thing!!)

lilsirecho · 2007-03-12 15:58:11

I have a dual-layer Larch DVD with 3200 arch packages in a local repository which I generated through pacman with dependencies.

I find that ....pacman -Ud (program to load)..... worked for the packages I called up in Larch(I didn't try all 3200!) and they are loaded r/w in Larch through unionfs and pacman. The DVD disc can be used in normal arch boot as well.

It may be of interest to you.

s26c.sayan · 2007-03-13 06:42:01

pacman -Ud (program to load)

The man pages tell me that -d option forces pacman to skip all dependency checks! So, even if I am able to install packages this way, it seems that I'l have to install all the dependancies separately to get a working sytem!! This still has be in 'dependancy hell'....or am I missing something here!!

Anyway, I have managed to find a partial solution. The python script mentioned here : http://wiki.archlinux.org/index.php/Cus … Python_way does almost what I want.
However, this script seems to copy only the first 100 packages or so into the repository!! So, I had to split my package collection into sveral parts and create a custom repo from each. After that I added them separately to pacman.conf. Some arbitrary packages were still left out though, which I had to add individually using pacman -A.

gradgrind · 2007-03-13 07:18:07

s26c.sayan wrote:

Anyway, I have managed to find a partial solution. The python script mentioned here : http://wiki.archlinux.org/index.php/Cus … Python_way does almost what I want.
However, this script seems to copy only the first 100 packages or so into the repository!! So, I had to split my package collection into sveral parts and create a custom repo from each. After that I added them separately to pacman.conf. Some arbitrary packages were still left out though, which I had to add individually using pacman -A.

Really? Can you supply more details of what the problem there is? I of course didn't intend the script to have such a limit, but I haven't tested it with a large number of packages. Packages got left out? Which ones? Or is it different ones each time you run the script? Cactus also wrote a python script to do the same job: http://cactuswax.net/projects/pygenrepo.html
Maybe that would be worth a try too. And let us know how it goes.

s26c.sayan · 2007-03-13 10:46:02

This is what I get as output while running the script:

[root@myhost hda6]# ./gen_repo.py pkg/ sayan

Creating pacman database (sayan.db.tar.gz) file in pkg/
Do you want to do this? [Y/n] y
Traceback (most recent call last):
  File "/mnt/hda6/gen_repo.py", line 263, in ?
    create_db(dbname, pkgdir, ignore_list)
  File "/mnt/hda6/gen_repo.py", line 55, in create_db
    mkdesc(pkg_dir, pkg_dict)
  File "/mnt/hda6/gen_repo.py", line 135, in mkdesc
    f.write("%DESC%\n" + pkg_dict["pkgdesc"] + "\n\n")
TypeError: cannot concatenate 'str' and 'NoneType' objects
[root@myhost hda6]#

The sayan.db.tar.gz file is created alright, but when I view its contents, I find only the first 100 or so packages out of the total 491 packages in the specified pkg/ directory. Obviously , I am a total no-geek, and don't know what to make of these error messages!!:(

As for the 'left out' packages, I dont remember which ones, but there were only a couple so far. And that may also be due to the fact that I had copied parts of the pkg/ folder into several parts, and might have missed some packages!!

I will be trying out the other script as well ASAP.

Anyway, this script has been a very good one for my purpose. Thanks a lot!!!

s26c.sayan · 2007-03-13 17:14:34

The second script you mentioned seems to work better!!

BTW,. can anyone tell me what exactly does this *.db.tar.gz file contains?? Certainly it doesn't contain the packages themselves!!
Maybe only some files which contain the info about each of the packages in the respective repo???
Sorry if I sound too much like a n00b...but can't help it, I really AM one!!!

gradgrind · 2007-03-13 18:36:37

s26c.sayan wrote:

The second script you mentioned seems to work better!!
BTW,. can anyone tell me what exactly does this *.db.tar.gz file contains?? Certainly it doesn't contain the packages themselves!!
Maybe only some files which contain the info about each of the packages in the respective repo???
Sorry if I sound too much like a n00b...but can't help it, I really AM one!!!

Grrrr, one up to cactus!

Anyway, I have made a very little change to my script on the wiki - if you have time, I would be very grateful if you could test this. I suspect that the problem was a package without a description text, which wasn't expected. (Is there a spanish-inquisition package in Arch?)

And yes, the *.db.tar.gz files contain information about the packages. pacman uses it to resolve dependencies, check checksums, etc.

pelle.k · 2007-03-13 18:43:27

I've got an idea! Why don't you untar that archive and take a look at it?
It's basicly what it says it is; a db of a repository.

You can find the contents of a "pacman -Sy" in /var/lib/pacman/
You might even copy the contents from /var/lib/pacman/ from $source machine to $target machine, deploy everything from /var/cache/pacman/pkg and "pacman -S" stuff. I don't know. Haven't tried it...

Last edited by pelle.k (2007-03-13 18:43:59)

vacant · 2007-03-13 20:30:39

I've often installed Arch by "cp -a" from one hard disk to another, then adjust fstab, boot a Puppy Linux CD and install grub.

Background to this is that I look after a youth cybercafé with a cheap Internet connection, capped to a few GB per month. So at home I keep a HD with the Arch cybercafé setup (guest account, games etc) and every few weeks I update it, check it works, then take the pacman cache on a USB stick. But whenever I build a new Arch Cybercafé PC, I use the "cp -a" method.

Kardell · 2007-03-27 00:22:01

[root@localhost custompkg]# ./gen_repo.py /home/custompkg/custompkg-arch-i686/

Creating pacman database (custompkg-arch-i686.db.tar.gz) file in /home/custompkg/custompkg-arch-i686/
Do you want to do this? [Y/n]
Traceback (most recent call last):
  File "/home/custompkg/gen_repo.py ", line 263, in <module>
    create_db(dbname, pkgdir, ignore_list)
  File "/home/custompkg/gen_repo.py ", line 61, in create_db
    d = onlyname.match(d).group(1)
AttributeError: 'NoneType' object has no attribute 'group'
[root@localhost custompkg]#

What the heck?
^^

Last edited by Kardell (2007-03-27 00:23:44)

LiFo2 · 2007-03-29 15:13:56

Well, I was wondering the same thing, so I tried a "man pacman", and guess what ...

CONFIG: REPOSITORIES
       Each repository section defines a section name and at least one location where the packages can be found.   The  section
       name  is  defined  by  the  string within square brackets (eg, the two above are 'current' and 'custom').  Locations are
       defined with the Server directive and follow a URL naming  structure.   Currently  only  ftp  is  supported  for  remote
       servers.  If you want to use a local directory, you can specify the full path with a 'file://' prefix, as shown above.

USING YOUR OWN REPOSITORY
       Let's  say  you  have  a  bunch  of  custom  packages  in  /home/pkgs  and  their  respective  PKGBUILD files are all in
       /var/abs/local.  All you need to do is generate a compressed package database in the /home/pkgs directory so pacman  can
       find it when run with --refresh.

              # gensync /var/abs/local /home/pkgs/custom.db.tar.gz

       The  above  command  will  read  all  PKGBUILD  files  in  /var/abs/local  and  generate  a  compressed  database called
       /home/pkgs/custom.db.tar.gz.  Note that the database must be of the form {treename}.db.tar.gz, where {treename}  is  the
       name  of the section defined in the configuration file.  That's it!  Now configure your custom section in the configura-
       tion file as shown in the config example above.  Pacman will now use your package repository.  If you add  new  packages
       to the repository, remember to re-generate the database and use pacman's --refresh option.

Kardell · 2007-03-29 19:34:05

Yeah I red about it.
One thing that gensync and 1st script from Arch Wiki do not check dependencies, only generates db.
I want to create perfect repository without software holes. ;-P
So we need to wait for gradgrind intervetion. ;-)

gradgrind · 2007-03-30 09:59:36

Kardell wrote:

[root@localhost custompkg]# ./gen_repo.py /home/custompkg/custompkg-arch-i686/

Creating pacman database (custompkg-arch-i686.db.tar.gz) file in /home/custompkg/custompkg-arch-i686/
Do you want to do this? [Y/n]
Traceback (most recent call last):
  File "/home/custompkg/gen_repo.py ", line 263, in <module>
    create_db(dbname, pkgdir, ignore_list)
  File "/home/custompkg/gen_repo.py ", line 61, in create_db
    d = onlyname.match(d).group(1)
AttributeError: 'NoneType' object has no attribute 'group'
[root@localhost custompkg]#

What the heck?
^^

Could you try this version, please:
http://pastebin.archlinux.org/2178

and let me know what output you get.

And, by the way, please mail me bug reports in case I miss them in some obscure thread on the forum.

Kardell · 2007-03-30 15:18:45

gradgrind wrote:

Could you try this version, please:
http://pastebin.archlinux.org/2178
and let me know what output you get.
And, by the way, please mail me bug reports in case I miss them in some obscure thread on the forum.

Roger that!

I was trying this new script many times and everytime I`ve received "Segmentation fault". Python process is consuming almost 85% of processor (1GHz P3) and only 1.5% of SDRAM (390MB). Earlier version of script failed the same way, but more often finished the job. When I putted new dependent packages to my repository it caused AttributeError in further generation process.
That`s all!
^^

Last edited by Kardell (2007-03-30 15:29:20)

gradgrind · 2007-03-31 10:47:04

Kardell wrote:

[root@localhost custompkg]# ./gen_repo.py /home/custompkg/custompkg-arch-i686/

Creating pacman database (custompkg-arch-i686.db.tar.gz) file in /home/custompkg/custompkg-arch-i686/
Do you want to do this? [Y/n]
Traceback (most recent call last):
  File "/home/custompkg/gen_repo.py ", line 263, in <module>
    create_db(dbname, pkgdir, ignore_list)
  File "/home/custompkg/gen_repo.py ", line 61, in create_db
    d = onlyname.match(d).group(1)
AttributeError: 'NoneType' object has no attribute 'group'
[root@localhost custompkg]#

What the heck?
^^

Fixed in version 1.4 (and in pacin-2.3). Thanks for the bug report!
I can neither explain nor reproduce your seg-faults. I tested the new version on a pretty large repo and it seemed to work fine.

s26c.sayan · 2007-04-24 12:55:15

Can any of these scripts preserve the package group structure?

I mean, now when I try to to install a group like xfce4 from a repository created by a script, it says 'package not found'!

gradgrind · 2007-04-24 17:07:38

s26c.sayan wrote:

Can any of these scripts preserve the package group structure?
I mean, now when I try to to install a group like xfce4 from a repository created by a script, it says 'package not found'!

Do you mean gen_repo? I just did a very quick and simple test and it worked. So if you're having problems with it I would be grateful for some more details.

s26c.sayan · 2007-05-05 13:35:56

gradgrind wrote:

s26c.sayan wrote:
Can any of these scripts preserve the package group structure?
I mean, now when I try to to install a group like xfce4 from a repository created by a script, it says 'package not found'!
Do you mean gen_repo? I just did a very quick and simple test and it worked. So if you're having problems with it I would be grateful for some more details.

Well I gave it another try today after a long time, and it worked like a charm!!!!!! [:)]
Thanks a lot!! What a great script!!!!

BTW, the other script (by Cactus) doesn't seem to preserve groups! So one up to YOU Gradgrind this time!! [:D]

Arch Linux

#1 2007-03-12 14:04:23

Installing Arch without net (from backed up /var/cache)

#2 2007-03-12 15:58:11

Re: Installing Arch without net (from backed up /var/cache)

#3 2007-03-13 06:42:01

Re: Installing Arch without net (from backed up /var/cache)

#4 2007-03-13 07:18:07

Re: Installing Arch without net (from backed up /var/cache)

#5 2007-03-13 10:46:02

Re: Installing Arch without net (from backed up /var/cache)

#6 2007-03-13 17:14:34

Re: Installing Arch without net (from backed up /var/cache)

#7 2007-03-13 18:36:37

Re: Installing Arch without net (from backed up /var/cache)

#8 2007-03-13 18:43:27

Re: Installing Arch without net (from backed up /var/cache)

#9 2007-03-13 20:30:39

Re: Installing Arch without net (from backed up /var/cache)

#10 2007-03-27 00:22:01

Re: Installing Arch without net (from backed up /var/cache)

#11 2007-03-29 15:13:56

Re: Installing Arch without net (from backed up /var/cache)

#12 2007-03-29 19:34:05

Re: Installing Arch without net (from backed up /var/cache)

#13 2007-03-30 09:59:36

Re: Installing Arch without net (from backed up /var/cache)

#14 2007-03-30 15:18:45

Re: Installing Arch without net (from backed up /var/cache)

#15 2007-03-31 10:47:04

Re: Installing Arch without net (from backed up /var/cache)

#16 2007-04-24 12:55:15

Re: Installing Arch without net (from backed up /var/cache)

#17 2007-04-24 17:07:38

Re: Installing Arch without net (from backed up /var/cache)

#18 2007-05-05 13:35:56

Re: Installing Arch without net (from backed up /var/cache)

Board footer