Download all packages and them move them to home

mfaridi · 2010-12-15 21:04:59

I have internet connection in work place and I do not have internet connection in home , so I can not connect to internet in home
I want install Arch linux in home , and start use it and , after install I need packages ,
So I want download all packages for AMD64 and then move them to home and make local repo there and install all thing I need ,
so I use this command for download all packages

 
rsync -avrt --bwlimit=175 rsync://ftp.byfly.by/pub/archlinux/core/os/any/   /home/narges/suse && rsync -avrt --bwlimit=85 rsync://ftp.byfly.by/pub/archlinux/core/os/x86_64/ /home/narges/suse/ && rsync -avrt --bwlimit=85 rsync://ftp.byfly.by/pub/archlinux/extra/os/any/ /home/narges/suse/ && rsync -avrt --bwlimit=85 rsync://ftp.byfly.by/pub/archlinux/extra/os/x86_64/ /home/narges/suse/

after one days ., I test I see this command download for me more than 1.5GB of packages , but after two days I see this command download for me 700MB of packages and many of packages are delete .
can I use or find another way for download all packages and then move them to house and then start install packages I need ?
Is this method I use is good or correct ?

Last edited by mfaridi (2010-12-15 21:06:10)

karol · 2010-12-15 21:31:01

       -w, --downloadonly
           Retrieve all packages from the server, but do not install/upgrade anything.

So 'pacman -Sw <list packages here>', put it on e.g. a pendrive and https://wiki.archlinux.org/index.php/Pa … repository at home.

Arch is a rolling release distro so packages come and go pretty fast (a new version arrives).
Many old packages can be found here: http://arm.konnichi.com/search/

Last edited by karol (2010-12-15 21:34:29)

mfaridi · 2010-12-16 06:14:53

karol wrote:

       -w, --downloadonly
           Retrieve all packages from the server, but do not install/upgrade anything.
So 'pacman -Sw <list packages here>', put it on e.g. a pendrive and https://wiki.archlinux.org/index.php/Pa … repository at home.
Arch is a rolling release distro so packages come and go pretty fast (a new version arrives).
Many old packages can be found here: http://arm.konnichi.com/search/

Thanks,
Do I understand good ? I must use rsysnc with

option .
but I at work place I use another OS and do not use Arch linux , So I want download all packages in work place and themn move them to home

karol · 2010-12-16 06:28:17

No, the 'w' option is for pacman - 'pacman -Sw'. What is the best way w/o using pacman - I don't know. pacman should work on many distros - what OS are you using?
I think it would be better if you could print a bunch of urls and download them with wget instead of rsyncing the whole mirrror:

[karol@black ~]$ sudo pacman -Syup
:: Synchronizing package databases...
 core is up to date
 extra                                        454,2K  496,0K/s 00:00:01 [########################################] 100%
 heftig is up to date
 xyne-any is up to date
 unarch is up to date
 archlinuxfr is up to date
 archstuff is up to date
 arch-games is up to date
 dragonlord is up to date
 herecura-stable                               12,5K  130,3K/s 00:00:00 [########################################] 100%
 andrwe is up to date
 cake is up to date
 cinan is up to date
 rfad is up to date
 community                                    405,6K  484,1K/s 00:00:01 [########################################] 100%
:: Starting full system upgrade...
http://mir.archlinux.fr/core/os/i686/bash-4.1.009-3-i686.pkg.tar.xz
http://mir.archlinux.fr/core/os/i686/run-parts-3.4.1-1-i686.pkg.tar.xz
http://mir.archlinux.fr/core/os/i686/ca-certificates-20090814+nmu2-1-any.pkg.tar.xz
http://mir.archlinux.fr/extra/os/i686/chromium-8.0.552.224-1-i686.pkg.tar.xz
http://mir.archlinux.fr/core/os/i686/filesystem-2010.12-1-any.pkg.tar.xz

You can save those urls to a file and bring it to work. You need current package dbs though to generate the correct output. I think you can download them by hand.

Using Arch Linux w/o internet connection is a bit masochistic.

mfaridi · 2010-12-16 07:32:56

karol wrote:

No, the 'w' option is for pacman - 'pacman -Sw'. What is the best way w/o using pacman - I don't know. pacman should work on many distros - what OS are you using?
I think it would be better if you could print a bunch of urls and download them with wget instead of rsyncing the whole mirrror:
[karol@black ~]$ sudo pacman -Syup
:: Synchronizing package databases...
 core is up to date
 extra                                        454,2K  496,0K/s 00:00:01 [########################################] 100%
 heftig is up to date
 xyne-any is up to date
 unarch is up to date
 archlinuxfr is up to date
 archstuff is up to date
 arch-games is up to date
 dragonlord is up to date
 herecura-stable                               12,5K  130,3K/s 00:00:00 [########################################] 100%
 andrwe is up to date
 cake is up to date
 cinan is up to date
 rfad is up to date
 community                                    405,6K  484,1K/s 00:00:01 [########################################] 100%
:: Starting full system upgrade...
http://mir.archlinux.fr/core/os/i686/bash-4.1.009-3-i686.pkg.tar.xz
http://mir.archlinux.fr/core/os/i686/run-parts-3.4.1-1-i686.pkg.tar.xz
http://mir.archlinux.fr/core/os/i686/ca-certificates-20090814+nmu2-1-any.pkg.tar.xz
http://mir.archlinux.fr/extra/os/i686/chromium-8.0.552.224-1-i686.pkg.tar.xz
http://mir.archlinux.fr/core/os/i686/filesystem-2010.12-1-any.pkg.tar.xz
You can save those urls to a file and bring it to work. You need current package dbs though to generate the correct output. I think you can download them by hand.

Using Arch Linux w/o internet connection is a bit masochistic.

I use FreeBSD at work place and I want do this with FreeBSD , with this method I download all packages for OpenSuse 11.3 and Fedora 14 too and then make localrepo , I do this with rsync and use above command
I want download all packages for Arch ,

karol · 2010-12-16 07:40:20

How often OpenSuse and Fedora update their packages? Arch does that pretty often.
IIRC pacman should work on FreeBSD.

mfaridi · 2010-12-16 07:54:11

karol wrote:

How often OpenSuse and Fedora update their packages? Arch does that pretty often.
IIRC pacman should work on FreeBSD.

OK
but I need way for download all packages and make localrepo

Xyne · 2010-12-16 08:07:23

"pacman -Sw..." will not work in this case.

If you had an internet connection at home then you could update the sync database with "pacman -Sy". After that you could generate download lists with "pacman -Syup [pkgnames]" or create metalinks with powerpill.

Without any connection at home your options are very limited. Rsyncing the databases will be a waste of bandwidth because you will download many packages that you will never use and you will download them often because of the frequent upgrades.

I've had an idea for something for a while and this thread finally convinced me to write it. Take a look at Powerpill Portable.
I've opened a thread about it here. Please post any feedback there.

Last edited by Xyne (2010-12-16 08:18:20)

mfaridi · 2010-12-16 08:17:51

Xyne wrote:

"pacman -Sw..." will not work in this case.
If you had an internet connection at home then you could update the sync database with "pacman -Sy". After that you could generate download lists with "pacman -Syup [pkgnames]" or create metalinks with powerpill.
Without any connection at home your options are very limited. Rsyncing the databases will be a waste of bandwidth because you will download many packages that you will never use and you will download them often because of the frequent upgrades.

I've had an idea for something for a while and this thread finally convinced me to write it. Take a look at the following:
http://xyne.archlinux.ca/scripts/pacman … l-portable

thanks
in home I do not have internet connection and I want download all packages in work place and then move them to home and install arch with all packages I need , and I do not update it .
it is not important how much packages I download at work place . maybe many of them never use

Xyne · 2010-12-16 08:19:43

mfaridi wrote:

thanks
in home I do not have internet connection and I want download all packages in work place and then move them to home and install arch with all packages I need , and I do not update it .
it is not important how much packages I download at work place . maybe many of them never use

That's why I wrote Powerpill Portable. It should do exactly what you need.

mfaridi · 2010-12-16 08:27:50

Xyne wrote:

mfaridi wrote:
thanks
in home I do not have internet connection and I want download all packages in work place and then move them to home and install arch with all packages I need , and I do not update it .
it is not important how much packages I download at work place . maybe many of them never use
That's why I wrote Powerpill Portable. It should do exactly what you need.

I see Powerpill script , but I think this script does not work on FreeBSD Box
I need script for download all packages only and work on FreeBSD box

Xyne · 2010-12-16 08:39:02

mfaridi wrote:

Xyne wrote:
mfaridi wrote:
thanks
in home I do not have internet connection and I want download all packages in work place and then move them to home and install arch with all packages I need , and I do not update it .
it is not important how much packages I download at work place . maybe many of them never use
That's why I wrote Powerpill Portable. It should do exactly what you need.
I see Powerpill script , but I think this script does not work on FreeBSD Box
I need script for download all packages only and work on FreeBSD box

You have not understood. Please read the information on the page that I linked. I have written a tool to do what you need.

Download the "powerpill_portable.sh" script from my site.
Install "powerpill" and "rsync" on your home computer if you don't have them.
Mount a USB drive or something similar on your home computer and change directory to the drive.
Run powerpill_portable.sh in the drive directory. It will create a directory named "portable".
Unmount the drive and take it to work, then mount it there.
Install Perl and Aria2 on the work computer if you don't have them.
Open "portable" on your work computer and change into the directory.
Run "./pp -Syu" to download all upgradable packages for your home computer.
Run "./pp -S [pkgnames]" to download any other packages (replace "[pkgnames]" with actual package names, e.g. "python2").
Unmount the drive and take it home, then mount it there and change into the "portable" directory.
Run "./sync" (as root) to update the home computer's sync database and copy the packages to the cache.
Run "pacman -Su" to upgrade your system.
Run "pacman -S [pkgnames] to install the other packages that you downloaded.
Run "powerpill_portable.sh" again to repeat.

Last edited by Xyne (2010-12-16 08:40:14)

mfaridi · 2010-12-18 14:44:31

sorry for cross-post , but my next post is different from this post , but two post has same subject
in next post I say I downloaded all packages and i have many packages right now , in this link
https://wiki.archlinux.org/index.php/Of … f_Packages
I see I need only packages are in these directories

 core

 extra

 community

 mulilibe

what I must do with other packages are in other directories ?
can I use them later ?

Xyne · 2010-12-18 19:13:29

@mfaridi
I do not believe that you have understood my previous posts. There was no reason for you to download 22 GB of packages as you describe here. It is a waste of server bandwidth. It seems that you have not even tried to use the tool that I created for you. It is very frustrating to try to help someone who does not listen to others' suggestions, but I understand that there is a language barrier here.

I will try to answer your questions from the other post.

mfaridi wrote:

1- which version of arch work better i686 or x86_64 ?

Use the forum search form to find threads comparing i686 to x86_64. If you have a 64-bit processor then you should probably use x86_64.

mfaridi wrote:

2- how I can say pacman or other application like pacman , use my hard disk for install packages , or how I can make localrepo in arch ?

The answer is in /etc/pacman.conf:

# An example of a custom package repository.  See the pacman manpage for
# tips on creating your own repositories.
#[custom]
#Server = file:///home/custompkgs

mfaridi wrote:

3- can I put all packages from all directories in one folder or one directory and make localrepo ? or I have to put all packages in different directories and then make localrepo ?

You can combine [core], [extra], [community], and [multilib] in a single repo. Do not include [testing] and [community-testing] because they may be unstable.

Again, if you use the tool that I posted (Powerpill Portable) then you do not need to create a local repo or download 22 GB of packages. It is much simpler.
It has been added to the wiki page: https://wiki.archlinux.org/index.php/Of … l_Portable

Last edited by Xyne (2010-12-18 19:13:55)

mfaridi · 2010-12-19 07:12:57

Thanks all guys
My big problem is this
I do not have internet connection in home , for work powerpill I need to download 17MB of packages like perl , libxml2, and rsync and ... other packages . without this packages powerpill does not work

Last edited by mfaridi (2010-12-19 07:17:56)

karol · 2010-12-19 07:44:08

mfaridi wrote:

Thanks all guys
My big problem is this
I do not have internet connection in home , for work powerpill I need to download 17MB of packages like perl , libxml2, and rsync and ... other packages . without this packages powerpill does not work

Downloading many GBs of packages in not a problem, but downloading and installing 50 MB is?

You can do it like this: one day you grab the current DBs at work and bring them home, run 'pacman -Syup' [1] like I suggested earlier and save the urls to a file. Take that file to work, download the packages using curl / wget etc. and bring them home.
https://wiki.archlinux.org/index.php/Of … od:_Pacman but you download the DBs by hand.

[1] Actually, you should 'pacman -Sup' - without the 'y'. My mistake.

Last edited by karol (2010-12-19 07:50:21)

mfaridi · 2010-12-19 08:07:05

karol wrote:

mfaridi wrote:
Thanks all guys
My big problem is this
I do not have internet connection in home , for work powerpill I need to download 17MB of packages like perl , libxml2, and rsync and ... other packages . without this packages powerpill does not work
Downloading many GBs of packages in not a problem, but downloading and installing 50 MB is?
You can do it like this: one day you grab the current DBs at work and bring them home, run 'pacman -Syup' [1] like I suggested earlier and save the urls to a file. Take that file to work, download the packages using curl / wget etc. and bring them home.
https://wiki.archlinux.org/index.php/Of … od:_Pacman but you download the DBs by hand.

[1] Actually, you should 'pacman -Sup' - without the 'y'. My mistake.

I can download all packages at work m but I do not have internet connection in home , so for run or work powerpill I need perl and other dependencies in home computer , so in home I can not download packages

karol · 2010-12-19 08:46:00

You can download perl or any other package by hand http://www.archlinux.org/packages/core/ … /download/ (for 32-bit)
Of course that means you have to take care of the dependencies manually.

Plus you've got perl on the install iso if you still have it around.

Last edited by karol (2010-12-19 08:50:16)

lymphatik · 2010-12-19 08:55:51

Perhaps he does not have root access at work. So installing something might be complicated ;(

karol · 2010-12-19 08:56:41

lymphatik wrote:

Perhaps he does not have root access at work. So installing something might be complicated ;(

He can still download the packages using wget / curl etc.

hokasch · 2010-12-19 11:45:13

l I need perl and other dependencies in home computer , so in home I can not download packages

Yes, but since you already downladed a snapshot of the whole repo - you caneither manually install them with "pacman -U <package>", (start with powerpill/rsync and run down the dependencies) or just use

#[custom]
#Server = file:///home/custompkgs

for the moment.

Last edited by hokasch (2010-12-19 11:46:00)

Xyne · 2010-12-19 16:12:14

mfaridi wrote:

I can download all packages at work m but I do not have internet connection in home , so for run or work powerpill I need perl and other dependencies in home computer , so in home I can not download packages

So install powerpill and rsync from the packages that you have downloaded now, and use powerpill-portable in the future.

mfaridi · 2010-12-19 16:57:20

thanks all guys
If during of installation of Arch some important packages like perl , wvdial and ... installed by default it was good and great . for example powerpil and all dependency installed by default it is good

Xyne · 2010-12-19 18:40:01

mfaridi wrote:

thanks all guys
If during of installation of Arch some important packages like perl , wvdial and ... installed by default it was good and great . for example powerpil and all dependency installed by default it is good

One of the main philosophies of Arch is to let users decide what to install. It might make sense to include certain packages on the offline installation disk though, but they should not be installed by default.

mfaridi · 2010-12-20 05:51:46

thanks all guys
this link help me install Arch Linux with some packages like xorg , kde , ....
Arch is very faster than Fedora
https://wiki.archlinux.org/index.php/Of … od:_Pacman
I download all packages from sites with wget , all packages are over 25GB and then move them to

/var/cache/pacman/pkg

and then I run these command

mkdir -p /var/lib/pacman/sync/{core,extra,community}
rm -r /var/lib/pacman/sync/{core,extra,community}/*
tar -xzf core.db.tar.gz      -C /var/lib/pacman/sync/core
tar -xzf extra.db.tar.gz     -C /var/lib/pacman/sync/extra
tar -xzf community.db.tar.gz -C /var/lib/pacman/sync/community
tar -xzf multilib.db.tar.gz  -C /var/lib/pacman/sync/multilib

and after that I use this command to install packages

 pacman -S mc

and after that MC install.
but I have three questions ?
1- why we run this command

 rm -r /var/lib/pacman/sync/{core,extra,community}/*

2- after this how ّّI can have up to date system ?
3- how I can have offline update ?
and right now I have Arch

Arch Linux

#1 2010-12-15 21:04:59

Download all packages and them move them to home

#2 2010-12-15 21:31:01

Re: Download all packages and them move them to home

#3 2010-12-16 06:14:53

Re: Download all packages and them move them to home

#4 2010-12-16 06:28:17

Re: Download all packages and them move them to home

#5 2010-12-16 07:32:56

Re: Download all packages and them move them to home

#6 2010-12-16 07:40:20

Re: Download all packages and them move them to home

#7 2010-12-16 07:54:11

Re: Download all packages and them move them to home

#8 2010-12-16 08:07:23

Re: Download all packages and them move them to home

#9 2010-12-16 08:17:51

Re: Download all packages and them move them to home

#10 2010-12-16 08:19:43

Re: Download all packages and them move them to home

#11 2010-12-16 08:27:50

Re: Download all packages and them move them to home

#12 2010-12-16 08:39:02

Re: Download all packages and them move them to home

#13 2010-12-18 14:44:31

Re: Download all packages and them move them to home

#14 2010-12-18 19:13:29

Re: Download all packages and them move them to home

#15 2010-12-19 07:12:57

Re: Download all packages and them move them to home

#16 2010-12-19 07:44:08

Re: Download all packages and them move them to home

#17 2010-12-19 08:07:05

Re: Download all packages and them move them to home

#18 2010-12-19 08:46:00

Re: Download all packages and them move them to home

#19 2010-12-19 08:55:51

Re: Download all packages and them move them to home

#20 2010-12-19 08:56:41

Re: Download all packages and them move them to home

#21 2010-12-19 11:45:13

Re: Download all packages and them move them to home

#22 2010-12-19 16:12:14

Re: Download all packages and them move them to home

#23 2010-12-19 16:57:20

Re: Download all packages and them move them to home

#24 2010-12-19 18:40:01

Re: Download all packages and them move them to home

#25 2010-12-20 05:51:46

Re: Download all packages and them move them to home

Board footer