You are not logged in.

#1 2005-02-25 18:37:02

lilsirecho
Veteran
Registered: 2003-10-24
Posts: 5,000

Multiple "cloop" application :LiveDVD 3.8GB burned

The following is copied from Knoppix DVD forum on this date;
=====================================================================================

If you use Linux 2.6 with driver mapper support you can go around 2Gb limit a little bit differently - split your uncompressed image into chunks (on a block boundary) which would compressed into under 2Gb sized images, use separate cloop devices for each of them and then use 'linear' target to put them all together. Try something like this (assuming 65536 block size and 16Gb uncompressed image):

split -b 4294967296 KNOPPIX-UNCOMPRESSED.iso KNOPPIX.iso.
create_compressed_fs KNOPPIX.iso.aa 65536 > KNOPPIX0-gz.iso
create_compressed_fs KNOPPIX.iso.ab 65536 > KNOPPIX1-gz.iso
create_compressed_fs KNOPPIX.iso.ac 65536 > KNOPPIX2-gz.iso
create_compressed_fs KNOPPIX.iso.ad 65536 > KNOPPIX3-gz.iso

modprobe cloop
losetup /dev/loop0 KNOPPIX0-gz.iso
losetup /dev/loop1 KNOPPIX1-gz.iso
losetup /dev/loop2 KNOPPIX2-gz.iso
losetup /dev/loop3.KNOPPIX3-gz.iso

dmsetup create knoppix <<EOF
0 8388608 linear /dev/loop0 0
8388608 8388608 linear /dev/loop1 0
16777216 8388608 linear /dev/loop2 0
25165824 8388608 linear /dev/loop3 0

mount -r -t iso9660 /dev/mapper/knoppix /KNOPPIX

So, here - you'll have your under 2Gb file sizes on iso9660 and a nice 16Gb uncompressed file system. 
The only problems are that Linux 2.4 does not have device mapper and you need to adjust your mounting schema for /KNOPPIX during boot.

Hope it helps,
Cheers,
Igor

P.S. It may not work with the current create_compressed_fs due to a small bug - it adds an extra block if uncompressed image size is evenly divided by the block size. It doesn't show up when you have just one image, but with many it will show up in the middle of your file system.
If you have python installed you can use "my cloop python utilities"  ftp://belyi.ac93.org/pub/cloop-pyutils.tar.gz as a workaround. Otherwise, you should apply a tiny patch provided with a debian bug report: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=296179


lilsirecho


Joined: 13 Jul 2004


Posts: 166


Location: Ca

Posted: Sat Feb 26, 2005 3:44 am

belyi;

=====================================================================================

The above describes a possible application to establish a 16GB (uncompressed) filesystem by applying cloop technique multiple times.


Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

Offline

#2 2005-02-26 02:26:59

lilsirecho
Veteran
Registered: 2003-10-24
Posts: 5,000

Re: Multiple "cloop" application :LiveDVD 3.8GB burned

A question popped into my mind about the multiple "cloop" idea.

How would the cheatcodes interface in such an arrangement?


Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

Offline

#3 2005-02-27 01:50:01

lilsirecho
Veteran
Registered: 2003-10-24
Posts: 5,000

Re: Multiple "cloop" application :LiveDVD 3.8GB burned

The author of the multi cloop procedure has provided his details for a split-knoppix DVD boot of Knoppix26.  These are reprinted below;
=================================================================================For a proof of concept I've created a new minirt26.gz which recognizes KNOPPIX.0 .. KNOPPIX.3 images as a split KNOPPIX image. I've tried it on KNOPPIX 3.7 CD by spliting its compressed image in half.
You can get this 1.4Mb minirt26.gz from ftp://belyi.ac93.org/pub/KNOPPIX/minirt26.gz but I have very small upload speed so it may be easier if I tell you what I did.

1. Mounted the original minirt26:
modprobe cloop
mount -o loop KNOPPIX-3.7.iso /mnt/src
losetup /dev/cloop0 /mnt/src/KNOPPIX/KNOPPIX
mount /dev/cloop0 /mnt/root
gzip -c /mnt/src/boot/isolinux/minirt26.gz > minirt26
mount -o loop minirt26 /mnt/minirt

2. Edited /mnt/minirt/linuxrc (here's the patch: ftp://belyi.ac93.org/pub/KNOPPIX/linuxrc-dm.patch)

3. Statically compiled losetup and dmsetup utils (simply added -static switch to the linker) and copied them into /mnt/minirt/static

4. Statically compiled a small program dmtables (ftp://belyi.ac93.org/pub/KNOPPIX/dmtables.c and copied it into static as well.
gcc -static -o dmtables dmtables.c
cp dmtables /mnt/minirt/static

5. Copied driver mapper kernel module:
cp /mnt/root/lib/modules/2.6.9/kernel/drivers/md/dm-mod.ko /mnt/minirt/modules/dm-mod.o

6. Created additional cloop nodes:
pushd /mnt/minirt/dev
mv cloop cloop0
ln -s cloop0 cloop
mknod cloop1 b 240 1
mknod cloop2 b 240 2
mknod cloop3 b 240 3
popd

7. Unmount and compress minirt26 back:
umount /mnt/minirt
gzip minirt26

8. Create your splitted KNOPPIX image:
rsync -a --exclude=KNOPPIX/KNOPPIX /mnt/src/ newimage
extract_compressed_fs /mnt/src/KNOPPIX/KNOPPIX | split -d -a 1 -b 973696k - knoppix.decomp.
create_compressed_fs knoppix.decomp.0 > newimage/KNOPPIX/KNOPPIX.0
create_compressed_fs knoppix.decomp.1 > newimage/KNOPPIX/KNOPPIX.1
cp minirt26.gz newimage/boot/isolinux
mkisofs -pad -l -r -J -v -V "KNOPPIX" -no-emul-boot -boot-load-size 4 -boot-info-table -b boot/isolinux/isolinux.bin -c boot/isolinux/boot.cat -hide-rr-moved -o KNOPPIX-3.7-splited.iso newimage

9. Burn the image and then boot it using knoppix26

10. If it works you can start fill KNOPPIX with your favorite programs without worry for 2Gb limitation of iso9660. 


Note 1. dmtables just simulates the job of blockdev --getsz, plus some arithmetics to align those split images nicely for driver mapper. I suspect you can do its job easily with some other methods.

Note 2. This is just quick-and-dirty work - it is not intended for mass production.

Note 3. I didn't use extrace_compressed_fs and create_compressed_fs programs - I used my own utils but it shouldn't be matter as long as you use the patched version.


I appologize for being too verbose... Spending 20 minutes to create an image and then finding a typo in linuxrc during boot (repeated number of times) do get on my nerves. I thought I should spill it on somebody else too. 

Cheers,
Igor
===================================================================================

The knoppix forum for DVD has the original reference in the first DVD posting.  The minirt26.gz file is available at 1.4MB from the link provided therein.

I am not expert enough to be able to try this out, but someone will!!!


Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

Offline

#4 2005-02-27 03:41:36

lilsirecho
Veteran
Registered: 2003-10-24
Posts: 5,000

Re: Multiple "cloop" application :LiveDVD 3.8GB burned

See also knoppix ideas forum post.....managing cloop images with python scripts....author belyi.


Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

Offline

#5 2005-02-27 15:54:48

lilsirecho
Veteran
Registered: 2003-10-24
Posts: 5,000

Re: Multiple "cloop" application :LiveDVD 3.8GB burned

Latest in Knoppix DVD forum is a post from chris-uk.

He reports a torrent which provides Knoppix for windows that takes a week to leech at hgih download speed!!!!  Probably three or four days....?

He provides a url for the torrent.

He comments that the 2GB limit will be a thing of the past with Blue Ray.

Standards for Blue Ray seem nebulous at present unless I haven't seen the latest info.

Probable that 100GB with a single cloop Knoppix will be enough to "choke" a user..enough is too much!!!  Boot time?

Another "shot across the bow" to awaken the experimenter from his boredom!


Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

Offline

#6 2005-03-02 18:51:59

lilsirecho
Veteran
Registered: 2003-10-24
Posts: 5,000

Re: Multiple "cloop" application :LiveDVD 3.8GB burned

Belyi offers"split .iso" method allowing kernel2.4 to have 2GB and kernel 2.6 have 2GB providing 4.4GB total apps on a DVD.

It would seem possible to "split" the latest LiveCD , record to DVD .iso format and add .cmg files after booting to permit deleteable apps up to the 2GB limit in either 2.4 or 2.6 kernel options.

This seems possible since each .iso of the "split" .iso have 2GB capability.

The total of the two cannot exceed 4.4GB on a dvd disc.

Perhaps this technique will increase read-speed as well.

Check it out in Knoppix DVD forum..


Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

Offline

#7 2005-03-07 04:14:27

lilsirecho
Veteran
Registered: 2003-10-24
Posts: 5,000

Re: Multiple "cloop" application :LiveDVD 3.8GB burned

I have successfully attempted to utilize the split idea proposed by Belyi but have used a different technique, along with his new-minirt26.gz file.

My technique utilizes the LiveCD for Knoppix2.6 as copied into folder CD in root.  One such copy is only of the KNOPPIX (and sub KNOPPIX) and associated smaller files but no copy of the "Boot" directory.

This CD is then converted to an .iso with the following;

     mkisofs -pad -l -r -J -v -V "KNOPPIX" -hide-rr-moved -o /root/KNOPPIX-suffix.iso /root/CD

This setup ignores all "boot" commands.

A second copy of the LiveCD was placed in CD1.  The "KNOPPIX" folder is deleted from CD1.

Next, the minirt26.gz is deleted.  The new-minirt26.gz from Belyi is copied into CD1(I renamed it minirt26.gz)

Then CD1 is operated on my mkisofs;

    mkisofs -pad -l -r -J -v -V "KNOPPIX" -no-emul-boot -boot-load-size 4 -boot-info-table -b boot/isolinux/isolinux.bin -c boot/isolinux/boot.cat -hide-rr-moved -o /root/KNOPPIX-prefix.iso /root/CD1

The combined (concatenated) .iso's are processed as follows;

   cat KNOPPIX-prefix.iso KNOPPIX-suffix.iso > KNOPPIX-full.iso

The KNOPPIX-full.iso is then burned to DVD+R/W on my system.

This procedure provided a 699MB bootable DVD in kernel2.6 with myconfig=scan fully operational.

I assume it is a split format at this time due to the use of Belyi's new-minirt26.gz in its formation.

Maybe someone can make this play at 4GB with added apps.

I could be wrong in assuming it has that capability!!!


Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

Offline

#8 2005-03-09 02:26:56

lilsirecho
Veteran
Registered: 2003-10-24
Posts: 5,000

Re: Multiple "cloop" application :LiveDVD 3.8GB burned

Belyi commented on the procedure I used and reports that it is a split but a limited split.

The suffix.iso is still limited to 2GB if my procedure is used.

He expects in the coming weeks to generate a new minirt26.gz procedure to enable a more simply added split for suffix.iso application.

The new procedure should provide a straightforward insertion of Knoppix material according to his advisory.


Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

Offline

#9 2005-03-12 01:23:56

lilsirecho
Veteran
Registered: 2003-10-24
Posts: 5,000

Re: Multiple "cloop" application :LiveDVD 3.8GB burned

Belyi in the Knoppix DVD forum has posted a new simplified technique for splitting the .iso to produce a greater than 2GB DVD+RW disc burn.

He supplies the minirt26.gz file needed via ftp and also provides a linuxrc diiff file  and a patch for those in need of same.

My use of the procedure for Knoppix 3.7 was successful in that it booted the DVD with Knoppix 3.7 CD apps and it operated correctly with myconfig.

I have yet to try a larger than 2GB install.

Have a look-see in Knoppix forum DVD....


Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

Offline

#10 2005-03-16 03:37:36

lilsirecho
Veteran
Registered: 2003-10-24
Posts: 5,000

Re: Multiple "cloop" application :LiveDVD 3.8GB burned

Belyi has posted in KnoppixDVD forum a split format for the new Knoppix3.8 Cebit download.

This represents the second Knoppix version formatted for split.

Split format is the simplest for generating large files (>2GB) using Knoppix OS in these cases.

As previously described, multiple .iso's can be combined to generate 16GB of apps with 4 such units.

My download of Knoppix3.8 Cebit was successfully split using the latest minirt.gz and I am using it at the moment to post.

Many thanks due to Belyi! big_smile


Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

Offline

#11 2005-04-03 06:10:54

lilsirecho
Veteran
Registered: 2003-10-24
Posts: 5,000

Re: Multiple "cloop" application :LiveDVD 3.8GB burned

After cleaning out my files and re-doing the re-master of KNOPPIX3.8 Cebit, I was successful in burning a bootable DVD of 748MB using the procedure provided by Belyi, called knxapp.gz as outlined in the KNOPPIX DVD forum.

The added apps appear in KDE menu and the DVD operates normally (writing this in the DVD boot at present).

The procedure outlined should allow ~3.7GB of added apps to the basic KNOPPIX 3.7 or 3.8 Live CD , producing a LiveDVD of 4.4GB compressed.

This is awesome!!!!!
yikes

Kudos to Belyi for his excellent support.

Speak softly..use braille.


Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

Offline

#12 2005-04-03 09:25:24

iphitus
Forum Fellow
From: Melbourne, Australia
Registered: 2004-10-09
Posts: 4,927

Re: Multiple "cloop" application :LiveDVD 3.8GB burned

who are you talking to? lol

if you want to be talking to the archie live cd devs, they're on www.user-contributions.org/forums/

Offline

#13 2005-04-03 14:09:15

lilsirecho
Veteran
Registered: 2003-10-24
Posts: 5,000

Re: Multiple "cloop" application :LiveDVD 3.8GB burned

This forum is LINUX Discussion forum...not limited to discussion about archlinux.

I am aware of Archie.  I have a LiveCD of Archie.

The Knoppix procedure referenced herein outstrips all other LiveCD capabilities.  It permits the loading of 4.4MB compressed to a DVD which with the usual procedures is limited to 2GB.  My demo trial shows that it works and has the potential.

I know of no other LiveCD/DVD doing likewise.

Its LINUX DISCUSSION and the information applies only to Knoppix at this time.

It may be useful to other users as well.

As a discussion of Knoppix in an Archie forum seems out of place, I prefer to use this forum.  If it applies to Archie LiveCD as it evolves, with appropriate mods, it may be adopted by the developer in some form.

The developer has been informed of these Knoppix capabilities but expressed a lack of time in his efforts with Archie to study the Knoppix situation

It is likely that the same capability will be forthcoming should the community move in that direction.  A large 4.4Gb DVD is an awesome upgrade for any LiveOS media and may not be popular.  Download time is a big factor.

Lean and mean may be best!

Speak softly..use braille.


Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

Offline

#14 2005-04-22 17:42:59

lilsirecho
Veteran
Registered: 2003-10-24
Posts: 5,000

Re: Multiple "cloop" application :LiveDVD 3.8GB burned

Utilizing the "split.iso" method designed by Belyi, I tested its performance with large files.

At first, tried to load the Video in KNOPPIX/KNOPPIX file but could not get a compressed file therein.

I then surmised I could add it to the "prefix" .iso.

After using too much Video in the first trial, I reduced it to 2.4GB after which a successful burn was made.

The Video was added as a subdirectory in the newimage/KNOPPIX directory, in the steps utilized to generate the "prefix.iso".  (Details in KNOPPIX DVD forum).

The concatenation of the "prefix.iso" and the "suffix.iso" generated a LiveDVD+RW of 3.8GB (3/16 radius left on DVD disc!).

This exercise was performed to test the "split" system.  It results in an interesting combination of file systems....compressed cloop, uncompressed Video, UNIONFS files, and Klik compressed image files.  These are all operable on this LiveDVD+RW (re-burnable,too)!

The Video files are accessible on the CDROM.  To eliminate transfer delays, copying the Video to Desktop permits the Video to run in RAM, uncompressed!

My system is athlon 1900+ with up to 3GB RAM and I use a Sony DRU530 DVD burner on x4 media.  My motherboard is asus a7v333 with 2100RAM sticks.  My hdd is 80GB 133spd.

I used about 15GB of HDD in producing this DVD, managing the content at each step to reduce the HDD required and keep RAM use manageable.

I have two HDD's available each having 2GB of swap available, thus with 3GB of RAM and 4GB of swap, The total useable is near 7GB (some reserved for system operations,housekeeping,etc.).

Compressed files must be processed in the cloop system provided for the KNOPPIX version being "split" and requires that the KNOPPIX be booted to enable the use of the cloop system to perform the compression. AFAIK....

This was a test only to check the "split" system in enhancing the utilization of DVD recording space-on-disc which is normally held to file size of 2GB.  As such, the test proves that the full DVD+RW disc space can be utilized with the "split" system.

Compressing a full sized DVD probably requires 9GB of uncompressed applications.  That is thousands of applications!


Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

Offline

#15 2005-06-28 17:12:44

lilsirecho
Veteran
Registered: 2003-10-24
Posts: 5,000

Re: Multiple "cloop" application :LiveDVD 3.8GB burned

Klaus Knopper has generated KNOPPIX 4.0 "maxi" DVD with 4GB of program files, cloop compressed for a 9GB uncompressed Live DVD.

Bittorent for this DVD is available...see the KNOPPIX.net forum NEWS.

The cheat codes are available to obtain language selection and myconfig can be implemented.

This method employs two /KNOPPIX files, each 2GB and cloop compressed.  Unionfs is employed to generate the additional 2GB /KNOPPIX file.

The technique  is similar to the prefix/suffix arrangement but improves it by providing cloop compression in the added 2GB, a cloop feature I haven't had success with in my attempts to generate the full-size DVD.

The 4GB size of the DVD permits FAT system use of the DVD .iso.

In operation, the system is slower than the CD version.  Boot time is near 3 minutes with KDE on i686 machine.

It is compatible with the CMG KLIK system.  Background image is time-lapse night driving the autobahn.

Theoretically, 400mb of data can still be added to the DVD to utilize the last of the disc space.

Opera is available as a cmg file and extends the internet capability with minimal impact.  Selection of OPERA via cmg is much faster access than the selection of firefox which is a component of the DVD, although OPERA requires a few selection clicks to enable.

It is a big, big, big LIVE DVD!!!!


Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

Offline

Board footer

Powered by FluxBB