You are not logged in.

#1 2004-11-25 11:58:55

dp
Member
From: Zürich, Switzerland
Registered: 2003-05-27
Posts: 3,378
Website

fast compression of primitive ASCII?

the problem:
--------------

big files (~200-1600MB) with primitive contents (genetic sequences: containing only "1" "2" "3" "4" and "-" and "|" chars)

for a project i'm doing myself (coded in java) i need this data to be plain (because i didn't add any compressor/decompressor lib)

what i'm now searching is a way to use less harddisk space but still be able to access (scroll, jump ...) the sequences (the big files) at a fast speed (say: goto(538285) would jump imediately to char 538285 and read +1000 and -1000 chars from this position)


the ways i'm thinking about:
--------------------------------

- adding some lib to the project to let java compress/decompress the data (PRO easy implementable CONTRA slow )

- using a cloop partition (PRO faster CONTRA no idea how to do it with java (out of the app))

- ???

to anyone with more experience in coding and compression: what would you suggest? any literature you would suggest me to read about compressing?

thanx in advance


The impossible missions are the only ones which succeed.

Offline

#2 2004-11-25 15:52:33

z4ziggy
Member
From: Israel
Registered: 2004-03-29
Posts: 573
Website

Re: fast compression of primitive ASCII?

i think any use of compressed filesystem like cloop should be be better, since all the compression will be transparent to your java program, and u dont need to alter your sources (lazy-ass coders like me...), and u can always revet easily to non-compressed data... downside is, your program rely on the user providing the compression... but i see this also as advantage.
[EDIT]
also, user compression lets u change the compression library used on the fly.

Offline

#3 2004-11-25 19:25:30

Michel
Member
From: Belgium
Registered: 2004-07-31
Posts: 286

Re: fast compression of primitive ASCII?

There is a file-attribute to enable compression for that file or not, but it isn't supported wel yet I believe ...

Offline

#4 2004-11-25 19:51:16

dp
Member
From: Zürich, Switzerland
Registered: 2003-05-27
Posts: 3,378
Website

Re: fast compression of primitive ASCII?

z4ziggy wrote:

i think any use of compressed filesystem like cloop should be be better, since all the compression will be transparent to your java program, and u dont need to alter your sources (lazy-ass coders like me...), and u can always revet easily to non-compressed data... downside is, your program rely on the user providing the compression... but i see this also as advantage.
[EDIT]
also, user compression lets u change the compression library used on the fly.

seen like this, you are absolutely right! as addition, the data would be still human-readable, as it is transperently compressed

i'll go for the cloop solution - thanx for reply


The impossible missions are the only ones which succeed.

Offline

Board footer

Powered by FluxBB