Minimalist apps and file search question

ParanoidAndroid · 2013-09-28 01:08:28

I'm reorganizing my applications and interface method and I have a few questions:

1. I currently use mailx as my MUA, but I would like to know if there is anything more minimal. Just plain-text, no ncurses, just to compose mail (reply, compose with attachments, etc) and to read and manage my maildir inbox. I use external programs to handle getting and sending mail, I just need a minimal program to integrate with the others.

2. I would like to be able to do a full-text search of all plain-text files in my home folder from the command line, such that I can find any file by searching for a phrase or string in it's contents. I'm aware grep can do this, but it can't search using a string of more than one word, so far as I can tell. This is ill-suited to what I have in mind. If I could do such a search, then I could simply rename all my text files with a 5-character alphanumeric code, instead of names. In this way I can call up a file by searching it's contents and entering it's code. I can write a function to handle this, but I need the text-search capacity in order to do so.

3. I need to generate such an ID for all files. That is, I need to rename every text file I have, excepting dotfiles, with a random 5-character string. How could I do this, say, with a script? A for loop with the mv command and mkpassword to generate the filename, but I'm not sure exactly how to go about that.

4. what are some raw-text (no ncurses) apps that I can use for common tasks? I already use vim, mailx, mpc, calc, and utilities like cat, ls, mv, etc. to manage files. My goal is to get all needed functionality with the most minimal programs possible.

Trilby · 2013-09-28 01:58:00

1) ~~vim~~ ed
2) grep can do more than one word. ~~But (m)locate seems better suited to what you describe~~
3) random? Or just unique? If unique, just run them in sequence.
4) what common tasks?

Last edited by Trilby (2013-09-28 02:36:56)

cfr · 2013-09-28 02:20:51

How do you get (m)locate to find files by content? I always thought it worked with a database describing directories and their contents but not the contents of the files themselves.

@ParanoidAndroid,
May I ask why you want to rename all your text files with randomly generated ID numbers? That is, not why you want them to be random but why you want to rename them that way at all. It does not seem the most user-friendly solution!

Unless you have relatively few text files, it is going to take a while to find the file you want by grepping everything in your home directory. If something like locate really can do this for content, that will be quicker but it will not update in real time because it relies on periodic updates to a database.

The whole thing sounds incredibly annoying to me which is why I'm really curious as to why you want to do it at all. (Not the minimal bit - that's fine. The renaming all your files with meaningless sequences bit is the aspect I find puzzling.)

Trilby · 2013-09-28 02:26:35

Yes, perhaps I spoke to quickly. Grep could do it, but would be very slow for any real use. An indexer is what is needed, and while locate does index, I wasn't thinking it through and I don't believe it indexes content. NotMuch indexes content of email - and if needed one could force it to index any text files - but there are likely similar tools for that purpose.

As for the names, also why 5 digits. If you can allow for a couple more digits, you could just rename every file to it's inode. This would ensure unique names for each file, and would allow for easy generation of the "name" of any new file without having to check any listing or database for uniqueness (at least not an additional check over what the filesystem would be doing anyways).

EDIT: even if you're restricted to 5 digits, you could code inodes in base 62 (0-9A-Za-z), then you'd definitely never need the 6th digit (could have just under 1 bilion inodes) - probably not even the 5th (~1.5 million inodes).

Last edited by Trilby (2013-09-28 02:49:13)

WonderWoofy · 2013-09-28 02:29:55

@Trilby, vim depends on ncurses.

Trilby · 2013-09-28 02:31:15

True, OP said they already use vim though - so if it's there anyways.

But without vim, just use ed.

ParanoidAndroid · 2013-09-28 15:59:05

I know vim relies on Ncurses, but I simply couldn't live without it. It's fantastic.

@Trilby,
Thank you for the inode bit. How would one go about that, exactly?

As for 'common tasks'? Music, mail, all office needs, web browsing, media playing, and so on. I have most if not all of this taken care of except for mail, I use (in order of the above) mpc; mailx, which I'm looking to replace; vim, txt2tags, unoconv, when+cal, calc, and magicpoint; uzbl for browsing and mplayer for video. I would like some input, though, if you guys can think of anything even more minimalist and command line oriented. I use uzbl instead of lynx because I need a graphical, java-capable web browser and uzbl was perfect in terms of minimalism and flexibility. My desktop environment comprises xmonad, conky, and urxvtd/urxvtc. Conky and uzbl (plus lowriter as a dependency for unoconv - curse it) are about the only graphical apps I have aside from urxvt.

@ cfr,
Well, I ran across a wikipedia article describing Archy, a radically new type of computer interface. I'm trying to mimmick some of it's properties, such as:

- persistence

- universal searchability based around text files

naming documents by content - this is what got me thinking about the proposed naming scheme. Instead of naming a
file, I would simply search for it's contents with a command-line utility and then either open it using it's identifier string or pipe the output of the search command to vim (something vim always gets pissed at me for doing, so I'm not sure about that last one.)

- applications that work on text files, i.e. vim, mailx, etc.

- crash resistance - changes are written to disk as they occur, instead of when memory pressure flushes the cache to disk. This is a behavior I'm unsure how to implement, mostly because I have an SSD and I'm wondering whether this would degrade lifespan and performance. I currently use JFS as my filesystem, and it seems pretty good at crash recovery already.

WonderWoofy · 2013-09-28 17:01:01

It just seems as though you are trying to make things harder than is necessary. I'm not trying to discourage you from your plan here, but it just seems that since you already have ncurses as a dependecy for vim, why not use it? I am a happy mutt user, and in combination with an indexer (I have tried a few, but always come back to notmuch-mutt), it seems like all the functionality that you require can be had with that… with the exception of the "no ncurses" thing of course.

I guess I'm just curious why you are adamant about moving to tools that don't use ncurses even though it sounds as though you aren't about to give up vim any time soon.

cfr · 2013-09-28 20:36:52

I still don't see why you want to rename all the files. Nothing you describe requires that that I can see.

At a minimum, I would suggest testing the system without renaming them first so that you can determine whether you find the latency acceptable and whether the system actually enables you to find the files you want.

ParanoidAndroid · 2013-09-29 22:46:15

@WonderWoofy,
mostly it's just a matter of personal preference. I like using non-ncurses, raw-text tools because a) if my system is built on them, I can script for anything b) I like the idea of entering a one-liner or interactive text sessions rather than mucking about in visual menus c) I believe the GNU philosophy (write apps to handle text streams, since that is a universal interface) is a good one, and d) I feel that it's faster in a keyboard-driven environment compared to an ncurses app like mutt. Feel free to dispute any of those points, but that's just my personal opinion on the matter. As for why I'm using vim, well... I know it uses ncurses, but it's just so nifty and useful that I couldn't live without it.

@cfr,
As I said, I got the idea from Archy, which sounded really intuitive and awesome to me. But again, that's just my opinion. Since it's not likely I'll get anywhere with that, though, due to various technical constraints, may I return to the earlier bit of my initial question? I'm looking for a better non-ncurses alternative to mailx, if such a thing exists, one that is purely designed to handle creating mail and managing my inbox, a pure MUA that doesn't do any fancy filtering or sending or so on, merely passes a message on to an external program to send or reads a maildir to present/manage to the user.

WonderWoofy · 2013-09-30 00:23:48

ParanoidAndroid wrote:

@WonderWoofy,
mostly it's just a matter of personal preference. I like using non-ncurses, raw-text tools because a) if my system is built on them, I can script for anything b) I like the idea of entering a one-liner or interactive text sessions rather than mucking about in visual menus c) I believe the GNU philosophy (write apps to handle text streams, since that is a universal interface) is a good one, and d) I feel that it's faster in a keyboard-driven environment compared to an ncurses app like mutt. Feel free to dispute any of those points, but that's just my personal opinion on the matter.

I wasn't trying to tell you that you were wrong, I was just honestly curious. The whole idea of being able to script things is certainly a strong argument.

As for why I'm using vim, well... I know it uses ncurses, but it's just so nifty and useful that I couldn't live without it.

Damn straight its nifty! When I started to learn how to use it I remember thinking "WTF?!" But after I got used to it, I don't know that I would be able to keep my sanity if it were somehow taken away.

Trilby · 2013-09-30 01:16:50

WonderWoofy wrote:

I don't know that I would be able to keep my sanity if it were somehow taken away.

No, you wouldn't keep your sanity: you'd use emacs.

WonderWoofy · 2013-09-30 01:41:30

Trilby wrote:

WonderWoofy wrote:
I don't know that I would be able to keep my sanity if it were somehow taken away.
No, you wouldn't keep your sanity: you'd use emacs.

Exactly my point!

…not that I'm claiming outright sanity to begin with.

ewaller · 2013-09-30 02:14:34

Hey! I use emacs. No, wait...

ANOKNUSA · 2013-09-30 03:07:07

STUFFS is an in-house project that seems to be along the lines of what you're asking for for filesystem organization and searches. It's pretty new, though, so frankly I don't know how reliable it is in a production environment. It definitely doesn't seem to be usefull as a full-block-level filesystem. I've played with it a bit, and it's pretty interesting, though a bit odd; I've been thinking about trying to organize my ebook and comics collections with it for quick searches, to see how well it might hold up.

ElKatrina · 2013-09-30 19:13:10

For office stuff (documents and presentations), try LaTex or XeTeX or one of the other TeX variants. No more mucking about in a word processor---you type your document in a text editor (vim), and then call LaTeX to typeset it for you (into a PDF or whatever other format---much more universal than .doc(x) or .odt).

This would help you, as you can still grep/search your documents (since they are written in marked-up plain text) but you can still create nice-looking documents.

For spreadsheet stuff, use plain text files, and gnuplot to make graphs out of them.

Edit And ten thumbs up for using text streams. I just finished writing a script to wget my internet usage data from my ISP, write it into a text file, run gnuplot to make it into a pretty graph, and then typeset the graph and some other information into a PDF using LaTeX. Try doing THAT with Calc and friends!

Last edited by ElKatrina (2013-09-30 19:15:33)

ParanoidAndroid · 2013-10-01 19:53:59

@ElKatrina,
I currently use txt2tags as a markup system, but since I'll be off to college soon LaTex looks like a better option. Much more flexible and powerful, though it looks a bit intimidating. Gnuplot looks useful, too.

Would anyone happen to know of a MUA that fits my stated criteria? I used to use mutt, but I don't like the interface all that much and the color scheme can't be specified in hex, so I switched to mailx. I'd be happy with mailx, except that it doesn't fit my philosophy. NMH seems ideal, except that it doesn't read from maildirs. Are there any similar programs or suites of programs that can function like that or as described in my first post?

WonderWoofy · 2013-10-01 19:57:53

Just so that it is noted in this thread, the default heirloom-mailx is apparently no longer actively developed, and recently, a fork called s-nail was put into [testing]. It will replace heirloom-mailx in both the repos and the "base" group. It is apparently a drop-in replacement (though I don't use either, so cannot attest to this). So anyone reading this thread should know that if they are trying to follow this route of mail use, they should be looking for s-nail rather than heirloom-mailx.

Okay, carry on…

Trilby · 2013-10-01 20:34:10

If you have tools to sync/download all your mail to a local maildir, what exactly do you want out of a MUA?

Just open your mail in your favorite text editor, and draft emails in the same way. A few simple bash scripts can automate the repetative parts of dealing with the headers - but so too will nearly any program that manages/creates the maildir.

ParanoidAndroid · 2013-10-01 20:46:33

I need an MUA that can do the following:
1. compose/reply to mail (with attachments) and hand it off to msmtp to be sent
2. display, manage, and navigate the contents of my maildir (display messages with less, delete messages, etc)
3. download/handle attachments in an email
4. display html mail through lynx (dumping the mail to stdin or less)

and that's it. No fancy filtering, no built-in imap or pop, nothing.

Trilby · 2013-10-01 22:11:07

I just don't see anything in that list that a text editor with a few well chosen macros wouldn't do.

ElKatrina · 2013-10-01 22:18:21

ParanoidAndroid wrote:

I currently use txt2tags as a markup system, but since I'll be off to college soon LaTex looks like a better option. Much more flexible and powerful, though it looks a bit intimidating.

I found it pretty easy to learn, at least for basics. For college it is exceptionally good because you also get BibTeX for bibliographies, such a timesaver. The only document you'll need for a good long while is the Not so short guide to LaTeX, it's great to learn LaTeX. Then when you get more advanced, there are plenty of resources on the web. It's like arch: it doesn't babysit you, but there's plenty of good help to be had on the web.

ParanoidAndroid wrote:

I used to use mutt, but I don't like the interface all that much and the color scheme can't be specified in hex,

I can use 256 colours in my mutt, though they are defined using numbers from 0-255. If this really bothers you, it should be fairly simple to define user-defined variables as hex codes that have the values of the colour names, e.g.

$my_ff0000 = 196
$my_ff005f = 197
...

and then you just set colours like

color tree normal $my_ff0000

(I haven't actually tried this though).

You could write a pretty trivial script that takes the output from the colortest script and generates those variables for you

ParanoidAndroid · 2013-10-01 22:32:37

@Trilby,
Can you give an example of some macros? My mail-fetching program (isync, mbsync is the executable name) uses lengthy UID-type things to identify messages, so opening them with vim individually and directly isn't intuitive. My biggest concern is attachments. I could just use cat message.txt | msmtp -a gmx to send mail, but I can't send attachments directly through msmtp, as far as I know. I need a program to handle message composition with attachments, or at least that's what my endless Google-quest has shown me.

Trilby · 2013-10-01 22:35:50

I'll put some of my stuff up in a github repo in the next couple of days. It's a buch of little things that work together pretty well, but they are very much a work in progress as I'm still generally pretty happy with mutt.

But all of them use vim (which depends on ncurses too). But basically it's a tiny program - which could just as well be a shell script - that produces a list of subject line and sender names. Then there are a couple vim bindings for navigating between the "index" type view and the content of each message.

ewaller · 2013-10-01 22:42:35

ParanoidAndroid wrote:

I need an MUA that can do the following:
1. compose/reply to mail (with attachments) and hand it off to msmtp to be sent
2. display, manage, and navigate the contents of my maildir (display messages with less, delete messages, etc)
3. download/handle attachments in an email
4. display html mail through lynx (dumping the mail to stdin or less)
and that's it. No fancy filtering, no built-in imap or pop, nothing.

emacs will do all all of that right out of the box.
If you add in emacs-w3m, I think you can even display html email rendered in all its spammy glory right in the editor.

Edit: That includes all of the necessary GPG hooks for encryption and signing

Last edited by ewaller (2013-10-01 22:43:34)

Arch Linux

#1 2013-09-28 01:08:28

Minimalist apps and file search question

#2 2013-09-28 01:58:00

Re: Minimalist apps and file search question

#3 2013-09-28 02:20:51

Re: Minimalist apps and file search question

#4 2013-09-28 02:26:35

Re: Minimalist apps and file search question

#5 2013-09-28 02:29:55

Re: Minimalist apps and file search question

#6 2013-09-28 02:31:15

Re: Minimalist apps and file search question

#7 2013-09-28 15:59:05

Re: Minimalist apps and file search question

#8 2013-09-28 17:01:01

Re: Minimalist apps and file search question

#9 2013-09-28 20:36:52

Re: Minimalist apps and file search question

#10 2013-09-29 22:46:15

Re: Minimalist apps and file search question

#11 2013-09-30 00:23:48

Re: Minimalist apps and file search question

#12 2013-09-30 01:16:50

Re: Minimalist apps and file search question

#13 2013-09-30 01:41:30

Re: Minimalist apps and file search question

#14 2013-09-30 02:14:34

Re: Minimalist apps and file search question

#15 2013-09-30 03:07:07

Re: Minimalist apps and file search question

#16 2013-09-30 19:13:10

Re: Minimalist apps and file search question

#17 2013-10-01 19:53:59

Re: Minimalist apps and file search question

#18 2013-10-01 19:57:53

Re: Minimalist apps and file search question

#19 2013-10-01 20:34:10

Re: Minimalist apps and file search question

#20 2013-10-01 20:46:33

Re: Minimalist apps and file search question

#21 2013-10-01 22:11:07

Re: Minimalist apps and file search question

#22 2013-10-01 22:18:21

Re: Minimalist apps and file search question

#23 2013-10-01 22:32:37

Re: Minimalist apps and file search question

#24 2013-10-01 22:35:50

Re: Minimalist apps and file search question

#25 2013-10-01 22:42:35

Re: Minimalist apps and file search question

Board footer