Wrote the code... now what

Jack Spade · 2011-08-27 20:41:45

Hi all,

I am not - by any stretch - a professional, nor trained programmer. But I've always tinkered with computers and programming.

As is often said most great software comes from a developer "scratching a personal itch." I have had a few personal itches lately - and I don't have great software, but perhaps something interesting.

I've written a (currently very rough) curses-based console spreadsheet editor. Yes I know of Oleo and a couple other projects, but they just weren't quite what I wanted and I couldn't figure out if they could be configured to be what I wanted. So in the end I wrote my own.

It definitely follows the KISS principle, but as I've tried out a few features just for fun I've realized some were fairly easy to add without substantially increasing the program size or memory usage or decreasing speed. I now use my little tool for all my .csv file editing (I do a good bit of data analysis at work). It can read any similarly delimited file such as .tsv.

The first goal was for it to be much like a simple text editor (think nano) that would hide the commas and align the columns. It has come a little way since then. The files can be edited (of course) columns or rows deleted or inserted, and it has undo functionality.

I've just added a plotting function which uses GNU R (stats software) to generate plots of one, two, or three variables.

Its all written in C, and compiled is under 60KB.

The next version will include what I'm calling 'filters' for current lack of a better term. These are commands that can be given from within the program to filter the current file through any command line command or script such as awk, sed, perl/bash/python scripts, etc.

I am interested in getting feedback on how it works for others. But I am also curious what the next step in developing it should be. I'd like to think I'm pretty good at writing the code (in C), but try as I might I have never really gotten a hang of the GNU autotools stuff. I'm not sure if I need it for such a small program. The program is pretty basic - ncurses is the only thing (beyond standard C headers like stdlib.h stdio.h and string.h) that it should require. Of course it requires R for plotting - but it will compile and run just fine without R ... it just wouldn't plot.

I'm also not familiar with all the tools that seem to be used in more elaborate multi-developer projects or projects that are hosted on some server like (perhaps) git.

So, to boil down this long post I have two questions - Is anyone interested in trying out my code? and, what should be my next step in helping my baby grow up?

I have, for the moment, been calling the project 'mange' .... as all spreadsheet editors suck - but (I hope) this one just sucks less. [I have no connection whatsoever with the mutt developers - I just love the email client and hope my little project may someday be as useful to console-lovers].

As a side note, I've also just started a ncurses-based word processor - yes, I know of wordgrinder, but for similar reasons as above - I want my own.

Thanks for reading, I look forward to some feedback.

Last edited by Jack Spade (2011-08-28 15:00:49)

brebs · 2011-08-27 20:53:19

github seems popular.

karol · 2011-08-27 20:56:13

Any plans of pushing it to AUR? You can even use dropbox to hold the code (releases).

Jack Spade · 2011-08-27 21:02:02

EDIT: RE to Brebs:
Thanks - I'll look into that some more, but I should clarify that my little tool is not a collaborative project (which seems to be the purpose of github and the like). I just mentioned that because as I learn a bit about how other software gets 'out there' that seems to be discussed a good bit. So I suppose my question is more basic: is that even a step I should worry about for a single-man project?

Last edited by Jack Spade (2011-08-27 21:02:28)

Jack Spade · 2011-08-27 21:08:05

Karol, I would love to be able to contribute to the AUR at some point.

I've seen all the guides for *using* the AUR as a user - but I'm not sure what the first steps are for contributing to it.

Again, all I have is a few C source files. They compile smoothly on a few of my machines but I don't they just use a handcoded makefile (no autoconf or other such tools) and I have yet to learn about PKGBUILDS (except to know they exist).

I know there is a wealth of information on the wiki for each of these aspects ... each step is well explained in the wikis, but I guess I'm feeling a bit lost in terms of which steps I should be taking.

I am looking into dropbox now (even this is new to me ... again, on writing code I'm fairly proficient, on every other aspect of putting together a software project I'm a complete noob).

karol · 2011-08-27 21:08:47

From what I've seen a good deal of one-man-project uses github fro sharing the code with the users: it let's you track changes made to the projects etc.

Edit: For now you can just post those C files, makefile, README etc. somewhere on the net and give us the links so those feeling adventurous can have a go at it.

Last edited by karol (2011-08-27 21:11:16)

Jack Spade · 2011-08-27 21:20:30

I just set up a dropbox. The code is on my other computer, so it'll be a couple hours before I get the files up.

Thanks for the help and interest.

RichAustin · 2011-08-27 21:28:48

karol wrote:

Any plans of pushing it to AUR? You can even use dropbox to hold the code (releases).

I'd suggest using www.unfuddle.com for the code. I use it all the time, and it supports subversion and git repositories. You can have, free, as many repositories as you like. I've used it for about 3 years with no probs at all.

I use RapidSVN as gui for handling the repositories. I just prefer subversion to git that's all, but there isn't much difference. It's well worth looking at launchpad as well I would think. Never tried it other than downloading a zillion cool oss things from it though.

Last edited by RichAustin (2011-08-27 21:31:13)

ibrunton · 2011-08-27 22:15:57

One advantage of github and other version-control tools is that they make it very easy for others to download the code to build the software, and then keep it up-to-date. There are lots of one-person projects on github.

Also, don't be intimidated by PKGBUILD. It can be used to do some very complex things, but it can be pretty straightforward too. The wiki has excellent instructions.

Jack Spade · 2011-08-28 02:10:34

Thanks everyone. I'll start looking into these options.

For the time being I have the source in dropbox here:
http://dl.dropbox.com/u/39577570/mange_v0-1b.tar.gz

Again this is just a hand-coded configure/makefile - but I suspect it should compile without issue. I'm sure it's quite rough around the edges - but I am using it daily for editing data files.

Thanks in advance to anyone daring enough to try it out - please send bug reports or feature requests to the address in the docs.

Jack Spade · 2011-08-28 02:11:57

PKGBUILD does require the autotools suite to be used right? If so, I suppose spending some time learning the ins and outs of that would be the first step. If not ... then it's PKGBUILD time.

falconindy · 2011-08-28 02:35:13

PKGBUILD mandates nothing as far as the build system for the package-to-be. There's plenty of examples on the AUR where there's just a plain Makefile (or no Makefile at all).

karol · 2011-08-28 02:40:50

A very simple PKGBUILD - the package is just a shell script https://aur.archlinux.org/packages/do/d … e/PKGBUILD
Some C code hosted on github https://aur.archlinux.org/packages/sx/sxiv-git/PKGBUILD

Jack Spade · 2011-08-28 03:39:05

Wow! I finally started reading the PKGBUILD wiki and checked out those examples. That's a lot easier than I was expecting (I've been using that phrase a lot since I started using arch).

I should be able to put one of those together pretty quickly.

Jack Spade · 2011-08-28 06:06:28

Thanks again everyone - particularly Karol.

This *seemed to* go pretty smoothly. I made the PKGBUILD and put it in the AUR.

The package name is mange. If anyone is up for humoring me and seeing if I did anything horribly wrong I'd appreciate it.

For anyone else facing the same question(s), this was far easier than I thought. It seems the source code can be hosted anywhere, and all that's really needed is a short text file (PKGBUILD) that basically just says where the source code is.

karol · 2011-08-28 13:16:01

https://aur.archlinux.org/packages.php?ID=51920 builds and installs fine, but the 'files' metadata is an empty file.

[karol@black ~]$ cat /var/lib/pacman/local/mange-0.1b-1/files 
[karol@black ~]$ pacman -Ql mange
[karol@black ~]$

falconindy · 2011-08-28 13:29:34

Not cool:

install: mange
  sudo cp ./mange /usr/bin/mange
  sudo cp ./mange.1.gz /usr/share/man/man1/mange.1.gz

Your Makefile needs to:

a) Not call sudo
b) Not try to install directly to the filesystem (invalidating the point of a package)
c) Honor DESTDIR

Jack Spade · 2011-08-28 15:00:35

I was worried about that too falconindy.

When I first uploaded it it read

install: mange
cp ./mange $(DESTDIR)/usr/bin/mange
cp ./mange.1.gz /usr/share/man/man1/mange.1.gz

But apparently the DESTDIR passed to the makefile from the PKGBUILD didn't exist so installation failed with an error on the cp command. How should a makefile install?

[ "SOLVED" removed. Let me know if this should instead be a new topic. ]

falconindy · 2011-08-28 15:13:22

It should definitely work... it's how I write all my PKGBUILDs and Makefiles. Example:

https://github.com/falconindy/cower/blo … r/Makefile
http://aur.archlinux.org/packages/co/cower/PKGBUILD

Jack Spade · 2011-08-28 16:03:44

Thanks falconindy,

I used those as templates and it seems to work now.

I tried it using an AUR helper (sorry not cower) and worked 'right' rather than just worked.

pacman -Ql now gives a sensible response.

falconindy · 2011-08-28 16:12:20

Jack Spade wrote:

Thanks falconindy,
I used those as templates and it seems to work now.
I tried it using an AUR helper (sorry not cower) and worked 'right' rather than just worked.
pacman -Ql now gives a sensible response.

Wasn't intended to be a sales pitch Glad it was helpful.

karol · 2011-08-28 16:17:04

Jack Spade wrote:

pacman -Ql now gives a sensible response.

Just to make it clear, w/o a proper 'files' file, you get conflicts like this:

==> Installing package mange with pacman -U...
resolving dependencies...
looking for inter-conflicts...

Targets (1): mange-0.1b-2 [0,04 MB]

Total Download Size:    0,00 MB
Total Installed Size:   0,05 MB

Proceed with installation? [Y/n] 
(1/1) checking package integrity                           [################################] 100%
(1/1) checking for file conflicts                          [################################] 100%
error: failed to commit transaction (conflicting files)
mange: /usr/bin/mange exists in filesystem
mange: /usr/share/man/man1/mange.1.gz exists in filesystem
==> WARNING: Failed to install built package(s).

and a non-working 'pacman -Ql foo' is just a "side-effect" :-)

Mange installed fine with '--force', thanks.

Jack Spade · 2011-08-28 16:34:47

karol wrote:

Just to make it clear, w/o a proper 'files' file, you get conflicts like this: ...

Hmm ... this may sound foolish - but I've looked around and checked some other AUR packages and I'm not sure what/where a proper 'files' file would be. I'm assuming it would just list the files (all two of them) that the package installs(?).

Any references for this? I am interested in learning how to polish things up to be done the proper way.

falconindy · 2011-08-28 16:57:31

Every locally installed package has 2 metadata files in the local pacman DB: desc, files. If you want to check the sanity of your package before installing it, it's much easier to look at the .PKGINFO file created in the pkg/ directory and the contents of the package tarball itself (`bsdtar tf' is helpful here if you're not using lesspipe). You shouldn't, as a user, ever need to worry about what's in pacman's database, but you're free to nose around in it: /var/lib/pacman/local/

Last edited by falconindy (2011-08-28 16:59:50)

Jack Spade · 2011-08-28 19:24:16

Ah, the files file is there.

Its previous absence was probably from the (innapropriate :S) shortcut I took the first time around.

Arch Linux

#1 2011-08-27 20:41:45

Wrote the code... now what

#2 2011-08-27 20:53:19

Re: Wrote the code... now what

#3 2011-08-27 20:56:13

Re: Wrote the code... now what

#4 2011-08-27 21:02:02

Re: Wrote the code... now what

#5 2011-08-27 21:08:05

Re: Wrote the code... now what

#6 2011-08-27 21:08:47

Re: Wrote the code... now what

#7 2011-08-27 21:20:30

Re: Wrote the code... now what

#8 2011-08-27 21:28:48

Re: Wrote the code... now what

#9 2011-08-27 22:15:57

Re: Wrote the code... now what

#10 2011-08-28 02:10:34

Re: Wrote the code... now what

#11 2011-08-28 02:11:57

Re: Wrote the code... now what

#12 2011-08-28 02:35:13

Re: Wrote the code... now what

#13 2011-08-28 02:40:50

Re: Wrote the code... now what

#14 2011-08-28 03:39:05

Re: Wrote the code... now what

#15 2011-08-28 06:06:28

Re: Wrote the code... now what

#16 2011-08-28 13:16:01

Re: Wrote the code... now what

#17 2011-08-28 13:29:34

Re: Wrote the code... now what

#18 2011-08-28 15:00:35

Re: Wrote the code... now what

#19 2011-08-28 15:13:22

Re: Wrote the code... now what

#20 2011-08-28 16:03:44

Re: Wrote the code... now what

#21 2011-08-28 16:12:20

Re: Wrote the code... now what

#22 2011-08-28 16:17:04

Re: Wrote the code... now what

#23 2011-08-28 16:34:47

Re: Wrote the code... now what

#24 2011-08-28 16:57:31

Re: Wrote the code... now what

#25 2011-08-28 19:24:16

Re: Wrote the code... now what

Board footer