Need some ideas for a large-ish Python project

Peasantoid · 2010-02-20 05:01:35

Title says all -- I've got coder's block. I desperately want to write something cool in Python, but I don't know what. I did a partial reimplementation of less, which was an interesting challenge (especially working with ncurses), and now I'm stuck. I thought I'd implement this, but then I realized it wouldn't be very fun.

So. Any ideas? I'll accept anything at this point, even half-baked silliness.

Michael C. · 2010-02-20 05:13:52

I just wrote a portknocking client and server, found it to be a very fun project... though I did it in C, might be too easy in Python.

Peasantoid · 2010-02-20 05:19:36

Michael C. wrote:

I just wrote a portknocking client and server, found it to be a very fun project... though I did it in C, might be too easy in Python.

Sounds interesting, though I wouldn't know where to begin nor how to test it (don't really use a firewall since my router does NAT, though I wonder if I should anyway). I suppose I could do C if Python presented an insufficient challenge.

Although what I'm really looking for is something the user can interact with, like a terminal application or such.

Last edited by Peasantoid (2010-02-20 05:27:47)

tomd123 · 2010-02-20 05:33:18

Peasantoid wrote:

Michael C. wrote:
I just wrote a portknocking client and server, found it to be a very fun project... though I did it in C, might be too easy in Python.
Sounds interesting, though I wouldn't know where to begin nor how to test it (don't really use a firewall since my router does NAT, though I wonder if I should anyway). I suppose I could do C if Python presented an insufficient challenge.
Although what I'm really looking for is something the user can interact with, like a terminal application or such.

Create a fast full text file search.
Basically, find a way to have a fast grep -iR sometext /

Peasantoid · 2010-02-20 05:35:33

tomd123 wrote:

Create a fast full text file search.
Basically, find a way to have a fast grep -iR sometext /

This sounds fun. Will regex be a requirement?
The 'fast' constraint implies C. *cracks knuckles, fires up vim*

n0dix · 2010-02-20 05:35:53

tomd123 wrote:

Peasantoid wrote:
Michael C. wrote:
I just wrote a portknocking client and server, found it to be a very fun project... though I did it in C, might be too easy in Python.
Sounds interesting, though I wouldn't know where to begin nor how to test it (don't really use a firewall since my router does NAT, though I wonder if I should anyway). I suppose I could do C if Python presented an insufficient challenge.
Although what I'm really looking for is something the user can interact with, like a terminal application or such.
Create a fast full text file search.
Basically, find a way to have a fast grep -iR sometext /

if the text is big you can implement threads,

Peasantoid · 2010-02-20 05:38:15

n0dix wrote:

if the text is big you can implement threads,

Uh, why don't I put that on the back burner for now. *coughs, edges away*

Peasantoid · 2010-02-20 08:03:29

Okay, I think the reason I'm having problems writing code is that I don't have any itches to scratch, so I'm not motivated to fix them. Thanks for all your ideas, though.

pointone · 2010-02-20 14:38:19

Why not write a program that generates project ideas? Problem solved!

n0dix · 2010-02-20 15:13:23

pointone wrote:

Why not write a program that generates project ideas? Problem solved!

the world would be easy if the programs can do something like that.

Eratosthenes · 2010-02-20 17:49:44

I fell into a python project that makes heavy use of regular expressions. A good gui reg ex testing tool would be very useful, especially if it supported cool things like named match groups, re.MULTILINE, and re.VERBOSE. I used to use the one in EPIC (the Eclipse perl plug in), but it is too limited.

http://weitz.de/regex-coach/tutorial1.html

is the general idea, but it is windows only. I have seen several others kicking around the web, but none that did a complete job with the python regex engine. (But I would certainly be happy to hear suggestions.)

Oh yeah...I know real programmers don't need fancy gui tools and I got the job done without one. This would make writing and degugging complex regexes a lot more fun.

MrAllan · 2010-02-20 20:36:20

I recently created a program that calculates the sunrise and sunset times for a given day at a given location. Even though this sounds trivial, it is surprisingly difficult - although it may not be a "large-ish" project.

Mr.Elendig · 2010-02-20 22:43:49

Create a _good_ new irc bot using twisted, as a nice replacement for phrik

Peasantoid · 2010-02-20 23:20:20

Mr.Elendig wrote:

Create a _good_ new irc bot using twisted, as a nice replacement for phrik

Okay, now *that* sounds exciting. Will need to research IRC protocol of course.

jorpheus · 2010-02-20 23:35:22

Peasantoid wrote:

Mr.Elendig wrote:
Create a _good_ new irc bot using twisted, as a nice replacement for phrik
Okay, now *that* sounds exciting. Will need to research IRC protocol of course.

Research Twisted.words, it'll save you time reinventing the wheel.

Peasantoid · 2010-02-20 23:59:21

jorpheus: Thank you greatly for that advice. Means I can concentrate on writing real functionality rather than dicking around with the protocol.

Peasantoid · 2010-02-21 06:43:08

What features do y'all find desirable in an IRC bot? I need some input.

linkmaster03 · 2010-02-21 14:31:13

An IRC bot will be fun. I made one a couple months back. Of course, I used sockets instead of twisted, but I suspect it will still be a good time. Make sure you post it up if you go through with it!

Last edited by linkmaster03 (2010-02-21 14:31:33)

Cdh · 2010-02-21 14:37:08

I would like some console music player with some more features especially for internet radio etc. or maybe just a frontend for existing players.
I think it should not be too hard when using shoutcast-search from aur to search and parse shoutcast stations and using mplayer as output. Then the jamendo database (which is an xml file) and I would be really happy.

Peasantoid · 2010-02-21 14:56:52

Okay, plans for IRC bot are as follows:
- name is 'pyrik' or some other variation on 'phrik'
- Lisp-alike for the command language
- some back-end for data persistence, perhaps via pickle

Still need some input though.

Cdh: Sounds like it would be pretty simple to hack one up in Python. You should try it.

Last edited by Peasantoid (2010-02-21 14:58:16)

karol · 2010-04-02 20:26:17

From http://peasantoid.org/ublog.cgi
31.03.2010 16:12:06 *yawn* Bored.

How 'bout a python3 / sqlite3 pacman backend?
/me ducks

I can supply a wishlist and test the thing, but don't count on patches.

tomd123 · 2010-04-02 20:56:46

Peasantoid wrote:

tomd123 wrote:
Create a fast full text file search.
Basically, find a way to have a fast grep -iR sometext /
This sounds fun. Will regex be a requirement?
The 'fast' constraint implies C. *cracks knuckles, fires up vim*

Not necessarily. I'm talking about implementing some fast algorithms in any language. AFAIK grep -iR searches all the file's contents in real time, which makes it easy to code, but it has horrible time and disk activity

karol · 2010-04-02 21:05:06

@tomd123

find "./" \! -name "*~" -type f -exec grep -isI "$1" {} \; -print

That's what I use, but I have only a couple MB of files. How much do you have?

gohu · 2010-04-02 21:16:52

tomd123 wrote:

Peasantoid wrote:
tomd123 wrote:
Create a fast full text file search.
Basically, find a way to have a fast grep -iR sometext /
This sounds fun. Will regex be a requirement?
The 'fast' constraint implies C. *cracks knuckles, fires up vim*
Not necessarily. I'm talking about implementing some fast algorithms in any language. AFAIK grep -iR searches all the file's contents in real time, which makes it easy to code, but it has horrible time and disk activity

Well I haven't read grep's sourcecode but I think it is well optimized, it's been around for quite some time. For an 'interactive' tool, that is.
A compiled lexical analyzer will very probably be faster.
So if you want a fast grep for large corpuses of text you should automate the generation of a lexical analyzer and its launch.
You may want to look into lex.

About the disk activity, I believe grep is well coded and doesn't read in 1 byte chunks or something, so I don't think you could ameliorate on that.
Or maybe were you thinking of a system based on indexation ?

Oh and BTW for that purpose I use ack ( http://betterthangrep.com/ )
It's really excellent. And I just benchmarked it agains grep, it's 5 times faster. (test procedure: find penguin in the kernel sources, results: 25 seconds for grep, 5 for ack. Yay !)

Last edited by gohu (2010-04-02 21:20:55)

Peasantoid · 2010-04-02 22:47:34

karol wrote:

From http://peasantoid.org/ublog.cgi
31.03.2010 16:12:06 *yawn* Bored.
How 'bout a python3 / sqlite3 pacman backend?
/me ducks
I can supply a wishlist and test the thing, but don't count on patches.

I'd need to track down one of several libalpm modules for Python (or write my own). Shouldn't be too hard, but at the moment I'm more focused on developing FCM.

As for the grep slowness thing, I imagine a solution would be to fork off a pipe-connected subprocess for each file (or directory if there are many files) in the background and then select() on the pipes, so the whole thing runs in parallel and it only takes as long as the largest file.

Last edited by Peasantoid (2010-04-02 22:51:36)

Arch Linux

#1 2010-02-20 05:01:35

Need some ideas for a large-ish Python project

#2 2010-02-20 05:13:52

Re: Need some ideas for a large-ish Python project

#3 2010-02-20 05:19:36

Re: Need some ideas for a large-ish Python project

#4 2010-02-20 05:33:18

Re: Need some ideas for a large-ish Python project

#5 2010-02-20 05:35:33

Re: Need some ideas for a large-ish Python project

#6 2010-02-20 05:35:53

Re: Need some ideas for a large-ish Python project

#7 2010-02-20 05:38:15

Re: Need some ideas for a large-ish Python project

#8 2010-02-20 08:03:29

Re: Need some ideas for a large-ish Python project

#9 2010-02-20 14:38:19

Re: Need some ideas for a large-ish Python project

#10 2010-02-20 15:13:23

Re: Need some ideas for a large-ish Python project

#11 2010-02-20 17:49:44

Re: Need some ideas for a large-ish Python project

#12 2010-02-20 20:36:20

Re: Need some ideas for a large-ish Python project

#13 2010-02-20 22:43:49

Re: Need some ideas for a large-ish Python project

#14 2010-02-20 23:20:20

Re: Need some ideas for a large-ish Python project

#15 2010-02-20 23:35:22

Re: Need some ideas for a large-ish Python project

#16 2010-02-20 23:59:21

Re: Need some ideas for a large-ish Python project

#17 2010-02-21 06:43:08

Re: Need some ideas for a large-ish Python project

#18 2010-02-21 14:31:13

Re: Need some ideas for a large-ish Python project

#19 2010-02-21 14:37:08

Re: Need some ideas for a large-ish Python project

#20 2010-02-21 14:56:52

Re: Need some ideas for a large-ish Python project

#21 2010-04-02 20:26:17

Re: Need some ideas for a large-ish Python project

#22 2010-04-02 20:56:46

Re: Need some ideas for a large-ish Python project

#23 2010-04-02 21:05:06

Re: Need some ideas for a large-ish Python project

#24 2010-04-02 21:16:52

Re: Need some ideas for a large-ish Python project

#25 2010-04-02 22:47:34

Re: Need some ideas for a large-ish Python project

Board footer