How do I go about something like this?

wirenik · 2008-09-26 16:00:36

Hey, I'm a newbie programmer with a project in mind. I buy all my music from Amazonmp3.com, but since the shopping's done in a browser, it gets a little annoying to wait for the whole page to load just to can sample some music. I'd like to make a simple, light app to search Amazon's mp3 downloads and play samples, possibly with an option to buy the music via a browser afterward.

So my questions are, is this even possible? What language should I use (I'm thinking Perl)? And, most importantly, how do you go about something like this? Do I need to find a way to get to Amazon's database directly, or maybe query it like a regular user and iterate through the results, or something?

Responses are really appreciated.

daf666 · 2008-09-26 18:18:27

I would think something in the lines of a Firefox add-on probably be the quickest...

Xyne · 2008-09-26 18:37:46

Hi wirenik

I'd use Perl for this, but I'm sure you'll get a few other suggestions. I think the choice comes down to what you're most comfortable using as several languages will probably be able to accomplish this relatively easily.

If you can access the database or some minimal info page then do that as it will save some bandwidth and time, but I suspect that you'll be downloading the normal user page through your script and parsing it (which is easy and fun with Perl's regexes, plus it will still be fast as the page isn't parsed and rendered as it is in the browser). I'd start by looking at the source code of the normal page (assuming that there is no simplified database interface) to find unique tags that identify the information that you're after. Save that source code to a file and start by writing a script that can open the page and display the information that you want (e.g. printing the song name and url). Once you have that working, add the download functionality.

To actually download the pages, take a look at the LWP modules:
http://search.cpan.org/~gaas/libwww-per … lib/LWP.pm
http://search.cpan.org/~gaas/libwww-per … /Simple.pm
http://search.cpan.org/~gaas/libwww-per … erAgent.pm
http://search.cpan.org/~gaas/libwww-per … Cookies.pm
http://search.cpan.org/~jjore/LWP-Simpl … Cookies.pm

They're included in the package "perl-libwww" in the "extra" repo.

Alternatively, you could use "curl" to load the page directly into a variable in your script with something like this:

$page = `curl http://www.example.com`;

After you have the website interactivity working the way that you want it, you'll (*just*) need to figure out how you want to download and play the samples. Some ideas:
Collect all the mp3 download links then pass them to a download accelerator like aria2 (found in extra, very good and one that I highly recommend). You can pipe a list of urls to aria2 directly from your perl script (just ask me if you need some help with the syntax). If you do that, create an html page for all the files with the relevant links for buying the music later if you want.

Alternatively, just create a single html page with a table containing all of the mp3s linked to their urls along with links for buying them later. To download all of the mp3s, use a build-in browser download accelerator (I'd recommend downthemall for firefox).

I hope this helps, good luck.

EDIT
ornitorrincos' post reminded me...
Check out the mechanize module too: http://search.cpan.org/~petdance/WWW-Me … chanize.pm

Last edited by Xyne (2008-09-26 20:55:21)

ornitorrincos · 2008-09-26 20:40:26

python with mechanize could do the work if you want to to in something that is not perl(although you preffer perl)

Arch Linux

#1 2008-09-26 16:00:36

How do I go about something like this?

#2 2008-09-26 18:18:27

Re: How do I go about something like this?

#3 2008-09-26 18:37:46

Re: How do I go about something like this?

#4 2008-09-26 20:40:26

Re: How do I go about something like this?

Board footer