Help with getting specific web pages to my pc

Lone_Wolf · 2006-02-04 17:20:56

I have started to write a program to get stats from a game to my pc for processing them.

this page : http://www.lordsoflegend.com/user_rank.php is giving me trouble.
Atm the only way to get the 'All Races' and 9 race-specific pages is to select an option with the mouse, then click 'rank by race' and save the page.
This needs to be repeated 10 times to get all pages.
The url of the page doesn't change when you do this.
Below i have copied the part of the source-code that i think deals with those actions.

Can you help with automating this ?

        <form name="form1" method="post" action="">
         <select name="rank_race" class="textfieldcss"> <option value='0'  > All Races </option>";
 <option value='1'>Humans</option><option value='2'>Elves</option><option value='3'>Dwarves</option><option value='4'>Orcs</option><option value='5'>Trolls</option><option value='6'>Undead</option><option value='7'>Dark Elves</option><option value='8'>Draconians</option><option value='9'>Fangolians</option>              </select>

              <input name="submit" type="submit" class="burronsection"  value="Rank by race">
         </form>

Dusty · 2006-02-04 17:28:25

ITs a POST request, as opposed to GET (get is easy to encode in the URL). If you're using Python, you'll need to look up the functions and format of an HTTP post request and set each of the variables that are set in the form to what you like. I'm not sure how this is done, I always use the urllib library for web processing, but I'm not sure if they have POST processing, or you'll have to go one level down.... the Python reference will be your friend here.

Dusty

Lone_Wolf · 2006-02-04 23:24:12

Thanks, it was easier to find it than i thought.

params = urllib.urlencode({'rank_race': 9})
f = urllib.urlopen("http://www.lordsoflegend.com/user_rank.php", params)

I'll try to do more searching myself before asking my next question.

BTW, i think Python is my kind of language.

Arch Linux

#1 2006-02-04 17:20:56

Help with getting specific web pages to my pc

#2 2006-02-04 17:28:25

Re: Help with getting specific web pages to my pc

#3 2006-02-04 23:24:12

Re: Help with getting specific web pages to my pc

Board footer