Linux equivalent of Website-Watcher?

Kirby · 2005-09-12 23:43:54

I had it up to here with Winblows...almost all software I use now is open-source, and there is really no reason to keep it anymore (except for games, but even Cedega will make that useless)

I have found Linux equivalent for all my programs, except for Website-Watcher....for all who don't know, website-watcher checks website text for updates (very useful if the website doesn't have RSS feeds).

For a second, I thought RSSOwl could do something like that, but I guess not :cry:

So I'm asking if there are Linux alternatives out there that are like Website-Watcher

beniro · 2005-09-13 03:03:16

I don't know of a program that does it, but it would be a trivial bash script to write...ahem...not...ummm...for ME...I mean...trivial for someone who knows what they're doing.

paranoos · 2005-09-13 06:33:17

well, links has the ability to save a txt file of a website's content, so you could use that, with diff to see if the page has changed.

if you don't want to use links, you could simply use sed to strip out html tags and just keep the text ... sed "s,<[^>]*>,,g" ... or something like that. the sed version might work better, as links may put the alt text of images in the text file, which would change simply from a dynamic ad.

Dusty · 2005-09-13 14:30:44

Wouldn't it be easier to check the 'last modified' date returned in the http header? Or is that not always updated correctly on the server side?

Dusty

FJ · 2005-09-19 07:08:24

Does this work for you?

#!/bin/sh
#urlcompare
#Deps: snarf

#Enter URL at the command line without http://
URL=$1
Location=$HOME/$URL

if ! [ -d $Location ];
then mkdir -p $Location
fi

if [ -s $Location/LocalCopy ];
then
        #Get the new page, find out if it's different,
        #then make the new page the base to compare
        #to next time.

        snarf http://$URL $Location/NewCopy && 
        diff -qs $Location/LocalCopy $Location/NewCopy
        mv -f $Location/NewCopy $Location/LocalCopy
else
        #Create a copy of the page.
        snarf http://$URL $Location/LocalCopy
        echo "No previous copy of the page found, created new one."
fi

You need snarf (but wget can probably do the same thing). Chmod it 755 then simply go

./urlcompare www.foo.bar

It'll create a new copy of the page. Then when you need to check it for updates later run the same command. Haven't tested it too much but it seems to work in that it will also check date stamps on the pages.

Arch Linux

#1 2005-09-12 23:43:54

Linux equivalent of Website-Watcher?

#2 2005-09-13 03:03:16

Re: Linux equivalent of Website-Watcher?

#3 2005-09-13 06:33:17

Re: Linux equivalent of Website-Watcher?

#4 2005-09-13 14:30:44

Re: Linux equivalent of Website-Watcher?

#5 2005-09-19 07:08:24

Re: Linux equivalent of Website-Watcher?

Board footer