You are not logged in.

#1 2014-09-11 07:42:17

mir91
Banned
Registered: 2013-11-17
Posts: 143

Aggregate new postings from different (forum) sources

There are a couple of different forums, that don't have an RSS feed, where I want to aggregate new postings from.
I don't need any fancy database stuff, a simple diff would do and the changes be concatenated as they appear to a local html file that I'd just refresh in my web browser.
However, diff would of course also deliver unwanted info such as changed dates, visitor counters etc.
So before I start reinventing a wheel by creating a bash script that contains a lot of diffing and grepping, I thought I'd ask if there's maybe already such a script available somewhere.

Offline

#2 2014-09-11 09:14:25

willemw
Member
Registered: 2013-02-19
Posts: 116

Re: Aggregate new postings from different (forum) sources

https://www.archlinux.org/packages/comm … /urlwatch/ is one of those tools. With it you can filter out parts of the html pages. Written in Python.

Offline

#3 2014-09-11 13:13:05

mir91
Banned
Registered: 2013-11-17
Posts: 143

Re: Aggregate new postings from different (forum) sources

Thanks

Offline

#4 2014-09-16 08:21:06

willemw
Member
Registered: 2013-02-19
Posts: 116

Re: Aggregate new postings from different (forum) sources

For accessing forum threads, where a thread can be spread across several html pages, you could also have a look at 'flexget' and its 'html' plugin (http://flexget.com/wiki/Plugins/html).

Offline

Board footer

Powered by FluxBB