So, today I’ve been working on my new weblog system. The first thing I thought I’d do is write the RSS Aggregator, because I thought MovableType was doing a decent job of being my current weblog system. This was a bit silly really because “SharpReader”:http://www.sharpreader.net is doing a decent job at being my RSS reader, but hey I’ve started it now.
This page here shows it in action, but all it is doing is displaying the data in an order, with none of the functionality I want in the future. I just want to talk about the technology behind this one simple page, mainly for my own benefit, but perhaps you want to know too.
At the heart of things is “The Ultra Liberal RSS Parser”:http://www.diveintomark.org/projects/rss_parser/ by Mark Pilgrim. Then, in a directory called “thing”[1] is a smallish python script that uses rssparser.py to grab items and stuff them into a MySQL database (don’t start Aq). I store stuff like etag and last-modified so rssparser.py behaves itself, as well as description, time etc. In fact, the only trouble I had here was with the dates. I’m thankful to “another Mark”:http://markpasc.org/weblog/2003/06/01_prrarf_01b_rss_dates.html for being two steps ahead of me here. So, with this little program, you can add feeds, subscribe to feeds, unsubscribe to feeds and update the feed items.
Long term, I’m planning to output everything from the database as XML and then use some XSLT to pump it out to the relevant format, but I didn’t have time for that and used “PyMeld”:http://www.entrian.com/PyMeld/index.html instead.
This is really neat and very easy to use, and I might just use this for everything in the end. Only one problem; at the start the docs give an example of how to use it and then say how slow that method is. Then, they give a different method, which I tested and found to be really quick. Trouble was, that this didn’t take into account nested objects, and I couldn’t work out how to do what I wanted with that method. I went back to the old method, and whilst it outputted as I wanted, it was rather slow (and the Linux box is this running on is no slow-coach).
Next step: a bunch of other things!
I’m celebrating by listening to British Sea Power as “mentioned by Vaughan”:http://www.whereveryouare.org.uk/weblog/archives/week_2003_06_01.html#003122 to whom I’m now indebted. I saw BSP at Reading Festival 2002 and thought they were blooming marvellous. I was very disappointed to find I couldn’t find their only EP anywhere, and hadn’t noticed that they had an album out.
[1] Working title for the weblog system 