What does recently updated mean?

You might be familiar with the Recently updated blogs page, which shows a list of blogs ordered by those that posted most recently. It can be a nice way to see what people are blogging about right now. In fact, in a recent blog post, Warren Ellis said:

the ooh.directory recently updated blogs list is what I have instead of social media if I’m in the mood to “waste” time on my phone

Which sounds like a great idea. However, a while back I realised a small problem with the page. 

(If this is too much to read, jump down to the bold paragraph below for what’s actually changed.)

The blogs were ordered by when their most recent post was published. Which makes sense. Except there’s a delay between when a post is published and when ooh.directory next looks at that blog’s feed and notices the new post.

The frequency with which we check blogs’ feeds varies. A blog that posts often will be checked more frequently. But each time we check a feed and find there’s no new post, we increase the time between checks. There’s no point checking a blog every hour if it only posts once a year.

The biggest gap we leave between checking a blog’s feed is currently 24 hours. Which can mean that if a rarely-updated blog publishes a new post at noon on a Friday, it might only appear on ooh.directory as late as noon on Saturday (although on average it’ll be sooner).

So, imagine ooh.directory only picks up that post at noon on Saturday. And you visit the Recently updated blogs page at 12.01pm that day. That post – only just fetched – was published a whole day ago. Between then and now, many other blogs will have published new posts, so that blog would appear quite a way down the list, probably a few pages away.

Given that blogs that update rarely can be among the most interesting, this seemed a shame.

To stop rarely-updated blogs from getting lost in this list, the page is now ordered by when each blog’s latest post was found by ooh.directory, instead of when it was published.

This does mean that a post appearing on the first page, marked “4 minutes ago” might actually have been published hours ago, or even the previous day. But it only arrived on the site four minutes ago. This feels fairer and more transparent. It’s everything coming in to ooh.directory, as it arrives.

Over time, with more resources (and confidence in the server and code) we might increase the frequency with which we fetch feeds for all blogs, which would reduce the potential gap between a post being published, and a post arriving on the site. But for now we’ll see how this change goes – hopefully it means rarely-updated blogs will be more visible in the stream when you’re “wasting” time on your phone.

If you visit the Recently updated blogs page frequently, and have any thoughts on this change, or how else to improve the page, then do get in touch via Mastodon, Twitter or email.