I woke up this morning thinking about User Agents (ok – I was also thinking about Naomi Watts and Sean Penn who were amazing in 21 Grams.) A commenter on my Personalize Feed post pointed out that most of the big online aggregators include subscriber counts in their user-agent headers when the aggregator polls the RSS feed.
While I agree that using the User Agent to report the number of subscribers to a feed is a good approach, I’ve noticed a bizarre pattern lately. Since I use FeedBurner to manage my feed, I’ve got great visibility into which aggregators are providing their subscriber numbers. I’ve got enough subscribers at this point (> 4500) that the law of large numbers are working for me – while I won’t claim to have perfect “aggregator market share data”, I’ve got a pretty good feel for it (which is really useful given my investment in NewsGator.) As I wrote in my Blog Analytics post, I love numbers and have long invested in, benefited from, and paid attention to web analytics. So – the number of subscribers to my feed (among other feed and blog related data) are near and dear to my heart.
However, it turns out that a number of aggregators don’t report subscriber count. I’ve got 128 distinct aggregators polling my feed on a daily basis according to FeedBurner. 76 of them have one subscriber which most likely means they don’t report the number of subscribers via User Agent (yeah – some of them probably only have one subscriber, but not all of them.) The top 10 aggregators polling my feed (in order) are Bloglines, NewsGator Online, FeedDemon, NetNewsWire, Firefox Live Bookmarks, Rojo, Google Desktop, SharpReader, NewsGator Enterprise, and Thunderbird.
Ok – that’s sort of interesting – but more interesting is the number of aggregators that don’t report number of subscribers. Google Reader doesn’t. Microsoft is noticeably missing (and has a User Agent is lodged down in the subscriber = 1 category). And – the grand daddy of them all (My Yahoo) – after reporting a number that landed them regularly in my top four, recently stopped reporting number of subscribers and now shows one. Others in the subscriber = 1 list that stand out include OddPost (Yahoo again), Pluck Firefox and Web Edition (ironically the Pluck Internet Explorer Edition reports, just not in the top 10, although that might be because it’s not an online reader), Pubsub, and SearchFox.
Since a web-based aggregator only needs to request the feed once and then uses that cached version for all its users, putting the subscriber count in the User Agent is a good citizen move to simply help the publisher of the feed (e.g. me – I want to know the number of subscribers I have.) I’ve got to believe that it’s useful to the aggregator to report the number of subscribers since it helps the publisher understand how popular the aggregator is and publishers will try to promote the services where they have a lot of subscribers. Plus – as a publisher – while I don’t know the demographics of my subscribers, at least I’d like to know how many I have. Given how trivial it is for the aggregator to report this, it baffles me why some of the aggregators, including Google, Microsoft, and Yahoo, don’t do this.
Of course, there are issues with the simply reporting the aggregator count. While I get an aggregate subscriber count, I don’t get any real information about the number of subscribers that are actively reading my blog via an aggregator. Also, people don’t tend to delete their user-ids at online aggregators if they don’t use the service anymore (e.g. I know I have a Kinja account – remember Kinja – they don’t report subscribers either – with feeds in it that I imagine happily pools stuff daily for me). Finally, some services subscribe a bunch of feeds automatically skewing the number of subscribers for that aggregator.
That said, the User Agent approach is a good one that I figured was worth decomposing more based on the comment I got suggesting most of the big online aggregators support subscription counts. While a lot of aggregators are playing nice (at least 33% of the ones that poll my feed), a bunch aren’t, including Microsoft, Google, and Yahoo. What’s up with that guys?