Will RSS Readers Clog the Web?

28885 Resource 11678 24 21 1083386363799 1213209016066 Controlled Big, Hungry, Orange RSS Alligators

Imagine a swamp that stretches to the horizon...

<div style="BORDER-RIGHT: rgb(204,204,204) 1px solid; PADDING-RIGHT: 6px; BORDER-TOP: rgb(204,204,204) 1px solid; PADDING-LEFT: 6px; FLOAT: right; MARGIN-BOTTOM: 8px; PADDING-BOTTOM: 6px; MARGIN-LEFT: 8px; BORDER-LEFT: rgb(204,204,204) 1px solid; WIDTH: 181px; PADDING-TOP: 6px; BORDER-BOTTOM: rgb(204,204,204) 1px solid"><img height="85" alt="Big, Hungry, Orange Alligators" hspace="0" src="http://blogsite.com/styles/blogsite/Blogsite/content/orange-aligator.jpg" width="169" border="0" /> <div style="FONT-WEIGHT: bold; FONT-SIZE: 9pt; TEXT-ALIGN: center">Overly Aggressive Feed Readers</div></div>The swamp is teaming with orange alligators as far as the eye can see, and they're all hungry. And in the middle of the swamp, perched on a small rock outcropping with open crates labeled 'Bandwidth Sandwiches', we pitch them in as fast as possible; our only hope is that the alligators will eat us last.This is an ugly scenario. What happens when everyone discovers the power of alligators; er, I mean aggregators? I agree with the premise of <a href="http://www.wired.com/news/infostructure/0,1377,63264,00.html">this Wired Magazine article </a>– an Internet society that polls for everything will probably not work well.Seriously – this is a problem. We serve about 5,000 RSS feeds off a single <a href="http://intel.com/">Intel CPU</a>, but not without some tense moments. Imagine 100,000 readers each pinging a single feed at 60 minute intervals (many at the top of the hour). That’s 2,400,000 HTTP requests per day. Now add 20 feeds.<a href="http://www.cafeshops.com/rsstore.9000417"><img class="imageborder" style="WIDTH: 150px; HEIGHT: 150px" height="150" alt="Bib" hspace="6" src="http://storetn.cafepress.com/7/9000417_F_store.jpg" align="left" border="0" /></a>MyST content can be delivered via FTP and Web services to file systems and into <a href="http://www.corante.com/importance/archives/002223.html">BitTorrent </a>(theoretically). But the future requires a web services approach that elevates intelligent process between the client and server. Additionally, the RSS server must reward readers for good behavior and punish them when they waste bandwidth. We have many RSS delivery mechanisms that we’re considering for the future and we already have algorithms that punish certain readers that are blatantly wasteful. In the next quarter we hope to publish a paper on this subject. The general polling architecture behind RSS readers (while popular today) will give way to smarter approaches soon. "Once all the covered wagons show up and an RSS reader is folded into Microsoft's Internet Explorer or Outlook, we're doomed," -- <a href="http://www.teledyn.com/mt">Gary Lawrence Murphy</a>Actually we’re doomed if reader tools and technologies don’t get smarter and listen to servers with as much as they ask questions.The issue of bandwidth consumption will likely give rise to a new type of server; an RSS server. The <a>MyST platform</a> is designed to handle transactions like this and many others types of processes that embrace loose information couplings in intelligent ways."The trouble is, aggregators are greedy. They constantly check websites that use RSS, always searching for new content." -- By <a href="http://www.wired.com/news/feedback/mail/1,2330,0-742-63264,00.html">Ryan Singel</a>And this is exactly why RSS is so useful. People need machines to help them wade through information and determine the delta (i.e., what's different from yesterday). Machines work best with machine code and that's what RSS is - a semantically useful format that an RSS reader (i.e., your agent) can use to help you deal with the <a href="http://sims.berkeley.edu/research/projects/how-much-info/index.html">information tsunami </a>that is still miles off shore.The nice thing about the relationship that an RSS server has with its reader agents – it can augment the feed to inform users that they will not be serviced [with content updates] until they upgrade their reader client to something that doesn’t suck the life out of the information architecture. In fact, I wouldn't be surprised if certain highly demanded feeds will be served rapidly to registered customers and a bit slower to non-paying content consumers. Imagine paying a premium for high-velocity performance of feeds important to your business. Or paying a surcharge to dine on RSS content at a peak demand period. A whole new world of RSS services will be required to perform the myriad of RSS acrobatics that will be required.The swamp is teaming with orange beasts and the growth rate (in the near term) is likely to benefit those making bandwidth hoagies. In the meantime, be careful what you syndicate; it may require an extra server or two. ;-) 9 application/xml false aggregator alligator alligators bandwidth hungry orange Will RSS Readers Clog the Web? News aggregators may be the best new tools to appear on the Web since the browser, but as the programs and the underlying RSS standard grow more popular, some question whether the Internet will be able to handle the traffic. http://www.wired.com/news/infostructure/0,1377,63264,00.html Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727) 127.0.0.1 127.0.0.1 aseidl 68.40.167.222 false true false false true 190014 false false true false false false true 11678 blog blogsite/MySTHome/web