HTTP Bot Features

  • User-Agent header: "JabRSS (http://jabrss.cmeerw.org)"
  • conditional HTTP GET (Last-Modified and ETag supported)
  • gzip and deflate encoded HTTP requests
  • RSS parser supports RSS 0.90, RSS 0.91, RSS 2.0, RDF 1.0 and Atom 0.3
  • adaptive polling intervals based on the update frequency of the feed (every 30 minutes up to once per day)
  • feeds not supporting conditional HTTP requests will be slightly penalized
  • support for widely-used character data encodings (including UTF-8, UTF-16, ISO8859-1)
  • mostly compliant with RFC 3023
  • proper handling of redirects