Skip to content

Use If-Modified-Since Header when refreshing feeds - #224

Merged
swanson merged 1 commit into
stringer-rss:masterfrom
mherb:fetch-feed-caching
Jul 16, 2013
Merged

swanson merged 1 commit into
stringer-rss:masterfrom
mherb:fetch-feed-caching

Conversation

@mherb

@mherb mherb commented Jul 13, 2013

Copy link
Copy Markdown
Contributor

Fetch feeds with the If-Modified-Since HTTP header set to the last refresh timestamp.
This should save us (and the feed providers) quite some traffic since we don't have to fetch the whole feed when the content hasn't changed

Fetch feeds with the If-Modified-Since HTTP header set to the last refresh timestamp.
This should save us (and the feed providers) quite some traffic since we don't have the fetch the whole feed when the content hasn't changed
@swanson

swanson commented Jul 14, 2013

Copy link
Copy Markdown
Collaborator

This would be good to add. Let me dig around the Feedzirra source on this before I merge - I don't necessarily trust all the magic of how they detect new stories.

@mherb

mherb commented Jul 14, 2013

Copy link
Copy Markdown
Contributor Author

So here is what I found out while writing this:

We use feed.last_modified from the parsed Feedzirra feed. In case the server responded with a Last-Modified HTTP header, it will return this value. If that header has not been set, then Feedzirra will go through all the stories in the feed and return the latest publication date of all stories. If there is also no publication date, then this will be nil.

So there are two situations we should take care about:

  1. The server responded with the Last-Modified Header. In that case we supply exactly that timestamp as our If-Modified-Since value, so we should only get a (full) response in case it really has been modified
  2. In case the server did not respond with any Last-Modified header then the server will likely also ignore the If-Modified-Since header. Even if the server does not ignore the request header, that shouldn't cause too much trouble since the publication dates should be at least roughly accurate.

swanson pushed a commit that referenced this pull request Jul 16, 2013
Use If-Modified-Since Header when refreshing feeds
@swanson
swanson merged commit 4e00820 into stringer-rss:master Jul 16, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants