Skip to content
This repository was archived by the owner on Mar 11, 2021. It is now read-only.

Calais error handling #287

Open
cometman wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Calais error handling #287

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c820223
adding open calais error handling
cometman Oct 16, 2015
e62f98e
Adding reference to gem
cometman Oct 16, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Gemfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ source 'https://rubygems.org'
gem 'iconv'
gem 'rails', '~>4.2.0'
gem 'curb', '~>0.8.4'
gem 'open_calais', :github => "documentcloud/open_calais", :branch=> "fuzziness" # Supports Open Calais v2
gem 'open_calais', :github => "documentcloud/open_calais", :branch=> "error_handling" # Supports Open Calais v2
gem 'rest-client', '~> 1.8.0'
gem 'bcrypt', '~> 3.1.1', :require => 'bcrypt'
gem 'rubyzip', '~> 1.1.0' #0.9.9'
Expand Down
13 changes: 9 additions & 4 deletions lib/dc/import/calais_fetcher.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,21 +62,26 @@ def retry_calais_errors
attempts = 0
begin
yield
rescue Faraday::Error, Timeout::Error => e
rescue CalaisError::RateLimitExceeded => e
attempts += 1
sleep 10
Rails.logger.warn 'waiting 10 seconds. Calais rate limit exceeded'
retry if attempts < 5
rescue CalaisError::ServiceUnavailable, CalaisError::ServerError => e
Rails.logger.warn e.message
return nil if e.message == 'Calais continues to expand its list of supported languages, but does not yet support your submitted content.'
Rails.logger.warn 'waiting 10 seconds'
Rails.logger.warn 'Calais server had an error. Retrying...waiting 10 seconds'
attempts += 1
sleep 10
retry if attempts < 5
rescue RuntimeError => e
rescue RuntimeError => e # Do we still need this catch?
Copy link
Contributor Author

@cometman cometman Oct 16, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@knowtheory do we need this RuntimeError catch still? Was this exception used to catch errors if Calais received a request that was too large? If so, we have an exception that can handle this.

Copy link
Member

@knowtheory knowtheory Oct 19, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cometman So long as the branch there is covered and that the content too large error is caught through other means we can remove it def.

return nil if e.message == 'content is too large'
puts e.message
puts e.class
raise e
rescue Exception => e
puts e.message
puts e.class
LifecycleMailer.exception_notification(e).deliver_now
raise e
end
end
Expand Down