Showing posts with label API. Show all posts
Showing posts with label API. Show all posts

Thursday, 15 January 2015

Api Development in Ruby On Rails

Recently I wrote a series of blog posts about the best practices of API development in RoR and how to develop API in general.

Let me summarize those posts:


Happy API development :)

API development in Rails error path


In my series of of API development I haven't covered the error handling.

Let me share my experience with you about what I have learned about proper error handling.
I call error path the case when the user can't achieve what he wants.

This case can happen by the following reasons:

  • The user calls an invalid url
  • The user addresses a missing resource
  • The user misses a mandatory parameter
  • The uses passes a wrong parameter
  • Some other internal error occurs
  • ...

Since thousands of calls can be made daily or hourly against the API there is no way to stop the server and debug it. So all the information related to the erroneous call must be recorded. Basically all the information to reproduce the error must be recorded. I call this ApiCallAudit.

Such an ApiCallAudit must contain:

  • All the incoming parameters
  • The type of call (GET, POST, DELETE, ...)
  • cookies
  • backtrace
  • created_at
I added some additional fields to it for filtering purposes:

  • level. It serves to quickly determine the possible source of errors. It could be parameter_error, unexpected_error, parameter_missing_error, entity_missing_error
  • status. It is a default error message.
  • code. An error code which uniquely identifies the error.
In the beginning the error code was missing from my design and the mobile clients were using the default error message. This has some disadvantages:
  • The mobile UI is usually developed in a different codebase. And the server side error message modification is not possible. Specially if the same chunk of server side code is serving many applications.
  • -The language used on the mobile UI can vary. For example the UX developer can decide to use:
    1. "You haven't provided the group" - First person complaining style.
    2. "The group is missing" - Passive objective style
    3. "Please provide a group" - Proactive gentle style
    4. "Select a group!" - Imperative style
    5. ...
In order to formulate the sentences which reflects the application mood the mobile developer needs to interpret the returned error based on its error_code and formulate its own corresponding message.

Happy coding :)



API Development Restfull vs Facebook style


In the last months I developed mainly API's using grape gem and their related gems.

In this blog post I will express my opinion how to organise the API which manages the entities.

Generally speaking I like the REST concept and I organize my API to conform that way. For cases when there are no association among the entities this philosophy is good enough and clear.

However I like the API to reflect the associations when we are dealing with has_many or has_and_belongs_to_many associations. Just like in case of  Facebook Graph API. In these cases I find that style more intuitive and I am following that style whenever I deal with associations.

Examples:

Lets suppose that cars and manufacturers can be reviewed.

class Car < ActiveRecord::Base
  has_many :reviews, as: :reviewable

class Manufacturer < ActiveRecord::Base
  has_many :reviews, as: :reviewable

class Review < ActiveRecord::Base
  belongs_to :user
  belongs_to :reviewable, polymorphic: true

In this case the review API would be:
  • creation of reviews:
    POST /api/cars/{id}/reviews
    POST /api/manufacturers/{id}/reviews
  • retrieval of reviews:
    GET /api/cars/{id}/reviews
    GET /api/manufacturers/{id}/reviews
  • deletion of reviews:
    DELETE /api/reviews/{id}


In the classic restful style the review creation would be:

    params do
      requires :comment, type: String, desc: "The comment"
      requires :reviewable_id, type: Integer, desc: "The id of a reviewable entity"
      requires :reviewable_type, type: String, desc: "The type of a reviewable entity"
    end
    POST /api/reviews

Both solutions have pros and cons. The restful style is more DRY. The Facebook Graph Api style relieves more information about the associations ergo it is more intuitive and easier to use.

Happy coding :)



Friday, 2 January 2015

Specing the API


When you cover your API with specs the first rule is to cover everything. 
Since the API can be called by third parties you need to be sure what is happening in every case. You must be able to reproduce every scenario any time and you must have the same results. :) Does it seems scientific? Well... Indeed it is.

Let me show you some examples:
-You might need to check that only the calls with developer key can access the API. In the opposite case the response is 401, it is still a JSON and maybe an audit log is created.
-You might need to check that the user is authenticated. If not then the response is 401, it is still a JSON and again an audit log is created.
-If some of the parameters are missing or wrong then the response is 400, it is still a JSON and an audit log is created by logging all the incoming parameters.
-If everything is alright then the response is 200/201, the response is a JSON and the entities are massaged as needed by that specific case.

You also need to keep your specs DRY
This will reduce maintenance effort of the test code and it will keep your specs more readable and you will have easy time to add new features and you will feel more happy and in control.

As you can see the above steps like checking the response code, checking if the response format is JSON, checking that the audit log is created are repetitive tasks and can be DRY-ed with rspecs shared examples. The only specific thing which changes from one api endpoint to another is "the entities are massaged as needed by that specific case".

RSpec.shared_examples "returning 401" do
  specify "returns 401" do
    api_call params, developer_header
    expect(response.status).to eq(401)
  end
end
RSpec.shared_examples "being JSON" do
  specify 'returns JSON' do
    api_call params, developer_header
    expect { JSON.parse(response.body) }.not_to raise_error
  end
end
...

Then in your specs you can use these shared examples in particular cases:

context '/API/learn_by_playing/' do
  def api_call *params
    get "/api/learn_by_playing", *params
  end
  let(:api_key) { create :apikey }
  let(:developer_header) { {'Authorization' => api_key.token} }
  context 'GET' do
    let(:required_params) do
      {
        :first_name => 'Botond',
        :last_name => "Orban"
      }
    end
    let(:params) { required_params }

    it_behaves_like 'restricted for developers'

    context 'wrong parameters' do
      required_params.keys.each do |s|
        context 'when the #{s} parameter is blank' do
          let(:params) { required_params.merge({s => ''}) }
          it_behaves_like 'returning 400'
          it_behaves_like 'being JSON'
          it_behaves_like 'creating an audit log'
        end
        context 'when the #{s} parameter is missing' do
          let(:params) { required_params.except(s) }
          it_behaves_like 'returning 400'
          it_behaves_like 'being JSON'
          it_behaves_like 'creating an audit log'
        end
      end
    end
    context 'valid params' do
      specify '...whatever you need to really assert for in this particular case...' do
        api_call params, developer_header
        expect ...
      end
    end
  end
end

As you can see from the above example using Rspec.shared examples a lot of repetitive lines from your Rspec can be thrown out.

On one of my projects I managed to reduce the specs size by 20% and raise the visibility a lot :) I can't measure and therefore I can't express in numbers what "a lot" means but everybody was satisfied  and pleased with the result.

Professional DRY hacking ;)

Sunday, 28 December 2014

API Development in Ruby On Rails, Nested Entities

There are times when there is one-to-many association amongst the entities. And the mobile developer needs to show all of them in one screen. In these cases it is very easy to fell into the bad habit to execute n+1 queries from mobile side toward the server or from the server to the sql-server.

Here are the entities:

class DetailedGameEntity < Grape::Entity
  ...
  expose :reviews, using: ReviewEntity do |game, options|
    game.reviews
  end
end

class ReviewPresenter < Grape::Entity
  expose :user_id, as: :owner_id
  expose :comment
  expose :rating

  expose :image_urls do |review, options|
    review.images.map{|image| image.image}
  end
end

They are designed to return everything the mobile screen needs. The games contains their reviews. And the reviews contain their images. So in one query the whole mobile screen can be populated.

Let's see how a programmer in a hurry develops all this API:

games = Game.actual.limit(500)
present games, with: DetailedGameEntity

By developing the API this way the call will result in these queries:
...
  Review Load (0.5ms)  SELECT "reviews".* FROM "reviews"  WHERE "reviews"."reviewable_id" = $1 AND "reviews"."reviewable_type" = $2  [["reviewable_id", 2359], ["reviewable_type", "Game"]]
  Review Load (0.4ms)  SELECT "reviews".* FROM "reviews"  WHERE "reviews"."reviewable_id" = $1 AND "reviews"."reviewable_type" = $2  [["reviewable_id", 2358], ["reviewable_type", "Game"]]
...

As you can see for each returned game game another select is executed to fetch the belonging reviews. The same is done to fetch the belonging user and to fetch the images belonging to the review. This is a very ineffective way because as the review count grows more and more queries need to be executed. This problem is called n+1 query problem.

I do this in my code in these cases:

games = Game.actual.limit(500)
games = games.includes(reviews: [:images, :user] )
present games, with: DetailedGameEntity

The above code will do this sql query:

 Review Load (1.0ms)  SELECT "reviews".* FROM "reviews"  WHERE "reviews"."reviewable_type" = 'Schedule' AND "reviews"."reviewable_id" IN (2195, 2198, 2197, 9567, 9572, 9573, 9574, 9575, 9576, 9571, 9570, 9569, 9568, 2196, 2204, 2210, 2211, ...

So, the underlying Rails code will execute only one query per entity.

It is nice, isn't it?

Please use it ;)

Saturday, 27 December 2014

API Development in Ruby On Rails, Entities

In my last blog post I presented in general the features I like the most about the grape gem. Now I will present the grape entities which are related to the presentation of the returned data. I like these entities a lot because they are an OO way to present data and they help a lot to keep my presentation layer DRY.

The last statement of every grape call is returned to the caller as JSON.
Despite this simple efficiency I like to have more control over the returned data and I use the grape entities to format the returned data wherever I can. I can also rspec them and I am doing it extensively because I am a Test Driven Guy ;)

Let me present some examples of entities their usage and how do I spec them:

Here is the spec for the entity:

describe GameEntity do
  describe 'fields' do
    subject(:subject) { GameEntity }
    it { is_expected.to represent(:id) }
    it { is_expected.to represent(:user_id).as(:owner_id) }
...


Here is the entity itself:

class GameEntity < Grape::Entity
  expose :id
  expose :user_id, as: :owner_id
...

The part of the grape API:

  namespace :games do
    desc "Retrieve all the games"
    params do
      optional :include_reviews, type: Boolean, default: true, desc: 'Accepts: true/false.'
    end
    get do
      games = Game.actual.limit(500)
      present games, with: GameEntity

This was simple so far. But I can use a more sophisticated GameEntity in case I want to present the included reviews. In all other cases I will use my old simple GameEntity to present the Game.

Let's see that case too:

I modify the grape API to this:

    ...
    if params[:include_reviews]
      games = games.includes(:reviews)
      present games, with: DetailedGameEntity
    else
      present games, with: GameEntity
    end

I also spec the new entity:

describe DetailedGameEntity do
  describe 'special fields' do
      subject(:detailed_game) { DetailedGameEntity(game).as_json }
      specify { expect(detailed_game[:reviews]).to be_an(Array) }
  end

Of course the above was a very simple example.

In real world scenarios we can find ourselves that we are reusing the entities like this:

class DetailedGameEntity < Grape::Entity
  expose :game, using: GameEntity do |game, options|
    game
  end

  expose :venue, using: VenueEntity do |game, options|
    game.the_venue
  end

  expose :visitor_team, using: TeamEntity do |game, options|
    game.visitor_team
  end

  expose :host_team, using: TeamEntity do |game, options|
    game.host_team
  end
end

As you can see we are keeping our entity codebase DRY by calling the TeamEntity twice in the DetailedGameEntity. This is a typical use of delegation of the Entities.

Let me show a more complex example of our entities:

The usage of the entities:

  desc "Retrieve all the reviews belonging to a team"
    get do
      ...
      present reviews, :with => ReviewPresenter, user: current_user
    end

The entity itself:
  class ReviewPresenter < Grape::Entity
    expose :current_user_likes do |review, options|
      review.likes.pluck(:user_id).include? options[:user].id if options[:user].present?
    end
  end

And finally the spec for the entity:
describe ReviewPresenter do
  describe 'special fields' do
    context 'without passing a user' do
      subject(:presented_review) { ReviewPresenter.new(image.review).as_json }
      specify { expect(presented_review[:current_user_likes]).to be_nil }
    end

    subject(:presented_review) { ReviewPresenter.new(image.review, user: current_user).as_json }
    ...
      context 'user is the current_user' do
        specify { expect(presented_review[:current_user_likes]).to eq(true) }
      end
    end
    ...
  end
end

What I was doing here is to populate the field 'current_user_likes'
 to true or false depending if the current_user likes or not the presented review.

As you can see the presentation layer of any grape API could became more DRY and versatile by using the grape entities.

Enjoy using them ;)

Friday, 26 December 2014

API Development in Ruby On Rails

I met a technology and a set of gems which were designed to develop any API in Ruby On Rails about half years ago.

The set of gems were:


At first glance it seemed that it adds complexity to the Rails application compared to standard Rails JSON returns. After I played a bit with it I suddenly realized that its API formulating DSL is much more superior. So I started to like it :)

Let me list the elements of DSL I like most:

  • ability to mount API endpoints. This can be very handy if I want to unmount certain unused parts of the API.

      mount Caesars::Ping
      mount Caesars::PingProtected

  • ability to describe the API endpoint. This description will appear in the Swagger documentation. It is a help for the API user.
  • ability to specify the parameters (type, mandatory or required).
      params do
        optional :user_ids, desc: "JSON array of the user ids to be deleted."
        requires :group_id, type: Integer, desc: "The id of the group"
      end

By specifying the type our parameters will be converted automagically to the specified type. It is easier this way for us developers to write our core logic.
  • a complex DSL to organize the API hierarchy and use route params. I consider this the most powerful feature. I can easily organize the API to look more like Facebook graph API or Philips Hue API than standard Rails restfull API.
      namespace :teams do
        route_param :id do
          desc 'Retrieve all games belonging to teams'
          get 'scores' do
            #DO logic here
          end
        end
      end

The grape features are many more including versioning. Take a look for a complete description at grape site ;)