Bloggy #5 - Enforcing the media type and improving the controllers of Bloggy

We now have Yumi implemented but we’re missing something.

Our API still works with the media type application/json! Since the JSON API specification requires using application/vnd.api+json, we need to update our API.

Since I’ve decided that we won’t be supporting application/json anymore, we also need to change how we expect the requests to look like. The JSON API specification luckily specifies how those are supposed to be formatted.

For example, here is a request to create a resource:

POST /photos HTTP/1.1
Content-Type: application/vnd.api+json
Accept: application/vnd.api+json

{
  "data": {
    "type": "photos",
    "attributes": {
      "title": "Ember Hamster",
      "src": "http://example.com/images/productivity.png"
    },
    "relationships": {
      "photographer": {
        "data": { "type": "people", "id": "9" }
      }
    }
  }
}

Source: JSON API Specification

We are going to change our endpoints to expect this format and handle it correctly. Let’s get started!

Master Ruby Web APIs

APIs are evolving. Are you prepared?

The Bloggy series

You can find the full list of jutsus for this series in this jutsu and the jutsu before this one is available there.

Getting the code

You can get the source code to follow this tutorial from GitHub if you didn’t follow the previous jutsu.

Coding Time \o/

1. Enforcing the media type

So first, we need to restrict the media type to application/vnd.api+json.

1.1 Testing the water

Let’s see what happens currently when we make a request. If it’s not running, start your server with shotgun config.ru.

curl -I http://localhost:9393/api/v1/posts

Output:

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 2164
Server: WEBrick/1.3.1 (Ruby/2.2.0/2014-12-25)
Date: Tue, 15 Mar 2016 06:40:37 GMT
Connection: Keep-Alive

1.2 Removing support for application/json

Enforcing the media type application/vnd.api+json can be a bit tricky with Grape. Luckily, there is a way to do it.

First, we need to tell Grape which content_type to use. We will do that with this code:

format :json
formatter :json, -> (object, _env) { object.to_json }
content_type :json, 'application/vnd.api+json'

Then we need to add a callback before any request to check which media type was sent by the client. We will use the following code for this:

before do
  unless request.content_type == 'application/vnd.api+json'
    error!('Unsupported media type', 415)
  end
end

Now let’s see how it integrates within our application.rb file. Here is the full content of the file if you’re lazy ;)

# application.rb
require 'grape'
require 'mongoid'

Mongoid.load! "config/mongoid.config"

META_DATA = {
  name: 'Bloggy',
  description: 'A simple blogging API built with Grape.'
}

# Load files from the models and api folders
Dir["#{File.dirname(__FILE__)}/app/models/**/*.rb"].each { |f| require f }
Dir["#{File.dirname(__FILE__)}/app/api/**/*.rb"].each { |f| require f }
Dir["#{File.dirname(__FILE__)}/app/yumi/**/*.rb"].each { |f| require f }
Dir["#{File.dirname(__FILE__)}/app/presenters/**/*.rb"].each {|f| require f}

# Grape API class. We inherit from it in our controllers.
module API
  class Root < Grape::API
    prefix :api

    format :json
    formatter :json, -> (object, _env) { object.to_json }
    content_type :json, 'application/vnd.api+json'

    helpers do
      def base_url
        "http://#{request.host}:#{request.port}/api/#{version}"
      end

      def invalid_media_type!
        error!('Unsupported media type', 415)
      end

      def json_api?
        request.content_type == 'application/vnd.api+json'
      end
    end

    before do
      invalid_media_type! unless json_api?
    end

    # Simple endpoint to get the current status of our API.
    get :status do
      { status: 'ok' }
    end

    mount V1::Admin::Posts
    mount V1::Comments
    mount V1::Posts
  end
end

# Mounting the Grape application
DevblastBlog = Rack::Builder.new {

  map "/" do
    run API::Root
  end

}

1.3 Retrying our request

Let’s try the same request we did earlier. But this time, it shouldn’t work and return 415, Unsupported Media Type.

curl -I http://localhost:9393/api/v1/posts

Output:

HTTP/1.1 415 Unsupported Media Type
Content-Type: application/vnd.api+json
Content-Length: 34
Server: WEBrick/1.3.1 (Ruby/2.2.0/2014-12-25)
Date: Tue, 15 Mar 2016 06:51:41 GMT
Connection: Keep-Alive

Yes, it’s working! We have what we wanted and we’re receiving a 415 back as expected in the JSON API specification.

1.4 Making a functional request

Now, how do we make our requests work? Well, we just need to specify the right media type in the Content-Type header. We can do this with cURL by using the option -H:

curl -I http://localhost:9393/api/v1/posts -H 'Content-Type: application/vnd.api+json'

Output:

HTTP/1.1 200 OK
Content-Type: application/vnd.api+json
Content-Length: 2164
Server: WEBrick/1.3.1 (Ruby/2.2.0/2014-12-25)
Date: Tue, 15 Mar 2016 06:55:51 GMT
Connection: Keep-Alive

Remove -I if you want to see the body.

And it works! We’re getting our JSON API document back with the right headers. We’re done enforcing the media type, now it’s time to handle the JSON API format for requests sent by the clients.

2. Updating our controllers

Time to fix our API. We only have to change the Comments controller because the Posts controller doesn’t have any requests with a payload, only GET requests.

2.1 Getting rid of the strong parameters exception

Before we actually change anything, we have a small tweak to do. Currently, when we try to update a comment for example, we have the following code:

comment.update!({
  author:  params[:author],
  email:   params[:email],
  website: params[:website],
  content: params[:content]
})

That’s a problem. Since we are going to change the HTTP verb from put to patch for our updates, we shouldn’t replace the whole resource anymore, only the specified fields. So we need to have something like this where nil parameters are ignored:

comment.update!(params)

But since we are not using Rails or the Strong Parameters gem, we are going to get an exception from Mongoid saying that the params are not safe. The thing is that Grape already provide us with parameters validation in the following form:

params do
  requires :id, type: String
  requires :author, type: String
  requires :email, type: String
  requires :website, type: String
  requires :content, type: String
end

So what can we do? Luckily, there is a little gem called hashie-forbidden_attributes that will prevent this exception. Obviously, we need to be carefully but we will be using Grape to validate the parameters so everything should be fine.

Let’s add the gem to our Gemfile.

# Gemfile
source 'https://rubygems.org'

gem 'grape'
gem 'mongoid', '~> 5.0.0'
gem 'pry-byebug'
gem 'shotgun'

# Add this
gem 'hashie-forbidden_attributes'

gem 'rspec'
gem 'rack-test'
gem 'factory_girl'
gem 'mongoid_cleaner'

Followed by a quick bundle install to get everything in place.

We also need to require this gem in the application.rb file.

# application.rb
require 'grape'
require 'mongoid'

# Add this
require 'hashie-forbidden_attributes'

# Rest of the file...

Don’t forget to restart your application.

We are safe now! Let’s proceed.

2.2 The Comments controller

Now we can fix the Comments controller. Here is the list of stuff we need to do:

1. Validate the parameters of the request with Grape
2. Use declared(params) to access those validated parameters
3. Only update the specified attributes in the update action

So first, we need to change the params we accept from this:

params do
  requires :author, type: String
  requires :email, type: String
  requires :website, type: String
  requires :content, type: String
end

To something that follows the JSON API specification like this:

params do
  requires :post_id, type: String
  requires :data, type: Hash do
    requires :type, type: String
    requires :attributes, type: Hash do
      requires :author, type: String
      optional :email, type: String
      optional :website, type: String
      requires :content, type: String
    end
  end
end

I like this feature of Grape because of how easy it is to see what the expect parameters hash is supposed to look like.

Then we will need to use the method declared offered by Grape to access those validated parameters:

post.comments.create!(declared(params)['data']['attributes'])

We also need to make the attributes optional in the update action so a client can only send what needs to be updated.

# ...
  requires :attributes, type: Hash do
    optional :author, type: String
    optional :email, type: String
    optional :website, type: String
    optional :content, type: String
  end
# ...

By default, Grape will assign a nil value to optional parameters that are not present. We can easily clean that up using the following code:

comment_params = declared(params)['data']['attributes'].reject { |k, v| v.nil? }
comment.update_attributes!(comment_params)

That’s about everything we have to change in the file so let’s do it now. Here is the full content for the Comments controller:

# app/api/v1/comments.rb
module API
  module V1
    class Comments < Grape::API
      version 'v1', using: :path, vendor: 'devblast-blog'

      # Nested resource so we need to add the post namespace
      namespace 'posts/:post_id' do
        resources :comments do

          desc 'Create a comment.'
          params do
            requires :post_id, type: String
            requires :data, type: Hash do
              requires :type, type: String
              requires :attributes, type: Hash do
                requires :author, type: String
                optional :email, type: String
                optional :website, type: String
                requires :content, type: String
              end
            end
          end
          post do
            post = Post.find(params[:post_id])
            comment = post.comments.create!(declared(params)['data']['attributes'])
            Presenters::Comment.new(base_url, comment).as_json_api
          end

          desc 'Update a comment.'
          params do
            requires :post_id, type: String
            requires :id
            requires :data, type: Hash do
              requires :type, type: String
              requires :attributes, type: Hash do
                optional :author, type: String
                optional :email, type: String
                optional :website, type: String
                optional :content, type: String
              end
            end
          end
          patch ':id' do
            post = Post.find(params[:post_id])
            comment = post.comments.find(params[:id])
            comment_params = declared(params)['data']['attributes'].reject { |k, v| v.nil? }
            comment.update_attributes!(comment_params)
            Presenters::Comment.new(base_url, comment.reload).as_json_api
          end

          desc 'Delete a comment.'
          params do
            requires :id, type: String
          end
          delete ':id' do
            post = Post.find(params[:post_id])
            post.comments.find(params[:id]).destroy
          end

        end
      end

    end
  end
end

2.3 Testing the Comments controller

We will write automated tests in the next jutsu. For now, here are a few cURL requests to ensure that everything is working correctly.

Get your list of posts. (If you don’t have any, create one using the racksh console).

curl http://localhost:9393/api/v1/posts -H 'Content-Type: application/vnd.api+json'

Get one of your post ID and replace POST_ID in the request below. Now try to create a comment by running the command.

curl -i http://localhost:9393/api/v1/posts/POST_ID/comments -H 'Content-Type: application/vnd.api+json' -d '{"data":{"type":"comments","attributes":{"author":"thibault","email":"thibault@example.com","website":"devblast.com","content":"Cool"}}}'

Your post should be created correctly! Now let’s try to only update the author (changed from ‘thibault’ to ‘tibo’). Don’t forget to replace the POST_ID and the COMMENT_ID in the command!

curl -i http://localhost:9393/api/v1/posts/POST_ID/comments/COMMENT_ID -X PATCH -H 'Content-Type: application/vnd.api+json' -d '{"data":{"id":"COMMENT_ID","type":"comments","attributes":{"author":"tibo"}}}'

It should return the comment with the updated author ‘tibo’. Finally, clean up the mess by calling the delete endpoint. Once again, replace POST_ID and COMMENT_ID in the following command:

curl -i http://localhost:9393/api/v1/posts/POST_ID/comments/COMMENT_ID -X DELETE -H 'Content-Type: application/vnd.api+json'

Our Comments controller is performing well, neat!

2.4 Updating the Admin controller

Updating the admin controller for posts follows the same logic. I won’t go into the details because it’s pretty much the same thing so just take a look at the code to see what’s going on.

# app/api/v1/admin/posts.rb
module API
  module V1
    module Admin
      class Posts < Grape::API
        version 'v1', using: :path, vendor: 'devblast-blog'

        namespace :admin do

          helpers do
            def base_url
              "http://#{request.host}:#{request.port}/api/#{version}/admin"
            end
          end

          resources :posts do

            desc 'Returns all posts'
            get do
              Presenters::Post.new(base_url, Post.all.ordered).as_json_api
            end

            desc "Return a specific post"
            params do
              requires :id, type: String
            end
            get ':id' do
              Presenters::Post.new(base_url, Post.find(params[:id])).as_json_api
            end

            desc "Create a new post"
            params do
              requires :data, type: Hash do
                requires :type, type: String
                requires :attributes, type: Hash do
                  requires :slug, type: String
                  requires :title, type: String
                  optional :content, type: String
                end
              end
            end
            post do
              post = Post.create!(declared(params)['data']['attributes'])
              Presenters::Post.new(base_url, post).as_json_api
            end

            desc "Update a post"
            params do
              requires :id, type: String
              requires :data, type: Hash do
                requires :type, type: String
                requires :id, type: String
                requires :attributes, type: Hash do
                  optional :slug, type: String
                  optional :title, type: String
                  optional :content, type: String
                end
              end
            end
            patch ':id' do
              post = Post.find(params[:id])
              post_params = declared(params)['data']['attributes'].reject { |k, v| v.nil? }
              post.update_attributes!(post_params)
              Presenters::Post.new(base_url, post.reload).as_json_api
            end

            desc "Delete a post"
            params do
              requires :id, type: String
            end
            delete ':id' do
              Post.find(params[:id]).destroy
            end

          end
        end
      end

    end
  end
end

2.5 Testing the admin controller with cURL

Here are some cURL commands you can run to see if everything is going smoothly with the admin controller.

Get all the posts:

curl -i http://localhost:9393/api/v1/admin/posts -H 'Content-Type: application/vnd.api+json'

Get a specific posts. Replace POST_ID below.

curl -i http://localhost:9393/api/v1/admin/posts/POST_ID -H 'Content-Type: application/vnd.api+json'

Create a post.

curl -i http://localhost:9393/api/v1/admin/posts -d '{"data":{"type":"posts","attributes":{"slug":"media-type","title":"Media Type!","content":"Cool"}}}' -H 'Content-Type: application/vnd.api+json'

Update a post. Replace POST_ID below.

curl -i http://localhost:9393/api/v1/admin/posts/POST_ID -X PATCH -d '{"data":{"type":"posts","id":"POST_ID","attributes":{"title":"New Post 1"}}}' -H 'Content-Type: application/vnd.api+json'

Delete a post. Replace POST_ID below.

curl -i http://localhost:9393/api/v1/admin/posts/POST_ID -X DELETE -H 'Content-Type: application/vnd.api+json'

Everything’s working? Awesome!

That’s it, we’re done updating the controllers. Our endpoints are ready for the future front-end application we will create in the next series of jutsus. But we are not completely done with Bloggy yet: in the next jutsu, we will write automated tests to ensure that we never break anything in the future!

3. What did we skipped?

To be honest, Bloggy is not totally compliant with the JSON API specification. There are things that I didn’t cover in the series either because it would make things too long or because I just go lazy… Sorry I’m also human! :)

Here is the list of what Bloggy doesn’t support and that you can implement if you want:

• HTTP codes

We didn’t implement all the mandatory HTTP codes in our responses. I will give more details about that below.

• Delete Response

We didn’t implement all the correct HTTP status code when a resource is correctly deleted (204 No Content) and we simply went with Grape default 200 OK. Responding with 200 is actually fine but we then need to responds with top-level meta data and Yumi doesn’t allow for cherry-picking which top-level keys we want so we didn’t do it. Maybe a feature to add for you ;)

• Errors for creating and updating

We are currently not handling any error so any validation error will return a 500. Fixing this is not too hard and by using a few checking, you could return the correct status codes.

• Pagination for posts

• Endpoints to change relationships

Our JSON documents contain links to access related resources or relationships but we did not implement them in our API.

• Belongs To

Yumi only supports has_many relationships which means presenting a comment, for example, wouldn’t come with its parent post as a related resource. Adding the belongs_to relationship is actually not that hard and I encourage you to do it yourself ;)

What’s next?

This series is almost finished. We built a complete Grape API from scratch with its own JSON API implementation. But no application is complete without automated tests and that’s exactly what we are going to do in the next jutsu.

Source Code

The source code is available on GitHub.