Chapter 6

Handling Errors

The curl requests we have used so far have all been expertly crafted to work. We haven’t tried to game the API or make it fail, because we are nice people.

Not everyone is like this, unfortunately! And it’s not only people with bad intentions we need to look out for, but also simple mistakes. Anyone can make a mistake - it happens. That does not mean we should laugh at them for trying and send back some indecipherable message.

Descriptive messages are a very important part of any web API. Web APIs are used by our fellow developers and, by building an API, we take the responsibility of providing a good service for them. Any service should be easy to use. Put yourself inside a user’s shoes and pinpoint the issues someone could have with your service.

The obvious problem in our current web API is that we don’t send back meaningful errors when something goes wrong.

Let’s take the following curl request as an example. Notice anything wrong? We forgot to close the JSON document. Before we can realize our mistake, we send the request…

curl -i -X POST http://localhost:4567/users \
     -H "Content-Type: application/json" \
     -d '{"first_name":"Samuel", "last_name":"Da Costa", "age":19'

and we end up with a scary 500 Internal Server Error. This seems to mean that there was something wrong with the server, not with our request.

HTTP/1.1 500 Internal Server Error
Content-Type: text/plain
Content-Length: 4679
Connection: keep-alive
Server: thin

[JSON::ParserError]

Luckily, Sinatra sends back the whole stack trace, so we can easily debug it. Not every framework is that nice, though. That’s the server’s duty - to provide descriptive errors to the client. A 500 is as descriptive as looking at a black box that just says: “Sorry bro, it didn’t work”.

HTTP comes with a set of features to handle errors. Indeed, thanks to all its HTTP status codes, we can configure our API to send back descriptive error messages to a client in a format that it can understand.

6.1. The different classes of HTTP status codes

There are five classes of HTTP codes:

1XX Informational: Status codes starting with 1 are known as informational codes. Most of them are rarely used nowadays.
2XX Success: These codes indicate that the exchange between the server and the client was successful.
3XX Redirection: 3XX codes indicate that the client must take additional action before the request can be completed.
4XX Client Error: There was something wrong with the request the client sent and the server cannot process it.
5XX Server Error: The client sent a valid request but the server was not able to process it successfully.

We will be using status codes from some of these classes in this chapter.

6.2. Global

We will add error handling for each route, but let’s first see the errors we need to handle for every request.

6.2.1. 405 Method Not Allowed

This HTTP status code can be used when the client tries to access a resource using an unsupported HTTP method. For example, in our API, what would happen if a client tried to use the PUT method with the /users URI?

Let’s give it a try.

curl -i -X PUT http://localhost:4567/users

Output

HTTP/1.1 404 Not Found
Content-Type: text/html;charset=utf-8
X-Cascade: pass
Content-Length: 467
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
Connection: keep-alive
Server: thin

<!DOCTYPE html>
<html>
<head>
<MORE HTML>

That’s not very helpful… Plus, it returns 404. While we do have a resource living at that URI, it’s not the right HTTP method… we can do better. All we need to do is catch any request accessing /users with an unsupported method: put, patch and delete. With a bit of metaprogramming, it’s not hard to do.

In the code below, we loop through each of the HTTP methods we don’t support and define a Sinatra route that returns 405 Method Not Allowed for each one of them.

# webapi.rb
# /users
# head /users
# get /users
# post /users

[:put, :patch, :delete].each do |method|
  send(method, '/users') do
    halt 405
  end
end

# options /users/:first_name

Let’s give it another try after restarting the server, shall we?

curl -i -X PUT http://localhost:4567/users

Output

HTTP/1.1 405 Method Not Allowed
Content-Type: text/html;charset=utf-8
Content-Length: 0
[MORE HEADERS]

Alright! It’s working correctly, let’s proceed.

6.2.2. 406 Not Acceptable

We already implemented one HTTP status code in the previous chapters. If the client requests a media type that our API does not support, we will return 406 Not Acceptable. To the client receiving this, it means that the server was not able to generate a representation for the resource according to the criteria the client had fixed. The response should also include what formats are actually available for the client to pick from.

# webapi.rb
# users
# ...

helpers do

  # def json_or_default?
  # def xml?

  def accepted_media_type
    return 'json' unless request.accept.any?

    request.accept.each do |type|
      return 'json' if json_or_default?(type)
      return 'xml' if xml?(type)
    end

    content_type 'text/plain'
    halt 406, 'application/json, application/xml'
  end

  # def type
  # def send_data

end

Let’s try with a fake media type that we do not support.

curl -i http://localhost:4567/users -H "Accept: moar/curl"

Output

HTTP/1.1 406 Not Acceptable
Content-Type: text/plain;charset=utf-8
Content-Length: 33
... More Headers ...

application/json, application/xml

With this, the client can understand what is actually supported and act accordingly. Sadly, it’s not defined more clearly in a standard.

Alternatively, we could actually return a JSON representation instead of telling the client that we can’t give it anything. We would set the Content-Type header to application/json to let the client do its own checking before it parses the response. Both this option and the previous one are acceptable as defined in the HTTP RFC.

# webapi.rb
# users
# ...

helpers do

  # def json_or_default?
  # def xml?

  def accepted_media_type
    return 'json' unless request.accept.any?

    request.accept.each do |type|
      return 'json' if json_or_default?(type)
      return 'xml' if xml?(type)
    end

    'json'
  end

  # def type
  # def send_data

end

Don’t forget to restart the server.

curl -i http://localhost:4567/users -H "Accept: moar/curl"

Output

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 203
... More Headers ...

[
  {"first_name":"Thibault", "last_name":"Denizet", "age":25, "id":"thibault"},
  {"first_name":"Simon", "last_name":"Random", "age":26, "id":"simon"},
  {"first_name":"John", "last_name":"Smith", "age":28, "id":"john"}
]

If your server cannot generate a representation the way the client requests, you can either return 406 (with or without a list of actually supported media types), or just return the default format and let the client handle it.

6.3. POST /users

For the post /users route, we are currently not handling any errors. If you run the following curl request, it will crash and send you back 500 and the whole stack trace.

curl -X POST -i http://localhost:4567/users \
     -H "Content-Type: application/fake" \
     -d 'Weirdly Formatted Data'

Output

HTTP/1.1 500 Internal Server Error
Content-Type: text/plain
Content-Length: 4647
Connection: keep-alive
Server: thin

JSON::ParserError: 757: unexpected token at 'Weirdly Formatted Data'
[Stack Trace]

This does not help the client in understanding what happened. It does help us humans, because we can read the Ruby stack trace and see that there was a JSON::ParserError. But to an automated client, this only means that something went wrong on the server. We need to fix this.

6.3.1. 415 Unsupported Media Type

To fix it we will use the 415 HTTP status code. 415 Unsupported Media Type is exactly what its name implies - we can return it to the client when we don’t understand the format of the data sent by the client.

To prevent the execution of the code there, we will first check if the data was sent as a JSON document, the only format we support for requests. Our code must be guarded from unwanted media types.

We just need the following line to do it.

halt 415 unless request.env['CONTENT_TYPE'] == 'application/json'

And it comes in the post /users route like this:

# webapi.rb
# Stuff
# ...
post '/users' do
  halt 415 unless request.env['CONTENT_TYPE'] == 'application/json'

  users[user['first_name'].downcase.to_sym] = user

  url = "http://localhost:4567/users/#{user['first_name']}"
  response.headers['Location'] = url
  status 201
end

Don’t forget to restart the server.

Now, let’s see what happens when we send the curl request seen in the previous section.

curl -X POST -i http://localhost:4567/users \
     -H "Content-Type: application/fake" \
     -d 'Weirdly Formatted Data'

Output

HTTP/1.1 415 Unsupported Media Type
[MORE HEADERS]

That’s much more descriptive, right? The only problem is that this does not tell the client which media types are actually supported. We will see what we can do about this later. For now, our endpoint won’t crash anymore whatever the received format is.

6.3.2. 400 Bad Request

The 400 Bad Request is used way too often in modern web applications. Its real purpose is to indicate that the server could not understand the request due to some syntax issue. For example, a malformed JSON document.

Let’s give it a try and break some stuff.

curl -X POST -i http://localhost:4567/users \
     -H "Content-Type: application/json" \
     -d '{"first_name":"Mark"'

Output

HTTP/1.1 500 Internal Server Error
Content-Type: text/plain
Content-Length: 4645
Connection: keep-alive
Server: thin

JSON::ParserError: 757: unexpected token at '{"first_name":"Mark"'
[Stack Trace]

Boom! Broken. Since our JSON document, {"first_name":"Mark", is not valid, our server crashes when trying to parse it. Here is how we can fix this. We are going to catch the exception raised when parsing an invalid JSON document (JSON::ParserError). In the rescue, we will send back the error to the client, in the format of its choice (specified by the Accept header). Note that we are still using the send_data method we wrote earlier.

# webapi.rb
# Stuff
# ...
post '/users' do
  halt 415 unless request.env['CONTENT_TYPE'] == 'application/json'

  begin
    user = JSON.parse(request.body.read)
  rescue JSON::ParserError => e
    halt 400, send_data(json: -> { { message: e.to_s } },
                        xml:  -> { { message: e.to_s } })
  end

  users[user['first_name'].downcase.to_sym] = user
  url = "http://localhost:4567/users/#{user['first_name']}"
  response.headers['Location'] = url
  status 201
end

Don’t forget to restart your server!

curl -X POST -i http://localhost:4567/users \
     -H "Content-Type: application/json" \
     -d '{"first_name":"Mark"'

Output

HTTP/1.1 400 Bad Request
Content-Type: application/json
Content-Length: 65
X-Content-Type-Options: nosniff
Connection: keep-alive
Server: thin

{"message":"757: unexpected token at '{\"first_name\":\"Mark\"'"}

Great! We are getting a 400 back and the body contains the error message.

6.3.3. 409 Conflict

Currently, if we try to re-create a user that already exists, it’s just going to override the existing one. Not good. This should throw up some kind of conflict error saying that a user cannot be overridden. The problem here is how we save users in a Ruby hash. Since we use the first name as a key, we don’t want to allow the override of an existing user.

Luckily, that’s what the 409 Conflict HTTP code is for. Note that this code is only allowed in situations where the client can actually fix the conflict. In our case, the client can either change the first name or use another endpoint to update the user.

Our use of the first name as a key is there only for simplicity and any production application should not use such a mechanism. Unique generated IDs are way better!

To prevent any override, we will use this code. If we find a user with the first name sent by the client, we’ll immediately halt and send back a response to the client with a 409 status code and an explanation message.

if users[user['first_name'].downcase.to_sym]
  message = { message: "User #{user['first_name']} already in DB." }
  halt 409, send_data(json: -> { message },
                      xml:  -> { message })
end

Here is our full POST endpoint:

# webapi.rb
# Stuff
# ...
post '/users' do
  halt 415 unless request.env['CONTENT_TYPE'] == 'application/json'

  begin
    user = JSON.parse(request.body.read)
  rescue JSON::ParserError => e
    halt 400, send_data(json: -> { { message: e.to_s } },
                        xml:  -> { { message: e.to_s } })
  end

  if users[user['first_name'].downcase.to_sym]
    message = { message: "User #{user['first_name']} already in DB." }
    halt 409, send_data(json: -> { message },
                        xml:  -> { message })
  end

  users[user['first_name'].downcase.to_sym] = user
  url = "http://localhost:4567/users/#{user['first_name']}"
  response.headers['Location'] = url
  status 201
end

Let’s try to recreate an existing user (for example, Thibault), after restarting the server.

curl -X POST -i http://localhost:4567/users \
     -H "Content-Type: application/json" \
     -d '{"first_name":"Thibault", "last_name":"Denizet", "age":25}'

Output

HTTP/1.1 409 Conflict
Content-Type: application/json
Content-Length: 42
X-Content-Type-Options: nosniff
Connection: keep-alive
Server: thin

{"message":"User Thibault already in DB."}

Great, we’re telling the client that its request triggered a conflict that needs to be solved. We can now detect when updates conflict with the users’ data stored in the users hash.

6.4. GET /users/:first_name

Let’s now turn our attention to the get /users/:first_name route. We know our endpoint works fine when the user exists. But what happens when it doesn’t?

curl -i http://localhost:4567/users/frank

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 4
... More Headers ...

null

What the heck is this null doing here? Plus, we are telling the client it’s a JSON document with the Content-Type. We really need to fix this!

6.4.1. 404 Not Found

The 404 Not Found status code can help us. This code is meant to tell the client that nothing was found at the URI specified by the client; in other words, the requested resource does not exist.

This fits what we need for the get /users/:first_name route pretty well. Let’s add a little halt that will return 404 if the user is not found in our hash.

# webapi.rb
# Stuff
# ...
get '/users/:first_name' do |first_name|
  halt 404 unless users[first_name.to_sym]

  send_data(json: -> { users[first_name.to_sym].merge(id: first_name) },
            xml:  -> { { first_name => users[first_name.to_sym] } })
end

Restart the server and send the following request.

curl -i http://localhost:4567/users/frank

Output

HTTP/1.1 404 Not Found
... More Headers ...

Looks good!

6.4.2. 410 Gone

Sadly 404 Not Found can seem quite generic to a client. We can use other codes to give a more specific response. One of them is 410 Gone which indicates that a resource used to live at the given URI, but doesn’t anymore. It has probably been deleted since the server does not have the new location.

Currently, if we delete a user and then try to access it, we will just get 404, as if it never existed.

It’s not always possible for a server to indicate a deleted resource and there is nothing forcing us to use this HTTP status. It is helpful for a client though, since it will let the client know that this URI does not point to anything anymore. It shouldn’t be considered as a bug since there used to be something there. The client should take note that it’s now gone and move on with its life.

Let’s quickly add a mechanism to keep track of what has been deleted. In a production application with a real database, it is possible to achieve this kind of mechanism by using a boolean in order to see if something has been deleted or not. There are other ways to do it, but we won’t be covering them.

The first thing we need is a hash, named deleted_users, that will contain all the deleted users (duh!).

# webapi.rb
# require
users = {
  thibault: { first_name: 'Thibault', last_name: 'Denizet', age: 25 },
  simon:    { first_name: 'Simon', last_name: 'Random', age: 26 },
  john:     { first_name: 'John', last_name: 'Smith', age: 28 }
}

deleted_users = {}

# Helpers
# Routes

Then we need to update the get /users/:first_name route to halt with 410 if the deleted_users hash contains the specified user.

# webapi.rb
# stuff
get '/users/:first_name' do |first_name|
  halt 410 if deleted_users[first_name.to_sym]
  halt 404 unless users[first_name.to_sym]

  send_data(json: -> { users[first_name.to_sym].merge(id: first_name) },
            xml:  -> { { first_name => users[first_name.to_sym] } })
end

Finally, we need to fill the deleted_users hash every time a user gets deleted in the delete /users/:first_name route.

# webapi.rb
# stuff
delete '/users/:first_name' do |first_name|
  first_name = first_name.to_sym
  deleted_users[first_name] = users[first_name] if users[first_name]
  users.delete(first_name)
  status 204
end

Everything is in place now and we can start sending some test requests. First, restart the server. Then delete the simon user with this command:

curl -i -X DELETE http://localhost:4567/users/simon

Output

HTTP/1.1 204 No Content
[ ... ]

Now, what happens when we try to access the Simon resource?

curl -i http://localhost:4567/users/simon

HTTP/1.1 410 Gone
Content-Type: text/html;charset=utf-8
Content-Length: 0
[More Headers]

Our little API is getting more expressive by the minute!

6.5. PUT /users/:first_name

For this route, we can handle 415 Unsupported Media Type and 400 Bad Request. However, I’m not going to show you how. Instead, I want you to do it.

Testing 415

curl -X PUT -i http://localhost:4567/users/thibault \
     -H "Content-Type: application/fake" \
     -d 'Weirdly Formatted Data'

Testing 400

curl -X PUT -i http://localhost:4567/users/thibault \
     -H "Content-Type: application/json" \
     -d '{"first_name":"Tibo"'

6.6. PATCH /users/:first_name

For this route, we can handle 415 Unsupported Media Type, 404 Not Found, 410 Gone and 400 Bad Request. You can get inspired by what we did for POST /users/:first_name and GET /users/:first_name. Below, you will find curl commands to test if you have correctly implemented each status code.

Testing 415

curl -X PATCH -i http://localhost:4567/users/thibault \
     -H "Content-Type: application/fake" \
     -d 'Weirdly Formatted Data'

Testing 400

curl -X PATCH -i http://localhost:4567/users/thibault \
     -H "Content-Type: application/json" \
     -d '{"first_name":"Tibo"'

Testing 404

curl -X PATCH -i http://localhost:4567/users/mark \
     -H "Content-Type: application/json" \
     -d '{"first_name":"Marc"}'

Testing 410

curl -i -X DELETE http://localhost:4567/users/simon

curl -X PATCH -i http://localhost:4567/users/simon \
     -H "Content-Type: application/json" \
     -d '{"first_name":"Super Simon"}'

6.7. Wrap Up

In this chapter, we’ve seen how to implement errors to let our clients understand exactly what went wrong when something didn’t go as planned.

In the next chapter, we will tackle a sensitive subject: versioning!

Previous - Next

Master Ruby Web APIs

Chapter 6

Handling Errors