Chapter 1

The Beginning

Welcome to Master Ruby Web APIs!

You are about to learn how to build awesome web APIs using the wonderful Ruby language.

1.1. Introduction

The purpose of this book is to teach you everything about building modern web APIs (Application Programming Interfaces). These days, the web API landscape is very confusing. You hear buzzwords that most people don’t actually understand being thrown everywhere, and a lack of resources explaining what things really are and why they are like that.

RESTful is a good example. Most people don’t understand what it actually means and confuse RESTful APIs with HTTP APIs. The truth is there are almost no RESTful APIs out there; most of them are only HTTP APIs. We will learn about the difference soon.

For now, let’s see exactly how this book is structured and what you will learn from it.

1.1.1. Who is this book for?

This book starts slowly enough for beginners to understand, before switching to more complex topics. The minimum requirement is having some programming background and being able to understand some basic Ruby code.

I believe any developer will have to work on a web API one day, and this experience will be much easier with the knowledge contained in this book. That’s not all. Most of the techniques we will discuss in this book also apply to regular web applications.

Basically, if you are a web developer, this book is for you. I can also assure you that you will learn new stuff.

1.1.2. Requirements

There are not too many requirements for Master Ruby Web APIs. You can actually follow the whole book without writing a line of code, but I would recommend following the flow and building the applications while reading the book.

To build them, you will need a Linux or Unix based system. I provide a lot of commands that can be run easily in a terminal on those platforms and that will make your life simpler.

If you are currently using Windows, I recommend setting up Virtual Box with the Ubuntu Linux distribution.

1.1.3. How is this book organized?

There are 3 modules in this book. Each module is a set of chapters focusing on a common topic. Here are the 3 modules with details for each one of them.

Module 1: Simple Sinatra APIs to learn HTTP

This first module is about two things: HTTP and Sinatra. With those two tools, we will review the basics of HTTP before talking about more advanced topics like caching, versioning, and authentication.

  • Chapter 2 - A simple Sinatra API to learn The Basics
  • Chapter 3 - Understanding HTTP requests with curl
  • Chapter 4 - Media Type fun
  • Chapter 5 - Using HTTP verbs
  • Chapter 6 - Handling Errors
  • Chapter 7 - Versioning
  • Chapter 8 - Caching
  • Chapter 9 - Authentication
  • Chapter 10 - Module Conclusion

Module 2: Alexandria, an E-Commerce Website Selling Digital Books

With everything we learned in the first module, we will start building Alexandria with Rails 5 and the new API mode. We will design it and define our expectations before implementing the code. Finally, we will deploy Alexandria on Heroku.

  • Chapter 11 - Alexandria
  • Chapter 12 - Alexandria: Setup Time
  • Chapter 13 - Adding Books and Authors
  • Chapter 14 - Exposing The Books Resource
  • Chapter 15 - Building Representations
  • Chapter 16 - The First Query Builder: Paginator
  • Chapter 17 - Query Builders
  • Chapter 18 - Representation Builders
  • Chapter 19 - Finishing up our controllers
  • Chapter 20 - Full-Text search
  • Chapter 21 - Client Authentication
  • Chapter 22 - Users
  • Chapter 23 - User Authentication and Authorization
  • Chapter 24 - Selling Books
  • Chapter 25 - Compression, Caching, CORS, Versioning
  • Chapter 26 - Writing documentation
  • Chapter 27 - Deployment
  • Chapter 28 - Module Conclusion

Module 3: REST

In the last module, we will talk about REST and clarify what it is. We will review the REST constraints and analyze why Alexandria is not RESTful. We will also see how we can fix it before taking a look at some modern hypermedia formats.

  • Chapter 29 - REST
  • Chapter 30 - Alexandria and the REST constraints
  • Chapter 31 - Exploring More Hypermedia Formats
  • Chapter 32 - Module Conclusion

1.1.4. How To Read This Book

This book is meant to be read from start to finish. While you could probably cherry-pick some chapters in the modules, you will probably end up confused if you try to follow the code in a similar way.

Note that the best format to read this book on a computer, and build the applications, is the static HTML website. The chapters navigation is simplified, and you will be able to easily copy and paste the code. If you just want to read through, which is totally possible as long as you really try to understand the source code, the PDF/EPUB/MOBI formats will be perfect for you.

1.1.5. Code Blocks

There is a large amount of code in this book. Actually, more like a huge amount. I believe reading code is the best way to learn coding. Don’t worry though, every example comes with enough explanation to understand what’s happening.

Code blocks will look like this:

def code_block
  'This is a code block!'
end

I also use them for commands that are meant to be run in a terminal. I don’t prefix the lines with $ in order to make them easier to copy. You should have no problem differentiating the code blocks from the terminal command blocks.

touch my_new_file.rb

In some occasions, I had to reduce the length of the lines of code in order to fit them in the boxes used in the book. Here is an example:

# Does not fit in PDF version
def something
  response.headers['Location'] = "http://localhost:4567/users/#{user['first_name']}"
end

# Alternative
def something
  url = "http://localhost:4567/users/#{user['first_name']}"
  response.headers['Location'] = url
end

1.1.6. Errors, Bugs, Problems

While I did my best to avoid mistakes, be it in spelling or in the code, you might encounter some of those. If you do, I would really appreciate if you could send an email to bo@samurails.com with the details.

You can also check this GitHub repository and create an issue there.

1.2. A Short History of the Internet

What follows is not always taught in university because it’s not really required to develop anything. Especially if you’ve worked on web applications before, you can easily make the jump to web APIs without overthinking it. The thing is that, when you create web applications, you are following widely-accepted standards without even realizing it - You are coding something that will be run in a browser.

Why do you think the web application you made runs in (almost) the same way on every browser you try? You can thank standards for this. Companies which develop browsers need guidelines about how things should be implemented in order to make your life easier and allow as many people as possible to access your content.

What happens now when you build a web API? Think about it, and we’ll come back to it soon.

Let’s start our theoretical adventure into the wonderful World Wide (more like Wild!) Web and learn more about standards and how everything comes together.

First, we need to understand where we are coming from in order to know where we are going. The Internet has grown at a crazy rate in the past 30 years. Today, everyone has access to it using all kinds of devices from small to big.

Let’s see how it all came together.

1.2.1. The Origin

More than one hundred years ago, the only way people could communicate was through telegraph. Soon enough, the telephone and radio were invented. These media became the links that human beings needed to reveal their potential.

The Intergalactic Computer Network was a computer networking concept imagined in the early 1960s by J.C.R Licklider, who was working at the Pentagon’s ARPA (Advanced Research Projects Agency). It’s basically the same Internet we have today.

From this first concept, the first version of ARPANET (Advanced Research Projects Agency Network) was made. It expanded into a network of networks in the following years before becoming the Internet as we know it.

The fundamental challenge when creating ARPANET was to make computers communicate with each other. It’s quite funny when you think that this is pretty much the focus of this book, but on a completely different level. Thanks to all the progress we have made, we are not doing low-level communication anymore. Today it’s much easier - and thanks to standards, we can use awesome technologies like the HTTP protocol to communicate with pretty much anything that supports it, from home automation devices to smartphones, (without forgetting your computers, of course).

Box 1.1. Tesla and the First Logic Gate

You probably know that the device you are using to read this book is powered by millions of transistors. Widely viewed as the greatest invention of the 20th century, the transistor allows computers to run all the binary operations they need using thousands of ‘AND’ and ‘OR’ logic gate operations. Officially, it was first conceived by Julius Lilienfeld in 1926.

What you may not know is that the first logic gate was actually created by Nikola Tesla in 1898 for the first radio controlled device (a miniature boat) which was named ‘teleautomaton’ by Tesla. It was used to decode and encode Hertzian waves.

Nikola Tesla was truly a man ahead of his time.

1.2.2. The Importance of Documentation (a.k.a Standards)

The Internet is not one technology. It is not a black box that magically runs Google, Facebook, and your blog.

The World Wide Web is not an objective thing. It’s a patchwork of technologies and a set of agreements. More importantly, it is a social construct made by humans for humans.

In making the Web, there are as many abstract discussions as there are practical implementations. That means people need a place to read and comment about how things should be because, let’s be honest, a collective output is stronger than any individual output. Today it seems normal to have public access to all the standards and specifications that make the Internet what it is today.

But 40 years ago, none of that existed. It all started in 1969 with S. Crocker, who created the first RFC (Request For Comments) notes. They were a way for him to share ideas in an informal way with other researchers.

Box 1.2. The First RFC Notes

When those first RFC notes were written, our Internet didn’t exist in the same way yet. FTP (File Transfer Protocol) would soon be used to share the RFCs, but until then, everything was printed on paper and sent via regular mail.

Those RFCs ended up focusing more and more on protocol standards and, thanks to their public accessibility, they were used by engineers and teachers in their work.

Today, web standards are maintained by organizations like the IETF (Internet Engineering Task Force) or the W3C (World Wide Web consortium) which publish them as recommendations. The impact of these standards is tremendous.

Box 1.3. The Core Standards

The Internet relies on three core agreements, or standards, which are present in pretty much every connection: HTTP (RFC 2616), HTML (W3C Recommendation), and ECMA-262 (ECMA standard).

Not only do they provide optimal solutions for us to implement, but they also make it easier to create things! By adhering to those standards, your products will require less work and, for example, be better indexed. It also provides ways to help everyone, including blind people and people with poor eyesight, access the information from any device.

Standards give us accessibility and stability.

1.2.3. Documentation

We’ve seen before how important documentation has helped the Word Wide Web rise to the top. Nowadays, documentation is written by pretty much every developer. Showing how to use a library or a tool is mandatory if you’re looking for people to actually use your stuff.

We can mention Google’s and AWS’s documentation as examples of corporate documentation. I’m sure you’ve also written documentation before, so there shouldn’t be anything new here.

However, there are also more significant groups like the W3C (World Wide Web Consortium) or the IETF that take care of writing another kind of documentation. We have mentioned them before, but I’m obviously talking about web standards. In other words, this is not the standard documentation you write about code or tools, but rather a theoretical approach trying to create the best possible solutions for specific problems. Their creation can span years to allow time for feedback and improvements, and they end up being the “best way to do something” according to the community, the corporations, and the organization drafting them.

You don’t actually need web standards to create stuff for the web, but they make everything simpler. Think about CSS and the custom browser properties you have to use sometimes. What if you had to create three or four stylesheets, one for each browser you support? Luckily, browsers follow standards, which basically make your life much easier.

Following works pretty well for developing websites/web applications, but web APIs are kind of lagging behind. Most people today just create new fiat standards for every project, re-inventing the wheel every time. This also means they need to write a new set of documentation to cover everything about the web API functionalities. Using standards would save some precious time by removing the need to write walls of text to explain how things work, since this wall of text would already exist elsewhere (and presumably, already be understood by the wider community). You would still have to describe your application semantics, but this part could actually be addressed by upgrading to a hypermedia API with embedded documentation. We will learn more about this very soon.

The goal behind the creation of web standards is the long term growth of the World Wide Web (a.k.a WWW).

For now, let’s have a quick overview of how standards work and how to use them.

1.2.4. The Web Standards Organizations

There are a few organizations responsible for creating and maintaining the web standards of the World Wide Web.

The ones we are interested in as software engineers are the W3C, the IETF, and Ecma International. If you love standards, you can also take a look at the ISO, the ANSI and the Unicode Consortium, but we won’t be covering them here.

W3C - World Wide Web Consortium

The W3C is an international organization where companies (Google, Cisco, etc.) work with a full-time staff (408 members) as well as the public in order to develop web standards published as recommendations. It was founded by the creator of the World Wide Web, Tim Berners-Lee. Most of the standards that interest us as web developers come from this institution.

Recommendations examples: HTML, CSS, DOM

IETF - Internet Engineering Task Force

The IETF creates Internet standards, especially standards related to the infrastructure of the Internet. They are published as RFC (Request for Comments) and are called Internet-Drafts before that. All participants in this non-profit organization are volunteers, and there are no membership requirements.

RFC examples: HTTP, DHCP, FTP

Ecma International

ECMA is an international membership-only non-profit organization which aims to “facilitate the timely creation of a wide range of global Information and Communications Technology and Consumer Electronics standards.” One of its standards is the famous ECMAScript (ECMA-262), which gave birth to the JavaScript language.

Standards examples: ECMAScript, NFC

1.2.5. The Different Types of “Standards”

There are different types of standards depending on whom they are targeted at and available for. There are no official names for these different types, and are provided here more to give you an idea of the different categories of standards.

Personal Standards

You are building a web application, or web APIs, and you decide that your JSON documents should look like this:

{
  "businessCard": {
    "firstName": "John",
    "lastName": "Doe",
    "url": "http://example.com"
  }
}

You made an arbitrary choice to present data like this, and you will probably have to write some human-readable documentation for this.

Box 1.4. Personal Standard Documentation

The given JSON document represents a person’s business card.

  • businessCard: top-level attribute containing the business card details.
    • firstName: The person’s first name.
    • lastName: The person’s last name.

That’s what you could call a personal standard. It’s only used by you or your colleagues on a specific project.

Corporate Standards

The corporate standard is one level higher. It is defined inside a corporation to reduce development times for their specific use case.

If we take our previous example, there would be a document defining the standard at the organization level. It could contain the following:

Box 1.5. Documentation

Business cards MUST be represented with a top-level bCard attribute that MUST include the first name of the person as fName and the last name of the person as lName.

MUST, SHOULD and MAY are part of the terminology used to write RFCs.

By following this corporate standard, all representations of business cards in the company’s applications SHOULD look like this:

{
  "bCard": {
    "fName": "John",
    "lName": "Doe",
    "website": "http://example.com"
  }
}

From an individual or small team, we went to the corporate level. What’s next? The world level.

Web Standard

Now we are getting to the interesting part. Instead of letting people or corporations reinvent the wheel, nonprofit organizations work to create global web standards for everyone. Those standards can take years before being published and sometimes require revisions (like HTTP/1.1).

These are the RFC made by the IETF and the recommendations of the W3C. Those standards are meant to go further than just your next application; they are meant to allow people to bring out the full potential of the World Wide Web.

In our previous examples, we were representing a business card using JSON. We are now going to do the same using the JSON-LD standard associated, with the vocabulary defined on schema.org.

{
  "@type": "http://schema.org/Person",
  "@context": {
    "givenName": "http://schema.org/givenName",
    "familyName": "http://schema.org/familyName",
    "homepage": {
      "@id": "http://schema.org/url",
      "@type": "@id"
    }
  },
  "givenName": "John",
  "familyName": "Doe",
  "homepage": "http://example.com"
}

Our JSON document follows a well-known standard and people can easily find out what means what. Even computers can now understand that the home page is a related resource.

Plus, it fixed a flaw in our previous JSON documents; we were using “first name” and “last name”. Those words are mostly targeted at people in the Western hemisphere. In other parts of the world, the family name actually comes first, which makes our use of “first name” nonsensical for users living there.

1.3. Getting Started

Before we start the first module, let’s set up our environment. First, we need to install Ruby. To ensure that everything is correctly installed, we will also build a simple web API with Ramaze.

Installing RVM

I personally like to use RVM (Ruby Version Manager) to handle my Ruby versions. It also allows you to compartmentalize your gems by project, which makes it easier to work on multiple projects with different gem versions.

Open a terminal and use the commands below to install RVM. If you have any problems, check the RVM website for solutions. You will need the gpg command line tool.

gpg --keyserver hkp://keys.gnupg.net \
    --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
\curl -sSL https://get.rvm.io | bash -s stable

If you prefer a more lightweight tool to manage your Ruby versions, take a look at rbenv.

Installing Ruby

We can now use RVM to install Ruby. We will be using the 2.5.0 version in this book, so let’s install it. This can take a while.

rvm install ruby 2.5.0
rvm use 2.5.0

Finally, create a gemset which will contain all the gems we will be using to build the web APIs in this book. The gems we install in this gemset will be isolated from the gems in other gemsets.

rvm gemset create mrwa
rvm use 2.5.0@mrwa --default

Our first API

With Ruby installed, we can start working on our first API. To keep things organized, let’s create a folder to contain all the projects we will build in Master Ruby Web APIs. We also need an introduction/ folder for our first project.

mkdir -p master_ruby_web_apis/chapter_01 && \
  cd master_ruby_web_apis/chapter_01

Time to install Ramaze!

Ramaze is a very simple and straightforward web-framework. Its philosophy can be expressed as a mix of KISS (Keep It Simple, Stupid) and POLS (Principle Of Least Surprise) - trying to make simple things simple and complex things both possible and fun.

Ramaze Website

Install the Ramaze gem.

gem install ramaze --no-ri --no-rdoc

Then, create a file for the code.

touch hello_world.rb

Here is our web API:

# hello_world/hello_world.rb
require 'ramaze'

class HelloWorld < Ramaze::Controller
  map '/hello_world'
  def index
    'Hello World!'
  end
end

Ramaze.start

Start the server with the command below:

ruby hello_world.rb

It’s time to check whether our first web API is working. Open your browser and head over to http://localhost:7000/hello_world. Hello World! See Figure 1 for reference.

https://s3.amazonaws.com/devblast-mrwa-book/images/figures/01/01
Figure 1

Stop the server with CTRL-C.

We built our first web API. It was obviously super simple, because it was a simple Hello World example.

1.4. Wrap Up

The purpose of this chapter was to ease you into the book by presenting what we are going to do. Now, it’s time to get started with the first module and build our first Sinatra APIs.