Adrian Mejia’s Blog

var life = [‘work_smart’, ‘have_fun’, ‘make_history’];

Creating RESTful APIs With NodeJS and MongoDB Tutorial (Part II)

| Comments

Welcome to this RESTful API using Node.js (Express.js) and MongoDB (mongoose) tutorial. You can follow alone to make a stand alone API endpoint, or you could also check out our AngularJS or BackboneJS tutorials to build a javascript-client that connects with the endpoint we are going to built.

Part II: RESTful API with NodeJS and MongoDB

What RESTful API really means?

REST stands for Representational State Transfer. It is an architecture that allows client-server communication through a uniform interface. They are also stateless, cachable and has property called idempotence, which means that the side effect of multiple identical requests have the same effect as the same single request.

HTTP RESTful API’s are compose of:

  • HTTP methods, e.g. GET, PUT, DELETE, PATCH, POST, …
  • Base URI, e.g. http://adrianmejia.com
  • URL path, e.g. /blog/2014/10/01/creating-a-restful-api-tutorial-with-nodejs-and-mongodb/
  • Media type, e.g. html, JSON, XML, Microformats, Atom, Images

Here’s is a summary what we want to implement:

Resource (URI) POST (create) GET (read) PUT (update) DELETE (destroy)
/todos create new task list tasks N/A (update all) N/A (destroy all)
/todos/1 error show task ID 1 update task ID 1 destroy task ID 1

NOTES:

  • Format will be JSON.
  • Bulk updates and bulk destroys are not safe, so we will not be implementing those.
  • POST, GET, PUT, DELETE == CREATE, READ, UPDATE, DELETE == CRUD.

Setup

The two main components of the MEAN stack are… NodeJS and MongoDB.

Note: If already have installed NodeJS, MongoDB (Mongoose), ExpressJS and knows about them separately then you can jump to wiring the stack, which is where the hands on start. Otherwise, if you want to review/learn about each member of the stack then follow alone and enjoy the ride!

NodeJS

For short NodeJS is Javascript running outside the browser, in this case in the server.

To install it, you can go to NodeJS Website. But if you are using Mac and brew you can do brew install nodejs and in ubuntu use nvm to install it. Once you have continue.

Check node version and npm (node package manager) version:

1
2
3
4
5
node -v
# => v0.10.30

npm -v
# => 2.0.0-alpha-5

ExpressJS

ExpressJS is web application framework that runs on NodeJS. Allows you to build web applications and APIs endpoints. (mode details later).

Install it using npm:

1
npm install -g express

Notice -g. It will install express globally and add it the PATH, so you can run it from anywhere.

Check version:

1
2
express -V
# => 4.9.0

MongoDB

MongoDB is a document-oriented NoSQL database (Big Data ready). It stores data in JSON-like format and allows to perform SQL-like queries against it.

You can installed following the instructions here. If you have a Mac and brew it’s just: brew install mongodb && mongod. In ubuntu sudo apt-get -y install mongodb.

Check version:

1
2
3
4
5
6
7
8
9
# Mac
mongod --version
# => db version v2.6.4
# => 2014-10-01T19:07:26.649-0400 git version: nogitversion

# Ubuntu
mongod --version
# => db version v2.0.4, pdfile version 4.5
# => Wed Oct  1 23:06:54 git version: nogitversion

Understanding the MEAN Stack

If follow the previous steps you should have all you need to complete this tutorial. Basically, we are going to build an API that allow users to CRUD (Create-Read-Update-Delete) Todo tasks from database.

Mongoose CRUD

CRUD = Create-Read-Update-Delete

We can play with Mongoose in the console. In the todoApp type node to enter in the node CLI. Then:

1
2
3
4
5
6
7
8
9
10
11
12
/* prompt> */ var mongoose = require('mongoose');

/* prompt> */ mongoose.connect('mongodb://localhost/test3');

/* prompt> */ var TodoSchema = new mongoose.Schema({
  name: String,
  completed: Boolean,
  note: String,
  updated_at: { type: Date, default: Date.now },
});

/* prompt> */ var Todo = mongoose.model('Todo', TodoSchema);

Mongoose Create

1
2
3
4
5
6
7
8
/* prompt> */ var todo = new Todo({name: 'Master NodeJS', completed: false, note: 'Getting there...'});

/* prompt> */ todo.save(function(err){
    if(err)
        console.log(err);
    else
        console.log(todo);
});

You can also build the object and save in one step using create:

1
2
3
4
/* prompt> */ Todo.create({name: 'Master Javscript', completed: true, note: 'Getting better everyday'}, function(err, todo){
    if(err) console.log(err);
    else console.log(todo);
});

Mongoose Read and Query

There are multiple options for reading/querying data:

  • Model.find(conditions, [fields], [options], [callback])
  • Model.findById(id, [fields], [options], [callback])
  • Model.findOne(conditions, [fields], [options], [callback])

Some examples:

Find all
1
2
3
4
/* prompt> */ Todo.find(function (err, todos) {
  if (err) return console.error(err);
  console.log(todos)
});

You can also add queries

Find with queries
1
2
3
4
5
6
7
8
9
10
/* prompt> */ var callback = function (err, data) {
  if (err) return console.error(err);
  else console.log(data);
}

// Get all completed tasks
/* prompt> */ Todo.find({completed: true }, callback);

// Get all tasks ending with `JS`
/* prompt> */ Todo.find({name: /JS$/ }, callback);

You can chain multiple queries, e.g.:

Chaining queries
1
2
3
4
5
6
7
8
/* prompt> */ var oneYearAgo = new Date();
oneYearAgo.setYear(oneYearAgo.getFullYear() - 1);

// Get all tasks staring with `Master`, completed
/* prompt> */ Todo.find({name: /^Master/, completed: true }, callback);

// Get all tasks staring with `Master`, not completed and created from year ago to now...
/* prompt> */ Todo.find({name: /^Master/, completed: false }).where('updated_at').gt(oneYearAgo).exec(callback);

Mongoose Update

Each model has an update method which accepts multiple updates (for batch updates because doesn’t return an array with data). Alternatively, the method findOneAndUpdate could be used to update just one and return an object.

  • Model.update(conditions, update, [options], [callback])
  • Model.findByIdAndUpdate(id, [update], [options], [callback])
  • Model.findOneAndUpdate([conditions], [update], [options], [callback])
Todo.update and Todo.findOneAndUpdate
1
2
3
4
5
6
7
8
9
10
11
// Model.update(conditions, update, [options], [callback])
// update `multi`ple tasks from complete false to true

/* prompt> */ Todo.update({ completed: false }, { completed: true }, { multi: true }, function (err, numberAffected, raw) {
  if (err) return handleError(err);
  console.log('The number of updated documents was %d', numberAffected);
  console.log('The raw response from Mongo was ', raw);
});

//Model.findOneAndUpdate([conditions], [update], [options], [callback])
/* prompt> */ Todo.findOneAndUpdate({name: /JS$/ }, {completed: false}, callback);

Mongoose Delete

update and remove mongoose API are identical, the only difference it is that no elements are returned.

  • Model.remove(conditions, [callback])
  • Model.findByIdAndRemove(id, [options], [callback])
  • Model.findOneAndRemove(conditions, [options], [callback])

ExpressJS and Middlewares

ExpressJS is a complete web framework solution. It has HTML template solutions (jade, ejs, handlebars, hogan.js) and CSS precompilers (less, stylus, compass). Through middlewares layers, it handles: cookies, sessions, caching, CSRF, compression and many more.

Middlewares are a stack of processors that runs on each request made to the server. You can have any number of middlewares that will process the request one by one in a serial fashion. Some might alter the request input, log outputs, add data and pass it to the next() middleware in the chain.

Middlewares are added to ExpressJS stack using app.use for any method or the app.VERB (e.g., app.get, app.delete, app.post, app.update, …)

Let’s say you want to log the IP of the client on each request:

Log the client IP on every request
1
2
3
4
5
app.use(function (req, res, next) {
  var ip = req.headers['x-forwarded-for'] || req.connection.remoteAddress;
  console.log('Client IP:', ip);
  next();
});

You can also specify a path that you want the middleware to activate on.

Middleware mounted on “/todos/:id” and log the request method
1
2
3
4
app.use('/todos/:id', function (req, res, next) {
  console.log('Request Type:', req.method);
  next();
});

And finally you can use app.get to catch GET request with maching route, reply the request with a response.send and end the middleware chain. Let’s use what we learned on mongoose read to reply with a user data matching the id.

Middleware mounted on “/todos/:id” and returns
1
2
3
4
5
6
app.get('/todos/:id', function (req, res, next) {
  Todo.findById(req.params.id, function(err, todo){
    if(err) res.send(err);
    res.json(todo);
  });
});

Notice that all previous middlewares called next() except this last one, because it sends a respond (in JSON) to the client with the requeste todo data.

Hopefully, you don’t have to develop a bunch of middlewares besides routes, since ExpressJS has a bunch of middlewares available.

Default Express 4.0 middlewares

  • morgan: logger

  • body-parser: parse the body so you can access parameters in requests in req.body. e.g. req.body.name.

  • cookie-parser: parse the cookies so you can access parameters in cookies req.cookies. e.g. req.cookies.name.

  • serve-favicon: exactly that, serve favicon from route /favicon.ico. Should be call on the top before any other routing/middleware takes place to avoids unnecessary parsing.

Other ExpressJS Middlewares

The following middlewares are not added by default, but It’s nice to know they exist at least:

  • compression: compress all request. e.g. app.use(compression())

  • session: create sessions. e.g. app.use(session({secret: 'Secr3t'}))

  • method-override: app.use(methodOverride('_method')) Override methods to the one specified on the _method param. e.g. GET /resource/1?_method=DELETE will become DELETE /resource/1.

  • response-time: app.use(responseTime()) adds X-Response-Time header to responses.

  • errorhandler: Aid development, by sending full error stack traces to the client when an error occurs. app.use(errorhandler()). It is good practice to surround it with an if statement to check process.env.NODE_ENV === 'development'.

  • vhost: Allows you to use different stack of middlewares depending on the request hostname. e.g. app.use(vhost('*.user.local', userapp)) and app.use(vhost('assets-*.example.com', staticapp)) where userapp and staticapp are different express instances with different middlewares.

  • csurf: Adds a Cross-site request forgery (CSRF) protection by adding a token to responds either via session or cookie-parser middleware. app.use(csrf());

  • timeout: halt execution if it takes more that a given time. e.g. app.use(timeout('5s'));. However you need to check by yourself under every request with a middleware that checks if (!req.timedout) next();.

API clients (Browser, Postman and curl)

I know you have not created any route yet. However, in the next sections you will. These are just three ways to retrieve, change and delete data from your future API.

Curl

Create tasks
1
2
3
4
5
# Create task
curl -XPOST http://localhost:3000/todos -d 'name=Master%20Routes&completed=false&note=soon...'

# List tasks
curl -XGET http://localhost:3000/todos

Browser and Postman

If you open your browser and type localhost:3000/todos you will see all the tasks (when you implement it). However, you cannot do post commands by default. For further testing let’s use a Chrome plugin called Postman. It allows you to use all the HTTP VERBS easily and check x-www-form-urlencoded for adding parameters.

Websites and Mobile Apps

Probably this is the main consumers of the APIs. You can interact with RESTful APIs using jQuery’s $ajax and its wrappers, BackboneJS’s Collections/models, AngularJS’s $http or $resource, among many other libraries/frameworks and mobile clients.

In the end, we are going to explain how to use AngularJS to interact with this API.

Image from CodeSchool

Wiring up the MEAN Stack

Bootstrapping ExpressJS

After a detour in Node CLI, MongoDB, Mongoose, tools and middlewares land we are back to our express todoApp. This time to create the routes and finalize our RESTful API.

Create the app typing express -e todoApp, install dependencies cd todoApp && npm install and run the app DEBUG=todoApp ./bin/www:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
express -e todoApp

# =>   create : todoApp                  # app directory  
# =>   create : todoApp/package.json     # file containing all the dependencies
# =>   create : todoApp/app.js           # Entry point of the application: defines middleware, initialize database connections, routes and more.
# =>   create : todoApp/public           # all files contained here are accessible through to public (browser or API calls).
# =>   create : todoApp/public/javascripts
# =>   create : todoApp/public/images
# =>   create : todoApp/public/stylesheets
# =>   create : todoApp/public/stylesheets/style.css
# =>   create : todoApp/routes           # containes all the routes files
# =>   create : todoApp/routes/index.js
# =>   create : todoApp/routes/users.js
# =>   create : todoApp/views            # contains all the HTML templates
# =>   create : todoApp/views/index.ejs
# =>   create : todoApp/views/error.ejs
# =>   create : todoApp/bin              # contains executable files
# =>   create : todoApp/bin/www          # bootstrap the app: loads app.js, and set the port for the webserver.
# =>
# =>   install dependencies:
# =>     $ cd todoApp && npm install
# =>
# =>   run the app:
# =>     $ DEBUG=todoApp ./bin/www

Connect ExpressJS to MongoDB

Hopefully, you have installed MongoDB in the setup section, and you can start it typing:

1
mongod

Install the MongoDB driver for NodeJS called mongoose:

1
npm install mongoose --save

Notice --save. It will add it to the todoApp/package.json

Next, you need to require mongoose in the todoApp/app.js

1
2
3
4
5
6
7
8
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/todoApp', function(err) {
    if(err) {
        console.log('connection error', err);
    } else {
        console.log('connection successful');
    }
});

Now, When you run npm start or ./bin/www, you will notice the message connection successful. Did you? Great!

You can find the repository here and the diff code at this point: diff

Creating the Todo model with Mongoose

It is show time! All the above was setup and preparation for this moment. Let bring the API to life.

Create a models directory and a Todo.js model:

1
2
mkdir models
touch models/Todo.js

In the models/Todo.js:

1
2
3
4
5
6
7
8
9
10
var mongoose = require('mongoose');

var TodoSchema = new mongoose.Schema({
  name: String,
  completed: Boolean,
  note: String,
  updated_at: { type: Date, default: Date.now },
});

module.exports = mongoose.model('Todo', TodoSchema);

diff

What’s going up there? Isn’t MongoDB suppose to be schemaless? Well, it is schemaless and very flexible indeed. However, very often we want bring sanity to our API/WebApp through validations and enforcing a schema to keep a consistent structure. Mongoose does that for us, which is nice.

You can use the following types:

  • String
  • Boolean
  • Date
  • Array
  • Number
  • ObjectId
  • Mixed
  • Buffer

ExpressJS Routes

To sum up we want to achieve the following:

Resource (URI) POST (create) GET (read) PUT (update) DELETE (destroy)
/todos create new task list tasks error error
/todos/:id error show task :id update task :id destroy task ID 1

Let’s setup the routes

Create a new route called `todos.js` in the `routes` folder or rename `users.js`
1
mv routes/users.js routes/todos.js

In app.js add new todos route or just replace ./routes/users for ./routes/todos

Adding todos routes
1
2
var todos = require('./routes/todos');
app.use('/todos', todos);

All set! Now, let’s go back and edit our routes/todos.js.

List: GET /todos

Remember mongoose query api? Here’s how to use it in this context:

routes/todos.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
var express = require('express');
var router = express.Router();

var mongoose = require('mongoose');
var Todo = require('../models/Todo.js');

/* GET /todos listing. */
router.get('/', function(req, res, next) {
  Todo.find(function (err, todos) {
    if (err) return next(err);
    res.json(todos);
  });
});

module.exports = router;

Harvest time! We don’t have any task in database but at least we verify it is working:

Testing all together
1
2
3
4
5
6
7
8
9
# Start database
mongod

# Start Webserver (in other terminal tab)
DEBUG=todoApp ./bin/www

# Test API (in other terminal tab)
curl localhost:3000/todos
# => []%

diff

If it returns an empty array [] you are all set. If you get errors, try going back and making sure you didn’t forget anything or write a comment at the end of the post for help.

Create: POST /todos

Back in routes/todos.js, we are going to add the ability to create using mongoose create. Could you make it work before looking at the next example?

routes/todos.js (showing just create route)
1
2
3
4
5
6
7
/* POST /todos */
router.post('/', function(req, res, next) {
  Todo.create(req.body, function (err, post) {
    if (err) return next(err);
    res.json(post);
  });
});

diff

Few things:

  • We are using the router.post instead of router.get.
  • You have to stop and run the server again: DEBUG=todoApp ./bin/www. From now on, use nodemon to refresh automatically. npm install nodemon and then run nodemon.

Show: GET /todos/:id

This is a snap with Todo.findById and req.params. Notice that params matches the placeholder name we set while defining the route. :id in this case.

routes/todos.js (showing just show route)
1
2
3
4
5
6
7
/* GET /todos/id */
router.get('/:id', function(req, res, next) {
  Todo.findById(req.params.id, function (err, post) {
    if (err) return next(err);
    res.json(post);
  });
});

diff

Test it in POSTMAN using an _id from you created elements. E.g. localhost:3000/todos/542d7d290a705126360ac635.

Update: PUT /todos/:id

Back in routes/todos.js, we are going to update tasks. This one you can do it before looking at the example bellow, review findByIdAndUpdate and give it a try!

routes/todos.js (showing just update route)
1
2
3
4
5
6
7
/* PUT /todos/:id */
router.put('/:id', function(req, res, next) {
  Todo.findByIdAndUpdate(req.params.id, req.body, function (err, post) {
    if (err) return next(err);
    res.json(post);
  });
});

diff

Test it in POSTMAN :)

Destroy: DELETE /todos/:id

Finally, the last one! Almost identical to update, use findByIdAndRemove.

routes/todos.js (showing just update route)
1
2
3
4
5
6
7
/* DELETE /todos/:id */
router.delete('/:id', function(req, res, next) {
  Todo.findByIdAndRemove(req.params.id, req.body, function (err, post) {
    if (err) return next(err);
    res.json(post);
  });
});

diff

Is it working? Cool, you are done then! Is NOT working? take a look at the full repository.

What’s next?

Connecting AngularJS with this endpoint:

Related tutorials

Comments