Another shoddy Mongo/Node/Express tutorial to add to the pile
I’ve never actually set up my own Mongo database before, I’ve always been the least useful person on the team. I thought I’d give it a go and document it.
I’ve never actually set up any sort of database before! To use the idea developed here this is like a dam that’s been holding me back for years. Now I can surge ahead like a boss. I’ve always found writing tutorials was a good way to learn, so if I’ve made a horrific mistake let me know.
Here’s a repo with some working code and here’s the working app.
It’s on Heroku because they seem to have the lowest friction to getting going. Once you’ve got your head around how Mongo and node actually work then you can get your hear around how to configure an AWS thingo.
One thing to think about is that you don’t put data directly into the database. That could get messy. You can think about it like a library where the librarian puts the books back for you. We use some code running on the same server as the database to act as our librarian.
Getting all the pieces in place
The steps to get a mongo/node instance up and running so that you can play with it.
Get a Heroku account
The reason for picking Heroku is that it’s got very little friction to getting started. You can use whatever you want.
Do this tutorial up until the ‘Provision a database’ step
except in Prepare the app when it asks you to clone the repo, clone this repo instead.
The following steps are from here:
add a new database:
$ heroku addons:create mongolab
Then find its connection uri:
$ heroku config | grep MONGODB_URI
At the bottom of your index.js
change this line to be your URI (i.e. not mine):
// Connect to Mongo on start
var mongoURL = 'mongodb://heroku_d58bh1wm:[email protected]:63899/heroku_d58bh1wm'; // <-- but put in YOUR one from the step above
Then git commit and git push heroku master
Your app should be working now!
Putting things in
Databases aren’t much use if there isn’t anything in them. Let’s throw some junk into Mongo.
URL params
The simplest way to put data into the database is with URL params. Here’s an example:
/in-url-params?rssi=50&base=home
The route is the /in-url-params
bit, and the params are everything after the ?
. They are in key=value
pairs, so in rssi=50
the word rssi
is the key and 50
is the value. They are separated by an &
and you can have something like 1000 characters of these before the browser gets grumpy with you.
On the server side, you need to intercept the route. That’s done by this bit of code:
app.get("/in-url-params", function (req, res) {
//assuming a url that looks like /in-url-params?rssi=50&base=home
var rssi = req.query.rssi || "unknown";
var base = req.query.base || "unknown";
console.log("rssi: ", rssi);
console.log("base: ", base);
// in here you'd probably do something smart to
// make sure that the data isn't bullshit or malicious
var collection = db.get().collection("beaconData");
collection.insert({ rssi: rssi, base: base }, function (err, result) {
console.log("that seemed to work", result);
res.send(result);
});
});
Breaking that down, the first line says: app.get('/in-url-params'
when someone sends a request to that route (something that starts with /in-url-params
) then do this function function(req, res) {
. req
and res
mean request and response. Request is what is being sent to the server, response is what it sends back.
var rssi = req.query.rssi || 'unknown';
does two things. It’s declaring a variable (duh). Then req.query.rssi
gets the query
out of the request, then tries to get the rssi
property from it. There’s a bit of magic happening around the back because the url params are being turned into properties of the query. The tricky thing here is that if that fails it assigns the string unknown to the rssi
variable. This might happen if the URL sent was /in-url-params?arseeye=50&base=home
then the .rssi
propery would be undefined
. In js, undefined is falsy, so it’d trip the second half of the conditional assignment.
Everything up to the comment in the middle should make sense now. It’s just getting the data out of the request. The comment in here you’d probably do something smart to make sure that the data isn’t bullshit or malicious is pretty self explanatory. An example of that would be checking that the date is in the past (no future dates) or that the rssi was a number, or that the base was on the list of legit bases.
The next bit is actually putting the data into the database.
var collection = db.get().collection('beaconData')
tells you what collection we’re about to insert into.
collection.insert({rssi: rssi, base: base},
actually does the insert. The rest deals with what to do next. This is because we dont’ want everything to hang while we wait, so we say do this, and when you’re done, do this. The bit that gets done after is this:
function(err, result){
console.log('that seemed to work', result);
res.send(result);
});
err
and result
are bit confusing. Only one of them will ever be defined. If there’s an error then we can write a bit of code that looks like if(err){...}
to handle it. In this case we’re being optimistic and just assuming that it’ll work.
In the first line of that function we’re logging to the console. It’s important to note that the console is on the server. The second line sends a response to the client (the client is the browser that sends the request). Because the server and the client are usually in different places you would do both so that both parties know what’s going on. This is about the simplest example of putting something into a database I can make up.
POSTing JSON
A more useful and flexible way to send JSON in the body of the request. The URL would look nice and simple e.g. /in-json
and all the data are carried as a payload.
Sending
This is a very simple python script that sends some data to the server. In the repo it’s called render.py
and has a bit more detail in the JSON to make the point, but really it’s the same.
import requests
import simplejson as json
url = "https://infinite-castle-45974.herokuapp.com/in-json"
data = {'some': 'json'}
headers = {'Content-type': 'application/json'}
r = requests.post(url, data=json.dumps(data), headers=headers)
print r
Something to note is that neither requests
or simplejson
is a built in module. That means that the script will fail if you don’t have them installed. Use $ sudo pip install requests simplejson
if you have pip installed, or
on OSX you can use sudo easy_install -U requests simplejson
if you have easy_install installed. (I ripped most of this from here.)
Receiving
This took a bit longer to get working, but is actually a bit simpler once you are over the initial hump.
You need to install the body-parser node module by:
npm install body-parser -s
There are a few lines at the top of index.js
that are there to make body-parser work, but you can safely ignore them for now. This lets us get directly to req.body
.
app.post("/in-json", function (req, res) {
console.log(req.body);
var collection = db.get().collection("beaconData");
collection.insert(
req.body, //req.body is already JSON
function (err, result) {
console.log("that seemed to work", result);
res.send(result);
}
);
});
Here’s the low-down on what’s going on in this function. We’re _post_ing data this time instead of _get_ing it. Don’t worry about the distinction for now. The rest is just a cut down version of /in-url-params
.
Because req.body
is json we can actually throw it straight into the database. (Of course this is a terrible idea because it might be all kinds of junk data, but that’s a problem for another day.)
And taking things out
/out-url-params-base?base=home
this gets it back out again.
Also I’ve tried to privilege readability over good software engineering.