{ "blyg": "0.4", "id": "3w871j05tx0qwner9c9f7z4dhf", "kind": "thread", "origin": "https://notionparallax.co.uk/", "page": "https://notionparallax.co.uk/2016/mongo", "author": { "name": "Ben Doherty", "url": "https://notionparallax.co.uk/" }, "created": "2016-05-06T20:25:59Z", "updated": "2016-05-06T20:25:59Z", "version": 1, "content_md": "# Another shoddy Mongo/Node/Express tutorial to add to the pile\n\n\nI've never actually set up my own Mongo database before, I've always been the least useful person on the team. I thought I'd give it a go and document it.\n\nI've never actually set up any sort of database before! To use the idea developed [here](http://notionparallax.co.uk/2016/progress-enabling-more-progress) this is like a dam that's been holding me back for years. Now I can surge ahead like a boss. I've always found writing tutorials was a good way to learn, so if I've made a horrific mistake let me know.\n\n[Here's a repo](https://github.com/notionparallax/SimpleMongNode) with some working code and [here's the working app](https://infinite-castle-45974.herokuapp.com/).\n\nIt's on Heroku because they seem to have the lowest friction to getting going. Once you've got your head around how Mongo and node actually work then you can get your hear around how to configure an AWS thingo.\n\nOne thing to think about is that you don't put data directly into the database. That could get messy. You can think about it like a library where the librarian puts the books back for you. We use some code running on the same server as the database to act as our librarian.\n\n## Getting all the pieces in place\n\nThe steps to get a mongo/node instance up and running so that you can play with it.\n\n### Get a Heroku account\n\nThe reason for picking Heroku is that it's got very little friction to getting started. You can use whatever you want.\n\nDo [this tutorial](https://devcenter.heroku.com/articles/getting-started-with-nodejs##introduction) up until the 'Provision a database' step\n\n_except_ in **Prepare the app** when it asks you to clone the repo, clone this repo instead.\n\nThe following steps are from [here](https://devcenter.heroku.com/articles/mongolab##adding-mlab-as-a-heroku-add-on):\n\nadd a new database:\n\n $ heroku addons:create mongolab\n\nThen find its connection uri:\n\n $ heroku config | grep MONGODB_URI\n\nAt the bottom of your `index.js` change this line to be your URI (i.e. not mine):\n\n // Connect to Mongo on start\n var mongoURL = 'mongodb://heroku_d58bh1wm:m3f37fh3q0sodg1lct1vbb97s0@ds063899.mlab.com:63899/heroku_d58bh1wm'; // <-- but put in YOUR one from the step above\n\nThen git commit and `git push heroku master`\n\nYour app _should_ be working now!\n\n## Putting things in\n\nDatabases aren't much use if there isn't anything in them. Let's throw some junk into Mongo.\n\n### URL params\n\nThe simplest way to put data into the database is with URL params. Here's an example:\n\n[`/in-url-params?rssi=50&base=home`](https://infinite-castle-45974.herokuapp.com/in-url-params?rssi=50&base=home)\n\nThe route is the `/in-url-params` bit, and the params are everything after the `?`. They are in `key=value` pairs, so in `rssi=50` the word `rssi` is the key and `50` is the value. They are separated by an `&` and you can have something like 1000 characters of these before the browser gets grumpy with you.\n\nOn the server side, you need to intercept the route. That's done by this bit of code:\n\n```js\napp.get(\"/in-url-params\", function (req, res) {\n //assuming a url that looks like /in-url-params?rssi=50&base=home\n var rssi = req.query.rssi || \"unknown\";\n var base = req.query.base || \"unknown\";\n console.log(\"rssi: \", rssi);\n console.log(\"base: \", base);\n\n // in here you'd probably do something smart to\n // make sure that the data isn't bullshit or malicious\n\n var collection = db.get().collection(\"beaconData\");\n collection.insert({ rssi: rssi, base: base }, function (err, result) {\n console.log(\"that seemed to work\", result);\n res.send(result);\n });\n});\n```\n\nBreaking that down, the first line says: `app.get('/in-url-params'` when someone sends a request to that route (something that starts with `/in-url-params`) then do this function `function(req, res) {`. `req` and `res` mean request and response. _Request_ is what is being sent to the server, _response_ is what it sends back.\n\n`var rssi = req.query.rssi || 'unknown';` does two things. It's declaring a variable (duh). Then `req.query.rssi` gets the `query` out of the request, then tries to get the `rssi` property from it. There's a bit of magic happening around the back because the url params are being turned into properties of the query. The tricky thing here is that if that fails it assigns the string _unknown_ to the `rssi` variable. This might happen if the URL sent was `/in-url-params?arseeye=50&base=home` then the `.rssi` propery would be `undefined`. In js, undefined is falsy, so it'd trip the second half of the conditional assignment.\n\nEverything up to the comment in the middle should make sense now. It's just getting the data out of the request. The comment _in here you'd probably do something smart to make sure that the data isn't bullshit or malicious_ is pretty self explanatory. An example of that would be checking that the date is in the past (no future dates) or that the rssi was a number, or that the base was on the list of legit bases.\n\nThe next bit is actually putting the data into the database.\n\n`var collection = db.get().collection('beaconData')` tells you what collection we're about to insert into.\n\n`collection.insert({rssi: rssi, base: base},` actually does the insert. The rest deals with what to do next. This is because we dont' want everything to hang while we wait, so we say _do this, and when you're done, do this_. The bit that gets done after is this:\n\n```js\nfunction(err, result){\n console.log('that seemed to work', result);\n res.send(result);\n});\n```\n\n`err` and `result` are bit confusing. Only one of them will ever be defined. If there's an error then we can write a bit of code that looks like `if(err){...}` to handle it. In this case we're being optimistic and just assuming that it'll work.\n\nIn the first line of that function we're logging to the console. It's important to note that the console is _on the server_. The second line sends a response to the client (the client is the browser that sends the request). Because the server and the client are usually in different places you would do both so that both parties know what's going on. This is about the simplest example of putting something into a database I can make up.\n\n### POSTing JSON\n\nA more useful and flexible way to send JSON in the body of the request. The URL would look nice and simple e.g. `/in-json` and all the data are carried as a payload.\n\n#### Sending\n\nThis is a very simple python script that sends some data to the server. In the repo it's called `render.py` and has a bit more detail in the JSON to make the point, but really it's the same.\n\n```python\nimport requests\nimport simplejson as json\n\nurl = \"https://infinite-castle-45974.herokuapp.com/in-json\"\ndata = {'some': 'json'}\nheaders = {'Content-type': 'application/json'}\nr = requests.post(url, data=json.dumps(data), headers=headers)\nprint r\n```\n\nSomething to note is that neither `requests` or `simplejson` is a built in module. That means that the script will fail if you don't have them installed. Use `$ sudo pip install requests simplejson` if you have pip installed, or\non OSX you can use `sudo easy_install -U requests simplejson` if you have easy_install installed. (I ripped most of this [from here](http://stackoverflow.com/questions/17309288/importerror-no-module-named-requests).)\n\n#### Receiving\n\nThis took a bit longer to get working, but is actually a bit simpler once you are over the initial hump.\n\nYou need to install the [body-parser](https://www.npmjs.com/package/body-parser) node module by:\n\n`npm install body-parser -s`\n\nThere are a few lines at the top of `index.js` that are there to make body-parser work, but you can safely ignore them for now. This lets us get directly to `req.body`.\n\n```js\napp.post(\"/in-json\", function (req, res) {\n console.log(req.body);\n\n var collection = db.get().collection(\"beaconData\");\n collection.insert(\n req.body, //req.body is already JSON\n function (err, result) {\n console.log(\"that seemed to work\", result);\n res.send(result);\n }\n );\n});\n```\n\nHere's the low-down on what's going on in this function. We're \\_post_ing data this time instead of \\_get_ing it. Don't worry about [the distinction](http://stackoverflow.com/questions/3477333/what-is-the-difference-between-post-and-get) for now. The rest is just a cut down version of `/in-url-params`.\n\nBecause `req.body` is json we can actually throw it straight into the database. (Of course this is a terrible idea because it might be all kinds of junk data, but that's a problem for another day.)\n\n## And taking things out\n\n[`/out-url-params-base?base=home`](https://infinite-castle-45974.herokuapp.com/out-url-params-base?base=home) this gets it back out again.\n\nAlso I've tried to privilege readability over good software engineering.\n", "content_html": "

Another shoddy Mongo/<wbr>Node/<wbr>Express tutorial to add to the pile

\n

I’ve never actually set up my own Mongo database before, I’ve always been the least useful person on the team. I thought I’d give it a go and document it.

\n\n

I’ve never actually set up any sort of database before! To use the idea developed here this is like a dam that’s been holding me back for years. Now I can surge ahead like a boss. I’ve always found writing tutorials was a good way to learn, so if I’ve made a horrific mistake let me know.

\n\n

Here’s a repo with some working code and here’s the working app.

\n\n

It’s on Heroku because they seem to have the lowest friction to getting going. Once you’ve got your head around how Mongo and node actually work then you can get your hear around how to configure an AWS thingo.

\n\n

One thing to think about is that you don’t put data directly into the database. That could get messy. You can think about it like a library where the librarian puts the books back for you. We use some code running on the same server as the database to act as our librarian.

\n\n

Getting all the pieces in place

\n\n

The steps to get a mongo/node instance up and running so that you can play with it.

\n\n

Get a Heroku account

\n\n

The reason for picking Heroku is that it’s got very little friction to getting started. You can use whatever you want.

\n\n

Do this tutorial up until the ‘Provision a database’ step

\n\n

except in Prepare the app when it asks you to clone the repo, clone this repo instead.

\n\n

The following steps are from here:

\n\n

add a new database:

\n\n
$ heroku addons:create mongolab\n
\n\n

Then find its connection uri:

\n\n
$  heroku config | grep MONGODB_URI\n
\n\n

At the bottom of your index.js change this line to be your URI (i.e. not mine):

\n\n
// Connect to Mongo on start\nvar mongoURL = 'mongodb://heroku_d58bh1wm:m3f37fh3q0sodg1lct1vbb97s0@ds063899.mlab.com:63899/heroku_d58bh1wm'; // <-- but put in YOUR one from the step above\n
\n\n

Then git commit and git push heroku master

\n\n

Your app should be working now!

\n\n

Putting things in

\n\n

Databases aren’t much use if there isn’t anything in them. Let’s throw some junk into Mongo.

\n\n

URL params

\n\n

The simplest way to put data into the database is with URL params. Here’s an example:

\n\n

/in-url-params?rssi=50&base=home

\n\n

The route is the /in-url-params bit, and the params are everything after the ?. They are in key=value pairs, so in rssi=50 the word rssi is the key and 50 is the value. They are separated by an & and you can have something like 1000 characters of these before the browser gets grumpy with you.

\n\n

On the server side, you need to intercept the route. That’s done by this bit of code:

\n\n
app.get(\"/in-url-params\", function (req, res) {\n    //assuming a url that looks like /in-url-params?rssi=50&base=home\n    var rssi = req.query.rssi || \"unknown\";\n    var base = req.query.base || \"unknown\";\n    console.log(\"rssi: \", rssi);\n    console.log(\"base: \", base);\n\n    // in here you'd probably do something smart to\n    // make sure that the data isn't bullshit or malicious\n\n    var collection = db.get().collection(\"beaconData\");\n    collection.insert({ rssi: rssi, base: base }, function (err, result) {\n        console.log(\"that seemed to work\", result);\n        res.send(result);\n    });\n});\n
\n\n

Breaking that down, the first line says: app.get('/in-url-params' when someone sends a request to that route (something that starts with /in-url-params) then do this function function(req, res) {. req and res mean request and response. Request is what is being sent to the server, response is what it sends back.

\n\n

var rssi = req.query.rssi || 'unknown'; does two things. It’s declaring a variable (duh). Then req.query.rssi gets the query out of the request, then tries to get the rssi property from it. There’s a bit of magic happening around the back because the url params are being turned into properties of the query. The tricky thing here is that if that fails it assigns the string unknown to the rssi variable. This might happen if the URL sent was /in-url-params?arseeye=50&base=home then the .rssi propery would be undefined. In js, undefined is falsy, so it’d trip the second half of the conditional assignment.

\n\n

Everything up to the comment in the middle should make sense now. It’s just getting the data out of the request. The comment in here you’d probably do something smart to make sure that the data isn’t bullshit or malicious is pretty self explanatory. An example of that would be checking that the date is in the past (no future dates) or that the rssi was a number, or that the base was on the list of legit bases.

\n\n

The next bit is actually putting the data into the database.

\n\n

var collection = db.get().collection('beaconData') tells you what collection we’re about to insert into.

\n\n

collection.insert({rssi: rssi, base: base}, actually does the insert. The rest deals with what to do next. This is because we dont’ want everything to hang while we wait, so we say do this, and when you’re done, do this. The bit that gets done after is this:

\n\n
function(err, result){\n  console.log('that seemed to work', result);\n  res.send(result);\n});\n
\n\n

err and result are bit confusing. Only one of them will ever be defined. If there’s an error then we can write a bit of code that looks like if(err){...} to handle it. In this case we’re being optimistic and just assuming that it’ll work.

\n\n

In the first line of that function we’re logging to the console. It’s important to note that the console is on the server. The second line sends a response to the client (the client is the browser that sends the request). Because the server and the client are usually in different places you would do both so that both parties know what’s going on. This is about the simplest example of putting something into a database I can make up.

\n\n

POSTing JSON

\n\n

A more useful and flexible way to send JSON in the body of the request. The URL would look nice and simple e.g. /in-json and all the data are carried as a payload.

\n\n

Sending

\n\n

This is a very simple python script that sends some data to the server. In the repo it’s called render.py and has a bit more detail in the JSON to make the point, but really it’s the same.

\n\n
import requests\nimport simplejson as json\n\nurl = \"https://infinite-castle-45974.herokuapp.com/in-json\"\ndata = {'some': 'json'}\nheaders = {'Content-type': 'application/json'}\nr = requests.post(url, data=json.dumps(data), headers=headers)\nprint r\n
\n\n

Something to note is that neither requests or simplejson is a built in module. That means that the script will fail if you don’t have them installed. Use $ sudo pip install requests simplejson if you have pip installed, or\non OSX you can use sudo easy_install -U requests simplejson if you have easy_install installed. (I ripped most of this from here.)

\n\n

Receiving

\n\n

This took a bit longer to get working, but is actually a bit simpler once you are over the initial hump.

\n\n

You need to install the body-parser node module by:

\n\n

npm install body-parser -s

\n\n

There are a few lines at the top of index.js that are there to make body-parser work, but you can safely ignore them for now. This lets us get directly to req.body.

\n\n
app.post(\"/in-json\", function (req, res) {\n    console.log(req.body);\n\n    var collection = db.get().collection(\"beaconData\");\n    collection.insert(\n        req.body, //req.body is already JSON\n        function (err, result) {\n            console.log(\"that seemed to work\", result);\n            res.send(result);\n        }\n    );\n});\n
\n\n

Here’s the low-down on what’s going on in this function. We’re _post_ing data this time instead of _get_ing it. Don’t worry about the distinction for now. The rest is just a cut down version of /in-url-params.

\n\n

Because req.body is json we can actually throw it straight into the database. (Of course this is a terrible idea because it might be all kinds of junk data, but that’s a problem for another day.)

\n\n

And taking things out

\n\n

/out-url-params-base?base=home this gets it back out again.

\n\n

Also I’ve tried to privilege readability over good software engineering.

\n", "content_hash": "sha256:1068e3d0c135a587cc96042aab349a115f5a393c595688cf1e27f4fd6e60a05f", "media": [], "changelog": [ { "version": 1, "at": "2016-05-06T20:25:59Z", "note": null } ], "transclusions": [] }