Systems Status

Announce RSS Feed RSS

Blog RSS Feed RSS

Help Topics

Deschutes Facilities

Campus Facilities

User Account Info

Roundcube Mail

Contact Systems

ipv6 ready

Introduction

These are step-by-step instructions (a howto document) for creating a MongoDB database on our shared departmental server ix. Most MongoDB tutorials I have found assume the database will be created by a user with 'root' privileges on a dedicated machine. These instructions are for setting up MongoDB for ordinary (student and faculty) users of the shared departmental server, using the mongoctl software provided by UO CIS systems staff.

Background: What's the issue?

MongoDB is a widely used document database, convenient particularly for web apps because its data model (as well as its programming model) is based on JavaScript. This makes it quite different from MySQL, Oracle, and other relational databases whose data model is tables (relations) and whose programming model is the SQL query language.

Python has a pymongo module for accessing MongoDB databases. This makes it very convenient for a web application using Flask. Although MongoDB is designed for very high volume, high performance databases, it will work just fine for very small databases for our class projects.

Most of the difficulty you will encounter is in setting up the MongoDB database. Good tutorials are available, but they mostly assume that you are setting up MongoDB on a dedicated machine on which you are "super-user", otherwise known as "root". A typical instruction (from https://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/) instructs you to type

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927

That might work for your personal computer or virtual machine running Linux, but it isn't going to work on a shared server like ix.cs.uoregon.edu. Fortunately UO CIS systems staff has already installed MongoDB. It is set up in a way that allows each regular user on ix create their own MongoDB server process, and to be the administrator of that MongoDB server. That's what you need to do.

Install MongoDB

On ix, use the program mongoctl to configure your own MongoDB server. (mongoctl is a script written by UO CIS systems staff.)

When you run mongoctl install, mongoctl will configure a MongoDB server for you, and as part of the installation it will create an administrative user account in your MongoDB database. The administrative user account name will be the same as your user name on ix. You will need to provide a password for the administrative user. Do not use your ix password or any other password that you use for more sensitive accounts. Because of the way we will be using MongoDB, the password will be relatively easy to steal, so make it a password that you don't mind losing.

$ mongoctl install
This script is designed to setup an individual ngoDB server
Database password (DO NOT use your unix password):
Verify password:
The latest information about MongoDB is available on the web at http://docs.mongodb.org/v2.4 .

At this point there is one user, with the provided password, in the 'admin' database. There is also an etc/mongodb.conf file under your home directory giving process id, etc. Most importantly, your mongodb.conf file says what port Mongo will attempt to use:

dbpath=/home/faculty/michal/mongodb
port=9999

This port will be used when MongoDB is started via mongoctl. The first time mongoctl starts mongodb, it will create the master user:

$ mongoctl start
about to fork child process, waiting until server is ready for connections.
forked process: 21270
all output going to: /home/faculty/michal/mongodb/mongodb.log
child process started successfully, parent exiting
Started mongod on port 9999
MongoDB shell version: 2.4.9
connecting to: 127.0.0.1:9999/test
switched to db admin
{
        "user" : "michal",
        "pwd" : "f6f736f36d2104da987a6316b5699db2",
        "roles" : [
                "userAdminAnyDatabase",
                "readWriteAnyDatabase",
                "dbAdminAnyDatabase"
        ],
        "_id" : ObjectId("5536abb232b7867b16f82278")
}
bye

Now mongod is running in the background. You can give commands to the server process through a command shell called mongo. Start the mongo command shell, specifying the port (9999 in this example) and authorizing as the site administrator. Substitute your own MongoDB port number for 9999.

$ mongo --port 9999 -u michal -p myPassword admin
MongoDB shell version: 2.4.9
connecting to: 127.0.0.1:9999/admin
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
        http://docs.mongodb.org/
Questions? Try the support group
        http://groups.google.com/group/mongodb-user
> db.system.users.find()
{ "_id" : ObjectId("5536abb232b7867b16f82278"), "user" : "michal",
"pwd" : "f6f736f36d2104da987a6316b5699db2", "roles" : [
"userAdminAnyDatabase",  "readWriteAnyDatabase",  "dbAdminAnyDatabase"
] }

Note the password is stored as a hash.

Now we can quit the shell, leaving the server running in the background:

> quit()

Start and Stop the Database Server

The MongoDB server process is called mongod. To start a database server process, use the mongoctl start command.

$ mongoctl start
about to fork child process, waiting until server is ready for connections.
forked process: 23999
all output going to: /home/faculty/michal/mongodb/mongodb.log
child process started successfully, parent exiting
Started mongod on port 9999

It is ok to leave the server process running for weeks at a time, although sometimes it will die because the whole server has been rebooted. To check on your mongod process, use mongoctl status:

$ mongoctl status
mongod (pid 23999) listening on ix-trusty:9999

To stop the server process, use mongoctl stop:

$ mongoctl stop
Stopping mongod process 23999

If we restart the server too soon after shutting it down, the port will be busy and it will choose another. If we have waited several minutes, it will be able to reclaim it's configured port.

$ mongoctl start
about to fork child process, waiting until server is ready for connections.
forked process: 23832
all output going to: /home/faculty/michal/mongodb/mongodb.log
child process started successfully, parent exiting
Started mongod on port 9999

Create a database

MongoDB maintains a set of named databases, and within each database a set of named collections. In general you create one database for an application, and one collection for each set similar data in your project. The distinction between "databases" and "collections" is mostly in how permissions are granted and managed. You can create multiple user accounts in MongoDB, and each user account can have access to one or more databases. A user account with, say, ReadWrite access to database Foobars would have ReadWrite access to all the collections in Foobars.

The admin database is always present. We can inspect it using the mongo shell:

$ mongo --port 9999 -u michal -p imNotTelling admin
MongoDB shell version: 2.4.9
connecting to: 127.0.0.1:9999/admin

> use admin
switched to db admin

> db.system.users.find()
{ "_id" : ObjectId("5536abb232b7867b16f82278"), "user" : "michal", "pwd" : "f6f736f36d2104da987a6316b5699db2", "roles" : [  "userAdminAnyDatabase",  "readWriteAnyDatabase",  "dbAdminAnyDatabase" ] }

In this database, I have already created a database for GPS tracks, with a user (actually a program) that I have called 'tracker'. I can switch to that database and view the record of the user:

> use tracks
switched to db tracks
> db.system.users.find()
{ "_id" : ObjectId("5536aec06120e30bb54c3459"), "user" : "tracker", "pwd" : "d339d2ed360bdec659ca232a0e095141", "roles" : [  "readWrite" ] }
>

Databases are created implicitly by creating users that can write to them. Here I want to create a database of reindeer, by creating a user who can insert reindeer into the database:

$ mongo --port 9999 -u michal -p notMyPasswd admin
MongoDB shell version: 2.4.9
connecting to: 127.0.0.1:9999/admin
>  use reindeer
switched to db reindeer
>  db.addUser( {user: "santa", pwd: "hohoho", roles: ["readWrite"]})
{
        "user" : "santa",
        "pwd" : "47c1a34088fb2afe6331f4c125b1225a",
        "roles" : [
                "readWrite"
        ],
        "_id" : ObjectId("56b50fd0537861f1109e8784")
}
>  

Note that capitalization matters in mongo commands. I frequently mis-type commands by not using camelCase, like this:

>  db.adduser( {user: "santa", pwd: "hohoho", roles: ["readWrite"]})
Fri Feb  5 13:14:51.502 TypeError: Property 'adduser' of object reindeer is not a function

If you get an error message like this, check your spelling and capitalization.

Read and write database records in Python

MongoDB records are BSON structures, which are essentially the same as JSON. The pymongo module (which you will need to install in your virtual environment) manages communication with the MongoDB server process and also converts automatically between Python dicts and MongoDB BSON objects.

First, set include a URL for the MongoDB database in your configuration file:

MONGO_URL =  "mongodb://santa:hohoho@localhost:9999/reindeer"

Notice that the URL uses the mongodb protocol instead of http; it also includes the port (9999 in this case) and the database you wish to access, as well as the user and password (santa:hohoho in this case). Keeping the password in the configuration file, in plain text like this, is why it is important that you do not use your ix password or any other important password.

Now you can write a simple Python program that inserts three reindeer into your MongoDB database and then selects the brown reindeer for printing:

"""
Read and write reindeer
"""
from pymongo import MongoClient
import CONFIG
try:
    dbclient = MongoClient(CONFIG.MONGO_URL)
    db = dbclient.reindeer
    collection = db.flying

except:
    print("Failure opening database.  Is Mongo running? Correct password?")
    sys.exit(1)


collection.insert( { "name": "Dasher", "color": "Brown" } )
collection.insert( { "name": "Prancer", "color": "White" } )
collection.insert( { "name": "Dancer", "color": "Brown" } )

for record in collection.find( { "color": "Brown" } ):
    print(record)

Running this program shows that in addition to the attributes “name” and “color“ that I specified in the code, MongoDB has created a unique object identifier “_id” for each record:

$ python3 db_test.py
{'_id': ObjectId('56b5166eb282c754865f68eb'), 'name': 'Dasher', 'color': 'Brown'}
{'_id': ObjectId('56b5166eb282c754865f68ed'), 'name': 'Dancer', 'color': 'Brown'}
Edited: March 10, 2016, at 03:38 PM
Copyright © 2017, University of Oregon, All rights reserved
Privacy Policy