Setting Up Nginx, uWSGI, and Django on Amazon's EC2

A Supplement to the Official uWSGI Tutorial for Novices that Holds Your Hand and Whispers Soothing Words
by Oliver; 2014
                       

Introduction

I wanted to set up Django on my EC2 instance to create dynamic websites. The reason? Mostly ignorance. I haven't used any of this stuff before, but I liked the fact that (1) Django has a large user base; and (2) it's written in Python, a full-fledged language with applications not limited to the web. Because I have zero experience with Nginx, uWSGI, Django, or system administration, you may benefit from this step by step tutorial.

Finding a Tutorial

There are a million blogs that purport to show you how to do this, but by far the best one I found was a non-amateur guide on the uWSGI website (which is also recommend in the Django docs). It's the gold-standard of how-tos for this task: This page is a supplement to the above guide, which I'm going to follow exactly, filling in details where I stumbled or had questions. Remember: I have no experience with this stuff, so I just plodded ahead in a literal-minded fashion.

Jumping in

I'll go quickly because most of the nitty-gritty is on the tutorial's site. As they note, the end goal is this:

the web client <-> the web server (nginx) <-> the socket <-> uwsgi <-> Django

The first step is to install virtualenv to create a safe virtual environment which keeps whatever Python packages you install there safely isolated from the rest of your system. Let's get this, and some other packages, installed on your system:

$ sudo yum install python-pip
$ sudo yum install python-devel
$ sudo yum install python-setuptools
$ sudo pip install virtualenv
$ sudo pip install uwsgi

Good.

Next step: make a directory for your project, via virtualenv, and start virtualenv:

$ virtualenv --no-site-packages proj1     # proj1 is our quasi root directory
$ cd proj1 
$ source bin/activate                     # start virtualenv
$ easy_install Django                     # install Django
$ bin/django-admin.py startproject site1  # start a new Django project, site1
$ # deactivate                            # (FYI, stop virtualenv if you need to) 

Inside proj1/, our directory structure looks like this:

site1
├── manage.py
├── site1
    ├── __init__.py
    ├── __init__.pyc
    ├── settings.py
    ├── settings.pyc
    ├── urls.py
    ├── urls.pyc
    ├── wsgi.py
    └── wsgi.pyc

The name of our outer directory, site1/ (where manage.py resides), doesn't matter. However, the name of our inner directory, site1, is super important because this is the name of our Python module. I apologize for nomenclatural confusion caused by nesting two directories with same name.

Testing Django, Testing uwsgi, Testing Them Together

Before proceeding, you need to open up a port for Django or uWSGI to use. Go to your AWS Console: and to: and click on the security group associated with your instance. Click the Inbound tab and allow traffic from all IP addresses (0.0.0.0/0) for port 8000 (or any other free port you fancy). If you don't do this, permissions won't be enabled and your whole effort will capsize. Now try this:

$ cd site1
$ manage.py runserver 0.0.0.0:8000 

And you should see your site—let's call it mysite.com—on http://www.mysite.com:8000/, confirming Django's working:


As the tutorial notes, now try using uwsgi by making a script, test.py:

# test.py
def application(env, start_response):
    start_response('200 OK', [('Content-Type','text/html')])
    return "Hello World"

and running:

uwsgi --http :8000 --wsgi-file test.py

You should see a "Hello World" message on http://www.mysite.com:8000/.

Next, try getting uwsgi talking to Django:

uwsgi --http :8000 --module site1.wsgi

Pitfall: This command won't work unless it's run from the directory where manage.py resides (in our example: proj1/site1/). If this is not the case, you'll get an error like this:

ImportError: No module named mysite.wsgi

Running Nginx

Now that that's working, we're going to start nginx. Let's try starting it as follows:

$ sudo /etc/init.d/nginx start

Go to your URL, www.mysite.com, and you should see the following:


The page you're seeing resides, on my system, at Let's stop it now as follows:

$ sudo /etc/init.d/nginx stop

Nginx is taking cues from its configuration file, which we can look at:

$ cat /etc/nginx/nginx.conf

Configuration files have their own rules. You can read about nginx's here or here. In any case, let's point out a couple of simple features:

user  nginx;
error_log  /var/log/nginx/error.log;
http {
	include /etc/nginx/conf.d/*.conf;
	# other stuff
}

Each of these lines tells us something useful. The first one says that nginx's worker processes will run with a user id of nginx. The second one says that any errors will be logged in /var/log/nginx/error.log—an invaluable piece of information for troubleshooting. And the third thing to note is that we can include configuration blocks, or files, by using the include directive.

Let's follow the tutorial's lead and make a configuration block specially for our site:

$ sudo mkdir /etc/nginx/sites-enabled                   # make this dir
$ sudo touch /etc/nginx/sites-enabled/site1_nginx.conf  # make this file

Into /etc/nginx/sites-enabled/site1_nginx.conf paste this:

# mysite_nginx.conf

# the upstream component nginx needs to connect to
upstream django {
    server unix:///path/to/proj1/site1/site1.socket; # for a file socket
    # server 127.0.0.1:8001; # for a web port socket (we'll use this first)
}

# configuration of the server
server {
    # the port your site will be served on
    listen      8000;
    # the domain name it will serve for
    # substitute your machine's IP address or FQDN
    server_name .mysite.com; 
    charset     utf-8;

    # max upload size
    client_max_body_size 75M;   # adjust to taste

    # Django media
    location /media  {
	# your Django project's media files - amend as required
	alias /path/to/site1/site1/media;  
    }

    location /static {
	# your Django project's static files - amend as required
	alias /path/to/site1/site1/static; 
    }

    # Finally, send all non-media requests to the Django server.
    location / {
        uwsgi_pass  django;
	# the uwsgi_params file you installed
	include     /path/to/site1/uwsgi_params; 
    }
}

If your Django project is still inchoate, as mine is, we can make some of these directories by hand. In proj1/site1 (where manage.py lives) make the directories and files alluded to in the config file:

$ mkdir site1/media 
$ mkdir site1/static
$ touch uwsgi_params 

In uwsgi_params, write this:

uwsgi_param  QUERY_STRING       $query_string;
uwsgi_param  REQUEST_METHOD     $request_method;
uwsgi_param  CONTENT_TYPE       $content_type;
uwsgi_param  CONTENT_LENGTH     $content_length;

uwsgi_param  REQUEST_URI        $request_uri;
uwsgi_param  PATH_INFO          $document_uri;
uwsgi_param  DOCUMENT_ROOT      $document_root;
uwsgi_param  SERVER_PROTOCOL    $server_protocol;
uwsgi_param  HTTPS              $https if_not_empty;

uwsgi_param  REMOTE_ADDR        $remote_addr;
uwsgi_param  REMOTE_PORT        $remote_port;
uwsgi_param  SERVER_PORT        $server_port;
uwsgi_param  SERVER_NAME        $server_name;

(or just copy the file /etc/nginx/uwsgi_params, which should be identical to this, into the directory)

The final bit is to make sure the main config file, /etc/nginx/nginx.conf, knows about you site's configuration file, /etc/nginx/sites-enabled/site1_nginx.conf. To do this, add an include statement in the main configuration file so it looks like this:

user  nginx;
error_log  /var/log/nginx/error.log;
http {
	include /etc/nginx/conf.d/*.conf;
	# add this:
	include /etc/nginx/sites-enabled/*.conf;
	# other stuff
}

Pitfall: If you just paste the config file from the tutorial and rename it to be /etc/nginx/nginx.conf, you'll get this error when you try to start nginx:

$ sudo /etc/init.d/nginx start
Starting nginx: 
nginx: [emerg] "upstream" directive is not allowed here in /etc/nginx/nginx.conf:4
                                                           [FAILED]

The tutorial recommends placing an image file in the site1/media directory to see if everything's going as it should, and we can at least serve static files.

Tip: Here's how to rsync to your account (via this Stackoverflow post):

rsync -azv --progress -e "ssh -i your_amazon_key.pem.txt" ~/Desktop/corgi.gif \
your-user-name@your-amazon-address:~/path/to/site1/site1/media

Put an image file in the directory—I used a Corgi gif—and start nginx. And now, at http://www.mysite.com:8000/media/corgi.gif, you should see the glory:


If the website is down, which it very likely will be, it's probably on account of a permission issue which we'll tackle below. (Image source: Everyone's being an asshole today, so here's some pixel art of a swimming corgi.)

Getting it Up

We're almost there. Let's start nginx:

$ sudo /etc/init.d/nginx start

Let's start uwsgi, as described in the tutorial. From within the site1/ directory (where manage.py resides), do this:

$ # the less permissive way:
$ uwsgi --socket site1.socket --module site1.wsgi --chmod-socket=664 
$ # the more permissive way:
$ uwsgi --socket site1.socket --module site1.wsgi --chmod-socket=666 

In both cases, I went to http://www.mysite.com:8000 and got:


The good news was that ngnix was indeed serving pages, but what went wrong? I consulted the logs at /var/log/nginx/error.log and found an error that looked something like this:

2013/12/10 01:01:23 [crit] 24332#0: *8 connect() to 
unix:///path/to/site1/site1.socket failed (13: Permission denied) 
while connecting to upstream

Permission Issues! They're ubiquitous, but at least they're a known quantity and we should be able to fix them. One thing to note off the bat is that nginx's worker process is a different user than you are. E.g.:

$ ps -Af | head -1; ps -Af | grep nginx
UID      PID   PPID   C STIME TTY   TIME     CMD
root     29921     1  0 22:02 ?     00:00:00 nginx: master process /usr/sbin/nginx \
		 		               		-c /etc/nginx/nginx.conf
nginx    29923 29921  0 22:02 ?     00:00:00 nginx: worker process                   
ec2-user 29938 19605  0 22:43 pts/0 00:00:00 grep --color nginx

Hence, if your files are only readable or writable by you, you may have a problem. Where is your proj1/ directory? Mine was in my home and, after checking the permission of each nested directory—e.g.:

$ ls -hl /some/long/path
$ ls -hl /some/long
$ ls -hl /some

I discovered, of course, that my home directory had permission, so only my user could do anything with the files in there. I fixed this courtesy of: and that got the following working:

$ uwsgi --socket site1.socket --module site1.wsgi --chmod-socket=666

I wanted to get the less permissive command working, too. To do that, I granted group permissions and added the user nginx to my group. On your system, check your own user name and the groups you belong to. In my case it was:

$ whoami
ec2-user
$ groups
ec2-user wheel

To add the user nginx to the group ec2-user, I looked up the sysadmin command to add a user to a group:

$ sudo usermod -a -G ec2-user nginx

After you've done this, check group assignments with the command id:

$ id nginx
uid=220(nginx) gid=498(nginx) groups=498(nginx),500(ec2-user)

Now stop nginx and start it again. The command:

$ uwsgi --socket site1.socket --module site1.wsgi --chmod-socket=664

should work now, and once again you should see:


Score!

Following the tutorial, we can put the parameters for uwsgi into a file called uwsgi.ini:

[uwsgi]

# ----- Django-related settings -----

# the base directory (full path)
chdir=/path/to/site1

# Django's wsgi file
module=site1.wsgi

# the virtualenv (full path)
home=/path/to/proj1

# pidfile=/path/to/site1.pid

# max-requests=5000

# ----- process-related settings -----

# master
master=true

# maximum number of worker processes
processes=1

# the socket (use the full path to be safe)
socket=/path/to/site1/site1.socket

# ... with appropriate permissions - may be needed
chmod-socket=664

# clear environment on exit
vacuum=true

See the tutorial for more sophisticated things to do with this file and other details. We can daemonize it, run it with nohup, and so on. For now, I'm just going to be super lazy and run it in the background, so we can move on to actually building our site with Django:

$ uwsgi --ini uwsgi.ini > log/site1.uwsgi.o 2> log/site1.uwsgi.e &