Notes: Making a Simple Website with Flask

Notes for a First Flask-based Website
by Oliver; June 6, 2015
   
web
 

Introduction

Notes on making a simple lab homepage in Flask.

Project:
(very simple) print lab publications and lab member info from a database onto the web.

Stack:

image

server → nginxuWSGIFlask + SQLiteBootstrap + HTML CSS JS

In these notes, I assume the nginx and uWSGI part has already been taken care of.

Code on Github:
labsite

Finished product:
rabadan.c2b2.columbia.edu

nginx

"Nginx is a web server with a strong focus on high concurrency, performance and low memory usage."
---Wikipedia

Overtaking rival Apache in popularity

“nginx consists of modules which are controlled by directives specified in the configuration file.”
---official docs

See a sample configuration file for the project here.

Web Frameworks

A web framework is used to serve dynamic webpages. It's a bridge between the land of old-school static HTML and the land of programming. The "MVC (model, view, controller)" pattern is an oft-heard buzzword.

Some popular frameworks: Practically, Flask
  • has machinery to hook up to a database
  • allows you to use web templates
  • allows you to program in python

HTML Review

Fancy frameworks aside, do you know H.T.M.L. ("How To Meet Ladies")? Here's a 2 minute review. HTML is a markup language, which means you put tags around text to mark it up, giving it meta attributes like italicization. An example:
<p>This line is within paragraph tags.</p>

<p><i>This</i> is italic.</p>

<p><b>This</b> is bold.</p>

<h1>This is a header</h1>

<p><a href="http://www.nytimes.com">This is a link to The NY Times</a>.</p>
This yields:

This line is within paragraph tags.

This is italic.

This is bold.

This is a header

This is a link to The NY Times.

The larger structure of an HTML page is as follows:
<!DOCTYPE html>

<html>
        <head>
        </head>

        <body>
        </body>
</html>
Most HTML documents have a head and a body. The head has page attributes like title and is where you can link to CSS and JS files. The body is where you code your page proper.

Two useful tags are:
<div>
<span>
because they do little (<div> adds a newline; <span> does nothing) and thus serve as tabulae rasae which can be bent to your purpose via CSS. How does this work? Tags can have class and id:
<div id=“myid” class=“myclass”>
which means that:
<div class="text1">We can style this text one way</div>
<div class="text2">and style this text another way</div>
with a CSS file like:
.text1
{
        /* my styles for any element of class == text1 */
        font:72px bold arial,sans-serif;
        padding: 15px;
}

.text2
{
        /* my styles for any element of class == text2 */
        font:10px arial,sans-serif;
        padding: 5px;
        color: blue;
        border:1px solid rgb(200,200,200);
        background: yellow;
}
The dot in .text1 denotes that "text1" is a class. If it were an id, we'd use a hash tag: #text1

Database

Before you even start to think about the web, think about defining your data models as tables in your database. If you're modeling a user, for example, what attributes will it have? Perhaps it will have a name, email, age, gender, etc. Here's my schema.sql. I have three tables: one for lab members, one for publications, and one for press. Some of the attributes are a little sloppy and have slightly misleading names, but the point is to get the ball rolling.
drop table if exists people;
create table people (
  id integer primary key autoincrement,
  iscurrent integer not null,
  name text not null,
  title text not null,
  bio text not null,
  email text,
  imagefile text,
  webpage text
);

drop table if exists publications;
create table publications (
  id integer primary key autoincrement,
  ishightlight integer,
  year integer not null,
  title text not null,
  authors text not null,
  journal text not null,
  journal2 text,
  doi text,
  doi2 text,
  authors_first text,
  authors_corresponding text,
  journal_url text,
  journal_url2 text,
  notes text
);

drop table if exists press;
create table press (
  id integer primary key autoincrement,
  year integer not null,
  mytext text not null,
  title text,
  journal_url text
);
For this project, I had all the data already, so the next step was to load it into the database.

See the Python docs for how to script SQLite in Python.

Starting Out

Follow the Flask tutorial.

My main flask script, the (very small) brain of the site, is called entry.wsgi and looks like this:
#!/usr/bin/env python

from flask import Flask, render_template

application = Flask(__name__)
application.debug = True

@application.route('/')
def index():
    return render_template('home.html')

@application.route('/publications')
def pubs():
    return render_template('publications.html')

@application.route('/press')
def press():
    return render_template('press.html')

@application.route('/people')
def people():
    return render_template('people.html')

@application.route('/alumni')
def alum():
    return render_template('alumni.html')

@application.route('/formembers')
def formembers():
    return render_template('formembers.html')

@application.route('/courses')
def courses():
    return render_template('courses.html')

@application.route('/contact')
def contact():
    return render_template('contact.html')

if __name__ == "__main__":
    application.run()
This is the URL routing. If the URL for the site is:
https://my-base-url
then going to:
https://my-base-url/publications
will be invoke the function pubs(). For now, this function doesn't do much, but it will get more interesting.

Flask uses the awesome template engine Jinja—think html plus a bit of logic (loops, variables, conditional statements) and inheritance (so you can, e.g., propagate your page's header and footer). My pages inherit from layout.html, which started as:
<!doctype html>
<html>

  <!-- Head --> 
  <head>
    {% block head %}

    <link rel="stylesheet" href="{{ url_for('static', filename='style.css') }}">

    <title>Rabadan Lab | {% block title %}{% endblock %}</title>

    {% endblock %}
  </head>

  <!-- Body --> 
  <body>

    <div id="container">

      <!-- Header --> 
      <div id="header">
        {% block header %}
        {% endblock %}
      </div>

      <!-- Content --> 

      <div id="content">
        {% block content %}
        {% endblock %}
      </div>

      <!-- Footer --> 
      <div id="footer">
        {% block footer %}
        &copy; 2015 The Rabadan Lab 
        {% endblock %}
      </div>

    </div>

  </body>

</html>
The homepage, home.html, was:
{% extends "layout.html" %}

{% block title %}Home{% endblock %}

<!-- Header -->
{% block header %}

  <h2>Rabadan Homepage</h2>

{% endblock %}

<!-- Content -->
{% block content %}

  <i>Welcome on my awesome homepage.</i>
  <br>
  <br>

{% endblock %}
Thus far the homepage looked like this:

image

Adding a bit of CSS turned it into this:

image

Directory Structure

The directory structure of the site looks something like this (going only one level down):
labsite
├── README.md
├── dbs
│   ├── labsite.db
│   └── schema.sql
├── entry.wsgi
├── scripts
│   ├── load_db
│   ├── load_db_2
│   ├── load_db_3
│   └── make_schema
├── static
│   ├── bootstrap-3.3.4-dist
│   ├── css
│   └── js
└── templates
    ├── contact.html
    ├── courses.html
    ├── error.html
    ├── home.html
    ├── layout.html
    ├── people.html
    ├── press.html
    ├── publications.html
    ├── research.html
    └── software.html
  • dbs contains the sqlite database, labsite.db
  • entry.wsgi is the main script, which will read the db and render the templates
  • scripts contains helper scripts (in my case, to parse text files and load data into the db)
  • static contains the folders css and js and any external css/js packages I've downloaded (e.g., bootstrap)
  • templates contains the Jinja html templates

Reading from the Database

Following the Flask docs, the next step was to try reading from the database and rendering it. To get publications out of the database, entry.wsgi became:
import os
import sqlite3
from flask import Flask, request, session, g, redirect, url_for, abort, \
     render_template, flash
from contextlib import closing

# configuration
DATABASE=os.path.join(os.getcwd(), 'labsite/dbs/labsite.db')
# Never leave debug mode activated in a production system, because it will allow users to execute code on the server!
DEBUG = True
SECRET_KEY = 'development key'
USERNAME = 'admin'
PASSWORD = 'default'

# create our application
application = Flask(__name__)
application.config.from_object(__name__)

def connect_db():
    """Connects to the specific database."""
    return sqlite3.connect(application.config['DATABASE'])

@application.before_request
def before_request():
    # without the try/except block, nginx will implode if there's an exception
    try:
        g.db = connect_db()
    except:
	print("Doh! Error connecting to db")

@application.teardown_request
def teardown_request(exception):
    try:
        db = getattr(g, 'db', None)
        if db is not None:
            db.close()
    except:
        print("Doh! Error closing db")

@application.route('/')
def index():
    return render_template('home.html')

@application.route('/publications')
def pubs():
    cur = g.db.execute('select title, authors from publications order by id desc')
    entries = [dict(title=row[0], authors=row[1]) for row in cur.fetchall()]
    return render_template('publications.html', entries=entries)

if __name__ == "__main__":
    application.run()
And the publications template became:
{% extends "layout.html" %}

{% block title %}Publications{% endblock %}

<!-- Header -->
{% block header %}

  <h2>Rabadan Homepage</h2>

{% endblock %}

<!-- Content -->
{% block content %}

  <i>Publications</i>
  <br>
  <br>
  {% for entry in entries %}
    <b>{{ entry.title }}</b>
    <br>
    <small>{{ entry.authors|safe }}</small>
  {% else %}
    <em>Unbelievable.  No entries here so far</em>
  {% endfor %}

  <br>
  <br>

{% endblock %}
entries is the magic variable. In python terms, this is a list of dicts, where each dict contains all the attributes of the publication—e.g.:
[ {authors:'Mathias Klein, Raul Rabadan', title:'Orientifolds with discrete torsion'}, {authors:'J Park, R Rabadan, A M Uranga', title:'N=1 type IIA brane configurations, chirality and T duality'}, {authors:'J Park, R Rabadan, A M Uranga', title:'Orientifolding the conifold'}, ... ]
This produced:

image

And thus the first shoddy bridge between the database and the web was built!

Bootstrap

I decided to do the front end in Bootstrap and the front page with the carousel template. I don't think I would make this decision again because I don't like ceding control to Bootstrap—when something goes wrong, it's much harder to fix than if you wrote it yourself. Moreover, it's "heavy", loading a ton of code even though I only used a tiny bit of it (mainly the navbar). On the other hand, it can be convenient. The first stab at Bootstrap looked like this:

image

The publication section became:

image

Static Content

To serve static content—images, css and js files—you don't need a web-development framework whose raison d'etre is to serve dynamic content. The Flask docs say, "To generate URLs for static files, use the special 'static' endpoint name:"
url_for('static', filename='style.css')
However, it's more efficient to bypass the machinery of Flask entirely and put something like this in your ngnix config file:
# wsgi entry point
location / {
  include uwsgi_params;
  uwsgi_param UWSGI_APPID entry;
  uwsgi_param UWSGI_FILE  entry.wsgi;
  uwsgi_pass   unix:///var/run/uwsgi/example.edu;
}
# static files
location /static {
  alias /somepath/example.edu/html;
}
So anything you put in the special static path will get served without invoking all the overhead of Flask. E.g., to link the CSS file style.css in the <head> of your HTML document, you'd say something like this:
<link rel="stylesheet" href="/static/css/style.css">
rather than:
<link rel="stylesheet" href="{{ url_for('static', filename='style.css') }}">

Graphic Design

At some point, your page becomes an art problem, too. Find an artistic friend or prepare to wax artistic. For image manipulation, photoshop is indispensable. If you're working with batches of images on the command line, check out ImageMagick.

Here's a picture I took to make the graphic representing publications you saw above:

image

Here's a picture I took to make a "whiteboard scrawl" background:

image

My boss sent me this picture of "fancy math":

image

I photoshopped them together, plus some other pictures, into a nice muted white:

image

For a potential logo, I photoshopped some DNA into the Columbia Lion's mouth:

image

This was vetoed via democratic process, however, and we chose a flu virus logo designed by a former coworker, Antony:

image

Heading over to Favicon & App Icon Generator, I used this as the favicon, too:

image

For garnish, I salvaged a doodle drawing of virions I'd done some years ago:

image

More Front End Stuff

With a little more tweaking, the front page became:

image

I added in a javascript package qTip2, for speech bubble tips, and Hover.css, for some Hover effects. Again, there's a trade off between adding a lot of code for a few small effects and writing something yourself which gets the job done with precision and no waste. And I threw the "whiteboard scrawl" picture into the background:

image

My labmates gave me two good suggestions: (1) the homepage needed a welcome message, and (2) having research highlights on the front was too busy. Incorporating these and a bit more fiddling yielded:

image

I think the graffiti look is cool :)

Check out the homepage template here. As mentioned before, the homepage (along with every other page) inherits from the parent layout.html. The styling uses a combination of Bootstrap's CSS classes plus this simple custom.css.

Adding Complexity

In the publication section, I wanted to group the publications by year and highlight notable papers. In the people section, I wanted to group people by title (e.g., Postdocs, Doctoral Students, Staff, etc.) as well as distinguish between current employees and alumni. Doing this added complexity to entry.wsgi:
#!/usr/bin/env python

"""
    Lab Website
    ~~~~~~
    A first Flask + sqlite3 project - making a homepage for the lab,
    borrowing from Armin Ronacher's https://github.com/mitsuhiko/flask/tree/master/examples/flaskr
"""

import os
import sqlite3
from flask import Flask, request, session, g, redirect, url_for, abort, \
     render_template, flash
from contextlib import closing
from datetime import date

# configuration
# DATABASE = '/tmp/flaskr.db'
DATABASE=os.path.join(os.getcwd(), 'labsite/dbs/labsite.db')
# Never leave debug mode activated in a production system, because it will allow users to execute code on the server!
DEBUG = False
SECRET_KEY = 'development key'
USERNAME = 'admin'
PASSWORD = 'default'

# print statements will show up in the uWSGI logs
# print("debug")
# print(DATABASE)

# create our application
application = Flask(__name__)
application.config.from_object(__name__)
# if set an environment variable called FLASKR_SETTINGS to specify a config file 
# to be loaded which will then override the default values
# application.config.from_envvar('FLASKR_SETTINGS', silent=True)

# --- db functions --- #

def connect_db():
    """Connects to the specific database."""
    return sqlite3.connect(application.config['DATABASE'])

@application.before_request
def before_request():
    # without the try/except block, nginx will implode if there's an exception
    try:
        g.db = connect_db()
    except:
	print("Doh! Error connecting to db")

@application.teardown_request
def teardown_request(exception):
    try:
        db = getattr(g, 'db', None)
        if db is not None:
            db.close()
    except:
        print("Doh! Error closing db")

def get_people():
    """get people entries from the db"""
    cur = g.db.execute('select iscurrent, name, title, bio, email, imagefile, webpage from people order by id')
    return [dict(iscurrent=row[0], name=row[1], title=row[2], bio=row[3], email=row[4], imagefile=row[5], webpage=row[6]) for row in cur.fetchall()]

def get_pubs():
    """get publication entries from the db"""
    cur = g.db.execute('select id, year, title, authors, journal, journal2, doi, doi2, journal_url, journal_url2, notes, ishightlight from publications order by id')
    return [dict(id=row[0], year=row[1], title=row[2], authors=row[3], journal=row[4], journal2=row[5], doi=row[6], doi2=row[7], journal_url=row[8], journal_url2=row[9], notes=row[10], highlight=row[11]) for row in cur.fetchall()]

def get_press():
    """get press entries from the db"""
    cur = g.db.execute('select id, year, mytext, journal_url from press order by id')
    return [dict(id=row[0], year=row[1], mytext=row[2], journal_url=row[3]) for row in cur.fetchall()]

# --- URL routing --- #

@application.route('/')
def index():
    return render_template('home.html')

@application.route('/publications/')
@application.route('/publications/<int:myyear>')
def pubs(myyear=0):
    # db query
    entries = get_pubs()

    # default template is publications.html
    mytemplate='publications.html'

    # initialize 
    entries_year = []

    # if a year in the appropriate range is supplied
    if (myyear >= 1999 and myyear <= date.today().year):
        # entries_year is a list of entry dicts for a specific year
        entries_year = filter(lambda y: y['year'] == myyear, entries)
	# use a different template (a little bit of a DRY violation but oh well)
        mytemplate='publications_by_year.html'
    # else get complete publication list for all years
    else:
        # entries_year is a list of lists of entry dicts, segregated by year
        for i in reversed(range(1999,date.today().year + 1)):
            entries_year.append(filter(lambda y: y['year'] == i, entries))

    try:
        return render_template(mytemplate, mydata=entries_year, myyear=myyear)
    except:
        return render_template('error.html')

@application.route('/press')
def press():
    # db query
    entries = get_press()

    try:
        return render_template('press.html', mydata=entries)
    except:
        return render_template('error.html')

@application.route('/people')
@application.route('/people/<mystatus>')
def people(mystatus=None):
    entries = get_people()
    # data struct is a big list of entry dicts which looks something like this:
    # entries = [{'imagefile': u'raul.jpg', 'name': u'Raul Rabadan', 'iscurrent': 1, 'title': u'Principal Investigator', 'webpage': u'-', 'email': u'rabadan@dbmi.columbia.edu'}, {'imagefile': u'hossein.jpg', 'name': u'Hossein Khiabanian', 'iscurrent': 1, 'title': u'Associate Research Scientists', 'webpage': u'-', 'email': u'hossein@c2b2.columbia.edu'}, {'imagefile': u'jiguang.gif', 'name': u'Jiguang Wang', 'iscurrent': 1, 'title': u'Associate Research Scientists', 'webpage': u'-', 'email': u'-'}, {'imagefile': u'franny.png', 'name': u'Francesco Abate', 'iscurrent': 1, 'title': u'Postdoctoral Researchers', 'webpage': u'-', 'email': u'-'}]

    # list of lists of people dicts, segregated by title
    entries_title = []

    # titles of current members
    titles = ["Principal Investigator", "Associate Research Scientists", "Postdoctoral Researchers", "Doctoral Students", "Staff", "Master's Students"]
    # titles of alumni
    if (mystatus == "alum"):
        titles = ["Alumni"]

    # make data struct 
    for i in titles:
	# if nonempty (i.e., there are members with the title)
	if (filter(lambda y: y['title'] == i, entries)):
            entries_title.append(filter(lambda y: y['title'] == i, entries))

    try:
        return render_template('people.html', mydata=entries_title, mystatus=mystatus)
    except:
        return render_template('error.html')

@application.route('/courses')
def courses():
    return render_template('courses.html')

@application.route('/research')
def research():
    return render_template('research.html')

@application.route('/software')
def softw():
    return render_template('software.html')

@application.route('/contact')
def contact():
    return render_template('contact.html')

if __name__ == "__main__":
    application.run()
The function pubs passes a list of lists to publications.html. Each sub-list contains publications for one specific year. The elements of the list are dicts, as usual. The publications.html template loops through these nested lists. Similarly, the people.html template gets a list of lists grouped by job title.

The publications section became:

image

Adding the fonts at GoogleFonts, as is in vogue, plus my own CSS, improved the look and feel:

image

My labmates didn't like the grey for highlighted publications, so it morphed into:

image

with some very simple jQuery to display only the highlighted publications on button click.

The people section:

image

The software section:

image

Further Reading

The lab homepage is a first try with Flask to learn the fundamentals. Many of the choices I made reflect my newbie status. See how the pros develop with Flask: Reading aside, it's really nice to have somebody explain it to you:

(Video credit: YouTube: Miguel Grinberg: Flask by Example - PyCon 2014)
Advertising

image


image


image