Flying with Django on Google App Engine
Google App Engine is a powerful tool for web developers. I am sure that it is useful and every developer should taste it =) Python was the first programming language supported by App Engine, and is a programming language with a lot of web frameworks. So, you can use some of these frameworks on Google App Engine. In a series of three blog posts, I will show how to use three Python web frameworks on App Engine: Django, Flask and web2py (not necessarily in this sequence).
The first framework is Django, the most famous of all Python frameworks and maybe is used the most.
Django models is the strongest Django feature. It is a high level database abstraction layer with a powerful object-relational mapper, it supports a lot of relational database management systems, but App Engine doesn’t use a relational database. The database behind App Engine is called BigTable, which is a distributed storage system for managing structured data, designed to scale to a very large size (Reference: Bigtable: A Distributed Storage System for Structured Data). It is not based on schemas, tables, keys or columns, it is like a big map indexed by a row key, column key and a timestamp. We can not use native version of Django models with Bigtable, because the Django models framework was not designed for non relational databases.
So, what can we do? There is a Django fork, the django-nonrel project, which aims to bring the power of the Django model layer to non-relational databases. I will use the djangoappengine sub-project to build the sample application of this post, that will be deployed on Google App Engine :)
The sample application is the default: a blog. A very simple blog, with only a form protected by login (using Django built-in authentication system instead of Google Accounts API) and a public page listing all blog posts. It is very easy and simple to do, so let’s do it.
First, we have to setup our environment. According the djangoappengine project documentation, we need to download 4 zip files and put it together. First, I downloaded the django-testapp file, extract its contents and renamed the project directory from django-testapp to blog_gae. After this step, I downloaded the other files and put it inside the blog_gae directory. Here is the final project structure:
“django” directory is from the django-nonrel zip file, “djangoappengine” directory is from djangoappengine zip file and “djangotoolbox” directory is from djangotoolbox zip file. Look that is provided an app.yaml file, ready to be customized. I just changed the application id inside this file. The final code of the file is the following:
version: 1
runtime: python
api_version: 1
default_expiration: '365d'
handlers:
- url: /remote_api
script: $PYTHON_LIB/google/appengine/ext/remote_api/handler.py
login: admin
- url: /_ah/queue/deferred
script: djangoappengine/deferred/handler.py
login: admin
- url: /media/admin
static_dir: django/contrib/admin/media/
- url: /.*
script: djangoappengine/main/main.py
I will use one version for each part of the series, so it is the first version because it is the first part =D In settings.py, we just uncomment the app django.contrib.auth line inside the INSTALLED_APPS tuple, because we want to use the built-in auth application instead of the Google Accounts API provided by App Engine.
All settings are ok now, it is time to create the core application. In the Django project, we will use the core application to manage models and serve some views. We just start it using the following command:
It is a famous Django command, that creates the application structure, which is a Python package containing 3 Python modules: models, tests and views. Now we have to create the Post model. Here is the code of models.py file:
from django.contrib.auth.models import User
class Post(models.Model):
title = models.CharField(max_length = 200)
content = models.TextField()
date = models.DateTimeField(auto_now_add = True)
user = models.ForeignKey(User)
Now we just need to “install” the core application putting it on INSTALLED_APPS tuple in settings.py file and Django will be ready to play with BigTable. :) We will use the django.contrib.auth app, so let’s run a manage command to create a superuser:
After create the superuser, we need to setup login and logout URLs, and make two templates. So, in urls.py file, put two mappings to login and logout views. The file will look like this:
urlpatterns = patterns('',
('^$', 'django.views.generic.simple.direct_to_template',
{'template': 'home.html'}),
('^login/$', 'django.contrib.auth.views.login'),
('^logout/$', 'django.contrib.auth.views.logout'),
)
Here is the registration/login.html template:
{% block content %}
<p>Fill the form below to login in the system ;)</p>
{% if form.errors %}
<p>Your username and password didn't match. Please try again.</p>
{% endif %}
<form method="post" action="{% url django.contrib.auth.views.login %}">{% csrf_token %}
<table>
<tr>
<td>{{ form.username.label_tag }}</td>
<td>{{ form.username }}</td>
</tr>
<tr>
<td>{{ form.password.label_tag }}</td>
<td>{{ form.password }}</td>
</tr>
</table>
<input type="submit" value="login" />
<input type="hidden" name="next" value="{{ next }}" />
</form>
{% endblock %}
And registration/logged_out.html template:
{% block content %}
Bye :)
{% endblock %}
See the two added lines in highlight. In settings.py file, add three lines:
LOGOUT_URL = '/logout/'
LOGIN_REDIRECT_URL = '/'
And we are ready to code =) Let’s create the login protected view, where we will write and save a new post. To do that, first we need to create a Django Form, to deal with the data. There are two fields in this form: title and content, when the form is submitted, the user property is filled with the current logged user and the date property is filled with the current time. So, here is the code of the ModelForm:
class Meta:
model = Post
exclude = ('user',)
def save(self, user, commit = True):
post = super(PostForm, self).save(commit = False)
post.user = user
if commit:
post.save()
return post
Here is the views.py file, with the two views (one “mocked up”, with a simple redirect):
from django.shortcuts import render_to_response
from django.template import RequestContext
from django.http import HttpResponseRedirect
from django.core.urlresolvers import reverse
from forms import PostForm
@login_required
def new_post(request):
form = PostForm()
if request.method == 'POST':
form = PostForm(request.POST)
if form.is_valid():
form.save(request.user)
return HttpResponseRedirect(reverse('core.views.list_posts'))
return render_to_response('new_post.html',
locals(), context_instance=RequestContext(request)
)
def list_posts(request):
return HttpResponseRedirect('/')
There is only two steps to do to finally save posts on BigTable: map a URL for the views above and create the new_post.html template. Here is the mapping code:
('^posts/$', 'core.views.list_posts'),
And here is the template code:
{% block content %}
<form action="{% url core.views.new_post %}" method="post" accept-charset="utf-8">
{% csrf_token %}
{{ form.as_p }}
<p><input type="submit" value="Post!"/></p>
</form>
{% endblock %}
Now, we can run on terminal ./manage.py runserver and access the URL http://localhost:8000/posts/new on the browser, see the form, fill it and save the post :D The last one step is list all posts in http://localhost:8000/posts/. The list_posts view is already mapped to the URL /posts/, so we just need to create the code of the view and a template to show the list of posts. Here is the view code:
posts = Post.objects.all()
return render_to_response('list_posts.html',
locals(), context_instance=RequestContext(request)
)
And the list_posts.html template code:
{% block content %}
<dl>
{% for post in posts %}
<dt>{{ post.title }} (written by {{ post.user.username }})</dt>
<dd>{{ post.content }}</dd>
{% endfor %}
</dl>
{% endblock %}
Finished? Not yet :) The application now is ready to deploy. How do we deploy it? Just one command:
Done! Now, to use everything that we have just created on App Engine remote server, just create a super user in that server and enjoy:
You can check this application flying on Google App Engine: http://1.latest.gaeseries.appspot.com (use demo for username and password in login page).
You can check this application code out in Github: http://github.com/fsouza/gaeseries/tree/django.
Enjoy :)
>Cool, thanks for your work man!
>Looking forward to the next parts of your series. Thanks a lot for writing this tutorial!
>How did you choose those frameworks? I've been learning Python and Appengine using Tipfy. It seems best suited for appengine.
>Hi Anonymous :)
It looks great, I'll check it ;)
>Can't wait to see how Flask is done, post 'er to reddit!
>Thank you for your tuto but I'm blocked at the first command :
$ ./manage.py startapp core
I've an exeption
Exception: Could not get appid. Is your app.yaml file missing? Error was: No module named yaml
my file app.yaml is in the current folder and I've changed the "ctst" with my id
>Hi, the error mesage is telling that the yaml module was not found. That module is distributed with Google App Engine, so you can add to your PYTHONPATH the following directories:
yaml – ($GOOGLE_APP_ENGINE_HOME)/lib/yaml/lib
antlr3 – ($GOOGLE_APP_ENGINE_HOME)/lib/antlr3
ipaddr – ($GOOGLE_APP_ENGINE_HOME)/lib/ipaddr
webob – ($GOOGLE_APP_ENGINE_HOME)/lib/webob
($GOOGLE_APP_ENGINE_HOME = your GAE SDK root)
Thanks ;)
>Thanks it works well now
>This is a great tutorial, I love how your started from the beg and explained all the details.
One question, how can one use this with an authentication that is EMAIL & Password based?
>You can use the built-in Google Accounts API for authentication: http://code.google.com/intl/en/appengine/docs/python/users/
>@franciscosouza.net, thanks but that won't work. I'm looking to allow users to create accounts with their EMAIL/Password, there email being their work email as their web app instance is based on their domain. Any ideas? Also, if you're looking to make this tutorial even more amazing, it would be great in the code samples if by every line there was a comment saying what this is doing, since this tutorial is targeted for begginers. Thanks!
>Hi dude,
I see what you want to do, but there is nothing out of box for using in Django or Google App Engine for registering and login with email and password. In that way, you will need to implement by yourself this solution =)
>Hi Francisco, thank you for this great tutorial.
I got stuck at the following two steps though.
"Now we just need to "install" the core application putting it on INSTALLED_APPS tuple in settings.py file and Django will be ready to play with BigTable. :) We will use the django.contrib.auth app, so let's run a manage command to create a superuser:
$ ./manage.py createsuperuser
"
I added a line 'core', to the INSTALLED_APPS part of settings.py (I hope that was the right syntax?) and then tried to run ./manage.py createsuperuser, but I get the following error
File "/Library/Frameworks/Python.framework/Versions/6.2/lib/python2.6/site-packages/django/db/utils.py", line 91, in __getitem__
backend = load_backend(db['ENGINE'])
File "/Library/Frameworks/Python.framework/Versions/6.2/lib/python2.6/site-packages/django/db/utils.py", line 49, in load_backend
raise ImproperlyConfigured(error_msg)
django.core.exceptions.ImproperlyConfigured: 'djangoappengine.db' isn't an available database backend.
Try using django.db.backends.XXX, where XXX is one of:
'dummy', 'mysql', 'oracle', 'postgresql', 'postgresql_psycopg2', 'sqlite3'
Error was: No module named djangotoolbox.db.creation
Any ideas what I'm doing wrong? The directory structure looks identical to the one you've posted above…. Thanks man
>Hey man,
you can check the source code out here: http://github.com/franciscosouza/gaeseries/tree/django
I think that there is a problem with your INSTALLED_APPS tuple.
>Thanks Francisco! Your git repo works like a charm! Good Karma your way amigo :)
(Just in case in anyone else runs into (at least one) my problems, the djangotoolbox directory contains another djangtotoolbox folder inside when you download the zipped versions; moving that directory to the main project folder and deleting the original fixed (partially) the problem. Unfortunately, I still had problems running createsuperuser –>Traceback: … "/Users/project/djangotoolbox/db/basecompiler.py", line 241, in execute_sql
raise NotImplementedError('The database backend only supports count() queries')
NotImplementedError: The database backend only supports count() queries
Using Francisco's code fixed whatever typo I had made causing this error. Thanks!
[...] is a bonus part in the series (after a looooooooooooong wait =P)! On the first blog post, about Django, I received a comment about the use of tipfy, a small Python web framework made specifically for [...]
[...] the third part of using Python frameworks in Google App Engine. I wrote before about web2py and Django, and now is the time of Flask, a Python microframework based on Werkzeug, Jinja2 and good [...]
[...] is a bonus part in the series (after a looooooooooooong wait =P)! On the first blog post, about Django, I received a comment about the use of tipfy, a small Python web framework made specifically for [...]
Hi Francisco, thanks for this walk-through.
I was able to get things up on my localserver fine, but when I deploy the app to App Engine, I see a “Error: Server Error” message (below). Any ideas what the cause could possibly be? Thanks.
************
Error: Server Error
The server encountered an error and could not complete your request.
If the problem persists, please report your problem and mention this error message and the query that caused it.
Hi Rohan :)
That is a generic server error message. You should check what happened in the application admin panel:
- Go to http://appengine.google.com
- Login if necessary :)
- Select the application
- Click on “Logs” at the left menu bar and check the error :)
Ah, I figured it out. thanks for helping out a django noob :D
Francisco first off, thank you for your posts!excellent work!
I have a question:
In the djangoappengine tutorial (http://www.allbuttonspressed.com/projects/djangoappengine#installation) it is mentioned that you also need to download the django-dbindexer application. Why don’t you use it?
thx
Hello!
I can’t remember of seeing this django-dbindexer tool. I think it was not on djangoappengine tutorial when I wrote this post.
Anyway, thank you for this comment! :)
Hi
I can’t figure out how to use static files with nonrel. I just add the static_dir lines described in the google-app docs. Do i need additional configurations in the settings.py and urls.py
Thx in advance
Did you add the static handler before the django-nonrel handler in your app.yaml? Just like this:
- url: /static
static_dir: static
- url: /.*
script: djangoappengine/main/main.py