Freshen is an acceptance testing framework for Python, inspired by Cucumber and has the same goal: make BDD fun, but using Python instead of Ruby. Freshen uses the same syntax of Cucumber (Gherkin Syntax) and runs as a plugin of Nose, a powerful Python tool for using daily when you with TDD =)
Important: If you don’t know Django, this article is not recommended for you.
Nose is plugin based, so to use Django under Nose, we need to install a plugin called NoseDjango. To work isolated from the database, we need a mock framework. Let’s use Mock, wich is a simple and powerful mock framework for Python. We can install Django, Freshen, Mock, Nose and NoseDjango with a simple command:
It works in any operating system with Python and PIP.
After installing everything we need, we just start a Django Project, called “library” (just supposing that we are developing a software for the library on the next corner):
Looking quickly on Django settings, in the settings.py file, only database and templates settings:
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'data.db',
'USER': '',
'PASSWORD': '',
'HOST': '',
'PORT': '',
}
}
# Other settings here
import os
project_root = os.path.dirname(os.path.abspath(__file__))
template_root = os.path.join(project_root, 'templates')
TEMPLATE_DIRS = (
template_root,
)
# Other settings here
After start and setup the project, let’s start our Django application. Let’s start an application called “loans”. It app will manage the loans of our library:
After start the application, let’s define our Book model and install the application. You can see below the code of models module inside the loans app:
class Book(models.Model):
title = models.CharField(max_length=100)
author = models.CharField(max_length=100)
Now, we need to describe our acceptance tests, or as Freshen and Cucumber call: our system features. Let’s describe the loans application’s features inside the loans application, creating a directory called “features” inside the application.
The first feature of the application will be a feature that describes the access to a list containg all the books on the path “/loans/books”:
In order to loan a book
As a system user
I want to see the list of all books in the system
Scenario: List all books
Given there are 20 books in the system
When I navigate to the books list page
Then I should see the title of the 20 books
Let’s understand the feature: A Freshen feature is a description of a system feature! :) On the list of books, we have a scenario where we list all books. That scenario is described using some steps: given, when and then. You can learn more about Given, When and Then looking for some resources about BDD.
After describe a feature and it’s scenarios, we need to define the steps of each scenario. So, we put these definitions in a Python module called steps.py in our features directory. The scenario List all books has three steps: “Given there are 20 books in the system”; “When I navigate to the books list page” and “Then I should see the title of the 20 books”. First, let’s define the first step: Given there are 20 books in the system. Each step definition is a Python function decorated with the keyword that defines that step.
So, to define “Given” step, Freshen provides the @Given decorator. Let’s define a function that creates and saves 20 books in the database. But we will not really save it in the database, because we are working isolated from the database. We will just store the book list in a context object.
def create_books(total_of_books):
scc.books = []
for i in xrange(int(total_of_books)):
book = Book()
book.title = 'Book %d' %(i + 1)
book.author = 'Francisco Souza'
scc.books.append(book)
We use a regular expression to determine a pattern to receive a parameter in the function. The function create_books receives a parameter wich indicates the number of books that will be created. All parameters obtained from step specification is an instance of str and need to be converted properly if you want to use it as a number.
The second line of the create_books function defines an empty list called books inside an object called scc. Freshen provides three objects for context storage: glc, the global context, wich is never cleaned; ftc, the feature context, cleaned at the start of each feature and scc, the scenario context, wich is cleaned at the start of each scenario. Once we put the books object list in the scenario context, we can use this list in all steps of that scenario.
Now, let’s define the second step: When I navigate to the books list page. Now we will use the @When decorator to decorate a function called navigate_to_books_list. Here is the code:
def navigate_to_books_list():
with patch('loans.models.Book') as MockClass:
Book.objects.all = Mock()
Book.objects.all.return_value = scc.books
client = Client()
scc.response = client.get('/loans/books')
Book.objects.all.assert_called_with()
This function just makes a GET request on the URL /loans/books and saves the response in a object called response, stored in scenario context. The Book model was mocked here, using the patch decorator from Mock. We said to the mock: “when Book.objects.all() is called, return my list of books, stored in scenario context”. The last line of the function asserts if the method is really called in our GET request.
Now we can run the nosetests and see what happens. To execute Nose with Django and Freshen, we need to run the following command:
The output at this point is:
======================================================================
ERROR: List of books: List all books
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/francisco/Projetos/library/loans/features/steps.py", line 19, in navigate_to_books_list
scc.response = client.get('/loans/books')
File "/usr/local/lib/python2.6/dist-packages/django/test/client.py", line 290, in get
response = self.request(**r)
File "/usr/local/lib/python2.6/dist-packages/django/core/handlers/base.py", line 127, in get_response
return callback(request, **param_dict)
File "/usr/local/lib/python2.6/dist-packages/django/views/defaults.py", line 13, in page_not_found
t = loader.get_template(template_name) # You need to create a 404.html template.
File "/usr/local/lib/python2.6/dist-packages/django/template/loader.py", line 157, in get_template
template, origin = find_template(template_name)
File "/usr/local/lib/python2.6/dist-packages/django/template/loader.py", line 138, in find_template
raise TemplateDoesNotExist(name)
TemplateDoesNotExist: 404.html
>> in "I navigate to the books list page" # loans/features/list_books.feature:8
----------------------------------------------------------------------
Ran 2 tests in 0.174s
FAILED (errors=1)
Destroying test database 'default'...
The test fails with the following exception: TemplateDoesNotExist: 404.html. So, let’s create that template and see what happens.
After add a 404.html template to the templates directory in the root of the project, just run nosetests again, using the verbose mode:
List of books: List all books ... FAIL
test_model_validation_title_author (library.loans.tests.BookTest) ... ok
======================================================================
FAIL: List of books: List all books
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/francisco/Projetos/library/loans/features/steps.py", line 24, in navigate_to_books_list
Book.objects.all.assert_called_with()
File "/usr/local/lib/python2.6/dist-packages/mock.py", line 150, in assert_called_with
assert self.call_args == (args, kwargs), 'Expected: %s\nCalled with: %s' % ((args, kwargs), self.call_args)
AssertionError: Expected: ((), {})
Called with: None
----------------------------------------------------------------------
Ran 2 tests in 0.176s
FAILED (failures=1)
Destroying test database 'default'...
Step fails again, why? On function “navigate_to_books_list” the last line defines that the method Book.objects.all() should be called with no parameters in the GET request (lines above), but this method was not called anywhere, once that GET request returned a 404 response, because there is not an URL mapping to a view that responds by the action ‘/loans/books’, and there is not a view wich makes the expected method call. So, let’s make the view and map it to the “/loans/books” path. Here is the view code:
books = Book.objects.all()
return HttpResponse('')
Look that the view has only two lines of code and returns a HttpResponse object with blank content, it’s the enough code to make the Step definition works. To map the view to a URL path, create a urls.py module inside the application directory, with the following code:
urlpatterns = patterns('loans.views',
url(r'^books/', 'books', name = 'books_list'),
)
And include it on the main urls.py of project, that will contain the code bellow:
# Uncomment the next two lines to enable the admin:
# from django.contrib import admin
# admin.autodiscover()
urlpatterns = patterns('',
# Example:
# (r'^library/', include('library.foo.urls')),
# Uncomment the admin/doc line below and add 'django.contrib.admindocs'
# to INSTALLED_APPS to enable admin documentation:
# (r'^admin/doc/', include('django.contrib.admindocs.urls')),
# Uncomment the next line to enable the admin:
# (r'^admin/', include(admin.site.urls)),
(r'^loans/', include('loans.urls')),
)
Run nosetests again and see what happens:
List of books: List all books ... UNDEFINED: "I should see the title of the 20 books" # loans/features/list_books.feature:9
test_model_validation_title_author (library.loans.tests.BookTest) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.181s
OK (UNDEFINED=1)
Destroying test database 'default'...
Freshen said: I found an undefined step (“I should see the title of the 20 books”). Let’s define that step, using the @Then decorator to decorate our function called check_presence_of_the_title_of_books:
def check_presence_of_the_title_of_books(total_of_books):
expected_string = ''
for book in scc.books:
expected_string += '<li>%s</li>' % book.title
assert_true(expected_string in scc.response.content)
The function checks if the books titles is included in a li HTML element. Let’s call nosetests again and see the test failing:
List of books: List all books ... FAIL
test_model_validation_title_author (library.loans.tests.BookTest) ... ok
======================================================================
FAIL: List of books: List all books
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/francisco/Projetos/library/loans/features/steps.py", line 25, in check_presence_of_the_title_of_books
assert_true(expected_string in scc.response.content)
AssertionError
----------------------------------------------------------------------
Ran 2 tests in 0.178s
FAILED (failures=1)
Destroying test database 'default'...
The content returned by the GET request (an empty string) is not the content expected by the defined step, so we see an AssertionError. Now, we need to make this test pass. To do it, we need only to refactor the view:
books = Book.objects.all()
return render_to_response('books_list.html', {
'books' : books
}, context_instance=RequestContext(request)
)
And we run nosetests with Django and Freshen, in verbose mode again, and see that everything works fine =D
List of books: List all books ... ok
test_model_validation_title_author (library.loans.tests.BookTest) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.190s
OK
Destroying test database 'default'...
That is all, folks! :) You can download this Django Project at Github.
>Nice article!
Congrats for the initiative of starting a blog in English!
:)
>Sooooo awesome! Been looking for a testing solution just like this for Django. Everyone keeps saying to use fixtures, but give me mocks or give me death.
>Thanks for your article. Personally I like mock-approach rather then fixture approach from my Java-background.
But there is need of fixtures anyway in Django development. It's too hard to mock QuerySets as even generic views and custom managers utilize much more of features of it than just it's iterator.
Right now I'm trying to figure out how to get together Lettuce/Freshen with traditional fixtures. Not really obvious for me so if you have any idea how to do this kind of thing – please write about it (sooner is better because I might end up with mental disease trying to figure out it :)
>Great blog post. Thanks! Definitely going to use this in my upcoming project at work.
Thanks for the tutorial. I am getting stuck at the first test. Here is my output:
C:\My Web Sites\library>nosetests –with-freshen –with-django
E.
============================================================
: List all booksooks
———————————————————————-
Traceback (most recent call last):
File “C:\My Web Sites\library\loan\features\steps.py”, line 16, in navigate_to
_books_list
with patch(‘loans.models.Book’) as MockClass:
File “C:\Python27\lib\site-packages\mock.py”, line 711, in patch
target = _importer(target)
File “C:\Python27\lib\site-packages\mock.py”, line 506, in _importer
thing = __import__(import_path)
ImportError: No module named loans
” # loan\features\one.feature:8list page
———————————————————————-
Ran 2 tests in 0.128s
FAILED (errors=1)
Destroying test database ‘default’…
I have never used ‘with’ or Mock before, so I am in a little over my head. I am using python 2.7 and django 1.2.3.
I also noticed you assumed we would know to import the appropriate modules, this is what is in my steps.py:
from loan.models import Book
from freshen import Given, When, scc
from mock import patch, Mock
The rest of the code is copied/pasted from above. I named the feature file one.feature.
Thanks.
Hi Steve,
I see that your application directory is called “loan”, but I used “loans” (plural) on code.
I also recommend the use of Lettuce [1] instead of Freshen. It also has a powerful Django pluggable app [2] :)
[1] http://lettuce.it
[2] http://lettuce.it/recipes/django-lxml.html
Keep in touch,
Francisco Souza
Thanks Francisco. It’s moments like this that make me wonder if I should be a programmer!
I had seen both freshen and lettuce, and had read that lettuce had better docs but freshen was more mature. Why do you prefer lettuce?
Lettuce has a more complete documentation and has also a good test coverage (Freshen is a test framework not tested =/), but yes: Freshen is more mature and has more features than Lettuce.
freshen tests itself, using itself. :) Look in the features folder.
Francisco, thanks for getting the Python world fired up about Cucumber, etc. I’m a Python guy, but I’ve been coding in RoR for the last 1.5 years. I really liked Cucumber, so I’m glad to see I can do the same things in Python.
A few minor comments:
One thing I really like about Cucumber is that it’s integration testing. I’m very suspicious of mocks because you can easily misspell something in a mock, and your tests will pass, but your code won’t work (for instance, if the code and the mock use one spelling, but the database uses another). Hence, I’m really a fan of running my Cucumber tests against a test database. I don’t think there’s a real need to mock out the database if you have one running locally. Mocking out a payment gateway makes a lot of sense, but databases–not so much. I wonder if you’ve ever set up what RoR calls “transactional fixtures”, i.e. where each Freshen test runs in its own transaction so that rollbacks at the end of each test are really easy and fast.
By the way, I’m not a fan of fixtures. In the RoR world, I use this thing called factory_girl. Hence, I can create a simple database record when I need to, but my “setup data” varies on a per-test basis–which is pretty close to what you were doing, except I actually use a database.
I’m worried that you were testing for HTML using escaped HTML. I think you should either a) not bother testing for the li tags, but just instead look for the content you want or b) use an XML parsing library to actually parse the response and then use XPath to find the content.
Anyway, thanks again for the useful blog post, and happy hacking!