Posts Tagged django
This is a quick guide to running your [python/django] project on PyPy, the fast JIT-based Python interpreter (and optionally benchmark re: cPython)
Follow the clearly detailed instructions in: http://pypy.readthedocs.org
For example, for 32-bit linux I used:
$ wget https://bitbucket.org/pypy/pypy/downloads/pypy-1.8-linux.tar.bz2 $ tar xvf pypy-1.8-linux.tar.bz2
This will create a folder named
pypy-1.8, which matches Python 2.7.2:
$ ./pypy-1.8/bin/pypy --version Python 2.7.2 (0e28b379d8b3, Feb 09 2012, 19:41:19) [PyPy 1.8.0 with GCC 4.4.3]
Create your virtual environment
Install distribute and pip for pypy:
$ wget http://python-distribute.org/distribute_setup.py $ wget https://raw.github.com/pypa/pip/master/contrib/get-pip.py $ ./pypy-1.8/bin/pypy distribute_setup.py $ ./pypy-1.8/bin/pypy get-pip.py
Make sure you have at least virtualenv 1.6, else it won’t work with PyPy:
$ virtualenv --version 1.6.4
Install virtualenvwrapper, and create a new virtual environment for your projectx:
$ ./pypy-1.8/bin/pip install virtualenvwrapper $ mkvirtualenv --no-site-packages --distribute --python=/path/to/pypy-1.8/bin/pypy projectx-pypy $ python --version Python 2.7.2 (0e28b379d8b3, Feb 09 2012, 19:41:19) [PyPy 1.8.0 with GCC 4.4.3]
Now we have a virtual environment named
projectx-pypy based on
pypy that we can use as just any other virtualenv.
Setup your project
Install your project’s requirements as usual:
$ workon projectx-pypy $ pip install -r requirements.txt -r test-requirements.txt
Benchmark if desired
Now I can easily compare timings between cPython and PyPy. I created another identical virtualenv based on cPython (2.7.2) and run the same tests (one testcase replicated 100000 times using nose test generators, just to have something time consuming and CPU-intensive).
$ workon projectx-pypy $ nosetests ... ---------------------------------------------------------------------- Ran 100000 tests in 8.624s
$ workon projectx $ nosetests ... ---------------------------------------------------------------------- Ran 100000 tests in 38.180s
Of course this is not a representative sample, but just a simple test. In this one case, PyPy takes approximately 20% the execution time of cPython. Not bad, huh?
The PyPy people have more comprehensive benchmarks in their speed center.
So give PyPy a try, and share your results.
This post is built on some assumptions.
First, I assume that you already know that writing unit tests is good for you. Well, to be honest, if you are not systematically writing tests for your code, you shouldn’t be calling yourself a software engineer anyway. No excuses.
In consequence I also assume that your latest Django project includes its dose of unit testing. But do you have a clear idea of which parts of your Django site are not being tested? Are you taking action to improve on that area? In other words, are you already obtaining and analysing coverage data for your project?
If so, lucky you. I didn’t, decided it was about time, and set out to the task.
I will try to demistify the process, since it takes very little effort and you can reap substantial benefits from it – provided, of course, that you take a look at the coverage reports on a regular basis, and add tests for the uncovered methods… but you promise you will do that, won’t you? Great!
These are the steps remaining from our previous article. In order to complete our desired setup we must configure Apache with
mod_wsgi pointing to the new virtual environment.
But before we can even do that, we need to setup
mod_wsgi, which in our case will require building it and installing it from source.
mod_wsgi with our virtual Environment
Detailed explanations for using virtual environments in
mod_wsgi can be found here.
Just by reading over a bit, and based on some prior experience (2+ years ago, though), I was expecting this to be the main pain area of the whole process. From the docs I read that
mod_wsgi has to be compiled against the same version of Python your code will be running under. Which means I will have to build from source, since Ubuntu 9.04′s version of mod_wsgi is linked with its included version of Python 2.6.
First, we must build and install
mod_wsgi to use the correct version of Python. The installation guide for
mod_wsgi is very clear, you just need to follow it. Below are the distilled steps my installation needed.
We start by installing the Apache2 development libraries (which we will need in order to install
mod_wsgi), and downloading
mod_wsgi from its subversion repository.
sudo apt-get install apache2-dev svn co http://modwsgi.googlecode.com/svn/branches/mod_wsgi-2.X mod_wsgi-2.x cd !$
My problem: I have a server running my Django apps that I set up when I had very little experience with Django, and just some more with Python. The configuraton is using Apache and mod_wsgi on Ubuntu, running on the system’s default Python installation (2.6) which has Django 1.1 installed.
I need to update my site, which has been upgraded to use Django 1.2, and would benefit from Python 2.7 (although the site itself runs on 2.6 as well, running the tests requires 2.7, since I use some of the new convenient unittest assertions from that version, e.g.
Upgrading the site will require some migrations, which are not backwards compatible (e.g. some table and field renames). Since all in all will be a delicate intervention, I’d like to create a totally separate environment for testing the revamped site while the previous version is still online. Even if it was not risky, now I know better than when I started, and it’s a good time to switch to a virtual environment setup.
Therefore, in order to proceed we will:
- Install python 2.7 with
- Create a
virtualenvwith that python version
- Install all requirements with
- Make a Copy of the production DB, and migrate
- Run unit tests and test using the development server
- Switch to a production setup with
mod_wsgi, and the new
This article covers all steps but the
mod_wsgi setup, which will have its specific followup.
Read the rest of this entry »