My problem: I have a server running my Django apps that I set up when I had very little experience with Django, and just some more with Python. The configuraton is using Apache and mod_wsgi on Ubuntu, running on the system’s default Python installation (2.6) which has Django 1.1 installed.
I need to update my site, which has been upgraded to use Django 1.2, and would benefit from Python 2.7 (although the site itself runs on 2.6 as well, running the tests requires 2.7, since I use some of the new convenient unittest assertions from that version, e.g.
Upgrading the site will require some migrations, which are not backwards compatible (e.g. some table and field renames). Since all in all will be a delicate intervention, I’d like to create a totally separate environment for testing the revamped site while the previous version is still online. Even if it was not risky, now I know better than when I started, and it’s a good time to switch to a virtual environment setup.
Therefore, in order to proceed we will:
- Install python 2.7 with
- Create a
virtualenvwith that python version
- Install all requirements with
- Make a Copy of the production DB, and migrate
- Run unit tests and test using the development server
- Switch to a production setup with
mod_wsgi, and the new
This article covers all steps but the
mod_wsgi setup, which will have its specific followup.
All instructions and command-line examples are biased towards Ubuntu.
Install pythonbrew and Python 2.7
pythonbrew automates the building and installation of Python in the users $HOME. It is very convenient in order to e.g. install versions of python newer than those supported by your OS version (in my case the server still runs Ubuntu 9.04, which uses python 2.6).
Since pythonbrew builds python on your system, and I will need
sqlite3 support, I will need to install its development packages (I know this because I failed earlier). Same applies to libxml and bz2 development packages, because I’ll be using lxml and bz2. While I’m at it, I’ll also install curl:
sudo aptitude install libsqlite3-dev libbz2-dev libxml2-dev libxslt-dev curl
$ curl -kL http://github.com/utahta/pythonbrew/raw/master/pythonbrew-install | bash $ echo "source $HOME/.pythonbrew/etc/bashrc" >> ~/.bashrc
… and reload the shell for the command to be available.
Installing python 2.7 with pythonbrew may take quite a while, since it downloads and compiles from source (and runs tests):
$ pythonbrew install --verbose 2.7.2
After that, you may activate and run your new python interpreter in the current shell with:
$ python -V Python 2.6.2 $ pythonbrew use 2.7.2 Using `Python-2.7.2` $ python -V Python 2.7.2
Install virtualenv and virtualenvwrapper
virtualenv is a tool to create isolated Python environments. An isolated environment includes a specific version of python plus a set of installed packages. With virtualenv you can easily manage multiple such environments with controlled subsets of package versions in the same machine.
virtualenvwrapper is a set of extensions to
virtualenv to simplify the organisation and switching between multiple virtual environments.
Install on linux:
sudo aptitude install python-virtualenv pythonbrew use 2.7.2 pip install virtualenvwrapper
[EDIT: make sure that you install virtualenvwrapper within each of your python brews, i.e. your specific version was activated (with
pythonbrew use) before installing.]
Create a directory to hold your virtual environments:
… then edit ~/.bashrc, and add at the end:
export WORKON_HOME=$HOME/.virtualenvs export VIRTUALENVWRAPPER_PYTHON=$HOME/.pythonbrew/pythons/Python-2.7.2/bin/python source $HOME/.pythonbrew/pythons/Python-2.7.2/bin/virtualenvwrapper.sh
Relogin, or apply with
Create the new virtual environment
We are now ready to create and switch to the new virtual environment. We want to make sure it will use the proper python we installed earlier with
$ pythonbrew use 2.7.2 Using `Python-2.7.2` $ mkvirtualenv --no-site-packages fabula (fabula)$ python -V Python 2.7.2 (fabula)$ which python /home/carles/.virtualenvs/fabula/bin/python
(Note that virtualenv adds the name of the active environment within parenthesis in your prompt, for easy identification.)
Additional goodies… Install
yolk, so that we can obtain a useful list of installed packages.
$ pip install yolk $ yolk -l Python - 2.7.2 - active development (/home/carles/.pythonbrew/pythons/Python-2.7.2/lib/python2.7/lib-dynload) pip - 1.0.1 - active setuptools - 0.6c11 - active wsgiref - 0.1.2 - active development (/home/carles/.pythonbrew/pythons/Python-2.7.2/lib/python2.7) yolk - 0.4.1 - active
Install the project’s dependencies
All of the required packages are in the project’s
requirements.txt file used by pip (if you’re still not using this, you should). My requirements file contains:
django==1.2.5 South lxml
Installing all of these with
pip is a breeze.
(fabula)$ pip install -r requirements.txt
We can check that all’s fine, and we have separate environments as expected (
pythonbrew off are virtualenv’s and pythonbrew’s respective commands to leave the controlled environment and return to default versions):
# go back to default versions (fabula)$ deactivate $ pythonbrew off $ python -V Python 2.6.2 $ python -c "import django; print(django.VERSION)" (1, 1, 0, 'final', 0) # reactivate controlled environments $ workon fabula (fabula)$ python -V Python 2.7.2 (fabula)$ python -c "import django; print(django.VERSION)" (1, 2, 5, 'final', 0)
We’re cool! Let’s proceed with testing our new version of the application.
Migrate data and test
Our goal now is to set up a different database than production, migrate it and run our tests. We will test the migrations with a replica of production data, by exporting the production database and importing it in the staging site. With postgreSQL replicating the DB may be as easy as:
create database fabula_v11 with template fabula_v10;
We change our settings in staging to point to the new DB (
DATABASE_NAME=fabula_v11), and perform the validations (
manage is my alias for django’s manage.py/django-admin.py command):
(fabula)$ manage migrate (fabula)$ manage test ....................................................................................................... ---------------------------------------------------------------------- Ran 103 tests in 15.033s (fabula)$ manage runserver 0.0.0.0:8186
… and we have a test server running on port 8186, we may open our browser to http://hostname:8186/admin/ and voilà! there we have a running test site prefilled with production data that we may further test. Once we have verified that the site functionally works, we can stop the development server and get ready to go live with a real production setup with
mod_wsgi, which I will cover on a follow-up article .