Publishing Django Test Coverage Reports in Jenkins

This post is built on some assumptions.

First, I assume that you already know that writing unit tests is good for you. Well, to be honest, if you are not systematically writing tests for your code, you shouldn’t be calling yourself a software engineer anyway. No excuses.

In consequence I also assume that your latest Django project includes its dose of unit testing. But do you have a clear idea of which parts of your Django site are not being tested? Are you taking action to improve on that area? In other words, are you already obtaining and analysing coverage data for your project?

If so, lucky you. I didn’t, decided it was about time, and set out to the task.

I will try to demistify the process, since it takes very little effort and you can reap substantial benefits from it – provided, of course, that you take a look at the coverage reports on a regular basis, and add tests for the uncovered methods… but you promise you will do that, won’t you? Great!

We will start by generating the reports manually, and will then move on to automating them into Jenkins, our friendly build butler.
Continue reading

Running our Django site with mod_wsgi and virtualenv (part 2)

These are the steps remaining from our previous article. In order to complete our desired setup we must configure Apache with mod_wsgi pointing to the new virtual environment.

But before we can even do that, we need to setup mod_wsgi, which in our case will require building it and installing it from source.

Setting up mod_wsgi with our virtual Environment

Detailed explanations for using virtual environments in mod_wsgi can be found here.

Just by reading over a bit, and based on some prior experience (2+ years ago, though), I was expecting this to be the main pain area of the whole process. From the docs I read that mod_wsgi has to be compiled against the same version of Python your code will be running under. Which means I will have to build from source, since Ubuntu 9.04’s version of mod_wsgi is linked with its included version of Python 2.6.

Build mod_wsgi

First, we must build and install mod_wsgi to use the correct version of Python. The installation guide for mod_wsgi is very clear, you just need to follow it. Below are the distilled steps my installation needed.

We start by installing the Apache2 development libraries (which we will need in order to install mod_wsgi), and downloading mod_wsgi from its subversion repository.

sudo apt-get install apache2-dev
svn co http://modwsgi.googlecode.com/svn/branches/mod_wsgi-2.X mod_wsgi-2.x
cd !$

Continue reading

Setting up our Django site environment with pythonbrew and virtualenv

My problem: I have a server running my Django apps that I set up when I had very little experience with Django, and just some more with Python. The configuraton is using Apache and mod_wsgi on Ubuntu, running on the system’s default Python installation (2.6) which has Django 1.1 installed.

I need to update my site, which has been upgraded to use Django 1.2, and would benefit from Python 2.7 (although the site itself runs on 2.6 as well, running the tests requires 2.7, since I use some of the new convenient unittest assertions from that version, e.g. assertIsNotNone or assertIn).

Upgrading the site will require some migrations, which are not backwards compatible (e.g. some table and field renames). Since all in all will be a delicate intervention, I’d like to create a totally separate environment for testing the revamped site while the previous version is still online. Even if it was not risky, now I know better than when I started, and it’s a good time to switch to a virtual environment setup.

Therefore, in order to proceed we will:

  1. Install python 2.7 with pythonbrew
  2. Install virtualenv and virtualenwrapper
  3. Create a virtualenv with that python version
  4. Install all requirements with pip
  5. Make a Copy of the production DB, and migrate
  6. Run unit tests and test using the development server
  7. Switch to a production setup with mod_wsgi, and the new virtualenv

This article covers all steps but the mod_wsgi setup, which will have its specific followup.
Continue reading

Getting the delicious plugin to work on Firefox updates (4, 5…)

Every time Firefox updates their browser version, it’s the same story: the delicious extension is said not to be compatible. It looks like the new delicious owners are quite slow on the uptake…

But fortunately there is a decently simple workaround that works. The problem is not that the extension is not compatible, but it is not declared to be so. Simply modify the description file for the extension, tell it that you Firefox version is fine, and try it out. It has always worked for me so far.

Under Linux, locate your Firefox profile. In my case it is called zm84ac6z.default. Within this you will find an extensions folder:

$ cd ~/.mozilla/firefox/zm84ac6z.default/extensions

There you will find several subfolders, each named with some UUID, a hex string that looks sort of {2fa4ed95-0317-4c6a-a74c-5f3e3912c1f9}.

Locate the one for the delicious extension. In my case it is the one above, but you can always locate it:

$ find -name install.rdf -print -exec grep delicious.com {} \;

Once you’ve found the proper install.rdf file, edit it and look for something like em:maxVersion="4.1". Replace this with something like em:maxVersion="6.*" and you should be good to go for a while… It’s working fine for me on Firefox 5.0 on Ubuntu.

Good luck!

BTW: the process in Windows is similar, only the folder names are different…

Thanks to my colleague Marc Sallés for the tip!

Generating e-books with Sphinx

Sphinx

I find Sphinx a great tool for writing technical documentation. Born with the purpose of writing the Python documentation at docs.python.org, Sphinx specially excels at generating HTML for text-intensive documents that include code samples.

Although part of the Python ecosystem, there is nothing that binds Sphinx to just documenting Python programs. We use it at Isigma e.g. for documenting the integration APIs for our applications, such as PortaSigma or ISMe, which are mostly Java.

Sphinx serves as a good complement to in-code documentation tools like Javadoc/Doxygen/Epydoc, and is very well suited for writing programming tutorials and technical documents or books.

e-books

I recently started playing around with e-book readers and getting acquainted with the possibilities of developing software for these devices. I just started to familiarise myself with the epub format, which seems to be the best-suited format for e-books. For practical purposes, an epub file is just a ZIP file containing HTML, CSS and XML files. Which got me thinking, wouldn’t it be nice to be able to convert Sphinx documentation into an e-book?

I browsed the Sphinx documents and voilà, the epub format is stated to be available in experimental stage. You just have to do a specific build for it. It turned out to be extremely simple.
Continue reading

Regaining Control of the User Experience when SSL Root Certificates are not Recognised by Browsers

(Originally published in isigmaglobal.com)

So, Browsers Scare your Users by Saying your SSL Certificates Suck, hmmm?

Yup, we all know SSL is the way to go when protecting your site’s traffic from the bad guys, and it is not that difficult to set up. But what happens when the browsers don’t have pre-installed CA root certificates for the issuer of your SSL certificates? They tend to present very scary messages. Some versions of Internet Explorer heartily recommend users to get away from your site as quickly as possible, lest they be eaten by evil monsters (well, not exactly, but it has a similar effect).

Scare tactics may work well for browser manufacturers, so that certificate issuers are willing to make their certificates recognised and pre-installed in these browsers (or OS) releases. This process might involve some payment to the browser’s manufacturer… (warning: this is my personal unproven claim… call me suspicious if you wish).

In any case, once the user clicks on a link to your secure SSL-protected HTTPS site, funnily enough, the browser’s alarm messages may make it appear as a more insecure choice than using HTTP. What the …?

It looks like you don’t have control over the user’s experience… or do you?
Continue reading

Tracking Events with Google Analytics in Struts Actions

Google Analytics

Google Analytics is a great marketing tool that can be used in order to get detailed information about the users of your web sites and their navigation patterns. The standard approach is to track page visits (where statistics can be aggregated by visited URLs), but there are times where you would want a higher granularity of which actions your users do, where these can’t be tracked exclusively by URL.

Analytics provides a mechanism called event tracking for that purpose. Events can be tracked by calling a JavaScript function _trackEvent. A sample usage would be:

var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));

// get a tracker with your user account code
var pageTracker = _gat._getTracker("UA-XXXXXX-X"); 

// optionally track the page view:
pageTracker._trackPageview();

// track an event:
pageTracker._trackEvent(category, event, label);

The parameters to track event (category, event and label) can be used within Analytics to obtain three levels of statistical aggregation of data. E.g. in our case we may use categories for login, document, subscription… Within document we have events for upload, sign, reject, add comment… As a label to document operations we may assign a user id, a document type, or however we wish the document events to be labelled for later analysis.

Integrating event tracking with Struts actions

Event tracking is based on a Javascript API, which means that you need to generate specific Javascript code within your pages depending on the results of your actions. This code must be present in the pages rendered as a consequence of the action, which sometimes may result in a redirect or action forward (for example, to a document list). A specific action may generate more than one event (e.g. “document was signed”, “user entered comments”).

The approach we used in order to be able to track events from within Struts can be summarised in the following steps:

  • An object associated with the user session keeps a list of generated events
  • A custom tag present in all pages flushes all pending events in the page (generates the Javascript code and clears the event list)
  • We include this custom tag as part of the default site template, so that it appears in all pages

Continue reading