Showing posts with label Python. Show all posts
Showing posts with label Python. Show all posts

Thursday, May 24, 2012

Django 1.4 raises PicklingError for Forms with Fields of Type BooleanField

The latest version of Django as of writing this post is 1.4, which contains a regression regarding the picklability of forms that contain fields of type BooleanField. The code for the widget that renders the BooleanField as a checkbox contains a lambda, which is not picklable. There is a fix in the Django github repository, but you might want to use the latest release version for production code.

I recently ran into this issue when trying to store a form object in the session. Regardless of whether this is a good idea, if you find yourself staring at the error below and scratching your head, you might be interested in a quick and dirty workaround.


Traceback:

File "/Users/hs/code/venv_django_fail/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  188.                 response = middleware_method(request, response)
File "/Users/hs/code/venv_django_fail/lib/python2.7/site-packages/django/contrib/sessions/middleware.py" in process_response
  36.                 request.session.save()
File "/Users/hs/code/venv_django_fail/lib/python2.7/site-packages/django/contrib/sessions/backends/db.py" in save
  52.             session_data=self.encode(self._get_session(no_load=must_create)),
File "/Users/hs/code/venv_django_fail/lib/python2.7/site-packages/django/contrib/sessions/backends/base.py" in encode
  79.         pickled = pickle.dumps(session_dict, pickle.HIGHEST_PROTOCOL)

Exception Type: PicklingError at /
Exception Value: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

Patching the Django code in my virtualenv seemed like a terrible idea. So instead I took the fixed code from the current Django development version and put it into a widgets.py file in my app directory. Let's say you have this form:


class FunnyForm(forms.Form):

    name = forms.CharField(label=u"What's your name?")
    is_awesome = forms.BooleanField(label=u'Are you awesome?', initial=True)

All you need to do is put this widgets.py file into your app directory, add
from widgets import PicklableCheckboxInput
to the top of your views.py file and change the widget for the BooleanField in the form like this:


class FunnyForm(forms.Form):

    name = forms.CharField(label=u"What's your name?")
    is_awesome = forms.BooleanField(label=u'Are you awesome?', initial=True,
        widget=PicklableCheckboxInput)


Tuesday, June 2, 2009

Using Sphinx and git as a simple CMS

Suppose you want to be able to easily publish content to a website without using a CMS like Plone or Drupal. Let’s say in order to minimize the attack surface of your website you don’t want any dynamic HTML generation to take place. In this post I’ll demonstrate one way this can be done using existing tools. The main building blocks are Sphinx and git. First, I’ll set up a self-contained installation using virtualenv:
$ mkdir sphinx_playground
$ virtualenv --no-site-packages sphinx_playground
New python executable in sphinx_playground/bin/python
Installing setuptools............done.
$ cd sphinx_playground
$ source bin/activate
(sphinx_playground)$ easy_install -U Sphinx
Searching for Sphinx
Reading http://pypi.python.org/simple/Sphinx/
Reading http://sphinx.pocoo.org/
Best match: Sphinx 0.6.1
Downloading http://pypi.python.org/packages/2.6/S/Sphinx/Sphinx-0.6.1-py2.6.egg#md5=0c5baac650e48792124f71eabb0eb029
[...]
Finished processing dependencies for Sphinx
(sphinx_playground)$
  
Installing Sphinx with easy_install installs all dependencies, including docutils and Pygments. The next step is to create directories for the git repository containing files in restructuredText and for the HTML that Sphinx will produce from these files. Sphinx includes a script named sphinx-quickstart, that will do this, as well as create some initial files to get started.
(sphinx_playground)$ sphinx-quickstart
[...]
(sphinx_playground)$ cd source
(sphinx_playground)$ git init
Initialized empty Git repository in /some/path/sphinx_playground/source/.git/
(sphinx_playground)$ git add conf.py index.rst
(sphinx_playground)$ git commit -m "initial commit"
Created initial commit ce317ec: initial commit
 2 files changed, 214 insertions(+), 0 deletions(-)
 create mode 100644 conf.py
 create mode 100644 index.rst
(sphinx_playground)$
  
sphinx-quickstart asks a few questions regarding the configuration. I chose seperate directories for source and build and accepted the default for most of the other choices. The script created a makefile, which I can use to generate the HTML by typing
(sphinx_playground)$ make html
  
The HTML files will end up in build/html. By default, the directory containing the stylesheets used in these files is build/html/_static. You probably want to change how the resulting HTML looks. Luckily, Sphinx has the concept of themes. I won’t go into detail on how to use this feature, since I don’t have the slightest clue. It’s probably not very hard, though. Check out the Sphinx documentation on how to use HTML themes. What’s left is adding a post commit hook in the git repository. Creating hooks in git is just a matter of putting a file named post-commit (or pre-commit, etc.) in /path/to/repo/.git/hooks. Here’s an example for a simple post-commit script that calls make to generate the HTML:
(sphinx_playground)$ cat source/.git/hooks/post-commit
#!/bin/sh

PROJECT_DIR=/some/path/sphinx_playground

cd $PROJECT_DIR
make html
(sphinx_playground)$ chmod 744 source/.git/hooks/post-commit
  
Make sure the script is executable and then modify the file source/index.rst and commit the changes.
(sphinx_playground)$ cat index.rst
.. sphinx-playground documentation master file, created by
sphinx-quickstart on Sun May 31 23:01:49 2009.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.

Welcome to sphinx-playground's documentation!
=============================================

I'm a headline! Look at me!
---------------------------

Contents:

.. toctree::
:maxdepth: 2

Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

(sphinx_playground)$ git add index.rst
(sphinx_playground)$ git commit -m "added silly headline"
sphinx-build -b html -d build/doctrees   source build/html
Making output directory...
Running Sphinx v0.6.1
loading pickled environment... not found
building [html]: targets for 1 source files that are out of date
updating environment: 1 added, 0 changed, 0 removed
reading sources... [100%] index
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
writing additional files... genindex search
copying static files... done
dumping search index... done
dumping object inventory... done
build succeeded.

Build finished. The HTML pages are in build/html.
Created commit 43a506c: added silly headline
1 files changed, 3 insertions(+), 0 deletions(-)
(sphinx_playground)$
  
If you point your browser to /some/path/build/html/index.html you should see the new headline on the page. Updates work as well, just change the file and commit again:
(sphinx_playground)$ cat index.rst
.. sphinx-playground documentation master file, created by
sphinx-quickstart on Sun May 31 23:01:49 2009.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.

Welcome to sphinx-playground's documentation!
=============================================

I'm a different headline! Look at me!
-------------------------------------

Contents:

.. toctree::
:maxdepth: 2

Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

(sphinx_playground)$ git add index.rst
(sphinx_playground)$ git commit -m "changed silly headline"
sphinx-build -b html -d build/doctrees   source build/html
Running Sphinx v0.6.1
loading pickled environment... done
building [html]: targets for 1 source files that are out of date
updating environment: 0 added, 1 changed, 0 removed
reading sources... [100%] index
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
writing additional files... genindex search
copying static files... done
dumping search index... done
dumping object inventory... done
build succeeded.

Build finished. The HTML pages are in build/html.
Created commit 0d6ef5a: changed silly headline
1 files changed, 2 insertions(+), 2 deletions(-)
(sphinx_playground)$
  
You might wonder whether Sphinx regenerates the HTML for all source files, regardless of them being out of date or not. Luckily, Sphinx is smarter than that, as you can see below:
(sphinx_playground)$ cat newfile.rst
I'm just a new file
===================

Move along, there is nothing to see here.

(sphinx_playground)$ git add newfile.rst
(sphinx_playground)$ git commit -m "added new file"
sphinx-build -b html -d build/doctrees   source build/html
Running Sphinx v0.6.1
loading pickled environment... done
building [html]: targets for 0 source files that are out of date
updating environment: 1 added, 0 changed, 0 removed
reading sources... [100%] newfile
looking for now-outdated files... none found
pickling environment... done
checking consistency... /some/path/sphinx_playground/source/newfile.rst:: WARNING: document isn't included in any toctree
done
preparing documents... done
writing output... [100%] newfile
writing additional files... genindex search
copying static files... done
dumping search index... done
dumping object inventory... done
build succeeded, 1 warning.

Build finished. The HTML pages are in build/html.
Created commit 5a287cd: added new file
 1 files changed, 5 insertions(+), 0 deletions(-)
 create mode 100644 newfile.rst
(sphinx_playground)$
  
As you can see in the output, Sphinx correctly determined, that existing source files have not changed. Also note, that Sphinx emits a warning, saying that the new file is not mentioned in a table of contents. Well, that’s pretty much it. Obviously you can enhance the post commit hook in any way you see fit. Like uploading the generated HTML to a web server via FTP or ssh, for example.

Friday, March 20, 2009

First Post!

Update: Since writing this post, I've moved the blog to Blogger. Turns out, I'm not fond of system administration. Besides, Blogger has shiny gadgets, widgets and whatnots.

So I decided to try this newfangled blogging thing that everyone is talking about. Maybe it's not just a fad after all. At this adoption rate expect me to start using twitter in about five years. ;)

In good internet tradition the first post is completely self-referential, describing what software I used to set this blog up.


Tools

One of the coolest features of the Django framework is its support for reusable applications. The concept of a generic foreign key is crucial for this to work, as well as some conventions like naming url patterns and adding a parameter for template names to view functions. By using some existing reusable apps, I was able to create this blog in a short amount of time while still having more flexibility for modification and extension than using a shrink-wrapped blog app like wordpress would provide. It seems to be a rite of passage for django programmers to write their own blog. I find the idea of doing that to be boring and pointless. Luckily, there's basic-apps. Since there are so many blogs built with Django out there I stole everything that seemed useful from them, like syntax highlighting and the help text for comment formatting.

Development and deployment

Using virtualenv I created a bootstrap script with an after_install function that installs pip and fetches some code from svn repositories.
import os, subprocess
def after_install(options, home_dir):
     subprocess.call([join(home_dir, 'bin', 'easy_install'), 'pip'])
     src = join(home_dir, 'src')
     if not os.path.exists(src):
         os.makedirs(src)
     curdir = os.getcwd()
     os.chdir('src')
     subprocess.call(['svn', 'co', 
      'http://django-basic-apps.googlecode.com/svn/trunk/', 'basic'])
     subprocess.call(['svn', 'co',
      'http://django-trackback.googlecode.com/svn/trunk/', 'django-trackback'])
     os.chdir(curdir)
     target = join(curdir, 'src', 'basic')
     link = join(curdir, 'lib', 'python2.5', 'site-packages', 'basic')
     os.symlink(target, link)
     target = join(curdir, 'src', 'django-trackback', 'trackback')
     link = join(curdir, 'lib', 'python2.5', 'site-packages', 'trackback')
     os.symlink(target, link)

Calling svn in a subprocess and symlinking to site-packages is pretty kludgy. Normally those repository URLs belong in the requirements.txt file. But basic-apps and django-trackback were missing setup.py files when I checked out the source. Maybe pip has a way to deal with that. Maybe I'll get around to look for this in the docs some day.

Speaking of pip, here's how my requirements.txt file looks like. I use rope for code completion in vim. Along the same line, ipython, django-extensions and Werkzeug are useful for development and debugging, but not really neccessary for running the blog.
flup
ipython
rope
markdown
docutils
BeautifulSoup
Werkzeug
pygments
-e svn+http://django-tagging.googlecode.com/svn/trunk/#egg=django-tagging
-e svn+http://code.djangoproject.com/svn/django/trunk/#egg=django-trunk
-e git+git://github.com/django-extensions/django-extensions.git#egg=django-extensions

After creating those files I created a virtualenv and ran pip to install the requirements.
$ mkdir myblog && cd myblog
$ python myblog-boot.py --no-site-packages .
[...]
$ source bin/activate
$ pip install -r requirements.txt


Django project

To glue all this goodness together I performed the following steps.

  • Create django project
  • $ django-admin startproject myblog
  • Edit settings.py, add to INSTALLED_APPS
'django.contrib.admin',<br />'django.contrib.markup',<br />'django.contrib.comments',<br />'django_extensions',<br />'tagging',<br />'basic.inlines',<br />'basic.blog',<br />'trackback',<br />

Here's my urls.py

# -*- coding: UTF-8 -*-
# vim: set fileencoding: utf-8

from django.conf import settings
from django.conf.urls.defaults import *
from django.contrib import admin
from basic.blog import views as blog_views
from basic.blog.feeds import BlogPostsFeed
from basic.blog.feeds import BlogPostsByCategory
from feeds import AllCommentsFeed
from feeds import AtomAllCommentsFeed
from feeds import AtomBlogPostsFeed
from feeds import AtomBlogPostsByCategory
from feeds import AtomCommentsForEntryFeed
from feeds import CommentsForEntryFeed
import views

admin.autodiscover()

rss_feeds = {
    'entries': BlogPostsFeed,
    'full-entries': BlogPostsFeed,
    'categories': BlogPostsByCategory,
    'entry-comments': CommentsForEntryFeed,
    'comments': AllCommentsFeed,
}

atom_feeds = {
    'entries': AtomBlogPostsFeed,
    'full-entries': AtomBlogPostsFeed,
    'categories': AtomBlogPostsByCategory,
    'entry-comments': AtomCommentsForEntryFeed,
    'comments': AtomAllCommentsFeed,
}

urlpatterns = patterns('',
    url(r'^(?P\d{4})/(?P\w{3})/(?P\d{1,2})/(?P[-\w]+)/$',
        view=blog_views.post_detail,
        name='blog_detail'),

    url(r'^(?P\d{4})/(?P\w{3})/(?P\d{1,2})/$',
        view=blog_views.post_archive_day,
        name='blog_archive_day'),

    url(r'^(?P\d{4})/(?P\w{3})/$',
        view=blog_views.post_archive_month,
        name='blog_archive_month'),

    url(r'^(?P\d{4})/$',
        view=blog_views.post_archive_year,
        name='blog_archive_year'),

    url('^$',
        view=blog_views.post_list,
        name='blog_index'),

    url('^archive/$',
        view=views.archive_list,
        name='archive_list'),

    url(r'^categories/(?P[-\w]+)/$',
        view=blog_views.category_detail,
        name='blog_category_detail'),

    url (r'^categories/$',
        view=blog_views.category_list,
        name='blog_category_list'),

    url (r'^search/$',
        view=blog_views.search,
        name='blog_search'),

    url(r'^page/(?P\w)/$',
        view=blog_views.post_list,
        name='blog_index_paginated'),

    url(r'^ping/', include('trackback.urls')),

    (r'^rss/(?P.*)/$', 'django.contrib.syndication.views.feed',
        {'feed_dict': rss_feeds}),

    (r'^atom/(?P.*)/$', 'django.contrib.syndication.views.feed',
        {'feed_dict': atom_feeds}),

    (r'^comments/', include('django.contrib.comments.urls')),

    (r'^admin/doc/', include('django.contrib.admindocs.urls')),
    (r'^admin/(.*)', admin.site.root),

    (r'^dev/random/$', views.randomize),

    (r'^pygments_lexers/$', views.pygments_lexers),
)

I wrote three little view functions of my own, but didn't bother creating an app for them. One is a primitive archive page, the second is a list of all lexers that pygments has installed and the last one is /dev/random. It will return either 4 or NINE NINE NINE NINE NINE NINE. I've linked to stackoverflow so you can see that I even stole that idea somewhere. But at least I improved it by adding the second value. It's actually using the python random module making it a true random device. ;)

For now I can only receive trackbacks/pingbacks, not send them. And I haven't really tried that, so it probably doesn't work either. I'll worry about that later, when there is some evidence that someone is actually reading this blog. The same strategy will be used for dealing with comment spam. ;)

Webserver Configuration

  • lighttpd
  • FastCGI
  • Hint: You might want to add FORCE_SCRIPT_NAME = "" to your settings.py.
TODO:

Yeah, I kinda got lazy in the end. ;) Anyway, that's it. Let's see whether I can come up with something more interesting for the next post.