Thursday, May 24, 2012

Django 1.4 raises PicklingError for Forms with Fields of Type BooleanField

The latest version of Django as of writing this post is 1.4, which contains a regression regarding the picklability of forms that contain fields of type BooleanField. The code for the widget that renders the BooleanField as a checkbox contains a lambda, which is not picklable. There is a fix in the Django github repository, but you might want to use the latest release version for production code.

I recently ran into this issue when trying to store a form object in the session. Regardless of whether this is a good idea, if you find yourself staring at the error below and scratching your head, you might be interested in a quick and dirty workaround.


Traceback:

File "/Users/hs/code/venv_django_fail/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  188.                 response = middleware_method(request, response)
File "/Users/hs/code/venv_django_fail/lib/python2.7/site-packages/django/contrib/sessions/middleware.py" in process_response
  36.                 request.session.save()
File "/Users/hs/code/venv_django_fail/lib/python2.7/site-packages/django/contrib/sessions/backends/db.py" in save
  52.             session_data=self.encode(self._get_session(no_load=must_create)),
File "/Users/hs/code/venv_django_fail/lib/python2.7/site-packages/django/contrib/sessions/backends/base.py" in encode
  79.         pickled = pickle.dumps(session_dict, pickle.HIGHEST_PROTOCOL)

Exception Type: PicklingError at /
Exception Value: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

Patching the Django code in my virtualenv seemed like a terrible idea. So instead I took the fixed code from the current Django development version and put it into a widgets.py file in my app directory. Let's say you have this form:


class FunnyForm(forms.Form):

    name = forms.CharField(label=u"What's your name?")
    is_awesome = forms.BooleanField(label=u'Are you awesome?', initial=True)

All you need to do is put this widgets.py file into your app directory, add
from widgets import PicklableCheckboxInput
to the top of your views.py file and change the widget for the BooleanField in the form like this:


class FunnyForm(forms.Form):

    name = forms.CharField(label=u"What's your name?")
    is_awesome = forms.BooleanField(label=u'Are you awesome?', initial=True,
        widget=PicklableCheckboxInput)


Saturday, April 23, 2011

Emacs vs vi

Last night I watched The Social Network and Tron: Legacy. In the first movie, Mark Zuckerberg uses Emacs for coding some stupid website in PHP. In the second movie, Kevin Flynn uses vi for coding laser control software in C, that transports human beings into the ultimate virtual reality.

I rest my case.

Friday, June 25, 2010

Using flash.utils.Proxy objects inside a DataGrid

Today, I've learned something about Adobe Flex. That's not surprising, since I'm using Flex for the first time. So this is just a quick "note-to-self" kind of post. I was trying to use objects of a subclass of flash.utils.Proxy inside a mx.controls.DataGrid component. Displaying them worked as expected, but I could only select the last row in the list. After spending about half an hour stepping through DataGrid and related code from the Flex framework, I finally got it to work. Maybe this post will spare someone the dubious pleasure of wading through heaps and heaps of frameworky do-nothing-and-do-lots-of-it code. It turns out, that DataGrid uses a unique ID to keep track of the objects it is supposed to display. This ID is usually provided by the runtime. There is even documentation about it. Of course I only found that afterwards. Finding out whether reading this would have solved my problem might have taken just as long... Anyway, here is some code to illustrate the problem and the solution. Apologies for the FooBar nonsense. I couldn't come up with a simple example.

Let's say we have a model object called "FooBar" with two properties, "spam" and "eggs".


package flex_proxy_example
{
    import mx.collections.ArrayCollection;
    
    public class FooBar
    {
        public var spam:String;
        public var eggs:int;


        public function FooBar(spam:String, eggs:int)
        {
            this.spam = spam;
            this.eggs = eggs;
        }
        
        public static function getAll():ArrayCollection
        {
            var result:ArrayCollection = new ArrayCollection();
            result.addItem(new FooBar("foo", 5));
            result.addItem(new FooBar("bar", 23));
            result.addItem(new FooBar("baz", 42));
            result.addItem(new FooBar("spam", 1337));
            result.addItem(new FooBar("egs", 4711));
            return result;
        }
    }
}


We want to display a collection of these in a DataGrid. For some reason we want to massage the value of the "eggs" property before displaying it. This can be done by using a collection of proxy objects as the dataProvider of our DataGrid, instead of the model objects themselves. Flex 3 provides a class to create proxy objects, sensibly called "Proxy". The proper way to use this class is to subclass it and implement (at least) the methods getProperty and callProperty. Here's how our FooBarProxy class might look like:


package flex_proxy_example
{
    import flash.utils.Proxy;
    import flash.utils.flash_proxy;
    
    public class FooBarProxy extends Proxy
    {
        private var foobar:FooBar;
        
        public function FooBarProxy(foobar:FooBar)
        {
            this.foobar = foobar;   
        }
        
        override flash_proxy function getProperty(name:*):*
        {
            if (name.toString() == "eggs")
                return foobar.eggs + 12345;
            return foobar[name];
        }       
        
        override flash_proxy function callProperty(methodName:*, ... args):*
        {
            return foobar[methodName].apply(foobar, args);
        }
    }
}


And here is a minimal Flex application, using the above classes:


<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" layout="absolute" creationComplete="creationComplete()">
    <mx:Script>
        <![CDATA[
            import flex_proxy_example.FooBarProxy;
            import flex_proxy_example.FooBar;
            import mx.collections.ArrayCollection;
            
            private var bunchOfFoobars:ArrayCollection;
            
            public function creationComplete():void
            {
                bunchOfFoobars = new ArrayCollection();
                
                for each (var fb:FooBar in FooBar.getAll())
                {
                    bunchOfFoobars.addItem(new FooBarProxy(fb));    
                }
                dg.dataProvider = bunchOfFoobars;
            }
        ]]>
    </mx:Script>
    <mx:DataGrid id="dg">
      <mx:columns>
         <mx:DataGridColumn dataField="spam" />
         <mx:DataGridColumn dataField="eggs" />
      </mx:columns>
     </mx:DataGrid>
</mx:Application>


There is just one little problem. Selecting anything but the last element of the DataGrid won't work. This is because the proxy objects all produce the same string as their UID. As the above linked documentation suggests, we can add our own UID property to the FooBarProxy class and everything will be peachy. This is how the fixed code looks:


package flex_proxy_example
{
    import flash.utils.Proxy;
    import flash.utils.flash_proxy;
    
    import mx.core.IUID;
    import mx.utils.UIDUtil;
    
    public class FooBarProxy extends Proxy implements IUID
    {
        private var foobar:FooBar;
        private var _uid:String;
                
        public function FooBarProxy(foobar:FooBar)
        {
            this.foobar = foobar;   
            this._uid = UIDUtil.createUID(); 
        }
        
        public function get uid():String
        {
            return _uid;
        }
        
        public function set uid(u:String):void
        {
            // ignored
        }
        
        override flash_proxy function getProperty(name:*):*
        {
            if (name.toString() == "eggs")
                return foobar.eggs + 12345;
            return foobar[name];
        }       
        
        override flash_proxy function callProperty(methodName:*, ... args):*
        {
            return foobar[methodName].apply(foobar, args);
        }
    }
}


I was too lazy to upload the example app somewhere and embed it into this post. It's pointless anyway. The code is available at http://hg.haikoschol.com/flex_proxy_example/src, simply because I have so much free space on Bitbucket. ;)

Sunday, March 21, 2010

FizzBuzz? SRSLY?

I was playing with Visual Studio 2010 and could not resist. Does that mean I'm not a good software developer? Anyway, without further ado, I present to you my ridiculously overengineered, gold-plated version of FizzBuzz in C++. Complete with lazy evaluation via a custom random access iterator. I'm not a C++ guru, so I probably got a lot of stuff wrong. If you take the time to write a comment about my mistakes, it won't have been a complete waste of time. Well, not for me at least.


#include <iostream>
#include <string>
#include <sstream>
#include <iterator>
#include <stdexcept>

class fizzbuzz {
public:
 class const_iterator;
 typedef std::pair<int, int> range;

 fizzbuzz(const range& range = range(0, 100),
    int fizz_num = 3,
    int buzz_num = 5,
    const std::string& fizz_word = "Fizz",
    const std::string& buzz_word = "Buzz")
  : _range(range),
    _fizz_num(fizz_num),
    _buzz_num(buzz_num),
    _fizz_word(fizz_word),
    _buzz_word(buzz_word) {}

 range get_range() const { return _range; }

 int get_fizz_num() const { return _fizz_num; }

 int get_buzz_num() const { return _buzz_num; }

 std::string get_fizz_word() const { return _fizz_word; }

 std::string get_buzz_word() const { return _buzz_word; }

 int get_fizz_num() { return _fizz_num; }

 int get_buzz_num() { return _buzz_num; }

 std::string get_fizz_word() { return _fizz_word; }

 std::string get_buzz_word() { return _buzz_word; }

 void set_fizz_num(int num) { _fizz_num = num; }

 void set_buzz_num(int num) { _buzz_num = num; }

 void set_fizz_word(const std::string& word) { _fizz_word = word; }
 
 void set_buzz_word(const std::string& word) { _buzz_word = word; }

 const_iterator begin() const { return const_iterator(this); }

 const_iterator end() const { return const_iterator(this, true); }

 class const_iterator {
 public:
  typedef std::string value_type;
  typedef std::random_access_iterator_tag iterator_category;

  const_iterator(const fizzbuzz* fb, bool end=false) :
    _fb(fb) {
   if (end) {
    _current = _fb->get_range().second;
   } else {
    _current = _fb->get_range().first;
   }
  }

  const_iterator(const const_iterator& other) {
   *this = other;
  }

  const_iterator& operator=(const const_iterator& rhs) {
   if (this == &rhs)
    return *this;
   this->_fb = rhs._fb;
   this->_current = rhs._current;
   return *this;
  }

  value_type operator*() const {
   if ((_current < _fb->get_range().first) ||
    (_current >= _fb->get_range().second))
    throw std::out_of_range("fizz went buzz(t)");
   return _fb->get_value(_current);
  }

  const_iterator& operator++() {
   _current++;
   return *this;
  }

  const_iterator& operator++(int) {
   _current++;
   return *this;
  }
  
  const_iterator& operator--() {
   _current--;
   return *this;
  }

  const_iterator& operator--(int) {
   _current--;
   return *this;
  }

  value_type operator[](int index) const {
   if ((index < _fb->get_range().first) ||
    (index >= _fb->get_range().second))
    throw std::out_of_range("fizz went buzz(t)");
   return _fb->get_value(index);
  }

  bool operator==(const const_iterator& rhs) const {
   return rhs._current == _current;
  }

  bool operator!=(const const_iterator& rhs) const {
   return !(*this == rhs);
  }

 private:
  const fizzbuzz* _fb;
  int _current;
 };

private:
 bool fizz(int num) const { return num % _fizz_num == 0; }

 bool buzz(int num) const { return num % _buzz_num == 0; }

 const std::string get_value(int num) const {
  if (fizz(num) && buzz(num))
   return _fizz_word + _buzz_word;

  if (fizz(num))
   return _fizz_word;

  if (buzz(num))
   return _buzz_word;

  std::stringstream s;
  s << num;
  return s.str();
 }

 range _range;
 int _fizz_num;
 int _buzz_num;
 std::string _fizz_word;
 std::string _buzz_word;
};

int main()
{
 fizzbuzz fb(fizzbuzz::range(0, 100));
 for (fizzbuzz::const_iterator it(fb.begin()); it != fb.end(); ++it) {
  std::cout << *it << std::endl;
 }
 
 std::cout << std::endl;
 std::cout << "And now for some random access..." << std::endl << std::endl;

 fizzbuzz::const_iterator it(fb.begin());
 std::cout << "it[5]: " << it[5] << std::endl;
 std::cout << "it[15]: " << it[15] << std::endl;
 std::cout << "it[33]: " << it[33] << std::endl;
 std::cout << "it[49]: " << it[49] << std::endl;
 return 0;
}

Tuesday, January 26, 2010

Running Groovy Code from XML Files

For an upcoming Java project, I'm having a look at Groovy. An obvious first opportunity for using it is unit tests. Here's another use case I just made up (that has nothing to do with the project whatsoever ;). Suppose you have two domain objects. One representing a mailing address, maybe stored in a database. The other one representing some kind of form that should be filled from the available data, which includes, among other things, an address. So you might come up with an XML format that maps fields from one object to the other. Maybe one contains a field "city" and the other one a field named "town" and you have something like

<mapping>
    <destination>town</destination>
    <source>city</source>
<mapping>

That's not very exciting. But then you have something like "streetName" and "streetNumber" in one object and "street" in the other one. Now you need to do some simple string manipulation to map fields from source to destination. So, extend the XML! Maybe like this?

<mapping>
    <destination>street</destination>
    <source>
        <combine>
            <field>streetName</field>
            <field>streetNumber</field>
        </combine>
    </source>
</mapping>

Doesn't look so bad, does it? Until you need to do the opposite, split up a field. You end up extending your XML format, slowly implementing a programming language. In XML. Not exactly pretty, IMHO. And lot's of pointless work to implement. What's the alternative? Don't use XML if you need logic! Use a *programming* language! so you write the mappings in Java. This  is very time consuming as well. Suppose you want to generate some of these mappings automatically. Maybe give your users a tool to create them easily, with an intuitive interface. Generating Java code from that might not be such a good idea.

Ok, enough of coming up with reasons and examples why you'd want to do this. Let's just agree, that we want to embed little snippets of Groovy code in an XML mapping document to perform simple transformations of source to destination fields. The scope in which the snippets are run should contain the source object and they should return a string, which is the content of the destination field. Our domain objects might look like this:

public class Address {
    private HashMap<String, String> fields = new HashMap<String, String>();

    public Address() {
        setField("firstName", "");
        setField("lastName", "");
        setField("street", "");
        setField("zipcode", "");
        setField("city", "");
    }

    public Address(
                String firstName,
                String lastName,
                String street,
                String zipcode,
                String city) {
        setField("firstName", firstName);
        setField("lastName", lastName);
        setField("street", street);
        setField("zipcode", zipcode);
        setField("city", city);
    }

    public void setField(String key, String value) {
        fields.put(key, value);
    }

    public String getField(String key) {
        return fields.get(key);
    }

    public String toString() {
     return fields.toString();
    }
    
    public String getFirstName() {
        return getField("firstName");
    }

    public void setFirstName(String firstName) {
        setField("firstName", firstName);
    }

    public String getLastName() {
        return getField("lastName");
    }

// ...

and this:

public class FormWithAddress {
    private HashMap<String, String> fields = new HashMap<String, String>();

    public FormWithAddress() {
        setField("name", "");
        setField("streetName", "");
        setField("streetNumber", "");
        setField("zipcode", "");
        setField("city", "");
    }

    public FormWithAddress(
                String name,
                String streetName,
                String streetNumber,
                String zipcode,
                String city) {
        setField("name", name);
        setField("streetName", streetName);
        setField("streetNumber", streetNumber);
        setField("zipcode", zipcode);
        setField("city", city);
    }

    public void setField(String key, String value) {
        fields.put(key, value);
    }

    public String getField(String key) {
        return fields.get(key);
    }

    public String toString() {
     return fields.toString();
    }
    
    public String getName() {
        return getField("name");
    }

    public void setName(String name) {
        setField("name", name);
    }

    public String getStreetName() {
        return getField("streetName");
    }

    public void setStreetName(String streetName) {
        setField("streetName", streetName);
    }


// yadda yadda yadda

So a mapping with a Groovy snippet in it might look like this:

<mapping>
    <source></source>
    <execute><![CDATA[
        firstName = address.getField("firstName")
        lastName = address.getField("lastName")
        return "${lastName}, ${firstName}".toString()
    ]]></execute>
    <destination>name</destination>
</mapping>

Now on to the actual point of this blog post. How do we execute the Groovy code that we pull out of the XML file as a string? There are several ways to run Groovy code from Java. As far as I know, some of them allow having a restricted security context. If you take arbitrary text from the network and run it as Groovy scripts, you want to make absolutely sure, that it's not possible to call System.exit() or worse (much, much worse ;). Come to think about it, you probably never want to do something like that. Anyway, we'll just look at the one I tried first and found to be working just fine; GroovyShell. For more information about available options have a look at these docs or this blog post.

Say we have a class that does the mapping. It reads the XML file, pulls data out of the source object, puts it into the destination object and when it encounters a non-empty tag, it runs the code inside. For the sake of loose coupling, we'll introduce a horrendously named interface that we can use as parameters for source and destination objects.

public interface FieldsGettableSettable {
    public String getField(String key);
    public void setField(String key, String value);
}

When I made up this example, I didn't realize that it would either require something like this setField()/getField() stuff or reflection. Well, maybe there's another, better way. I didn't think about this all that much. If you're used to coding in Python, you tend to take it for granted to be able to seamlessly go back and forth between strings containing the names of and actual functions/method/members. Gotta get back into the Java mindset I guess. Anyway, a rough sketch of the mapper class:

public class FormMapper {


    public FormMapper(FieldsGettableSettable from, FieldsGettableSettable to,
                      String mappingFilename) {
        // ...
    }
 
    public void performMapping() {
        // ...
    }
 
    private void mapField(String source, String destination, String execute) {
        if ((source != null && !source.trim().equals("")) &&
            (destination != null && !destination.trim().equals(""))) {
            to.setField(destination, from.getField(source));
        } else {
            applyTransformation(execute, destination);
        }
    }

    private void applyTransformation(String transformation,
                                     String destination)
                             throws CompilationFailedException {
            Binding binding = new Binding();
            binding.setVariable("address", from);
            GroovyShell shell = new GroovyShell(binding);
            Object result = shell.evaluate(transformation);
            assert result instanceof String;
            to.setField(destination, (String) result);
    }
}

So there it is, inside applyTransformation(). Shouldn't come as a surprise, since all the documentation and blog posts on the subject contain this code. But since I had to do the legwork of putting this together anyway, I figured I might as well blog about it. Considering how little I've managed to blog at all so far... As a little bonus, you may now laugh at my very first Groovy code that I wrote to generate those domain classes that consist of 120% Pure Premium Boilerplate(tm). It's probably not idiomatic and could be written in half as many lines of code. The capitalizeFirstLetter() function looks particularily cumbersome to me. I just ran with the first thing that worked inside groovysh. The template mechanism that's built into Groovy strings is neat. I probably don't need to toString() calls, likewise in the XML file.


#!/usr/bin/env groovy

def capitalizeFirstLetter(str, result="") {
result += str[0].toUpperCase()
result += str[1..str.length()-1]
return result
}

def generateGetter(fieldName) {
methodName = capitalizeFirstLetter(fieldName, "get")
return """\
public String ${methodName}() {
return getField("${fieldName}");
}

"""
}

def generateSetter(fieldName) {
methodName = capitalizeFirstLetter(fieldName, "set")
return """\
public void ${methodName}(String ${fieldName}) {
setField("${fieldName}", ${fieldName});
}

"""
}

def generateDefaultConstructor(name, fields) {
result = " public ${name}() {\n";
for (field in fields) {
result += " setField(\"${field}\", \"\");\n"
}
result += " }\n\n"
return result
}

def generateFieldsConstructor(name, fields) {
result = " public ${name}(\n"
len = fields.size()
fields.eachWithIndex() { field, i ->
if (i+1 < len)
result += " String ${field},\n"
else
result += " String ${field}) {\n"
}

for (field in fields) {
result += " setField(\"${field}\", ${field});\n"
}
result += " }\n\n"
return result
}

def generateClass(name, pkg, fields) {
code = """\
package ${pkg};
import java.util.HashMap;

public class ${name} {
private HashMap<String, String> fields = new HashMap<String, String>();

"""
code += generateDefaultConstructor(name, fields)
code += generateFieldsConstructor(name, fields)

code += """\
public void setField(String key, String value) {
fields.put(key, value);
}

public String getField(String key) {
return fields.get(key);
}

public String toString() {
return fields.toString();
}

"""

for (field in fields) {
code += generateGetter(field)
code += generateSetter(field)
}

code += """\
}
"""
return code.toString()
}

if (args.size() < 3) {
print "usage: ./GenCls.groovy <Class Name> <Package> <Field 1> [Field 2] ..."
System.exit(1)
}
print generateClass(args[0], args[1], args[2..args.size()-1])

For what it's worth, all of this stuff is available at http://hg.zeropatience.net/groovyinxml/. Ah, right, an example... So we use the mapper class like this:

public class Main {

    public static void main(String[] args) {
        Address address = new Address("Homer",
                                      "Simpson",
                                      "742 Evergreen Terrace",
                                      "0xBADC0DE",
                                      "Springfield");
        
        FormWithAddress formWithAddress = new FormWithAddress();
        
        try {
            FormMapper formMapper = new FormMapper(address,
                                            formWithAddress,
                                            "Address2FormWithAddress.xml");
            formMapper.performMapping();
        } catch (Exception ex) {
            System.err.println(ex.getLocalizedMessage());
            System.exit(1);
        }
        
        System.out.println("\nMAPPED\n\n");
        System.out.println(address);
        System.out.println("\n\nTO\n\n");
        System.out.println(formWithAddress);    
    }
}

And get the following, exhilarating output:

MAPPED


{lastName=Simpson, zipcode=0xBADC0DE, street=742 Evergreen Terrace, firstName=Homer, city=Springfield}


TO


{zipcode=0xBADC0DE, name=Simpson, Homer, streetNumber=742, streetName=Evergreen Terrace,

city=Springfield}

That's all.