Login

Since this is my first post on planet python: Hello !!

A few days ago, I commited a change that enables relatorio to output charts in png. I use PyCha as the plotting module since its graphics looked appealing and the syntax was simple enough to be used by lambda users. Since PyCha does not support negative values, I patched it and while I was working on it I also add a few features that I hope to see included upstream.

The nice stuff about this is that I can finally include a chart into an ODF file without to much hurdle, it works just like an image, the OpenDocument plugin will detect you are using a relatorio report and make the necessary call to create the chart.

This is all good and fine except that I don't like the way I templatize (is that a word) the chart: I use a genshi template of a yaml file like this one:

options:
    background: {hide: true}
    legend: {hide: true}
    padding: {bottom: 10, left: 70, right: 10, top: 10}
chart:
    type: pie
    width: 600
    height: 400
    dataset:
    {% for line in o.lines %}
      - - ${line.item.name}
        - - [0, $line.amount]
    {% end %}

When this file is generated with the relevant data, the options dictionary is transmitted to PyCha and the chart dictionary is then used to create the chart. Going through the yaml step feels like it is unnecessary but I fail to see how I can make this work without requesting the user to learn python.

Lately, I added some new features to relatorio.

• The use of ConTeXt to generate pdf files, it thus remove the dependancy on the trml2pdf that has not seen a release in age. It also shows how you can use a shell command to generate documents.
• Support for the opendocument spreadsheet and presentation files, the help of Udo Spallek has been precious for this. So with relatorio, you can now generate a dump of your data, and use it to construct pivot table (we geeks do not know what it is but believe me it is a key knowledge for every big business consultant), and presentations.

The openoffice change involve a change in the way you add your variables to the template. In previous versions we were using <text:placeholder> tags, but it was not possible to add them through OO UI (and that they get replaced if you add them by hand) in Calc. We choose to replace it by <text:a> tags, the URL must start with relatorio:// and be followed by your directive or expression. As some people have noted, this could create hard to find bugs if the href differs from the content of the tag (ie what is displayed), thus I included a warning in that case. But I think that if you are using the genshi interpolation feature there should be only structural information in the A tags.

Another nice addition to the openoffice templating mechanism, is that now we support inner documents. Thus if you include an opendocument spreadsheet in a text document, the data inside the spreadsheet will be interpreted as well.

UPDATE: finally, relatorio now supports both the old <text:placeholder> syntax and the new one.

Lately I got a lot of help from Cédric of b2ck (another bunch of ex-tiny employees) so that they can replace the openerp way to do things (they are using the deprecated .sxw format) by relatorio openoffice reports.

In order to meet their needs, I complety reviewed the way the openoffice template is written. We work now much more similarly to other genshi templates and it brings some nice goodies :

• generate outputs a genshi Stream that can be rendered as an OpenOffice document,
• testing is way more easy,
• you can apply filters to this stream in order to translate the document,
• styles.xml is now templateable, that way you can make use of openoffice template documents

Cédric also pointed me to bugs in my setup.py and in the way I handled the namespaces from the OO documents. Opening relatorio development really brought us a lot of good ideas and feedback. I am really glad we did this and we will keep doing it at a faster pace then previously planned.

I took some time today and yesterday to switch the relatorio repository to mercurial. The hardest thing in the migration was the Trac plugin that had a SyntaxError and some mess with easy_install.

Next on my schedule: migrating our framework and our client to mercurial. I expect this to be somewhat more difficult since they share the same subversion repository and I'd like to split them in two different hg repository.

In order to play with other to a blind test music game, I created a small pygame script that plays a song and wait for the player to press a button, wait for its response and display the artist/title of the song. It was my first experiment in game programming and pygame made it very easy.

To make it more fun, I decided to use my wiimotes as input devices. Gathering information from the GNU/Linux Port for the Wii and a script from Cadex found on the web I created this piece of code to send wiimote events to pygame.

class Wiimote(object):
    wii_buttons = {'A1300008': 'A', 'A1300004': 'B'}
    def __init__(self, address, event):
        self.event = event
        self.address = address
        self.status = "Disconnected"
        self.receivesocket = bluetooth.BluetoothSocket(bluetooth.L2CAP)
        self.controlsocket = bluetooth.BluetoothSocket(bluetooth.L2CAP)
        self.connect()
    def connect(self):
        self.receivesocket.connect((self.address, 0x13))
        self.controlsocket.connect((self.address, 0x11))
        if self.receivesocket and self.controlsocket:
            if self.event == WIIMOTE1:
                data = "521110"
            else:
                data = "521120"
            self.controlsocket.send(data.decode('hex'))
            self.status = "Connected"
            thread.start_new_thread(self.receive, ())
    def receive(self):
        self.receivesocket.settimeout(0.1)
        while self.status == "Connected":
            try:
                data = self.receivesocket.recv(23).encode('hex').upper()
                if data in self.wii_buttons:
                    event = pygame.event.Event(self.event,
                                               button=self.wii_buttons[data])
                    pygame.event.post(event)
            except bluetooth.BluetoothError:
                pass
        self.receivesocket.close()
        self.controlsocket.close()
        self.status = "Disconnected"

What this class does is creating two sockets to communicate with the wiimote located at address. Once the connection is established, we send a simple command to light the led on the joypad and we create a thread that will receive all the data and post the events comming from the pad to the pygame event loop.

I am currently writing with Beautifulsoup and Mutagen a little python script to parse wikipedia pages and retrieve the released year of an album. Much to my surprise I received an "Forbidden" error code.

What is happening is that wikipedia does not want to allow crawlers. Hopefully I could make urllib use an authorized User-Agent but WTF ? This kind of protection is so easily circumvented that all it does is annoying people.

How to display numbers and monetary value in a locale aware way

>>> import locale
>>> locale.setlocale(locale.LC_ALL, '')
>>> locale.currency(6512.345)
'6512,35 \xe2\x82\xac'
>>> print locale.currency(6512.345)
6512,35 €
>>> print locale.currency(6512.345, grouping=True)
6.512,35 €
>>> print locale.currency(6512.345, grouping=True, international=True)
6.512,35 EUR
>>> locale.format('%#2.3f', 6512.345, True, False)
'6.512,345'

More info in the python documentation for the locale module

Today, I'm very proud to announce the birth of our new project : relatorio, a reporting module able to generate a whole lot of different document types from templates. relatorio is a Portuguese word that means report, I choose Portuguese because the Spanish/French words did not sound good enough.

Technically, we are using genshi and trml2pdf (the homepage does not work anymore) to create PDF and OpenOffice documents.

In this post I'll show you how easy it is to create OpenOffice documents using relatorio. First we need some objects to work on. Let's create a fake invoice object:

class Invoice(dict):
    @property
    def total(self):
        return reduce(operator.add, (l['amount'] for l in self['lines']), 0)
    @property
    def vat(self):
        return self.total * 0.21
inv = Invoice(customer={'name': 'John Bonham',
                        'address': {'street': 'Smirnov street',
                                    'zip': 1000,
                                    'city': 'Montreux'}},
              lines=[{'item': {'name': 'Vodka 70cl',
                               'reference': 'VDKA-001',
                               'price': 10.34},
                      'quantity': 7,
                      'amount': 7*10.34},
                     {'item': {'name': 'Cognac 70cl',
                               'reference': 'CGNC-067',
                               'price': 13.46},
                      'quantity': 12,
                      'amount': 12*13.46},
                     {'item': {'name': 'Sparkling water 25cl',
                               'reference': 'WATR-007',
                               'price': 0.4},
                      'quantity': 1,
                      'amount': 0.4},
                     {'item': {'name': 'Good customer rebate',
                               'reference': 'BONM-001',
                               'price': -20},
                      'quantity': 1,
                      'amount': -20},
                    ],
              id='MZY-20080703',
              status='late')

So we created an invoice for the famous Led Zeppelin's drummer and his favorite addiction.

The next thing to do is to create a template for invoices. We will use the one displayed below. To create the genshi directives, you just need to create a text-type placeholder [1] field, and fill it with the expression you want to use.

You can now start to use relatorio to create John Bonham's invoice.

from relatorio.templates.odt import Template
basic = Template(source=None, filepath='basic.odt')
file('bonham_basic.odt', 'w').write(basic.generate(o=inv).getvalue())

On the first line we import the odt Template engine. This class has the same signature as the one from genshi but uses only the filepath arguments. It returns a StringIO object that can be used to write the report on the disc.

And so here is our invoice with all the fields completed according to the Invoice object we created earlier. Notice how the style we set in the template are also applied in the resulting invoice.

We made use of the py:for directive. But it is not the only genshi directive supported by the relatorio odt plugin, it also support py:if, py:choose/py:when/py:otherwise and py:with.

We also included a simple report repository that allows you to link your reports to a class and retrieve them by the name you gave them or by the mimetype they output.

>>> import relatorio
>>> repos = relatorio.ReportRepository()
>>> repos.add_report(Invoice, 'application/vnd.oasis.opendocument.text',
...                  'basic.odt', report_name='basic')
>>> repos.add_report(Invoice, 'application/vnd.oasis.opendocument.text',
...                  'invoice.odt', report_name='complicated')
>>> repos.reports[Invoice]['basic']
('application/vnd.oasis.opendocument.text',
 <relatorio report on /home/nicoe/tmp/relatorio_demo/basic.odt>)
>>> repos.reports[Invoice]['application/vnd.oasis.opendocument.text']
[('basic',
  <relatorio report on /home/nicoe/tmp/relatorio_demo/basic.odt>),
 ('complicated',
  <relatorio report on /home/nicoe/tmp/relatorio_demo/invoice.odt>)]

Moreover, the report repository works with a TemplateLoader object that automatically reloads a template when the file used to create it has been modified.

When using the report repository, you're also working with the relatorio reports, those are proxies over a Template that enables you to pre-process data before rendering it. The default behavior is to bind your object to the template variable o and all other arguments to the variable args. Here is an example showing you how you could use it in cherrypy to make the request object available to all your reports.

class OHFactory(relatorio.DefaultFactory):
   def __call__(self, obj, **kwargs):
       data = {}
       data['o'] = obj
       data['args'] = kwargs
       data['request'] = cherrypy.request
       return data

I hope you will find this module usefull and abuse it in every way.

I did not notice it but a new release of genshi is available. The team at edgewall already famous for trac is doing a great job to ease the generation of xml templates as well as for text templates.

Lately I had to create a list of the available pages on the blog and I was wondering how to do it with a mix of <py:for> and <py:if>. But then I realized that it was easier to use the marvelous <py:choose> in this way:

<py:for each="pagenum in range(1, max_pages+1)">
  <py:choose test="">
  <li py:when="pagenum==page"><b>$pagenum</b></li>
  <li py:when="pagenum==1"><a href="/blogs${base}?page=1">1</a></li>
  <li py:when="pagenum==max_pages">
     <a href="/blogs${base}?page=$pagenum">$pagenum</a>
  </li>
  <li py:when="abs(page-pagenum)==1">
     <a href="/blogs${base}?page=$pagenum">$pagenum</a>
  </li>
  <li py:when="abs(page-pagenum)==2"><span py:strip="True">...</span></li>
  </py:choose>
</py:for>

This little piece of code will bold the current page, display a link to the first, the last, the previous and the next pages.