Prose practice

Crackbook

2011-08-31T00:21:00.004+03:00

Shortly before starting my job at Google, I had invested a little time into writing a Chrome extension called Crackbook. It helps against wasting time on the web by delaying access to selected sites (see this XKCD cartoon for an illustration, and check out the hint too). Because access is just delayed rather than blocked, you are less likely to just disable the tool after a while, which is exactly what happened to the other tools I tried to use.

Having found a little free time in the evenings, I have just updated the extension. If you use Chrome, please try it and give feedback!

The extension also collects anonymous statistics on what domains people tend to mark as junk and how often the domains are hit. Some very basic results are available at crackbook.info. Now that a fair amount of data has been collected, a deeper statistical inspection is in the plans. Stay tuned.

Amazon Kindle 3

2011-04-02T20:11:00.001+03:00

For the past couple years I had been reading very little, two or three books a year.

For the past month my average has been two books per week.

The culprit is the Amazon Kindle 3 I bought during my stay in the US. I just can not recommend it enough. For starters, it is quite affordable at $139. The image quality is wonderful, and reading in daylight or even in direct sunlight is pleasant. This is a welcome change from a backlit screen as the eyes get noticeably less strained. The format of the device is even more convenient than of a real book (the size is perfect for plain text, start thinking about the Kindle DX if and only if you intend to read PDFs). The Kindle is quite thin and convenient to hold (and you do not need to keep it from closing as with paperback bindings). Battery life is on the order of weeks. Extra functionality (e.g., web browsing) is available, but not convenient, but that is almost an advantage as distractions are kept at bay. The Amazon store is great, I have already shelled out over $50 with little remorse (my time is much more expensive than the measly $9.99 a typical book costs). There are quite a few free books around too. And to top it off, the "screensaver" with pictures of writers is a very nice touch.

The Kindle is also useful for going through longer articles you would normally be reading on your laptop. The Chrome extension Send to Kindle has worked well for me, do check it out.

The most important drawback of the Kindle is that it is slow to turn pages, so it is not easy to browse a book. Turning a page takes a second or so. This is extremely annoying when reading in non-linearly, e.g. taking in study material, when you constantly need to go back to check up details of previous material. A PDF interpreter is available, but the screen is too small to show a full PDF page (a Kindle DX or an iPad would probably be better) and scrolling is painful because of the delays. Wireless seems to drain the battery noticeably, there have been a few cases where I forgot to turn it off and found a dead Kindle a few days later. I wish the WiFi connection would manage itself intelligently somehow.

The Kindle drove the point home for me that dead-tree books will be going the way of vinyl soon, much sooner than I had expected. And for good reasons. Go order now if you do not have an e-book reader yet (and do not forget to order a sleeve too, given the time you will be spending carrying the device it will come in very handy).

Google Munich

2011-03-16T13:57:00.001+02:00

It looks like I have made it through most of the hurdles of the Google hiring process, and I should be starting my new job at Google's Munich office soon. Now I just need to haul my stuff to Munich and get the paperwork done. It sure looks like this is going to be a very interesting year.

Trip to Bay Area coming to an end

2011-03-06T04:37:00.002+02:00

My trip to the Bay Area is coming to the end. It has been surprisingly productive. Here's some things I have done in the nearly two weeks I have been here:

got acquainted with San Francisco
interviewed for a job at a startup
visited Stanford University & Berkeley University campuses
visited Google Mountain View and San Francisco offices
hanged out with folks from Google, from Twitter and from some very promising startups, also a Stanford professor, a researcher from Berkeley and a Berkeley undergrad student, among others
hanged out with people from The Singularity Institute for Artificial Intelligence
spent a day biking in the Golden Gate National Recreation Area

While I am getting a little travel-weary, I know I will be sad leaving this place.

Off to San Francisco!

2011-02-23T04:01:00.000+02:00

The interview at Munich seemed to go well. Well, at least, I had fun. We'll see how that turns out.

For now, I'm off again for another job interview, this time to Palo Alto. This will be my first time to the West Coast, so I have postponed the return flight for two weeks to familiarize myself with the proverbial area. If anyone's from around the San Francisco Bay Area, I would be very happy to meet over lunch/dinner/beer!

Interviewing with Google

2011-02-16T08:21:00.001+02:00

I somehow made it through Google's phone interviews in one piece, and now they are flying me in for an on-site interview in Munich (Germany), which will take place tomorrow. Keep your fingers crossed for me!

I will be landing in Munich in a few hours, and today I am free to look around the city. Anyone from here up for lunch or a beer?

Lithuanian (AltGr) keyboard for Windows

2010-12-19T22:59:00.000+02:00

In GNU/Linux I usually use an AltGr-based keyboard layout for entering Lithuanian characters. That is, you can enter Lithuanian diacritics by holding down AltGr (right Alt)). I had lived with separate modes for entering English and Lithuanian until I found out that having a single layout was a huge relief.

Windows does ship with a Lithuanian layout where AltGr switches to English characters, but I use the English layout much more, so I need it the other way around. So, I grabbed Microsoft's Keyboard Layout Creator and made the layout I needed. The result is available on a GitHub repository. You may also download the installer directly.

In Linux, keymaps, being plain text files, are more transparent, but to be fair, in this case the Windows way wins. Not only do you get a GUI tool to create the layout, but also an installer can be generated automatically for deployment of the layout.

Mobile App Camp

2010-12-19T19:13:00.001+02:00

On the last weekend my team and I participated in the Mobile App Camp (link in Lithuanian) organized by Omnitel, which is one of the dominant mobile operators in Lithuania. The event seems to be part of Omnitel's push for increasing the share of smartphones in the Lithuanian market, which has been lagging behind the European trends.

Our team consisted of Povilas Kytra, who is behind the TV.LT project, Mantas Kanaporis from A-Gain and me. In the weekend we built an app that shows the TV programme for the coming day for all the Lithuanian TV channels. (The app is not yet available on the Android Market, but we are working on it.)

Here's a screenshot of the main screen of the application:

The app was built using the standard Android Development Toolkit on Eclipse. The app gets the content from a Rails-based server containing a simple database and couple JSON views.

I had some experience of developing for Android before, but it was mostly about working with graphics on canvas, while in this app we used some standard GUI controls (with some nifty styling).

For the source control, we used a private repository on bitbucket.org. That one was a huge letdown: a 'hg push' or 'hg pull' would take ages (or at least that's how it seemed to me in comparison to GitHub), and we had no end of trouble with merging, partly due to the number of commands needed to get the repositories in sync (hg pull; hg update; hg merge; hg commit; hg push). Even Subversion would probably have worked better.

The event itself was great fun. It had been a while since I had last coded intensively for the entire weekend. There were quite a few decent ideas presented by the participants, and some of them were successfully implemented.

The rating system was somewhat disappointing though. The event was supposed to be a contest, with four predefined criteria for winning apps (still available on the website): uniqueness in the Lithuanian market, magnitude of the target group, value provided to the user and creativity. In the end, however, the jury nominated apps for three different awards (best app built on an existing database, best app built anew, and the "hope" nomination) with one app awarded from each category. Our app scored second in the first nomination, so we did not get an award, even though we would probably have been in the global top three, were the original criteria upheld.

To be fair, we did not stand much chance against the winner in our category, an app based on vaistai.lt, which sported a database of pharmaceuticals with detailed usage instructions, information about drug stores in Lithuania with maps and inventory status, and even a barcode scanner. Hats off to them. Another winner was "Alaus radaras" ("Beer radar") with locations of local beer bars and inventory info. The third one was "3 milijonai teisėjų" (Three million judges"), which, as far as I understood, was a conception for a basketball-throwing game (basketball is very big in Lithuania, it is a second national religion).

To conclude, it was a fun event and I wish we will be having more of those in Vilnius, even though the Monday after the long weekend was very unproductive.

Windows (III): Packaging & nitpicks

2010-12-07T10:00:00.001+02:00

(This is a part of a series of posts on my recent experience with Windows; see Windows (I) and Windows (II): User Interface.)

Every GNU/Linux desktop distribution hangs on a package management system as a foundation, and as a result the packaging systems are quite advanced, while Windows is stuck with those pesky installation wizards. There have been several stabs to implement a package system for Windows (see here), but none seem to have caught on.

There are three main disadvantages on not having good package management: tedium of manual installation, no dependency resolution and no system-wide automatic updates. In practice they are less important than it might seem. Installation wizards are indeed tedious, but in terms of time I found them to be quick enough to get through. Moreover, apps tend to be less specialized and more encompassing than Linux utilities, and as a result you do not need to install as many things as in Linux. For the same reason dependency resolution is not as important: there are few dependencies in general. As for automatic updates, most high-profile apps have integrated mechanisms to deal with that, however, I found that those mechanisms tended to result in nagging the user with messages. After a while I started ignoring the update notifications. I suspect that happens to many users.

The one thing that Windows has going for it is simplicity. You download installation binaries and you run them, and that's it. Linux package systems tend to be very complicated, and if (when) they break, you will need serious sysadmin chops to deal with the problem. Also installing specific versions (either older ones, or newer ones not yet available in the distro) of specific apps/libraries can be unnecessarily complicated, and the dependency graph limits your actions. In my experience, Windows does well enough without all this complexity as long as you don't reinstall your system too often.

As for the other complaints, I frequently run into the restriction of moving/editing files opened by another application. Coming from Linux, that one is stupidly annoying. Furthermore, it is frustrating not to be able to move inactive (or stuck) background windows. I really miss my unified filesystem namespace. Also there's quite a bit of nagging in general on Windows which I would rather do without. In the end, these are not dealbreakers though.

In my next (and probably final) post I will discuss software development on Windows.

Windows (II): User Interface

2010-12-01T17:13:00.002+02:00

(This is a part of a series of posts on my recent experience with Windows. See Windows (I) for the first post.)

The Windows user interface is definitely acceptable. The charge that it is too colorful or toy-like is completely unfounded. Rather, perhaps geeks should spend less time looking at their gray Motif boxy controls. In terms of speed, the UI is generally more responsive than on Linux, though maybe less so than previous versions of Windows (that could be because my Intel video IGP is a slouch).

An interesting observation is that apps are almost as heterogeneous in terms of interface as in GNU/Linux, even though the standard controls are ubiquitous. Even apps by Microsoft can be separated into different "generations" of UI (e.g., folder windows, Control Panel, Microsoft Office). Also it is clear that generally more attention is paid to UI and usability by app developers, although perhaps not as much as in Apple products.

The greatest asset and the worst offender at the same time, by far, is the overall GUI orientation of the system. Needless to say, it aids discoverability, but reduces scripting capabilities. Problem is, even if you do not need to write scripts per se, command-line actions are useful because they can be repeated and chained very easily, using shell history. In Windows, I occasionally find myself doing much repetitive clicking that would likely be an "Alt-Tab Up Enter" (or sometimes just one key) sequence in Linux. Moreover, the lack of good standartized scripting is a huge pain during app deployments which tend be repetetive. I did not like Windows on servers before, and I do not like it now.

To be fair, my complaints about scripting capabilities may be partly moot because I have only used vanilla Windows batch files, and have not looked at all into Windows PowerShell, which is a new-generation scripting tool. The examples are fairly impressive. The language has a nice look and is clearly powerful, but also looks somewhat complicated which keeps me away until I find a good reason to learn it.

Initially I missed workspaces. The one app I found that was supposed to emulate workspaces would take several seconds to switch the workspace, so it was completely unusable. I learned to do without them rather quickly though. Actually I'm starting to think that using many workspaces is a sign that you are doing too many things at once and also they are an invitation to distraction (how many of you have a "blog reader" workspace right aside your "work" workspace, ready at your fingertips at a moment's notice?). Workspaces do have one killer feature: you can jump directly to a given workspace instead of cycling through windows. This is very useful when working with more than two apps, in which case without workspaces you are forced to think about the morphing Alt-Tab queue when switching windows.

Speaking about UI generations, I much prefer the ribbon toolbars of the new Office. The still-prevalent toolbars with zillions of old-style 32x32 (or 16x16) toolbar icons are really ugly. Typically only a few of the toolbar buttons are actually useful, and they make the user interface unnecessarily cramped and busy. The move to fewer and larger toolbar buttons is definitely on the right track.

The explorer context menus also tend to grow crazy long. Every app wants to get in there, and in the end you have a context menu that takes up half a screen vertically. Needless to say, most of those items are not used much. Sure, you can opt out during app installation, but at that time it is difficult to say how useful the context menu item will be, and there's no easy way (i.e. easier than just ignoring the cruft) to remove the entries afterwards.

In my coming posts I will cover package management and application development on Windows.

Windows (I)

2010-11-25T06:15:00.003+02:00

I have been helping with an overdue web app project lately, and the project uses ASP.NET MVC, so I have had to use Windows for most of my day. Since I had been using GNU/Linux almost exclusively since 2003, this is quite an adventure (imagine trying out GNU/Linux after a seven year break). It is not nearly as bad as I thought it would be, even though I am using the not-so-well-liked Windows Vista which came pre-installed with my computer.

The first thing that struck me is how much less important the OS is when compared to the situation a few years ago: a lot of the applications I depend on are web-based (GMail, Google Calendar, Google Docs, Springpad), and the ones that are not web-based are multiplatform and installable with a few clicks (GVim comes to mind). In fact, I do not think there is a single piece of software that I really missed in Windows. (An interesting corollary of this observation is that it should be equally easy to migrate to GNU/Linux nowadays.)

The one thing where the operating system matters is hardware support, and I do not think I have to say much here. After years of whack-a-mole GNU/Linux upgrades which would fix support for one component but break another (for things as essential as video and sound), it was absolutely weird having all hardware work properly at the same time (sound after suspend! no video glitches! high quality webcam picture!). There is no point recounting the reasons why hardware support in Linux is as it is, but now I am starting to think that Linux users do tend to suffer from a sort of Stockholm syndrome regarding hardware support; I know I did.

The other important thing that operating systems do is resource management, and I did not have problems here either. In fact, I liked the virtual memory behaviour much better: when Windows ran out of memory (I had filled up the smallish Windows partition and there was no space left for the swap file to grow), it started rejecting malloc calls from applications, but the system was kept alive and responsive, and I could close apps and clean up the mess without much trouble. In Ubuntu, with the default settings, whenever a runaway process ate too much RAM, the system would start thrashing so hard that it would be very difficult to fix. My standard procedure (switch to text console, log in, find and kill app using command line tools) would take minutes and was so annoying that I would sometimes just reset the computer instead. And in the end, even if I did not reset, an important app would possibly be killed anyway. I know there are kernel parameters for tweaking this behaviour (swappiness?), but I never got around to figuring them out as this situation does not arise that frequently, and testing is painful.

I did not like the Windows FS security model. ACLs are much more complicated and thus less transparent that the Unix approach, even though they are more powerful. My main gripe with them is that I do not know a good way to display them easily (as with ls -l in Unix). I have not had to deal with them much though.

I have had no stability issues. It seems that this argument should be put to rest.

Long blog posts are annoying, so I will leave the user interface, the developer experience and my complaints with the system for separate posts (edit: you can now read my next post about the UI).

Laptop trouble

2010-11-21T01:18:00.001+02:00

My trusty somewhat-old laptop has been under the weather lately. It will not boot when it's cold, and it's fairly picky. ~20 °C seems too cold already; so I have to heat it up at little.

Now that winter is coming, this needs to be done almost every morning, and that is extremely annoying. Does anybody have an idea of what could be wrong? It seems that the problem is not just a loose connection, as the laptop goes up in stages:

1) when the power button is pressed the power LED will blink for a split-second and nothing will happen,
2) the LED will light up for slightly longer, but the laptop still will not boot,
3) the laptop starts booting but shuts off spontaneously after while on the BIOS screen,
4) the laptop goes through the BIOS screen, but shuts down before showing the boot menu.
Past that, apparently the CPU generates enough heat for stable operation.
This is fairly complicated behaviour for a hardware problem. I wonder, maybe the power supply is at fault? Any ideas, anyone? This is a good excuse to buy a new laptop, but I would rather not splurge right before Christmas season.

Verdi's Requiem

2010-11-15T01:22:00.001+02:00

The choir I sing in (Pro Musica) will be performing on January 22nd in Bremen, Germany. Together with two local choirs we will perform Verdi's Requiem, which is absolutely fabulous. See for yourself:

(Also try Solti's version for better sound quality, but no video.)
It's going to be a busy month preparing for the concert. Verdi's melodies are not complicated, but he's crazy with dynamics: I think it's the first time I've seen a quintuple pianissimo (ppppp) or a quadruple fortissimo (ffff) in sheet music.

Living a monastic life for 10 days

2010-11-03T22:55:00.000+02:00

About a month ago, at the end of August, I finished a 10-day Vipassana meditation course. This was probably one of the more extreme activities I have ever participated in, and it's well worth a blog post.

I found out about the course in a ridiculously trivial way, by following a link in a Reddit comment (I wonder if I could track down that comment...). The website referenced by the comment contained (almost) no new-agey mumbo-jumbo and looked legit in general. I also appreciated the fact that this was a relatively serious and intensive course, which should be enough to form a reasonably founded opinion on meditation in general. When learning sporadically, in small bits, it is hard to say whether the material itself is no good, or you just do not understand it yet and need to study more; I was quite skeptical towards the practice, but wanted to give it fair trial. I was quite impressed that there was a course available in Lithuania, a few hours by train from where I live. Finally, the course was free (by tradition Vipassana is taught without monetary compensation). I went ahead and signed up for a standard introductory 10-day course.

Vipassana (also called insight meditation) is a meditation technique allegedly coming back from Gautama Buddha himself (well, all the traditions like to claim that origin...). It has been experiencing a renaissance, partly due to an organization set up by a meditation teacher S. N. Goenka which has founded meditation centers all over the world.

The essence of Vipassana is a method of teaching your mind to act instead of merely reacting to outside stimuli such as pain, anger of other people or hardships. The technique provided is to monitor your sensations, watch them with equanimity and suspend any reaction, and slowly it sinks in that all things are temporary and you can choose what to react to and how. That's the basic idea; I don't want to go deep into exposition of Vipassana as there is plenty of information about that (also at the same time the information is almost useless because you have to experience it to really understand it). Instead, I will sure some of the more personal remarks and experiences.

The course took place in an cottage near Kretinga, housing around 50 people. The living conditions were absolutely great, and this would have been a true holiday if not for the regime and the rules. The day would start very early, at 4 AM (ending at around 9:30 PM), and we'd do about 12 hours of meditation in total inbetween, with occasional breaks, but the most gruesome part is that no communication at all was allowed with anyone except the manager of the camp (and the teacher during a specified time). That gets to you after a few days, but still, there are people around you, so it's bearable; on the other side, the senses (touch, smell) become very acute. The food was taken care of by volunteers (students who have finished the course), and I could say it was quite good actually. The goal, of course, was to keep the environment from interfering with meditation, and I think it was done very well.

The presentation of the actual technique also did not disappoint. Each day a new step would be introduced, with the whole day to practice before moving on. Looking back, the pacing worked well. Every day in the evening there were lectures on theory. They were somewhat informative, but also sometimes annoyingly primitive and verbose. The little mistakes, holes in logic and slight doses of mumbo-jumbo almost drove me nuts by the middle of the course, until I learned not to react to them. Despite the shortcomings of presentation, the simplicity and lucidity of the practice and lightness of theoretical baggage really appealed to me. The whole organisation in general gave the impression of complete transparency, with no strings attached, no financial interests in play and no agitation for further involvement.

While there are many meditation techniques, I am happy that I've gone with Vipassana in particular for one reason: Vipassana meditation uses no artificial tools to help you concentrate, such as mantras, visualisations, counting, etc. Such methods make concentration much easier, but they also create a dependence on the methods themselves. I appreciate the purity of Vipassana in this regard.

Not everything in the practice is roses though. First, a particular worldview supported by Vipassana is presented as "scientific" and supposedly "the right way to live", and if even if you don't agree, "the truth" of Vipassana will become obvious to you with time. This is a distinctive mark of ideologies. The thing that makes everything break is the reincarnation, which is generally assumed as true, even if it's not shoved down the student's throat. Every rationalization of a world view has a "white lie" (usually related to the concept of eternity somehow) at the core, and in this context reincarnation is it. Given what we know now about the relationship between the mind and the body, reincarnation is absolutely ridiculous and completely untenable, and that breaks all "moral" reasoning for practicing meditation as opposed to doing something else. I'm not the first to bring up this problem, there's plenty of material about it online.

The second slippery idea is that the concept of "negative happiness" is central in Vipassana, that is: happiness is defined as lack of suffering. This point is not that unreasonable actually, from personal experience I could indeed call the state of non-suffering happiness, but there is definitely something missing when compared to the state of mind when I'm truly, positively happy. Whether you consider that something important or not, it's definitely there.

Third, Vipassana suggests that it is only you who is at fault when your inner state is disturbed; the world affects you as much as you let it, and in order to find happiness you should learn to control the "interface" rather than the outside world, which will usually not behave as you want. This may be correct, but sometimes it could just be easier to fix the world rather than change yourself, and I have the suspicion that it this is much more frequently the case in the modern world compared to the pre-modern environment. (This is one of the points which break when reincarnation is rejected, as with reincarnation you have the whole eternity to fix yourself.)

While the complaints are serious, they do not mean complete rejection of Vipassana but rather integration among the various other techniques available. The actual meditation technique is still great for gaining and maintaining mental composure, reducing anxiety and improving well-being. Now, if I could only get used to meditating regularly...

If you have the chance, I highly recommend you try out an introductory 10-day course. There is little to lose and a lot to gain. And if you do go ahead, I have two hints. First, do not even think about leaving the course before it's over. Any hardships are temporary and will be overcome quickly. Second, use the time with the teacher, those interactions were very helpful to me. Also, given the "noble silence", talking to someone once in a while will not hurt.

Python course projects

2010-07-29T02:33:00.009+03:00

This spring I read a course on Python in Vilnius University, Department of Mathematics and informatics. The course was not mandatory, which I suspect is one of the reasons why my students were quite a bright and motivated bunch. I began the course with a brief introduction to the Python language and the standard library, then dedicated a few lectures to Django and in the remainder I covered various libraries (GUI toolkits, Pygame) and techniques (testing, debugging, optimization).

One of the major tasks of the course (and the deciding factor in the final valuation) was to develop a project using Python. The students were given complete freedom over the topic of the project and the tools used, as long as Python was involved. The quality of the result varied, of course; had I given concrete tasks, the low bar would definitely have been higher. However, I am happy with how things turned out.

Probably the best-executed project was 1000 Online, an online multiplayer card game (see rules). It was developed by Vytautas Karpavičius. Vytautas apparently had prior experience with Python and Django, nevertheless, the quality of the app (and the code behind it) is impressive. Among the other projects, Gegute.lt, a Lithuanian clone of Reddit is still up and running. It's nothing too fancy, but looks quite clean for a university project. Other projects perhaps were not perfectly executed, but were very interesting conceptually. Jonas Keturka developed a bot (BitBucket) for the online strategy game Travian. Marius Damarackas developed a web-based guess-the-song game. He used YouTube for the song material and Last.fm (IIRC) to make you guess among similar songs, so that it would not be too easy. Andrius Chamentauskas developed a Puyo Pop clone (GitHub) with PyGame, complete with multiplayer and an AI. Paulius Budzinskas worked towards his course paper by developing a bacteria movement modelling and visualisation app (GitHub; Lithuanian only). David Abdurachmanov developed a spell-checker bot for WikiAnswers.com, with a really baroque and sophisticated architecture. I believe he has applied to Google Summer of Code with that project; I wonder how that is working out.

For reference, you can find links to all student repositories on GitHub/BitBucket in this list. Beware: some of them may be documented/commented in Lithuanian.

Another interesting effect of the course was that people started using Python in other related courses, for example in Mathematical Modelling and Numeric Methods. I even heard that the lecturer was impressed and expressed interest in learning Python. Another lecturer, this time a statistician, was interested in teaching and using SAGE at lab practice. Feeling such 'ripples' going through the department sure felt nice.

By the way, the university also offers lectures on Ruby, read by Saulius Grigaitis. I took some inspiration from him for the course form and content.

Riga marathon

2010-05-24T19:30:00.001+03:00

I participated in the Riga marathon and ran the full 42km! The time (4:39) isn't great at all, but I'm happy to have finished at all!

Android apps

2009-12-16T21:52:00.003+02:00

I've recently gotten myself an Android phone. The Android app market is not as mature as iPhone's App Store, but there is a lot of applications, and plenty of chaff to separate the wheat from. I found most of the apps I use by browsing "top Android apps" lists on the web, so I am putting up a similar list here in case someone finds it useful.

Everyday apps
- Astrid Task/Todo List - a great To-Do list app that integrates with Remember The Milk
- Quick Calendar - an "upcoming events" widget
- CardioTrainer - nice jogging app, records routes using GPS, integrates with music player
Language tools
- Thesaurus Free - a thesaurus
- Free Dictionary Org - reference dictionary
- QuickDic German Dictionary - an English-German-English dictionary
Media content
- Listen - an app to find audio content online
- Mother TED - search and view TED talks
- TuneWiki - music, song lyrics
Games
- Lumbricidae - guide a worm using your accelerometer
- Flying High - plane flying (also accelerometer-based)
- miniMatcher - memory game
- Iconic Memory - another memory game
Cool apps
- Phonalyzr - pretty phone call (and SMS) stats
- Shazam - identify a song by recording a short snippet; possibly one of the coolest apps available
- SnapTell - look up product information by scanning a barcode (or even by snapping a photo!)
- Backgrounds - background images
- Google Sky Map - shows constellations
- Layar Reality Browser 3.0 - a fancy-looking Augmented Reality app
Music
- Sonorox - tune composition for everyone (the pentatonic scale is great, isn't it?)
- Loops! Lite - another music composition app
- Tuner - gStrings - a guitar tuner
Miscellaneous
- Task Manager - a process monitor utility, useful for checking up on app resource usage
- Currency - a currency converter
- Sleep Logger - a very simple, but useful app for tracking your sleep habits

Piano improv #2

2009-09-28T01:44:00.003+03:00

Here's a recording of a recent improvisation. It's very simple, but I'm quite happy with it.

Lęšis.lt

2009-05-14T01:15:00.003+03:00

A friend of mine has recently launched a contact lens e-shop, lesis.lt. It only ships in Lithuania at the moment, but if you're based here, have a look, you might find it useful.

Piano improv

2009-04-15T15:35:00.002+03:00

My today's experiments on piano:

Peer reviewed blogging

2009-03-04T17:20:00.002+02:00

Here's an idea: what if there was a website where you could upload a topical blog post, which would then be reviewed and edited by experts in the area and then posted on that website? Such a process would ensure that articles are at least well-named, appropriately tagged, well-structured and not methodologically deficient, and could result in a database of articles of decent quality. Would you upload your blog posts that are juicier in terms of content to such a site? Would you volunteer to do reviews and editing?

Hacker spaces

2008-12-26T22:14:00.003+02:00

I just found out about hacker spaces. For a nice example of a hack space, see The Metalab. Such places seem quite suitable for work on individual public projects and for growing teams by sorting out the wheat from the chaff.

It's a pity there are no hacker spaces east of Vienna. I wonder if one could be set up in Vilnius. There probably are enough people around. but quite a bit of organising work (see Hackerspace Design Patterns [PDF]) needs to be done to establish a hacker space. I would do it, but I am overbooked as it is. Any takers?

Python 3.0

2008-12-04T15:22:00.003+02:00

Python 3.0 (also known as Python 3000) has been released yesterday, so I thought I'd give it a try. It turns out that a recent release candidate is in official Ubuntu 8.10 (intrepid) repositories, so I did not even need to compile anything: apt-get install python3.0 and we're done. You may want to read the summary of changes from Python 2 beforehand.

Here's a quick demo of unicode identifiers:

>>> 二 = 2
>>> 四 = 二 + 二
>>> 四
4
>>> import math
>>> π = math.pi
>>> α = π/3
>>> math.sin(α)
0.8660254037844386

This isn't quite APL yet, but we are getting closer!

Silly translations

2008-11-28T23:01:00.003+02:00

Since I do some work on software translation, I have seen my fair share of silly translations (in my case from English to Lithuanian). Here's my latest find: in Gmail there is a choice of chat list for every person in the address book, and there are four options: Auto, Always, Never, and Block. The Lithuanian translation of the option "Auto" is ... "Automobiliai" which means "Cars". Took me a while to figure out what happened the first time I saw it. This ranks right next to the already classic wizzard button translation "Kitas", "Kitas", "Suomių" for "Next", "Next", "Finish" (translated back to English that would be "Next", "Next", "Finnish" – as the language of Finns) and my personal favourite: Finnish translated as "Finikiečių" (Phoenician).

FWIW I tried to apply for access to Gmail translations (there are many more inconsistencies, mistakes and screwups in there), but it seems that Google is not interested. I wonder if they have any translation quality control at all.

Computer-generated music

2008-11-21T23:55:00.002+02:00

EMI (Experiments in Musical Intelligence) by David Cope changed my mind on the state of computer music generation software. I had come across numerous "music generation" programs, but it would have been a long stretch to call their results "music". They would produce, at best, strange patterns of beeps. I had heard of a live "jazz improvisation gig partner" machine (probably this one), but EMI is on a completely different level – just listen to the MP3 samples. Sure, in the end it is mostly splicing and rehashing of existing pieces, but Cope claims that in blind tests even experienced listeners would sometimes confuse an original piece by a composer and a composition by EMI. In other words, EMI passed the equivalent of a Turing test for music generation software. If that is not impressive, I do not know what is.

Unfortunately David Cope's site is skimpy on the details of how EMI works, even though there is an extensive bibliography (note the time span: 20 years!). At least some information can be gathered from the webpages of his numerous books.

Book meme

2008-11-20T22:33:00.002+02:00

I am a bit of a bookworm, so I could not resist this meme:

Grab the nearest book.
Open it to page 56.
Find the fifth sentence.
Post the text of the sentence in your journal along with these instructions.
Don't dig for your favorite book, the cool book, or the intellectual one: pick the CLOSEST.

The book is actually a Lithuanian translation, but I'll translate the sentence into English for you:

After entering the palace, he seduced the king's wife, killed the king with her help and seized his throne.

Can you guess which book the sentence is from?

Non-native LCD resolutions

2008-11-14T14:55:00.003+02:00

Now that LCD monitors have spread, I keep noticing one thing: lots of public places with computers have the resolution set to some non-native value. Blur in cafes, libraries, everywhere! It seems that people just do not notice the blurriness, and I do not really blame them – how could they know that there's a better resolution? This makes me think that it would be a good idea to warn the user on the desktop environment configuration level about reduced quality whenever the screen is an LCD and they are trying to switch to a non-native resolution. A non-intrusive, but informative message could do the trick, I think.

Python syntax highlighting in LaTeX

2008-09-13T15:49:00.004+03:00

LaTeX provides a listings package to typeset code. The standard output is great for black-and-white printouts, but not for beamer presentations, where a more dynamic color scheme would be suitable. Having recently worked on slides with Python code, I have prepared a file that you can include to get nice colourful code that looks good on a presentation:

My code is based on a snippet found on Ubuntu forums. I added some more keywords and tweaked the color scheme to correspond to the one that IDLE uses (that color scheme is not ideal, but my audience uses IDLE for coding and I wanted some consistency).

To use the package, copy the file to the same folder as your document and include it in the header:

\include{pythonlisting}

Then, to render code, use python sections:

\begin{python}
def foo():
    print "bar"
\end{python}

Note that if you want to use this in a beamer frame, you will have to mark the frame that contains Python code as fragile, like this:

\begin{frame}[fragile]{Some Python code}

\begin{python}
def foo():
    print "bar"
\end{python}

\end{frame}

Better Russian-English dictionaries

2008-09-13T15:22:00.003+03:00

A reader called Kuno notes that there are now some good Russian-English dict-format dictionaries available online. They are very likely better than the ones I extracted a couple years ago. The dictionary files suggested by Kuno can be downloaded directly, fetched with rsync (rsync rsync://dictd.xdsl.by/dicts/) or the server can be queried with a dict client (try "dict -h dictd.xdsl.by -D" for a list of available dictionaries).

Buffalo buffalo

2008-08-18T20:03:00.002+03:00

"Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo" is a gramatically correct sentence! See Wikipedia for an explanation.

IronPython on Ubuntu

2008-05-30T01:56:00.010+03:00

After reading on Jimmy Schementi's weblog that IronPython, the .NET implementation of Python, is just an apt-get install ironpython away, I thought I'd give it a whirl. Here's what I got.

Huh? Maybe Microsoft took that "significant whitespace" thing a little too literally...

It turns out that IronPython really wants you to use a dark background (bug report).

Ah, so that's why the background of Jimmy's blog is black!

As you can see from the screenshot, it's also a bit difficult to quit from the Ubuntu version of IronPython. That weird symbol is a result of pressing Ctrl+D. Even Ctrl+\, which usually helps in such cases, only produces a nasty .NET CLR traceback with no ill effects to the interpreter.

On the bright side, this is a good start. IronPython seems to be more or less functional, with speed comparable to CPython, even though the example code included in Debian looks a little "heavy" (see /usr/share/doc/ironpython/Tutorial).

Europanto

2008-05-07T23:41:00.003+03:00

This made my day! Here's an excerpt if you're too lazy to click on the link:

"Als humanos, tambien nations, after seine geborn, become bambinos und grow adult. Aber als humanos, sommige nations never become adult und necessite psychiatrico help", dixit once Sigmund Freud aan Herr Kanzelier Otto Von Bismarck attemptante de le convince que Germania eine tumultuosa puberty was traversante.
Otto Von Bismarck ignored die wise parolas des fader des psychanalyse und wat happened aan Germania next, wir know mucho well.
Die freudiana diagnose noch todag applicable und validissima est, by exemplo, por wat Grosse Britannia betreft. Eine nation die encore play mit kleine soldatos und make carnaval parades out des season, mature est suremente nicht.
Wat pushe flegmaticos aged britannicos de dresse in hooliganicos costumes und parade mit tambouros und trompettes in der nordirlandico landscape? Sicheremente eine mucho neglected mentale desease.
Eine people waar hombres can wel dress in skirts, waar animals habe more rights dann mensen, waar man drive op de wronge side des strada, must echtemente seriose problemas van personnality habe. Eine people die nicht only play cricket und darts, aber les regarde tambien op TV, bastante insane must definitamente esse.
(...)

There are more texts in this, er, language, on the Europanto main site.

Edit: fixed link

Calibrating the keyboard autorepeat setting

2008-04-19T00:55:00.002+03:00

The default keyboard autorepeat setting on most systems is boringly slow. Even worse, the delay before autorepeat kicks in is so long. This is great for novice users, but we programmers deserve better. Here's a calibration procedure that I use:

The autorepeat delay should be as short as possible, but a concentrated, damp but short keypress should not cause multiple keypress events.
To determine the most suitable autorepeat rate, test your accuracy. Open an editor and try to enter multiple lines of about 50 characters using the autorepeat feature. Pick the highest setting with which you can consistently get the line length within 2-3 characters of the target. (The 2-3 character margin comes from the practice of releasing the autorepeated key a little early and entering the remaining 1-3 characters, if any, with separate keystrokes.)

For me an autorepeat delay of ~250ms and 80 char/s rate works well. To get your numbers, you can use the command xset q | grep 'repeat rate' .

This tweak make the standard direction keys (and particularly the backspace & delete keys) much more useful: you can frequently get away without using special editor commands or, say, having to select things to delete them and thus resetting the X buffer. It makes things a little faster and more responsive. Try reverting to the standard setting after a few weeks of fast repeat to feel the difference.

Perl program 'design'

2008-04-13T13:59:00.005+03:00

Acme::Eyedrops allows you to 'design' your Perl programs (if you have any; you'd better not). Basically, given a Perl program and a "shape" file, it produces a perl program that visually resembles the given shape and behaves like the former program. Here's my favorite example:

    ''=~(        '(?{'        .('`'        |'%')        .('['        ^'-')
    .('`'        |'!')        .('`'        |',')        .'"'.        '\\$'
    .'=='        .('['        ^'+')        .('`'        |'/')        .('['
    ^'+')        .'||'        .(';'        &'=')        .(';'        &'=')
    .';-'        .'-'.        '\\$'        .'=;'        .('['        ^'(')
    .('['        ^'.')        .('`'        |'"')        .('!'        ^'+')
   .'_\\{'      .'(\\$'      .';=('.      '\\$=|'      ."\|".(      '`'^'.'
  ).(('`')|    '/').').'    .'\\"'.+(    '{'^'[').    ('`'|'"')    .('`'|'/'
 ).('['^'/')  .('['^'/').  ('`'|',').(  '`'|('%')).  '\\".\\"'.(  '['^('(')).
 '\\"'.('['^  '#').'!!--'  .'\\$=.\\"'  .('{'^'[').  ('`'|'/').(  '`'|"\&").(
 '{'^"\[").(  '`'|"\"").(  '`'|"\%").(  '`'|"\%").(  '['^(')')).  '\\").\\"'.
 ('{'^'[').(  '`'|"\/").(  '`'|"\.").(  '{'^"\[").(  '['^"\/").(  '`'|"\(").(
 '`'|"\%").(  '{'^"\[").(  '['^"\,").(  '`'|"\!").(  '`'|"\,").(  '`'|(',')).
 '\\"\\}'.+(  '['^"\+").(  '['^"\)").(  '`'|"\)").(  '`'|"\.").(  '['^('/')).
 '+_,\\",'.(  '{'^('[')).  ('\\$;!').(  '!'^"\+").(  '{'^"\/").(  '`'|"\!").(
 '`'|"\+").(  '`'|"\%").(  '{'^"\[").(  '`'|"\/").(  '`'|"\.").(  '`'|"\%").(
 '{'^"\[").(  '`'|"\$").(  '`'|"\/").(  '['^"\,").(  '`'|('.')).  ','.(('{')^
 '[').("\["^  '+').("\`"|  '!').("\["^  '(').("\["^  '(').("\{"^  '[').("\`"|
 ')').("\["^  '/').("\{"^  '[').("\`"|  '!').("\["^  ')').("\`"|  '/').("\["^
 '.').("\`"|  '.').("\`"|  '$')."\,".(  '!'^('+')).  '\\",_,\\"'  .'!'.("\!"^
 '+').("\!"^  '+').'\\"'.  ('['^',').(  '`'|"\(").(  '`'|"\)").(  '`'|"\,").(
 '`'|('%')).  '++\\$="})'  );$:=('.')^  '~';$~='@'|  '(';$^=')'^  '[';$/='`';

The meaning of code

2008-04-05T01:17:00.004+03:00

I had an "Aha!" moment today when I came to a Wittgensteinian realization that while the sense of a piece of code corresponds to the formal behaviour of that code, the meaning of the snippet is the set of possible contexts where this code could be used. This idea has changed my view of teaching programming (I have been teaching Python in a local school for a couple months now).

This attitude is nothing new, but for some reason I have not come across it, or rather, I have never seen it applied. (The fashionable problem-based learning thing has some similarities, but isn't quite it.) Having an idea to start from, it was not hard to find good articles expanding the idea. I especially liked Robert Strandh's short writeup on idioms and Elliot Soloway's Learning To Program = Learning To Construct Mechanisms And Explanations (1986). Experienced programmers will not learn much, but anyone dealing with CS education, teachers and students alike, ought to have a look.

The moral is that emphasis should be placed on reading exemplary code ("experienced programmers do X this way") rather than definitions ("function Y does Z"). Programming seems to be closer to language use than to mathematics, and teaching methods should reflect that.

"Home" toolbar button on Firefox 3

2008-03-29T15:22:00.003+02:00

The "Home" toolbar button seems to have been removed from the default configuration of Firefox 3. Moreover, it not trivial to get it back as it is not available in the "Customize toolbar" list of optional buttons either. It turns out that to add it you have to drag it from the bookmarks instead.

Reverting to packaged translations on Launchpad

2008-03-25T21:43:00.002+02:00

It turns out that I did a poor job of looking for information on how to revert to packaged translations on Launchpad Translations. There is a bug report (#107737) on the very same issue with some useful information. Interestingly, the German team which reported the problem was even contemplating reverting all local Launchpad translations.

Furthermore, there is already a high-priority Launchpad blueprint for reverting translations of whole packages. There is no news on its implementation, but at least the idea is there.

Pianofiles

2008-03-22T02:16:00.003+02:00

pianofiles.com, a sheet music exchange site which I had mentioned two years ago, has gotten a nice facelift. It used to be very slow and neither very pretty nor convenient. Most of these shortcomings have now been corrected. It's a great place to look particularly for modern tunes which, unlike classical works (which are usually public domain anyway), do not have a canonical version, but heaps of arrangements and transcriptions of live performances. Quality of course varies, but point is that you can get your hands on many versions and compare them. Typical latency of requests is about a day. For example, just yesterday I fired off a few emails, and today I have got three versions of "Nardis" by Bill Evans. Feel free to ask for any sheets I've got.

Launchpad translations revisited

2008-03-21T01:39:00.004+02:00

I am trying to add some order to the mess that is the Launchpad translation repository. Problem is, strings translated just once through Launchpad stick around forever, even if you change them in upstream later. In effect, a lot of translation bugfixes were not reaching Ubuntu users. I thought I had that covered. Unfortunately it's not the case.

I just noticed that in the Launchpad package list sorting by changed strings only sorts the first page! Initially I had noticed random package names and mistakenly thought that the whole package list was sorted. The whole list (~1500 packages, shown in 75 package chunks) is sorted by "importance", so it's basically random. Good luck finding modified packages in that haystack.

Seeking computer-literate monkeys.
Earn up to five bananas per hour!
Must be able to use mouse.

Unless... how about increasing the batch size? Turns out that in Launchpad the batch size is not capped, so by changing a URL parameter you can just set it to a high number, wait a few short moments, and get the complete list of packages. Sort that by changed strings and we're done, right?

We're done finding the local translations, but not fixing (reverting) them. The new, larger list reveals that there are more than a few of those with quite more than a few strings that should in 99% of the cases be reverted to packaged versions. Here's how that works:

Monkey operator instruction manual

Click on 'Packaged' radio button for all 10 entries on page (no batch size modifications allowed here...)
Click 'save'
Repeat until the page says 'No more translations'
Done. Not! When you click the 'Save button', the starting offset is changed for the next page. However, the translations you just marked are removed from the list at the same time. So, if you were looking at translations 1-10, after pressing 'Save & Continue' you will actually be looking at translations 21-30, not 11-20! To get to the 'lost' translations, you have to retrieve the list from the start again. So, load the initial page and go to step #1 to fix the remaining half of translations. Next time you reach this step, half of those will be left. Next time, half of half of those. Isn't that fun!

Where's that monkey ad again?

A brute force solution could be to write a script. That could save some bananas. Not that we could run out of them as this whole thing is pretty much bananas. That does not look like a clean solution though. Does anyone know if there there is a better way?

Trickle the userspace traffic limiter

2008-03-20T01:38:00.003+02:00

trickle (available in a package repository near you) is a neat little tool to limit network bandwidth of a particular app. It's very useful for those situations where you don't want your connection gobbled up, but the application in question does not implement traffic limiting. For example, to run a dist-upgrade capped at 30 KB/s, run this:

trickle -s -d 30 apt-get dist-upgrade

(Care to guess what I'm doing now?)

Note that trickle does not seem to work across the sudo boundary: don't do trickle -s -d 30 sudo foo, do sudo trickle -s -d 30 foo .

Launchpad translation synchronization

2008-03-12T23:46:00.003+02:00

Ubuntu package translation storage in Launchpad used to annoy me, primarily because the synchronization workflow between Launchpad and upstream was unclear. My questions have been partly answered on Launchpad Answers. Others working with translations in Launchpad might be interested in this information.

Here's another tip for translation coordinators. I used to worry about translation pollution in Launchpad, until I found that you can clean up efficiently by sorting packages by the number of translations changed locally in Launchpad (to try it out, you can go here and click twice on the "Changed" column). Click on the number of changed translations in a given package and you will be presented with a list of the changed translations, which you can reset to upstream versions one by one by choosing "packaged" versions. It seems to be a good idea to perform such a cleanup once in a while, to keep the amount of cruft to a minimum.

Opening files quickly in Vim

2007-09-15T23:37:00.000+03:00

My Python development environment consists of a GVim window for editing code and a gnome-terminal for running the unit tests. This works out well, but leaves some integration to be desired. In particular, jumping to a test failure point is a little cumbersome. I used to do it this way: copy the filename from the traceback, switch to vim, then type :e, paste the filename with middle-click (now I know I can use * instead) and press Enter, then enter the line number and press G. That is barely acceptable; this particular wart has even contributed to pushing a ex-co-worker towards Emacs (the horror!).

I have written a small vimscript function to make the aforementioned use case slightly easier. Now to jump to a failure point, just triple-click the corresponding line in the traceback (that selects the whole line), switch to Vim and invoke the function, which scrapes the clipboard for a filename and line number and opens it in the active buffer.

Installation: paste the OpenFileFromClipboard function in your ~/.vimrc and bind it to a key by adding the following lines to your .vimrc:

nmap  :call OpenFileFromClipboard()
imap  :call OpenFileFromClipboard()

These commands bind the key F4 to our function. The first command handles normal mode, and the second handles insert mode (hence the to momentarily escape the insert mode). You can of course change the shortcut to whatever you like.

The function should handle following types of references from the clipboard:

/etc/passwd
John saved his work to ~john/doc.txt and left.
File "/home/gintas/devel/project/tests.py", line 123, in tests.py
/usr/include/stdio.h:333:extern int sprintf (char *__restrict __s,

I hope someone finds this useful.

Localization coordination for Debian #6

2006-08-14T02:47:00.000+03:00

With the deadline approaching, it appears that my productivity is rising. Seems like the typical efficiency=1/days_left (and at T=0 my brain explodes).

Basically I'm happy with what I have for DDTP. It will need some testing to replace the current system. One thing that I would like to have is package priorities: this has some support in the old system. I am not sure how to approach this yet, but I'm sure I'll think of something. Anyway, this is not a priority and will probably be settled after the deadline.

Priorities for next (and last) week are to get the relational backend running and to write a guide to the API. I have already started work on the RDB backend. I am using SQLAlchemy as the wrapper. A nice side-effect is that SQLAlchemy supports SQLite, which means that getting the thing to run would be a snap, without requiring fancy configuration of MySQL or PostgreSQL. On the other hand, having these options allows better scalability as the database and the web server(s) can be separated. As for the guide, I am aiming for a style similar to library documentation on python.org. I expect no trouble here because I already have some experience with L^AT_EX.

Last but not least, I have to remember to clean up quite a few leftover TODO and XXX markers in the things that I have. Most of them are benign, however.

Localization coordination for Debian #5.5

2006-08-10T03:22:00.000+03:00

I have been working on getting the DDTP to work with Pootle. Import from DDTP translation files works fine now (although it takes a while) as well as export to Translation-?? file format. I have also created a layer to split the translations up in groups by name, to avoid humongous multi-megabyte .po files.

More information and plans for next week coming up tomorrow.

Firefox tip-of-the-day

2006-08-01T22:37:00.000+03:00

A small tip for Firefox: use mouse gestures for basic operations. They are provided by the mouse gestures extension. I am no big expert here, as I only use three gestures: go back (left), go forward (right) and close tab (down,right). Nevertheless, having these around is already very convenient, because they are so simple and so frequently needed. I usually do browsing with (surprise) my mouse and these gestures streamline the process because now I don't have to care about the toolbar (and the close tab buttton in particular) any more.

If you are only interested in the simple gestures, I found that it helps to disable diagonal gestures: in the extension's properties, Additional Settings tab, set diagonal tolerance to 0 percent. This will disable some advanced gestures but I never wanted those anyway, so it reduces the chance of my gesture being incorrectly recognized. I also disable the New window (down) gesture ("General" tab, "Edit gestures" button) because it would sometimes be activated instead of close tab when the "right" in my "down, right" motion were too small.

Mouse gestures, like anything, need some time to get used to, but after a while they require much less effort than seeking out a button. In a few days you will learn that very little motion is required for the gesture to be recognized. For other applications usually the keyboard is actively used, so keystroke shortcuts would be faster, but for web browsing in particular, mouse gestures fit the bill very well in my opinion.

By the way, Opera supports mouse gestures natively.

In other news, my work on Debian l10n project was mentioned on Debian Weekly News. I found it very funny that they linked to Worldforge, which is an open sorce MMORPG, instead of WordForge, the localisation project. Admittedly I had made this mistake myself previously, and unfortunately WordForge has a much lower profile than WorldForge. Oh well, real news publishers do this kind of stuff all the time :)

Localization coordination for Debian #5

2006-08-01T03:22:00.000+03:00

There have not been updates for an extended period of time as I have taken a completely unplanned and unexpected holiday. Now I'm back on track.

Work I have accomplished since the last update:

updated TranslationUnit and the .po parser. In the end I decided that it's not worth making it too different from Pootle's pofile. Comments are now dealt with and the header is treated specially.
merging templates. I defined an interface for objects able to merge translation stores and provided a simple implementation. Pootle already has an implementation of this too, I'll try to encapsulate that as well.
help with move to Subversion for the Wordforge project. The Subversion repository is already operational at https://svn.sourceforge.net/svnroot/translate/trunk, although it's read-only at this moment.
some small shufflings in the API. Generally the defined interface are holding out fairly well new code, so only small modifications are needed.

Here's what I want to do next week:

Wire this thing up to DDTP. Should be easy, I just need to ask about a good way to fetch DDTP translations. Probably this will be a download-only demo in the beginning.
Same for debconf l10n templates.
Come up with a translation review and approval model. This is one of the cornerstones of the API and may be tricky to get right. I think that it is essential to have a sort of a 'diff' tool working at translation unit level (much like ordinary 'diff' works on text). To get usable diffs several copies of the template (upstream & local) should be stored. With this in place it shouldn't be hard to implement pushing of new DDTP / debconf translations upstream.
further updates to the API as flaws and missing features pop up.

Localization coordination for Debian #4

2006-07-08T17:05:00.000+03:00

This was a week of prototyping.

I performed some more basic changes to the API. I introduced folders so that arbitrary tree-like grouping structures would be possible. I also moved the language layer below the module layer (previously it was above, like in pootle), because it eliminates parallel hierarchies in the whole structure. It also just appears more logical to me. While the bottom-most TranslationUnit class needs more work until it's complete, I hope that the container interfaces will not need to be changed much more, as there would be more and more work to update the corresponding implementations.

I have written a prototype .po parser that uses Wordforge's tools to parse a .po file and converts the result into structures defined by the new API. Admittedly it's at a very early stage. There is also a preliminary implementation that can read Pootle's backend storage directory structure. It is read-only at the moment, and does not read Pootle's metadata files yet, but it is usable. There's also an XML-RPC server that demonstrates this backend: you can send XML-RPC requests to translate words in specified Pootle projects.

After having dealt with containers this week, next week I will probably concentrate on the actual translation objects and look deeper into structure of XLIFF to ensure decent compatibility. I would like to accommodate most of what Wordforge currently supports in this regard. I will also probably start thinking about the security subsystem (I have a feeling that this will spur a few discussions on the mailing list).

I have also tried to gently push Wordforge to migrate to Subversion. Sourceforge has its own migration facilities, so hopefully the transition will be easy and painless. CVS has been a little of a pain, and now I noticed that it's actually threatening me!

...
revision 1.1
date: 2006/07/08 12:06:43;  author: gintautasm;  state: dead;
branches:  1.1.2;
...

There was some interest on the Wordforge mailing list about using the XML-RPC interface to co-operate with thick clients (KBabel, etc.). For the moment I have enough work on Pootle itself and XML-RPC is not a priority, but if anyone would be interested in implementing the client side for some translation application, I would be very happy to cooperate and coordinate the server's XML-RPC interface.

Hopefully next week the weather will be a bit cooler too: it's so hot that it is difficult to fall asleep because of the heat. The heat definitely does not help to get actual work done.

Localization coordination for Debian #3

2006-06-30T17:57:00.000+03:00

This time the weekly report is late, sorry about that. I spent the better part of the week without access to the internet.

I did have my computer with me, and I have worked a bit on the Pootle backend API, which was received without major complaints on the Pootle mailing list. There were some very slight changes to the interfaces. Most importantly I whipped up a simple proof-of-concept implementation of the interfaces. That revealed a few more small problems, but now I am confident that the interfaces are consistent and can actually be implemented. This demo implementation is not persistent, but it might be useful as a base for other implementations and in tests, or it could be trivially made persistent with use of pickle.

The current API and the implementation are in the Pootle CVS repository (see here).

Next, I will attempt to wrap .po file classes that Pootle currently has. Implementing basic functionality should not be very difficult there, especially with recent changes by David Fraser.

Localization coordination for Debian #2

2006-06-20T02:48:00.000+03:00

I sent a new revision of the suggested translation storage API to the mailing list, addressing all the shortcomings that have been pointed out. Hopefully it will not change much any more and I can start working on an implementation this week.

My plan is to first produce a very simple pickle-based implementation of the API. This should be relatively quick to build and would highlight the weak points of the API. I think that this can be finished this week. After that (and possibly revisions to the API) comes the important part: writing a nice wrapper for the existing pofile class. This should not take too long either, another week at most (especially since my exams would already be over).

When a .po-based implementation is in place, the fun starts. I'd then like to try to have another application that concurrently uses Pootle's backend storage (currently po-based). Another week of work.

Localization coordination for Debian #1

2006-06-13T18:18:00.000+03:00

I have been working on an integrated l10n infrastructure for Debian. Instead of starting from scratch, the Wordforge project was chosen to be used as the base. Currently Wordforge has a toolkit for various l10n operations as well as a web frontend Pootle that uses this toolkit. My job is to adapt these tools to Debian needs.

I see three major areas of my future work in Wordforge:

scalability: importing all translations of all packages in Debian into Wordforge as it is would require a very powerful computer. I believe that a planned transition to a relational database as the backend of Wordforge (instead of plain old files) will help immensely here.
frontends: there is only one frontend to the system, Pootle. Debian will need an e-mail interface, an XML-RPC interface, a Subversion interface and possibly others in addition to the web interface. This is more difficult than it sounds because Pootle is intertwined with the backend. Work is underway to define a clear API for the backend that could be used by various clients.
process: Debian translators have strict translation review and ownership processes. These would have to be incorporated into Pootle.

A (to me almost distressingly) large amount of discussions took place on the debian-i18n and Wordforge mailing lists on these topics and more as I dug into Wordforge to better understand the current situtation and plans of other developers. After the initial communication peak, now things are really starting to roll. Here's what we have now:

A general concensus for a move in Wordforge towards a relational database. This took quite a bit of convincing ;)
Agreement on the ideas of a new backend API
A sketch (in code) of the new API

I posted the sketch of the API to the Pootle mailing list. There was little opposition and mostly constructive remarks. Unless a disagreement pops up, I would expect the API design to be finished by next week.

My longer term plans:

write an implementation for the new API (a week)
cover the current file-based backend under the API (difficult to estimate)
migrate Pootle to use the new backend (difficult to estimate)
write a new frontend or two: e-mail, XML-RPC (a week)
write a backend based on a relational database (a week)

One thing that is still keeping me down a is a couple of exams due in a week. After that I should have much more time to work on this project.

z3reload moved to the Zope 3 base

2006-05-16T00:12:00.000+03:00

A small project of mine, z3reload, which implements very basic view code reloading, has moved to the Zope 3 Base. Hopefully that will make it a bit more visible. Zope 3 is already fragmented as is, some consolidation can't hurt.

Russian to English dictionaries on Linux

2006-05-01T00:49:00.001+03:00

Update (2008-09-13): the dictionaries in http://dictd.xdsl.by/dicts are probably more complete than the ones provided here, see this post for more information.

It is very convenient for me to have a local dictd server set up, not only to look up obscure English words and jargon, but also for translation from foreign languages. Dictd is quite fast and powerful, with a multitude of dict clients to choose from. There are some pretty good free dictionaries too. For German, the dictionary in the Debian (Ubuntu) package dict-de-en (German-English + English-German) is fairly good. For Russian, the Mueller English-Russian dictionary (mueller7accent-dict) is great.

Unfortunately, the Mueller dictionary does not contain translations from Russian to English. I have been using ksocrat for that purpose, but KSocrat is a little buggy and its usability does not shine. I looked around on the web for a downloadable Russian-English dictionary in dict format, but surprisingly could not find anything useful. There are quite a few online translation sites, heaps of small DOS-era utilities and some home-made Windows stuff, but nothing that would be easy to feed to dictd. What I did find was Sdictionary.

Sdictionary is a dictionary application that has a fairly large community. Its Russian-English dictionary with 300 000 words was more than adequate for my purposes, but the database format was incompatible with dictd, and I couldn't find a converter on the web, so I wrote one.

The converter from Sdictionary to Dictd formats is a bash script that dumps an Sdictionary file to a plain text file and massages it a bit so that it can be fed to dictfmt. To use the script you will need to install PTkSdict from swaj.net (Debian package) and the Ubuntu packages dictfmt and dictzip. Then just drop the script in the same directory as some dictionary foo.dct and run ./sdict2dictd.sh foo (where foo is the name of the dct file). The script should produce foo.dict.gz and foo.index which you can install into /usr/share/dictd. The script is far from perfect and discards some formatting but it's better than nothing. The best solution would probably be to extend dictfmt so that it could read the Sdict dump files directly.

If you just want a dictd Russian-English dictionary, I have uploaded the files converted from Sdict here: rus_eng_full.dict.dz (8MB), rus_eng_full.index (15MB).

By the way, when running in daemon mode, dictd eats an unreasonable amount of RAM (it's taking 40MB resident memory on my machine). If the dict server is only for personal use, it makes sense to run it through inetd as described in /usr/share/doc/dictd/README.inetd.gz. It appears that the speed difference is not noticeable on modern computers.

Kalba ir kalbininkai

2005-12-22T15:35:00.000+02:00

Sorry, English speakers, this post is in Lithuanian; writing it in English would be an oxymoron.

Neseniai užtikau internete tipišką žmogaus, nusistačiusio prieš informatikos terminijos lietuvinimą, puslapį. Tokią nuomonę turi nemažai žmonių, ir kai kurie mėgsta apie tai garsiai pašnekėti internete ar forumuose, tuo tarpu iš kitos barikadų pusės tokių aktyvistų gerokai mažiau. Pabandysiu apginti (nebūčiau Gintautas!) kalbininkų darbą.

Pirmiausia pulsiu lengviausiai paneigiamą teiginį — kad kalba nesvarbi tautiniam identitetui ir kad ji savaime nėra vertybė. Manau, kad vienas iš dalykų, apibrėžiančių tautą, yra jos žmonių mąstysena, o kalba yra tos mąstysenos atspindys. Esmė ta, kad ryšys abipusis — kalba savo ruožtu veikia mąstymą. Nors mintys ir nėra rišlūs sakiniai, vis vien remiamės kalba. Ši idėja tiksliau išreikšta Sapiro-Vorfo hipotezėje. Paprasčiau kalbant, pabandykite įsivaizduoti vokietį, nekalbantį vokiškai. Arba prancūzą, nekalbantį prancūziškai. Japoną, nekalbantį japoniškai. Kiną? Suomį? Indą? Italą? Pagaliau anglą? Manau, kad tai pakankamai stiprus argumentas, kad kalba yra vienas iš pagrindinių, jei ne pats pagrindinis tautiškumo pagrindas ir labai svarbi kultūros dalis (tai, beje, argumentas, kodėl kalbos paskirtis nėra vien informacijos perdavimas). Savimonė taip pat svarbu, be savimonės tauta negali egzistuoti, tačiau savęs supratimas nebūtinai susijęs su tautiškumu. Žmonės kartais save apgaudinėja — jiems atrodo, kad jie skiriasi nuo kitų, nors iš tiesų tų skirtumų labai nedaug.

Manęs visai nežavi idėjos, kad visas pasaulis kalbėtų viena kalba; gerai, kad tai nelabai įmanoma. Būdas su žmonėmis susikalbėti jau egzistuoja senų seniausiai: tiesiog reikia mokytis užsienio kalbų. Aišku, tai nėra paprasta, tačiau ne tik suteikia galimybę bendrauti, tačiau ir apskritai praplečia akiratį.

Dar įmesiu akmenį į airių daržą. Gal anglų kalba jiems ir atnešė ekonominį klestėjimą investicijomis iš užsienio, tačiau kartu atkeliavo ir nelabai gerų dalykų. Jei neklystu, Airija buvo viena iš didžiausių programinės įrangos patentų propaguotojų. Viena iš priežasčių — Airijoje plačiai įsikūręs Microsoft galėjo tampyti politikų virveles ekonominiais argumentais. Taigi airiai praranda ne tik kalbą, bet ir galią savo pačių valstybėje.

Dabar galime pereiti prie, atrodytų, universaliai nekenčiamų kalbininkų. Tie dykaduoniai tik nieko nesuprasdami išradinėja naujus žodžius ir kiša nosį ne į savo reikalus, ar ne? Ne. Mano supratimu, kalbininkų tikslas — išsaugoti lietuvių kalbos grynumą. Manau, kad tai nėra kvailas tikslas, nes lietuvių tauta nėra didelė, ir todėl kitos kalbos (anglų ir rusų) nesunkiai daro įtaką. Aš didžiuojuosi savo kalba, ir nenorėčiau po 50 metų turėti lietrusių ar anglietuvių kalbos. Kalbininkai stengiasi neįsileisti visiškai netinkamų skolinių todėl, kad jei įleidžiame keletą, tada visi pamato, kad taip galima, ir nebelieka priežasties neįleisti ir daugiau. Taip kalba gali pasikeisti smarkiai per trumpą laiką. Gal tai atrodo kaip ėjimas prieš žmones, bet mano galva, žmonės tiesiog linkę rinktis paprastesnį sprendimą nemąstydami apie ilgalaikes pasekmes, kurias svarstyti ir yra kalbininkų (ir apskritai valstybės) pareiga, panašiai kaip rūpinimasis aplinkos tarša.

Nesutinku su populiaria nuomone, kad kalbininkai nesugeba atlikti savo darbo. Pirmiausia, jie yra savo darbo specialistai, ir aš mieliau palikčiau vertimą jiems, negu kokiems nors inžinieriams. Imant pavyzdį iš programų kūrimo konteksto, jei reikia parašyti verslo programą, juk darbą atlieka programuotojas, o ne verslininkas. Programuotojo pareiga yra išnagrinėti dalykinę sritį ir tada dirbti savo darbą. Verslininkas gali galvoti, kad jis daug geriau parašytų programą, nors tai dažniausiai netiesa. Apskritai, dažnai verslininkas net nežino, kokios programos iš tiesų nori. Lygiai taip pat, kalbininko pareiga yra išnagrinėti dalykinę sritį ir tada dirbti savo darbą. Programuotojas gali galvoti, kad jis daug geriau susitvarkytų su terminais, nors tai dažniausiai netiesa. Apskritai, dažnai programuotojas net nežino, kokių terminų iš tiesų nori.

Teisybė, kartais kalbininkai sugalvoja keistų, gal net juokingų žodžių. Kartais jie iš tiesų nepataiko, bet man atrodo, kad dauguma sugalvotų terminų pakankamai geri kasdieniam naudojimui. O žodžių juokingumas yra labai miglota savoka. Pabandykite lėtai, nesigilindami į prasmę, ištęsdami, pasimaivydami ištarti bet kokį paprastą žodį, pavyzdžiui, „piešti“ arba „šluota“. Man po tokios procedūros vos ne bet koks žodis atrodo juokingas, ir ne dėl to, kad juokingai ištariamas -- tas keistas tarimas reikalingas tam, kad būtų galima atskirti žodį nuo jo tiesioginės prasmės ir suvokti tik garsą ir ryšius su kitais žodžiais. Daug naujų žodžių gali atrodyti juokingi dėl deminutyvų („skreitinukas“) arba moteriškos giminės („derintuvė“). Jie atrodo juokingi todėl, kad neįprasti. Čia bėda ta, kad anglų kalba daiktams nepriskiria giminės ir neturi deminutyvinių formų, todėl tokie vertimai į lietuvių kalbą (mes įpratę prie svetimžodžio klijuoti vyriškos giminės galūnę be jokių priesagų) atrodo keisti. Tuo tarpu kasdieniuose žodžiuose tokios formos gana dažnos („mikrobangų krosnelė“, „degtukas“, „mentelė“). Štai ir pavyzdys, kaip skoliniai skurdina kalbą.

Neprotinga šnekėti, kad kalbininkai nori pakeisti profesinį žargoną. Tarpusavyje bendraujantys profesionalai greičiausiai kaip ir šnekėjo, taip ir šnekės pusiau angliškai, pusiau lietuviškai (deja, ir man tenka taip daryti; mielai pabandyčiau naudoti lietuviškus terminus, jei neprieštarautų bendradarbiai). Iš tiesų sprendžiamas svarbus klausimas: ką daryti, kai reikia parašyti apie dalyką knygą, arba išdėstyti medžiagą universitete? Aš manau, kad mano lietuvių kalbos jausmas neblogas, ir man labai nemalonu rasti sudarkytų skolinių profesionaliame tekste. Manau, kad daug lietuvių jaustųsi panašiai.

Galiausiai norėčiau pastebėti įdomią vidinę prieštarą. Dažnai žmonės, besipriešinantys šiems naujiems terminams, šiaip yra progresyvūs, domisi naujomis technologijomis, gyvena modernų gyvenimą ir laiko save atvirais kitų nuomonei. Tuo tarpu priešinimasis šiems terminams, mano supratimu, yra labai konservatyvus veiksmas: „Aš jau naudoju šį terminą tris mėnesius, todėl prašom jo nekeisti, ir apskritai, jūsų alternatyvos man atrodo juokingos.“ Alternatyvos juokingos vien dėl to, kad tai ne tas pats skolinys (kuris, beje, „aiškus“ turbūt todėl, kad akivaizdus ryšys su anglišku atitikmeniu). Pamažu tie žodžiai įeis į bendrąją kalbą ir bus visiškai natūralūs.

Hassle-free IPv6 connectivity

2005-11-02T23:37:00.000+02:00

I have recently discovered that nowadays it's very easy to get IPv6 access if you're using Debian (or Ubuntu):

apt-get install tspc

Try ping6 www.kame.net, it should work out of the box. A 6to4 tunnel should have been automatically established to Hexago, which hosts www.freenet6.net. You don't even have to register for an account, although it might be a good idea to get one if you are really planning to use IPv6, as the anonymous tunnel broker is rather slow (I had ping times on the order of 300-500ms), and you will probably want a statically allocated address space. Moreover, a tunneling protocol based on UDP which supports NAT punching is now available, so you do not need a public IP address to connect. This situation is orders of magnitude better than the one when I tried to set up IPv6 on my router at home a couple years ago.

At the moment there is not much you can do with your new shiny IPv6 connection IPv6 is still "basking" in obscurity, there's not much to see on the IPv6 web yet: dancing kame, some stats about fellow IPv6 users... Nothing really practical. However, given the trivial setup procedure, I'm sure some uses could be found. One that comes to mind immediately is SSH access. Since the tunnel can punch NAT's, even machines behind routers can be reached without any trouble. While 300ms lag is not pleasant, there is also some security through obscurity to be gained - who would bother scanning 128bit IP addresses? Of course, if you completely disable SSH on IPv4, a tunnel failure could be very unpleasant. There is still some sense having both running. For example, you can restrict SSH access to several static IPv4 addresses that you usually use; you can connect from external sites using IPv6. Besides, it's quite nice to have the ability to fix those nasty firewall setup slips when you lock yourself out by accident (that has happened to me several times).

While IPv4 is not giving way to IPv6 yet, a fair amount of software and hardware already supports the new standard. A concise overview of the advantages can be found at NetBSD IPv6 FAQ.

Design by Contract in Python

2005-10-16T13:53:00.000+03:00

In the light of recent lectures on Design by Contract in the university I decided see if there are any nice Python implementations.

To begin with, Design by Contract is a technique to increase reliability of programs, and in particular for reusable components. Invented by Bertrand Meyer more than ten years ago, it is a central notion in the programming language Eiffel that he created. There is a nice introduction to Design by Contract available. I'll try to summarize it very quickly.

Basically, in the context of software engineering a contract is a collection of obligations for a component. These obligations are divided into three major categories:

Class invariants are supposed to be (almost) always valid. For example, the attribute ISBN of a class Book should always be valid according to the ISBN checksum algorithm. A possible invariant for a class Window would be "not (window.maximized and window.minimized".
Preconditions are input conditions, they validate the arguments of a method or function call. For example, a precondition for the function sqrt(x) could be that its argument must be non-negative (assuming that it cannot deal with complex numbers). The caller of the function is responsible for supplying arguments that don't violate preconditions.
Postconditions are result obligations. They validate the result of a function/method and state changes of an object. An example of a postcondition for sqrt(): "abs(result * result - x) < 0.001". Postconditions can also access attributes and ensure that they have (or have not) been changed appropriately.

DBC might not look very pleasant for pythoneers at first glance, because it makes programs more rigid. Besides, unit and integration tests should catch all the problems, right? Well, not quite. I have experienced numerous bugs caused by violations of informally (if at all) described invariants. The problem with unit tests is that when you find a bug where one component (the "client") uses another component (the "server") incorrectly, you can fix the client and test for regressions in that client, but you cannot ensure that other clients have the same problem. Well, you can add assertion statements on the server, but I'm not particulary fond of assert statements as they clutter up the code and they are not suitable for checking a set of invariants in multiple locations. A nice thing about DBC is that, unlike static typing, which is an all-or-nothing affair, you can go only as far as you wish.

Preconditions in particular can be very beneficial to beginners. I have heard at least several people complain about dynamic typing in Python because they were frustrated by mysterious errors coming from the depths of a large framework which turned out to be a result of incorrect API usage. Comprehensive preconditions can eliminate such problems much better than any static typing system.

In Zope 3 applications that I worked with, a fair number of assumptions was supposed to hold at all times, they were mostly mentioned in interface docstrings, but only occasionally checked in actual code, and certainly not systematically. We've had a fair share of problems with objects referencing other objects that should have been deleted ("hanging in the air") and with processing inconsistent data structures. In relational databases, such requirements could be checked using constraints and triggers, but I am not aware of similar mechanisms for ZODB. In the end we bolted on a component that would swoop through all the objects in our application and check various things, but this component had to be invoked externally, using a cron job or manually pointing the browser at a particular URL. However, this was still far from perfect, because the component was separate, so requirements for components were not localized adjacent to the corresponding code. Using the Zope 3 component architecture would help with that, but it would increase the overhead for adding checks.

A possible complication with DBC in Python is the performance hit taken by all the extra checks. For compiled languages it's not nearly as bad as in Python where running unit-tests already takes minutes for larger applications. However, Python has always favoured convenience and doing the right thing over performance, so, given the benefits, this is not against the ideology at all. I do not like the current situation in zope.interface where if you want something checked against the interface, you have to run the check manually.

Of the Python implementations of DBC, I liked Contracts for Python most. In fact, its author Terence Way even proposed to include DBC in Python (see PEP-316), but the PEP was deferred. The general idea of the implementation is to declare the contract in docstrings, similar to doctest. This is much more lightweight and convenient than the other approaches which require to inherit from special classes (or set metaclasses) and/or define new special methods. Here is an example of a contract from the implementation's homepage:

def sort(a):
    """Sort a list *IN PLACE*.

    pre:
        # must be a list
        isinstance(a, list)

        # all elements must be comparable with all other items
        forall(range(len(a)),
               lambda i: forall(range(len(a)),
                                lambda j: (a[i] < a[j]) ^ (a[i] >= a[j])))

    post[a]:
        # length of array is unchanged
        len(a) == len(__old__.a)

        # all elements given are still in the array
        forall(__old__.a, lambda e: __old__.a.count(e) == a.count(e))

        # the array is sorted
        forall([a[i] >= a[i-1] for i in range(1, len(a))])
    """

There is not much point in reiterating the concise documentation found in the package. You can find more examples on the "Contracts for Python" homepage.

It is unfortunate that the implementation is a bit stale, last touched more than two years ago as of today. I managed to find a trivial bug (patch) related to importing packages to be processed, but otherwise it seems to be still working fine. I also tried to make it work properly with Zope interfaces, so that implemented interfaces are treated as superclasses. Problem is, interfaces are not quite ordinary classes, so there were some complications. I might look further into it if anyone is interested. By the way, speaking about Zope interfaces, they already include support for invariants (see zope.interface.invariant), although I prefer the "Contracts for Python" way. Furthermore, preconditions and postconditions are not supported.

Zope 3 views reloaded

2005-08-20T21:21:00.000+03:00

One of the few things I dislike most about Zope 3 is the time it takes to start the server, which becomes an annoyance when running functional tests or when tinkering with view code and checking the results in the browser. It's so refreshing sometimes to work with page templates where the a browser reload is enough. I would frequently think how useful it would be if at least the views behaved the same too.

A little bit of peeking in Zope 3 code and a little bit of coding, and we've got a package called z3reload (tarball) that does exactly that, reloads the view code before each render. This only really works for views, but in my experience views tend to be the largest and most complex part of the code, so writing them consumes the bulk of the time programming, and they need manual testing most (unit tests are usually enough for the model classes).

Installation of z3reload is relatively simple: just drop some files in Zope3 package-includes, and specify which views to patch in z3reload-configure.zcml. Be sure to look through the README file.

z3reload might come in handy to speed up functional testing iterations as well, if you use the script runfdoctests.py by my co-worker Marius Gedminas (I hope he doesn't mind me publishing this script). Drop it into your Zope 3 directory and run it. This script runs the Zope 3 initialization routines just once, and then you can edit functional doctests and immediately run them without the overhead of reinitializing everything. The script is already a great timesaver. Thing is, now with z3reload you not only get to modify the functional test source without having to reinitialize Zope, but you can edit the view code as well.

It is a bit puzzling to me why such a thing has not been done before (well, at least I was not able to find it). I have heard there was something similar in Zope 2, and now Zope 3 people seem to have the notion that code reloading is not worth it, it's too complex and bug-prone to implement. Sure, my version is very limited, but I think it has significant productivity advantages.

Beware, I have not used or tested z3reload much. It's quite possible that it breaks horribly in some typical circumstances. Needless to say, don't use it on production servers or precious databases.

For my co-workers stuck with an old revision of Zope 3, from the times Zope 3 had services, I have put up a backported version of z3reload.

EuroPython 2005, continued

2005-07-11T23:08:00.000+03:00

I promised to review some of the talks that I attended. I will cover those which were most memorable or useful.

Kit Blake presented the Document Library, which is basically a document archival application. What's interesting is that it's built using Zope 3 technologies. I think that this need for this type of applications is going to grow and it's nice to have a free implementation. Besides, since it's in Zope 3, it should be easy to integrate into other projects.

AlphaFlow, presented by Christian Theune, revealed to me how libraries can help manage workflow. Besides, I followed a link from AlphaFlow's webpage and found Workflow Patterns, which has some nice research on workflow.

Edward K. Ream, an impressive person by himself, introduced Leo, which is similar to an outliner, but much more powerful. It is not very easy to precisely define Leo because it is a generic tool to manage information. A nice introduction is on the What Is Leo page. I was quite fascinated with Leo, but found it to be imperfect in some regards. Maybe it's just me, but the Tkinter GUI looks outdated and the whole application does not look very polished. Leo is also very ambitious and provides its own editor, which I don't think I'd like to switch to, especially since I'm a vim user. It does provide some Emacs-like shortcuts. It has some novel ideas, but I ran out of patience trying to find an efficient way to work with it. I also did not want to lose many very useful tools that work on the standard filesystem and integrate with vim/Emacs. It also seems to me that versioning should be in the big picture too. It's obviously a hard problem though.

Two talks by Theo de Ridder were energizing and even frenetic. It's obvious that he is an extremely smart person, but most people from the audience lost the thread about halfway through his talks, myself included. The fact that I think it's my fault shows that the presentations were excellent in other regards.

Kamaelia, presented by Michael Sparks, is a framework for asynchronous applications constructed from components linked via communication channels. The nice thing is that Python generators are used to achieve multitasking, which makes components much easier to write. It was interesting to hear that the best way they found to design parallel systems was to first write a simple modularized version and parallelise afterwards. I liked the basic idea of small components talking together to do complex tasks. It's like a simple and elegant application of parallelization to component-based architectures, which are more frequently built as libraries or have much more complex synchronization mechanisms. I was a bit disappointed by the state of the project (they don't even have Kamaelia packaged for easy download...).

Both Steve Alexander's talks that I attended were brilliant as usual. In particular the talk about Zope 3 security was interesting to me. The slides elucidated the Zope 3 security mechanisms and offered a nice use case in Launchpad. Even though the presentation took 60 minutes, it didn't seem long at all (unlike the other one, about Leo, which could easily have been compressed).

Tommi Virtanen talked about Twisted news. We had used Twisted in SchoolTool previously and it's possible that we will be using it in the future (there is, or at least was, talk of Zope 3 integration with Twisted). It was nice to hear about all the new things being worked on.

The talk on PyPy by its main contributors Armin Rigo, Holger Krekel, Christian Tismer and Carl Friedrich Holz was lucid and helped understand the structure and goals of PyPy. I was impressed by their early demonstrations of simple Python code compilation to C, giving a tenfold speed increase. They have gone a fair way since the last Europython, where I attended their presentation on PyPy too. In fact, I participated in the PyPy sprint after the conference, I helped a bit with the standard library update. PyPy uses the CPython standard library, but there were a few issues to be resolved to update the library to that in Python 2.4.

Michael Hudson talked about Recoverable Exceptions. He was being a bit too apologetic, but the talk came out OK, at least it was interesting for me. He talked about smart handling of exceptions, that is, other behaviours than just aborting to the nearest except block without a way back when an exception is raised. While I'm currently quite satisfied with the standard exceptions that Python has, it's some food for thought.

Armin Rigo talked about greenlets, coroutines in Python. They are based on Python generators and use some C-level magic stack manipulation. The presentation helped me finally understand what PEP 342 is all about. Now that Guido van Rossum has accepted PEP 342 and I believe the implementation has been checked in, most of what greenlets offer will be in the Python core eventually.

Web Application testing using Selenium was not particularly impressive, perhaps because I knew most of the things being presented. The idea and technology behind Selenium itself is sound and very practical though. If you are doing web application development, be sure to have a close look.

Michael Salib is another one of those energetic speakers that deliver memorable talks. In this Europython he talked about Xapwrap, a wrapper for the Xapian text indexing library, and q2q, a peer-to-peer connection management protocol. The former might come out a viable alternative for Lucene in some cases. q2q is rather immature at the moment but it seems to be solving the right problems the right way. It's a pity that slides for the q2q presentation don't seem to be available, they were hilarious.

I spent most of my time on the Python lightning talks track rather than the Zope lightning talks. Python talks were so much more fun. I forgot most of the content, but I had a great time. I actually did a lightning talk about darcs, David Roundy's Revision Control System, which I found very useful. Michael Sparks, who spoke about Kamaelia, has a very nice summary of Python lightning talks. By the way, to answer his doubts about my talk: yes, darcs is not written in Python, but it's great for managing Python code as any other, a very useful tool for any developer. Steve Alexander did a talk on bazaar-NG, which also looks very promising, but at the moment in my opinion darcs is much more mature and usable.

Most of the descriptions were rather terse, you will have to excuse me for that. There is just too much to cover. Reinout van Rees has a more thorough overview of Europython 2005 talks that he attended (there is not much overlap with my list), you should have a look there too.

Europython 2005 report

2005-06-30T15:09:00.000+03:00

Europython 2005 is now officially over.

I did a talk on Monday that covered gtktest, a small collection of helpers to make unit-testing pyGTK applications easy. Check out the slides (PDF).

I also delivered a lightning talk on darcs, which is a very well-designed revision control system. Slides (PDF) are available. I use darcs for managing gtktest code, and it has been great so far, much more pleasant to use that Arch and much more powerful than Subversion which I use at work.

I have compiled a list of talks that I attended. You can find a description of each talk on the Europython website, but I was too lazy to add direct hyperlinks on each and every talk. I did include links to day timetables, where you can find complete lists of talks (with hyperlinks to descriptions and slides).

Monday (schedule)

The art of giving a talk (Hellwig)
Architecture of a large Zope 3 system (Alexander)
A Python Framework for Rapid Application Development (Goodwin, Wrigley)
Document Library (Blake)
AlphaFlow (Theune)
MayaVi2 (Ramachandran)
The world according to Leo (Ream)
Teaching computational engineering (Fangohr)
Enabling bare Python as universal connector for ad-hoc networks (de Ridder)
Pulling Java Lucene into Python: PyLucene (Vajda)

Tuesday (schedule)

Kamaelia (Sparks, Lord)
Complex security with Zope 3 and an RDB (Alexander)
Twisted news (Virtanen)
PyPy as a compiler (Bolz, Krekel, Tismer, Rigo)
Recoverable Exceptions In Python (Hudson)
Greenlets: coroutines aren't stranger than generators (Rigo)
Where metaclasses surpass decorators (de Ridder)
Solving puzzles with Python (Niemeyer)

Wednesday (schedule)

Stupidity and laser cat toys: Indexing the US Patent Database with Xapian and Twisted (Salib)
The Python revolution in the publishing industry (Masini)
ItsATree - creating a multimedia editor (Gietz)
Web Application Testing with Selenium (Roeder, Roeder)
WYSIWYG interface design with CPSSkins and CPSPortlets (Orliaguet, Anguenot)
The Personal Internet Endpoint: Using Python and Twisted to write Reliable Peer-To-Peer Programs (Salib)
Lots of Python lightning talks (I did one too)

I will cover the talks that I liked best later.

Back again

2005-06-21T23:57:00.000+03:00

It's been quite a while since my last post. Now, having finished my exam session, I hope to resume regular posting.

So, what's been up lately? Well, just a few days ago version 1.1.1 of the calendaring server SchoolBell, free software that I'm working on, has been released. I am fairly satisfied with the reliability of this version as it fixes most problems that surfaced in SchoolBell 1.1, which was tested by quite a few people. SchoolBell will not be developed further on its own in the near future, instead we will be working on SchoolTool Calendar (version 0.10 released recently), which is an extension for SchoolBell that accommodates some education-related use cases. You can find some nice screenshots on the webpage and in the "screencasts" for SchoolTool and SchoolBell.

In other news: I'm leaving for EuroPython in Sweden on Saturday with my colleagues. I will be doing a presentation there on unit-testing PyGTK applications, which is not quite my area of expertise, but, well, they accepted the talk, so they can't complain. The code behind the idea still needs a lot work but I intend to clean it up a little bit and make it public before I leave.

SpamBayes

2005-04-06T00:08:00.000+03:00

I was fed up with the low, but increasingly annoying flow of spam into my mailboxes, so I have finally decided to set up a spam filter. I chose SpamBayes, as I had heard some good things about it (besides, it's written in Python).

As I use Debian (unstable), installation was just an apt-get install spambayes away. Setup and integration with Evolution, my mail client, was a bit more tedious. (By the way, SpamAssassin might have been a sensible choice, as it integrates well with Evolution.) First, I investigated the approach of using SpamBayes by piping messages to one of the SpamBayes scripts, then I even found a script (sb_evoscore.py) that is for use with Evolution specifically, but these solutions had a few drawbacks, so in the end I settled with the standard proxy server approach.

The user interface of the SpamBayes server impressed me. The server sports a simple web server which you can use to configure SpamBayes, review messages, train the filter or view statistics. Configuration of the server was straightforward.

The server is started by running the script sb_server.py, residing in /usr/bin, so it should be in your path. I was slightly annoyed by the fact that the script would immediately litter the working directory with files, and that it had no way of daemonizing, i.e., detaching from the terminal. I created the directory .spambayes in my home directory for storing the SpamBayes database. To run the server automatically, I whipped up a simple init script. It runs sb_server.py in the background as the specified user (just one user though - this will not work for a multiuser system with several people running the SpamBayes server). You will need to create /etc/default/spambayes where the variables DBDIR (the directory for the databases) and RUNAS (the name of the user) would be specified, e.g.:

DBDIR=/home/gintas/.spambayes
RUNAS=gintas

I have not yet figured out why, but after changing the network the SpamBayes server would sometimes wedge up and refuse to connect to a POP3 server because it could not resolve the domain name. For now, I added /etc/init.d/spambayes restart to my suspend script as a workaround.

As I had anticipated setting up a Bayesian spam filter, I have been marking my mail as spam in Evolution rather than simply deleting it for a while. However, when I wanted to train the filter, I could not find the spam folder on my filesystem (Evolution stores mail in the mbox format, in ~/.evolution/mail/local). My first try at training the filter was to simply copy the contents of the Spam folder to a temporary mail folder, which would show up as a file, and feed that as "spam" to SpamBayes, and the other mailboxes as "ham". However, I noticed that the filter didn't work well. Then I found out why the Spam mail folder was not showing up as a file - Spam is actually a virtual folder, and when a message is marked as spam, it is simply hidden from the view rather than moved to a different mailbox. It makes some sense - in case you change your mind about the message, you don't have to know where it came from, it will appear where it was. Therefore the spam training went fine, but supplying the "good" mailboxes was a mistake, because they included the spam too. In the end I had to create another temporary mail folder and copy some good messages to it, and use that one to train the filter.

Wiring up Evolution to use the proxy was easy. I had to change the POP3 server settings in my Evolution accounts to point to localhost:proxied_port as the server, so that Evolution would get messages with the spam indication headers. To use the filtering, I added two rules, one for messages tagged as spam by SpamBayes, and another one for "unsure" (the tag can be found in the header "X-SpamBayes-Classification"). I set the former one to give the message Spam status and mark it read, so I wouldn't even notice it, and the latter to mark the message as Spam but leave it unread, so that I would have a look at it before discarding it. These rules suit me well, as I have never had a false positive, and most of the "unsure" messages (21 out of 24) are spam.

After you train SpamBayes, remember to run a sanity check by querying some common "ham" / "spam" words - that's how I discovered my blunder. Such words as "money", "rich" should show up as spam content. As for ham content, you know best what words are most frequent in your emails (in my case "python" was a clear shot at 87 ham messages vs. 0 spam messages).

Further training of SpamBayes is performed either by reviewing the messages through the web, or running a proxy for the outgoing mail server so that you can send emails to fictitious addresses used for notifying SpamBayes about mistakes. I went for the former. Now once in a while I visit the message review page to classify the unsure messages, though even that is probably unnecessary, as SpamBayes should be chugging along well enough with the existing database.

In conclusion, with zero positives, zero negatives and just a handful of unsure messages to date, I'm quite satisfied with SpamBayes. I had tried to look around on the web for information on using Evolution with SpamBayes and found very little, so I hope that this article will turn out to be useful to someone.

Poetry in Translation

2005-03-21T00:52:00.000+02:00

I have recently discovered a very funny page which abuses Google translation services to get really funny results. Ever found oddly simple insights in automatic translation? Well, there's a fair dose of them in Poetry in Translation, which translates English to German to French to German to English. The comments section is a bit indecent, but there are some outright hilarious results:

"I have a broken heart" -> "I have a defective heart"
"Get busy living, get busy dying" -> "If you receive a life employed, you receive death employed"
"The quick brown fox jumped over the lazy dog." -> "the fast brown fox jumped on the putrefied dog."
"Just die, why don't you?" -> "Cubic straight lines, why not him?"
"you won't fool the children of the revolution" -> "children of rotation tromp you"
"the pen is mightier" -> "the feather is more powerful"
"Oh how i love my girl" -> "How how my girl is expensive!"
"One for all, all for one" -> "For all, all"
"Be Afraid, Very Afraid" -> "Have fear very timidly,"

Hard drive benchmarking

2005-03-21T00:35:00.000+02:00

Marius Gedminas experimented a bit with hdparm and his results showed no difference in linear disk read rate at the beginning of the disk as compared to at the end. That made me curious. I whipped up a very simple Python script to time some plain reads from the disk and the results are consistent with ones from dd, and with another benchmark of my new drive. Yes, using Python for benchmarking is a stupid idea, and the results are not stable, but I do consistently get almost 40MB/s at the start and no more than 28MB/s at the end of the disk. You can try the script for yourself:

#!/usr/bin/env python

import sys
import time

MiB = 2**20

BLOCKS = 128 # number of megabytes to read at a time
SPACING = 4 * 1024 # number of megabytes to seek forward


if len(sys.argv) < 2:
    print "You must supply a device (e.g., '/dev/hda') as an argument"
    sys.exit(1)
try:
    f = open(sys.argv[1], 'r')
except IOError, e:
    if e.errno == 13:
        print ("Permission denied to read the device, you"
               " may need root privileges")
        sys.exit(1)

offset = 0
while True:
    start = time.time()
    try:
        f.seek(offset * MiB)
    except IOError:
        break # We probably hit the end of the disk
    for i in range(BLOCKS):
        f.read(MiB)
    delta = time.time() - start
    rate = BLOCKS / delta
    print 'Offset: %d GB, read rate: %3.2f MB/s' % (offset / 1024, rate)
    offset += SPACING

print 'Finished.'

The Commonly Confused Words Test

2005-03-14T23:21:00.000+02:00

English Genius
You scored 100% Beginner, 93% Intermediate, 93% Advanced, and 88% Expert!

You did so extremely well, even I can't find a word to describe your excellence! You have the uncommon intelligence necessary to understand things that most people don't. You have an extensive vocabulary, and you're not afraid to use it properly! Way to go!

Thank you so much for taking my test. I hope you enjoyed it!

For the complete Answer Key, visit my blog: http://shortredhead78.blogspot.com/.

My test tracked 4 variables How you compared to other people your age and gender:

You scored higher than 77% on Beginner
You scored higher than 39% on Intermediate
You scored higher than 61% on Advanced
You scored higher than 97% on Expert

Link: The Commonly Confused Words Test written by shortredhead78 on Ok Cupid

New hard disk for my laptop

2005-03-14T01:16:00.000+02:00

Just a few days ago I bought a new 7200rpm Hitachi hard disk (Travelstar E7K60) to replace my old 4200rpm one by Toshiba that came together with the laptop. Now I really regret that I did not do this earlier. This is easily the best investment in a laptop's performance, unless you have a really old CPU or less than 256MB RAM.

The speed increase is very noticeable. Bootup now takes only half the time it used to, and applications start significantly faster. Seeks are more silent in the new drive, and I have not noticed any background noise because of the increased rotational speed. Battery usage has not changed at all. In general, I noticed only improvements and no regressions after upgrading.

While partitioning the new disk, I noticed that in my old Toshiba drive the root (/) partition was located at the very end of the disk, because of hysterical raisins. I did not have a separate partition for /usr, so its contents were there too. Make sure not to make my mistake of putting frequently accessed data at the end: hard disks are usually faster at the start. This is because the rotational speed of the disk is constant, but the circumference of outer tracks is larger, therefore, if the data density is uniform over the disk, the transfer rate is greater.

Speed of some drives may be more sensitive to track diameter than others. If you are curious, you can do a quick benchmark. My new one shows about 38MB/s linear read speed on the outer tracks (start of disk) and about 27MB/s on the inner ones (end of disk), a quite significant difference. To get these numbers, I used this command on Linux: sudo dd if=/dev/hda of=/dev/null bs=1M count=1024 skip=0. It reads a gigabyte of data from a given offset in the disk, the operation should take about half a minute. dd even counts the transfer rate for you. To measure performance on the inner tracks, adjust the skip parameter (e.g., about 37000 for a 40GB drive). You might want to repeat the command a few times and average the results. Do not pick an amount of data (the count parameter) less than twice your RAM, because the results may be skewed because of caching.

One-button testing

2005-03-06T01:56:00.000+02:00

I wrote about some inefficiencies in my procedure of running tests a while ago. I really dislike the repetitiveness of commands to switch context, as I use the vim editor for editing code and a terminal for running the tests. Had it been code, I would have refactored it long ago. Now I decided that it's time to optimize this part a little bit.

My very first idea which I had come up with long ago was to write a small script called loop, which would loop a command given as an argument infinitely, waiting for a keypress between runs. The script was an extremely simple one-liner:

while true; do $@ ; read; done

However, it only helped me with losing the Up,Enter habit a little bit, as Enter would be sufficient.

The second go at the problem was on the right track. I decided to write a small Python script to behave much like the loop script, but it would register a global shortcut handler so that I could do an iteration without having to switch to the terminal.

The idea of handling global shortcuts was OK, but the implementation gave me some pains. I tried to look around for some examples of registering global shortcuts with GNOME, but found nothing really useful. I then remembered that a multitude of KDE apps register global shortcuts, and decided to try the KDE Python bindings. In the end I wasted several hours scouring the web for information and watching my app segfault because of odd reasons. It took me a long while to get the details mostly right, and because of reentrancy problems I managed to wedge my keyboard completely so that I had to login remotely and kill the Python process to get control back. I did not quite like having to load KDE libraries either, which took a whole second to import on script startup. In the end I dumped this solution for a more simplistic approach.

After playing with the KDE shortcuts for a while, I finally understood that I don't really need a global shortcut, as 99% of the time I need to run the tests while I am working in Vim. This allowed for a much simpler system. The loop script has remained, but has evolved significantly from the one-liner. Notably, it now checks the return status of the executed command and prints a red or a green horizontal bar with a timestamp. It is very nice to have coloured feedback on whether the tests have passed. There is also an option to run an alternate hardcoded command.

I implemented interprocess communication in a slightly hacky way, by having the loop script invoke another shell script for user input (this way I sidestepped smart signal handling in a shell script). The client, in our case vim, can then send a signal to the primitive sub-script with a simple killall command. Using the process name as a unique identifier is not very clean, but good enough for me.

So, there are three scripts in total:

loop (to be used from the command line)
dumbass.sh (the stupid sub-process, only used internally)
dumbass-kick.sh, which abstracts the killall command in case I want to implement it in a cleaner way. It accepts '1' or '2' as an argument. If you pass '2', loop runs the alternate command.

To be able to run the tests from vim by a single keypress, I added these lines to my .vimrc to bind the given command to F12, and the hardcoded alternate command to Shift+F12:

nmap <silent> <F12> :wall<CR>:silent !dumbass-kick.sh 1<CR>
imap <silent> <F12> <C-o>:wall<CR><C-o>:silent !dumbass-kick.sh 1<CR>
nmap <silent> <S-F12> :wall<CR>:silent !dumbass-kick.sh 2<CR>
imap <silent> <S-F12> <C-o>:wall<CR><C-o>:silent !

These commands also save all active files before running the tests. I added that because occasionally I forget or mistype the write command in vim and then waste some time trying to understand why the tests are misbehaving. All in all, this provides true one-button testing, you don't even have to exit from insert mode.

If you really want a global mapping, you can map a key to invoke dumbass-kick.sh in your window manager. That's a more lightweight solution than importing KDE libraries just to use their shortcut mechanism.

Even though I mostly use these scripts for running unit tests, they could be useful whenever you need to repeat the same command lots of times. For example, I have been toying around with Lilypond (music typesetting software) a bit, and I used loop on lilypond to generate DVI output on a keypress so that I could see my changes immediately without stopping to type and switching windows. The script could be useful with make when working with compiled languages.

Dictionaries

2005-03-01T01:38:00.000+02:00

Having a computer look up words for you in a dictionary can be a great timesaver sometimes, especially if you need to check many words. There are several choices of a computerised dictionary.

A simple and straightforward choice is to use a web-based dictionary. You can usually find some quite complete and verbose ones with many examples. Some sites even provide other linguistic data (this database really impressed me). Besides, Google is always handy to search for extra information.

If you have a text that has many unknown words, it might be faster to run it through a general-purpose translator, such as translate.google.com. You will lose precision, but at least you might get a good laugh from the results.

Having the dictionary installed locally is more convenient (you do not need an internet connection) and faster (almost zero latency). For some languages specialised dictionary software is available, but there is also the dict network protocol which defines a standard way for a generic dictionary client and a dictionary server to communicate. This model is quite powerful.

Setting up a dict server on Linux is not hard, in Debian it's just an apt-get install dictd away. Note that you will need to install the dictionaries yourself. Debian provides some dictionaries, e.g., dict-de-en. There is also a number of dict-freedict-* packages, but I have the impression that they are not very complete.

Now that you have a server running, you need a client to use it. There are quite a few clients available, I will mention several:

dict - the command-line client. Just type dict foo in a terminal and you'll get the query results immediately. Very handy but not convienient for looking up many words
gnome-dictionary - the GNOME dictionary client (screenshot, look on the right - not to the point, but will do). Looks nice at first but in my opinion it is not very usable, I hate the popup window when no matches are found for a query. And it pushes GNOME's "live preferences" to the uncomfortable limit - when you enter a new server, the same preferences dialog box is immediately reconfigured to that server, which looks very awkward.
kdict - the KDE dictionary client (screenshot). I slightly disliked it because the input box would lose focus after executing a query, so I would have to use the mouse to enter another word. Jeez, even the web-based dictionaries get this right with some JavaScript. The problem can be worked around with by mapping Ctrl+L to the "Clear Input Field" action. I would rather it selected the word instead of deleting it, but this solution is satisfactory. In addition, kdict offers database sets, which turns out to be very helpful. In most dict clients you can only query either a single dictionary or all dictionaries available. Database sets are like virtual dictionaries. An example of a use case is when you want to translate from English to another language and you have several dictionaries for this purpose, but you don't want the general-purpose ones to get in the way.

After finding out about the Ctrl+L tweak I liked KDict best. I would prefer it to be a GTK+ application rather than Qt, but it is practical, which matters most to me.

Piano resources

2005-02-13T00:57:00.000+02:00

I get most of my piano sheet music from the internet, for free. The guitar players are probably in a even better position, with tablature for most pop songs, (and classical pieces too, I suppose) freely available. However, there is a fair share of free sheet music for the piano as well.

Because of the copyright laws, most sites do not contain material created later than the early 20th century. Finding modern pieces is a problem. However, there are plenty of public-domain classical (in the broad sense) pieces available. When searching for a piece you will probably want to visit several sheet music repositories before you turn to Google, which will give you heaps of trash to wade through.

If you are looking for a piece by a very famous composer, chances are that there is a dedicated site where you can download sheet music / recordings (e.g., www.chopinmusic.net, www.jsbach.org). You may want to look at such sites first, they are usually quite complete and the quality is good.

When I am looking for classical music, my first stop is usually the The Sheet Music Archive. The info page says that the site contains over 4000 pages of sheet music. It is a pity that it only allows two downloads per day (unless you are smart and guess the filename of the piece), but that is usually enough.

If you can read Russian, you might find Boris Tarakanov's Sheet Music Archive at notes.tarakanov.net very handy. I have only discovered it recently, but already found a few pieces which I had been looking for. You do not need to actually understand Russian to browse the site — an online translation service might work if you can't read text in Cyrillic.

mfiles.co.uk seems to have a little bit of everything. You can find various well-known pieces and some comments on them here. This site is nice to browse when you want are looking for new material.

It is sometimes helpful to hear a piece to decide if you like it before looking for the score. In other cases, you only know the composer and the melody, but not the name / number of the piece. kunstderfuge.com has an extensive collection of classical MIDI files. Of course, MP3s are more pleasant to the ear, but they are also harder to obtain and take much longer to download. MIDIs can usually give you the basic idea. Googling for MIDI files is easier than searching for free sheet music, but still a pain, so look here before you wander off.

Sibelius, the most popular notation software package, has a large repository of Sibelius scores on sibeliusmusic.com. You do not need to have Sibelius, but you will have to install Scorch (Windows and Mac only), a free browser plugin to view sheet music and play the pieces. Although all pieces on the site can be played and viewed for free, most (but not all) will cost a few dollars if you want to print them out. As this is a community site, there is also a lot of music here that is not worth your time, but there are quite a few gems to be discovered too.

If you are willing to pay money for the sheet music, the best sites are probably sheetmusicplus.com and virtualsheetmusic.com. This is just my impression however, as I have not used either of these services.

Finally, if you are after a relatively modern or rare piece, you might want to check out pianofiles.com. You will not find sheet music on the site itself, but it can provide you with e-mail addresses of people who have the piece you are after. You may then contact them by e-mail and ask politely to send the score to you. You might be asked for something in return. You can search the database anonymously, but the system will not show any e-mail addresses until you register. You can find a list of sheet music that I have on my member page.

Cryptonomicon

2005-02-06T21:38:00.000+02:00

I have recently finished reading "Cryptonomicon" by Neal Stephenson, kindly lended to me by Marius. To begin with, it was quite a bit thicker than I'd like (almost a thousand pages), but in the end this book was worth the time.

I think there is no point in retelling the plot, as you can find that in lots of places. In fact, I think that it was not quite top-notch. Most of the book feels a little sluggish and I could not see where things were going. Only the last hundred pages were really interesting plot-wise for me. I might have been happier, had the remaining part been shorter.

This book seems to excel at style, however. Stephenson is not afraid to spend lots of time describing elaborately crafted environments and delving into details. There is a fair bit of intelligent humour and sarcasm thrown in.

There are some geeky parts, about computers and cryptography, that made me slightly uneasy. Some I might consider insulting my intelligence, like the large sections about modular arithmetic or simple text transformations. The scenes concerning computers seemed out of place somewhat (why would I care if Randy was writing a bash or a Perl script, or what UNIX commands he was typing?). Maybe it's just me because I have a technical background, perhaps Stephenson just paid these details as much attention as he did to the non-technical ones.

Frankly, I do not have much to compare the book to, so take my comments with a grain of salt. Personally, I found the book quite enjoyable.

PyQLogger reloaded

2005-02-03T23:08:00.000+02:00

The author of PyQLogger (a nice and functional PyQt-based blogging client) has been very helpful and has promptly fixed most of the problems I had mentioned. Even an improved spellchecking interface is in the works. A version with the fixes included has not been released yet, but you can check it out from the project's Subversion repository:

svn checkout svn://svn.berlios.de/pyqlogger/stable-1.x pyqlogger

In addition, a Debian package is now available. Unlike the source tarball, the APT package does include most of the recent updates and fixes.

Tidbits for developers who use the Vim editor

2005-01-30T15:11:00.000+02:00

Vim is an excellent general-purpose text editor, but it is relatively stupid by itself in some situations. Thankfully, it can be scripted. You can find many scripts on www.vim.org/scripts, I will mention a few useful ones here.

Every decent code editor has a shortcut to quickly comment / uncomment a block of code. A similar effect can be achieved on Vim by installing FeralToggleCommentify. This plugin works with a great number of different filetypes. I especially like to use it to comment out HTML/XML markup because XML comment tags are tedious to type (the script comments each line of the selection separately though). The binding C-c, although on a nasty key, is quite useful as well - it makes a copy of the current line and comments it out.

If you work with Subversion and write commit messages with Vim, you might find svn-diff handy (you will need Python scripting support in vim). It shows the diff of your commit in a pane below the commit message. Besides, Vim does syntax highlighting, so the patch is easier to read than on the console. Remember that if you see something wrong in the diff, you can always abort the commit by not saving the commit message file or, if you have already saved it, deleting the file (:!rm %).

Python coders will appreciate the smart handling of indentation by the alternative python indent script. It is sometimes too smart and therefore annoying, but works well in most cases.

The XML editing plugin is also nice to have if you deal a lot with HTML / XML. Its functions appear to be quite useful (I keep forgetting the bindings): jumping between opening and closing tags, enclosing content in tags and deleting enclosing tags, etc.

If you work in Vim a lot, I highly recommend to remap Escape. The standard position makes you move your hand away from the home row. I have remapped Escape to Caps Lock which I never use. It takes a few minutes to readjust to the standard Escape position when working in Vim on other computers, but that is a small price to pay for the increased productivity. You can remap the key on X by using xmodmap. Run xmodmap ~/.xmodmaprc after creating the .xmodmaprc file in your home directory that contains these two lines:

clear lock
keycode 66 = Escape

Make sure to have a look at a post by Marius Gedminas about CTags and id-utils if you are not familiar with these two timesavers.

I have uploaded my vimrc, maybe you will find something useful in there. I highly encourage to go through the Vim internal features that are turned on in the script, you will probably want to use most of them. Do not expect the script to work out of the box, you will have to remove the sections that depend on other scripts and plugins.

Time bug in kernel 2.6.10

2005-01-30T14:56:00.000+02:00

The kernel 2.6.10 has an uncanny ability to lose track of time when it is sleeping. Whenever I would suspend my computer and then wake it up, I would find the clock hours or even days ahead from reality.

I tried to circumvent the problem with hwclock, but that made things even worse. Evolution would get extremely confused the moment I resumed my laptop and it would suck up all CPU and then some. In the end I would have to switch to a text console, login (wait 20 seconds for the prompt to come up), killall evolution evolution-alarm-notify evolution-data-server-1.0, wait another 5 seconds for the processes to actually get killed and then see the flurry of messages from modules being loaded by my resume script.

To fix the timeshifting problem I had to modify the code as described in this LKML post. The fix is in 2.6.11pre. If you decide to try out 2.6.10, and you use suspend, make sure to apply the patch.

A new Scheme standard in the works

2005-01-30T14:55:00.000+02:00

I was surprised to find out that R⁷RS, a new revision of the Scheme standard, is being prepared. There is a report (PDF) on the progress of the standard.

The most important thing in the new standard is that a module system will be defined. Some interesting things have been considered, such as language case-sensitivity (no decision), non-hygienic macros (no decision), square brackets equivalent to parentheses (passed). Lots of practical suggestions (Unicode support, regular expressions, I/O, hash tables, object-oriented programming, network programming, serialisation, etc.) are also being reviewed.

I was a bit disappointed that a lot of ideas are either still not decided upon or have been postponed for R⁷RS. It is still nice to know that the language is still evolving and improving

Nasty memory leak in X

2005-01-27T00:18:00.000+02:00

Quite recently a friend of mine has found the cause of a leak in X that has been plaguing me for ages. The problem is that the X cursor library does not free animated cursors properly. As a result, my X server would hoard hundreds of megabytes of RAM after a few days of use.

If you are using an animated mouse cursor theme, you might want to check if you are affected (most machines seem not to be for reasons that I have not yet found out). Run top and find the XFree86 process (you might want to sort by memory usage by pressing M). Then have a look at the value in the 'RES' column (it is typical for X to hoard lots of virtual memory, so you should not pay attention to the other values). If it is significantly more than about 40m, chances are that you are experiencing the aforementioned leak. I would suggest working around the issue for now by switching to standard non-animated X cursors.

More information about the bug and a sample application to reproduce the problem is available in the Debian bug tracking system. As far as I can see, the problem has not yet been fixed. By the way, my tests indicate that the problem is pertinent to X.Org too.

PyQLogger

2005-01-26T23:59:00.000+02:00

Since I found out about Gnome Blog, had a look, liked what I saw, and then discovered the fact that it could not set post titles on blogger.com, I have been looking around for other blog applications that can cooperate with blogger.com. I have found several, and finally settled with PyQLogger.

While PyQLogger does not look as neat as Gnome Blog, it is quite functional. It uses a smart interface to blogger.com - posts can also be retrieved from blogger.com, edited and republished without touching the web browser. I really liked the idea of saving unfinished drafts. HTML highlighting and easily accessible post preview are quite nice to have. Spellchecking is also supported, although it is not very convenient.

Since PyQLogger is based on PyQt, it does not blend in with my Gnome desktop, but I do not really mind. There are many worse things. The interface is a bit overcrowded. There are some very annoying bugs. The hyperlink dialog is buggy (the "Cancel" button works as "OK", and the "OK" button does not work at all). And it is annoying that Home/End keys jump to the start / end of the paragraph rather than the active line. The OSD library seems to be used out of plain vanity in situations where a statusbar message would do.

Still, PyQLogger does its job. If someone weeded the bugs out and cleaned up the interface, it would be excellent. For now, it is acceptable, but still better than most of the other clients I have tried.

Kernel 2.6.10 on a Toshiba laptop

2005-01-24T23:50:00.000+02:00

I upgraded the Linux kernel to 2.6.10 about a week ago. I had been using 2.6.8. Here are some impressions.

There were a few pleasant changes. I have already covered CD packet writing earlier. ACPI support has also improved a bit. A minor but very pleasant detail is that my laptop automatically resumes now when I open the lid.

I was surprised to find out that suspend-to-disk kind-of-works now. It used to hang during resume, now it resumes successfully, but maims USB. After resuming the kernel starts spewing messages such as these:

Jan 16 17:26:42 localhost kernel: usb 1-2: khubd timed out on ep0out
Jan 16 17:26:49 localhost kernel: usb 1-2: khubd timed out on ep0in

As my USB mouse stops working after this, suspend-to-disk is still not viable for everyday use to me, but at least it seems to be getting there.

I have also noticed that the clock would at least a few hours off after each resume. I have worked around by saving the system time to the hardware clock right before suspending, and then restoring the time just after resuming. Nonetheless I keep finding premature Evolution alerts after resuming. Really annoying, I hope this will be fixed.

3D support in X still breaks after suspend. It's not really important though, I can restart the X server when I really need 3D working. Besides, it is quite slow anyway.

People who own a Toshiba A15-S157 laptop might find my kernel configuration and suspend script useful.

JScheme

2005-01-24T01:54:00.000+02:00

I have recently come across an interesting project called JScheme, which, as far as I understand, is something similar to Jython, but for the Scheme language. As Scheme syntax differs a lot from Java (Python syntax is very similar in comparison), JScheme adopted special syntax called the Java dot notation to manipulate Java objects, which is quite simple but adequate.

Since JScheme is implemented in Java, you can run it as an applet too. An online demo is available. The interface is cumbersome and ugly, but enough to demonstrate the concept.

Packrats

2005-01-22T23:40:00.000+02:00

It is interesting that people have absolutely no problem filling up latest storage devices despite the rapid advances of data storage technology. This topic has seen some attention from the Slashdot crowd. I do not think that the amount of useful data on a typical computer changes at the same astounding rate. So, where does the space go?

Probably most of the data is multimedia - movies and songs. I wonder how much of those are just played once or twice and then left alone for the rest of time, or until the hard drive dies, whichever comes first. It is none of my business what people do with their multimedia, but I think that it is foolish to hoard things that you will never use.

Movies, in my experience, are a one-off thing. Some do deserve several viewings, but then you do not wait a year before watching them again. I can not think of a good reason not to delete the movie, unless it is one of the few very specials that you actually would want to see after a long period of time passes. Could it be because it "took me a week to download, so it's precious!" Perhaps a friend might want it some time in the future (I can hear the MPAA growling, but I'm not in the USA so I do not care), but then save him/her a few hours and lend a good book instead.

Songs are a slightly different matter. In particular because I have a larger store of them than I would want to (80MB of MIDI from the old times, 5GB of MP3s just locally on my laptop and 350MB of sheet music). I could argue that I do listen to various pieces occasionally. I understand why people are reluctant to delete music. However, I still maintain that collecting for the sake of collection is not a bright thing to do.

Collecting trash is not smart either. Some store all bits of information that they have come in contact with. I say, who cares about the SMS messages, archives of mailing lists, notes, articles that you found "interesting" at the time, artifacts of experiments and primitive one-off scripts. In theory, they could come in useful one day, but in practice they do not by overwhelming proportions.

I suspect that some people put up with the trash just because they are lazy to clean it up. Unorganised files pile up and then in a few years they have trouble finding anything. Well, I think that just like in real life it pays to have a tidy work environment.

The big question is, why am I discussing this? I think that usability and efficiency problems arise out of the sheer amounts of cruft accumulated, and we do not notice. Technical ones too, but that is not important. Database-like file systems are promising, but maybe we could go along with what we have now if we revised our habits a bit. I am not comfortable with the fact that the amount of useless information is increasing so rapidly, and we are battling it by improving searching technology. It would be much better if the signal-to-noise ratio could be improved.

That is a sensible message for developers too: users should be encouraged to throw away what they do not need.

Such an approach of tidying things up does not work on distributed systems, for obvious reasons. That is why we do need good search capabilities and the semantic web. However, in most cases, you are the boss of your computer, so locally you can organize things however you want.

There are practical reasons to keep only useful data around other than searching. Backups are smaller and therefore easier to do, therefore you do them more frequently, therefore your data is safer, QED. Furthermore, there is less risk to accidentally throw away something important while cleaning up junk when you need some extra space. And for me it's a nice feeling that my computer is not a huge pile of virtual trash with the important things buried somewhere inside.

CD packet writing on Linux

2005-01-16T14:24:00.000+02:00

The CD packet writing mode on Linux allows you to mount rewritable compact discs as ordinary read/write media. It brought my attention when I was upgrading my Linux kernel to 2.6.10. Apparently, patches to implement packet writing were floating around previously for some time, but now it has made it into the mainline kernel (darn, can't wait until reiser4 is merged).

It is fairly obvious that packet writing support makes using rewritable discs much easier. I have abhorred floppy discs since I got a CD writer. Floppies are so unreliable and their capacity is appalling by modern standards. However, burning CDs involved much overhead - I had to start a burning application, find an unused CD, erase it, create a new compilation, and then finally burn it. That made CDs inconvenient for storing and exchanging small documents of several megabytes.

Probably the biggest disadvantage of packet writing is that you lose about a sixth of the disc capacity - GNOME shows 550MiB free on a fresh 700MB disc. That means that you will have to burn movies the old-fashioned way, otherwise they will not fit.

Packet writing is slower than burning normal images, which is not surprising. I did a small benchmark. Copying a directory of 11 files, about 10 megabytes each, 90MB in total, took about two minutes of real time. My CD burner supports 10x (1500KB/s). I included spin-up time and unmount time. Linux seems to cache writes, so work with the disc is snappy (well, until you fill the buffer), but unmounting takes some time. In theory, the minimum amount of time required would be one minute plus spin-up time. The result is not so bad. If I was burning the data as an image, I would have had to first erase the disc and then wait while lead-in and lead-out were written out in addition to the data itself.

Compatibility with Windows is extremely important. Modern versions support UDF without problems. I just tried to read the CD on Windows XP. It had the title of "LinuxUDF", included the folder "lost+found", and a empty file "Unallocated space", but other files were read properly. Oh, and that darn GNOME created a directory called .Trash-gintas which I had not noticed. Out of curiosity I tried to write to the CD on Windows by drag and drop, and it did not work, therefore Linux CD-RW support is superior.

Here is a guide to get CD packet writing working on Debian GNU/Linux (with udev):

Upgrade the kernel to 2.6.10. If you are compiling it yourself, you will need UDF read/write support and packet writing support (CDROM_PKTCDVD).
modprobe pktcdvd (add pktcdvd to /etc/modules if you want). You obviously do not need it if packet writing support is statically compiled into the kernel.
apt-get install udftools You need not create the device nodes when asked to, they are for the older packet writing interface.
Edit the file /etc/default/udftools. Uncomment the DEVICES declaration, set the value to your CD device, "/dev/cdrom" should work in most cases. You will need to run/etc/init.d/udftools start afterward.
Find an unused rewritable disc, preferably one that supports fast speeds, and insert it.
Run cdrwtool -d /dev/cdrom -q. The CD will be erased and formatted, an UDF file system will be created on the disc.
While the CD is being prepared, which will take a while, mounting can be set up. Create the directory /mnt/cdrw, then edit /etc/fstab, add this line:
```
/dev/pktcdvd/0 /mnt/cdrw      udf       rw,user,noauto,noatime 0 0
```
After the CD has been prepared, mount /mnt/cdrw.
I had some permission problems - /mnt/cdrw was owned by root with no word-write permissions. If you make changes to its permissions, they appear to be saved into the CD. I used chmod 1777 /mnt/cdrw. Maybe there is a better way.
Try copying files to the mounted disc. I especially like the fact that deleting files is trivial. Run umount /mnt/cdrw when you're done playing. If you use GNOME, you should also be able to mount and eject UDF discs by a single click, just like standard CDs. Use the CD-RW icon instead of CD-ROM in the Computer window.

By the way, you can use any file system, not just UDF, but remember to use the noatime option so that the disc is not written to on every read. You do not want to put too much wear on the disc, as they can endure about a thousand rewrites.

On blogger.com

2005-01-15T01:15:00.000+02:00

Here's an easy one for the Friday night.

A few people have been asking about blogger.com, I thought that I could briefly cover my experiences here.

Things I like most are the fact that there is no hassle with setting up software on a local server, and that design is a matter of choosing a template. The former has a few drawbacks - I can not have trackbacks as they are not yet supported by blogger.com, and backups are not trivial. I do not care much about either of these. As for design, I can make things look neat, but not attractive and beautiful. Preset templates saved a whole lot of time, although I did spend almost an hour picking one as they are all quite nice.

My absolute worst complaint about blogger.com is that it treats paragraphs in a completely braindead way. There are two modes: either no paragraph breaks are inserted (you have to write and tags yourself), or each line break is replaced with an tag. That means that you can not have normal paragraphs in tags. Even if you leave an empty line, two breaks are inserted and that's it. I decided to stick with the manual markup option. The developers of blogger.com really ought to address this problem, it does not look hard to fix.

I was a bit appalled by customisation of links to other blogs. Let me tell how templates work: you choose one from a list (previews are available) and then you can further fine-tune the HTML source. Well, you have to edit the source to add links, which means that if you decide to change the template, they will be lost. I had hoped for a more generic method - links are just a collection of titles and hyperlinks, how hard can it be to implement a page to deal with them?

Today Marius Gedminas brought my attention to gnome-blog, a small GTK app for posting to blogs, written in Python. It looks a bit unpolished, but seems to be quite functional. And judging from the code it should insert those dreaded tags to form paragraphs. This is my first post through this nice program, I will soon find out if publishing works correctly.

In conclusion, I could recommend blogger.com as a way to publish blogs without hassle. It may not have the latest features, but there should be more than enough for most people.

gnome-blog report: it did handle the paragraph tags correctly (yay!), but the title I entered was put as the first line of the post, and the real title of the post was left empty. Well, since it's Python, such a minor problem should be easy to fix.

Me, I, umm, myself...

2005-01-14T01:32:00.000+02:00

While writing English texts, especially this blog, I frequently shuffle with discomfort as can not find a way around using the words "I" and "me" several times in almost each sentence of every paragraph. I guess that using these words a lot is natural, as I am writing a lot about myself, yet it takes some getting used to for me. Using passive voice is an alternative, but it causes more stylistic problems than it solves. I think that some prolific writers structure their sentences to dodge the problem somehow, but I just can not grasp, much less adopt it.

The awkwardness probably comes from my native language. In Lithuanian there is no need to repeat the "I"'s because all verbs have a first-person form, if my phrasing is correct. I can only think of the word "am" in English, as in "I am here", as an analogy, other verbs do not have the difference ("I live" vs. "you live" vs. "they live"). Actually I frequently spot naïve translations from English which indicate the agents explicitly when it is not needed, and this makes the sentences more blunt and unfriendly. I seem to remember a text which discussed fundamental differences between languages where the agent is implied and ones where it is not, so I might have covered just the tip of the iceberg (note to self: cut down on clichés).

One could speculate if using "I" and "me" frequently has a subconscious effect on the writer (or the reader). However, as I am not a psychologist, I will refrain from discussion.

I will again slightly change topic here and use the chance to mention E-Prime. In short, E-Prime is a subset of English which does not include forms of the verb "to be". There are already enough resources to explain it on the web (such as this one), so I need not do that here. I just thought I would mention it as it tries to deal with other aspects of bluntness of the English language.

¿Punctuation?

2005-01-13T01:50:00.000+02:00

Most of my friends know my uncanny ability to notice typos everywhere. I am not sure why that is so, as I do not read all that much (that I do intend to change). There is a bit of irony that I am working on SchoolTool (school administration software) at the moment. Darn, I wish I was good at punctuation instead of spelling, which is becoming less useful as automatic spellcheckers get better. Which brings me to the point.

Recently I have come across an interesting essay called "The Philosophy of Punctuation" by Paul Robinson. It is rather wordy, but well worth reading. I have picked a few things that I could apply to my own writings.

Most importantly the essay brought my attention to the fact that I am overusing punctuation. I used to like long, intricate sentences with dashes, parentheses and semicolons. Connecting ideas in various ways seemed interesting and original. However, Robinson claims that overusing punctuation indicates a lack of writing ability. "Expository prose is linear," and therefore ideas should be presented sequentially. Dashes and semicolons "betoken stylistic laziness," they are a sign of weakly connected ideas.

With regard to long sentences, I read somewhere on the web that varying sentence length helps keep the reader interested. I still rarely use short sentences, but I'm getting better. At least my recent writings are stylistically lighter and more lucid than ones I wrote a few years ago. I am still dissatisfied with structure of my writings, that will probably take time to improve.

I found the personified descriptions of the punctuation marks highly entertaining and persuasive. The style also helps the message that it is not a good idea to try to outwit yourself. It is not clever to obfuscate content, and trying to be modern is a bad excuse. Making the text overly complex means that you distance yourself from the reader, which hinders not only readability of the text, but emotional response as well.

Robinson also expressed the old idea that "Good writing is as much a matter of subtraction as creation." This is good to be reminded of, and so obvious that I need not add anything.

For technical people who care about their writing I could suggest to have a look at Lyn Dupre's "BUGS in Writing: A Guide to Debugging Your Prose" (amazon.com). Although in my opinion sometimes Dupre overshoots while trying to attain lucidity, that may not be necessarily a bad thing as the goal is to help people who are already used to a formal, rigid writing style.

Python unit test runners

2005-01-07T21:19:00.000+02:00

As I do test-driven Python development almost every day, I care a lot about testing infrastructure.

Currently my development environment of choice is Vim + a terminal (for running PyUnit tests). It's rather low-tech, but it works reasonably well. I usually do not use Vim's QuickFix because it's a bit cumbersome to use (I do use quickfix in combination with idutils). I could outline two main inefficiencies of my current approach.

First, to run the tests I have to save the changed files (which I occasionally forget to do), then M-tab to the terminal window and use Up,Enter to run the previous command, which was most likely to run the tests. The latter step, which is now pretty much a reflex as I do it hundreds of times a day, does unexpected things when I forget that the previous command was something different. I have tried to improve upon things by using a shell snippet while true; do ./test -pv foo bar ; read; done, which requires only Enter to be pressed. I find it hard to drop the habit and still press Up,Enter most of the time. Well, at least unexpected things do not happen. Yet this is clearly a poor man's approach. It also makes changing the arguments to the test runner slightly harder.

Second, I am currently doing the jumping to errors manually. QuickFix could do it automatically, but it sometimes does not work as I would like it to, because sometimes I want to jump to a function higher up in the stack rather the one that raised the exception. Typing the line numbers in by hand is not that much trouble, but tedious.

The page that turned my attention into my testing routine inefficiencies is One Button Testing. It contains some interesting ideas. I especially liked the suggestion to use a smart test runner that automatically runs the tests when you save the code. Now I am thinking that, for starters, it might be quite easy to write a small script that would monitor a directory (either by polling or by utilising dnotify/inotify) and would invoke an ordinary text-mode test runner when a file changed.

I have thought up some features (in addition to the usual ones) that my imagined test runner should have:

GUI. Command-line interfaces are really powerful and great for scripting, but they require typing, which is exactly what we want to avoid. (I would not mind if the backend was a text-based test runner.)
Global keyboard shortcuts. Some examples: C-M-a to run all tests, C-M-l to only run tests that failed the last time, C-M-s to show and hide the test result window. All without having to switch away from the code editor.
Notification area support. It would be great to see without switching windows if a test run is in progress, whether the last run had failures or not, and other things.
File monitoring. This is the idea from OneButtonTesting. The critical thing here is to run the tests only for the module that has been updated. Otherwise we either have to run all the tests of a project (slow) or type in a specific subsystem to test (error prone and requires typing).
Integration with Vim and Emacs. As discussed, clicking on a traceback to jump to corresponding code in the editor would save some keystrokes. It would be great if it also did not require using the mouse (e.g., use Left/Right to switch between failed tests, Up/Down to walk between stack frames in a traceback, Enter to go to the highlighted code)

There are already quite a few test runners out there, and some of them already implement a few of the items in the above list. First, the text-only ones:

The Zope 3 test runner is a quite usable, and has some nifty features.
The SchoolTool test runner (download) has been written by Marius Gedminas, my colleague and a very bright Python hacker (if you are a Python programmer, make sure to have a look at his blog). I prefer this test runner over the Zope 3 one, because it is faster (demonstration) and neater.

There are also some GUI test runners out there already. In my opinion they are inferior to the text runners available, at least the ones I have tried. Here is a list:

The PyUnit test browser has integration with vim, idle and kate, but that's about it. It is hard to figure out, requires a few dependencies and in general looks unmaintained. Besides, it uses Tkinter, therefore it looks ugly.
The PyUnit GUI test runner (screenshot) is rather simple and not very useful. It uses Tkinter too.
Pycotine is for Macs, I have not tried it.
PyUnitOSX seems to be a port of the PyUnit GUI test runner to the Mac. I have not tried it either. Well, at least it looks nicer than the original.
The NUnit GUI test runner (screenshot) is not a Python test runner, but I wanted to mention it because it gets some things right. Notably, it sports file monitoring -- whenever you rebuild your project, the runner notices and resets status of all tests. Besides, it is not ugly. However, other things in my wishlist are missing.

As a sidenote, I would like to mention the std library by Armin Rigo and Holger Krekel, two top-notch Python hackers. However, it's buried in obscurity, I do not think it even has a web page. You can check it out from the CodeSpeak Subversion repository (here). The library makes unit tests more Pythonic by doing dissection of expressions (it provides values of context variables with the traceback). It also encourages use of the plain assert statement, which is so much cleaner than the Java-esque self.assertEquals and others. As assertEquals was primarily used so that the value of variables would be shown on failure, there is no need to use it if you use the std library. There are more cool ideas, go see for yourself.

I would implement the stuff I described myself, but I have quite a few unfinished matters already (I will cover them in other posts) and my conscience would not let me drop them and start a yet another project. Therefore, I am hoping that someone with too much free time and good Python skills will have a look at my wishlist, say "These are neat ideas!" and go on and implement them in a couple evenings. Make sure to let me know if you do. By the way, I have seen a pygtk-based GUI test runner by Marius Gedminas . I suspect that it is rather incomplete, but I would not be surprised if Marius, being so ingenious, came up with something really useful, as he has with the SchoolTool test runner.

Darn, another essay-post. Indeed, my original intent was to provide content, but this is too time-consuming to keep up. I will have to try to be more concise.

Why blog?

2005-01-06T01:30:00.000+02:00

Today I would like to discuss the reasons why I thought that blogging would be a good thing to pick up for me. While there are already numerous essays covering the psychology of the blogger (such as this one), I thought that I could present a few of my personal thoughts.

The most straightforward reason why I started this blog is that my attempts of creating a homepage were rather unsuccessful. I would only occasionally find time to reserve for building the page, and when I did, it was not apparent what to do next, what information to put up and how to organise it, not to mention my poor design skills getting in the way. Because the process was so slow, it was not fun and a negative feedback loop ensued. As a result, I have not touched the web page for half a year. Conversely, it is really easy to post to a blog, so I hope that blogging will be fun and will not die off.

A case for blogging is that nowadays there is little point in carefully organising information that you put up on the web. Chances are that the people who need the information will come through a web search engine, therefore, they won't see or care about the structure. Of course, it is a completely different story for large sites which harbour lots of information on a specific field, but I am talking about typical homepages that include information fragmented by nature, for example, "How to set up Linux on a Toshiba laptop", "Some useful Python tricks", "My favourite links", and "Public-domain sheet music of pieces that I like to play". Just post when you feel like it and the info will be scanned by Google in at most a week.

Technical reasons and fun factor aside, there are more serious reasons why I picked up blogging. Browsing the web started to seem too passive for me. I despise television because it makes people passive receivers of whatever is fed to them. The web is better in this regard, because you can (and have to) choose what to read. It is much more interactive. However, I think that to take in the information you need to put it into your own words, or, even better, to explain it to someone. Posting on a blog is communication (even though it is indirect), and you can also get comments, so I expect blogging to be intellectually stimulating.

By the way, Paul Graham has written an interesting text called The Age of the Essay, which discusses the idea of the essay and how it improves mind flow. I think that some ideas from the text may apply to blogging as well.

My memory is not something I am completely satisfied with. That might be the reason why when reading a book or a lengthy text I sometimes feel that reading is pointless. The words just seem to zoom by without a leaving a lasting trace. I sometimes recognise quotes from books (to my surprise). However, I find that I can not consciously recall most of the essence of a book not long after I have finished it. I have discovered that whereas rephrasing helps comprehension, writing down the rephrased thoughts helps memory. It would be reasonable to hope that jolting down my thoughts will help me remember things.

Most of the above does not explain why I would want to make my thoughts available to the public. (I find it funny that apparently only now have I realised why people would write diaries.) Fact is, I am now trying to be more open and transparent to other people. I used to be a rather reserved person, but I have found that it is not the best way. Besides, I do not think that I have much to hide - for example, this essay is completely open and honest, yet I do not think there is anything in here that I would want to keep secret.

I also find the idea of blogging appealing to reveal my personality to people who cannot meet me in person. Just as everyone, I communicate by e-mail with lots of people whom I have never met. The Internet is already an alienating medium in many ways, so it can not be a bad idea to make communication warmer and more personal. I, for one, would really appreciate the chance to find essays similar to this one by people I have to deal with but can't meet.

The last, trivial and a really simple reason to have a blog is to improve English skills. I am not quite satisfied with my English writing abilities. In particular, I would like to improve not just the quality of my writing (which is reasonable, but it still could be much better), but efficiency as well. If I develop a habit of posting regularly, surely there will have to be a noticeable improvement.

I hope you have learnt something about me from this essay. Even better if you found out something about yourself - in my opinion, that is the most important goal of most non-technical writings. Now go write something about yourself if you have not yet done that, there is nothing to lose, only to gain -- to personalise your online identity, and to find out things you might not have noticed.

Whew, now that was a whole essay rather than a post. Well, I had lots of things to say and this particular text would look weird posted a paragraph a day. I guess I made up for the lag - darn, five whole days since the first post, I will have to try and improve...