There are unlimited gnome improvements which I don't care about, but some things stand out for me in this release.

• Firefox 3 is much better than 2. The memory improvements and the renewed linux (gtk) focus helps. And luckily you can disable the stupid new urlbar in about:config with browser.urlbar.maxRichResults = 0 .
• The way non-free installation for media-playback, video drivers and browser plugins is handled, is sweet! The installation of this software is painless here. I even got a nice 64-bit(!) Java firefox plugin, which I was unable to install in Debian (thanks redhat/icedtea).
• And most important: the colors in the gnome-terminal are smoothened. Now that is what _I_ call eye candy (see screenshot below). Less work and much more satisfaction than the integrated compiz.real ;-).

So thanks again for all free software developers!

To achieve this I bought the Gembird Silver Shield, a USB-switchable power adapter. I was prepared to do some nice USB snooping and C programming to get this device working in linux, but (un)fortunately there was already a working utility for this device.

Configuration for suspend/hibernate is not so easy and documentation is sparse for the user mode utilities. Since it took me more than the usual googling I will summarize my conclusions here for later use. First the gnome-screensaver measures the idle time. After this timeout is expired the gnome-power-manager starts to measure his own timeout (so before suspend the two times will stack). When the gnome-power-manager times out it will look at the Inhibit flag. If there is no inhibiting (for example my rhythmbox pushes Inhibiting, because I do not want to suspend when music is playing) your computer will suspend.

Since a custom script should be added to switch the power-switch off via usb with sispmctl the suspend-backend is important. Gnome power manager can use multiple backends to go to suspend mode. This works via hal (the hardware abstraction layer). Configuration for this is in /usr/share/hal/information. HAL is responsible for calling the suspend-backend. The default suspend-backend on my machine is pm-utils. (which can be tested with pm-(suspend/hibernate/power-save etc.). I also have a package called hibernate (which can also suspend, confusing isnt' it?). A third one is suspend2 (which can also hibernate....). These backends have different ways of adding custom hooks.

To add a hook to hibernate I added a file called local in /etc/hibernate/scriptlets.d/. The API is as follows (ugly in my book):

# -*- sh -*-

UsbPowerSocketDown() {
      /usr/bin/sispmctl -f1
}
UsbPowerSocketUp() {
      /usr/bin/sispmctl -o1
}

AddUsbOptions() {
      AddSuspendHook 10 UsbPowerSocketDown
      AddResumeHook 10 UsbPowerSocketUp
      return 0
}

AddUsbOptions

Pm-utils has a much nicer API. To add a custom hook add a file to /etc/pm/sleep.d . This uses init style ordering. So look in /usr/lib/pm-utils/sleep.d/ for a proper number. I needed to talk to the usb-bus AFTER the modules were loaded, so a number lower than 50. So I added /etc/pm/sleep.d/10usbpoweroptions with content like this:

#!/bin/bash
case $1 in
    suspend)
        /usr/bin/sispmctl -f1 
    ;;
    resume)
        sleep 1
        /usr/bin/sispmctl -o1 
    ;;
esac

After all this fiddling it works like a charm! Now hopefully one standard will emerge; because how to achieve the same result with KDE I don't know. I had to manually patch rhythmbox to change calling (via dbus) the Inhibit method from org.gnome.powermanager to org.freedesktop.powermanagement (because i used a wrong combination of versions..), so this suggests a move in the right direction.

Anyway, good 2008 to everyone!

One problem I encountered was getting a decent resolution (f.g. 1280x1024 or higher) in the guest xserver (xorg). Google was not really helpful this time so therefore I post this note for future reference. Qemu emulates two kind of video cards, standard is a cirrus, -std-vga provides you with a vesa one. Vesa was not able to help me (garbled screen), so using the (default) cirrus emulator is the way to go for a linux guest.

The trick was in changing the monitor sync and refresh rates, autodetection did not work properly (in my centos 4.0 guest). So here is my working snippet of /etc/X11/xorg.conf:

Section "Device"
  Identifier  "Generic Video Card"
  Driver      "cirrus"
EndSection

Section "Monitor"
  Identifier  "Generic Monitor"
  Option    "DPMS"
  HorizSync 28-64
  VertRefresh 43-60
EndSection

Sidenote for R. Stallman: when i am using gnu/linux 25 years I promise to make a gnu cake :-D.

Most in charge mplayer developers are pro OSS and do not want to switch. One of the arguments is that OSS is now GPL and is used other unices as well . There are also arguments about library vs. unix semantics. Alsa has an OSS compatibility mode (actually two, one in-kernel and one in-library). However using this blocks soft-mixing. This means that another program playing sounds can not do this while using default mplayer compile.

Both sides have some good arguments (i truly understand the points for OSS!, it is just not realistic), but discussing them in this context is useless and the way it is done on this mailing-list is very childish and out of reality. Mplayer should set Alsa as default output for linux, because 1) all major distributions do this and do this with good reason: 2) OSS in linux is deprecated. Reading this thread made me feel tired of (the immaturity of) the open source scene.

But today I am a happy guy again. Some troll suggested removing Alsa from the linux kernel and I was afraid of another useless thread. However I found a message from Takashi (one of Alsa's most productive driver and patchwriters:

Honestly, I'm not fully against changing the current code base (or crap, whatever, any childish name). There are indeed many misdesigns. But, replacing with the above is no option, IMO. The OSS have also many misdesigns, so the same argument would start again. One should learn something from history...

Anyway, if it's going to be more constructive, I'm willing to join in.

Takashi

Hannu (the main OSS developer) then said this:

We have no intention to push OSS back to the kernel or to replace ALSA. That alternative is not realistic any more. In addition OSS is a cross-platform product and staying more or less outside various kernel trees should provide better flexibility.

What we would like to push is that the old "deprecated" OSS/Free are removed from the kernel. OSS/Free is based on about years old OSS API version which was too limited for many applications. Having OSS/Free in the kernel doesn't serve any purpose.

Also we would like to stop the silly OSS vs ALSA war. OSS and ALSA are rather different. Both of them have some good points and bad points. For ordinary users it doesn't matter which API is used by the applications as long as they work. Just the application developers can see the real difference. Some of them prefer OSS while some other prefer ALSA and this should be their "freedom of choice".

I think the ideal solution would be that both ALSA and OSS APIs can co-exist by sharing the same low level drivers (which has already been demonstrated). The low level driver interfaces in both systems are practically identical. This means that ALSA's core can work with OSS' drivers and vice versa.

Today both OSS and ALSA teams have to spend significant amounts of time in emulating the "alien" APIs. Making OSS and ALSA to co-exist will require some work in both sides but that should be nothing when compared to the effort required for emulation.

Just my 2 cents.

Best regards, Hannu

Fortunately there are still a lot of sane people in the community. Thanks!

Perhaps I like it because perfectly reflects my own opinion though ;-).

You need to apt-get install courier-base. (and then /etc/init.d/courier-authdaemon stop; update-rc.d remove courier-authdaemon. This is not logical (I do not need courier at all, only postfix and mutt), so this post will help me and probably others in the future.

[update]
After hitting save I thought of an easier solution (?), use something like
echo 'function maildirmake { mkdir -p "$1"/{cur,new,tmp}; chmod -R 700 "$1";}' égt;égt; ~/.bashrc
[/update]

I like to watch movies, so now I have a problem... Or not? Fortunately the flash way of bringing movies is just an container trick around a normal mpeg movie, and my favorite audio/movieplayer mplayer can play them just fine.

So if you are on debian, add contrib to /etc/apt/sources.list and apt-get install iceweasel iceweasel-greasemonkey mozilla-mplayer (the mplayer plugin is broken for firefox in etch at this moment). Now you only need a way to let mozilla know how to feed the movies. This can be done with a simple script like Michael Sheldons's.

So finally I can be an ultrahip blagotuber and and bring you movies ;-)

Happy Christmas Everyone!

Also I added photos from an autumn week in Morocco. This trip is really recommended in october.

Since some time I am an user of encfs. I actually did want to encrypt my whole root filesystem (just for fun, nothing to hide ;-)), but the loopback way is a hack and the old weak initialization vectors make watermark attacks easy. The weak key management was also a showstopper. Now that luks is relatively standard I hoped that with ubuntus upcoming edgy the eft the dm crypt + luks setup would be well enough integrated.

Unfortunately this is not the case. The installer does not support this yet. There is a myriad of conflicting documentation on how to set this up properly. Most of these are misleading and outdated. There are thousand ways to do this.. but I wanted to do it in a way that will be supported properly in the future. This guide is the best there is on this topic and following it literally does work for Edgy Eft.

One thing Edgy has changed (compared to the guide, which is actually for Dapper Drake) is that the latest cryptsetup package already has encrypted root initramsfs hooks. With adding cryptopts in the kernel line (or via kopts in grub) or making a /etc/initramfs-tools/conf.d/cryptoroot file the system _should_ come up automatically with the passphrase question. After a lot of fiddling (every distro seems to have their on ways of specifying the parameters) I still did not get this working. If you use the kernel line options cryptsetup is not installed in the initrd and if you use the conf file option, the proper kernel modules are not loaded.

Usually I behave like a good open source citizen and file nice bug reports about this, instead of whining in a journal entry. This time the 1000 different ways of doing these + the already very confusing bug reports about this package, left me feeling disqualified to do so. In the end, just following the guide for ubuntu and writing my own initramfs non configurable hook functions (which conflict with the future cryptopts settings !!) seems the best way right now. What makes the situation even more difficult for starters is that google does not list the proper page for ubuntu but a very outdated howto, so hopefully this entry will help the proper guide bubble up.

Hopefully this is something that is going to get better in future! Distributors, please fix this and standardize! On the bright side, the debian future for this looks most promising, clean implementation and support in the beta installer..

Time to add some neural network learning.

Just before I left, I bought a new Canon 350D. One of the most mainstream (SLR) camera's. I had some positive experience with the analog version and reviews of the digital thing were ok. I am very happy with it. It works fine with gphoto2. (Sorry for the analog camera lovers).

These two items combined gives some new photo albums on this site. Enjoy.

XMMS and winamp were pretty usable and fast (no meta-data reading in advance, but newer versions tried to do the same database building as other new programs (and failed)). One feature was particularly missing from these players: the ability to play random albums. This should (imho) be the standard setting of any music player: You are working and want to listen to an album in the background. Also it would be nice if you would be able to give a subset of your collection (for example all jazz music) and than the software picks an album for you in this genre. XMMS and winamp lack these features.

Even newer music players (banshee, amarok, rhythmbox, XMMS2, beep, windows media player, mpd, etc.) cannot do this, although they come closer nowadays. However I have lots of trouble trying them. For years they crashed on loading my large (100Gb+) collection. Now they usually do not crash anymore, but start crunching for hours (sometimes days) and when they finish loading the library and you restart the application; it just crunches again on their index for minutes. And more often than not, they load the index (sometimes 100MB+) to memory. This is intolerable if you just want to listen to one song.

I realized pretty quick that just whining is not going to help, so I wrote my own player in a few lines of Perl. It is far from perfect: I really want to:

• NOT start gconf-tool externally
• use gstreamer-perl bindings, instead of mplayer
• thread/fork so better key-bindings are possible

Despite these drawbacks I use it for months already and it works flawlessly . Therefore I release the script to the world. It sets your album cover in the background and just starts playing random directories (=albums in my library).

download randomalbum.

For some years semi turing tests under the name of "captchas" can be found on the web, to prevent bots from filling in forms. When I first saw the visual variant I thought recognizing the characters with a computer algoritm should be easy. A bit of surfing and searching on the internet learned me that I was right, most were broken already. Reinventing the wheel is not very useful, so I left the topic alone.

Later I found a post about voice captchas. Since there was not too much information about this on the net and I was bored (ill at home), I decided to give it a shot. I started easy, willing to enhance the used algoritms to those used in speech recognition (like hmm, viterbi, baum-welch, entropy coding, etc.) when needed. This proved not to be necessary, the first feature complete (segmentation and matching) code worked relatively well on microsofts captchas. Later I tweaked it a bit to also work on google captchas.

On this page you can find proof of concept code to break voice captchas. Do not expect advanced software (pattern recnognition science is so much further) or code that can be used in other projects, I quitted the project when it worked. Initially (february 2006) I kept the code on my harddisk, but later (may 2006) I published it (see disclosure motivation).

How does it work

This is not a complete guide, but some pointers to the source (read it luke). As a starting point, consider the configtype struct:

typedef struct {
    int samplerate;
    int byterate;
    int winsize;
    int band_cnt;
    int word_length;
    int word_overlap;
    int threshold_energy;
    int file_offset;
    char trainfile[255];
} configtype;

The program starts with reading the audio file (in the header it could read the samplerate and byterate, but I am lazy). file_offset bytes are skipped in the beginning of the file, because google starts with a bell. The first step is that all samples are treated with a hamming window (arbitrary choice, most window types should do). The winsize is in samples (eg 512 samples on 8000 Hz provides a 64 ms window). Now the blocks are transformed into the frequency domain with a DFT After that the frequencies are put in band_cnt bins. These bins are not equally wide, the higher the frequency, the larger the band (this has to do with human hearing (mel/bark scale), but I doubt this is actually useful at the current incarnation of the program).

Now the program looks at the highest frequency bin. Every block that has more energy in a window than threshold_energy is considered a peak, and these peaks are used the segment the input file in the different spoken words. The word_length tells the program how many windows long a word is (so all words are considered the same length which is a current weakness of devoicecaptcha). word_overlap helps in localizing the peaks. When the locations of the words are know all frequency bins are written for word_length windows around the peaks. This is called the profile of the word.

The profiles for know words are put in trainfile. When a guess has to be made, the profiles for the words in the file are subtracted from those in the trainfile and the smallest deviation is chosen as the proper word. That is all.

The algoritms in devoicecaptcha are at this moment really naive. There are a lot of possible improvements. Perhaps in the future I will enhance the program a bit, for now I think the 33% (as on googles captchas) is good enough (and I am too lazy to reimplement htk, which should do the trick also (I guess)).

Proof of concept

The code is rather messy, but since this applies to most p0f code consider that 1337 ;-). Download devoicecaptcha.c and compile it with it:

gcc -lfftw3 -std=c99 devoicecaptcha.c

As you can see you need fftw, an allround fourier transform library, which is packaged for most distributions, so you can be lazy (apt-get install fftw3-dev or similar).

When started with ./a.out captcha.wav you also need a data set (a msn one and a google one are available. If you have downloaded the same captchas (see links) as I have, it will print a guess on stdout.

As said before, devoicecaptcha works with a comparison to a trained set. To build up a training set and test the effectiveness of various parameters you can start devoicecaptcha with a third bogus argument, eg ./a.out captcha.wav --print.

What I did was download a large set of captchas with lwp and transcribed them with the proper words with something like:

for i in google/*.wav; do aplay $i éamp;> /dev/null éamp;; read j; mv $i google/$j.wav; done

I ended up with a directory with filenames like "123456.wav" where 123456 are the digits spoken in the captcha. On this directory I unleased a small ruby script, which splits the files in a training and testing set, builds a training set and tests the rest. This script can be found under train.rb.

If you have broken other voice captchas with my code (or with an addition to my code), please let me know, so I can update this page.

MSN

MSN (passport) audio captchas are really weak. Only digits are used, there are always ten digits and the noise is weak and constant. The distance between the words is relatively constant. Devoicecaptcha guesses all ten digits correct on around the 75% of all cases, with a training set of about 40 files.

A data set which can be used for the english MSN (aka passport, aka msn live) voice captchas (which I got from passport.net) can be downloaded under the name msn.txt. It is also possible to create your own training data (see above).

Google

Googles voice captchas are more difficult to break than the captchas by microsoft. Google employs different speakers, uses better noise artifacts and a random number of words. The dictionary is (as microsofts) limited to digits only. The devoicecaptcha program scores around 33% on these voice captchas with a training set of 60 files. This is high enough for use in a bot.

A data set which can be used for the google captchas (in english, google also provides captchas in multiple languages) can be found under google.txt. The files were found at Google new account.

These captchas are also in use by blogger and blogspot for comments

Others

If you know other voice captcha systems, let me know. Perhaps I will have some time to look into them (and perhaps not). I will at least add them to the links section on this page, so together with the provided source other people can try to beat them.

Disclosure motivation

• I do not believe captchas actually work. Almost all visual captchas are broken already (read for example Microsofts paper on visual captchas or ez-gimpy which can defeat the "human insolvable" captchas at yahoo). Or look at the versatile pwntcha. Although I do believe humans can fool computers for quite some time in the future, I suspect that computers can always beat computer generated challenges (without a "secret" as in PKI).
• Spam is a human problem and I believe human problems should be solved by humans. Legislation and law enforcement are in my opinion the best ways to deal with spammers. If captchas did no harm anyone this would of course not be enough reason for releasing this code, but
• Captchas make the web a more difficult place for disabled people (see http://www.w3.org/ and more annoying for everyone. I hope the community will be motivated to find other solutions (and I am happy that the w3 organization cares for people with disabilities and a usable web for everyone, including the deaf-blind).
• I am a proponent of full disclosure.

Some people might ask what kind of solutions I do suggest for solving the spam problem. Spamassasin catches thousands of spam mails for me; it is expensive in cpu cycles (so putting spammers in jail is preferred), but the multi-tiered approach (neural network detection together with several lists of wrong-doers) works relatively well and can be applied to other forms of spam.

Playing the cat/mouse game with more difficult captchas, when the previous challenge is broken will work, but is not satisfactory in the end. I encounter more human unsolvable captchas everyday. I do understand that corporations play this game however; in the real world thresholds do help.

Links

• Wikipedias entry on captchas is a very good starting point on this topic (as usual) and contains links to pages about breaking visual captchas
• Inaccessibility of CAPTCHA by w3.org
• Microsofts paper on visual captchas
• PWNtcha - captcha decoder by sam hocevar

Broken voice captchas

• Google signup
• Microsofts passport service
• blogger and blogspot also use google voice captchas, so these are broken to.

Working on voice captchas

• Google.com A slightly better segmentation and better database gives me 60%+ recognition rate, but this is yet unreleased.
• Paypal audio captcha Relatively well done (best one I found actually), but I expect to release support for this captcha "any time now"
• Captchas.net HEAD works now. I will release this soon, but I am too lazy to generate a training set, so stuck at 33% recognition rate for now.

Not working on voice captchas

• Standards schmmandards proposal The use of generated speech is probably weak; positive point is the use of human parsing ("the three numbers are one, two three"). For now I haven"t seen it actually used.

Do you know different implementations of audio captchas? Please contact me.

Leiden was inhabited (at least) from 300 BC (actually the Romans lived in Katwijk, but this sounds better ;-)). In 900 AC, where the Oude rijn, the Nieuwe Rijn and the Mare come together, a tower was built, which began urbanization and in 1266 the city received city rights.

Leiden also housed the first dutch university and was home of the Nobel price winner Kamerlingh Onnes.

Just kidding of course ;), I actually like the Melua cd's. Especially Mike Bright Eyes Batt's songwriting is very smooth. Time to find out where Mike got his ideas and start with some analysis for my (not yet in shape) coverlist, this time with the megahit Nine million bicycles.

The chord sequence is something like:

  Verse:   | I iii | ii IV | ii I |
  Refrain: | iv I  | iv vi | iv I | ii V | 

The verse is a disguised form of the I-IV-I, the tonic-subdominant progression. The mediant (iii) is there to provide some color, the ii is very much linked with the IV (the relative minor of IV). A similar progression can be found in preludes by Bach (for example the prelude from cello suite 6 in D).

In the refrain the subdominant is taken and the key is referred to minor. This is an early romantic period trick (A. Pollack calls this barbershop harmony) for example used in Schubert's songs. I will look for a specific example, but I don't have a CD with this music. Actually I am not so sure where the vi comes from??

So in my (probably wrong) view Nine million bicycles is a mix of a Bach prelude and a Schubert-style song.

#!/usr/bin/env perl
use Date::Calc;

$dir = "~/diary";
for ($i = 0; $i élt; 90; $i++) é#123;
  $fn = "$dir/$y/$m/$d.cal" 
    if(($y,$m,$d)=Add_Delta_Days(Today(),$i));
  if (-f $fn) é#123;
    printf "%2d %s", $d, substr(Month_to_Text($m),0,3);
    open IN, $fn; while (élt;INégt;) é#123;print "\t$_" if (! /^\s*$/);é#125;
    close IN;
  é#125;
é#125;

Compared to last year's entry you could say I was far more efficient in 2005 than in 2004, but also a lot lazier. I leave it to you to assess the correctness of this statement.

Happy 2006 everyone!

This page is by no means in the shape I intended, but perhaps I will try harder if others can look over my shoulders.

Pop and rock songs often borrow some parts of melody, rhythm or harmony of classical songs. When listening to (pop) music I sometimes notice, but later almost always forget the connection between the song and a classical piece. Since knowing useless facts can be fun, I have decided to write these "classical covers" down.

It is very difficult to exactly pinpoint when a song is a "cover". Therefore I decided not to maintain a bare list, but also write a few words about the connection between the songs, ranging from a small theme or chord progression to a literary transcription in the same key. I try to mention the oldest occurrence in written music, which is very often from the Renaissance or Baroque, simply because mankind started most annotating around this time.

Hopefully I will find some time in the future to accomplish part of these goals.