Fix qTranslate with WordPress 3.9

April 17th, 2014 by Alexander Kirk | Comments No Comments »

When updating a blog of mine to WordPress 3.9 the page wouldn't load anymore because of qTranslate not able to cope with the update. In the error log it says:

PHP Catchable fatal error: Object of class WP_Post could not be converted to string in ../wp-content/plugins/qtranslate/qtranslate_core.php on line 455

The error is caused by this change: get_the_date() to accept optional $post argument

There is a proposed quick fix by Saverio Proto, but it doesn't take the problem at its root:

qTranslate registers the function qtrans_dateFromPostForCurrentLanguage($old_date, $format ='', $before = '', $after = '') for the hook get_the_date but it actually only accepts one parameter. With the new update it accepts a second parameter $post, which now wrongly fills the variable $before that is in the process being converted to a string.

So the solution simply is to delete the two parameters that were assigned the wrong meaning and have defaults anyway.

Add a Rate Limit to Your Website

April 19th, 2013 by Alexander Kirk | Comments Comments Off

Suppose you have a ressource on the web (for example an API) that either generates a lot of load, or that is prone to be abused by excessive use, you want to rate-limit it. That is, only a certain number of requests is allowed per time-period.

A possible way to do this is to use Memcache to record the number of requests received per a certain time period.

Task: Only allow 1000 requests per 5 minutes

First attempt:
The naive approach would be to have a key rate-limit-1.2.3.4 (where 1.2.3.4 would be the client's IP address) with a expiration time of 5 minutes (aka 300 seconds) and increment it with every request. But consider this:

10:00: 250 reqs -> value 250
10:02: 500 reqs -> value 750
10:04: 250 reqs -> value 1000
10:06: 100 reqs -> value 1250 -> fails! (though there were only 850 requests in the last 5 minutes)

Whats the problem?

Memcache renews the expiration time with every set.

Second attempt:
Have a new key every 5 minutes: rate-limit-1.2.3.4-${minutes modulo 5}. This circumvents the problem that the key expiration but creates another one:

10:00: 250 reqs -> value 250
10:02: 500 reqs -> value 750
10:04: 250 reqs -> value 1000
10:06: 300 reqs -> value 300 -> doesn't fail! (though there were 1050 requests in the last 5 minutes)

Solution:
Store the value for each minute separately: rate-limit-1.2.3.4-$hour$minute. When checking, query all the keys in the last 5 minutes to calculate the requests in the last 5 minutes.

Sample code:


foreach ($this->getKeys($minutes) as $key) {
    $requests += $this->memcache->get($key);
}

$this->memcache->increment($key, 1);

if ($requests > $allowedRequests) throw new RateExceededException;

For your convenience I have open sourced my code at github: php-ratelimiter.

Thinkery API launched

April 5th, 2013 by Alexander Kirk | Comments Comments Off

Just a quick note, we made the Thinkery API public.

If you don't know thinkery.me, it is a simple yet powerful tool for storing both notes and bookmarks. The contents of the saved page is stored in your Thinkery which you can keep even if the webpage goes down. With #hashtags you can easily categorize everything.

Check it out!

munin smart plugin: ignore error in the past

February 7th, 2013 by Alexander Kirk | Comments Comments Off

As a hard drive in my server failed, my hosting provider exchanged the drive with another one which obviously had some sort of error in its past, but now seems to be fully ok again. I would have wished to receive a drive without any problems but as my server is RAID 1, I can live with that.

I do my monitoring with Munin and for monitoring my hard drives I use the smart plugin. Now this plugin also monitors the exit code of smartctl, where smartctl sets bit no 6 if there was an error in the past, so now while everything is alright, the exit code is always numeric 64.

Now the smart plugin reports this as an error, if the exit code is > 0, i.e. now it always reports a problem.

I could set the threshold to 65, but then I wouldn't be notified of other errors which essentially makes the plugin useless.

I asked at Serverfault but no one seems to have a solution for that.

So I attacked the problem on my own and patched the plugin. In the source code the important line is here:


if exit_status!=None :
# smartctl exit code is a bitmask, check man page.
num_exit_status=int(exit_status/256)

which I have modified to look like this:

if exit_status!=None :
# smartctl exit code is a bitmask, check man page.
num_exit_status=int(exit_status/256)
# filter out bit 6
num_exit_status &= 191
if num_exit_status<=2 :
exit_status=None

if exit_status!=None :

Now it doesn't bug me anymore when bit 6 is set, but if any other bit goes on again, I will still be notified. The most interesting part is the line where there is a bitwise operation with 191: this is 0x11011111 in binary, so doing an AND operation with the current value it will just set bit no 6 to 0 while letting the other values untouched.

Therefore a value of 64 (as mine does) will be reported as 0 while a value of 8 would remain at 8. But also, very importantly, a value of 72 (bit 6 set as always and bit 3 set because the disk is failing) it would also report 8.

And there we have another reason, why it is good to be firm with knowledge about how bits and bytes behave in a computer. Saved me from a warning message every 5 minutes :-)

Genial Daneben Analyse

December 23rd, 2012 by Alexander Kirk | Comments Comments Off

Zur Abwechslung mal ein Post auf deutsch. Ich bin Fan der (inzwischen abgesetzten) Fernsehsendung Genial daneben. Es gibt da eine Genial Daneben Datenbank mit (nahezu) allen Fragen, die in den Sendungen vorkamen. Ohne konkreten Nutzen habe ich diese Daten aus der Text-Form in eine echte Datenbank konviertiert (Script hier) und bin zu folgender Tabelle gekommen. Für alle die es interessiert:

Fragen: 2732 (1087 beantwortet, das sind fast 40%)

Teilnehmer Teilnahmen gelöst gelöst pro Teilnahme
Bernhard Hoecker 399 223 0.56
Hella von Sinnen 390 202 0.52
Wigald Boning 120 56 0.47
Guido Cantz 96 51 0.53
Barbara Schöneberger 62 14 0.23
Dieter Nuhr 62 18 0.29
Bastian Pastewka 62 6 0.10
Ralf Schmitz 60 6 0.10
Oliver Kalkofe 55 6 0.11
Herbert Feuerstein 47 25 0.53
Georg Uecker 39 13 0.33
Olli Dittrich 38 11 0.29
Oliver Welke 37 11 0.30
Ingo Appelt 32 8 0.25
Martin Schneider 31 0 0.00
Michael Kessler 29 6 0.21
Thomas Hermanns 27 7 0.26
Anke Engelke 26 8 0.31
Matze Knop 23 2 0.09
Christoph Maria Herbst 20 3 0.15
Mario Barth 20 8 0.40
Jürgen von der Lippe 19 3 0.16
Lou Richter 19 7 0.37
Ingo Oschmann 13 2 0.15
Guildo Horn 13 1 0.08
Dirk Bach 12 3 0.25
Kim Fisher 12 1 0.08
Urban Priol 10 2 0.20
Cordula Stratmann 9 0 0.00
Eckart von Hirschhausen 9 0 0.00
Tetje Mierendorf 8 0 0.00
Oliver Pocher 8 0 0.00
Bodo Bach 8 0 0.00
Hennes Bender 7 1 0.14
Rüdiger Hoffmann 7 0 0.00
Elton 7 1 0.14
Johann Köhnich 7 0 0.00
Bürger Lars Dietrich 7 2 0.29
Helge Schneider 7 7 1.00
Anka Zink 6 0 0.00
Hans Werner Olm 6 0 0.00
Cindy aus Marzahn 6 0 0.00
Kaya Yanar 5 2 0.40
Mike Krüger 5 2 0.40
Horst Lichter 5 2 0.40
Susanne Pätzold 5 1 0.20
Jochen Busse 5 1 0.20
Karl Dall 5 0 0.00
Mirja Regensburg 4 1 0.25
Oli Petszokat 4 2 0.50
Janine Kunze 4 0 0.00
Michael Mittermeier 4 0 0.00
Paul Panzer 3 0 0.00
Florian Schroeder 3 0 0.00
Konrad Stöckel 3 2 0.67
Axel Stein 3 0 0.00
Gayle Tufts 3 1 0.33
Verona Pooth 3 1 0.33
Zack Michalowski 2 0 0.00
Atze Schröder 2 1 0.50
Emily Wood 2 1 0.50
Susanne Fröhlich 2 0 0.00
Gabi Decker 2 0 0.00
Helfried 2 1 0.50
Mirja Boes 2 1 0.50
April Hailer 2 0 0.00
Michael "Bully" Herbig 2 1 0.50
Roberto Cappelluti 2 0 0.00
Olaf Schubert 2 0 0.00
Sissi Perlinger 2 0 0.00
Ottfried Fischer 2 0 0.00
Klaus Eberhartinger 2 0 0.00
Rick Kavanian 2 0 0.00
Lisa Feller 1 0 0.00
Sascha Korf 1 0 0.00
Marc Metzger 1 0 0.00
Joachim Fuchsberger 1 1 1.00
Martin Klempnow 1 0 0.00
Tom Gerhard 1 0 0.00
Ralph Morgenstern 1 1 1.00
Valerie Bolzano 1 0 0.00
Kurt Krömer 1 0 0.00
Hubertus Meyer-Burkhardt 1 0 0.00
Smudo 1 1 1.00
Badesalz (Gerd Knebel und Hendrik Nachtsheim) 1 0 0.00
Markus Maria Profitlich 1 1 1.00
Sven Nagel 1 0 0.00
Bernd Stelter 1 0 0.00
Matthias Matschke 1 0 0.00
Otto Waalkes 1 0 0.00
Till Hoheneder 1 0 0.00
Waldemar Hartmann 1 0 0.00
Lisa Fitz 1 0 0.00
Fatih Cevikkollu 1 0 0.00
Matze Knop (als Franz Beckenbauer) 1 0 0.00
Ralf Morgenstern 1 0 0.00
Ulrike von der Gröben 1 0 0.00

Ich habe den Großteil der Daten nicht überprüft, aber im Groben dürfte es stimmen.

Android WebView: Web page not available

October 11th, 2012 by Alexander Kirk | Comments Comments Off

Just a quick note in order to save someone else searching for a solution to this problem.

When you want to display HTML content in an Android WebView do it like this:

String html = "my >b<HTML content>/b<. 100% cool.";
WebView webView = (WebView) findViewById(R.id.myWebView);
webView.loadData(">?xml version=\"1.0\" encoding=\"UTF-8\" ?<" + html.replace("%","%25"), "text/html", "UTF-8");

If you don't replace the % to its url encoded equivalent you will get a "Web page not available". Simple, arguable but not at all apparent.

Git tip: Changing your mind: Push pending changes to a (not-yet existing) new branch

February 15th, 2012 by Alexander Kirk | Comments 1 Comment »

It happens quite often to me that I start committing things and only afterwards decide I should have created a new branch.

So git status says something like:

Your branch is ahead of 'origin/master' by 4 commits.


but I don't want to push to origin/master but rather create a new branch. (Of course this works for any other branch, not named master)

So you can use this sequence of commands:

git checkout -b newbranch
git push origin newbranch
git checkout master
git reset --hard origin/master

Explanation: This…

1. creates a new branch pointing to the current changes (and switches to it)

2. pushes this new branch including the changes to the server

3. switches back to the branch master
4. and undoes the changes that were made locally

The important thing to know is that the changes remain in the repository because a branch is merely a pointer to a commit.

Afterwards you can continue to commit to master, for example:

(screenshots done with a fork of gitx)

hckr news

January 26th, 2012 by Alexander Kirk | Comments 1 Comment »

Just wanted to point you all to Wayne Larsen's alternate interface to Hackernews, called hckr news.

I really like the clean and fast interface, compared to the everlasting old interface on the original page (even though you can of course use User CSS to pep it up a little).

Wayne also created browser extensions that do a little more good to the old Hacker News interface. He also included my Collasible Threads bookmarklet. Thanks!

Use an authenticated feed in Google Reader

January 3rd, 2012 by Alexander Kirk | Comments Comments Off

You currently can't subscribe to an authenticated feed (for example in Basecamp) in Google Reader.

If you want to do it nonetheless you can use this script of mine which will talk to the server that needs authentication, passing through all the headers (so that also cookies and "not modified" requests will come through): download authenticated-feed-passthru.php


<?php
// change this url
$url = "https://username:password@proj.basecamphq.com/projects/123/feed/recent_items_rss";

$ch = curl_init($url);

if (isset($_SERVER['REQUEST_METHOD']) && strtolower($_SERVER['REQUEST_METHOD']) == 'post') {
    curl_setopt($ch, CURLOPT_POST, true);
    curl_setopt($ch, CURLOPT_POSTFIELDS, $_POST);
}

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_HEADER, true);

$headers = array();
foreach ($_SERVER as $name => $value) {
    if (substr($name, 0, 5) != 'HTTP_') continue;
    if ($name == "HTTP_HOST") continue;
    $headers[] = str_replace(' ', '-', ucwords(strtolower(str_replace('_', ' ', substr($name, 5))))) . ": " . $value;
}
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

list($header, $contents) = preg_split('/([\r\n][\r\n])\\1/', curl_exec($ch), 2);
curl_close($ch);

foreach (preg_split('/[\r\n]+/', $header) as $header) {
    header($header);
}

echo $contents;

If you don't mind giving away your credentials you can also use Free My Feed.

New Feature for HN Collapsible Threads: Collapse Whole Thread

December 6th, 2011 by Alexander Kirk | Comments Comments Off

I have added a feature to the HN Collapsible Threads bookmarklet that enables you to close a whole thread from any point within the thread:

This is useful when you are reading a thread and decided that you are having enough of it and want to move on to the next thread. Before you had to scroll all the way up to the top post and collapse that one.

Drag this to your bookmarks bar: collapsible threads

Install Greasemonkey script

preg_match, UTF-8 and whitespace

October 1st, 2011 by Alexander Kirk | Comments Comments Off

Just a quick note, be careful when using the whitespace character \s in preg_match when operating with UTF-8 strings.

Suppose you have a string containing a dagger symbol. When you try to strip all whitespace from the string like this, you will end up with an invalid UTF-8 character:

$ php -r 'echo preg_replace("#\s#", "", "†");' | xxd
0000000: e280

(On a side note: xxd displays all bytes in hexadecimal representation. The resulting string here consists of two bytes e2 and 80)

\s stripped away the a0 byte. I was unaware that this character was included in the whitespace list, but actually it represents the non-breaking space.

So actually use the u (PCRE8) modifier as it will be aware of the a0 "belonging" to the dagger:

$ php -r 'echo preg_replace("#\s#u", "", "†");' | xxd
0000000: e280 a0

By the way, trim() doesn't strip non-breaking spaces and can therefore safely be used for UTF-8 strings. (If you still want to trim non-breaking spaces with trim, read this comment on PHP.net)

Finally here you can see the ASCII characters matched by \s when using the u modifier.

$ php -r '$i = 0; while (++$i < 256) echo preg_replace("#[^\s]#", "", chr($i));' | xxd
0000000: 090a 0c0d 2085 a0
$ php -r '$i = 0; while (++$i < 256) echo preg_replace("#[^\s]#u", "", chr($i));' | xxd
0000000: 090a 0c0d 20

Functions operating just on the ASCII characters (with a byte code below 128) are generally safe, as the multi-byte characters of UTF-8 have a leading bit of one (and are therefore above 128).

Restoring single objects in mongodb

May 31st, 2011 by Alexander Kirk | Comments Comments Off

Today I had the need to restore single objects from a mongodb installation. mongodb offers two tools for this mongodump and mongorestore, both of which seem to be designed to only dump and restore whole collections.

So I'll demonstrate the workflow just to restore a bunch of objects. Maybe it's a clumsy way, but we'll improve this over time.

So, we have an existing backup, done with mongodump (for example through a daily, over-night backup). This consists of several .bson files, one for each collection.

  1. Restore a whole collection to a new database: mongorestore -d newdb collection.bson
  2. Open this database: mongo newdb
  3. Find the items you want to restore through a query, for example: db.collection.find({"_id": {"$gte": ObjectId("4da4231c747359d16c370000")}});
  4. Back on the command line again, just dump these lines to a new bson file: mongodump -d newdb -c collection -q '{"_id": {"$gte": ObjectId("4da4231c747359d16c370000")}}'
  5. Now you can finally import just those objects into your existing collection: mongorestore -d realdb collection.bson