Better code downloading with AJAX

I've been playing with Code downloading (or Javascript on Demand) a little more.

Michael Mahemoff pointed me at his great Ajaxpatterns in which he suggests a different solution:

if (self.uploadMessages) { // Already exists
return;
}
var head = document.getElementsByTagName("head")[0];
var script = document.createElement('script');
script.type = 'text/javascript';
script.src = "upload.js";
head.appendChild(script);

Via DOM manipulation a new script tag is added to our document, loading the new script via the 'src' attribute. I have put a working example here. As you can see this does not even need to do an XmlHttpRequest (XHR later on) so it will also work on browsers not supporting that.

So why use this approach and not mine? Initially I thought that it was not as good as doing it via XHR because you receive a direct feedback (i.e. a function call) when the script has been loaded. This is per se not possible with this technique. But as in good ol' times a simple function call at the end of the script file will do the same job (compare source codes from the last example and this one (plus load.js)).

Using this method to load code later on also provides another "feature" (thanks for that hint to Erik Arvidsson): Unlike XHRs Firefox also provides a cache for scripts loaded that way. There seems to be a disagreement about whether this is a bug or a feature (people complaining that IE caches such requests while it could be quite useful in this scenario).

When using dynamically generated javascript code you will also have to keep your HTTP headers in mind (scripts don't send them by default). The headers Cache-Control and Last-Modified will do usually (see section 6.1.2 of my thesis)

The method above is also the method used by Dojo, a developer (David Schontzler) commented, too. He says that Dojo also only loads the stuff the programmer needs, so little overhead can be expected from this project.

Also Alex Russell from Dojo left a comment about bloated javascript libraries. He has some good points about script size to say (read for yourself), I just want quote the best point of his posting:

So yes, large libraries are a problem, but developers need some of the capabilities they provide. The best libraries, though, should make you only pay for what you use. Hopefully Dojo and JSAN will make this the defacto way of doing things.

So hang on for Dojo, they seem to be on a good way (coverage of Dojo to follow).

Finally I want to thank you all for your great and insightful comments!

, , , , ,

No Google Office To Be Expected

According to an interview of Sergey Brin (founder of Google) by John Battelle, Google does not plan to publish a web based office, as rumored before (fueled by a new partnership between Google and Sun (owner of openoffice.org)).

"I don't really think that the thing is to take a previous generation of technology and port them directly," he said. I agree with that. We have to think about new tools to be our future web apps. Just porting office apps to the web won't work.

So what's this deal about now? Google seems to want a wider distribution of their toolbar. According to the Cnet coverage (via John Battelle again):

"Details about what exactly that will entail were vague at best, with the only nugget offered being that Sun, in the immediate future, will make Google's toolbar a standard part of the package when users download Sun's Java Runtime Environment from the server seller's Web site."

Sergey Brin further stated that distributed thin web applications allowed you to do "new and better things than the Office package and more." So further tools by Google can be expected.

, , , ,

Code downloading with AJAX

Earlier, I suggested to use Code Downloading in order to reduce the size of AJAX application. I left the term undescribed, but I will change this now:

As JavaScript is an interpreted language, it is quite easy to load additional code, even after the application has "started". This means that only code absolutely necessary to display the app has to be loaded initially.

In the following example, we define a function test() in the context of an object App. Then via Ajax the original code is overwritten. Naturally also new functions can be loaded.



The downloaded code is eval'uated, i.e. it is executed. You cannot only execute statements but also define variables and functions.
Source of load.js:

App.test = function() {
alert("additional code loaded");
}

I have set up an example implementation of this.

This allows more flexibility for larger apps. My "negative" example, Kiko, could use this method to enormously reduce the amount of code to be loaded initially.

I alse see the possibility to only store encrypted Javascript source code on the server and decrypt it on-the-fly (of course also this would only prevent script kiddies from stealing, but it could challenge some hacker a bit more).

,

Documenting prototype.js for AJAX

As the prototype.js library lacks documentation and I recommend to use the down cut version for AJAX, I thought it might be useful to document how to use the AJAX related functions.

A "normal" AJAX callback

When you just want to do a post-back to the server (for example to store some data the user has just changed) you'd use the following piece of code:




A simple corresponding PHP script (store.php) would look like this:

< ?php if (!isset($_POST["value"])) { die("no value given!"); } $db = mysql_connect(); mysql_select_db("test"); mysql_query("INSERT INTO table (value) VALUES ('" + addslashes($_POST["value"]) + "')"); mysql_close(); echo "successful"; ?>

The value the user entered in the prompting box will be stored to a MySQL database. The magic happens in line 2 of the function store(): new Ajax.Request will do an HTTP POST to the given script at the first parameter (store.php) with the parameters given in the second argument. The yet unusual notations within curly brackets denotes a hash (also called associative array) that stores key/value pairs. Here the pair parameters (key) and '&value=' + value (value) are stored. Other possible key/parameter functions can be found at the preliminary documentation of prototype.js by Sergio Pereira.

This brings up a problem common to AJAX applications: you explicitly need to inform the user about what happened, or if it failed. This can be done via an Event handler. The most useful one might be onComplete. It is inserted like this:


function store() {
value = prompt("Give me a value, please.");
new Ajax.Request("store.php", {
parameters: '&value=' + value,
onComplete: function (req) {
if (req.responseText == "successful") {
alert("Your value has been stored successfully");
} else {
alert("Something went wrong when storing your value");
}
}
});
}

This will display an appropriate alert box depending a successful status message by the script. In near future I will describe how to use the Ajax.Updater feature which allows to easily modify your existing page to instantly display results of a query.

, , ,

prototype.js just for AJAX

As I stated earlier, the prototype.js library is too large for just using AJAX. In its current version (1.4.0_pre10) it weighs 36KB and contains lots of other features that are most probably not needed when just dealing with AJAX.

I have therefore created a smaller version just for AJAX, based on 1.4.0_pre10: pt.ajax.js 8.9K

It now has only a quarter of size and still provides some nice features such as $ as a wrapper of document.getElementById.

Creating this was not too difficult: it is merely a combination of 4 files that make up prototype.js:

Just do copy and paste into a new files, i.e. copying each file to the bottom of your new javascript file.

As you can see, you can easily create your own customized (smaller!) version of prototype.js to fit your needs.

, ,

Rise of slow AJAX applications

The current movement towards AJAX is a good thing. If it really were a movement towards AJAX. In my eyes it is rather a higher acceptance for Javascript applications. Of course, it is quite naturally in the early stages of a "hyped" technology to observe many misuses; they use AJAX just for the sake of using AJAX.

Pages get more voluminous because so much code has to be loaded to the browser (which makes the browser slow again) so you could just begin to use the application. This somehow reminds me of all the flash apps. Waiting for hours to load the page and you'll stick to that page for half a minute. (I do have a broadband connection. Still a page with 30kb loads 10 times faster than a 300kb page).

A negative example for this is Kiko, a web calendar. It has a nice "Kiko loading" box which already hints that they are doing something wrong. All Javascript files are included via server side scripting instead of loading them via <script src="xyz.js"> which would allow the browser to cache the file.

Kiko is just one example, there are others doing similar mistakes.

I think that the current usage of AJAX is a misuse of the browser. They are designed to render web pages (i.e. (X)HTML pages). Javascript is a bonus. Large data strucures can slow down browsers enourmously (they are still interpreting Javascript just in time).

As a conclusion I want to come up with some essential features for AJAX applications:

  • Keep it bookmarkable. Don't load everything to one page, let users return to a certain section of your app via their bookmarks.
  • Don't overuse AJAX. Often simple Javascript without server interaction will do. Try to reduce the server callbacks.
  • Minimize the code to be loaded. When you don't have any other choice, consider code downloading.
  • Speed up your apps with AJAX. Use AJAX for what it was meant for: tune your existing application at points where postbacks need to reload the same page with only little change.

, , , ,

Bloated Ajax Applications Due to Libraries

When Ajax began to rise, there was quite a movement towards the prototype javascript library. This was also pushed by the great Ruby on Rails. Then came the visual effects of script.aculo.us. They look great, they really do. But for what price? Lots of KB of code.

alex@www:~/scriptaculous/$ du *.js -ch --apparent-size
23K controls.js
18K dragdrop.js
21K effects.js
28K prototype.js
899 scriptaculous.js
12K unittest.js
8.7K util.js
109K total

This is unacceptable for just a library. Most of these KB's have to be downloaded and do not provide any functionality per-se. Broadband is not an argument here. To load or not to load 100kb is relevant.

I therefor really like the Sack of Ajax. It takes only about 4K:

alex@www:~/sack/$ du *.js -ch --apparent-size
3.9K tw-sack.js
3.9K total

Now this does not give us all the script.aculo.us stuff. For that case I suggest to just reuse the relevant parts of it. Just let the user download what you really use. Maybe one day we will see a reduced script.aculo.us. Or an alternative using Sack.

UPDATE: I now recommend to use protoype.js again, in a reduced version just for AJAX.

, , ,

Setting up exim4 on Debian

i have been dealing with setting up a mail server, lately. debian seems to have a preference for exim. most of the mail admins i know also support this. so i had a look at this.

as it always happens to me, i start with a not so easy scenario, but after some figuring i got it, mostly by following this guide: Configuring Exim4 and Courier IMAP under Debian GNU/Linux.

what the article does not say (and might be common knowledge — i will still describe this here for anyone who does this the first time) to have mails addressed to domain xyz.com be sent to the appropriate mail server, there needs to be a so called MX entry (mail exchanger). See also section 5 of RFC 2821.

I've got some more useful links on this topic:
Virtual Domains with Exim + Courier-IMAP + MySQL
Eleven Examples for Configuring Exim
Secure Mail Relaying with Exim and OpenSSL

there also is the Exim FAQ and the Exim documentation, but I'm not to fond of lots of on-screen reading.

, ,

PHP and Multibyte

ever messed around with umlauts or other non [a-z] letters? it's quite horrible.

for the german speaking region there are mainly two encoding types: iso8859-1 and utf-8. the former encodes each letter with one byte by extending old 7-bit ascii with 127 more letters, amongst others also umlauts. utf-8 includes up to 32,640 more letters (ascii 0x80-0xff are used to select the range of the following byte). this is established by allowing multi-byte characters. in the case of utf-8 the maximum is two letters, but there exist utf-16 and utf-32 with up to 4 bytes per char.

so, what's the problem? with bandnews we have different sources for our data, meaning that we receive many pages with many different encodings and have to deliver a page that follows only one encoding. we chose to use utf-8 now, because a wide range of letters from many other encodings can be displayed which are not included in iso8859-1.

now it is important that you stop using strlen and substr because it can easily happen that you split an utf-8 character into parts, and forget comparing it to anything, then. alterenatives are mb_strlen and mb_substr and all other sorts of mb_* functions. well… this does not work out of the box, you need to specify what encoding is to be expected. this can be done like this:

mb_internal_encoding("UTF-8");

all mb_* commands use this encoding if no other is specified.

still, non-utf-8 code can come through to the browser, e.g. if you receive it from the database. but there is a chance to get around this quite comfortably:

mb_http_output("UTF-8");
ob_start("mb_output_handler");

the output buffer is cleared from wrong charactes by the mb_output_handler. it is also easily possible to have the output converted to iso8859-1, just by specifying it with the mb_http_output command.
a drawback is, though, that no other output filter can be applied, such as for output compression

ob_start("ob_gzhandler");

the manual states that instead zlib compression should be used, as specified in the php.ini file or via ini_set:

ini_set ('zlib.output_compression', 'on');
ini_set ('zlib.output_handler', 'mb_output_handler');
ob_start();

note that the output-handler for ob_start has to be empty and it is moved to the config option. this sounds great, but i was not able to get it to work. well, i must admit that i did not put so much time into it because i simply decided to move the responsibility to apache: mod_deflate. you might want to modify the configuration line, as i did:

AddOutputFilterByType DEFLATE text/html text/plain text/xml text/javascript text/css

have fun with character encoding. it works after some while. but its a lot of trial and error.