Archive for the 'Javascript' Category

04
Mar

HTML 5 Multi + Drag and Drop File Uploads with ExtJS

In anticipation of the new Tine 2.0 version Mialena which is scheduled for March 2010 I’d like to show you a sneak preview about the new file upload features.

Tine 2.0 is now able to upload multiple files at once. Additionally Files can be uploaded from directly out of the operating systems file manager or desktop using drag & drop.

Being an open source project, it’s extremely important for us to chose freely available standard technologies. Therefore it has to to be emphasized, that these new features run in FF, Chrome and Safari without any additional plugins.

The ExtJS wrappers for File Browsing are wrapped into
Ext.ux.file.BrowsePlugin

HTML 5 File uploading for ExtJS is wrapped into
Ext.ux.file.Uploader

20
Jan

Ext JS 3.0 Cookbook Review

After seeing quite a few reviews of the ‘Ext JS 3.0 Cookbook‘ on the Ext JS related blogs, I was curious to hold a copy of the book in my hands.

As you can guess from the title, the book contains recipes for solving GUI designs with Ext JS 3.0. In total the book comes with 109 recipes divided into 10 chapters.

Chapter 1 gives a short introduction of the Ext JS way of DOM manipulation and other basic things. Together with Chapter 10, which is about design patterns in Ext JS this are the Chapters I like most in the book, cause they help to understand the whole Ext JS picture and present content you won’t find easily in the internet.

The 8 Chapters in between contain receipts as you expect from a cookbook. People familiar with the examples provided in the Ext JS SDK will have sawn about 70% of the receipts presented in the book. In Contrast to the examples in the SDK the printed recipes come with a lot of explanation suitable for beginners with Ext JS and non Javascript geeks. The recipes which are not part of the Ext JS SDK are of high quality and fill the gap where beginners might miss examples.

Chapter 2 tries to give an introduction into the Ext JS layout engine. I use the word “try” as I would expect that the receipts are not enough to explain a beginner the insights needed to understand and create complex layouts. Moreover this chapter misses to explain the new vbox and hbox layouts introduced in Ext JS 3.0. In my view, this layouts are the most important additions to the Ext JS layout engine in the new version.

Each recipe contains the additional sections “How it works” and “There is more”. Both of this sections are helpful for people who don’t just want to cook, but want to gain more insights of the Ext JS framework.

While a recipes book always has newbies and beginners as target audience, this book is not for beginners only. The explanations often stress points which are of importance for experienced users of the Ext JS framework. Moreover the Author frequently gives hints which are of importance for big web applications.

With the precise explanations and the elegance of his own receipts the author proves himself to have an excellent skill level in providing Ext JS based solutions. The examples provided in this book are not just working somehow, they are assembled within the spirit and design constrains of the Ext JS framework itself.

Additionally, it is notable that the version number in the books title reflects the actual state of the contents. In a number of receipts the solution is done using new features which where introduced in the 3.0 version of Ext JS.

In a big online book store, I would rate this book with 4 of 5 stars.

09
Dec

Cross Site AJAX Window Name Proxy for extjs/ext-core

Cross site communication in the browser using the XHTTPRequest Object is prohibited by the Same Domain Policy. With HTML 5 this restriction is relaxed within the so called workers.

As of now, normally the browser can only communicate with the domain the document being displayed comes from. As always, most restrictions can be circumvented somehow. Quite a while now, the de facto standard approach for cross-site communication in the wild is JSONP.

In short: A javascript located at the foreign domain is inserted in background. Via the GET url parameters the data of the request is send. The script returned by the foreign server contains code which gets directly executed at include time of this script, e.g: callback({jsondata});

It is widely accepted that JSONP should be rated highly insecure for multiple reasons:
- Including a script of a foreign domain is a build in XSS attack.
- All parameters send to the foreign domain are part of the standard access logs of the foreign web servers.
- …

A new idea for secure cross site communication has come up a year ago utilizes the fact, that the name property of browser windows/frames remains unchanged when a window/frame loads a new document even from a foreign domain. With this, it is possible to create a transport by inserting a hidden iframe which switches its document between the origin- and foreign domain.

This approach has the major advantage, that the data returned by the foreign domain is encoded as a string and could be evaluated in a safely. Moreover the request could be done via POST’s which solves EOI issues with server logs.

While I was aware of this new technique, I never had the need to do cross site request from the browser. In general my approach is to proxy data from foreign domains via the server as this gives a better control. However for a Tine 2.0 related customer project I had the requirement to do cross site request from a website to a Tine 2.0 installation directly from the browser.

Unfortunately I didn’t find any extjs/ext-core implementation of the window.name technique. Moreover I realized, that the implementation of other libraries always POST their request using form submits. But doing all request via form Submits would require an additional API for Tine 2.0 as the standard JSON API talks JSONRPC.

So what I needed was a XHTTPRequest/AJAX solution to be able to utilize the standard Tine 2.0 API’s. For this I created a windowNameConnection Proxy which based on ext-core does AJAX requests to the foreign domain. This sequence diagram visualizes it’s simple workings:
windowNameConnectionDiagramm

You can find the latest version of the implementation here. The usage of this connection class is straight forward as it works like a normal extjs connection:


var connection = new Ext.ux.data.windowNameConnection({
    // this url is autodetected if windowNameConnection is loaded
    // from the foreign domain
    proxyUrl: https://foreign.domain/windowNameConnection.html
});

connection.request({
    url: https://foregin.domain/apiurl,
    header: {...}
    params: {...},
    success: someSuccessCb
});

Of course the ability to do AJAX requests is purchased by a minor performance loss compared to form posts. BUT: The windowNameProxy should only be seen as a fallback for Browsers which are not capable of the HTML5 workers. Users of these old browsers are obviously not keen on enhanced performance. Otherwise they would use an up to date browser.

18
Sep

Data Transport with JSON-RPC via Ext.Direct and Zend Framework Part 3

Now, with the results from Part 1 and Part 2, let’s take a deeper look into the Ext.Direct Providers to see how they work and what to do.

Prepareing a JSON Service Map Description

To use the Ext.direct framework we need to register a provider. For a remoting provider we need a service mapping description.

Ext.Direct.addProvider(Ext.apply(Ext.app.JSONRPC_API, {

    'type'     : 'zfprovider',
    'url'      : Ext.app.JSONRPC_API.target
}));

Ext.app.JSONRPC_API holds the service mapping description which in the Tine 2.0 case we deliver it with our registry data, but you can also fetch it e.g. by a javascript include as suggested in the “EXT.DIRECT REMOTING SPECIFICATION”.

<script src=”index.php?method=Tinebase.getServiceMap” type=”text/javascript”></script>

The Service Map could be generated like this:

$server = new Zend_Json_Server();

// set all classes you want to expose
$server->setClass('MyModule_Service_Json', 'MyModul');

$server->setTarget('index.php')
    ->setEnvelope(Zend_Json_Server_Smd::ENV_JSONRPC_2);

$smdArray = $server->getServiceMap()->toArray();
// save some bytes
unset($smdArray['methods']);

echo "Ext.app.JSONRPC_API = " . Zend_Json::encode($smdArray);

Ext.ux.direct.ZendFrameworkProvider

To Enable Ext.Direct to understand the JSON-SMD data and also communicate using the JSON-RPC protocol we create an own provider:

Ext.ux.direct.ZendFrameworkProvider = Ext.extend(Ext.direct.RemotingProvider, {

...

At first we need to parse the JSON-SMD and to create the stubs. This is done by overwriting the initAPI method:

// private

    initAPI : function() {
        for (var method in this.services){
            var mparts = method.split('.');
            var cls = this.namespace[mparts[0]] || (this.namespace[mparts[0]] = {});
            cls[mparts[1]] = this.createMethod(mparts[0], Ext.apply(this.services[method], {
                name: mparts[1],
                len: this.services[method].parameters.length
            }));
        }
    },


All we do is to transform the JSON-SMD definition data in to a from the original createMethod method can understand to create the stubs.

It’s important to understand, that the createMethod creates stubs which do only trigger further processing of the provider and do _NOT_ contain all the code to do requests and protocol processing.

Once a stub is called, it applies the doCall function of our provider. In order to support named parameters we inspect the call and transform a named parameter call into a positional parameter call. This is needed, cause the ZendFrameworks Zend_Json_Server is itself not able to understand named parameter calls witch by the way are indeed part of the JSON-RPC specification.

// private

    doCall : function(c, m, args) {
        // support named parameters
        if (args[args.length-1].paramsAsHash) {
            var o = args.shift();
            for (var i = 0; i < m.parameters.length; i++) {
                args.splice(i,0, o[m.parameters[i].name]);
            }
        }

        return Ext.ux.direct.ZendFrameworkProvider.superclass.doCall.call(this, c, m, args);
    },

After the parameters a prepared, the provider assembles the request according to the protocol definitions. This is done in the getCallData method.

// private

    getCallData: function(t){
        return {
            jsonrpc: '2.0',
            method: t.action + '.' + t.method,
            params: t.data,
            id: t.tid
        };
    },


This protocol encapsulation only deals with single transaction calls. The processing for batched calls is done elsewhere. Luckily The both protocols use the same simple outer array for batched calls, so that we don’t need to touch it.

Finally the response needs to be decoded according to the protocol and the callback function needs to get called using the decoded data. This is done in the onData method:

// private

    onData: function(opt, success, xhr) {
        var rpcresponse = Ext.decode(xhr.responseText);
        xhr.responseText = {
            type: rpcresponse.result ? 'rpc' : 'exception',
            result: rpcresponse.result,
            tid: rpcresponse.id
        };

        return Ext.ux.direct.ZendFrameworkProvider.superclass.onData.apply(this, arguments);
    }

And voila, thats it. This is our custom provider to enable Ext.Direct to use and talk pure standard JSON-PRC/JSON-SMD.

Finally let’s register the provider for lazy invocation

Ext.Direct.PROVIDERS['zfprovider'] = Ext.ux.direct.ZendFrameworkProvider;

Server Side Batched Requests

As noted in Part 2, the biggest problem with the Zend_Json_Server is the fact that it’s not capable to handle batched requests yet. To overcome this in a simple way, you can monitor the first character of an JSON request in your dispatcher and dispatch multiple request if it’s a ‘[‘. Of course this is far away from optimum, but it’s a start point till we have the feature in the Zend_Json_Server.

04
Sep

Data Transport with JSON-RPC via Ext.Direct and Zend Framework Part 2

As promised in my previous post, I’ll introduce Ext.Direct and discuss how it fits with the new JSON-RPC standards and especially with the Zend_Json_Server component.

Ext.direct is a namespace and a bit of a buzzword in the ExtJS 3.0 release. In short, the Ext.direct stuff introduces high level communication features.

The key component of these new features is the “EXT.DIRECT REMOTING SPECIFICATION” which covers the aspects ‘remote procedure call‘ and ’service description’. Ext.Direct implements this specifications within the ExtJS framework as service consumer and for multiple server side stacks as service provider.

Besides having a well defined and documented communication and transport layer Ext.direct also has the advantage, that other ExtJs classes dealing with data can work on the stubs, created based on the ’service map descriptions’. The Ext.tree.TreeLoader can be configured with a directFn to fetch it’s data. More over the Ext.data.DirectStore can be configured with a complete set of direct functions for CRUD actions.

An other cool thing about the javascript implementation of the Ext.direct.RemotingProvider is, that is queues request of the direct stubs for a configurable time span. After this span, it sends one batched request to the server.

However, its important to note, that the “EXT.DIRECT REMOTING SPECIFICATION” is different form the JSON-RPC/JSON-SMD specifications introduced in part 1.

While digging in the specs I found some pros and cons for the “EXT.DIRECT REMOTING SPECIFICATION” compared to the JSON-RPC standardization:

  • + It covers form posts which are needed for special actions like file upload
  • + It covers events send by the server
  • - It lacks of function parameter description
  • - It does not support named function parameters
  • - It does not support optional parameters
  • - It is not compatible to the JSON-RPC standardization efforts

The last point is IMHO the strongest point why not to use this specification. While the rest of the javascript/webservice addicted world tries to find a common standard, ExtJs goes its own way. There are already a number of implementations for the JSON-RPC and JSON-SMD out there, and more and more will follow. I also expect to see service consumers written in PHP which ease writing server-server web-services using the same API.

For that reasons I choose to take the standard Zend_Json_Server implementation and to write a Ext.ux.direct.ZendFrameworkProvider we can use in Tine 2.0 and other Zend Framework based projects.

It’s only fair to note, the the Zend_Json_Server also has several issues which needs to be improved. Most notably:

  • - It does not support batched requests
  • - It does not support named parameters

After all this theory I’ll cover the actual implementation of the Ext.ux.direct.ZendFrameworkProvider in part 3.

17
Aug

Data Transport with JSON-RPC via Ext.Direct and Zend Framework Part 1

By writing this post, I noticed that it’s quite some stuff, so I broke it up in parts. This part is about the new specifications which help to standardize JSON protocols/apis.

When we started to implement Tine 2.0 about 2 years ago, we got quite some confused request why we chose JSON over SOAP as our data transport between the server and the client. Technically the answer is very easy. JSON protocols only need a fraction of the bandwidths and even more important JSON has a significant better performance on client side, cause no XML parsing is needed.

On the other hand, two years ago SOAP was a mature protocol but JSON only an data encapsulation and no real standards did exist, so that we where forced to implement a custom protocol.

Fortunately this has changed in the last two years. There are three cool standardization efforts in the JSON community I’d like to introduce.

  1. JSON-RPC: It’s a lightweight remote procedure call (RPC) protocol. The clients request consist only of a single JSON data telegram and does not use HTTP parameters:
    --> {
        "jsonrpc":"2.0",
        "method":"Tinebase.login",
        "params": {
            "username":"tine20admin",
            "password":"lars"
        },
        "id":2
    }


    <-- {
    "id":"2",
    "jsonrpc":"2.0",
    "result":{
        "success":true,
        "account": {
            "accountId":"0e7d07cb453b15f520021e0a433956b234a7c0bd",
            "accountDisplayName":"Admin Account, Tine 2.0",
            "accountLastName":"Admin Account",
            "accountFirstName":"Tine 2.0",
            "accountFullName":"Tine 2.0 Admin Account",
            "contact_id":"1"
        },
        "jsonKey":"85160b59b4421bbe2f350c5b3888a2efdbfcb949",
        "welcomeMessage":"Welcome to Tine 2.0!"
    }}

    Multi batch calls are also possible. The client only has to send an array of request objects. Unfortunately the Protocol does not handle “Form-Posts” which are necessary for some special actions like file-uploads.

    But nevertheless libraries are starting to implement the spec and this is the most important thing about it!

  2. JSON SMD: It’s a JSON representation for describing a web service. The server ‘exposes’ it’s services to the client.
    {
    "transport":"POST",
    "envelope":"JSON-RPC-2.0",
    "contentType":"application/json",
    "SMDVersion":"2.0",
    "target":"index.php",
    "services": {
        "Tinebase.login":{
            "envelope":"JSON-RPC-2.0",
            "transport":"POST",
            "parameters":[{
                "type":"string",
                "name":"username",
                "optional":false
            }, {
                "type":"string",
                "name":"password",
                "optional":false}
            ],
    "returns":"array"}}}

    The client can use this descriptions to automatically create the stubs for calling the services which already encapsulates all the proxy and transport and protocol overhead. With this, instead of creating an AJAX request per hand you only have to call the method:

    Tinebase.login('tine20admin', 'lars', cb);

    where cb is a callback function, cause we do our requests asynchronous.

  3. JSON Schema: Quoting their website: “JSON Schema is a specification for a JSON-based format for defining the structure of JSON data.”
    Of course the idea is pretty old in the world of web-services, but it’s the first approach to bring this to the new area of javascript based RIA’s which use JSON for the reasons stated above.With this we not only expose the services to the client, but more over also the data types. The benefit is clearly that we don’t need to maintain the data models client side and also the client side data validation could be made client side automatically.

In Part 2 I’ll introduce Ext.Direct and go into the details how to apply this new standards within the Ext.Direct framework.

26
May

Event Calendar Views made with ExtJS core library

The last few weeks I worked on the calendar views for the upcoming Tine 2.0 release. For sure, the new calendar also will be made with the great UI technologies like all ExtJS widgets. Unfortunately, there was no Calendar widget out there, so I had to get my hands dirty and code my own one.

At the moment I have Day, Week and Monthview working. The Day and Week view supports direct creation of Events using dblclick or a mousedown/mousemove combination. All views support drag and drop of events. For fast switching of views, you can click the jumppads (dayheaders/weeknumer).

I made a litte MemoryProxy as Backend for the demo bellow, so you can change views and move around in time. The whole demo is Javascript only. The Calendar views are designed like normal ExtJS widgets and should be highly reusable. If you want to join development for them, please drop me a note.

27
Apr

First Impression of the new Tine 2.0 Calendar

Just a litte update for all of you, waiting for the new calendar:

nelius$ phpunit --verbose Calendar_AllTests AllTests.php
PHPUnit 3.2.19 by Sebastian Bergmann.

Tine 2.0 Calendar All Tests
Calendar_RruleTests
............

Calendar_Backend_SqlTests
.......

Calendar_Controller_EventTests
..................

Time: 24 seconds

 

OK (37 tests)

Most of hard work behind the scenes (aka backend) is finished. Implementing the recuring rules to behave correctly for participants in different timezones was quite a challenge, but I’m quite hopeful that with our implementation we won’t have the moving events after ‘Day light saving’ boundaries like we had in the other project.

This week I’ll start to implement the interfaces based on ExtJS. Anne already gave some great impact on the calendar organization schema. Please do support her survey coming up in the next days.

06
Apr

New Language Statistics Viewer

Today I updated the languate statistics viewer for Tine 2.0. You now can select the versision to see the statistics for.

langstats

After the campaign for new translators, we got new translators for Spanish(en), French(fr), Dutch(nl), Portuguese(pt) and  Traditional Chinese(zh_TW). At the moment, they are working on the translations for our current stable release (Lara). If the translations are finished, we’ll add them to our next service release for that version.

02
Mar

Progress on a Shiny Tine 2.0 Setup

As the Tine 2.0 community keeps growing, also the need for a setup GUI is growing. One the one hand we need to support the ‘non computer freak admins’ which are installing Tine 2.0 and on the other hand we need tools for the growing developers community to install/uninstall their own applications or to only particularly upgrade the system.

So here a little snapshot of current development progress, feel free to leaf you comments.

The installation requirements are not met. It's not possible to continue setup. Maybe we also need some kind of alert "Your server is not capable to run Tine 2.0. Sorry!"

The installation requirements are not met. It's not possible to continue setup. Maybe we also need some kind of alert "Your server is not capable to run Tine 2.0. Sorry!"

Setup checks are passed, but no config file is found. Here we need a little form to support creation of a config.inc.php file.

Setup checks are passed, but no config file is found. Here we need a little form to support creation of a config.inc.php file.

If a config file is present, you need to login into setup (credentials from config.inc.php)

If a config file is present, you need to login into setup (credentials from config.inc.php)

You have a config file and successfully logged in, but something is not working with your config, e.g. SQL or LDAP is down.

You have a config file and successfully logged in, but something is not working with your config, e.g. SQL or LDAP is down.

Application Manager

If all requirements are met, you are ready to run the application manager. This is the place to install / uninstall and upgrade your applications

 

Whats the scope of the setup?

In the setup all things needed to run Tine 2.0 itself should be configured. This is the main database connection on the one hand, and the authentication and account drivers on the other hand. Things like email, voip, langs etc could be configured directly from the admin section in Tine.