Archive for the 'php' Category

18
Sep

Data Transport with JSON-RPC via Ext.Direct and Zend Framework Part 3

Now, with the results from Part 1 and Part 2, let’s take a deeper look into the Ext.Direct Providers to see how they work and what to do.

Prepareing a JSON Service Map Description

To use the Ext.direct framework we need to register a provider. For a remoting provider we need a service mapping description.

Ext.Direct.addProvider(Ext.apply(Ext.app.JSONRPC_API, {

    'type'     : 'zfprovider',
    'url'      : Ext.app.JSONRPC_API.target
}));

Ext.app.JSONRPC_API holds the service mapping description which in the Tine 2.0 case we deliver it with our registry data, but you can also fetch it e.g. by a javascript include as suggested in the “EXT.DIRECT REMOTING SPECIFICATION”.

<script src=”index.php?method=Tinebase.getServiceMap” type=”text/javascript”></script>

The Service Map could be generated like this:

$server = new Zend_Json_Server();

// set all classes you want to expose
$server->setClass('MyModule_Service_Json', 'MyModul');

$server->setTarget('index.php')
    ->setEnvelope(Zend_Json_Server_Smd::ENV_JSONRPC_2);

$smdArray = $server->getServiceMap()->toArray();
// save some bytes
unset($smdArray['methods']);

echo "Ext.app.JSONRPC_API = " . Zend_Json::encode($smdArray);

Ext.ux.direct.ZendFrameworkProvider

To Enable Ext.Direct to understand the JSON-SMD data and also communicate using the JSON-RPC protocol we create an own provider:

Ext.ux.direct.ZendFrameworkProvider = Ext.extend(Ext.direct.RemotingProvider, {

...

At first we need to parse the JSON-SMD and to create the stubs. This is done by overwriting the initAPI method:

// private

    initAPI : function() {
        for (var method in this.services){
            var mparts = method.split('.');
            var cls = this.namespace[mparts[0]] || (this.namespace[mparts[0]] = {});
            cls[mparts[1]] = this.createMethod(mparts[0], Ext.apply(this.services[method], {
                name: mparts[1],
                len: this.services[method].parameters.length
            }));
        }
    },


All we do is to transform the JSON-SMD definition data in to a from the original createMethod method can understand to create the stubs.

It’s important to understand, that the createMethod creates stubs which do only trigger further processing of the provider and do _NOT_ contain all the code to do requests and protocol processing.

Once a stub is called, it applies the doCall function of our provider. In order to support named parameters we inspect the call and transform a named parameter call into a positional parameter call. This is needed, cause the ZendFrameworks Zend_Json_Server is itself not able to understand named parameter calls witch by the way are indeed part of the JSON-RPC specification.

// private

    doCall : function(c, m, args) {
        // support named parameters
        if (args[args.length-1].paramsAsHash) {
            var o = args.shift();
            for (var i = 0; i < m.parameters.length; i++) {
                args.splice(i,0, o[m.parameters[i].name]);
            }
        }

        return Ext.ux.direct.ZendFrameworkProvider.superclass.doCall.call(this, c, m, args);
    },

After the parameters a prepared, the provider assembles the request according to the protocol definitions. This is done in the getCallData method.

// private

    getCallData: function(t){
        return {
            jsonrpc: '2.0',
            method: t.action + '.' + t.method,
            params: t.data,
            id: t.tid
        };
    },


This protocol encapsulation only deals with single transaction calls. The processing for batched calls is done elsewhere. Luckily The both protocols use the same simple outer array for batched calls, so that we don’t need to touch it.

Finally the response needs to be decoded according to the protocol and the callback function needs to get called using the decoded data. This is done in the onData method:

// private

    onData: function(opt, success, xhr) {
        var rpcresponse = Ext.decode(xhr.responseText);
        xhr.responseText = {
            type: rpcresponse.result ? 'rpc' : 'exception',
            result: rpcresponse.result,
            tid: rpcresponse.id
        };

        return Ext.ux.direct.ZendFrameworkProvider.superclass.onData.apply(this, arguments);
    }

And voila, thats it. This is our custom provider to enable Ext.Direct to use and talk pure standard JSON-PRC/JSON-SMD.

Finally let’s register the provider for lazy invocation

Ext.Direct.PROVIDERS['zfprovider'] = Ext.ux.direct.ZendFrameworkProvider;

Server Side Batched Requests

As noted in Part 2, the biggest problem with the Zend_Json_Server is the fact that it’s not capable to handle batched requests yet. To overcome this in a simple way, you can monitor the first character of an JSON request in your dispatcher and dispatch multiple request if it’s a ‘[‘. Of course this is far away from optimum, but it’s a start point till we have the feature in the Zend_Json_Server.

04
Sep

Data Transport with JSON-RPC via Ext.Direct and Zend Framework Part 2

As promised in my previous post, I’ll introduce Ext.Direct and discuss how it fits with the new JSON-RPC standards and especially with the Zend_Json_Server component.

Ext.direct is a namespace and a bit of a buzzword in the ExtJS 3.0 release. In short, the Ext.direct stuff introduces high level communication features.

The key component of these new features is the “EXT.DIRECT REMOTING SPECIFICATION” which covers the aspects ‘remote procedure call‘ and ’service description’. Ext.Direct implements this specifications within the ExtJS framework as service consumer and for multiple server side stacks as service provider.

Besides having a well defined and documented communication and transport layer Ext.direct also has the advantage, that other ExtJs classes dealing with data can work on the stubs, created based on the ’service map descriptions’. The Ext.tree.TreeLoader can be configured with a directFn to fetch it’s data. More over the Ext.data.DirectStore can be configured with a complete set of direct functions for CRUD actions.

An other cool thing about the javascript implementation of the Ext.direct.RemotingProvider is, that is queues request of the direct stubs for a configurable time span. After this span, it sends one batched request to the server.

However, its important to note, that the “EXT.DIRECT REMOTING SPECIFICATION” is different form the JSON-RPC/JSON-SMD specifications introduced in part 1.

While digging in the specs I found some pros and cons for the “EXT.DIRECT REMOTING SPECIFICATION” compared to the JSON-RPC standardization:

  • + It covers form posts which are needed for special actions like file upload
  • + It covers events send by the server
  • - It lacks of function parameter description
  • - It does not support named function parameters
  • - It does not support optional parameters
  • - It is not compatible to the JSON-RPC standardization efforts

The last point is IMHO the strongest point why not to use this specification. While the rest of the javascript/webservice addicted world tries to find a common standard, ExtJs goes its own way. There are already a number of implementations for the JSON-RPC and JSON-SMD out there, and more and more will follow. I also expect to see service consumers written in PHP which ease writing server-server web-services using the same API.

For that reasons I choose to take the standard Zend_Json_Server implementation and to write a Ext.ux.direct.ZendFrameworkProvider we can use in Tine 2.0 and other Zend Framework based projects.

It’s only fair to note, the the Zend_Json_Server also has several issues which needs to be improved. Most notably:

  • - It does not support batched requests
  • - It does not support named parameters

After all this theory I’ll cover the actual implementation of the Ext.ux.direct.ZendFrameworkProvider in part 3.

27
Apr

First Impression of the new Tine 2.0 Calendar

Just a litte update for all of you, waiting for the new calendar:

nelius$ phpunit --verbose Calendar_AllTests AllTests.php
PHPUnit 3.2.19 by Sebastian Bergmann.

Tine 2.0 Calendar All Tests
Calendar_RruleTests
............

Calendar_Backend_SqlTests
.......

Calendar_Controller_EventTests
..................

Time: 24 seconds

 

OK (37 tests)

Most of hard work behind the scenes (aka backend) is finished. Implementing the recuring rules to behave correctly for participants in different timezones was quite a challenge, but I’m quite hopeful that with our implementation we won’t have the moving events after ‘Day light saving’ boundaries like we had in the other project.

This week I’ll start to implement the interfaces based on ExtJS. Anne already gave some great impact on the calendar organization schema. Please do support her survey coming up in the next days.

02
Mar

Progress on a Shiny Tine 2.0 Setup

As the Tine 2.0 community keeps growing, also the need for a setup GUI is growing. One the one hand we need to support the ‘non computer freak admins’ which are installing Tine 2.0 and on the other hand we need tools for the growing developers community to install/uninstall their own applications or to only particularly upgrade the system.

So here a little snapshot of current development progress, feel free to leaf you comments.

The installation requirements are not met. It's not possible to continue setup. Maybe we also need some kind of alert "Your server is not capable to run Tine 2.0. Sorry!"

The installation requirements are not met. It's not possible to continue setup. Maybe we also need some kind of alert "Your server is not capable to run Tine 2.0. Sorry!"

Setup checks are passed, but no config file is found. Here we need a little form to support creation of a config.inc.php file.

Setup checks are passed, but no config file is found. Here we need a little form to support creation of a config.inc.php file.

If a config file is present, you need to login into setup (credentials from config.inc.php)

If a config file is present, you need to login into setup (credentials from config.inc.php)

You have a config file and successfully logged in, but something is not working with your config, e.g. SQL or LDAP is down.

You have a config file and successfully logged in, but something is not working with your config, e.g. SQL or LDAP is down.

Application Manager

If all requirements are met, you are ready to run the application manager. This is the place to install / uninstall and upgrade your applications

 

Whats the scope of the setup?

In the setup all things needed to run Tine 2.0 itself should be configured. This is the main database connection on the one hand, and the authentication and account drivers on the other hand. Things like email, voip, langs etc could be configured directly from the admin section in Tine.

26
Aug

First Translations for Tine 2.0

After the german translations of Tine, which are provided by the core developers team, now the first contributed translations are integrated.

We are quite glad that we already have translators for russian, bulgarian, french, italian, polish and simplified chinese even before we started to ask actively for translations.

Specially I’d like to thank Ilya Yurkovetskiy, for the russian translation and the help to locate areas where the plural forms in the templates needed to be improved.

Unlike English and German, the Russian language has 3 plural forms. The first for e.g. 1, the second for e.g. 2 and the third for e.g. 5 items of a thing.

Also the cyrillic letters help to identify strings not in the translation system yet.

I set up a language statistics page here, to track the state of each translation. After Release of the next milestone we will start a call for translations.

14
Aug

Will DateTime abstraction come to Zend_Db?

In my view, the lack of proper date-time abstraction in the Zend_Db component is the most missing core feature in Zend Framework as any Database Brand expects date-time informations in a different representation.

However in Zend Framework date-time data are greatly abstracted via the Zend_Date component with full Time-zone support!

But when it comes to Zend_Db, this abstraction is of no use, as the component expects data for date-time fields in the db’s native representation.

There is a open issue in the Zend Tracker about this topic. The scheduled fix war for 0.8.0 release :-) .

Well, never give up Hope is the credo. The issue got updated 3 days ago, setting the target version again to next minor release. 

Lets all keep finger crossed, as the inclusion of this feature would greatly help to finish the Oracle Back-end of Tine 2.0.

16
Jun

To MVC or not to MVC?

Anybody speaks about the MVC pattern these days. When designing the Tine 2.0 application and api structure we also thought about implementing it with the standard MVC pattern.

While the clean 3-tier layout of the MVC pattern is crucial for business applications the MVC pattern brings some limitations we solved with small extensions to the pattern.

Single Page Application
Tine 2.0 is a so called ”’Single Page Application (SPA)”’. This means anything is served from a single URL, the index.php. If a login screen is shown or if a application does a complex transactional operation, it’s always handled by the index.php.

This layout simplifies the application flow dramatically, as there is no need to figure out the required controllers/actions by the url and there is no need to define rewrite rules. As a consequence we don’t use a traditional front controller and only have a light-way dispatcher in the index.php.

Service Orientated Architecture
Tine 2.0 has a service orientated architecture. The normal ”views” in MVC apps are perfectly sweetet for ”HTML output”. However the Tine 2.0 server returns almost no HTML, but is capable to serve the needed data in various representations. Moreover it also receives its actions and data also via various representations.

As a consequence Tine 2.0 applications normally don’t have view’s, but they have server classes, e.g a JSON class. This server classes handle all the input and output conversions from and to the the requested format, e.g. JSON.

Multi Back-end Capable
The Tine 2.0 application pattern is designed to support multiple back-ends. This means that the applications records store could be of totally different types. A simple example is the Address-book. It’s contacts can come from a SQL back-end or from a LDAP back-end.

In the standard MVC pattern the models do the persistence operations on a single persistence instance, which normally means they do all the SQL database handling. This does not fit the Tine 2.0 multi back-end requirement.

As a consequence the Tine 2.0 models are just simple data containers and have nothing to do with persistence. Controller store and get those simple models via the back-end classes which hide the complexity of the different physical back-ends by implementing a simple public interface.

Find out more about the Tine 2.0 application layout in our wiki.

10
May

Usability Of The Country Picker UI Element

While working on the country picker widget for Tine 2.0 I got remembered on a long discussion with a customer about the country picker in the traditional web application address book they where running.

The country selection user interface element there was a normal combo-box with 238 countries in it. If you want to select a country you have to search this alphabetical sorted list yourself. This is a quite annoying task for people working the whole day with the address book.

So they thought, it would be a tremendous amendment to make the country field a normal text field where the user just types in the country e.g. ‘Germany’. In normal operation most users only deal with only a few countries and do not need to have counties like ‘Saint Vincent and the Grenadines’.

This approach has two problems. First, If the country is a free text field, you will have typos in the data e.g. ‘Gremany’. You will miss such entries when you filter you addresses by the country ‘Germany’. Secondly, this only works if the company operates in one language, cause French employees would write ‘Allemagne’ in the field.

The worse situation is caused by limitations of an old fashioned web application. It could be solved nowadays by using up to date web techniques. Below is the current country selection we use in Tine 2.0. It’s a combination of a text field, a combo-box and a search field:

  • You could only select well defined values
  • You can perform a live search / type ahead
  • You can use it like a traditional combo-box
  • You can use the ENTER key to select, no mice input needed (please notice the fluent work flow: after pressing ENTER you can directly perform a new search)
  • You can use the COURSER keys to navigate in the results


BTW: After we noticed, that the old web address book (guess which) stores localised country names in the database, we switched the country selection to a text field as the data are messed up in any case.

03
May

Gleanings Of The PHPUnconference In Hamburg

Last weekend I attended the PHPUnconference in Hamburg.

The concept of an unconference was new to me. One the one hand I found the approach relaxing, the amount of free time between the sessions was really great to come in contact with the other attendes. One the other hand the selforganisation costed plenty of time and also brought some confusions. This seems to be the natural drawback of an unconference.

In Particular the talks about software-quality from Sebastian Bergmann and Manuel Pichler where pretty interesting to me. A complete list of sessions and notes of them could be found in the PHPUnconferences wiki.

Here are my impressions I got from the various conversations there:

  • The php web application world goes javascript for the user-interfaces. Although it was a PHP converence, you could talk with allmost anyone about the exciting world of javascript. Many people, who started with conservertive aproches and replaced server generated HTML on the client, are now in the process to switch to real javascript clients.
  • The php web application world goes to mission critical places. Where in the beginning of the php area it had the smack of a hobbyist sphere of activity, php applications do more and more mission critical computing in various business areas.
  • Php web application is not a synonym for open source code. In fact as a ambassador of a high quality mission critical open source web application, I felt like a bit of an exception there. The majority of attendees is into closed sourced ‘commercial only’ projects.
  • A lot of java coders switch to php. As php is not in the academic scope yet, many newcomers in the web-scene start coding in java. After their first contact with php they love the language due to it’s flexibility. Even in the open source world, one might find this tendency according to ohloh.

Finaly many thanks to the orga team for the great unconference and the extraordinary catering.

26
Mar

Back Online

After two weeks of nice holidays, I’m back online.

My father became 70 and we spend one week together as family to celebrate this. The other week we visited friends and enjoyed our free time.

Now, back in Office, I noticed that my colleges turned Tine 2.0 an other time inside out. I had to dig through more than 300 commits. Many cool new internal features and again simplifications in the backend. So I’ll have to spend a day to adopt the tasks application to the new standards.

Really cool is the new schemaproc we have now. We can manage tables and relations in a really advanced way. We havn’t found this features, needed for a complex application design, in the existing PHP frameworks and so we are pretty exited about the results. Maybe we can bring this stuff into the Zend Framework.

Also noteworthy are the beginnings of setup and migration. In short it should be possible to install Tine 2.0 without an existing egroupware installation. Also the migration of existing data from egroupware is on the way. Maybe I can manage to finish the infolog->tasks import this week. Really exiting with this migration approach is, that we are able to migrate from different databases/versions easily. So let’s se what future brings :-)

Also during my holidays, the usability efforts became a lot more concrete. I even found the first interface prototype in my Inbox. Next week I’ll start to implement the first results from this in the addressbook.

As a conclusion of all this progress I take that I realy should take more holidays, as Tine develops great in those times :-)