Archive for the 'Zend Framework' Category

21
Jan

New Features in Calendar for Tine 2.0 Neele (2011-01)

I just assembled a small video, describing the new feature which came in Tine 2.0Version Neele:

  • Printing
  • Color definition
  • Fast responses to invitations
  • Quick tip symbols
  • Out of view hints

09
Nov

How to Speed up the Slow Zend_Date Class

Is Zend_Date slow?

Well, it’s always easy to state that something is slow, so let me be a bit more precise: If you need to do a lot of complex date/time operations in you applications, Zend_Date might turn out to be the major bottleneck. If you just need localization for some date’s Zend_Date presumably won’t be your problem.

In the Calendar app of Tine 2.0 it turned out, that Zend_Date consumes 66% of the time of a Month view Request in a realistic szenario.

Just bashing Zend_Date would be to easy, as it has a remarkable feature set. Most notably in my view:
- solves the year 2038 problem on 32 bit systems
- full timezone support
- full localization support
- full support for ISO 8601 date string identifiers

As Date/Time is a really complex topic, here my full acknowledgement for this feature set at a time when PHP couldn’t help on all this.

On the other hand you don’t always need all the nice features it has, but you might want to use Zend_Date as generic date/time abstraction all over your applications. In this case Zend_Date is slow and has a lot of potential for improvements.

Patching Zend_Date

Zend_Date::__construct($date, $format, $locale)

Zend_Dates could be created from a date string in an arbitrary format and locale. This is great if you need to parse custom date strings.

When dealing with Zend_Date in backend operations, you normally only use very few date/time representation formats. e.g. ‘yyyy-MM-dd HH:mm:ss’ which is the mysql date/time format.

For this kind of input Zend_Date has no optimization. An easy performance patch for this problem could be:

if ($format === 'yyyy-MM-dd HH:mm:ss') {
    $matches = array();
    preg_match("/^(\d{4})-(\d{2})-(\d{2})[T ]{1}(\d{2}):(\d{2}):(\d{2})/", $_ISO, $matches);

    if (count($matches) == 7) {
        list($match, $year, $month, $day, $hour, $minute, $second) = $matches;
        // NOTE: PHP5 timestamp support is 32 bit and ends on 2038-01-19 03:14:07
        if ($year < 2038) {
           $date = mktime($hour, $minute, $second, $month, $day, $year);
           $format = Zend_Date::TIMESTAMP
       }
    }
}

Zend_Date::get($part, $locale)

When converting Zend_Dates to strings, the same as above applies. Here an easy patch could be:

if (! $part) {
    $part = self::TIMESTAMP;
}

if (array_key_exists($part, self::$_dateMap)) {
    $dt = new DateTime('@' . $this->getUnixTimestamp());
    $dt->setTimezone(new DateTimeZone($this->getTimezone()));

    $s = $dt->format(self::$_dateMap[$part]);
    switch($part) {
        case 'm':
            $s = (int) preg_replace('/^0/', '', $s);
            break;
        }
    return $s;
}

with this dateMap

private static $_dateMap = array(
    'yyyy-MM-dd HH:mm:ss' => 'Y-m-d H:i:s',
    'MM' => 'm',
    'm'  => 'i', // spechial handling required!
    'M'  => 'n',
    'd'  => 'j',
    'h'  => 'g',
    'H'  => 'G',
    'HH' => 'H',
    's'  => 's',
    'I'  => 'I',
    'z'  => 'T',
    'U'  => 'U',
    'eee' => 'N',
    'D'  => 'z',
    'e'  => 'w',
    'X'  => 'Z'
);

Zend_Date_DateObject::setTimezone($zone)

As ZF requires PHP 5.2.4 and above, we can use native PHP functions to improve setTimezone:

public function setTimezone($zone = null)
{
    try {
        $dtz = new DateTimeZone($zone);
        $this->_offset   = $dtz->getOffset(new DateTime('1970-02-01 00:00:00'));
        $this->_timezone = $zone;
    } catch (Exception $e) {
        require_once 'Zend/Date/Exception.php';
        throw new Zend_Date_Exception("timezone ($zone) is not a known timezone", $zone);
    }
    if (($zone == 'UTC') or ($zone == 'GMT')) {
        $this->_dst = false;
    } else {
        $this->_dst = true;
    }
    return $this;
}

This are just 3 very easy patches, but for a Tine 2.0 Calendar month view request it saves 41% of the request time!

Avoiding Zend_Date

Derick Rethans contributed a great DateTime class to PHP which is in included beginning with version 5.2 which got even more improved in version 5.3. This class solves the year 2038 problem, has full timezone support.

So my first idea was to write a new Zend_Date class which requires PHP version >= 5.3 to be used as a drop in replacement. Unfortunately it turned out, that the design of Zend_Date is not compatible with the DateTime, or to be more precise, the new class would also be slow due to two base restrictions in Zend_Date:
- ISO representation for date string identifiers
- Zend_Date also represents date intervals which have a separate class in PHP.

At the end I implemented on own wrapper class around DateTime which supports some old Zend_Date signatures, so that we don’t had to edit all our code. Zend_Date is only used when Server side locale handling is needed, like in exports.

Switching from Zend_Date to native DateTime brought us a speed up of 66% in a calendar month view request.

18
Sep

Data Transport with JSON-RPC via Ext.Direct and Zend Framework Part 3

Now, with the results from Part 1 and Part 2, let’s take a deeper look into the Ext.Direct Providers to see how they work and what to do.

Prepareing a JSON Service Map Description

To use the Ext.direct framework we need to register a provider. For a remoting provider we need a service mapping description.

Ext.Direct.addProvider(Ext.apply(Ext.app.JSONRPC_API, {

    'type'     : 'zfprovider',
    'url'      : Ext.app.JSONRPC_API.target
}));

Ext.app.JSONRPC_API holds the service mapping description which in the Tine 2.0 case we deliver it with our registry data, but you can also fetch it e.g. by a javascript include as suggested in the “EXT.DIRECT REMOTING SPECIFICATION”.

<script src=”index.php?method=Tinebase.getServiceMap” type=”text/javascript”></script>

The Service Map could be generated like this:

$server = new Zend_Json_Server();

// set all classes you want to expose
$server->setClass('MyModule_Service_Json', 'MyModul');

$server->setTarget('index.php')
    ->setEnvelope(Zend_Json_Server_Smd::ENV_JSONRPC_2);

$smdArray = $server->getServiceMap()->toArray();
// save some bytes
unset($smdArray['methods']);

echo "Ext.app.JSONRPC_API = " . Zend_Json::encode($smdArray);

Ext.ux.direct.ZendFrameworkProvider

To Enable Ext.Direct to understand the JSON-SMD data and also communicate using the JSON-RPC protocol we create an own provider:

Ext.ux.direct.ZendFrameworkProvider = Ext.extend(Ext.direct.RemotingProvider, {

...

At first we need to parse the JSON-SMD and to create the stubs. This is done by overwriting the initAPI method:

// private

    initAPI : function() {
        for (var method in this.services){
            var mparts = method.split('.');
            var cls = this.namespace[mparts[0]] || (this.namespace[mparts[0]] = {});
            cls[mparts[1]] = this.createMethod(mparts[0], Ext.apply(this.services[method], {
                name: mparts[1],
                len: this.services[method].parameters.length
            }));
        }
    },


All we do is to transform the JSON-SMD definition data in to a from the original createMethod method can understand to create the stubs.

It’s important to understand, that the createMethod creates stubs which do only trigger further processing of the provider and do _NOT_ contain all the code to do requests and protocol processing.

Once a stub is called, it applies the doCall function of our provider. In order to support named parameters we inspect the call and transform a named parameter call into a positional parameter call. This is needed, cause the ZendFrameworks Zend_Json_Server is itself not able to understand named parameter calls witch by the way are indeed part of the JSON-RPC specification.

// private

    doCall : function(c, m, args) {
        // support named parameters
        if (args[args.length-1].paramsAsHash) {
            var o = args.shift();
            for (var i = 0; i < m.parameters.length; i++) {
                args.splice(i,0, o[m.parameters[i].name]);
            }
        }

        return Ext.ux.direct.ZendFrameworkProvider.superclass.doCall.call(this, c, m, args);
    },

After the parameters a prepared, the provider assembles the request according to the protocol definitions. This is done in the getCallData method.

// private

    getCallData: function(t){
        return {
            jsonrpc: '2.0',
            method: t.action + '.' + t.method,
            params: t.data,
            id: t.tid
        };
    },


This protocol encapsulation only deals with single transaction calls. The processing for batched calls is done elsewhere. Luckily The both protocols use the same simple outer array for batched calls, so that we don’t need to touch it.

Finally the response needs to be decoded according to the protocol and the callback function needs to get called using the decoded data. This is done in the onData method:

// private

    onData: function(opt, success, xhr) {
        var rpcresponse = Ext.decode(xhr.responseText);
        xhr.responseText = {
            type: rpcresponse.result ? 'rpc' : 'exception',
            result: rpcresponse.result,
            tid: rpcresponse.id
        };

        return Ext.ux.direct.ZendFrameworkProvider.superclass.onData.apply(this, arguments);
    }

And voila, thats it. This is our custom provider to enable Ext.Direct to use and talk pure standard JSON-PRC/JSON-SMD.

Finally let’s register the provider for lazy invocation

Ext.Direct.PROVIDERS['zfprovider'] = Ext.ux.direct.ZendFrameworkProvider;

Server Side Batched Requests

As noted in Part 2, the biggest problem with the Zend_Json_Server is the fact that it’s not capable to handle batched requests yet. To overcome this in a simple way, you can monitor the first character of an JSON request in your dispatcher and dispatch multiple request if it’s a ‘[‘. Of course this is far away from optimum, but it’s a start point till we have the feature in the Zend_Json_Server.

04
Sep

Data Transport with JSON-RPC via Ext.Direct and Zend Framework Part 2

As promised in my previous post, I’ll introduce Ext.Direct and discuss how it fits with the new JSON-RPC standards and especially with the Zend_Json_Server component.

Ext.direct is a namespace and a bit of a buzzword in the ExtJS 3.0 release. In short, the Ext.direct stuff introduces high level communication features.

The key component of these new features is the “EXT.DIRECT REMOTING SPECIFICATION” which covers the aspects ‘remote procedure call‘ and ‘service description’. Ext.Direct implements this specifications within the ExtJS framework as service consumer and for multiple server side stacks as service provider.

Besides having a well defined and documented communication and transport layer Ext.direct also has the advantage, that other ExtJs classes dealing with data can work on the stubs, created based on the ‘service map descriptions’. The Ext.tree.TreeLoader can be configured with a directFn to fetch it’s data. More over the Ext.data.DirectStore can be configured with a complete set of direct functions for CRUD actions.

An other cool thing about the javascript implementation of the Ext.direct.RemotingProvider is, that is queues request of the direct stubs for a configurable time span. After this span, it sends one batched request to the server.

However, its important to note, that the “EXT.DIRECT REMOTING SPECIFICATION” is different form the JSON-RPC/JSON-SMD specifications introduced in part 1.

While digging in the specs I found some pros and cons for the “EXT.DIRECT REMOTING SPECIFICATION” compared to the JSON-RPC standardization:

  • + It covers form posts which are needed for special actions like file upload
  • + It covers events send by the server
  • - It lacks of function parameter description
  • - It does not support named function parameters
  • - It does not support optional parameters
  • - It is not compatible to the JSON-RPC standardization efforts

The last point is IMHO the strongest point why not to use this specification. While the rest of the javascript/webservice addicted world tries to find a common standard, ExtJs goes its own way. There are already a number of implementations for the JSON-RPC and JSON-SMD out there, and more and more will follow. I also expect to see service consumers written in PHP which ease writing server-server web-services using the same API.

For that reasons I choose to take the standard Zend_Json_Server implementation and to write a Ext.ux.direct.ZendFrameworkProvider we can use in Tine 2.0 and other Zend Framework based projects.

It’s only fair to note, the the Zend_Json_Server also has several issues which needs to be improved. Most notably:

  • - It does not support batched requests
  • - It does not support named parameters

After all this theory I’ll cover the actual implementation of the Ext.ux.direct.ZendFrameworkProvider in part 3.

27
Apr

First Impression of the new Tine 2.0 Calendar

Just a litte update for all of you, waiting for the new calendar:

nelius$ phpunit --verbose Calendar_AllTests AllTests.php
PHPUnit 3.2.19 by Sebastian Bergmann.

Tine 2.0 Calendar All Tests
Calendar_RruleTests
............

Calendar_Backend_SqlTests
.......

Calendar_Controller_EventTests
..................

Time: 24 seconds

 

OK (37 tests)

Most of hard work behind the scenes (aka backend) is finished. Implementing the recuring rules to behave correctly for participants in different timezones was quite a challenge, but I’m quite hopeful that with our implementation we won’t have the moving events after ‘Day light saving’ boundaries like we had in the other project.

This week I’ll start to implement the interfaces based on ExtJS. Anne already gave some great impact on the calendar organization schema. Please do support her survey coming up in the next days.

02
Mar

Progress on a Shiny Tine 2.0 Setup

As the Tine 2.0 community keeps growing, also the need for a setup GUI is growing. One the one hand we need to support the ‘non computer freak admins’ which are installing Tine 2.0 and on the other hand we need tools for the growing developers community to install/uninstall their own applications or to only particularly upgrade the system.

So here a little snapshot of current development progress, feel free to leaf you comments.

The installation requirements are not met. It's not possible to continue setup. Maybe we also need some kind of alert "Your server is not capable to run Tine 2.0. Sorry!"

The installation requirements are not met. It's not possible to continue setup. Maybe we also need some kind of alert "Your server is not capable to run Tine 2.0. Sorry!"

Setup checks are passed, but no config file is found. Here we need a little form to support creation of a config.inc.php file.

Setup checks are passed, but no config file is found. Here we need a little form to support creation of a config.inc.php file.

If a config file is present, you need to login into setup (credentials from config.inc.php)

If a config file is present, you need to login into setup (credentials from config.inc.php)

You have a config file and successfully logged in, but something is not working with your config, e.g. SQL or LDAP is down.

You have a config file and successfully logged in, but something is not working with your config, e.g. SQL or LDAP is down.

Application Manager

If all requirements are met, you are ready to run the application manager. This is the place to install / uninstall and upgrade your applications

 

Whats the scope of the setup?

In the setup all things needed to run Tine 2.0 itself should be configured. This is the main database connection on the one hand, and the authentication and account drivers on the other hand. Things like email, voip, langs etc could be configured directly from the admin section in Tine.

26
Aug

First Translations for Tine 2.0

After the german translations of Tine, which are provided by the core developers team, now the first contributed translations are integrated.

We are quite glad that we already have translators for russian, bulgarian, french, italian, polish and simplified chinese even before we started to ask actively for translations.

Specially I’d like to thank Ilya Yurkovetskiy, for the russian translation and the help to locate areas where the plural forms in the templates needed to be improved.

Unlike English and German, the Russian language has 3 plural forms. The first for e.g. 1, the second for e.g. 2 and the third for e.g. 5 items of a thing.

Also the cyrillic letters help to identify strings not in the translation system yet.

I set up a language statistics page here, to track the state of each translation. After Release of the next milestone we will start a call for translations.

14
Aug

Will DateTime abstraction come to Zend_Db?

In my view, the lack of proper date-time abstraction in the Zend_Db component is the most missing core feature in Zend Framework as any Database Brand expects date-time informations in a different representation.

However in Zend Framework date-time data are greatly abstracted via the Zend_Date component with full Time-zone support!

But when it comes to Zend_Db, this abstraction is of no use, as the component expects data for date-time fields in the db’s native representation.

There is a open issue in the Zend Tracker about this topic. The scheduled fix war for 0.8.0 release :-) .

Well, never give up Hope is the credo. The issue got updated 3 days ago, setting the target version again to next minor release. 

Lets all keep finger crossed, as the inclusion of this feature would greatly help to finish the Oracle Back-end of Tine 2.0.