A look at Qt's Declarative UI and CSS3 Animations

By Kenneth Rohde Christiansen, INdT OpenBossa Labs

Disclaimer:

The following article takes a look at two technologies which are both work in progress. Comparing such technologies can be difficult due to the lack on documentation as well as daily changes.

The CSS3 support is pretty much specified and as the standard it derives from WebKit, the support in WebKit could be considered 100%.

QML on the other hand, is in rapid development and most documentation is found in the source code itself.

Both lack examples and tutorials.

Declarative UI

Developing software for embedded devices such as mobile phones is quite hard, as consumers demand more and more features, which have to be integrated and fit onto a relatively small screen.

Due to this a lot of research has been going on, on how to improve the user experience by adding more customized interfaces than with common toolkits, that better use the screen realestate available. The iPhone was one of the first phones that showed, that with a bit more processing power and a touch screen, you can actually make a truly amazing interface.

At INdT we have spend the last three years working on various interfaces that fit into this above description, and we found a huge performance boost when we started using a declarative description language for building our user interfaces. Let's call this Declarative UI (DUI, for short)

So, what is a declarative UI?

Declarative UI is a way of making this kind of user interfaces by describing them in terms of simple elements; text, image, rectangle, and well as other objects. These are then build into components.

The reason why it is called declarative is that the UI is expressed as sets of property expressions, like, the x value of RectA is equal to the half of the width of RectB. These properties can then also be grouped into states, so for instance when a button is in Selected state the button image, which is a property as well, might be different.

This is different to imperative programming where you would normally give orders like, set this, set that, then do this, etc...

Declarative UI is very "designer like", as it makes it possible for you to do things like, add a rectangle that fits the whole screen, add a rectangle as a child inside the other one, add another rectangle to the right of the first child (with it x value being equal to the x value + the width of the former one)

A great declarative language would be rather simple, but still powerful enough to cover most use cases.

The WebKit Stack

During the last, at least 5 years, a lot of research has gone into developing web applications. Not just websites, but apps that act like real applications.

Google was one of the first companies bringing this to the mass market, by developing GMail as well as Google Maps.

Since then developing such apps has become easier, with the introduction of JavaScript libraries such as jQuery, as well as faster JavaScript implementations.

Things are looking really good actually as a lot of work has gone into developing a new HTML standard, mostly based on work by the WHATWG which is backed by Apple, Opera, Mozilla and more. This has let into the HTML5 standard which is work in progress.

As part of Apple's Dashboard support for Mac OS X, Apple developed a canvas extension to WebKit, which showed a lot of promise. Flash has long been hated by many users and web developers as it doesn't integrate with the web, breaks tab focusing, and is proprietary. It has been obvious that Apple is also not a bit fan of Flash, and they even opted to ship the iPhone with no Flash support whatever.

On the other hand, their developers looked at what was needed to provide the web developers with some of the features found in Flash, and came up with four CSS extensions, CSS3 Transitions, CSS3 Animations, CSS3 2D and 3D transformations, which are now all part of the CSS3 draft and supported in WebKit, using the -webkit- prefix.

Doing declarative UI with CSS3 only is not possible, you'll have to combine it with other web technologies such as at HTML. Actually HTML + CSS3 allows you to do many things, but combining the former with JavaScript makes the abbilities almost limitless, without that meaning that it is feasable without lots of coding.

The Qt Declarative UI

In the upcoming version 4.6 of Qt, the toolkit will bring declarativity to Qt.

The language that enables this is named QML and is a simple but yet powerful language.

Most of a user interface is described by a simple tree structure of property bindings. The power is that those property values can be any JavaScript expression - and because it’s a binding rather than just an assignment, the expressions are re-evaluated whenever their dependencies change. Very similar to web technologies.

Comparing the Two

Generally speaking, QML is made for making declarative user interfaces, where as CSS3 provides a very nice transition and animation framework to the already well proved web technologies.

Using the web for making actual mobile web applications hasn't been done much in the past, but developers showed that you could make applications for the iPhone that were quite native looking. These applications were still very simple in nature, but with the announcement that Palm would be bringing a phone to the market only based on web apps caught many peoples attention.

In this section we will look at the basic concepts of CSS3 and QML to see where they are similar in nature and where they differ. We will then conclude whether you would be able to accomplish almost when same using CSS3 as with QML, investing the same amount of efforts, or whether that is just not the case.

As QML is still in early development, we will also see if there are some ideas in CSS3 that could be applied to QML to make it better.

Concepts

The basic building block in QML and the WebKit stack, is the element; an HTML element in WebKit and a FxItem in Qt.

The most important block is the rectangle, the image as well as text.

In QML these are the Rect, the Image and the Text items, where as in WebKit these would be a div, an img (or div with image applied) as well as a text block, probably wrapped in a div.

Image

QML supports bordered images, which means that you can define a border area of an image that should be scaled differently when the image is scaled. You can think of it as the image split into 9 parts:


  TOPLEFT     |     TOP     |  TOPRIGHT    
-------------------------------------------
  LEFT        |   MIDDLE    |  RIGHT       
-------------------------------------------
  BOTTOMLEFT  |   BOTTOM    |  BOTTOMRIGHT 

Top left, top right, bottom left and bottom right are never scaled, where as top for instance would only be scaled horizontally, and left only vertically.

This leaves borders intact, which is really useful if you for instance used an image for painting a bottom and you want it to always have the same border.

In QML this is accomplished by using the scaleGrid properties of the Image


Image {
    id: "borderedImage"
    width: 100
    height: 160
    source: "label_keyboard.png"
    scaleGrid.left: 15 
    scaleGrid.right: 17
    scaleGrid.top: 15 
    scaleGrid.bottom: 17
}

WebKit supports the same by using a div with the CSS property border-image:


<div style=
 "width: 68px;
  height: 128px;
  border-width: 15 17 15 17;
  -webkit-border-image: url(label_keyboard.png) 15 17 15 17 stretch stretch;">
</div>

Notice that in the above example the border goes beyond the image width, thus width: 68 + 15 + 17 makes up the width of 100 in the QML example.

Layout Support

Layout support in QML and WebKit is a bit different. QML is very toolkit like and provides ways of boxing the components, and in the near future it will even support anchorlayout, where you can anchor the sides of elements to others, and make the resizing respect their min, max and preferred size values. A very powerful construct is being developed here at INdT in cooperation with Qt Software.

WebKit on the other hand is a web contents rendering engine, and as such supports the layout system of HTML and CSS. This is a very powerful system, but there is one conceptual difference, in that it is based on the layout of variable sized contents, mostly text. If the text grows, everything relayouts.

Normally when developing an interface with QML you want it the other way around: show text in this box, and if it doesn't fit, crop it, adding '...'.

CSS3 provides some of the former by introducing the CSS flex box, though we haven't confirmed whether this is yet supported by WebKit.

In WebKit and QML, by default children are relative to their parents, but in WebKit you can specify absolute position, where as in QML you can use the binding system and make the children relative to almost whatever you want.

Animation Support

Visual clues and animations can make a user interface a lot more welcoming, and after the launch of the iPhone, it is obvious that this is what the users want.

Both CSS3 as well as QML support animations, and we will try show how to accomplish various kind of animations with the two.

Fixed Animations

Fixed animations are animations that run some fixed animation, once or repeatingly.

Such animations are easy to do with QML, so let's look at an example and go on from there. Then later see how to accomplish the same with CSS3.

First of all, let's animate a rectangle from left to right, repeatingly:


Rect {
    width: 400
    height: 200
    color: "white"
    Rect {
        width: 40
        height: 40
        y: 80
        color: "red"
        x: SequentialAnimation {
            running: true
            repeat: true
            NumericAnimation {
                from: 0
                to: 360
                easing: "easeInOut"
                duration: 500
            }
        }
    }
}

The code is quite simple. First we create a rectangle of size (400, 200) and put a red rectangle inside it, which autoloads (running: true) and loops forever (repeat: true). This rectangle animates from 0 to 360 pixel along the x-axis, using an easy-in-out curve.

If not setting repeat and running, the above could be activated by using a trigger, leading us to talk about trigger actions.

CSS3 supports transitions and animations and those two are different as transitions are mostly used for trigger actions and animations are used for fixed animations.

Transitions in CSS3 are behaviour dependent. You do not define the actual animation, but a transition/animation that is applied when the values are changed! This is very web like, as in the web you will most likely start animations when the mouse is above a div, or a div received a click.

As CSS3 transition are the most simple, we will start by having a quick look at those. So let's see how to make a trigger action of the above, triggered on mouse over.


<html>
  <head></head>
  <body>
     <div style="background-color: red; -webkit-border-radius: 0.7em;
                 width: 40; height: 40;
                 position: absolute; top: 80px; left: 0px;
                 -webkit-transition-property: left;
                 -webkit-transition-duration: 0.5s;"
          onmouseover="this.style.left = 360"
          onmouseout="this.style.left = 0">
     </div>
  </body>
</html>

It is clear in the above that this is behavior dependent. It is simple an animation that is applied to a transtition when the style values are changed.

Something very powerful about this is that you can apply the same transition to any property of the element. Thus, we could have written:


-webkit-transition-property: left, color;

and if we did onmouseover="this.style.left = 360; this.style.color = green", the same transition would be applied to both properties.

So, let's see how we accomplish the same as in the QML example, using CSS3. In order to do this, we would need to define an animation. An animation is defined using keyframes:


@-webkit-keyframes animation-name {
   from { left: 0px; }
   to   { left: 360px; }
}

and as you can see it is not a behaviour dependent animation, as we will have to specify the properties we want to change. You can also specify the transition function by adding -webkit-animation-timing-function: to one of the keyframes (from: to: or any percentage)


@-webkit-keyframes animation-name {
   0%   { left: 0px; -webkit-animation-timing-function: ease-out; }
   100% { left: 360px; -webkit-animation-timing-function: ease-in; }
}

It is here important to notice that QML supports like 40 different kinds of animation timing functions where as CSS3 only supports: ease, linear, ease-in, ease-out, ease-in-out and cubic-bezier. The latter is very powerful though, as it makes you able to define a curve; powerful, but not easy to use.

The code that matches the behaviour of the QML example amounts to:


<html>
  <head>
     <style>
@-webkit-keyframes left-animation {
   from { left: 0px; }
   to   { left: 360px; }
}
     </style>
   </head>
   <body>
     <div style="background-color: red; -webkit-border-radius: 0.7em;
                 width: 40; height: 40;
                 position: absolute; top: 80px; left: 0px;
                 -webkit-animation-name: left-animation;
                 -webkit-animation-duration: 0.5s;
                 -webkit-animation-iteration-count: infinite;">
      </div>
    </body>
</html>

With the key frame and the transition support in CSS3, it is possible to do parallel animations. With transitions, we would just apply the transition to move properties and change them in parallel, like what we showed earlier. With animations we would just apply the animation to more properties:


@-webkit-keyframes left-animation {
   from { left: 0px; background-color: red; }
   to   { left: 360px; background-color: blue; }
}

QML supports parallel and sequential animations in conjunction, thus you can add parallel animations inside a sequential animation. We do not see any way to do this with CSS3, but CSS3 does support sequential animations by using the keyframes support.

Let's improve our first example by changing the color as well as adding a sequential animation:


x: SequentialAnimation {
    running: true
    repeat: true
    NumericAnimation {
        from: 0
        to: 360
        easing: "easeInOutQuad"
        duration: 1000
    }
    PauseAnimation {
        duration: 1000
    }
    NumericAnimation {
        from: 360
        to: 0
        easing: "easeInOutElastic"
        duration: 2000
    }
}
color: SequentialAnimation {
    running: true
    repeat: true
    ColorAnimation {
        from: "#FF0000"
        to: "#00FF00"
        duration: 2000
    }
    ColorAnimation {
        from: "#00FF00"
        to: "#FF0000"
        duration: 2000
    }
}

As seen, we now have an animation that takes 4 seconds in total and changes color and the x value (left in CSS).

Doing the same with CSS can be accomplished by using the keyframes:


@-webkit-keyframes left-animation {
    0%   { left: 0px; color: red;
           -webkit-animation-timing-function: ease-in-out; 
         }
    25%  { left: 360px; }
    50%  { left: 360px; }
    100% { left: 0px; color: green;
           -webkit-animation-timing-function: ease-in-out;
         }
}

One difference here is that the keyframes work almost as some kind of animation timing function, and can be reused for different durations, where as in QML the time is fixed.

Lets set it to 4 seconds as well:


-webkit-animation-name: left-animation;
-webkit-animation-duration: 4s;

One thing that we see here is that the CSS3 animation is somewhat simpler. We dont define the animation type at all, just as with transitions. It will find out how to interpolate and anime color and left. You can even try to anime something that doesn't support animation and it will just change the value without animation. This is not possible with QML as far as we can tell.

It would be very nice if I could do something similar with QML, but it would need some more thoughts.


KeyFrameAnimation {
    running: true
    repeat: true
    duration: 4000

    Frame(0) {
        x: 0
        color: red
        easing: "easeInOutQuad"
    }
    Frame(25) {
        x: 360
    }
    Frame(50) {
        x: 360 // x will stay at the current position
    }
    Frame(100) {
        x: 0 // back to start position
        color: green
        easing: "easeInOutElastic"
    }
}

Also, we do not like that you need to specify the kind of animation, NumericAnimation, ColorAnimation etc, and would like some dynamic support here as in CSS3, where it will just anime what it supports.

Doing color: ColorAnimation { ... } we are already operating on color, so no need to tell it that it is a color animation. The cool thing with this is that if supported behavior/transition animations like CSS3 (it does and we will come back to that) we could reuse the same animation for color and x, as we are the ones changing the values.


x, color: Animation { 
              duration: 2000
              easing: "easeInOutElastic"
          }

instead of doing:


x:     NumericAnimation {
           duration: 2000;
           easing: "easeInOutElastic"
       }
color: ColorAnimation {
           duration: 2000;
           easing: "easeInOutElastic"
       }

QML supports using different animations for color and x at the same time, with different durations (notice the changed durations!):


x: SequentialAnimation {
    running: true
    repeat: true
    NumericAnimation {
        from: 0
        to: 360
        easing: "easeInOutQuad"
        duration: 500
    }
    PauseAnimation {
        duration: 1000
    }
    NumericAnimation {
        from: 360
        to: 0
        easing: "easeInOutElastic"
        duration: 200
    }
}
color: SequentialAnimation {
    running: true
    repeat: true
    ColorAnimation {
       from: "#FF0000"
       to: "#00FF00"
       duration: 5000
    }
    ColorAnimation {
       from: "#00FF00"
       to: "#FF0000"
       duration: 5000
    }
}

We wondered how to do this with CSS3 or whether it is possible, and it indeed is. All you need to do it to define two keyframes animations and apply them with varying duration.


@-webkit-keyframes left-animation {
    0%   { left: 0px; -webkit-animation-timing-function: ease-in-out; }
    18%  { left: 360px; }
    25%  { left: 360px; }
    100% { left: 0px; -webkit-animation-timing-function: ease-in-out; }
}

@-webkit-keyframes color-animation {
    from { background-color: red; }
    50%  { background-color: green; }
    to   { background-color: red;}
}

and on the element:


-webkit-animation-name: left-animation, color-animation;
-webkit-animation-duration: 2.7s, 10s;
-webkit-animation-iteration-count: infinite, infinite;

Triggered Animations

It it our experience that when using touch devices, the click position isn't that precise because of big and/or shaking fingers, and it is normal to define invisible areas as even proxies.

For instance you have a button, but you actually want to make the area that captures button clicks larger than the actual button. In CSS3 we do not see any obvious way of acomplishing this, but with QML you can define event regions as shown below, using MouseRegion:


Rect {
    y: 100
    width: 50
    height: 50
    color: "blue"
    MouseRegion {
        anchors.fill: parent
    }
}

A MouseRegion supports function properties such as on Pressed, onReleased, etc:


onPressed: {
    print('press (x: ' + mouse.x + ' y: ' + mouse.y + ')')
    print(mouseButton)
    if(mouseButton=='Right') {
        parent.color = 'blue'
    } else {
        parent.color    = "red"
    }
}
onReleased: { 
    print('release (x: ' + mouse.x + ' y: ' + mouse.y 
          + ' isClick: ' + mouse.isClick + ' wasHeld: ' + mouse.wasHeld + ')')
}
onClicked: {
    print('click (x: ' + mouse.x + ' y: ' + mouse.y
          + ' wasHeld: ' + mouse.wasHeld + ')')
}
onDoubleClicked: {
    print('double click (x: ' + mouse.x + ' y: ' + mouse.y + ')')
}
onPressAndHold: {
    print('press and hold')
}
onExitedWhilePressed: {
    print('exiting while pressed')
}
onReenteredWhilePressed: {
    print('reentering while pressed')
}

We find this support really useful and important.

Behaviour Dependent Animations

Now that we have already touched CSS3 transition and mentioned that QML has something similar, it is time to have a look. Remember our earlier example:


-webkit-transition-property: left;
-webkit-transition-duration: 0.5s;"

onmouseover="this.style.left = 360px";

We set a transition with duration 500 ms for when ever the property left is changed.

you can do this with QML by using Behaviour:


x: Behaviour {
    NumericAnimation {
        property: "x"
        duration: 500
    }
}

MouseRegion {
    anchors.fill: parent
    onClicked: { x = mouseX; }
}

Recall that we proposed the below a bit earlier


x, color: Animation { 
              duration: 2000
              easing: "easeInOutElastic"
          }

Modifying this to the behaviour syntax would result in something along the lines of:


x, color: Behavior {
    Animation {
        duration: 500
    }
}

MouseRegion {
    anchors.fill: parent
    onClicked: {
        x = mouseX;
        if (color != blue) {
           color = blue;
        } else {
           color = red;
        }
    }
}

and it would just anime automatically, without having to do special declarations for x and color, using NumericAnimation and ColorAnimation.

Dragables and Slides

When designing user interfaces, slides, song trackers, volume bars etc are quite useful. Being able to make a rectangle dragable is very useful for this reason.

CSS supports dragable div elements, but we couldnt find an easy way to restrict the drag to a specific area, or fixed its draggability to just one axis, which is very useful when designing sliders. We would say this is one of the areas where CSS3 really fails for designing declarative user interfaces.

QML has good support for this, again using the MouseRegion.


Rect { 
    id: Slider
    x: 10
    y: 10
    width: 30
    height: 12
    color: "lightgray"
    radius: 6
    MouseRegion {
        anchors.fill: parent
        drag.target: parent
        drag.axis: "x"
        drag.xmin: 10
        drag.xmax: page.width - 120
    }   
}

Another place where CSS fails is that it doesn't have the bindability of QML. Yes, sure you can apply JavaScript magic to accomplish the same, but it is not easy and it will make the code less readable.

With the above QML code we would make an image slide along with the slider, thus if I move the slider, the image resizes! Cool stuff.


Image {
    id: "image"
    x: 10
    y:50
    width: 100 + Slider.x // NOTICE THE SLIDER HERE!
    height: 160
    source: "box.png"
}

States

One of the most useful and most powerful things in QML is the support of states and transitions. We will not go into details about this as it is not directly supported by CSS3 and would have to be coded with JavaScript.

Conclusion

Having looked at CSS3 support in WebKit and the upcoming Qt Declarative UI format, it is clear that while both add support for animations, Qt's QML is better for developing real application user interfaces, where as CSS3's support currently seem to just add animation support to html based content; just as animation triggers on mouse over, clicks etc, and some fixed animations that could be used to replace Flash ads. It is a very valuable addition, though.

QML on the other hand, has really amazed us. It is very well thought out, and the binding and javascript support (each property is a javascript expression) has really amazed us.

We were able to implement a song tracker that would modify the label showing the remaining time, in a few lines of code. The below code shows the code needed to show the remaining time.


Text {
    id: TotalTime
    x: parent.width + 20 // BINDING!
    font.size: 12
    font.bold: true
    color: "white"
    text: "-" + Math.floor((354 - Knob.x + 13) / 60) + ":"
              + Math.floor((354 - Knob    .x + 13) % 60) // JAVASCRIPT!
}

QML also has some support for loading it on demand over HTTP, much like Flash, and it seems that it has the potential for becoming a Flash killer with time.

A web browser plugin could easily be developed and Nokia could support it directly on all its S60 phones, when the Qt S60 port is ready. This way, third parties could develop applications that you run remotely and where you always would run the latest release!