Chapter 18. Media Resources

Table of Contents

1. Overview
2. Supported media types
2.1. Media for SWF SOLO applications
2.2. DHTML Limitations and workaround
2.3. Summary of Supported Media
3. Views and resources
3.1. Resources are not views
3.2. Summary of APIs that act on resources
3.3. Resource declarations and naming
4. Multi-frame resources
4.1. Frame rates
5. Resource loading
5.1. Compile time resources
5.2. Run-time resources
6. Using the <image> tag

This chapter describes how to include external media resources (images, audio, animation, and video) into OpenLaszlo applications.

1. Overview

LZX is designed to make it easy for you to integrate media resources into your application in a flexible and dynamic manner. You can include images, audio, video, and animations that were created using standard third-party content creation tools. This chapter describes the media formats that are currently supported as resources to views in OpenLaszlo 4.0. It then goes on to describes how to integrate and control media in your application. It concludes with some optimization tips.

Note that in addition to media that can be attached as resources to views, streaming audio and video can be played by videoviews. See Chapter 42, Audio and Video for a discussion of streaming media.

2. Supported media types

The list of supported media types varies slightly depending on the target runtime and whether the application is to be deployed SOLO or proxied.

2.1. Media for SWF SOLO applications

[Warning]
[SWF]

SWF only: The features described in this section only work in applications compiled to SWF. They do not work in applications compiled to other runtimes.

SOLO applications compiled to SWF depend on Flash to load images (whereas the proxied applications get help from the transcoder on the server). This means that SOLO applications are limited to file types supported natively by Flash.

2.1.1. Flash 7 and Flash 8 limitations

Flash player version 7 can only load a limited set of jpegs encodings; it cannot do interleaved or progressive jpeg. Flash player 8 can do a wider range of jpeg encoding. Set the compiler to compile for Flash 8 as a target runtime to get this expanded range of image formats (also PNG and GIF).

2.2. DHTML Limitations and workaround

[Warning]
[DHTML]

DHTML only: The features described in this section only work in applications compiled to DHTML. They do not work in applications compiled to other runtimes.

Browsers do not natively support .swf or audio or video formats. You must transcode them, omit them from your application, or embed a flash player in your application to handle them.

2.2.1. Transcoding .swf to .png

Files in .swf format can be transcoded to .png format for display by the DHTML runtime. OpenLaszlo provides a facility, called autopng, for doing this automatically. As a convenience, the server and compiler search for a corresponding png file if a swf file is referenced in the DHTML runtime. To use this mechanism in your own code, create a png file with the same basename as the swf file, and place it in the same directory as the swf resource you want it to replace.

2.2.2. Audio and Video in DHTML applications

[Note] Note

The following discussion applies to audio and video resources attached to views, not to videoviews. videoviews are not supported at all for the DHTML runtime.

In order to display swf-formatted audio and video within a DHTML application, you may embedd a swf player in the page. For a discussion of how to do this, see Chapter 35, Browser Integration.

2.3. Summary of Supported Media

OpenLaszlo 4.0 and above support the following media types:

JPEG

Depending on target runtime and mode of deployment, different .jpg and .jpeg files are supported. Progressive and interleaved files are not supported for Flash 7 SOLO deployment. See the W3C JPEG JFIF Specification.

[Note] Note

Progressive files are re-encoded in the OpenLaszlo Server.

GIF

All .gif files are supported. See the GIF Spec.

PNG

All Portable Network Graphics files ( PNG details at W3C information) are supported. This includes PNGs with alpha channels (transparency).

MP3
[Warning]
[SWF]

For proxied applications compiled to .swf, support is limited to audio MP3 files with sample-rates of 44.1 kHz, 22.050 kHz, 11.025 kHz, 8.000 kHz and 5.512 kHz.

Flash behaves poorly with mp3s it doesn't like, which makes tracking down problems difficult. If you run into a problem, a good idea would be to find out whether your mp3 plays in a standalone Flash app separate from OpenLaszlo. (You can use the mtasc.org compiler to test this, using Flash's Sound class to load an play your mp3.) If your mp3 does not work in standalone Flash, then the problem is not OpenLaszlo.

Mp3s are not natively supported by browsers, so applications that compile to DHTML should make provisions, as described above.

SWF

Macromedia Flash (SWF) files. (See Macromedia documentation).

For applications compiled to Flash, support is limited to SWF files that contain stills or animation only. Very limited ActionScript is allowed/supported (see below for more details).

For applications compiled to DHTML, .swf format files are not supported. See above for a workaround.

For up-to-date discussion of supported media and considerations about their deployment in various runtimes, see the OpenLaszlo Wiki.

3. Views and resources

The view is the most basic viewable element in an OpenLaszlo application and it is described in Chapter 26, Views. A view may have a resource attached to it. The resource represents a piece of media (image, audio, video, or animation) that the view controls. For images, the view typically displays the image. For time-based media (audio, video, animation), the view can be used to start, stop, or jump to a specific frame or time in the media.

For a simple informal introduction to attaching resources to views, see Chapter 8, Introduction to Media and Art Assets. The chapter you're reading now briefly goes over topics covered in that tutorial and then goes into greater depth.

Resources may be declared directly inside a view using the view's resource attribute:

Example 18.1. Importing a resource

<canvas height="110" width="100%">
  <view resource="resources/logo.gif"/>
</canvas>

Or they may be declared externally as first-class objects themselves:

Example 18.2. Importing a resource

<canvas height="110" width="100%">
  <resource name="myname" src="resources/logo.gif"/>
  <view resource="myname"/>
</canvas>

[Note] Note

You cannot set a resource on a view with subviews. The following code will result in an error:

Example 18.3. Setting a resource on a view with subviews

<canvas height="100" width="100%" debug="true">
    <resource name="myname" src="resources/logo.gif"/>
    <view>
        <handler name="oninit">
            this.setAttribute('resource', 'myname');
        </handler>
        <view bgcolor="red"/>
    </view>
</canvas>

3.1. Resources are not views

Note that a resource is not itself a view. Resources do not have x and y positions, or background color, or any of the 49 or so attributes that views have. A resource that is not attached to a view is essentially invisible to the application. As explained below, in order to manipulate a resource, you perform actions on the view to which the resource is attached.

3.2. Summary of APIs that act on resources

There are a number of view APIs (methods, fields, and events) that can be used to find out about and control a view's resource.

3.3. Resource declarations and naming

When a resource is declared externally, it must be given a name. The namespace for resources is global to the entire application. Resource tags can only be used as direct children of either the canvas or library tags.

This style of resource inclusion makes it simpler to separate out art assets from code and can make for improved designer/developer interactions.

When you directly import a resource without naming it, as in the first example, the LZX compiler assigns a unique name to that resource.

Behavior when two resources share the same name is undefined. Therefore you should take care to make sure that each resource has a unique and appropriate name.

4. Multi-frame resources

Laszlo supports the notion of a resource frame. A multi-frame resource is a resource that has a single name but refers to multiple independent pieces of media that are sequenced. Here is an example of declaring a multi-frame resource:

Example 18.4. Declaring a multi-frame resource

<canvas height="180" width="100%">
  <resource name="myresource">
    <frame src="resources/logo.gif"/>
    <frame src="resources/sky.jpg"/>
  </resource>
  <simplelayout spacing="5"/>
  <view id="myview" resource="myresource"/>
  <view layout="axis: x; spacing: 3">
    <text>Frame:</text>
    <radiogroup>
      <radiobutton selected="true">1</radiobutton>
      <radiobutton>2</radiobutton>
      <handler name="onvalue" args="v">
        myview.setAttribute('frame', Number(v));
      </handler>
    </radiogroup>
  </view>
</canvas>

You can use a multi-frame resource to construct an animation that moves between frames when you call the lz.view.play() method. Conversely, when you import a resource that contains an animation, each frame of the animation automatically becomes a frame in that resource. In other words, art assets that contain multiple frames themselves are multi-frame resources. Just as with multi-frame resources that are constructed in LZX by importing each frame individually, you can control the frame that is currently displayed.

Example 18.5. Importing an animation

<canvas height="220" width="100%">
  <simplelayout spacing="5"/>
  <resource name="myresource" src="resources/logo.swf"/>
  <button onclick="me.stop()"> Stop </button>
  <button onclick="me.play()"> Play </button>
  <text text="${me.frame + ' out of ' + me.totalframes + ' frame(s)'}"/>
  <view layout="axis: x; spacing: 4">
    <button isdefault="true" text="Set current frame:" onclick="me.setAttribute('frame', Number(tt.text))"/>
    <edittext id="tt" width="50"/>
  </view>
  <view id="me" resource="myresource"/>
</canvas>

Multi-frame resources are also useful for structuring the display of a single visual element or component that takes on different appearances:

Example 18.6. Using multi-frame resources

<canvas height="120" width="100%">
  <!-- create the multi-frame resource and give it a name -->
  <resource name="mybutton_rsrc">
    <!-- first frame is the mouseup state of the button -->     
    <frame src="resources/button-up.png"/>   
    <!-- second frame is the mouseover state of the button -->     
    <frame src="resources/button-over.png"/> 
    <!-- third frame is the mousedown state of the button -->     
    <frame src="resources/button-down.png"/> 
  </resource>

  <!-- attach the resource to a view with mouse events -->
  <view resource="mybutton_rsrc" onmouseover="setAttribute('frame', 2)" onmouseout="setAttribute('frame', 1)" onmousedown="setAttribute('frame', 3)" onmouseup="setAttribute('frame', 2)"/>

</canvas>

4.1. Frame rates

As of OpenLaszlo 3.1, all LZX applications are compiled to Macromedia Flash (SWF) files. With the exception of audio files noted below, SWF files are played by the Flash plugin at a fixed frame rate and all imported animations are played at that frame rate. This frame rate is fixed at 30 frames per second.

4.1.1. Audio rates

Animations that include an audio soundtrack are played at a rate that maintains the original audio/video synchronization in the incoming animation.

5. Resource loading

In the examples above, we've declaratively tied resources to view. But we haven't expressed anything about when or how the application should pull in the needed resource. We call this resource loading and LZX provides flexibility over how and when the application loads resources. The most important choice is whether a resource should be loaded at compile time or run time.

5.1. Compile time resources

Resources that are loaded at compile time are included in the application binary and add to the initial download size of the application.

Example 18.7. Including a resource at compile time

<resource name="myname"
        src="resources/logo.gif"/>

Compile-time resource inclusion is good for

  • small assets that are fixed for the life-time of the running application.

  • assets that are needed by all users of an application

Examples of resources that are included at compile time include most of the assets associated with user-interface components.

The LZX compiler may transcode compile-time resources for inclusion in the SWF application; the OpenLaszlo Server caches the results of these transcodes. The compiler checks the last-modified time of all compile-time-included resources to determine if an application needs to be recompiled.

5.2. Run-time resources

There are two main ways to load resources at run-time. The first is to use special syntax that is understood by the view's resource attribute. If you specify a value for this attribute that is an HTTP url, the compiler assumes that this resource should be loaded at runtime:

Example 18.8. Loading a resource at runtime

<canvas height="180" width="100%">
  <view resource="http:resources/logo.gif"/>
</canvas>

The second method is to use the setSource()method. The previous example is equivalent to the following one:

Example 18.9. Loading a resource at runtime using script

<canvas height="180" width="100%">
  <view oninit="this.setAttribute('source', 'http:resources/logo.gif')"/>
</canvas>

[Note] Note

LZX treats HTTP URLs that omit the two slashes //after the colon (:) as being relative to the application.

setSource() provides more control over the loading of the resource. Using this method, you can:

  • choose whether or not the server and/or client should cache the resource;

  • specify a URL that instructs the server to use a specific protocol to fetch the resource;

  • specify HTTP request headers that should be used when fetching the resource over HTTP.

5.2.1. Protocols for proxied applications

[Note] Note

This section does not apply to SOLO applications.

Before sending resources to a running, proxied, OpenLaszlo application, the OpenLaszlo Server first fetches (or proxies) the resource. The server uses the URL for the resource to determine how and where to get the resource. The term back-end is used to refer to the server that is ultimately providing the resource. Depending on your configuration, the back-end can be the OpenLaszlo Server host itself. Or it may be another distinct host.

The OpenLaszlo Server determines the protocol to use to communicate to the back-end based on the URL for the resource. By default, URLs that are to be loaded at runtime, that don't specify the protocol are assumed to be HTTP urls. OpenLaszlo supports communication to back-end hosts (including localhost itself) via the HTTP protocol. Relative HTTP URLs are thus fetched using a localhost HTTP transfer.

5.2.2. Caching

By default, in proxied applications, media that is accessed at runtime is cached both in the server and in the client browser. The OpenLaszlo Server uses the standard HTTP If-Modified-Since/Last-Modified caching mechanism specified in the HTTP 1.1 specification.

The OpenLaszlo Server actually maintains two media caches. One cache is used for resources that are requested at runtime. This is simply called the Media Cache. It also maintains a second cache for resources that are compiled into applications called the Compiler Media Cache. In general, the Compiler Media Cache settings only affect the speed of the OpenLaszlo compiler. The settings for the Media Cache can affect server performance. See the OpenLaszlo System Administrator's Guide for details.

The example below provides a text input area for you to type in urls that you'd like to test.

Example 18.10. Testing runtime resource loading

<canvas height="400" width="100%">
  <font src="helmetr.ttf" name="helvet"/> 
  <simplelayout spacing="3"/> 
  <view> 
    <simplelayout axis="x" spacing="2"/> 
    <text valign="middle">Type url:</text> 
    <edittext width="500" id="url">http:resources/logo.gif</edittext>
  </view>
  <text>(also try resources/logo.swf)</text> 
  <command onselect="butt.doit();" key="['Enter']" active="true"/> 
  <checkbox text="Cache in client" id="ccache"/> 
  <checkbox text="Cache in server" id="scache"/> 
  <button id="butt">
    Make request 
    <handler name="onclick"> this.doit(); </handler> 
    <method name="doit"><![CDATA[  
      status.setAttribute("text", "Requesting: " + url.text); 
      // Determine caching from checkboxes 
      var caching = 'none';
      if (ccache.getValue() & scache.getValue()) { 
          caching = 'both'; 
      } else if (ccache.getValue()) { 
          caching = 'clientonly'; 
      } else if (scache.getValue()) { 
          caching = 'serveronly'; 
      } 
      me.setAttribute("source", url.text, caching); 
    ]]></method> 
  </button> 
  <button onclick="me.stop()"> Stop </button> 
  <button onclick="me.play()"> Play </button> 
  <text text="${me.frame + ' out of ' + me.totalframes + ' frame(s)'}"/> 
  <text multiline="true" width="200" height="100" id="status"/> 
  <view id="me"> 
    <handler name="onload" args="e"> 
      status.setAttribute("text", 'loaded: ' + e); 
    </handler> 
    <handler name="onerror" args="e"> 
      status.setAttribute("text", 'error: ' + e); 
    </handler> 
    <handler name="ontimeout" args="e"> 
      status.setAttribute("text", 'timeout: ' + e); 
    </handler> 
  </view> 
</canvas>

6. Using the <image> tag

The image tag has the effect of declaring a view and associating a resource with it. This is a convenient shorthand that can make your programs easier to read and maintain. Do not confuse the image tag with the <img> tag. The latter is used to include images inside HTML text and behaves like the <img> tag in HTML.