Hacking SWF – PlaceObject and the Ratio Field

According to the SWF10 specification, PlaceObject2

.. can both add a character to the display list, and modify the attributes of a character that is already on the display list.

The placed character is usually defined earlier in the SWF, and can be anything supported by SWF, e.g., Shape, MorphShape, Sprite, Text, EditText etc. It stays on the display list until it is explicitly removed by the RemoveObject tag. PlaceObject2 might also tell the Flash Player that the placed object is to be treated as a mask (and what depth range will be masked), and might give the character an instance name (if it is a Sprite).

Additionally, PlaceObject2 might carry an optional ratio parameter. According to the SWF10 specification, it

.. specifies a morph ratio for the character being added or modified. This field applies only to characters defined with DefineMorphShape, and controls how far the morph has progressed. A ratio of zero displays the character at the start of the morph. A ratio of 65535 displays the character at the end of the morph. For values between zero and 65535 Flash Player interpolates between the start and end shapes, and displays an in- between shape.

For Flash users, this is better known as a “Shape Tween”.

However, the statement that  “this field applies only to characters defined with DefineMorphShape” is incorrect. It also applies to Sprite characters (“MovieClips”).

The Flash Player uses the value of the Ratio field to determine whether or not to reset the playhead in the placed Sprite to frame 1, when jumping to arbitrary frames in the parent timeline.

To illustrate that, let’s create a simple Flash movie using the Flash IDE. We create a one-frame MovieClip (called “square”) containing a simple shape. We create another MovieClip (called “animatedSquare”), which contains “square”, animated by a motion tween over 20 frames. We place “animatedSquare” on the main timeline (one frame only). When the Flash Player executes the resulting SWF, we see “animatedSquare” looping over all its 20 frames, as we would expect.

Here are the guts of the resulting SWF (simplified). Nothing surprising in there:

Now, let’s make the main timeline 10 frames long (each frame containing “animatedSquare”). The behavior doesn’t change, “animatedSquare” still loops over all of its 20 frames as expected. Also, the SWF tags still don’t reveal anything surprising, 9 more ShowFrame tags were added:

Finally, let’s remove “animatedSquare” from frame 5, leaving it only on frames 1-4 and 6-10:

This is where things get interesting. The character at depth 1 (our “animatedSquare” Sprite) is removed after frame 4, frame 5 is displayed without any content, and then “animatedSquare” is placed back on depth 1. Only now, the PlaceObject2 tag carries a value (5) in the Ratio field. Now why is that?

If we let Flash Player execute this SWF, you will notice the following behavior:

  1. In frames 1-4, the first 4 frames of “animatedSquare” are displayed
  2. In frame 5, a blank frame is displayed
  3. In frames 6-10, the first 5 frames of “animatedSquare” are displayed
  4. The main timeline then loops and jumps back to frame 1, displaying the first frame of “animatedSquare” again. Back to 1. Etc.

If we would put a gotoAndPlay(6) action on frame 10 of the main timeline, “animatedSquare” would reset to frame 1 once, and then loop over all 20 frames infinitely.

So what happens here is that once the Flash Player encounters a PlaceObject tag that attempts to place a Sprite on a depth that was previously occupied by the same Sprite, it looks at the ratio fields of the current and previous PlaceObject tags. If both carry the same value, the child Sprite keeps on playing normally. If not, the child Sprite’s playhead is reset to frame 1.

There is a little more to it yet, so if you are interested in digging deeper you should take a look at the Gnash Wiki, which lists many cases.

Talking about WebGL and Security…

i was just browsing MrDoob‘s beautiful three.js examples when i stumbled on something quite creepy.

This is a screenshot (click for full size image) of my browser rendering one of the demos:

What you see is a messed up background texture showing fragments of my local desktop. Note that some of the windows that you see weren’t even open anymore at the time the demo ran, like the Keychain one.. *shudder*

Every time i reload the demo, the background texture shows different, seemingly random fragments. This happens in both Chrome (13) and Firefox (6). I’m on a MBP featuring an Intel HD Graphics 3000, running Mac OS X 10.6.8.

What i suspect is happening here is that either WebGL, the drivers, or the GPU itself corrupts the GPU texture memory somewhere and pulls in content it shouldn’t pull. I’m certainly not an expert in this field, so this is really just an (un)educated guess.

Thoughts?

[Update: I logged a bug with Mozilla, and this bug was marked fixed and closed today. Thanks Benoit and everyone else from Mozilla and Google who was involved!]

ActionScript 3 Bitmap Tracer (Vectorizer): as3potrace

Ever wanted to convert a bitmap to vector shapes, at runtime, in Flash? Look no further. Let me introduce you to as3potrace, an ActionScript 3 library to trace bitmaps.

As the name suggests, this is a port of the well known C library potrace by Peter Selinger. To be more exact, it is a AS3 port of Vectorization, a C# port of potrace 1.8 by Wolfgang Nagl.

Like potrace, as3potrace is released under GPL. The SWC and source code are available on GitHub:

https://github.com/PowerflasherBR/as3potrace

Demo

This movie requires Flash Player 10

Demo source code: https://gist.github.com/940219 (Warning: ugly)

Usage example

A minimal example of how to trace a bitmap with as3potrace, and draw the result into a Shape:

Note that you can write your own backends to ease handling/processing of generated shapes. The one backend that i already wrote, GraphicsDataBackend, produces GraphicsData structures that you can immediately draw using Graphics.drawGraphicsData().

Alternatives

And as always, after i finished porting this little beauty, i found out that this has been done before by the amazing folks at LibSpark. Check out nitoyon’s PotrAs (also GPL’ed).

Adobe iPhone Packager 2.0

The whole Flash community has been jumping for joy ever since Apple revised their controversial App Store License Agreement in early September this year. Apps created with Adobe’s “iPhone Packager” are not longer banned from the App Store, and Adobe started working on the product again.

Section 3.3.1 now merely forbids the use of private APIs.. tooling and language restrictions have been lifted, especially the part that prohibits “linking to documented APIs through an intermediary translation or compatibility layer or tool” – the iPhone Packager uses LLVM to cross-compile ActionScript to iPhone-native byte code, which is now allowed.

I didn’t hear many people commenting on Section 3.3.2 though, which i find weird. I think the real good news is this very section. Code interpreters are now explicitly allowed, given that “all scripts, code and interpreters are packaged in the Application and not downloaded”.

This means that Adobe could potentially shelve the existing, LLVM based iPhone Packager, and instead deploy a regular AIR runtime with a separate SWF. This would certainly make the publishing process a lot faster, as there is no cross-compilation step anymore – a standard SWF is published, which is then just packaged together with an iOS AIR runtime. I am uncertain which approach would perform better though: interpreted and JIT compiled via virtual machine, or LLVM translated native code.

Also feasible: a Flash Player engine that developers can plug in to their Cocoa apps, to mix UIKit and SWF. Best of both worlds.

Where do you think the iPhone Packager is heading?

Mythbusting “HTML 5 Did Not Kill Flash”

So, i was thinking.. the Flash Player version number in the “HTML 5 Did Not Kill Flash” comic strip (258.1) … PLAUSIBLE?

It is obvious that we deal with Jean-Luc Picard, captain of the Starship USS Enterprise NCC-1701-D, which places the scene in the Star Trek TNG series. According to Wikipedia, Star Trek TNG is set in the 24th century from the year 2364 through 2370.

Today: 1285956121684 (Fri Oct 1 15:02:01 GMT-0300 2010)
Star Trek TNG start: 12433402800000 (Wed Jan 1 00:00:00 GMT-0300 2364)
Star Trek TNG end: 12654327600000 (Fri Jan 1 00:00:00 GMT-0300 2371)
Assumed 18 months release cycle, in ms: 47304000000
Release cycles until Star Trek start:
   12433402800000 - 1285956121684 = 11147446678316
   11147446678316 / 47304000000 = 236
   236 + 10 = 246
Release cycles until Star Trek end:
   12654327600000 - 1285956121684 = 11368371478316
   11368371478316 / 47304000000 = 240
   240 + 10 = 250

So assuming a strict 18 month release cycle, this places version 258.1 just about a decade or so after Star Trek TNG (versions 246-250) ends. If we further assume that new technologies will emerge in the coming 350 years that speed up release cycles a bit, i think we can safely say that Flash Player version 258.1 is indeed plausible.

Hypothetically speaking…

Flash IDE Inspired Flex 4 Timeline Component

Here’s a little demo of a Flex 4 component i wrote that displays a Flash IDE-like timeline for any SWF you load into it. The timeline data is reconstructed by as3swf (i discuss how that works in a previous blog post, SWF Timeline Reconstruction with as3swf). The timeline is not interactive in this demo, and only the root timeline is shown.

Continue reading

My Silly Gist Collection

I’ve been quite busy lately on both client and personal projects (can’t talk about the former, other than it being a big ass Flex 4 enterprise application i’ve been working on for the fine folks at Powerflasher; and it’s too early to talk about the latter.. stay tuned).

In the last few weeks i silently released random little experiments on Gist – mostly fallout from personal projects of mine. Nothing super impressive, all pretty rough, but i thought some of you might be interested.

Continue reading

SWF Timeline Reconstruction with as3swf

In case you havent heard of it yet, as3swf is an ActionScript 3 library to parse and publish SWF files. It does that rather well by now, providing full roundtrip publishing, plus some neat extra features like shape export to AS3 Drawing API, AS3 GraphicsObjects, FXG, and Objective-C.

Whenever i find some free time i’m working on adding new, useful features. One feature of as3swf, that i haven’t talked about much yet but is implemented for quite a while already, is the reconstruction of timelines as you know them from Adobe’s Flash IDEs.

In the first place, timelines in as3swf help you to export and render layered animations. Having a long list of parsed SWF tags won’t help much if you want to export or render frame X, as the Flash Player would display it.

Continue reading

The Grand SWF Archive

What if somebody would write a spider (which respects common conventions like robots.txt etc) that searches for and archives publicly available SWF files, and a service that makes graphical assets found in those SWFs (vector shapes, bitmap images, videos, fonts, etc) available for public browsing, similar to what the Internet Archive’s Wayback Machine does?

As SWF, at least according to Adobe, is an open format, this would be both technically and legally feasible, wouldn’t it? From a legal perspective, what line would need to be drawn where, and why exactly?

Posted in SWF

“Flash is as open as HTML5″ – No, it isn’t.

Lately, Adobe representatives and Flash fan boys alike became more vocal than usual about the alleged openness of Flash. This is probably spurred by the proposed feature set of HTML5, as well as the decisions of a certain vendor to ban Flash from some of their products, both potentially being threats to Adobe and the Flash Platform.

I originally posted the following article as comment to an article by Serge Jespers, “Flash is as open as HTML5“. Serge is an Adobe Platform Evangelist. I thought this comment deserves its own post, so here we go.

No, it isn’t.

And it is beyond me why so many independent Flash Platform developers fail to see it. I completely understand of course why Adobe evangelists downplay it.

It is irrelevant for the so called “open web” whether the Flash Player is going to get open sourced or not. That’s not the point. Microsoft won’t open source their browser, Opera won’t open source their browser, etc.

Relevant is who decides about the development of the data format that a runtime consumes, and the APIs a runtime provides to access that data. In the Flash world that’s SWF and the Flash Player APIs, both controlled by a single vendor: Adobe. In the HTML world that’s HTML and DOM, controlled by many vendors, including you and me, via standards bodies.

Adobe neither provides formal means for other companies and individuals to participate in the development of SWF and Flash Player APIs, nor does it provide detailed work-in-progress specs to the general public for discussion. This effectively rules out the possibility for third parties to provide alternative runtimes. The runtime and its specs are released to the public at the same time.

This is the exact opposite of “open”.

To make matters worse, the specs released by Adobe are incomplete and buggy (e.g. the SWF spec fails to explain how exactly shapes are supposed to be rendered and leaves out information on codecs, the ABC spec is plain wrong on some things) and generally infested with patented technologies.

Of course i understand why it is how it is (and likely always will be). If Adobe were to become truly open and put the development of SWF etc in the hands of standards bodies, the 1.5 year release cycle would become a 10+ years release cycle. Innovation would slow down significantly. I as a Flash Platform developer wouldn’t want that to happen.

However, sorry to say that, but to tout “Flash is as open as HTML5″ is pure FUD.