Names on NEXUS: Under the Hood

I nearly have the basic data model and data processing functions pinned down for Names on NEXUS. Once again, that's my project, hinted at in a paper of mine (Keesey 2007), to relate the data in NEXUS files (Maddison et al. 1997) to definitions of names as governed by the PhyloCode.

I've had to learn some new technologies and code packages to accomplish this. Here's a rundown of some key ones:

BioJava
This is the most recent addition. Originally I had built my own library in ActionScript 3.0 to parse NEXUS files. But it had some limitations. NEXUS is a rather old format (as bioinformatics formats go), and different applications produce somewhat different versions. So rather than use my own ad hoc library, I decided I should get an open source one.

There aren't any in ActionScript, of course, but there are some in Java. This meant I had to switch NEXUS parsing from the front end to the back end, but in some ways that's better. It means I can stored parsed data in the database instead of having client application parse NEXUS data every time. In fact, it means that the client never has to actually see raw NEXUS data—it can just fetch the pre-parsed data.

I first looked into using the NEXUS-parsing code in Mesquite, an open-source phylogenetic analysis program. But it's not set up for simply using the parsing engine on its own—the parser is tied into a whole file-browsing package. Then I found BioJava, which had exactly what I needed. Just looka this package!

Unfortunately there are still some problems with opening certain NEXUS files. I downloaded some samples from TreeBASE and they flagged errors in the TREES section. The reason, as I found after hours of searching and considering whether it might be better just to write my own parser after all, turns out to be an extra comma in the TRANSLATE section. Still not exactly sure how I'm going to solve that one. But it works when I remove the comma!

Hibernate
Remember how I wrote a post a while ago about building classes that map from the Java back-end to the database? Turns out that was all unnecessary. Hibernate is a persistence layer that provides pretty seamless integration between Java and a database (in this case, a PostgreSQL database). Augmented by Hibernate Annotations and Hibernate Validator, it makes it fairly easy to set up and use a complex, well-organized database.

Well, okay, there's a bit of a learning curve first, but it's totally worth it. Incidentally, the book I used to learn it has what is possibly the best title ever.

Flex Data Management Services
Basically, Hibernate is to Java and databases as mx.data is to Flex and Java. It provides a persistence layer so that I don't have to keep track of whether or not I need to request certain data from the Java back-end. I just create DataService objects, tie them to Assembler classes on the back end, and it's all taken care of.

FlexUnit and JUnit
I've already extolled the virtues of unit testing. These wonderful (and, yes, comically-named) packages (huh huh) make it possible. I haven't built enough unit tests, really, but the few I have have been enormously useful in hunting down peculiar errors. And aside from that, since Eclipse can run JUnit tests natively, I can even use them to perform certain important tasks, such as setting up the database from annotated classes via Hibernate.

So What's Left For Me To Do?
Plenty. Although these premade packages help out enormously, I've still had to build an entire mathematics library, a MathML parser, and some tools for handling URIs. I've still got tons of work left to do on the user interface. (Event bubbling is helping a lot with that, by the way.) And, even when stuff is already built, just hooking up one pipe to another pipe can be more complicated than it seems.

Here's a rough list of what's left:

• Finalize the servlet for uploading and parsing NEXUS data. (I'm very close on this one.)
• Finish the required behind-the-scenes "search" features. Some of these might be a bit involved, like the ones that suggest possible links between NEXUS taxa and species or specimens or between NEXUS character states and apomorphies.
• Overhaul the way Names on NEXUS entities (particularly specifiers) are referenced in MathML.
• Finish the user interface. So far I just have a few forms. I still have to do tree visualization, stylesheets, high-level navigation, transitions, etc.
• Constrain access to certain functionality. Names on NEXUS is going to be a pretty open, collaborative tool, but I need to set a few boundaries. (E.g., I can't have any old person delete data.)
• Make sure the server's all optimized, with a static, JNDI-named Hibernate factory, etc.

And here are some things that aren't, strictly speaking, essential, but would be awfully nice:

• Create a servlet to provide permanent links for Names on NEXUS entities.
• Create unit tests for all relevant classes.
• Add JavaDoc and ASDoc comments to all code.

Part of me is also thinking about renaming the project. I mean, it's a good name for what it does right now, but what if I start to bring formats other than NEXUS into the fold? (Not that there are many, but....) Well, I'll probably cross that bridge when I come to it.

My goal is to get an alpha version online sometime this Spring and go open source with it by the Fall. We'll see....

Of Document Classes and Timeline Code

My coworker Ezra and I just got through figuring out a bizarre bug in an application created with Flash CS3 (using ActionScript 3.0). Since I didn't see anything else about it online, I thought I'd post it here.

The Set-Up: A loader SWF loads in section SWFs. The section SWFs have timeline code that trigger events based on timeline animations.

The Problem: On their own, the section SWFs worked fine. But once loaded into the loader SWF, only their class code would execute; none of the timeline code would execute, not even traces.

The Really Strange Wrinkle: I created a minimal proof-of-concept test, and it worked fine. It even worked when I used the section SWF from the actual project. So the problem seemed to be in the loader (but it really wasn't).

Here's the code from the minimal proof-of-concept test. Here's the loader class:

package timelinetest {
 import flash.display.Loader;
 import flash.display.LoaderInfo;
 import flash.display.Sprite;
 import flash.errors.IOError;
 import flash.events.ErrorEvent;
 import flash.events.Event;
 import flash.events.IOErrorEvent;
 import flash.events.SecurityErrorEvent;
 import flash.net.URLRequest;
 public class TestLoader extends Sprite {
  public function TestLoader() {
   super();
   addEventListener(Event.ADDED_TO_STAGE, loadContent);
  }
  private function loadContent(event:Event):void {
   var loader:Loader = new Loader();
   var info:LoaderInfo = loader.contentLoaderInfo;
   info.addEventListener(IOErrorEvent.IO_ERROR, onContentIOError);
   info.addEventListener(SecurityErrorEvent.SECURITY_ERROR, onContentSecurityError);
   info.addEventListener(Event.INIT, onContentInit);
   loader.load(new URLRequest("./loaded.swf"));
  }
  private function onContentInit(event:Event):void {
   var info:LoaderInfo = event.target as LoaderInfo;
   addChild(info.content);
  }
  private function onContentIOError(event:IOErrorEvent):void {
   throw new IOError(event.text);
  }
  private function onContentSecurityError(event:SecurityErrorEvent):void {
   throw new SecurityError(event.text);
  }
 }
}

... and here's the section class:

package timelinetest {
 import flash.display.MovieClip;
 import flash.events.Event;
 public class TestLoaded extends MovieClip {
  public function TestLoaded() {
   super();
   stop();
   addEventListener(Event.ADDED_TO_STAGE, onAddedToStage);
  }
  private function onAddedToStage(event:Event):void {
   play();
  }
  public function onSpecialFrameReached():void {
   trace("special frame reached");
   stop();
  }
 }
}

... and on frame 10 of the root timeline in loaded.swf I had this code:

trace("calling onSpecialFrameReached()");
onSpecialFrameReached();

So on frame 10, it called onSpecialFrameReached(), which stopped playback. This worked fine. But similar code did not work fine in the actual project.

So What Was the Discrepancy?: Making the following modifications (underlined) to TestLoader.as caused the timeline code failure:

package timelinetest {
 import flash.display.Loader;
 import flash.display.LoaderInfo;
 import flash.display.Sprite;
 import flash.errors.IOError;
 import flash.events.ErrorEvent;
 import flash.events.Event;
 import flash.events.IOErrorEvent;
 import flash.events.SecurityErrorEvent;
 import flash.net.URLRequest;
 public class TestLoader extends Sprite {
  private var loaded:TestLoaded;
  public function TestLoader() {
   super();
   addEventListener(Event.ADDED_TO_STAGE, loadContent);
  }
  private function loadContent(event:Event):void {
   var loader:Loader = new Loader();
   var info:LoaderInfo = loader.contentLoaderInfo;
   info.addEventListener(IOErrorEvent.IO_ERROR, onContentIOError);
   info.addEventListener(SecurityErrorEvent.SECURITY_ERROR, onContentSecurityError);
   info.addEventListener(Event.INIT, onContentInit);
   loader.load(new URLRequest("./loaded.swf"));
  }
  private function onContentInit(event:Event):void {
   var info:LoaderInfo = event.target as LoaderInfo;
   loaded = info.content as TestLoaded;
   addChild(loaded);
  }
  private function onContentIOError(event:IOErrorEvent):void {
   throw new IOError(event.text);
  }
  private function onContentSecurityError(event:SecurityErrorEvent):void {
   throw new SecurityError(event.text);
  }
 }
}

See the difference? Before, there was no reference to TestLoaded in loader.swf. With this change, there is. Before, when loaded.swf was loaded, it contained the only reference to TestLoaded. This was as the document class, which had extra code from the timeline, so it was sort of a unique version of the class. But after the change, loader.swf had a copy of the TestLoaded class as well, a copy which had precedence over the unique version loaded in from loaded.swf. So the version with timeline code was ignored—no execution of timeline code.

The Remedy: This is something that was going to be done anyway—none of the loaded section SWFs should use the base version of the section/loaded class. They should all use their own subclasses. That way, nothing in loader.swf can block them.

Another solution (which isn't appropriate for our project, but it might be for others) would be to reference sections in loader.swf using an interface. (But even with this you'd still have to have different implementations in different SWFs, or they might conflict with each other—I think—should test that sometime.)

Really, though, there should be an option when assigning document classes, as there is when linking to a class from the library. You should be allowed to specify whether you want to use the class itself or an ad hoc subclass. If the former, the presence of timeline code should flag a warning message. Remind me to report this to Adobe.