Subject: [Dev] morpho/jalama/mellon roadmap
From:    Matt Jones <jones@nceas.ucsb.edu>
Date:    Wed, 18 Jun 2003 12:43:59 -0800
To:      NCEAS Developers <dev@nceas.ucsb.edu>, 
         Chris Jones <cjones@lifesci.ucsb.edu>

So here it goes.  I've been mulling this situation with our need for
a new Morpho that supports EML2, and how this relates to the Mellon
project goals for product deployment, and the Jalama project goals.
I appreciated the discussion that we had on the phone last week, and
understand the concerns.  I also talked this over with Mark, and feel that
I want to go in a different direction than was proposed during our call.
So, here is my proposal for a workplan for these projects based on our
recent discussions.  Feedback is appreciated.

First, here are the priorities as I see them:

  1) Deliver a stable, attractive version of Morpho that supports
  EML2 editing
  2) Deliver on the Jalama project goals
     -- schema/metadata driven user interface generation
     -- desktop metadata and data entry tool
     -- field-based (handheld) data entry tool
  3) Deploy an updated version of the OBFS/NRS Registry that supports
  editing


The goals of Jalama are to produce exactly the kind of technology
needed in Morpho.  Current Morpho implementations are an early version
of exactly what Jalama needs to do.  So....it makes sense to me to
use Jalama to improve Morpho, which has the benefit of eliminating
duplication of effort and makes it so that Jalama need not create its
own Morpho-like infrastructure.  We are far too small of a team to be
implementing the same products twice.  We don't have that luxury, and
so each of our product teams must accept responsibility for producing
usable and stable software over extremely short development periods. The
argument against is that Jalama, and in particular, the Mozilla-Java
bridge, has not been tested enough to be stable.  My feeling is that,
over the next 6 to 9 months that testing needs to happen anyways, so we
should do it as part of the effort to create a unified product.

We'll have to maintain the current version of Morpho simultaneously,
so I think we should create a branch for the current version to
which we can continue to make bug fixes and release minor patches,
and the HEAD should be used to integrate Jalama and support EML2.0.0.
The Mellon person would focus on the bug fixes and maintenance of the
current version, and then on comprehensive testing of the Jalama-ized
version. Eventually we might want to release this Jalama-ized version
as Morpho 3? Here's how I think we should focus our efforts:

1) Dan (9 months): Redesign and implement the data package subsytem
in morpho as we had discussed to allow new metadata standards to be
supported.  Begin setup of testing framework until Mellon person starts. 
Finish schema-driven editor for EML2.  Text import wizard recode. Revamp

2) Matthew:  Integrate Jalama engine in Morpho, at first focusing
on just a single screen (e.g., the attribute editor or access control
editor, then work on further components such as the data package wizard).
Do design work for all EML2 screens (screenshots) with Chris that others
can implement in XUL/XBL.  Help John/Chris with design work for XUL
generation engine based on your XUL/XBL experience.

3) John: Finish work on making Jalama work on Mac via webclient
(or whatever is needed for it to work on the mac). Work on interface
generation approaches for Jalama -- ie, how do we take a schema (XML
Schema, DTD, or EML document) and produce a data entry application
for that schema?  This is the Jalama version of the current metadata
editor in Morpho.  Schema and metadata in, XUL/XBL/JS interface out.
This interface should be able to be run in Morpho when it is output (ie,
package everything up and install as a plugin in Morpho to enable).
It should also be able to be run in other environments, possibly
standalone for the field devices.

4) Chris: Design work for user interface generation based on experience
with XUL/XBL and patterns.  Focus on how do we generate a usable interface
from schema and metadata. Project oversight.  Help with design work for
EML2 screens, which should give experience with design patterns without
the XUL/XBL development cost.

5) Mellon position: Develop the new editable web interface for the
OBFS/NRS registry, with user authentication (3 weeks).  Develop a
comprehensive testing and profiling framework for Morpho/Jalama.  Test and
fix morpho bugs in current version, profile and fix Morpho performance
issues, release as needed on branch.   Respond to user issues with Morpho,
fix them.  Throw lots of bad data at Jalama, find the problems, fix or
help fix them.  Write documentation for Morpho and EML and incorporate
it into Morpho in appropriate, context-sensitive ways.

6) LTER position (possible use of a new LTER programmer for KDI for
6 months).  Can help as needed if it comes through.

OK, so when should this be done?  Jalama really needs to produce a usable*
product that we can distribute by the next annual report, so I think that
should be our target goal.  The development cycle for Jalama needs to
be measured in months, not years:  we should probably have development
milestone goals for every 3 month period.  Morpho EML2 support needs
to be done yesterday.  Morpho stability and usability needed asap.
Here's a proposed set of milestones:

Timeline:
  Jul 1, 2003:  
  Aug 1, 2003: 
                M: Morpho Data Package redesigned (maybe w/ caching) (dan)
                J/M: Create Morpho 1.4, 1.5, Jalama branch for maintenance
                     (matt)
                M: Java data package wizard (matthew)
  Sep 1, 2003:  M: Integrate EML2 stylesheets in Morpho (matthew)
                M: Metacat support for inline data and other issues (jing)
                M: Morpho backend data package re-implemented (dan)
  Oct 1, 2003:  M: New schema-based editing package (dan)
                M: Text import wizard for EML2, incl. use new dp architecture
                (dan)
                M: Modify search system to not do triple backtracking (dan)
                M: Accomodate new EML2 data storage needs (e.g., inline) (Dan)
                M: Revise stylesheets for navigation improvements (matthew)
                R: Registry updated for editing (mellon)
  Nov 1, 2003:  M: Testing framework in place, critical tests developed (mellon)
                M: Morpho 1.5.0 beta 1 (dan)
  Dec 1, 2003:  
  Jan 1, 2004:  Incorporate EML input into generation engine
  Feb 1, 2004:  Morpho 1.5.0
  Mar 1, 2004:
  Apr 1, 2004:  XUL Generation engine w/ EML support released
                Plan/design for moving to field-based device
  May 1, 2004:
  Jun 1, 2004:
  Jul 1, 2004:  Implementation/alpha for field-based device
  Aug 1, 2004:
  Sep 1, 2004:
  Oct 1, 2004:  Implementation/beta for field based device
  Nov 1, 2004:
  Dec 1, 2004:
  Jan 1, 2005:  Final for field-based device
  Apr 1, 2005:  Testing/evaluation of system by PISCO/LTER scientists
  Jul 1, 2005:  ?
  Oct 1, 2005:  ?
  Jan 1, 2006:  ?


Well, that's pretty rough.  But, its at least something we can critique
and evaluate.  I'd like to keep a roadmap like this up-to-date from now on
(so I will check a copy into CVS for editing).  If we're falling off of
our goals, I'd like to know asap.  As we get to the less detailed areas
of the roadmap, we should fill out the monthly goals (ie, always have
a detailed plan for the next six months).

This is an initial attempt at this whole thing.  I'd like to discuss
it in more detail with you.  As you know, I'm headed on vacation, so
that'll have to wait.  But, hopefully soon.  I'm planning on being in
Santa Barbara July 21-24 (maybe 25), and would like to have a full day
on this topic, probably on July 24. Sound good?

Thanks for your attention and hard work.  Feedback appreciated.

Matt

*usable: to be of use, but not necessarily optimal, optimized, or fully
completed
