[kepler-dev] where do actor packages meant to work with Kepler 1.0 go in the repository?

Wed Jun 25 21:04:27 PDT 2008

I think this is testament to the requirement that actors (i.e. pPOD) and
library extentions i.e. (COMAD) not be part of the core kepler build and
they should be stored in their own project, and built with reference to the
kepler/ptolemy jars rather than built into them. However, there is obviously
a lot of work to be done before that's possible (I spent half an hour trying
to separate the ptolemy.actor.lib package out from ptolemy, but there are
too many other packages that require it to compile so i gave up rather
quickly).

What I think needs to happen after that, is the actor repository needs to be
revived and revamped. Kepler should be shipped with only a core set of
actors, but have an option from kepler to browse all the libraries available
in the repository and download the ones that want. metadata can be included
to say if the packages are untested unstable or stable for the specific
version of kepler the user is using, whether the package is certified by
kepler or not and anything else that could be useful. Developers of actor
packages can then simply upload their packages to the repository and not
worry about getting it into a specific release of kepler.

This doesn't really solve where to stick the code tho. But I think once my
proposed infrastructure was put into place, then it isn't a big deal where
the code is stored, as the repository could link to the packages home page
where a developer could find the source code.

I could throw out a few possible solutions, but i can't think of any that I
think of any that i would actually suggest doing. In my view, something like
what i've suggested above is the only good solution.

Kepler SDK anyone? (the thought just popped into my head i'll ponder on that
one some more)

Sorry I can't be of any real help :)
-Tristan

On Thu, Jun 26, 2008 at 12:59 PM, Timothy McPhillips <tmcphillips at mac.com>
wrote:

> Dear All,
> With the Kepler 1.0 release out and the migration to svn complete I am
> about to begin migrating into the Kepler repository some code we've been
> developing here at UCD.  Some of these checkins will be relatively
> straightforward, but other code will present some integration, source code
> management, and build system challenges I'd really like to get everyone's
> input and help with.
>
>
> *** Upcoming checkins ***
>
> 1.  First, I will be committing a new version of the COMAD framework to
> src/org/kepler/comad.  The current version in Kepler is included under
> src/org/nddp.  The major difference with the new version is that it includes
> comprehensive support for recording provenance from COMAD workflows and
> storing the results as provenance-annotated 'trace' files.
>
> 2.  Second, we would like to begin committing the code specific to the pPOD
> extension to Kepler (phylogenetics actors and data types for the AToL
> community).  This integration will be trickier.  Please see "complications"
> below.
>
> 3.  Third, when the COMAD and pPOD sources have been successfully
> integrated with the main Kepler system, I will delete the code under
> src/org/nddp, this package having been superseded by the two sets of
> checkins above.
>
>
> *** Complications  ***
>
> Ok, here's the conundrum.  The pPOD extension is mostly a set of actors for
> phylogenetics analyses.  Prior to the Kepler 1.0 release we provided a preview
> release <http://daks.genomecenter.ucdavis.edu/kepler-ppod/> of the pPOD
> actors (based on the COMAD framework) to the AToL <http://atol.sdsc.edu/> community
> (see a poster<http://daks.genomecenter.ucdavis.edu/kepler-ppod/poster.pdf>
>  and presentation<http://daks.genomecenter.ucdavis.edu/kepler-ppod/Kepler-Ppod.pdf> about
> the Kepler/ppod preview).  This was not meant for production use, but rather
> as a way to get feedback from the community on our work.  However, in the
> very near future we would like to release this code as a package of actors
> and community-specific customizations to the Kepler GUI.   Note that this
> release will not be targeted only at an abstract community of folks who
> might possibly try it out, but also at particular scientists who have asked
> for specific workflows based on the pPOD package and who need them very
> badly for their own research.)
>
> As you might expect, we do *not* want to:
>
> N1.  Wait for the next release of Kepler before distributing these actors.
>
> N2.  Release a package of actors that has been tested only against  the
> revision of Kepler that we happen to find at the trunk of the Kepler
> repository on a particular day.
>
> Instead, we want to (read 'must') release a package of actors that works
> with Kepler 1.0.  When another release of Kepler is made (Kepler 1.1, say),
> we will need to release a version of the pPOD actors that works with *that*
> version of Kepler.
>
> There are number of questions implied by this need to release a package of
> actors designed to work with a particular, supported release of Kepler:
>
> Q1. How do we build the pPOD actors against Kepler 1.0 rather than against
> the current revision in the repository?
>
> Q2.  Where should we put the pPOD actors and support classes in the Kepler
> repository directory structure?  If we put them in, say, src/org/ppod (under
> kepler), then all of this code will be seen by the build system and all
> kepler developers will find themselves building code that is not necessarily
> meant to be compiled against the current revision of Kepler in the
> repository.  If any classes the pPOD package depends on elsewhere in Kepler
> are renamed, moved, removed, or changed significantly, the pPOD code will no
> longer build against the current revision of Kepler in the repository.  The
> build will break.
>
> A further complication is that once the 1.0-compatible version of the pPOD
> package is out, we likely will want to begin building against the latest
> revision of Kepler in the repository until the next version of Kepler is
> out, at which point we'll want to build against *that* supported version of
> Kepler.
>
> In short, we need to be able to choose which revision of Kepler in the
> repository we build the pPOD package against, without disturbing folks
> working with the current revision of Kepler.
>
>
> *** Solutions? ***
>
> S1.  On the surface this problem sounds like one that could be solved with
> branches.  And I believe it possibly could, and that branches likely will be
> involved in some way (for example, we will still want to provide bug fixes
> to the 1.0-compatible version of the pPOD actors when we're working on the
> 1.1-compatible release).
>
> However, is this how we want to treat the Kepler 1.0 release branch in the
> repository?  Do we want to be checking hundreds of source files into the
> Kepler 1.0 release branch indefinitely, and almost certainly introducing new
> bugs and instabilities there?
>
> S2.  An alternative would be to create a separate source tree for pPOD in
> the svn repository (as a peer to kepler, for example).  The build system
> would not include this 'extension' when compiling the Kepler java sources.
>  Instead, we might build the pPOD package against the Kepler 1.0 release jar
> itself.  This approach might scale better and enable everyone providing
> their users 1.0-compatible actors to share their work with the community
> through the Kepler repository without requiring that their code work with
> the current revision of Kepler.
>
> The problem with this second approach is that it might not be so easy to
> build against the current revision of Kepler, should one choose to do so
> (and as we likely will between releases of the pPOD package).  At least, not
> without building the Kepler jar each time.
>
> S3.  A variant of S2 would be to enable developers to tell the build system
> to optionally include extensions in the build.  With the extensions stored
> as peers to the kepler directory in the repository, developers could choose
> which extensions to check out, and specify different branches or tags for
> Kepler and for each extension checked out.
>
> This final approach is the one we currently use at UCD.  We store our
> Kepler extensions in our own repository at UCD and use a tweaked version of
> Kepler's standard build.xml file that supports extensions in this way. One
> nice feature of this approach is that it does not require all extensions to
> be stored in the same repository.  The problem with our current approach is
> that it is harder for us to share our extensions with the rest of the
> community.  We'd much rather use the Kepler repository, but at the moment we
> can't as explained above.  In short...
>
>
> *** Help! ****
>
> Does anyone else have similar needs?  Does anyone have any thoughts or
> advice?  Are there other solutions?  The UCD team needs a solution soon
> because we want to share our source code widely with others in the
> community, *and* we need to make a release of the pPOD actors (as well as a
> number of other extensions we're currently working on outside the Kepler
> repository for the same reasons) to users (including people working here at
> the UCD Genome Center <http://genomecenter.ucdavis.edu/>) in the very near
> future (a few weeks from now).
>
> Thanks very much for your thoughts and help!
>
> Tim
>
>
> _______________________________________________
> Kepler-dev mailing list
> Kepler-dev at ecoinformatics.org
> http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
>
>

-- 
Tristan King
Research Officer,
eResearch Centre
James Cook University, Townsville Qld 4811
Australia

Phone: +61747816902
E-mail: tristan.king at jcu.edu.au www: http://eresearch.jcu.edu.au
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mercury.nceas.ucsb.edu/ecoinformatics/pipermail/kepler-dev/attachments/20080626/d028542a/attachment-0001.htm 