[kepler-dev] Modifying MoML files

Thu Apr 12 15:47:32 PDT 2012

I would try hard to avoid writing version specific code, it can get very messy and confusing, as there are a variety of scenarios to consider, and ways to trap yourself. A user can upgrade from N.x to N.y, or from N.x to N.z, or they can restart into any older version. If you can't find an alternate solution, I believe you can access your module's version numbers using Version.fromVersionString(yourModule.getName()).
I wonder if you can't just write your filters and leave them in place / update them as you release new versions of your module. I suppose it depends on how much you're changing (the same) things.
Depending on the complexity of the workflow and how many changes the users have made, at a certain point it also might make more sense to instead just distribute a new version of the workflow that requires changes and ask users to use it instead of their old copy.

I can't think of much else to recommend on diff'ing beyond what's been discussed. I'm also not sure on if rearranging the XML will break things, but I'm curious to hear.

Derik

On Apr 11, 2012, at 8:34 AM, Hogan, D. (GE Energy) wrote:

> Thanks Derik.  That answers another question that I had: since Kepler is
> versioned and BackwardCompatibility is global, how do I associate
> transformations with only certain versions?  I don't have an Initialize
> class in our modules so I'll look into adding one.
> 
> Re: LSID.  Your approach makes sense.  Do you have any recommendations
> on how I can get meaningful differences between kar files?  After I run
> it through the filter, there are many trivial differences.  If I apply a
> specific format to MoML files by running it through XSLT, will it break
> anything?  For instance, I'm not sure where it decides the order for I/O
> on actors in the GUI.  Is it from the XML or the order they are
> initialized in Java?
> 
> -----Original Message-----
> From: Derik Barseghian [mailto:barseghian at nceas.ucsb.edu] 
> Sent: Tuesday, April 10, 2012 8:08 PM
> To: Hogan, D. (GE Energy)
> Cc: kepler-dev at kepler-project.org
> Subject: Re: [kepler-dev] Modifying MoML files
> 
> I wouldn't take the approach of writing a script to unzip, process MoML,
> and zip your KARs. I don't have much experience with the filters, but
> Kepler utilizes them; I'd look into how it does. I think a better
> approach would be to make your changes to your actor(s), addFilters to
> Kepler's list(s) as you deem fit in your module's Intialize class, and
> distribute a new version of your module. Users get the updated module,
> open their old KARs, and the filters will be applied to the MoML (if the
> user simply opens the KAR and makes no changes, they will be prompted to
> save when closing since these changes have been automatically applied).
> I believe this way you'll avoid issues to do with proper
> versioning/identification of objects. Kepler maintains LSIDs and derived
> from LSID lists for components, workflows and KARs. E.g. when you modify
> an actor, its LSID revision is incremented, the workflow's LSID is
> incremented, and when you save this workflow, the KAR gets a new LSID,
> and its manifest is updated appropriately.
> 
> 
> Derik
> 
> On Apr 10, 2012, at 4:05 PM, Christopher Brooks wrote:
> 
>> You've found the ptolemy/moml/filter classes which is how we handle
>> sorts of changes at Berkeley.
>> 
>> My comments are interspersed with your message below.
>> 
>> On 4/10/12 10:02 AM, Hogan, D. (GE Energy) wrote:
>>> What's the best way to modify MoML inside of kar files?  Should I use
>>> ptolemy.moml.MoMLParser, some other class in Kepler, standard tools
>>> (XSLT/DOM) or something else?
>> Unfortunately, MoML is not well structured xml.  This is because
>> of the <configure> tag.  There is something in either the kepler email
>> archives or the ptolemy-hackers archives about this.
>> 
>> I have a test at ptolemy/configs/test/validDemos.tcl
>> that strips out the <configure> tags and runs a validating parser on
> the results.
>> However, the problem is that you need those configure tags for
> plotters etc.
>> So, using a validating parser will not be much help.
>> 
>> You could use other XSLT tools as long as the tool does not require
> valid xml.
>> 
>>> 
>>> I'm trying to do two types of operations with MoML.  After I update
>>> source code for actors, I want to have a way to update all of the
>>> affected kar files.  This could be adding, removing or renaming ports
>>> and parameters or changing types and class names across all local kar
>>> files.  All of these operations appear to be supported as filters in
> the
>>> MoML parser.
>> Right, the ptolemy/moml/filter classes are the way to go here.
>> Let me know if the filters need any changes like accessor or reset
> methods.
>> You would need to write a script that unjars (I think) the .kar files,
> updates
>> the moml file and recreates the kar file.
>> 
>>>  The second task is doing a diff on MoML files without GUI
>>> or documentation related elements.  The automated layout in the GUI
> is
>>> great, but small changes in the workflow can result in big changes in
>>> the layout.   If I give a kar to someone and they update it, I want
> to
>>> be able to see what changes they made besides GUI placement or
>>> documentation.  This also appears to be supported by MoMLParser since
>>> there are filters to remove parts of the file.
>> Right, ptolemy/moml/filter/RemoveGraphicalClasses.java will remove
>> the gui xml.  That class has a main() method so that it can be run
> from
>> the command line.
>> 
>> To see if there are any changes, you could get part way there by
>> using RemoveGraphicalClasses, then, if necessary removing the
>> <configure> tags and then using tool to do a diff
>>> When I tried ptolemy.moml.MoMLParser with
>>> org.kepler.moml.filter.BackwardCompatibility.initialize(), I get a
> lot
>>> of unnecessary changes.  It would cause problems if I checked in the
>>> MoML files and ran 'svn blame'.  I noticed there is a file
>>> ptolemy-kepler-2.3/src/adm/bin/updatemoml which calls
>>> ptolemy-kepler-2.3/src/adm/bin/updateMoMLFile.tcl and runs diff to
> see
>>> if there were any differences.  I don't think that would work
> properly
>>> on a MoML from Kepler because there would be many changes.
>> 
>> I had forgotten about org.kepler.moml.filter.BackwardCompatibility,
> which
>> is found at
> ./core/src/org/kepler/moml/filter/BackwardCompatibility.java
>> 
>> I'm not sure I totally understand your question, but are you saying
> that the
>> MoML files included in the Kepler .kar files change when the
> Kepler-specific
>> BackwardCompatibility filter is run on them?
>> 
>> In Ptolemy II, we don't run the BackwardCompatibility filter on all
> the files
>> before a release.  In other words - the ptII tree includes files that
> would be
>> different if the BackwardCompatibility filter was run on them.  We
> partly
>> do this so that we can test the filters.  Another reason is so that we
> don't
>> have arbitrary changes in the change log.  Running the
> BackwardCompatibility
>> filter would make a number of changes which would make using 'svn
> blame'
>> less useful.
>> 
>> You could either run the BackwardCompatibility filters on all the moml
> files in the kar
>> files and provide an updated release to your users or you could try
> running
>> the BackwardCompatibility filter after the fact and comparing the
> output.
>> In other words, for any given model, you would need to run the filter
> on the
>> moml file from the Kepler distribution and on the updated version.
>> 
>> Kar files probably have a versioning system associated with them.
>> Ptolemy II updates the VersionAttribute in each file with the version
> of the PtolemyII
>> that is used.
>> There is also ptolemy/vergil/kernel/attributes/IDAttribute.java, which
> is not used
>> very much, but could be used to store version information.
>> 
>> In general, library management is a big issue.  Ptolemy II uses Actor
> Oriented Classes,
>> where a model is declared to be a class definition and there are
> derived instances
>> that are what is actually run.  There are rules about what can be
> updated or deleted.
>> 
>> _Christopher
>>> 
>>> _______________________________________________
>>> Kepler-dev mailing list
>>> Kepler-dev at kepler-project.org
>>> http://lists.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-dev
>> 
>> -- 
>> Christopher Brooks, PMP                       University of California
>> CHESS Executive Director                      US Mail: 337 Cory Hall
>> Programmer/Analyst CHESS/Ptolemy/Trust        Berkeley, CA 94720-1774
>> ph: 510.643.9841                                (Office: 545Q Cory)
>> home: (F-Tu) 707.665.0131 cell: 707.332.0670
>> 
>> _______________________________________________
>> Kepler-dev mailing list
>> Kepler-dev at kepler-project.org
>> http://lists.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-dev
> 