[kepler-dev] [Bug 4702] KAR to module conversion utility

Thu Feb 11 04:00:14 PST 2010

http://bugzilla.ecoinformatics.org/show_bug.cgi?id=4702

--- Comment #3 from David Welker <welker4kepler at gmail.com> 2010-02-11 04:00:13 PST ---
Do any actual kar files from 1.0 actually contain jars? If so, is the number
sufficient to justify building a utility to do this conversion automatically
rather than having people just create modules using the tools already in place?

If the goal is backwards compatibility such that a 1.0 kar that actually uses
the ability of 1.0 kars to store jars actually loads in 2.0 has significant
implications. The main problem is determining where this automatically-created
module should be inserted into modules.txt. Perhaps it should be inserted at
the highest priority position. But this does risk havoc in case the jar
provided by the kar contains incompatible overrides, contains unstable code, or
contains semantically incompatible code with the code that already exists in
the modules specified in modules.txt. In effect, automatic conversion is
another name for distributing suites to people that have not been tested. The
result would be that Kepler crashes or behaves in an unpredictable manner in
some cases. (Although, perhaps such overrides would be rare.)

Should a module automatically created from a 1.0 KAR that contains jars be
inserted into modules.txt automatically? Should any module? Does it violate
expectations to modify modules.txt without user or developer control? Up to
now, modules.txt has been under the control of developers with the option for
advanced users to tweak it using the Module Manager. To make these sorts of 1.0
KARS "automatically" compatible with 2.0 would require modifying modules.txt.
At the very least, we would need to prompt the user asking them if they really
want to do this. Keep in mind also that if such a 1.0 kar were loaded, this
would require a restart of Kepler.

My own preference is for any 1.0 KARS that do contain jars to be converted to
modules manually. I am not even sure any such 1.0 KARS exist. But, if they do,
I believe that manual conversion is best, since that way there is an
opportunity to test them as part of a working suite before distributing them.
Automatic conversion and modification of modules.txt is basically a method of
distributing untested software. It is true that the Module Manager allows the
mixing and matching of modules to create unique modules.txt files that may or
may not work. But this is an advanced feature which will be used only by
advanced users/developers who have the time and skills to determine an
appropriate ordering for modules.txt or determine the source of inevitable
problems when they do occur OR by more casual users who are following the
advice of more advanced users/developers who have already tested a particular
combination of modules and found that it works. Basically, if you are playing
around with modules.txt, you should not be surprised if Kepler crashes or
exhibits other unexpected behavior. Therefore, playing around with modules.txt
is not something that would be recommended for users who are not prepared to
deal with unpredictability and frustration. Because an automatic conversion
utility that allowed Kepler 2.0 to read 1.0 KARS that contain jars would modify
modules.txt (in essence, as soon as the 1.0 KAR is loaded, Kepler becomes
essentially "untested" software), I do not think that such a utility is a good
idea.

On a related note, I think the most common use case for the Module Manager for
everyday users will be to simply load a suite. A suite represents an ordering
of modules in modules.txt that has been tested. A suite, therefore has much
more reliability than other ad hoc ordering of modules.txt that are enabled by
the Module Manager or that would be created by automatic mechanisms, as
proposed here.

I think moving forward post 2.0, we might think of situations where automatic
insertion of modules into modules.txt might be considered "safer" (although it
will never be entirely safe, since a failure in the execution of code in one
module can always bring down the JVM and, therefore, the rest of Kepler along
with it.) It may even be useful in the future to even think about the
situations where certain modules could be added to the mix without requiring a
restart. (Such modules would need to be known to contain no overrides, to
address a discrete area of functionality that is non-foundational, and be
semantically compatible with existing modules - i.e would need to conform to
the principles that enable hot deployment.) 

For now, the following points must be kept in mind:

(1) Any changes to modules.txt could destabilize the system. Therefore, such
changes must not be invoked casually.

(2) A different ordering of modules.txt is a different software product.
Therefore, any automatic changes to modules.txt is, in effect, distributing new
and untested software. Users accessing features that make automatic changes to
modules.txt (if we decide to implement any such features) should not be
surprised if Kepler crashes or exhibits unexpected behavior.

Even if in the future, we were to research and make use of hot deployment to
the greatest extent possible, these points would still be true. At best, hot
deployment would be a feature that is more convenient for developers, as it
would allow them to change the mix of modules without restarting Kepler. But,
since a newly loaded module is capable of causing the JVM to crash, even hot
deployment is not going to guarantee stability for our users. There is no and
never can be any entirely safe way to change modules.txt, except by having
developers actually test and certify particular orderings. This is precisely
what makes a suite a suite. A suite is an ordering of modules.txt that has been
purposely developed by a particular group or individual and tested. In that
sense, a suite is as much or more a social construct as it is a technical
construct.

-- 
Configure bugmail: http://bugzilla.ecoinformatics.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.