[kepler-dev] Proposal: Kepler should work off a snapshot or release of Ptolemy instead of the trunk of Ptolemy

[David Welker]

Hi all,

I wanted to see how people felt, including people working on Ptolemy, 
about the following proposal. Instead of having the trunk of Kepler run 
off the trunk of Ptolemy, instead we should have it run off a snapshot 
or release of Ptolemy. Then, according to a set schedule, Kepler could 
synchronize the trunk of Kepler with the trunk of Ptolemy. This way, we 
can remain highly coordinated with Ptolemy and the Ptolemy community, 
but not be impacted by minor (yet important) changes.

Overall, I am really impressed with how stable the trunk of Ptolemy is. 
Compilation problems and problems running are usually resolved quickly. 
However, sometimes, things are not so easy and it isn't anyone's fault. 
There are just too many moving parts.

I think today was a case in point. The build continues to evolve, as it 
must. Modules that depend on the build, such as loader and 
module-manager, also continue to evolve. Ptolemy continues to evolve. 
All of this evolution is absolutely necessary. But, it also conflicts 
with another need we have, and that is a stable foundation upon which 
developers can work.

Here is the story of what happened today:

Ptolemy decided to move some configuration files -- gtEffigyFactory.xml 
and gtTableauFactory.xml -- from one location to another. Specifically, 
from ptolemy/configs/gt/ to ptolemy/configs/. This strikes me as a 
perfectly sensible change. But, it also was pretty expensive in terms of 
developer time. Unfortunately, our configuration.xml file in Kepler 
referred to the old location of these files instead of the new locations.

It took me at least 3 hours to find this problem. Not least of which, 
because these changes in Ptolemy corresponded with significant changes I 
have been making to loader and module-manger, not to mention the build. 
When so many things are changing at once, it is challenging to isolate 
an error like this. I spent a significant amount of time looking for how 
the build system or loader was somehow causing this error, when in fact 
it was a change in ptolemy without a corresponding change to a 
configuration file that was the real culprit.

Of course, the amount of time it took me to find this error is only a 
small part of the cost. I think this change happened sometime yesterday. 
I didn't run into this error until I updated. It basically broke Tim's 
build yesterday, and at some point he had to stop working. I know it 
also broke Aaron's build too. So, we have what turns out to be a rather 
minor (yet important) change sucking up multiple hours for multiple 
developers. It seems pretty clear that the decision to synchronize the 
trunk of Kepler with the trunk of Ptolemy in real time is pretty 
expensive. So, we really have to scrutinize this decision.

Imagine a world where, instead of working off the trunk of Ptolemy, we 
work off a snapshot that is made on a regular basis, say once a month. 
When we decide to migrate the trunk of Kepler to work with the trunk of 
Ptolemy on a monthly basis, if we found that Kepler suddenly stopped 
working, we would immediately know where the problem was. There must 
have been a change in Ptolemy that made it incompatible with Kepler. So, 
this sort of problem would take much less time to hunt down, simply out 
of virtue that the set of potential causes would be much smaller. 
Secondly, and more significantly, the costs of such incompatibilities 
could be absorbed by one expert who would be well equipped to figure out 
how compatibility was breaking down, rather than subset of the entire 
Kepler community that chooses to work off the trunk of Kepler. Only when 
compatibility was ensured would a new snapshot that reflected the latest 
trunk of Ptolemy be used.

I don't really see any downsides to this proposal. We would still be 
working in a very collaborative manner with Ptolemy. Only, the costs of 
such collaboration would be much lower.

Does anyone else have any thoughts on this proposal?

David