[kepler-dev] Meeting about .kepler and related issues

Tue Jun 2 16:52:54 PDT 2009

Matt, Tim, Chad, Aaron and other interested parties:

Would you be available to meet on Thursday at 2 pm to discuss the issues 
identified below?

David
>
> Feel free to set up a meeting.
>
> David Welker wrote:
>> A couple of points:
>>
>> (1) The question of whether what support we are going to have for 
>> multiple installations of Kepler versus having one installation of 
>> Kepler that is configurable has yet to be fully developed via 
>> discussion. Your design assumes that we are supporting multiple 
>> installations of Kepler on the same machine and that each of these 
>> multiple installations have their own cache. Those are all reasonable 
>> ideas, but these ideas have yet to be discussed and agreed upon by 
>> Kepler management or the various development teams. This is not a 
>> minor implementation decision, and it has implications that need to 
>> be discussed.
>>
>> (2) The implications of the fact that Kepler could be installed in a 
>> read-only area are far reaching. It certainly affects the build 
>> system. Right now, modules.txt in the build-area is regularly 
>> rewritten with every issuance of the change-to command. But, what if 
>> Kepler is installed in a read-only area? It sounds like in that case, 
>> we are going to have to think about what we can do if modules.txt 
>> cannot be written. Like, for example, having a version of modules.txt 
>> in the .kepler directory that is read from for example. So, this is 
>> definitely not unrelated to the general work you are doing with 
>> .kepler right now. Also, we need to think about where kars are build. 
>> Originally, kars were built in the common module. Then, they were 
>> built in the kepler.modules directory. Then, they were build in the 
>> module where the came from. However, if Kepler is installed in a 
>> read-only locations, none of these solutions for where kars are built 
>> would be acceptable. We would probably want to build kars in the 
>> .kepler or similar directory in the user directory.
>>
>> (3) There are also implications here for the configuration system. In 
>> general, the system default configurations could be in a read-only 
>> area. Therefore, user changes to configuration need to be stored in a 
>> different location than the original configuration files, which would 
>> contain default configuration options.
>>
>> (4) For development purposes, I do not think we need to support 
>> caches for multiple installations of Kepler. However, for your 
>> testing purposes, it would be acceptable to have the build system 
>> generate these sorts of files, just as the installer would. But, 
>> before that, we need to think about what sort of support we are going 
>> to provide for multiple installations of Kepler and only then go 
>> forward with an implementation or strategy.
>>
>> (5) Kepler itself should not try to write files that in the normal 
>> course of events should be written by the installer only. Instead, 
>> the system should have a sensible default behavior that it utilizes 
>> when such files are absent.
>>
>> (6) If you delete "only one file" in an installation of Kepler or any 
>> other software, there is a good chance that this will in fact mess up 
>> the software. Of course, the more robust we can make the system, the 
>> better. However, there is no reason to think that people will be 
>> likely to delete this file. Especially if it is written to a less 
>> visible location in a subdirectory of the build-area folder. Of 
>> course, I must admit that when I see a file like this in 
>> kepler.modules, I am in fact tempted to delete it because it doesn't 
>> feel like it belongs there.
>>
>> (7) In general, we should avoid storing non module files in 
>> kepler.modules to reduce the probability of name clashes.
>>
>> (8) Also, if the purpose of the class is to uniquely identify a 
>> particular installation of Kepler, I think a better name than 
>> InstanceAuthNamespace should be selected. It is not at all clear from 
>> the file names what the function of this file is -- although one gets 
>> a sense of how it does whatever it does do (i.e. with namespaces). I 
>> think in general, it would be better to select file names thinking 
>> about "what" rather than "how," especially as implementations are 
>> liable to change in the future.
>>
>> Anyway, I think we need to have a discussion about these issues that 
>> includes Tim and Matt and other interested parties at a minimum, and 
>> not just you, me, and Chad. We really need to think about the 
>> implications of these decisions before moving forward.
>>
>> David
>>
>>>
>>> I think it's ok to have that file in the project root, alongside all 
>>> of the read-only module jars, the installer could initially generate 
>>> the file, but we still need the runtime mechanism for development 
>>> purposes.  It also increases the reliability quite a bit, imagine if 
>>> I had to rerun my installer just because I deleted that one file...
>>>
>>> Aaron
>>>
>>>
>>> David Welker wrote:
>>>> If you are going to have a file that uniquely identifies an 
>>>> installation and that file is going to be written to kepler.modules 
>>>> (I think a better location would be in a subfolder of the 
>>>> build-area folder where it not likely to be seen very often), then 
>>>> it must be written by the installer and not by any other process. 
>>>> In general, Kepler is likely to be installed in a read-only area of 
>>>> disk, so such a file must be generated only once at installation time.
>>>>
>>>> David
>>>>>
>>>>> I thought that would work (and actually moved it there yesterday) 
>>>>> but the problem arises that if you have two different 
>>>>> installations of Kepler on the same machine they both end up using 
>>>>> the .kepler/InstanceAuthNamespace file and therefore the same 
>>>>> cache.  Now this may be what you want but may not be what you 
>>>>> want.  Say I have a kepler base configuration installation and a 
>>>>> Kepler WRP installation, there are cases where I may want them to 
>>>>> use the same cache or I may not want to use the same cache.  If 
>>>>> the InstanceAuthNamespace is in .kepler then they both MUST use 
>>>>> the same cache.  If the InstanceAuthNamespace file is stored at 
>>>>> the project root then they can both use different caches or they 
>>>>> can both use the same cache (by copying the InstanceAuthNamespace 
>>>>> file to both project root directories).
>>>>>
>>>>> Aaron
>>>>>
>>>>> Chad Berkley wrote:
>>>>>> Could it go in the root of the .kepler directory so the system 
>>>>>> will know where to find it?  Or maybe in some 'common' directory 
>>>>>> in .kepler that doesn't rely on the unique id system?
>>>>>>
>>>>>> chad
>>>>>>
>>>>>>
>>>>>> Aaron Schultz wrote:
>>>>>>>
>>>>>>> Hi Chad,
>>>>>>>
>>>>>>> The InstanceAuthNamespace file contains a java serialized string 
>>>>>>> for the unique authority and namespace of any given kepler 
>>>>>>> installation (which is either retrieved from a webservice or 
>>>>>>> assigned via UUID).
>>>>>>>
>>>>>>> For each installation of kepler on a given machine there would 
>>>>>>> be (actually there now is) a subdirectory in the .kepler 
>>>>>>> directory that was named using the authority and namespace.
>>>>>>> Each of these .kepler/instance directories would contain it's 
>>>>>>> own cache as we discussed yesterday....
>>>>>>>
>>>>>>> So we need that one file to find all the other files for any 
>>>>>>> given installation of kepler...
>>>>>>>
>>>>>>> Aaron
>>>>>>>
>>>>>>> Chad Berkley wrote:
>>>>>>>> Hi Aaron,
>>>>>>>>
>>>>>>>> I've been seeing this file appear and wondering what it was.  I 
>>>>>>>> think things like this should be written to .kepler or some 
>>>>>>>> other user directory.  I agree with David that the kepler 
>>>>>>>> project directory should probably not be written to in 
>>>>>>>> general.  Maybe this type of file should be written to the 
>>>>>>>> cache so that it could be programatically purged if and when it 
>>>>>>>> needs to be.
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>> chad
>>>>>>>>
>>>>>>>>
>>>>>>>> David Welker wrote:
>>>>>>>>> Hi Aaron,
>>>>>>>>>
>>>>>>>>> In general, you need to avoid writing anything to the 
>>>>>>>>> kepler.modules project root directory. In the general case, 
>>>>>>>>> this directory is likely to be stored in a read-only area of 
>>>>>>>>> disk.
>>>>>>>>>
>>>>>>>>> David
>>>>>>>>>>
>>>>>>>>>> You will want to delete the InstanceAuthNamespace file in 
>>>>>>>>>> your project root directory.
>>>>>>>>>>
>>>>>>>>>> Otherwise you will have troubles getting proper object ids 
>>>>>>>>>> for LSIDs.
>>>>>>>>>>
>>>>>>>>>> Aaron
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Kepler-dev mailing list
>>>>>>>>>> Kepler-dev at kepler-project.org
>>>>>>>>>> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-dev
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Kepler-dev mailing list
>>>>>>>>> Kepler-dev at kepler-project.org
>>>>>>>>> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-dev
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>