[kepler-dev] Meeting about .kepler and related issues

[Chad Berkley]

works for me.

David Welker wrote:
> Matt, Tim, Chad, Aaron and other interested parties:
> 
> Would you be available to meet on Thursday at 2 pm to discuss the issues 
> identified below?
> 
> David
>>
>> Feel free to set up a meeting.
>>
>> David Welker wrote:
>>> A couple of points:
>>>
>>> (1) The question of whether what support we are going to have for 
>>> multiple installations of Kepler versus having one installation of 
>>> Kepler that is configurable has yet to be fully developed via 
>>> discussion. Your design assumes that we are supporting multiple 
>>> installations of Kepler on the same machine and that each of these 
>>> multiple installations have their own cache. Those are all reasonable 
>>> ideas, but these ideas have yet to be discussed and agreed upon by 
>>> Kepler management or the various development teams. This is not a 
>>> minor implementation decision, and it has implications that need to 
>>> be discussed.
>>>
>>> (2) The implications of the fact that Kepler could be installed in a 
>>> read-only area are far reaching. It certainly affects the build 
>>> system. Right now, modules.txt in the build-area is regularly 
>>> rewritten with every issuance of the change-to command. But, what if 
>>> Kepler is installed in a read-only area? It sounds like in that case, 
>>> we are going to have to think about what we can do if modules.txt 
>>> cannot be written. Like, for example, having a version of modules.txt 
>>> in the .kepler directory that is read from for example. So, this is 
>>> definitely not unrelated to the general work you are doing with 
>>> .kepler right now. Also, we need to think about where kars are build. 
>>> Originally, kars were built in the common module. Then, they were 
>>> built in the kepler.modules directory. Then, they were build in the 
>>> module where the came from. However, if Kepler is installed in a 
>>> read-only locations, none of these solutions for where kars are built 
>>> would be acceptable. We would probably want to build kars in the 
>>> .kepler or similar directory in the user directory.
>>>
>>> (3) There are also implications here for the configuration system. In 
>>> general, the system default configurations could be in a read-only 
>>> area. Therefore, user changes to configuration need to be stored in a 
>>> different location than the original configuration files, which would 
>>> contain default configuration options.
>>>
>>> (4) For development purposes, I do not think we need to support 
>>> caches for multiple installations of Kepler. However, for your 
>>> testing purposes, it would be acceptable to have the build system 
>>> generate these sorts of files, just as the installer would. But, 
>>> before that, we need to think about what sort of support we are going 
>>> to provide for multiple installations of Kepler and only then go 
>>> forward with an implementation or strategy.
>>>
>>> (5) Kepler itself should not try to write files that in the normal 
>>> course of events should be written by the installer only. Instead, 
>>> the system should have a sensible default behavior that it utilizes 
>>> when such files are absent.
>>>
>>> (6) If you delete "only one file" in an installation of Kepler or any 
>>> other software, there is a good chance that this will in fact mess up 
>>> the software. Of course, the more robust we can make the system, the 
>>> better. However, there is no reason to think that people will be 
>>> likely to delete this file. Especially if it is written to a less 
>>> visible location in a subdirectory of the build-area folder. Of 
>>> course, I must admit that when I see a file like this in 
>>> kepler.modules, I am in fact tempted to delete it because it doesn't 
>>> feel like it belongs there.
>>>
>>> (7) In general, we should avoid storing non module files in 
>>> kepler.modules to reduce the probability of name clashes.
>>>
>>> (8) Also, if the purpose of the class is to uniquely identify a 
>>> particular installation of Kepler, I think a better name than 
>>> InstanceAuthNamespace should be selected. It is not at all clear from 
>>> the file names what the function of this file is -- although one gets 
>>> a sense of how it does whatever it does do (i.e. with namespaces). I 
>>> think in general, it would be better to select file names thinking 
>>> about "what" rather than "how," especially as implementations are 
>>> liable to change in the future.
>>>
>>> Anyway, I think we need to have a discussion about these issues that 
>>> includes Tim and Matt and other interested parties at a minimum, and 
>>> not just you, me, and Chad. We really need to think about the 
>>> implications of these decisions before moving forward.
>>>
>>> David
>>>
>>>>
>>>> I think it's ok to have that file in the project root, alongside all 
>>>> of the read-only module jars, the installer could initially generate 
>>>> the file, but we still need the runtime mechanism for development 
>>>> purposes.  It also increases the reliability quite a bit, imagine if 
>>>> I had to rerun my installer just because I deleted that one file...
>>>>
>>>> Aaron
>>>>
>>>>
>>>> David Welker wrote:
>>>>> If you are going to have a file that uniquely identifies an 
>>>>> installation and that file is going to be written to kepler.modules 
>>>>> (I think a better location would be in a subfolder of the 
>>>>> build-area folder where it not likely to be seen very often), then 
>>>>> it must be written by the installer and not by any other process. 
>>>>> In general, Kepler is likely to be installed in a read-only area of 
>>>>> disk, so such a file must be generated only once at installation time.
>>>>>
>>>>> David
>>>>>>
>>>>>> I thought that would work (and actually moved it there yesterday) 
>>>>>> but the problem arises that if you have two different 
>>>>>> installations of Kepler on the same machine they both end up using 
>>>>>> the .kepler/InstanceAuthNamespace file and therefore the same 
>>>>>> cache.  Now this may be what you want but may not be what you 
>>>>>> want.  Say I have a kepler base configuration installation and a 
>>>>>> Kepler WRP installation, there are cases where I may want them to 
>>>>>> use the same cache or I may not want to use the same cache.  If 
>>>>>> the InstanceAuthNamespace is in .kepler then they both MUST use 
>>>>>> the same cache.  If the InstanceAuthNamespace file is stored at 
>>>>>> the project root then they can both use different caches or they 
>>>>>> can both use the same cache (by copying the InstanceAuthNamespace 
>>>>>> file to both project root directories).
>>>>>>
>>>>>> Aaron
>>>>>>
>>>>>> Chad Berkley wrote:
>>>>>>> Could it go in the root of the .kepler directory so the system 
>>>>>>> will know where to find it?  Or maybe in some 'common' directory 
>>>>>>> in .kepler that doesn't rely on the unique id system?
>>>>>>>
>>>>>>> chad
>>>>>>>
>>>>>>>
>>>>>>> Aaron Schultz wrote:
>>>>>>>>
>>>>>>>> Hi Chad,
>>>>>>>>
>>>>>>>> The InstanceAuthNamespace file contains a java serialized string 
>>>>>>>> for the unique authority and namespace of any given kepler 
>>>>>>>> installation (which is either retrieved from a webservice or 
>>>>>>>> assigned via UUID).
>>>>>>>>
>>>>>>>> For each installation of kepler on a given machine there would 
>>>>>>>> be (actually there now is) a subdirectory in the .kepler 
>>>>>>>> directory that was named using the authority and namespace.
>>>>>>>> Each of these .kepler/instance directories would contain it's 
>>>>>>>> own cache as we discussed yesterday....
>>>>>>>>
>>>>>>>> So we need that one file to find all the other files for any 
>>>>>>>> given installation of kepler...
>>>>>>>>
>>>>>>>> Aaron
>>>>>>>>
>>>>>>>> Chad Berkley wrote:
>>>>>>>>> Hi Aaron,
>>>>>>>>>
>>>>>>>>> I've been seeing this file appear and wondering what it was.  I 
>>>>>>>>> think things like this should be written to .kepler or some 
>>>>>>>>> other user directory.  I agree with David that the kepler 
>>>>>>>>> project directory should probably not be written to in 
>>>>>>>>> general.  Maybe this type of file should be written to the 
>>>>>>>>> cache so that it could be programatically purged if and when it 
>>>>>>>>> needs to be.
>>>>>>>>>
>>>>>>>>> thanks,
>>>>>>>>> chad
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> David Welker wrote:
>>>>>>>>>> Hi Aaron,
>>>>>>>>>>
>>>>>>>>>> In general, you need to avoid writing anything to the 
>>>>>>>>>> kepler.modules project root directory. In the general case, 
>>>>>>>>>> this directory is likely to be stored in a read-only area of 
>>>>>>>>>> disk.
>>>>>>>>>>
>>>>>>>>>> David
>>>>>>>>>>>
>>>>>>>>>>> You will want to delete the InstanceAuthNamespace file in 
>>>>>>>>>>> your project root directory.
>>>>>>>>>>>
>>>>>>>>>>> Otherwise you will have troubles getting proper object ids 
>>>>>>>>>>> for LSIDs.
>>>>>>>>>>>
>>>>>>>>>>> Aaron
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Kepler-dev mailing list
>>>>>>>>>>> Kepler-dev at kepler-project.org
>>>>>>>>>>> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-dev
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Kepler-dev mailing list
>>>>>>>>>> Kepler-dev at kepler-project.org
>>>>>>>>>> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-dev
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
> 
> _______________________________________________
> Kepler-dev mailing list
> Kepler-dev at kepler-project.org
> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-dev
>