[kepler-dev] Meeting Reminder

David Welker david.v.welker at gmail.com
Thu Jun 4 13:27:49 PDT 2009 

We will be meeting at 2:00 pm using Marratech at meet.nceas.ucsb.edu in 
the Silver room.

David
> David and I have started putting together a little table classifying 
> the various kinds of information stored by Kepler (on the scientist's 
> computer) so that we can evaluate proposals for where to store and how 
> to organize this information.  Can we use and update this table during 
> our discussion at 2 PM?
>
> https://kepler-project.org/developers/teams/framework/classification-of-persistent-system-state 
>
>
> Cheers,
>
> Tim
>
> On Jun 3, 2009, at 10:44 AM, Chad Berkley wrote:
>
>> works for me.
>>
>> David Welker wrote:
>>> Matt, Tim, Chad, Aaron and other interested parties:
>>> Would you be available to meet on Thursday at 2 pm to discuss the 
>>> issues identified below?
>>> David
>>>>
>>>> Feel free to set up a meeting.
>>>>
>>>> David Welker wrote:
>>>>> A couple of points:
>>>>>
>>>>> (1) The question of whether what support we are going to have for 
>>>>> multiple installations of Kepler versus having one installation of 
>>>>> Kepler that is configurable has yet to be fully developed via 
>>>>> discussion. Your design assumes that we are supporting multiple 
>>>>> installations of Kepler on the same machine and that each of these 
>>>>> multiple installations have their own cache. Those are all 
>>>>> reasonable ideas, but these ideas have yet to be discussed and 
>>>>> agreed upon by Kepler management or the various development teams. 
>>>>> This is not a minor implementation decision, and it has 
>>>>> implications that need to be discussed.
>>>>>
>>>>> (2) The implications of the fact that Kepler could be installed in 
>>>>> a read-only area are far reaching. It certainly affects the build 
>>>>> system. Right now, modules.txt in the build-area is regularly 
>>>>> rewritten with every issuance of the change-to command. But, what 
>>>>> if Kepler is installed in a read-only area? It sounds like in that 
>>>>> case, we are going to have to think about what we can do if 
>>>>> modules.txt cannot be written. Like, for example, having a version 
>>>>> of modules.txt in the .kepler directory that is read from for 
>>>>> example. So, this is definitely not unrelated to the general work 
>>>>> you are doing with .kepler right now. Also, we need to think about 
>>>>> where kars are build. Originally, kars were built in the common 
>>>>> module. Then, they were built in the kepler.modules directory. 
>>>>> Then, they were build in the module where the came from. However, 
>>>>> if Kepler is installed in a read-only locations, none of these 
>>>>> solutions for where kars are built would be acceptable. We would 
>>>>> probably want to build kars in the .kepler or similar directory in 
>>>>> the user directory.
>>>>>
>>>>> (3) There are also implications here for the configuration system. 
>>>>> In general, the system default configurations could be in a 
>>>>> read-only area. Therefore, user changes to configuration need to 
>>>>> be stored in a different location than the original configuration 
>>>>> files, which would contain default configuration options.
>>>>>
>>>>> (4) For development purposes, I do not think we need to support 
>>>>> caches for multiple installations of Kepler. However, for your 
>>>>> testing purposes, it would be acceptable to have the build system 
>>>>> generate these sorts of files, just as the installer would. But, 
>>>>> before that, we need to think about what sort of support we are 
>>>>> going to provide for multiple installations of Kepler and only 
>>>>> then go forward with an implementation or strategy.
>>>>>
>>>>> (5) Kepler itself should not try to write files that in the normal 
>>>>> course of events should be written by the installer only. Instead, 
>>>>> the system should have a sensible default behavior that it 
>>>>> utilizes when such files are absent.
>>>>>
>>>>> (6) If you delete "only one file" in an installation of Kepler or 
>>>>> any other software, there is a good chance that this will in fact 
>>>>> mess up the software. Of course, the more robust we can make the 
>>>>> system, the better. However, there is no reason to think that 
>>>>> people will be likely to delete this file. Especially if it is 
>>>>> written to a less visible location in a subdirectory of the 
>>>>> build-area folder. Of course, I must admit that when I see a file 
>>>>> like this in kepler.modules, I am in fact tempted to delete it 
>>>>> because it doesn't feel like it belongs there.
>>>>>
>>>>> (7) In general, we should avoid storing non module files in 
>>>>> kepler.modules to reduce the probability of name clashes.
>>>>>
>>>>> (8) Also, if the purpose of the class is to uniquely identify a 
>>>>> particular installation of Kepler, I think a better name than 
>>>>> InstanceAuthNamespace should be selected. It is not at all clear 
>>>>> from the file names what the function of this file is -- although 
>>>>> one gets a sense of how it does whatever it does do (i.e. with 
>>>>> namespaces). I think in general, it would be better to select file 
>>>>> names thinking about "what" rather than "how," especially as 
>>>>> implementations are liable to change in the future.
>>>>>
>>>>> Anyway, I think we need to have a discussion about these issues 
>>>>> that includes Tim and Matt and other interested parties at a 
>>>>> minimum, and not just you, me, and Chad. We really need to think 
>>>>> about the implications of these decisions before moving forward.
>>>>>
>>>>> David
>>>>>
>>>>>>
>>>>>> I think it's ok to have that file in the project root, alongside 
>>>>>> all of the read-only module jars, the installer could initially 
>>>>>> generate the file, but we still need the runtime mechanism for 
>>>>>> development purposes.  It also increases the reliability quite a 
>>>>>> bit, imagine if I had to rerun my installer just because I 
>>>>>> deleted that one file...
>>>>>>
>>>>>> Aaron
>>>>>>
>>>>>>
>>>>>> David Welker wrote:
>>>>>>> If you are going to have a file that uniquely identifies an 
>>>>>>> installation and that file is going to be written to 
>>>>>>> kepler.modules (I think a better location would be in a 
>>>>>>> subfolder of the build-area folder where it not likely to be 
>>>>>>> seen very often), then it must be written by the installer and 
>>>>>>> not by any other process. In general, Kepler is likely to be 
>>>>>>> installed in a read-only area of disk, so such a file must be 
>>>>>>> generated only once at installation time.
>>>>>>>
>>>>>>> David
>>>>>>>>
>>>>>>>> I thought that would work (and actually moved it there 
>>>>>>>> yesterday) but the problem arises that if you have two 
>>>>>>>> different installations of Kepler on the same machine they both 
>>>>>>>> end up using the .kepler/InstanceAuthNamespace file and 
>>>>>>>> therefore the same cache.  Now this may be what you want but 
>>>>>>>> may not be what you want.  Say I have a kepler base 
>>>>>>>> configuration installation and a Kepler WRP installation, there 
>>>>>>>> are cases where I may want them to use the same cache or I may 
>>>>>>>> not want to use the same cache.  If the InstanceAuthNamespace 
>>>>>>>> is in .kepler then they both MUST use the same cache.  If the 
>>>>>>>> InstanceAuthNamespace file is stored at the project root then 
>>>>>>>> they can both use different caches or they can both use the 
>>>>>>>> same cache (by copying the InstanceAuthNamespace file to both 
>>>>>>>> project root directories).
>>>>>>>>
>>>>>>>> Aaron
>>>>>>>>
>>>>>>>> Chad Berkley wrote:
>>>>>>>>> Could it go in the root of the .kepler directory so the system 
>>>>>>>>> will know where to find it?  Or maybe in some 'common' 
>>>>>>>>> directory in .kepler that doesn't rely on the unique id system?
>>>>>>>>>
>>>>>>>>> chad
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Aaron Schultz wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Chad,
>>>>>>>>>>
>>>>>>>>>> The InstanceAuthNamespace file contains a java serialized 
>>>>>>>>>> string for the unique authority and namespace of any given 
>>>>>>>>>> kepler installation (which is either retrieved from a 
>>>>>>>>>> webservice or assigned via UUID).
>>>>>>>>>>
>>>>>>>>>> For each installation of kepler on a given machine there 
>>>>>>>>>> would be (actually there now is) a subdirectory in the 
>>>>>>>>>> .kepler directory that was named using the authority and 
>>>>>>>>>> namespace.
>>>>>>>>>> Each of these .kepler/instance directories would contain it's 
>>>>>>>>>> own cache as we discussed yesterday....
>>>>>>>>>>
>>>>>>>>>> So we need that one file to find all the other files for any 
>>>>>>>>>> given installation of kepler...
>>>>>>>>>>
>>>>>>>>>> Aaron
>>>>>>>>>>
>>>>>>>>>> Chad Berkley wrote:
>>>>>>>>>>> Hi Aaron,
>>>>>>>>>>>
>>>>>>>>>>> I've been seeing this file appear and wondering what it 
>>>>>>>>>>> was.  I think things like this should be written to .kepler 
>>>>>>>>>>> or some other user directory.  I agree with David that the 
>>>>>>>>>>> kepler project directory should probably not be written to 
>>>>>>>>>>> in general.  Maybe this type of file should be written to 
>>>>>>>>>>> the cache so that it could be programatically purged if and 
>>>>>>>>>>> when it needs to be.
>>>>>>>>>>>
>>>>>>>>>>> thanks,
>>>>>>>>>>> chad
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> David Welker wrote:
>>>>>>>>>>>> Hi Aaron,
>>>>>>>>>>>>
>>>>>>>>>>>> In general, you need to avoid writing anything to the 
>>>>>>>>>>>> kepler.modules project root directory. In the general case, 
>>>>>>>>>>>> this directory is likely to be stored in a read-only area 
>>>>>>>>>>>> of disk.
>>>>>>>>>>>>
>>>>>>>>>>>> David
>>>>>>>>>>>>>
>>>>>>>>>>>>> You will want to delete the InstanceAuthNamespace file in 
>>>>>>>>>>>>> your project root directory.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Otherwise you will have troubles getting proper object ids 
>>>>>>>>>>>>> for LSIDs.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Aaron
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Kepler-dev mailing list
>>>>>>>>>>>>> Kepler-dev at kepler-project.org
>>>>>>>>>>>>> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-dev 
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Kepler-dev mailing list
>>>>>>>>>>>> Kepler-dev at kepler-project.org
>>>>>>>>>>>> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-dev 
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>> _______________________________________________
>>> Kepler-dev mailing list
>>> Kepler-dev at kepler-project.org
>>> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-dev
>> _______________________________________________
>> Kepler-dev mailing list
>> Kepler-dev at kepler-project.org
>> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-dev
>
>

More information about the Kepler-dev mailing list