[seek-kr] Re: [seek-dev] data typing in ptolemy

Bertram Ludaescher ludaesch at sdsc.edu
Mon Oct 13 10:47:03 PDT 2003

Yes, I'd like to be on it.

Actually Dave Thau will be here during that time, so he'll join us as


>>>>> "CB" == Chad Berkley <berkley at nceas.ucsb.edu> writes:
CB> OK, lets do this call at 10:00 am PDT Wednesday (10/15).  Is that ok 
CB> with you rich?  Bertram, do you want to be in on this?  This will allow 
CB> me time to send an email out to pthackers to see what they think.
CB> I don't think the 3 way calling hack will work with this many 
CB> participants so I'll just setup a regular at&t conf call.  I'll email 
CB> later with the code and number.
CB> So far, the participants are:
CB> Shawn
CB> Rich
CB> Matt
CB> Chad
CB> Amy
CB> Anyone else want to join this little party?  The topic is: Extending the 
CB> ptolemy type system.
CB> chad
CB> Shawn Bowers wrote:
>> I am available (it is about 9:15).  Either way is fine with me.  
>> If we want to discuss semantic issues, I think it would be good to have 
>> Rich involved in the discussion. So perhaps we should punt to Wed.
>> Shawn
>> On Mon, 13 Oct 2003, Matt Jones wrote:
>>> I can do it now, and would like to be involved.  Or Wednesday is fine too.
>>> Matt
>>> Chad Berkley wrote:
>>>> Hey,
>>>> Do we still want to have a conference call this morning or should we 
>>>> continue this debate over email and have a call on wednesday when Rich 
>>>> is around?
>>>> chad
>>>> Shawn Bowers wrote:
>>>>> On Fri, 10 Oct 2003, Amy Sundermier wrote:
>>>>>> I'll quote some stuff from your email for clarification:
>>>>>> "I think what is desired in Chad's example with extending the set of
>>>>>> values allowed in a Ptolemy token type, is to augment the edges in the
>>>>>> lattice so that ExtendedDouble is a subclass of Complex, and Double is a
>>>>>> subclass of ExtendedDouble.
>>>>>> Then, if I say that my port accepts the ExtendedDouble type, and I
>>>>>> receive a Double token, no big deal.  If I say that my port accepts
>>>>>> Double, I am guaranteed (by Ptolemy type checking) that I will never
>>>>>> receive null, or missing, values."
>>>>>> I am having some trouble believing the second paragraph.  If you declare
>>>>>> a port to accept an ExtendedDouble, are you sure it will typecheck ok to
>>>>>> pass it a subclass Double?  I don't know the semantics of Ptolemy type
>>>>>> checking and substitutibility rules.  Upcasting is usually ok but
>>>>>> downcasting needs explicit support.  Do they have the concept of runtime
>>>>>> casting?  I thought that was missing from the architecture according to
>>>>>> the documentation.
>>>>> I think it should be okay since ExtendedDouble would be a supertype of 
>>>>> Double. If ExtendedDouble were a subctype, then we would be in trouble.
>>>>> Assuming ExtendedDouble is a supertype of Double, say I have an actor A2
>>>>> and it has one input port that has type ExtendedDouble and one output 
>>>>> port
>>>>> that returns Integer. Then, the "signature" for A2 might look like this
>>>>> (using java-like syntax):
>>>>> Integer A2(ExtendedDouble arg)
>>>>> Then, if I have another actor A1 that takes a String and returns a 
>>>>> Double, its signature looks like this:
>>>>> Double A1(String arg)
>>>>> And, I want to connect A2 and A1 such that the output of A1 is sent to 
>>>>> A2, I have:
>>>>> String a1 = "..."
>>>>> Double a2 = A1(a1)
>>>>> Integer r = A2(a2).
>>>>> This is an "upcast" as you mentioned, which is what Ptolemy allows.
>>>>> The second case would be if there was an actor A3 that output an 
>>>>> ExtendedDouble and an actor A4 that took as input a Double. Their 
>>>>> signatures would look as follows.
>>>>> ExtendedDouble A3(String arg)
>>>>> Integer A4(Double arg)
>>>>> Then, if we connected them, it should give an error:
>>>>> String a3 = "..."
>>>>> ExtendedDouble a4 = A3(a3)
>>>>> Integer r = A4(a4)
>>>>> This is an example of "downcasting," which as you mentioned isn't
>>>>> supported in Ptolemy.
>>>>> Shawn
>>>>>> So I gather you are proposing that ExtendedDouble is basically an
>>>>>> abstract class with no attributes or methods that override superclass
>>>>>> methods, just a placeholder in the class hierarchy?  Because if Double
>>>>>> is a subclass then it would inherit from ExtendedDouble any attributes
>>>>>> or methods that made it nullible.  Only the class would be different.
>>>>>> Thanks again for the explanation, Amy
>>>>>> -----Original Message-----
>>>>>> From:    Shawn Bowers [mailto:bowers at sdsc.edu]
>>>>>> Sent:    Fri 10/10/2003 12:47 PM
>>>>>> To:    Amy Sundermier
>>>>>> Cc:    Chad Berkley; Rich Williams
>>>>>> Subject:    RE: [seek-kr] Re: [seek-dev] data typing in ptolemy
>>>>>> On Fri, 10 Oct 2003, Amy Sundermier wrote:
>>>>>>> I don't see how changing the TypeLattice will do much of anything for
>>>>>>> you.  If you want to put new types in the system, you have to be 
>>>>>>> able to
>>>>>>> create instances of both the Ptolemy classes and the Ptolemy 
>>>>>>> objects.  Why would you want to subvert the class hierarchy for 
>>>>>>> existing types?
>>>>>> In Chad's original email, he suggested adding a new type to Ptolemy, 
>>>>>> which he called ExtendedDouble that could support null, or missing 
>>>>>> values. To add a new Ptolemy type, you *have* to change the type 
>>>>>> lattice. Otherwise, it isn't recognized by Ptolemy as a new type.
>>>>>> I am basing my information on Chapter 12 of the Ptolemy documentation on
>>>>>> the Type System.  As I understand it, an instance of TypeLattice defines
>>>>>> the type hierarchy for Ptolemy structured types. For example, in the
>>>>>> default type lattice object, there is an edge assignment:
>>>>>  _basicLattice.addEdge(BaseType.DOUBLE, BaseType.COMPLEX);
>>>>>> BaseType.DOUBLE and BaseType.COMPLEX are instances of DoubleType and
>>>>>> ComplexType. Each type class in Ptolemy has exactly one instance (i.e.,
>>>>>> they are singleton classes). The instance represents the actual type in
>>>>>> the Ptolemy environment, e.g., if I want to state that a port takes
>>>>>> tokens whose types are double (or subclasses of double), then I use
>>>>>> BaseType.DOUBLE to represent this.
>>>>>> The statement above asserts that the Double type is a subtype of the
>>>>>> Complex type.  The Ptolemy codebase itself doesn't reflect this type
>>>>>> system at all (Ptolemy *doesn't* use Java's reflection capabilities for
>>>>>> typing). For example, both classes DoubleType and ComplexType are
>>>>>> subclasses of BaseType, i.e., the class DoubleType is not a subclass of
>>>>>> ComplexType. The type lattice specifies exactly the desired 
>>>>>> relationship.
>>>>>> Also, since everything is an object, the type heirarchy can be used
>>>>>> directly in Ptolemy for both design-time and runtime type checking of
>>>>>> Ptolemy ports. Ptolemy supports multiple inheritance in its type system,
>>>>>> which isn't possible in Java unless interfaces are used (which might be
>>>>>> one reason for their architecture).
>>>>>> I don't think we want to subvert the class hierarchy, as you say 
>>>>>> above, we
>>>>>> just want to add a new token/token type to the system.  I believe 
>>>>>> that the
>>>>>> type system is meant to be extensible (that is what I got from the 
>>>>>> Chapter
>>>>>> above and from some of their other papers).  Basically, if I want to 
>>>>>> add a
>>>>>> class such as ExtendedDouble to Ptolemy's type system, I would subclass
>>>>>> BaseType, create an object of ExtendedDouble, and then add an edge to 
>>>>>> the
>>>>>> new object in the lattice.  I think what is desired in Chad's example 
>>>>>> with
>>>>>> extending the set of values allowed in a Ptolemy token type, is to 
>>>>>> augment
>>>>>> the edges in the lattice so that ExtendedDouble is a subclass of 
>>>>>> Complex,
>>>>>> and Double is a subclass of ExtendedDouble. 
>>>>>> Then, if I say that my port accepts the ExtendedDouble type, and I 
>>>>>> receive a Double token, no big deal.  If I say that my port accepts 
>>>>>> Double, I am guaranteed (by Ptolemy type checking) that I will never 
>>>>>> receive null, or missing, values.
>>>>>> In theory, doing this would not require augmenting any of Ptolemy's code
>>>>>> since we only need to pass around a different TypeLattice object, 
>>>>>> which I
>>>>>> believe is the intention in Ptolemy. However, for some reason they
>>>>>> hard-coded the reference to a predefined Typelattice object, so it 
>>>>>> doesn't quite work as they claim.
>>>>>> Does that make sense?
>>>>>> Shawn
>>>>>>> -----Original Message-----
>>>>>>> From:    Shawn Bowers [mailto:bowers at sdsc.edu]
>>>>>>> Sent:    Fri 10/10/2003 9:28 AM
>>>>>>> To:    Chad Berkley
>>>>>>> Cc:    Amy Sundermier; Rich Williams
>>>>>>> Subject:    Re: [seek-kr] Re: [seek-dev] data typing in ptolemy
>>>>>>> I am available on Monday, or pretty much any time next week for a 
>>>>>>> conference call. Just let me know what you decide.  I know the 
>>>>>>> phones here can do three-way calling, but would need to figure out 
>>>>>>> how to do it, which shouldn't be a problem (I think Ilkay has done 
>>>>>>> it before).
>>>>>>> We can talk a bit about semantic typing as well, and some ideas for 
>>>>>>> how to integrate semantic types in Ptolemy.
>>>>>>> Shawn
>>>>>>> On Fri, 10 Oct 2003, Chad Berkley wrote:
>>>>>>>> Hey Amy, Rich and Shawn (and others),
>>>>>>>> Thanks for all your comments.  Shawn and I had a pretty in-depth 
>>>>>>>> discussion about this yesterday afternoon on irc and we hashed out 
>>>>>>>> a few ideas.  It seems we all have slightly different ideas about 
>>>>>>>> what needs to be done so I'd like to propose having a conference 
>>>>>>>> call on this issue. would sometime monday work?  I'll propose 9:00 
>>>>>>>> am PDT.  If you have a problem with that time, propose a different 
>>>>>>>> one.  I can setup a vnc connection so we can have a whiteboard to 
>>>>>>>> transfer ideas on.  shawn or amy, can your phones do 3 way 
>>>>>>>> calling?  Mine does, do if one of your phones can do it, we can 
>>>>>>>> chain together all 4 of us without having to spend 
>>>>>>>> $.60/minute/participant on a conf. call.  If anyone else wants to 
>>>>>>>> engage in an extremely technical discussion about this issue, you 
>>>>>>>> are more than welcome to join in (Bertram?  Matt?).
>>>>>>>> Amy, to answer your question about the Ptolemy source, we do have 
>>>>>>>> the ptolemy source code, but I've been trying my best not to alter 
>>>>>>>> it. We've realized that we may have to do so, but I'd like to avoid 
>>>>>>>> that if it is at all possible.  This would be a relatively simple 
>>>>>>>> thing to fix if we just hacked the existing ptolemy source.  (if 
>>>>>>>> you want the source, by the way, it's freely available at 
>>>>>>>> ptolemy.eecs.berkeley.edu).
>>>>>>>> Rich, and Amy, you should look at the class 
>>>>>>>> ptolemy.data.type.TypeLattice.  Shawn was under the impression that 
>>>>>>>> we could extend that class to subvert the entire class hierarchy 
>>>>>>>> within ptolemy.  we kind of went back and forth about this for a 
>>>>>>>> while but it looks interresting.  I'm going to try just hacking it 
>>>>>>>> to see if changing the TypeLattice will indeed allow us to change 
>>>>>>>> the hierarchy without changing the actual java class hierarchy 
>>>>>>>> (something I'm still not convinced of....but Shawn found 
>>>>>>>> documentation to that reguard).
>>>>>>>> chad
>>>>>>>> Amy Sundermier wrote:
>>>>>>>>> Hi Chad (and mailing list recipients),
>>>>>>>>> Do you have the source for Ptolemy?  It seems to me that one way 
>>>>>>>>> to solve your "missing value" problem is to have a boolean in the 
>>>>>>>>> base class of the hierarchy (Token) called "nullValue" that all 
>>>>>>>>> subclasses inherit.  Any token could then declare itself null by 
>>>>>>>>> setting that boolean to true.  Your code would have to check "if 
>>>>>>>>> (token.isNullValue())" instead of "if (token == null)" but that's 
>>>>>>>>> pretty self-documenting. 
>>>>>>>>> As long as the Ptolemy infrastructure code is not using reflection 
>>>>>>>>> to set and get values and making assumptions about what it will 
>>>>>>>>> find, you should be able to modify a superclass in this way 
>>>>>>>>> without breaking their code.
>>>>>>>>> This seems like an obvious solution so I wonder if you've made a 
>>>>>>>>> policy decision not to modify the Ptolemy source? 
>>>>>>>>> I'd be interested in learning more about the semantic typing 
>>>>>>>>> issues alluded to by these emails.  Looking forward to meeting you 
>>>>>>>>> all.
>>>>>>>>> Amy Sundermier
>>>>>>>>> Arizona State University 
>>>>>>>>> -----Original Message-----
>>>>>>>>> From:    Chad Berkley [mailto:berkley at nceas.ucsb.edu]
>>>>>>>>> Sent:    Thu 10/9/2003 10:46 AM
>>>>>>>>> To:    seek-dev at ecoinformatics.org; seek-kr at ecoinformatics.org
>>>>>>>>> Cc:   
>>>>>>>>> Subject:    [seek-dev] data typing in ptolemy
>>>>>>>>> Hi,
>>>>>>>>> Matt and I had a conversation on IRC the other day that we thought 
>>>>>>>>> might be of interrest to those on these lists.
>>>>>>>>> Basically, I am now dealing with typing issues within ptolemy.  
>>>>>>>>> Problems arise when you get missing values in the data.  Ptolemy's 
>>>>>>>>> type heirarchy does not allow missing values in a data tokens so 
>>>>>>>>> Matt and I were talking about extending the ptolemy typing system 
>>>>>>>>> to allow missing values.  It occured to us that the typing system 
>>>>>>>>> will need to be extended to allow for semantic typing in the future.
>>>>>>>>> The type class hierarchy currently looks like the following:
>>>>>>>>> Token
>>>>>>>>> |
>>>>>>>>> --------------------------
>>>>>>>>> |                        |
>>>>>>>>> ScalarToken             AbstractConvertableToken
>>>>>>>>> |                               |
>>>>>>>>> ---------------...*              -----------------
>>>>>>>>> |            |                 |               |
>>>>>>>>> DoubleToken  IntToken         BooleanToken  StringToken
>>>>>>>>> *Note that ScalarToken also includes LongToken and ComplexToken.
>>>>>>>>> In addition to this Token hierarchy (Tokens are the means by which 
>>>>>>>>> you pass data between actors over ports) there is also a port 
>>>>>>>>> typing hierarchy implemented in the class BaseType.  BaseType is 
>>>>>>>>> the means by which you actually specify a port's type.  It looks 
>>>>>>>>> like this:
>>>>>>>>> BaseType
>>>>>>>>> |
>>>>>>>>> ---------------------------------------------------------....
>>>>>>>>> |             |             |          |          |
>>>>>>>>> BooleanType   ComplexType   GeneralType   IntType   DoubleType ....*
>>>>>>>>> * BaseType also includes EventType, LongType, NumericalType, 
>>>>>>>>> ObjectType, SCalarType, StringType, UnknownType, UnsignedByteType
>>>>>>>>> Basically, in order to extend this typing system, we must extend 
>>>>>>>>> both of these hierarchies since Tokens are the means by which data 
>>>>>>>>> is transfered between ports and BaseTypes are the means by which 
>>>>>>>>> you allow (or disallow) a port to accept different types of data.
>>>>>>>>> Extending the hierarchy
>>>>>>>>> -----------------------
>>>>>>>>> There are two different ways that I see to extend the hierarchy.  
>>>>>>>>> The first is to extend the base class Token with our own tree of 
>>>>>>>>> token types extending from the root of the tree.  This will 
>>>>>>>>> probably allow us the most flexibility in implementing types the 
>>>>>>>>> way we need to, however, the main drawback I see to doing this is 
>>>>>>>>> that we would not be able to use most existing actors because 
>>>>>>>>> their ports are typed according to the current hierarchy.  I think 
>>>>>>>>> that one fact pretty much eliminates this approach from the options.
>>>>>>>>> The second approach I see is to extend each of the leaf token 
>>>>>>>>> types. For example, extend DoubleToken to ExtendedDoubleToken and 
>>>>>>>>> add our additional functionality there.  This keeps our type 
>>>>>>>>> system within the bounds of the current ptolemy hierarchy but 
>>>>>>>>> limits our flexibility in extension.  we are basically limited to 
>>>>>>>>> the hierarchy that already exists.  It is still unclear to me what 
>>>>>>>>> the affects of doing this will be on existing actors.  For 
>>>>>>>>> instance, if we extend DoubleToken to allow missing values, and an 
>>>>>>>>> actor with a port of BaseType.DoubleType gets an 
>>>>>>>>> ExtendedDoubleToken, it would still need to be able to handle 
>>>>>>>>> whatever value we assign as a missing value code.  This is 
>>>>>>>>> problematic, because we are then restricted to using an actual 
>>>>>>>>> double value as a missing value code (i.e. -999.999) which we've 
>>>>>>>>> always maintained was bad data practice.  This could also cause 
>>>>>>>>> problems because the actor cannot differentiate -999.999 from a 
>>>>>>>>> normal value and will operate on it normally.
>>>>>>>>> This same problem comes up (but to a lesser extent) when you think 
>>>>>>>>> about extending this system for semantics.  What does an existing 
>>>>>>>>> actor do with semantic information stored in the token?  It can 
>>>>>>>>> ignore it, but that may be detrimental to the analysis.
>>>>>>>>> Another possible option
>>>>>>>>> -----------------------
>>>>>>>>> The other possible solution to the missing value problem is to 
>>>>>>>>> simply not send any data over the port when a missing value is 
>>>>>>>>> encountered.  I have modified the EML ingestion actor to 
>>>>>>>>> dynamically create one typed port for each attribute in the data 
>>>>>>>>> package.  These ports can then be hooked up to other actors.  The 
>>>>>>>>> data is sent asyncronously and depends on the receiving ports to 
>>>>>>>>> queue the data until all the input data is present to run the 
>>>>>>>>> analysis.
>>>>>>>>> If I simply do not send a token when a missing value comes up, I 
>>>>>>>>> forsee major timing problems.  For instance, port A and port B are 
>>>>>>>>> mapped to input ports X and Y (res.) of a plotter.  port A sends a 
>>>>>>>>> token to X, then B gets a missing value.  It sends nothing.  The 
>>>>>>>>> plotter is then waiting for its second input.  the next record is 
>>>>>>>>> iterated into.  port A sends another token to X.  This causes an 
>>>>>>>>> exception.  The other scenario is, on the second iteration, A is a 
>>>>>>>>> missing value but B is not.  Then we are plotting two values from 
>>>>>>>>> different records when Y recieves data from B in the second 
>>>>>>>>> record.  This would be a nightmare to deal with given the current 
>>>>>>>>> directors.
>>>>>>>>> So, does anyone see something that I'm missing here?  What are the 
>>>>>>>>> needs of the semantic typing going to be as far as ptolemy goes?  
>>>>>>>>> Anyone have a better solution than the three that I've layed out?  
>>>>>>>>> This is a complex issue that I need to deal with before I can 
>>>>>>>>> continue moving forward with AMS.  I don't want to do anything 
>>>>>>>>> that will hinder the future semantic extensions of ptolemy and 
>>>>>>>>> this is just too much of a basic infrastructure item to try to 
>>>>>>>>> hack.  If anyone want to have an IRC chat about this, I'm on #seek.
>>>>>>>>> chad
>> _______________________________________________
>> seek-dev mailing list
>> seek-dev at ecoinformatics.org
>> http://www.ecoinformatics.org/mailman/listinfo/seek-dev
CB> -- 
CB> -----------------------
CB> Chad Berkley
CB> National Center for
CB> Ecological Analysis
CB> and Synthesis (NCEAS)
CB> berkley at nceas.ucsb.edu
CB> -----------------------
CB> _______________________________________________
CB> seek-dev mailing list
CB> seek-dev at ecoinformatics.org
CB> http://www.ecoinformatics.org/mailman/listinfo/seek-dev

More information about the Seek-dev mailing list