[seek-kr] Re: [seek-dev] data typing in ptolemy
Bertram Ludaescher
ludaesch at sdsc.edu
Mon Oct 13 10:47:03 PDT 2003
Yes, I'd like to be on it.
Actually Dave Thau will be here during that time, so he'll join us as
well.
Bertram
>>>>> "CB" == Chad Berkley <berkley at nceas.ucsb.edu> writes:
CB>
CB> OK, lets do this call at 10:00 am PDT Wednesday (10/15). Is that ok
CB> with you rich? Bertram, do you want to be in on this? This will allow
CB> me time to send an email out to pthackers to see what they think.
CB>
CB> I don't think the 3 way calling hack will work with this many
CB> participants so I'll just setup a regular at&t conf call. I'll email
CB> later with the code and number.
CB>
CB> So far, the participants are:
CB> Shawn
CB> Rich
CB> Matt
CB> Chad
CB> Amy
CB>
CB> Anyone else want to join this little party? The topic is: Extending the
CB> ptolemy type system.
CB>
CB> chad
CB>
CB> Shawn Bowers wrote:
>> I am available (it is about 9:15). Either way is fine with me.
>> If we want to discuss semantic issues, I think it would be good to have
>> Rich involved in the discussion. So perhaps we should punt to Wed.
>>
>>
>> Shawn
>>
>>
>>
>> On Mon, 13 Oct 2003, Matt Jones wrote:
>>
>>
>>> I can do it now, and would like to be involved. Or Wednesday is fine too.
>>>
>>> Matt
>>>
>>> Chad Berkley wrote:
>>>
>>>> Hey,
>>>>
>>>> Do we still want to have a conference call this morning or should we
>>>> continue this debate over email and have a call on wednesday when Rich
>>>> is around?
>>>>
>>>> chad
>>>>
>>>> Shawn Bowers wrote:
>>>>
>>>>
>>>>> On Fri, 10 Oct 2003, Amy Sundermier wrote:
>>>>>
>>>>>
>>>>>
>>>>>> I'll quote some stuff from your email for clarification:
>>>>>>
>>>>>> "I think what is desired in Chad's example with extending the set of
>>>>>> values allowed in a Ptolemy token type, is to augment the edges in the
>>>>>> lattice so that ExtendedDouble is a subclass of Complex, and Double is a
>>>>>> subclass of ExtendedDouble.
>>>>>>
>>>>>> Then, if I say that my port accepts the ExtendedDouble type, and I
>>>>>> receive a Double token, no big deal. If I say that my port accepts
>>>>>> Double, I am guaranteed (by Ptolemy type checking) that I will never
>>>>>> receive null, or missing, values."
>>>>>>
>>>>>> I am having some trouble believing the second paragraph. If you declare
>>>>>> a port to accept an ExtendedDouble, are you sure it will typecheck ok to
>>>>>> pass it a subclass Double? I don't know the semantics of Ptolemy type
>>>>>> checking and substitutibility rules. Upcasting is usually ok but
>>>>>> downcasting needs explicit support. Do they have the concept of runtime
>>>>>> casting? I thought that was missing from the architecture according to
>>>>>> the documentation.
>>>>>
>>>>>
>>>>>
>>>>> I think it should be okay since ExtendedDouble would be a supertype of
>>>>> Double. If ExtendedDouble were a subctype, then we would be in trouble.
>>>>>
>>>>> Assuming ExtendedDouble is a supertype of Double, say I have an actor A2
>>>>> and it has one input port that has type ExtendedDouble and one output
>>>>> port
>>>>> that returns Integer. Then, the "signature" for A2 might look like this
>>>>> (using java-like syntax):
>>>>>
>>>>> Integer A2(ExtendedDouble arg)
>>>>>
>>>>> Then, if I have another actor A1 that takes a String and returns a
>>>>> Double, its signature looks like this:
>>>>>
>>>>> Double A1(String arg)
>>>>>
>>>>> And, I want to connect A2 and A1 such that the output of A1 is sent to
>>>>> A2, I have:
>>>>>
>>>>> String a1 = "..."
>>>>> Double a2 = A1(a1)
>>>>> Integer r = A2(a2).
>>>>>
>>>>> This is an "upcast" as you mentioned, which is what Ptolemy allows.
>>>>> The second case would be if there was an actor A3 that output an
>>>>> ExtendedDouble and an actor A4 that took as input a Double. Their
>>>>> signatures would look as follows.
>>>>>
>>>>> ExtendedDouble A3(String arg)
>>>>> Integer A4(Double arg)
>>>>>
>>>>> Then, if we connected them, it should give an error:
>>>>>
>>>>> String a3 = "..."
>>>>> ExtendedDouble a4 = A3(a3)
>>>>> Integer r = A4(a4)
>>>>>
>>>>> This is an example of "downcasting," which as you mentioned isn't
>>>>> supported in Ptolemy.
>>>>> Shawn
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> So I gather you are proposing that ExtendedDouble is basically an
>>>>>> abstract class with no attributes or methods that override superclass
>>>>>> methods, just a placeholder in the class hierarchy? Because if Double
>>>>>> is a subclass then it would inherit from ExtendedDouble any attributes
>>>>>> or methods that made it nullible. Only the class would be different.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Thanks again for the explanation, Amy
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Shawn Bowers [mailto:bowers at sdsc.edu]
>>>>>> Sent: Fri 10/10/2003 12:47 PM
>>>>>> To: Amy Sundermier
>>>>>> Cc: Chad Berkley; Rich Williams
>>>>>> Subject: RE: [seek-kr] Re: [seek-dev] data typing in ptolemy
>>>>>>
>>>>>> On Fri, 10 Oct 2003, Amy Sundermier wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> I don't see how changing the TypeLattice will do much of anything for
>>>>>>> you. If you want to put new types in the system, you have to be
>>>>>>> able to
>>>>>>> create instances of both the Ptolemy classes and the Ptolemy
>>>>>>> objects. Why would you want to subvert the class hierarchy for
>>>>>>> existing types?
>>>>>>
>>>>>>
>>>>>> In Chad's original email, he suggested adding a new type to Ptolemy,
>>>>>> which he called ExtendedDouble that could support null, or missing
>>>>>> values. To add a new Ptolemy type, you *have* to change the type
>>>>>> lattice. Otherwise, it isn't recognized by Ptolemy as a new type.
>>>>>> I am basing my information on Chapter 12 of the Ptolemy documentation on
>>>>>> the Type System. As I understand it, an instance of TypeLattice defines
>>>>>> the type hierarchy for Ptolemy structured types. For example, in the
>>>>>> default type lattice object, there is an edge assignment:
>>>>>>
>>>>> _basicLattice.addEdge(BaseType.DOUBLE, BaseType.COMPLEX);
>>>>>>
>>>>>> BaseType.DOUBLE and BaseType.COMPLEX are instances of DoubleType and
>>>>>> ComplexType. Each type class in Ptolemy has exactly one instance (i.e.,
>>>>>> they are singleton classes). The instance represents the actual type in
>>>>>> the Ptolemy environment, e.g., if I want to state that a port takes
>>>>>> tokens whose types are double (or subclasses of double), then I use
>>>>>> BaseType.DOUBLE to represent this.
>>>>>>
>>>>>> The statement above asserts that the Double type is a subtype of the
>>>>>> Complex type. The Ptolemy codebase itself doesn't reflect this type
>>>>>> system at all (Ptolemy *doesn't* use Java's reflection capabilities for
>>>>>> typing). For example, both classes DoubleType and ComplexType are
>>>>>> subclasses of BaseType, i.e., the class DoubleType is not a subclass of
>>>>>> ComplexType. The type lattice specifies exactly the desired
>>>>>> relationship.
>>>>>> Also, since everything is an object, the type heirarchy can be used
>>>>>> directly in Ptolemy for both design-time and runtime type checking of
>>>>>> Ptolemy ports. Ptolemy supports multiple inheritance in its type system,
>>>>>> which isn't possible in Java unless interfaces are used (which might be
>>>>>> one reason for their architecture).
>>>>>>
>>>>>> I don't think we want to subvert the class hierarchy, as you say
>>>>>> above, we
>>>>>> just want to add a new token/token type to the system. I believe
>>>>>> that the
>>>>>> type system is meant to be extensible (that is what I got from the
>>>>>> Chapter
>>>>>> above and from some of their other papers). Basically, if I want to
>>>>>> add a
>>>>>> class such as ExtendedDouble to Ptolemy's type system, I would subclass
>>>>>> BaseType, create an object of ExtendedDouble, and then add an edge to
>>>>>> the
>>>>>> new object in the lattice. I think what is desired in Chad's example
>>>>>> with
>>>>>> extending the set of values allowed in a Ptolemy token type, is to
>>>>>> augment
>>>>>> the edges in the lattice so that ExtendedDouble is a subclass of
>>>>>> Complex,
>>>>>> and Double is a subclass of ExtendedDouble.
>>>>>> Then, if I say that my port accepts the ExtendedDouble type, and I
>>>>>> receive a Double token, no big deal. If I say that my port accepts
>>>>>> Double, I am guaranteed (by Ptolemy type checking) that I will never
>>>>>> receive null, or missing, values.
>>>>>> In theory, doing this would not require augmenting any of Ptolemy's code
>>>>>> since we only need to pass around a different TypeLattice object,
>>>>>> which I
>>>>>> believe is the intention in Ptolemy. However, for some reason they
>>>>>> hard-coded the reference to a predefined Typelattice object, so it
>>>>>> doesn't quite work as they claim.
>>>>>>
>>>>>> Does that make sense?
>>>>>>
>>>>>> Shawn
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Shawn Bowers [mailto:bowers at sdsc.edu]
>>>>>>> Sent: Fri 10/10/2003 9:28 AM
>>>>>>> To: Chad Berkley
>>>>>>> Cc: Amy Sundermier; Rich Williams
>>>>>>> Subject: Re: [seek-kr] Re: [seek-dev] data typing in ptolemy
>>>>>>>
>>>>>>>
>>>>>>> I am available on Monday, or pretty much any time next week for a
>>>>>>> conference call. Just let me know what you decide. I know the
>>>>>>> phones here can do three-way calling, but would need to figure out
>>>>>>> how to do it, which shouldn't be a problem (I think Ilkay has done
>>>>>>> it before).
>>>>>>>
>>>>>>> We can talk a bit about semantic typing as well, and some ideas for
>>>>>>> how to integrate semantic types in Ptolemy.
>>>>>>>
>>>>>>> Shawn
>>>>>>>
>>>>>>>
>>>>>>> On Fri, 10 Oct 2003, Chad Berkley wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Hey Amy, Rich and Shawn (and others),
>>>>>>>>
>>>>>>>> Thanks for all your comments. Shawn and I had a pretty in-depth
>>>>>>>> discussion about this yesterday afternoon on irc and we hashed out
>>>>>>>> a few ideas. It seems we all have slightly different ideas about
>>>>>>>> what needs to be done so I'd like to propose having a conference
>>>>>>>> call on this issue. would sometime monday work? I'll propose 9:00
>>>>>>>> am PDT. If you have a problem with that time, propose a different
>>>>>>>> one. I can setup a vnc connection so we can have a whiteboard to
>>>>>>>> transfer ideas on. shawn or amy, can your phones do 3 way
>>>>>>>> calling? Mine does, do if one of your phones can do it, we can
>>>>>>>> chain together all 4 of us without having to spend
>>>>>>>> $.60/minute/participant on a conf. call. If anyone else wants to
>>>>>>>> engage in an extremely technical discussion about this issue, you
>>>>>>>> are more than welcome to join in (Bertram? Matt?).
>>>>>>>>
>>>>>>>> Amy, to answer your question about the Ptolemy source, we do have
>>>>>>>> the ptolemy source code, but I've been trying my best not to alter
>>>>>>>> it. We've realized that we may have to do so, but I'd like to avoid
>>>>>>>> that if it is at all possible. This would be a relatively simple
>>>>>>>> thing to fix if we just hacked the existing ptolemy source. (if
>>>>>>>> you want the source, by the way, it's freely available at
>>>>>>>> ptolemy.eecs.berkeley.edu).
>>>>>>>>
>>>>>>>> Rich, and Amy, you should look at the class
>>>>>>>> ptolemy.data.type.TypeLattice. Shawn was under the impression that
>>>>>>>> we could extend that class to subvert the entire class hierarchy
>>>>>>>> within ptolemy. we kind of went back and forth about this for a
>>>>>>>> while but it looks interresting. I'm going to try just hacking it
>>>>>>>> to see if changing the TypeLattice will indeed allow us to change
>>>>>>>> the hierarchy without changing the actual java class hierarchy
>>>>>>>> (something I'm still not convinced of....but Shawn found
>>>>>>>> documentation to that reguard).
>>>>>>>>
>>>>>>>> chad
>>>>>>>>
>>>>>>>> Amy Sundermier wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>> Hi Chad (and mailing list recipients),
>>>>>>>>>
>>>>>>>>> Do you have the source for Ptolemy? It seems to me that one way
>>>>>>>>> to solve your "missing value" problem is to have a boolean in the
>>>>>>>>> base class of the hierarchy (Token) called "nullValue" that all
>>>>>>>>> subclasses inherit. Any token could then declare itself null by
>>>>>>>>> setting that boolean to true. Your code would have to check "if
>>>>>>>>> (token.isNullValue())" instead of "if (token == null)" but that's
>>>>>>>>> pretty self-documenting.
>>>>>>>>> As long as the Ptolemy infrastructure code is not using reflection
>>>>>>>>> to set and get values and making assumptions about what it will
>>>>>>>>> find, you should be able to modify a superclass in this way
>>>>>>>>> without breaking their code.
>>>>>>>>>
>>>>>>>>> This seems like an obvious solution so I wonder if you've made a
>>>>>>>>> policy decision not to modify the Ptolemy source?
>>>>>>>>> I'd be interested in learning more about the semantic typing
>>>>>>>>> issues alluded to by these emails. Looking forward to meeting you
>>>>>>>>> all.
>>>>>>>>>
>>>>>>>>> Amy Sundermier
>>>>>>>>> Arizona State University
>>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Chad Berkley [mailto:berkley at nceas.ucsb.edu]
>>>>>>>>> Sent: Thu 10/9/2003 10:46 AM
>>>>>>>>> To: seek-dev at ecoinformatics.org; seek-kr at ecoinformatics.org
>>>>>>>>> Cc:
>>>>>>>>> Subject: [seek-dev] data typing in ptolemy
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Matt and I had a conversation on IRC the other day that we thought
>>>>>>>>> might be of interrest to those on these lists.
>>>>>>>>>
>>>>>>>>> Basically, I am now dealing with typing issues within ptolemy.
>>>>>>>>> Problems arise when you get missing values in the data. Ptolemy's
>>>>>>>>> type heirarchy does not allow missing values in a data tokens so
>>>>>>>>> Matt and I were talking about extending the ptolemy typing system
>>>>>>>>> to allow missing values. It occured to us that the typing system
>>>>>>>>> will need to be extended to allow for semantic typing in the future.
>>>>>>>>>
>>>>>>>>> The type class hierarchy currently looks like the following:
>>>>>>>>>
>>>>>>>>> Token
>>>>>>>>> |
>>>>>>>>> --------------------------
>>>>>>>>> | |
>>>>>>>>> ScalarToken AbstractConvertableToken
>>>>>>>>> | |
>>>>>>>>> ---------------...* -----------------
>>>>>>>>> | | | |
>>>>>>>>> DoubleToken IntToken BooleanToken StringToken
>>>>>>>>>
>>>>>>>>> *Note that ScalarToken also includes LongToken and ComplexToken.
>>>>>>>>>
>>>>>>>>> In addition to this Token hierarchy (Tokens are the means by which
>>>>>>>>> you pass data between actors over ports) there is also a port
>>>>>>>>> typing hierarchy implemented in the class BaseType. BaseType is
>>>>>>>>> the means by which you actually specify a port's type. It looks
>>>>>>>>> like this:
>>>>>>>>>
>>>>>>>>> BaseType
>>>>>>>>> |
>>>>>>>>> ---------------------------------------------------------....
>>>>>>>>> | | | | |
>>>>>>>>> BooleanType ComplexType GeneralType IntType DoubleType ....*
>>>>>>>>>
>>>>>>>>> * BaseType also includes EventType, LongType, NumericalType,
>>>>>>>>> ObjectType, SCalarType, StringType, UnknownType, UnsignedByteType
>>>>>>>>>
>>>>>>>>> Basically, in order to extend this typing system, we must extend
>>>>>>>>> both of these hierarchies since Tokens are the means by which data
>>>>>>>>> is transfered between ports and BaseTypes are the means by which
>>>>>>>>> you allow (or disallow) a port to accept different types of data.
>>>>>>>>>
>>>>>>>>> Extending the hierarchy
>>>>>>>>> -----------------------
>>>>>>>>> There are two different ways that I see to extend the hierarchy.
>>>>>>>>> The first is to extend the base class Token with our own tree of
>>>>>>>>> token types extending from the root of the tree. This will
>>>>>>>>> probably allow us the most flexibility in implementing types the
>>>>>>>>> way we need to, however, the main drawback I see to doing this is
>>>>>>>>> that we would not be able to use most existing actors because
>>>>>>>>> their ports are typed according to the current hierarchy. I think
>>>>>>>>> that one fact pretty much eliminates this approach from the options.
>>>>>>>>>
>>>>>>>>> The second approach I see is to extend each of the leaf token
>>>>>>>>> types. For example, extend DoubleToken to ExtendedDoubleToken and
>>>>>>>>> add our additional functionality there. This keeps our type
>>>>>>>>> system within the bounds of the current ptolemy hierarchy but
>>>>>>>>> limits our flexibility in extension. we are basically limited to
>>>>>>>>> the hierarchy that already exists. It is still unclear to me what
>>>>>>>>> the affects of doing this will be on existing actors. For
>>>>>>>>> instance, if we extend DoubleToken to allow missing values, and an
>>>>>>>>> actor with a port of BaseType.DoubleType gets an
>>>>>>>>> ExtendedDoubleToken, it would still need to be able to handle
>>>>>>>>> whatever value we assign as a missing value code. This is
>>>>>>>>> problematic, because we are then restricted to using an actual
>>>>>>>>> double value as a missing value code (i.e. -999.999) which we've
>>>>>>>>> always maintained was bad data practice. This could also cause
>>>>>>>>> problems because the actor cannot differentiate -999.999 from a
>>>>>>>>> normal value and will operate on it normally.
>>>>>>>>>
>>>>>>>>> This same problem comes up (but to a lesser extent) when you think
>>>>>>>>> about extending this system for semantics. What does an existing
>>>>>>>>> actor do with semantic information stored in the token? It can
>>>>>>>>> ignore it, but that may be detrimental to the analysis.
>>>>>>>>>
>>>>>>>>> Another possible option
>>>>>>>>> -----------------------
>>>>>>>>> The other possible solution to the missing value problem is to
>>>>>>>>> simply not send any data over the port when a missing value is
>>>>>>>>> encountered. I have modified the EML ingestion actor to
>>>>>>>>> dynamically create one typed port for each attribute in the data
>>>>>>>>> package. These ports can then be hooked up to other actors. The
>>>>>>>>> data is sent asyncronously and depends on the receiving ports to
>>>>>>>>> queue the data until all the input data is present to run the
>>>>>>>>> analysis.
>>>>>>>>>
>>>>>>>>> If I simply do not send a token when a missing value comes up, I
>>>>>>>>> forsee major timing problems. For instance, port A and port B are
>>>>>>>>> mapped to input ports X and Y (res.) of a plotter. port A sends a
>>>>>>>>> token to X, then B gets a missing value. It sends nothing. The
>>>>>>>>> plotter is then waiting for its second input. the next record is
>>>>>>>>> iterated into. port A sends another token to X. This causes an
>>>>>>>>> exception. The other scenario is, on the second iteration, A is a
>>>>>>>>> missing value but B is not. Then we are plotting two values from
>>>>>>>>> different records when Y recieves data from B in the second
>>>>>>>>> record. This would be a nightmare to deal with given the current
>>>>>>>>> directors.
>>>>>>>>>
>>>>>>>>> So, does anyone see something that I'm missing here? What are the
>>>>>>>>> needs of the semantic typing going to be as far as ptolemy goes?
>>>>>>>>> Anyone have a better solution than the three that I've layed out?
>>>>>>>>> This is a complex issue that I need to deal with before I can
>>>>>>>>> continue moving forward with AMS. I don't want to do anything
>>>>>>>>> that will hinder the future semantic extensions of ptolemy and
>>>>>>>>> this is just too much of a basic infrastructure item to try to
>>>>>>>>> hack. If anyone want to have an IRC chat about this, I'm on #seek.
>>>>>>>>>
>>>>>>>>> chad
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>
>>
>> _______________________________________________
>> seek-dev mailing list
>> seek-dev at ecoinformatics.org
>> http://www.ecoinformatics.org/mailman/listinfo/seek-dev
CB>
CB>
CB> --
CB> -----------------------
CB> Chad Berkley
CB> National Center for
CB> Ecological Analysis
CB> and Synthesis (NCEAS)
CB> berkley at nceas.ucsb.edu
CB> -----------------------
CB>
CB> _______________________________________________
CB> seek-dev mailing list
CB> seek-dev at ecoinformatics.org
CB> http://www.ecoinformatics.org/mailman/listinfo/seek-dev
More information about the Seek-dev
mailing list