[seek-kr] Re: [seek-dev] data typing in ptolemy
Matt Jones
jones at nceas.ucsb.edu
Mon Oct 13 09:02:07 PDT 2003
I can do it now, and would like to be involved. Or Wednesday is fine too.
Matt
Chad Berkley wrote:
> Hey,
>
> Do we still want to have a conference call this morning or should we
> continue this debate over email and have a call on wednesday when Rich
> is around?
>
> chad
>
> Shawn Bowers wrote:
>
>> On Fri, 10 Oct 2003, Amy Sundermier wrote:
>>
>>
>>> I'll quote some stuff from your email for clarification:
>>>
>>> "I think what is desired in Chad's example with extending the set of
>>> values allowed in a Ptolemy token type, is to augment the edges in the
>>> lattice so that ExtendedDouble is a subclass of Complex, and Double is a
>>> subclass of ExtendedDouble.
>>>
>>> Then, if I say that my port accepts the ExtendedDouble type, and I
>>> receive a Double token, no big deal. If I say that my port accepts
>>> Double, I am guaranteed (by Ptolemy type checking) that I will never
>>> receive null, or missing, values."
>>>
>>> I am having some trouble believing the second paragraph. If you declare
>>> a port to accept an ExtendedDouble, are you sure it will typecheck ok to
>>> pass it a subclass Double? I don't know the semantics of Ptolemy type
>>> checking and substitutibility rules. Upcasting is usually ok but
>>> downcasting needs explicit support. Do they have the concept of runtime
>>> casting? I thought that was missing from the architecture according to
>>> the documentation.
>>
>>
>>
>> I think it should be okay since ExtendedDouble would be a supertype of
>> Double. If ExtendedDouble were a subctype, then we would be in trouble.
>>
>> Assuming ExtendedDouble is a supertype of Double, say I have an actor A2
>> and it has one input port that has type ExtendedDouble and one output
>> port
>> that returns Integer. Then, the "signature" for A2 might look like this
>> (using java-like syntax):
>>
>> Integer A2(ExtendedDouble arg)
>>
>> Then, if I have another actor A1 that takes a String and returns a
>> Double, its signature looks like this:
>>
>> Double A1(String arg)
>>
>> And, I want to connect A2 and A1 such that the output of A1 is sent to
>> A2, I have:
>>
>> String a1 = "..."
>> Double a2 = A1(a1)
>> Integer r = A2(a2).
>>
>> This is an "upcast" as you mentioned, which is what Ptolemy allows.
>> The second case would be if there was an actor A3 that output an
>> ExtendedDouble and an actor A4 that took as input a Double. Their
>> signatures would look as follows.
>>
>> ExtendedDouble A3(String arg)
>> Integer A4(Double arg)
>>
>> Then, if we connected them, it should give an error:
>>
>> String a3 = "..."
>> ExtendedDouble a4 = A3(a3)
>> Integer r = A4(a4)
>>
>> This is an example of "downcasting," which as you mentioned isn't
>> supported in Ptolemy.
>> Shawn
>>
>>
>>
>>
>>> So I gather you are proposing that ExtendedDouble is basically an
>>> abstract class with no attributes or methods that override superclass
>>> methods, just a placeholder in the class hierarchy? Because if Double
>>> is a subclass then it would inherit from ExtendedDouble any attributes
>>> or methods that made it nullible. Only the class would be different.
>>>
>>
>>
>>
>>
>>> Thanks again for the explanation, Amy
>>>
>>> -----Original Message-----
>>> From: Shawn Bowers [mailto:bowers at sdsc.edu]
>>> Sent: Fri 10/10/2003 12:47 PM
>>> To: Amy Sundermier
>>> Cc: Chad Berkley; Rich Williams
>>> Subject: RE: [seek-kr] Re: [seek-dev] data typing in ptolemy
>>>
>>> On Fri, 10 Oct 2003, Amy Sundermier wrote:
>>>
>>>
>>>> I don't see how changing the TypeLattice will do much of anything for
>>>> you. If you want to put new types in the system, you have to be
>>>> able to
>>>> create instances of both the Ptolemy classes and the Ptolemy
>>>> objects. Why would you want to subvert the class hierarchy for
>>>> existing types?
>>>
>>>
>>> In Chad's original email, he suggested adding a new type to Ptolemy,
>>> which he called ExtendedDouble that could support null, or missing
>>> values. To add a new Ptolemy type, you *have* to change the type
>>> lattice. Otherwise, it isn't recognized by Ptolemy as a new type.
>>> I am basing my information on Chapter 12 of the Ptolemy documentation on
>>> the Type System. As I understand it, an instance of TypeLattice defines
>>> the type hierarchy for Ptolemy structured types. For example, in the
>>> default type lattice object, there is an edge assignment:
>>>
>>> _basicLattice.addEdge(BaseType.DOUBLE, BaseType.COMPLEX);
>>>
>>> BaseType.DOUBLE and BaseType.COMPLEX are instances of DoubleType and
>>> ComplexType. Each type class in Ptolemy has exactly one instance (i.e.,
>>> they are singleton classes). The instance represents the actual type in
>>> the Ptolemy environment, e.g., if I want to state that a port takes
>>> tokens whose types are double (or subclasses of double), then I use
>>> BaseType.DOUBLE to represent this.
>>>
>>> The statement above asserts that the Double type is a subtype of the
>>> Complex type. The Ptolemy codebase itself doesn't reflect this type
>>> system at all (Ptolemy *doesn't* use Java's reflection capabilities for
>>> typing). For example, both classes DoubleType and ComplexType are
>>> subclasses of BaseType, i.e., the class DoubleType is not a subclass of
>>> ComplexType. The type lattice specifies exactly the desired
>>> relationship.
>>> Also, since everything is an object, the type heirarchy can be used
>>> directly in Ptolemy for both design-time and runtime type checking of
>>> Ptolemy ports. Ptolemy supports multiple inheritance in its type system,
>>> which isn't possible in Java unless interfaces are used (which might be
>>> one reason for their architecture).
>>>
>>> I don't think we want to subvert the class hierarchy, as you say
>>> above, we
>>> just want to add a new token/token type to the system. I believe
>>> that the
>>> type system is meant to be extensible (that is what I got from the
>>> Chapter
>>> above and from some of their other papers). Basically, if I want to
>>> add a
>>> class such as ExtendedDouble to Ptolemy's type system, I would subclass
>>> BaseType, create an object of ExtendedDouble, and then add an edge to
>>> the
>>> new object in the lattice. I think what is desired in Chad's example
>>> with
>>> extending the set of values allowed in a Ptolemy token type, is to
>>> augment
>>> the edges in the lattice so that ExtendedDouble is a subclass of
>>> Complex,
>>> and Double is a subclass of ExtendedDouble.
>>> Then, if I say that my port accepts the ExtendedDouble type, and I
>>> receive a Double token, no big deal. If I say that my port accepts
>>> Double, I am guaranteed (by Ptolemy type checking) that I will never
>>> receive null, or missing, values.
>>> In theory, doing this would not require augmenting any of Ptolemy's code
>>> since we only need to pass around a different TypeLattice object,
>>> which I
>>> believe is the intention in Ptolemy. However, for some reason they
>>> hard-coded the reference to a predefined Typelattice object, so it
>>> doesn't quite work as they claim.
>>>
>>> Does that make sense?
>>>
>>> Shawn
>>>
>>>
>>>
>>>>
>>>> -----Original Message-----
>>>> From: Shawn Bowers [mailto:bowers at sdsc.edu]
>>>> Sent: Fri 10/10/2003 9:28 AM
>>>> To: Chad Berkley
>>>> Cc: Amy Sundermier; Rich Williams
>>>> Subject: Re: [seek-kr] Re: [seek-dev] data typing in ptolemy
>>>>
>>>>
>>>> I am available on Monday, or pretty much any time next week for a
>>>> conference call. Just let me know what you decide. I know the
>>>> phones here can do three-way calling, but would need to figure out
>>>> how to do it, which shouldn't be a problem (I think Ilkay has done
>>>> it before).
>>>>
>>>> We can talk a bit about semantic typing as well, and some ideas for
>>>> how to integrate semantic types in Ptolemy.
>>>>
>>>> Shawn
>>>>
>>>>
>>>> On Fri, 10 Oct 2003, Chad Berkley wrote:
>>>>
>>>>
>>>>> Hey Amy, Rich and Shawn (and others),
>>>>>
>>>>> Thanks for all your comments. Shawn and I had a pretty in-depth
>>>>> discussion about this yesterday afternoon on irc and we hashed out
>>>>> a few ideas. It seems we all have slightly different ideas about
>>>>> what needs to be done so I'd like to propose having a conference
>>>>> call on this issue. would sometime monday work? I'll propose 9:00
>>>>> am PDT. If you have a problem with that time, propose a different
>>>>> one. I can setup a vnc connection so we can have a whiteboard to
>>>>> transfer ideas on. shawn or amy, can your phones do 3 way
>>>>> calling? Mine does, do if one of your phones can do it, we can
>>>>> chain together all 4 of us without having to spend
>>>>> $.60/minute/participant on a conf. call. If anyone else wants to
>>>>> engage in an extremely technical discussion about this issue, you
>>>>> are more than welcome to join in (Bertram? Matt?).
>>>>>
>>>>> Amy, to answer your question about the Ptolemy source, we do have
>>>>> the ptolemy source code, but I've been trying my best not to alter
>>>>> it. We've realized that we may have to do so, but I'd like to avoid
>>>>> that if it is at all possible. This would be a relatively simple
>>>>> thing to fix if we just hacked the existing ptolemy source. (if
>>>>> you want the source, by the way, it's freely available at
>>>>> ptolemy.eecs.berkeley.edu).
>>>>>
>>>>> Rich, and Amy, you should look at the class
>>>>> ptolemy.data.type.TypeLattice. Shawn was under the impression that
>>>>> we could extend that class to subvert the entire class hierarchy
>>>>> within ptolemy. we kind of went back and forth about this for a
>>>>> while but it looks interresting. I'm going to try just hacking it
>>>>> to see if changing the TypeLattice will indeed allow us to change
>>>>> the hierarchy without changing the actual java class hierarchy
>>>>> (something I'm still not convinced of....but Shawn found
>>>>> documentation to that reguard).
>>>>>
>>>>> chad
>>>>>
>>>>> Amy Sundermier wrote:
>>>>>
>>>>>> Hi Chad (and mailing list recipients),
>>>>>>
>>>>>> Do you have the source for Ptolemy? It seems to me that one way
>>>>>> to solve your "missing value" problem is to have a boolean in the
>>>>>> base class of the hierarchy (Token) called "nullValue" that all
>>>>>> subclasses inherit. Any token could then declare itself null by
>>>>>> setting that boolean to true. Your code would have to check "if
>>>>>> (token.isNullValue())" instead of "if (token == null)" but that's
>>>>>> pretty self-documenting.
>>>>>> As long as the Ptolemy infrastructure code is not using reflection
>>>>>> to set and get values and making assumptions about what it will
>>>>>> find, you should be able to modify a superclass in this way
>>>>>> without breaking their code.
>>>>>>
>>>>>> This seems like an obvious solution so I wonder if you've made a
>>>>>> policy decision not to modify the Ptolemy source?
>>>>>> I'd be interested in learning more about the semantic typing
>>>>>> issues alluded to by these emails. Looking forward to meeting you
>>>>>> all.
>>>>>>
>>>>>> Amy Sundermier
>>>>>> Arizona State University
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Chad Berkley [mailto:berkley at nceas.ucsb.edu]
>>>>>> Sent: Thu 10/9/2003 10:46 AM
>>>>>> To: seek-dev at ecoinformatics.org; seek-kr at ecoinformatics.org
>>>>>> Cc:
>>>>>> Subject: [seek-dev] data typing in ptolemy
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Matt and I had a conversation on IRC the other day that we thought
>>>>>> might be of interrest to those on these lists.
>>>>>>
>>>>>> Basically, I am now dealing with typing issues within ptolemy.
>>>>>> Problems arise when you get missing values in the data. Ptolemy's
>>>>>> type heirarchy does not allow missing values in a data tokens so
>>>>>> Matt and I were talking about extending the ptolemy typing system
>>>>>> to allow missing values. It occured to us that the typing system
>>>>>> will need to be extended to allow for semantic typing in the future.
>>>>>>
>>>>>> The type class hierarchy currently looks like the following:
>>>>>>
>>>>>> Token
>>>>>> |
>>>>>> --------------------------
>>>>>> | |
>>>>>> ScalarToken AbstractConvertableToken
>>>>>> | |
>>>>>> ---------------...* -----------------
>>>>>> | | | |
>>>>>> DoubleToken IntToken BooleanToken StringToken
>>>>>>
>>>>>> *Note that ScalarToken also includes LongToken and ComplexToken.
>>>>>>
>>>>>> In addition to this Token hierarchy (Tokens are the means by which
>>>>>> you pass data between actors over ports) there is also a port
>>>>>> typing hierarchy implemented in the class BaseType. BaseType is
>>>>>> the means by which you actually specify a port's type. It looks
>>>>>> like this:
>>>>>>
>>>>>> BaseType
>>>>>> |
>>>>>> ---------------------------------------------------------....
>>>>>> | | | | |
>>>>>> BooleanType ComplexType GeneralType IntType DoubleType ....*
>>>>>>
>>>>>> * BaseType also includes EventType, LongType, NumericalType,
>>>>>> ObjectType, SCalarType, StringType, UnknownType, UnsignedByteType
>>>>>>
>>>>>> Basically, in order to extend this typing system, we must extend
>>>>>> both of these hierarchies since Tokens are the means by which data
>>>>>> is transfered between ports and BaseTypes are the means by which
>>>>>> you allow (or disallow) a port to accept different types of data.
>>>>>>
>>>>>> Extending the hierarchy
>>>>>> -----------------------
>>>>>> There are two different ways that I see to extend the hierarchy.
>>>>>> The first is to extend the base class Token with our own tree of
>>>>>> token types extending from the root of the tree. This will
>>>>>> probably allow us the most flexibility in implementing types the
>>>>>> way we need to, however, the main drawback I see to doing this is
>>>>>> that we would not be able to use most existing actors because
>>>>>> their ports are typed according to the current hierarchy. I think
>>>>>> that one fact pretty much eliminates this approach from the options.
>>>>>>
>>>>>> The second approach I see is to extend each of the leaf token
>>>>>> types. For example, extend DoubleToken to ExtendedDoubleToken and
>>>>>> add our additional functionality there. This keeps our type
>>>>>> system within the bounds of the current ptolemy hierarchy but
>>>>>> limits our flexibility in extension. we are basically limited to
>>>>>> the hierarchy that already exists. It is still unclear to me what
>>>>>> the affects of doing this will be on existing actors. For
>>>>>> instance, if we extend DoubleToken to allow missing values, and an
>>>>>> actor with a port of BaseType.DoubleType gets an
>>>>>> ExtendedDoubleToken, it would still need to be able to handle
>>>>>> whatever value we assign as a missing value code. This is
>>>>>> problematic, because we are then restricted to using an actual
>>>>>> double value as a missing value code (i.e. -999.999) which we've
>>>>>> always maintained was bad data practice. This could also cause
>>>>>> problems because the actor cannot differentiate -999.999 from a
>>>>>> normal value and will operate on it normally.
>>>>>>
>>>>>> This same problem comes up (but to a lesser extent) when you think
>>>>>> about extending this system for semantics. What does an existing
>>>>>> actor do with semantic information stored in the token? It can
>>>>>> ignore it, but that may be detrimental to the analysis.
>>>>>>
>>>>>> Another possible option
>>>>>> -----------------------
>>>>>> The other possible solution to the missing value problem is to
>>>>>> simply not send any data over the port when a missing value is
>>>>>> encountered. I have modified the EML ingestion actor to
>>>>>> dynamically create one typed port for each attribute in the data
>>>>>> package. These ports can then be hooked up to other actors. The
>>>>>> data is sent asyncronously and depends on the receiving ports to
>>>>>> queue the data until all the input data is present to run the
>>>>>> analysis.
>>>>>>
>>>>>> If I simply do not send a token when a missing value comes up, I
>>>>>> forsee major timing problems. For instance, port A and port B are
>>>>>> mapped to input ports X and Y (res.) of a plotter. port A sends a
>>>>>> token to X, then B gets a missing value. It sends nothing. The
>>>>>> plotter is then waiting for its second input. the next record is
>>>>>> iterated into. port A sends another token to X. This causes an
>>>>>> exception. The other scenario is, on the second iteration, A is a
>>>>>> missing value but B is not. Then we are plotting two values from
>>>>>> different records when Y recieves data from B in the second
>>>>>> record. This would be a nightmare to deal with given the current
>>>>>> directors.
>>>>>>
>>>>>> So, does anyone see something that I'm missing here? What are the
>>>>>> needs of the semantic typing going to be as far as ptolemy goes?
>>>>>> Anyone have a better solution than the three that I've layed out?
>>>>>> This is a complex issue that I need to deal with before I can
>>>>>> continue moving forward with AMS. I don't want to do anything
>>>>>> that will hinder the future semantic extensions of ptolemy and
>>>>>> this is just too much of a basic infrastructure item to try to
>>>>>> hack. If anyone want to have an IRC chat about this, I'm on #seek.
>>>>>>
>>>>>> chad
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>
>
--
-------------------------------------------------------------------
Matt Jones jones at nceas.ucsb.edu
http://www.nceas.ucsb.edu/ Fax: 425-920-2439 Ph: 907-789-0496
National Center for Ecological Analysis and Synthesis (NCEAS)
University of California Santa Barbara
Interested in ecological informatics? http://www.ecoinformatics.org
-------------------------------------------------------------------
More information about the Seek-dev
mailing list