[seek-kr] Re: [seek-dev] data typing in ptolemy
Chad Berkley
berkley at nceas.ucsb.edu
Mon Oct 13 08:43:12 PDT 2003
Hey,
Do we still want to have a conference call this morning or should we
continue this debate over email and have a call on wednesday when Rich
is around?
chad
Shawn Bowers wrote:
> On Fri, 10 Oct 2003, Amy Sundermier wrote:
>
>
>>I'll quote some stuff from your email for clarification:
>>
>>"I think what is desired in Chad's example with extending the set of
>>values allowed in a Ptolemy token type, is to augment the edges in the
>>lattice so that ExtendedDouble is a subclass of Complex, and Double is a
>>subclass of ExtendedDouble.
>>
>>Then, if I say that my port accepts the ExtendedDouble type, and I
>>receive a Double token, no big deal. If I say that my port accepts
>>Double, I am guaranteed (by Ptolemy type checking) that I will never
>>receive null, or missing, values."
>>
>>I am having some trouble believing the second paragraph. If you declare
>>a port to accept an ExtendedDouble, are you sure it will typecheck ok to
>>pass it a subclass Double? I don't know the semantics of Ptolemy type
>>checking and substitutibility rules. Upcasting is usually ok but
>>downcasting needs explicit support. Do they have the concept of runtime
>>casting? I thought that was missing from the architecture according to
>>the documentation.
>
>
> I think it should be okay since ExtendedDouble would be a supertype of
> Double. If ExtendedDouble were a subctype, then we would be in trouble.
>
> Assuming ExtendedDouble is a supertype of Double, say I have an actor A2
> and it has one input port that has type ExtendedDouble and one output port
> that returns Integer. Then, the "signature" for A2 might look like this
> (using java-like syntax):
>
> Integer A2(ExtendedDouble arg)
>
> Then, if I have another actor A1 that takes a String and returns a Double,
> its signature looks like this:
>
> Double A1(String arg)
>
> And, I want to connect A2 and A1 such that the output of A1 is sent to A2,
> I have:
>
> String a1 = "..."
> Double a2 = A1(a1)
> Integer r = A2(a2).
>
> This is an "upcast" as you mentioned, which is what Ptolemy allows.
>
> The second case would be if there was an actor A3 that output an
> ExtendedDouble and an actor A4 that took as input a Double.
> Their signatures would look as follows.
>
> ExtendedDouble A3(String arg)
> Integer A4(Double arg)
>
> Then, if we connected them, it should give an error:
>
> String a3 = "..."
> ExtendedDouble a4 = A3(a3)
> Integer r = A4(a4)
>
> This is an example of "downcasting," which as you mentioned isn't
> supported in Ptolemy.
>
> Shawn
>
>
>
>
>>So I gather you are proposing that ExtendedDouble is basically an
>>abstract class with no attributes or methods that override superclass
>>methods, just a placeholder in the class hierarchy? Because if Double
>>is a subclass then it would inherit from ExtendedDouble any attributes
>>or methods that made it nullible. Only the class would be different.
>>
>
>
>
>
>>Thanks again for the explanation, Amy
>>
>>-----Original Message-----
>>From: Shawn Bowers [mailto:bowers at sdsc.edu]
>>Sent: Fri 10/10/2003 12:47 PM
>>To: Amy Sundermier
>>Cc: Chad Berkley; Rich Williams
>>Subject: RE: [seek-kr] Re: [seek-dev] data typing in ptolemy
>>
>>On Fri, 10 Oct 2003, Amy Sundermier wrote:
>>
>>
>>>I don't see how changing the TypeLattice will do much of anything for
>>>you. If you want to put new types in the system, you have to be able to
>>>create instances of both the Ptolemy classes and the Ptolemy objects.
>>>Why would you want to subvert the class hierarchy for existing types?
>>
>>In Chad's original email, he suggested adding a new type to Ptolemy, which
>>he called ExtendedDouble that could support null, or missing values. To
>>add a new Ptolemy type, you *have* to change the type lattice. Otherwise,
>>it isn't recognized by Ptolemy as a new type.
>>
>>I am basing my information on Chapter 12 of the Ptolemy documentation on
>>the Type System. As I understand it, an instance of TypeLattice defines
>>the type hierarchy for Ptolemy structured types. For example, in the
>>default type lattice object, there is an edge assignment:
>>
>> _basicLattice.addEdge(BaseType.DOUBLE, BaseType.COMPLEX);
>>
>>BaseType.DOUBLE and BaseType.COMPLEX are instances of DoubleType and
>>ComplexType. Each type class in Ptolemy has exactly one instance (i.e.,
>>they are singleton classes). The instance represents the actual type in
>>the Ptolemy environment, e.g., if I want to state that a port takes
>>tokens whose types are double (or subclasses of double), then I use
>>BaseType.DOUBLE to represent this.
>>
>>The statement above asserts that the Double type is a subtype of the
>>Complex type. The Ptolemy codebase itself doesn't reflect this type
>>system at all (Ptolemy *doesn't* use Java's reflection capabilities for
>>typing). For example, both classes DoubleType and ComplexType are
>>subclasses of BaseType, i.e., the class DoubleType is not a subclass of
>>ComplexType. The type lattice specifies exactly the desired relationship.
>>Also, since everything is an object, the type heirarchy can be used
>>directly in Ptolemy for both design-time and runtime type checking of
>>Ptolemy ports. Ptolemy supports multiple inheritance in its type system,
>>which isn't possible in Java unless interfaces are used (which might be
>>one reason for their architecture).
>>
>>I don't think we want to subvert the class hierarchy, as you say above, we
>>just want to add a new token/token type to the system. I believe that the
>>type system is meant to be extensible (that is what I got from the Chapter
>>above and from some of their other papers). Basically, if I want to add a
>>class such as ExtendedDouble to Ptolemy's type system, I would subclass
>>BaseType, create an object of ExtendedDouble, and then add an edge to the
>>new object in the lattice. I think what is desired in Chad's example with
>>extending the set of values allowed in a Ptolemy token type, is to augment
>>the edges in the lattice so that ExtendedDouble is a subclass of Complex,
>>and Double is a subclass of ExtendedDouble.
>>
>>Then, if I say that my port accepts the ExtendedDouble type, and I receive
>>a Double token, no big deal. If I say that my port accepts Double, I am
>>guaranteed (by Ptolemy type checking) that I will never receive null, or
>>missing, values.
>>
>>In theory, doing this would not require augmenting any of Ptolemy's code
>>since we only need to pass around a different TypeLattice object, which I
>>believe is the intention in Ptolemy. However, for some reason they
>>hard-coded the reference to a predefined Typelattice object, so it
>>doesn't quite work as they claim.
>>
>>Does that make sense?
>>
>>Shawn
>>
>>
>>
>>>
>>>-----Original Message-----
>>>From: Shawn Bowers [mailto:bowers at sdsc.edu]
>>>Sent: Fri 10/10/2003 9:28 AM
>>>To: Chad Berkley
>>>Cc: Amy Sundermier; Rich Williams
>>>Subject: Re: [seek-kr] Re: [seek-dev] data typing in ptolemy
>>>
>>>
>>>I am available on Monday, or pretty much any time next week for a
>>>conference call. Just let me know what you decide. I know the phones here
>>>can do three-way calling, but would need to figure out how to do it, which
>>>shouldn't be a problem (I think Ilkay has done it before).
>>>
>>>We can talk a bit about semantic typing as well, and some ideas for how to
>>>integrate semantic types in Ptolemy.
>>>
>>>Shawn
>>>
>>>
>>>On Fri, 10 Oct 2003, Chad Berkley wrote:
>>>
>>>
>>>>Hey Amy, Rich and Shawn (and others),
>>>>
>>>>Thanks for all your comments. Shawn and I had a pretty in-depth
>>>>discussion about this yesterday afternoon on irc and we hashed out a few
>>>>ideas. It seems we all have slightly different ideas about what needs
>>>>to be done so I'd like to propose having a conference call on this
>>>>issue. would sometime monday work? I'll propose 9:00 am PDT. If you
>>>>have a problem with that time, propose a different one. I can setup a
>>>>vnc connection so we can have a whiteboard to transfer ideas on. shawn
>>>>or amy, can your phones do 3 way calling? Mine does, do if one of your
>>>>phones can do it, we can chain together all 4 of us without having to
>>>>spend $.60/minute/participant on a conf. call. If anyone else wants to
>>>>engage in an extremely technical discussion about this issue, you are
>>>>more than welcome to join in (Bertram? Matt?).
>>>>
>>>>Amy, to answer your question about the Ptolemy source, we do have the
>>>>ptolemy source code, but I've been trying my best not to alter it.
>>>>We've realized that we may have to do so, but I'd like to avoid that if
>>>>it is at all possible. This would be a relatively simple thing to fix
>>>>if we just hacked the existing ptolemy source. (if you want the source,
>>>>by the way, it's freely available at ptolemy.eecs.berkeley.edu).
>>>>
>>>>Rich, and Amy, you should look at the class
>>>>ptolemy.data.type.TypeLattice. Shawn was under the impression that we
>>>>could extend that class to subvert the entire class hierarchy within
>>>>ptolemy. we kind of went back and forth about this for a while but it
>>>>looks interresting. I'm going to try just hacking it to see if changing
>>>>the TypeLattice will indeed allow us to change the hierarchy without
>>>>changing the actual java class hierarchy (something I'm still not
>>>>convinced of....but Shawn found documentation to that reguard).
>>>>
>>>>chad
>>>>
>>>>Amy Sundermier wrote:
>>>>
>>>>>Hi Chad (and mailing list recipients),
>>>>>
>>>>>Do you have the source for Ptolemy? It seems to me that one way to solve your "missing value" problem is to have a boolean in the base class of the hierarchy (Token) called "nullValue" that all subclasses inherit. Any token could then declare itself null by setting that boolean to true. Your code would have to check "if (token.isNullValue())" instead of "if (token == null)" but that's pretty self-documenting.
>>>>>
>>>>>As long as the Ptolemy infrastructure code is not using reflection to set and get values and making assumptions about what it will find, you should be able to modify a superclass in this way without breaking their code.
>>>>>
>>>>>This seems like an obvious solution so I wonder if you've made a policy decision not to modify the Ptolemy source?
>>>>>
>>>>>I'd be interested in learning more about the semantic typing issues alluded to by these emails. Looking forward to meeting you all.
>>>>>
>>>>>Amy Sundermier
>>>>>Arizona State University
>>>>>
>>>>>
>>>>>-----Original Message-----
>>>>>From: Chad Berkley [mailto:berkley at nceas.ucsb.edu]
>>>>>Sent: Thu 10/9/2003 10:46 AM
>>>>>To: seek-dev at ecoinformatics.org; seek-kr at ecoinformatics.org
>>>>>Cc:
>>>>>Subject: [seek-dev] data typing in ptolemy
>>>>>
>>>>>Hi,
>>>>>
>>>>>Matt and I had a conversation on IRC the other day that we thought might
>>>>>be of interrest to those on these lists.
>>>>>
>>>>>Basically, I am now dealing with typing issues within ptolemy. Problems
>>>>>arise when you get missing values in the data. Ptolemy's type heirarchy
>>>>>does not allow missing values in a data tokens so Matt and I were
>>>>>talking about extending the ptolemy typing system to allow missing
>>>>>values. It occured to us that the typing system will need to be
>>>>>extended to allow for semantic typing in the future.
>>>>>
>>>>>The type class hierarchy currently looks like the following:
>>>>>
>>>>> Token
>>>>> |
>>>>> --------------------------
>>>>> | |
>>>>>ScalarToken AbstractConvertableToken
>>>>> | |
>>>>>---------------...* -----------------
>>>>> | | | |
>>>>>DoubleToken IntToken BooleanToken StringToken
>>>>>
>>>>>*Note that ScalarToken also includes LongToken and ComplexToken.
>>>>>
>>>>>In addition to this Token hierarchy (Tokens are the means by which you
>>>>>pass data between actors over ports) there is also a port typing
>>>>>hierarchy implemented in the class BaseType. BaseType is the means by
>>>>>which you actually specify a port's type. It looks like this:
>>>>>
>>>>> BaseType
>>>>> |
>>>>> ---------------------------------------------------------....
>>>>> | | | | |
>>>>>BooleanType ComplexType GeneralType IntType DoubleType ....*
>>>>>
>>>>>* BaseType also includes EventType, LongType, NumericalType, ObjectType,
>>>>>SCalarType, StringType, UnknownType, UnsignedByteType
>>>>>
>>>>>Basically, in order to extend this typing system, we must extend both of
>>>>>these hierarchies since Tokens are the means by which data is transfered
>>>>>between ports and BaseTypes are the means by which you allow (or
>>>>>disallow) a port to accept different types of data.
>>>>>
>>>>>Extending the hierarchy
>>>>>-----------------------
>>>>>There are two different ways that I see to extend the hierarchy. The
>>>>>first is to extend the base class Token with our own tree of token types
>>>>>extending from the root of the tree. This will probably allow us the
>>>>>most flexibility in implementing types the way we need to, however, the
>>>>>main drawback I see to doing this is that we would not be able to use
>>>>>most existing actors because their ports are typed according to the
>>>>>current hierarchy. I think that one fact pretty much eliminates this
>>>>>approach from the options.
>>>>>
>>>>>The second approach I see is to extend each of the leaf token types.
>>>>>For example, extend DoubleToken to ExtendedDoubleToken and add our
>>>>>additional functionality there. This keeps our type system within the
>>>>>bounds of the current ptolemy hierarchy but limits our flexibility in
>>>>>extension. we are basically limited to the hierarchy that already
>>>>>exists. It is still unclear to me what the affects of doing this will
>>>>>be on existing actors. For instance, if we extend DoubleToken to allow
>>>>>missing values, and an actor with a port of BaseType.DoubleType gets an
>>>>>ExtendedDoubleToken, it would still need to be able to handle whatever
>>>>>value we assign as a missing value code. This is problematic, because
>>>>>we are then restricted to using an actual double value as a missing
>>>>>value code (i.e. -999.999) which we've always maintained was bad data
>>>>>practice. This could also cause problems because the actor cannot
>>>>>differentiate -999.999 from a normal value and will operate on it
>>>>>normally.
>>>>>
>>>>>This same problem comes up (but to a lesser extent) when you think about
>>>>>extending this system for semantics. What does an existing actor do
>>>>>with semantic information stored in the token? It can ignore it, but
>>>>>that may be detrimental to the analysis.
>>>>>
>>>>>Another possible option
>>>>>-----------------------
>>>>>The other possible solution to the missing value problem is to simply
>>>>>not send any data over the port when a missing value is encountered. I
>>>>>have modified the EML ingestion actor to dynamically create one typed
>>>>>port for each attribute in the data package. These ports can then be
>>>>>hooked up to other actors. The data is sent asyncronously and depends
>>>>>on the receiving ports to queue the data until all the input data is
>>>>>present to run the analysis.
>>>>>
>>>>>If I simply do not send a token when a missing value comes up, I forsee
>>>>>major timing problems. For instance, port A and port B are mapped to
>>>>>input ports X and Y (res.) of a plotter. port A sends a token to X,
>>>>>then B gets a missing value. It sends nothing. The plotter is then
>>>>>waiting for its second input. the next record is iterated into. port A
>>>>>sends another token to X. This causes an exception. The other scenario
>>>>>is, on the second iteration, A is a missing value but B is not. Then we
>>>>>are plotting two values from different records when Y recieves data from
>>>>>B in the second record. This would be a nightmare to deal with given
>>>>>the current directors.
>>>>>
>>>>>So, does anyone see something that I'm missing here? What are the needs
>>>>>of the semantic typing going to be as far as ptolemy goes? Anyone have
>>>>>a better solution than the three that I've layed out? This is a complex
>>>>>issue that I need to deal with before I can continue moving forward with
>>>>>AMS. I don't want to do anything that will hinder the future semantic
>>>>>extensions of ptolemy and this is just too much of a basic
>>>>>infrastructure item to try to hack. If anyone want to have an IRC chat
>>>>>about this, I'm on #seek.
>>>>>
>>>>>chad
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>>
--
-----------------------
Chad Berkley
National Center for
Ecological Analysis
and Synthesis (NCEAS)
berkley at nceas.ucsb.edu
-----------------------
More information about the Seek-dev
mailing list