[seek-kr] Re: [seek-dev] data typing in ptolemy
Chad Berkley
berkley at nceas.ucsb.edu
Mon Oct 13 10:30:54 PDT 2003
OK, lets do this call at 10:00 am PDT Wednesday (10/15). Is that ok
with you rich? Bertram, do you want to be in on this? This will allow
me time to send an email out to pthackers to see what they think.
I don't think the 3 way calling hack will work with this many
participants so I'll just setup a regular at&t conf call. I'll email
later with the code and number.
So far, the participants are:
Shawn
Rich
Matt
Chad
Amy
Anyone else want to join this little party? The topic is: Extending the
ptolemy type system.
chad
Shawn Bowers wrote:
> I am available (it is about 9:15). Either way is fine with me.
> If we want to discuss semantic issues, I think it would be good to have
> Rich involved in the discussion. So perhaps we should punt to Wed.
>
>
> Shawn
>
>
>
> On Mon, 13 Oct 2003, Matt Jones wrote:
>
>
>>I can do it now, and would like to be involved. Or Wednesday is fine too.
>>
>>Matt
>>
>>Chad Berkley wrote:
>>
>>>Hey,
>>>
>>>Do we still want to have a conference call this morning or should we
>>>continue this debate over email and have a call on wednesday when Rich
>>>is around?
>>>
>>>chad
>>>
>>>Shawn Bowers wrote:
>>>
>>>
>>>>On Fri, 10 Oct 2003, Amy Sundermier wrote:
>>>>
>>>>
>>>>
>>>>>I'll quote some stuff from your email for clarification:
>>>>>
>>>>>"I think what is desired in Chad's example with extending the set of
>>>>>values allowed in a Ptolemy token type, is to augment the edges in the
>>>>>lattice so that ExtendedDouble is a subclass of Complex, and Double is a
>>>>>subclass of ExtendedDouble.
>>>>>
>>>>>Then, if I say that my port accepts the ExtendedDouble type, and I
>>>>>receive a Double token, no big deal. If I say that my port accepts
>>>>>Double, I am guaranteed (by Ptolemy type checking) that I will never
>>>>>receive null, or missing, values."
>>>>>
>>>>>I am having some trouble believing the second paragraph. If you declare
>>>>>a port to accept an ExtendedDouble, are you sure it will typecheck ok to
>>>>>pass it a subclass Double? I don't know the semantics of Ptolemy type
>>>>>checking and substitutibility rules. Upcasting is usually ok but
>>>>>downcasting needs explicit support. Do they have the concept of runtime
>>>>>casting? I thought that was missing from the architecture according to
>>>>>the documentation.
>>>>
>>>>
>>>>
>>>>I think it should be okay since ExtendedDouble would be a supertype of
>>>>Double. If ExtendedDouble were a subctype, then we would be in trouble.
>>>>
>>>>Assuming ExtendedDouble is a supertype of Double, say I have an actor A2
>>>>and it has one input port that has type ExtendedDouble and one output
>>>>port
>>>>that returns Integer. Then, the "signature" for A2 might look like this
>>>>(using java-like syntax):
>>>>
>>>> Integer A2(ExtendedDouble arg)
>>>>
>>>>Then, if I have another actor A1 that takes a String and returns a
>>>>Double, its signature looks like this:
>>>>
>>>> Double A1(String arg)
>>>>
>>>>And, I want to connect A2 and A1 such that the output of A1 is sent to
>>>>A2, I have:
>>>>
>>>> String a1 = "..."
>>>> Double a2 = A1(a1)
>>>> Integer r = A2(a2).
>>>>
>>>>This is an "upcast" as you mentioned, which is what Ptolemy allows.
>>>>The second case would be if there was an actor A3 that output an
>>>>ExtendedDouble and an actor A4 that took as input a Double. Their
>>>>signatures would look as follows.
>>>>
>>>> ExtendedDouble A3(String arg)
>>>> Integer A4(Double arg)
>>>>
>>>>Then, if we connected them, it should give an error:
>>>>
>>>> String a3 = "..."
>>>> ExtendedDouble a4 = A3(a3)
>>>> Integer r = A4(a4)
>>>>
>>>>This is an example of "downcasting," which as you mentioned isn't
>>>>supported in Ptolemy.
>>>>Shawn
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>So I gather you are proposing that ExtendedDouble is basically an
>>>>>abstract class with no attributes or methods that override superclass
>>>>>methods, just a placeholder in the class hierarchy? Because if Double
>>>>>is a subclass then it would inherit from ExtendedDouble any attributes
>>>>>or methods that made it nullible. Only the class would be different.
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>Thanks again for the explanation, Amy
>>>>>
>>>>>-----Original Message-----
>>>>>From: Shawn Bowers [mailto:bowers at sdsc.edu]
>>>>>Sent: Fri 10/10/2003 12:47 PM
>>>>>To: Amy Sundermier
>>>>>Cc: Chad Berkley; Rich Williams
>>>>>Subject: RE: [seek-kr] Re: [seek-dev] data typing in ptolemy
>>>>>
>>>>>On Fri, 10 Oct 2003, Amy Sundermier wrote:
>>>>>
>>>>>
>>>>>
>>>>>>I don't see how changing the TypeLattice will do much of anything for
>>>>>>you. If you want to put new types in the system, you have to be
>>>>>>able to
>>>>>>create instances of both the Ptolemy classes and the Ptolemy
>>>>>>objects. Why would you want to subvert the class hierarchy for
>>>>>>existing types?
>>>>>
>>>>>
>>>>>In Chad's original email, he suggested adding a new type to Ptolemy,
>>>>>which he called ExtendedDouble that could support null, or missing
>>>>>values. To add a new Ptolemy type, you *have* to change the type
>>>>>lattice. Otherwise, it isn't recognized by Ptolemy as a new type.
>>>>>I am basing my information on Chapter 12 of the Ptolemy documentation on
>>>>>the Type System. As I understand it, an instance of TypeLattice defines
>>>>>the type hierarchy for Ptolemy structured types. For example, in the
>>>>>default type lattice object, there is an edge assignment:
>>>>>
>>>>> _basicLattice.addEdge(BaseType.DOUBLE, BaseType.COMPLEX);
>>>>>
>>>>>BaseType.DOUBLE and BaseType.COMPLEX are instances of DoubleType and
>>>>>ComplexType. Each type class in Ptolemy has exactly one instance (i.e.,
>>>>>they are singleton classes). The instance represents the actual type in
>>>>>the Ptolemy environment, e.g., if I want to state that a port takes
>>>>>tokens whose types are double (or subclasses of double), then I use
>>>>>BaseType.DOUBLE to represent this.
>>>>>
>>>>>The statement above asserts that the Double type is a subtype of the
>>>>>Complex type. The Ptolemy codebase itself doesn't reflect this type
>>>>>system at all (Ptolemy *doesn't* use Java's reflection capabilities for
>>>>>typing). For example, both classes DoubleType and ComplexType are
>>>>>subclasses of BaseType, i.e., the class DoubleType is not a subclass of
>>>>>ComplexType. The type lattice specifies exactly the desired
>>>>>relationship.
>>>>>Also, since everything is an object, the type heirarchy can be used
>>>>>directly in Ptolemy for both design-time and runtime type checking of
>>>>>Ptolemy ports. Ptolemy supports multiple inheritance in its type system,
>>>>>which isn't possible in Java unless interfaces are used (which might be
>>>>>one reason for their architecture).
>>>>>
>>>>>I don't think we want to subvert the class hierarchy, as you say
>>>>>above, we
>>>>>just want to add a new token/token type to the system. I believe
>>>>>that the
>>>>>type system is meant to be extensible (that is what I got from the
>>>>>Chapter
>>>>>above and from some of their other papers). Basically, if I want to
>>>>>add a
>>>>>class such as ExtendedDouble to Ptolemy's type system, I would subclass
>>>>>BaseType, create an object of ExtendedDouble, and then add an edge to
>>>>>the
>>>>>new object in the lattice. I think what is desired in Chad's example
>>>>>with
>>>>>extending the set of values allowed in a Ptolemy token type, is to
>>>>>augment
>>>>>the edges in the lattice so that ExtendedDouble is a subclass of
>>>>>Complex,
>>>>>and Double is a subclass of ExtendedDouble.
>>>>>Then, if I say that my port accepts the ExtendedDouble type, and I
>>>>>receive a Double token, no big deal. If I say that my port accepts
>>>>>Double, I am guaranteed (by Ptolemy type checking) that I will never
>>>>>receive null, or missing, values.
>>>>>In theory, doing this would not require augmenting any of Ptolemy's code
>>>>>since we only need to pass around a different TypeLattice object,
>>>>>which I
>>>>>believe is the intention in Ptolemy. However, for some reason they
>>>>>hard-coded the reference to a predefined Typelattice object, so it
>>>>>doesn't quite work as they claim.
>>>>>
>>>>>Does that make sense?
>>>>>
>>>>>Shawn
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>-----Original Message-----
>>>>>>From: Shawn Bowers [mailto:bowers at sdsc.edu]
>>>>>>Sent: Fri 10/10/2003 9:28 AM
>>>>>>To: Chad Berkley
>>>>>>Cc: Amy Sundermier; Rich Williams
>>>>>>Subject: Re: [seek-kr] Re: [seek-dev] data typing in ptolemy
>>>>>>
>>>>>>
>>>>>>I am available on Monday, or pretty much any time next week for a
>>>>>>conference call. Just let me know what you decide. I know the
>>>>>>phones here can do three-way calling, but would need to figure out
>>>>>>how to do it, which shouldn't be a problem (I think Ilkay has done
>>>>>>it before).
>>>>>>
>>>>>>We can talk a bit about semantic typing as well, and some ideas for
>>>>>>how to integrate semantic types in Ptolemy.
>>>>>>
>>>>>>Shawn
>>>>>>
>>>>>>
>>>>>>On Fri, 10 Oct 2003, Chad Berkley wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>>Hey Amy, Rich and Shawn (and others),
>>>>>>>
>>>>>>>Thanks for all your comments. Shawn and I had a pretty in-depth
>>>>>>>discussion about this yesterday afternoon on irc and we hashed out
>>>>>>>a few ideas. It seems we all have slightly different ideas about
>>>>>>>what needs to be done so I'd like to propose having a conference
>>>>>>>call on this issue. would sometime monday work? I'll propose 9:00
>>>>>>>am PDT. If you have a problem with that time, propose a different
>>>>>>>one. I can setup a vnc connection so we can have a whiteboard to
>>>>>>>transfer ideas on. shawn or amy, can your phones do 3 way
>>>>>>>calling? Mine does, do if one of your phones can do it, we can
>>>>>>>chain together all 4 of us without having to spend
>>>>>>>$.60/minute/participant on a conf. call. If anyone else wants to
>>>>>>>engage in an extremely technical discussion about this issue, you
>>>>>>>are more than welcome to join in (Bertram? Matt?).
>>>>>>>
>>>>>>>Amy, to answer your question about the Ptolemy source, we do have
>>>>>>>the ptolemy source code, but I've been trying my best not to alter
>>>>>>>it. We've realized that we may have to do so, but I'd like to avoid
>>>>>>>that if it is at all possible. This would be a relatively simple
>>>>>>>thing to fix if we just hacked the existing ptolemy source. (if
>>>>>>>you want the source, by the way, it's freely available at
>>>>>>>ptolemy.eecs.berkeley.edu).
>>>>>>>
>>>>>>>Rich, and Amy, you should look at the class
>>>>>>>ptolemy.data.type.TypeLattice. Shawn was under the impression that
>>>>>>>we could extend that class to subvert the entire class hierarchy
>>>>>>>within ptolemy. we kind of went back and forth about this for a
>>>>>>>while but it looks interresting. I'm going to try just hacking it
>>>>>>>to see if changing the TypeLattice will indeed allow us to change
>>>>>>>the hierarchy without changing the actual java class hierarchy
>>>>>>>(something I'm still not convinced of....but Shawn found
>>>>>>>documentation to that reguard).
>>>>>>>
>>>>>>>chad
>>>>>>>
>>>>>>>Amy Sundermier wrote:
>>>>>>>
>>>>>>>
>>>>>>>>Hi Chad (and mailing list recipients),
>>>>>>>>
>>>>>>>>Do you have the source for Ptolemy? It seems to me that one way
>>>>>>>>to solve your "missing value" problem is to have a boolean in the
>>>>>>>>base class of the hierarchy (Token) called "nullValue" that all
>>>>>>>>subclasses inherit. Any token could then declare itself null by
>>>>>>>>setting that boolean to true. Your code would have to check "if
>>>>>>>>(token.isNullValue())" instead of "if (token == null)" but that's
>>>>>>>>pretty self-documenting.
>>>>>>>>As long as the Ptolemy infrastructure code is not using reflection
>>>>>>>>to set and get values and making assumptions about what it will
>>>>>>>>find, you should be able to modify a superclass in this way
>>>>>>>>without breaking their code.
>>>>>>>>
>>>>>>>>This seems like an obvious solution so I wonder if you've made a
>>>>>>>>policy decision not to modify the Ptolemy source?
>>>>>>>>I'd be interested in learning more about the semantic typing
>>>>>>>>issues alluded to by these emails. Looking forward to meeting you
>>>>>>>>all.
>>>>>>>>
>>>>>>>>Amy Sundermier
>>>>>>>>Arizona State University
>>>>>>>>
>>>>>>>>-----Original Message-----
>>>>>>>>From: Chad Berkley [mailto:berkley at nceas.ucsb.edu]
>>>>>>>>Sent: Thu 10/9/2003 10:46 AM
>>>>>>>>To: seek-dev at ecoinformatics.org; seek-kr at ecoinformatics.org
>>>>>>>>Cc:
>>>>>>>>Subject: [seek-dev] data typing in ptolemy
>>>>>>>>
>>>>>>>>Hi,
>>>>>>>>
>>>>>>>>Matt and I had a conversation on IRC the other day that we thought
>>>>>>>>might be of interrest to those on these lists.
>>>>>>>>
>>>>>>>>Basically, I am now dealing with typing issues within ptolemy.
>>>>>>>>Problems arise when you get missing values in the data. Ptolemy's
>>>>>>>>type heirarchy does not allow missing values in a data tokens so
>>>>>>>>Matt and I were talking about extending the ptolemy typing system
>>>>>>>>to allow missing values. It occured to us that the typing system
>>>>>>>>will need to be extended to allow for semantic typing in the future.
>>>>>>>>
>>>>>>>>The type class hierarchy currently looks like the following:
>>>>>>>>
>>>>>>>> Token
>>>>>>>> |
>>>>>>>> --------------------------
>>>>>>>> | |
>>>>>>>>ScalarToken AbstractConvertableToken
>>>>>>>> | |
>>>>>>>>---------------...* -----------------
>>>>>>>> | | | |
>>>>>>>>DoubleToken IntToken BooleanToken StringToken
>>>>>>>>
>>>>>>>>*Note that ScalarToken also includes LongToken and ComplexToken.
>>>>>>>>
>>>>>>>>In addition to this Token hierarchy (Tokens are the means by which
>>>>>>>>you pass data between actors over ports) there is also a port
>>>>>>>>typing hierarchy implemented in the class BaseType. BaseType is
>>>>>>>>the means by which you actually specify a port's type. It looks
>>>>>>>>like this:
>>>>>>>>
>>>>>>>> BaseType
>>>>>>>> |
>>>>>>>> ---------------------------------------------------------....
>>>>>>>> | | | | |
>>>>>>>>BooleanType ComplexType GeneralType IntType DoubleType ....*
>>>>>>>>
>>>>>>>>* BaseType also includes EventType, LongType, NumericalType,
>>>>>>>>ObjectType, SCalarType, StringType, UnknownType, UnsignedByteType
>>>>>>>>
>>>>>>>>Basically, in order to extend this typing system, we must extend
>>>>>>>>both of these hierarchies since Tokens are the means by which data
>>>>>>>>is transfered between ports and BaseTypes are the means by which
>>>>>>>>you allow (or disallow) a port to accept different types of data.
>>>>>>>>
>>>>>>>>Extending the hierarchy
>>>>>>>>-----------------------
>>>>>>>>There are two different ways that I see to extend the hierarchy.
>>>>>>>>The first is to extend the base class Token with our own tree of
>>>>>>>>token types extending from the root of the tree. This will
>>>>>>>>probably allow us the most flexibility in implementing types the
>>>>>>>>way we need to, however, the main drawback I see to doing this is
>>>>>>>>that we would not be able to use most existing actors because
>>>>>>>>their ports are typed according to the current hierarchy. I think
>>>>>>>>that one fact pretty much eliminates this approach from the options.
>>>>>>>>
>>>>>>>>The second approach I see is to extend each of the leaf token
>>>>>>>>types. For example, extend DoubleToken to ExtendedDoubleToken and
>>>>>>>>add our additional functionality there. This keeps our type
>>>>>>>>system within the bounds of the current ptolemy hierarchy but
>>>>>>>>limits our flexibility in extension. we are basically limited to
>>>>>>>>the hierarchy that already exists. It is still unclear to me what
>>>>>>>>the affects of doing this will be on existing actors. For
>>>>>>>>instance, if we extend DoubleToken to allow missing values, and an
>>>>>>>>actor with a port of BaseType.DoubleType gets an
>>>>>>>>ExtendedDoubleToken, it would still need to be able to handle
>>>>>>>>whatever value we assign as a missing value code. This is
>>>>>>>>problematic, because we are then restricted to using an actual
>>>>>>>>double value as a missing value code (i.e. -999.999) which we've
>>>>>>>>always maintained was bad data practice. This could also cause
>>>>>>>>problems because the actor cannot differentiate -999.999 from a
>>>>>>>>normal value and will operate on it normally.
>>>>>>>>
>>>>>>>>This same problem comes up (but to a lesser extent) when you think
>>>>>>>>about extending this system for semantics. What does an existing
>>>>>>>>actor do with semantic information stored in the token? It can
>>>>>>>>ignore it, but that may be detrimental to the analysis.
>>>>>>>>
>>>>>>>>Another possible option
>>>>>>>>-----------------------
>>>>>>>>The other possible solution to the missing value problem is to
>>>>>>>>simply not send any data over the port when a missing value is
>>>>>>>>encountered. I have modified the EML ingestion actor to
>>>>>>>>dynamically create one typed port for each attribute in the data
>>>>>>>>package. These ports can then be hooked up to other actors. The
>>>>>>>>data is sent asyncronously and depends on the receiving ports to
>>>>>>>>queue the data until all the input data is present to run the
>>>>>>>>analysis.
>>>>>>>>
>>>>>>>>If I simply do not send a token when a missing value comes up, I
>>>>>>>>forsee major timing problems. For instance, port A and port B are
>>>>>>>>mapped to input ports X and Y (res.) of a plotter. port A sends a
>>>>>>>>token to X, then B gets a missing value. It sends nothing. The
>>>>>>>>plotter is then waiting for its second input. the next record is
>>>>>>>>iterated into. port A sends another token to X. This causes an
>>>>>>>>exception. The other scenario is, on the second iteration, A is a
>>>>>>>>missing value but B is not. Then we are plotting two values from
>>>>>>>>different records when Y recieves data from B in the second
>>>>>>>>record. This would be a nightmare to deal with given the current
>>>>>>>>directors.
>>>>>>>>
>>>>>>>>So, does anyone see something that I'm missing here? What are the
>>>>>>>>needs of the semantic typing going to be as far as ptolemy goes?
>>>>>>>>Anyone have a better solution than the three that I've layed out?
>>>>>>>>This is a complex issue that I need to deal with before I can
>>>>>>>>continue moving forward with AMS. I don't want to do anything
>>>>>>>>that will hinder the future semantic extensions of ptolemy and
>>>>>>>>this is just too much of a basic infrastructure item to try to
>>>>>>>>hack. If anyone want to have an IRC chat about this, I'm on #seek.
>>>>>>>>
>>>>>>>>chad
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>
>
> _______________________________________________
> seek-dev mailing list
> seek-dev at ecoinformatics.org
> http://www.ecoinformatics.org/mailman/listinfo/seek-dev
--
-----------------------
Chad Berkley
National Center for
Ecological Analysis
and Synthesis (NCEAS)
berkley at nceas.ucsb.edu
-----------------------
More information about the Seek-dev
mailing list