[seek-kr] Re: [seek-dev] data typing in ptolemy
Shawn Bowers
bowers at sdsc.edu
Mon Oct 13 09:20:14 PDT 2003
I am available (it is about 9:15). Either way is fine with me.
If we want to discuss semantic issues, I think it would be good to have
Rich involved in the discussion. So perhaps we should punt to Wed.
Shawn
On Mon, 13 Oct 2003, Matt Jones wrote:
> I can do it now, and would like to be involved. Or Wednesday is fine too.
>
> Matt
>
> Chad Berkley wrote:
> > Hey,
> >
> > Do we still want to have a conference call this morning or should we
> > continue this debate over email and have a call on wednesday when Rich
> > is around?
> >
> > chad
> >
> > Shawn Bowers wrote:
> >
> >> On Fri, 10 Oct 2003, Amy Sundermier wrote:
> >>
> >>
> >>> I'll quote some stuff from your email for clarification:
> >>>
> >>> "I think what is desired in Chad's example with extending the set of
> >>> values allowed in a Ptolemy token type, is to augment the edges in the
> >>> lattice so that ExtendedDouble is a subclass of Complex, and Double is a
> >>> subclass of ExtendedDouble.
> >>>
> >>> Then, if I say that my port accepts the ExtendedDouble type, and I
> >>> receive a Double token, no big deal. If I say that my port accepts
> >>> Double, I am guaranteed (by Ptolemy type checking) that I will never
> >>> receive null, or missing, values."
> >>>
> >>> I am having some trouble believing the second paragraph. If you declare
> >>> a port to accept an ExtendedDouble, are you sure it will typecheck ok to
> >>> pass it a subclass Double? I don't know the semantics of Ptolemy type
> >>> checking and substitutibility rules. Upcasting is usually ok but
> >>> downcasting needs explicit support. Do they have the concept of runtime
> >>> casting? I thought that was missing from the architecture according to
> >>> the documentation.
> >>
> >>
> >>
> >> I think it should be okay since ExtendedDouble would be a supertype of
> >> Double. If ExtendedDouble were a subctype, then we would be in trouble.
> >>
> >> Assuming ExtendedDouble is a supertype of Double, say I have an actor A2
> >> and it has one input port that has type ExtendedDouble and one output
> >> port
> >> that returns Integer. Then, the "signature" for A2 might look like this
> >> (using java-like syntax):
> >>
> >> Integer A2(ExtendedDouble arg)
> >>
> >> Then, if I have another actor A1 that takes a String and returns a
> >> Double, its signature looks like this:
> >>
> >> Double A1(String arg)
> >>
> >> And, I want to connect A2 and A1 such that the output of A1 is sent to
> >> A2, I have:
> >>
> >> String a1 = "..."
> >> Double a2 = A1(a1)
> >> Integer r = A2(a2).
> >>
> >> This is an "upcast" as you mentioned, which is what Ptolemy allows.
> >> The second case would be if there was an actor A3 that output an
> >> ExtendedDouble and an actor A4 that took as input a Double. Their
> >> signatures would look as follows.
> >>
> >> ExtendedDouble A3(String arg)
> >> Integer A4(Double arg)
> >>
> >> Then, if we connected them, it should give an error:
> >>
> >> String a3 = "..."
> >> ExtendedDouble a4 = A3(a3)
> >> Integer r = A4(a4)
> >>
> >> This is an example of "downcasting," which as you mentioned isn't
> >> supported in Ptolemy.
> >> Shawn
> >>
> >>
> >>
> >>
> >>> So I gather you are proposing that ExtendedDouble is basically an
> >>> abstract class with no attributes or methods that override superclass
> >>> methods, just a placeholder in the class hierarchy? Because if Double
> >>> is a subclass then it would inherit from ExtendedDouble any attributes
> >>> or methods that made it nullible. Only the class would be different.
> >>>
> >>
> >>
> >>
> >>
> >>> Thanks again for the explanation, Amy
> >>>
> >>> -----Original Message-----
> >>> From: Shawn Bowers [mailto:bowers at sdsc.edu]
> >>> Sent: Fri 10/10/2003 12:47 PM
> >>> To: Amy Sundermier
> >>> Cc: Chad Berkley; Rich Williams
> >>> Subject: RE: [seek-kr] Re: [seek-dev] data typing in ptolemy
> >>>
> >>> On Fri, 10 Oct 2003, Amy Sundermier wrote:
> >>>
> >>>
> >>>> I don't see how changing the TypeLattice will do much of anything for
> >>>> you. If you want to put new types in the system, you have to be
> >>>> able to
> >>>> create instances of both the Ptolemy classes and the Ptolemy
> >>>> objects. Why would you want to subvert the class hierarchy for
> >>>> existing types?
> >>>
> >>>
> >>> In Chad's original email, he suggested adding a new type to Ptolemy,
> >>> which he called ExtendedDouble that could support null, or missing
> >>> values. To add a new Ptolemy type, you *have* to change the type
> >>> lattice. Otherwise, it isn't recognized by Ptolemy as a new type.
> >>> I am basing my information on Chapter 12 of the Ptolemy documentation on
> >>> the Type System. As I understand it, an instance of TypeLattice defines
> >>> the type hierarchy for Ptolemy structured types. For example, in the
> >>> default type lattice object, there is an edge assignment:
> >>>
> >>> _basicLattice.addEdge(BaseType.DOUBLE, BaseType.COMPLEX);
> >>>
> >>> BaseType.DOUBLE and BaseType.COMPLEX are instances of DoubleType and
> >>> ComplexType. Each type class in Ptolemy has exactly one instance (i.e.,
> >>> they are singleton classes). The instance represents the actual type in
> >>> the Ptolemy environment, e.g., if I want to state that a port takes
> >>> tokens whose types are double (or subclasses of double), then I use
> >>> BaseType.DOUBLE to represent this.
> >>>
> >>> The statement above asserts that the Double type is a subtype of the
> >>> Complex type. The Ptolemy codebase itself doesn't reflect this type
> >>> system at all (Ptolemy *doesn't* use Java's reflection capabilities for
> >>> typing). For example, both classes DoubleType and ComplexType are
> >>> subclasses of BaseType, i.e., the class DoubleType is not a subclass of
> >>> ComplexType. The type lattice specifies exactly the desired
> >>> relationship.
> >>> Also, since everything is an object, the type heirarchy can be used
> >>> directly in Ptolemy for both design-time and runtime type checking of
> >>> Ptolemy ports. Ptolemy supports multiple inheritance in its type system,
> >>> which isn't possible in Java unless interfaces are used (which might be
> >>> one reason for their architecture).
> >>>
> >>> I don't think we want to subvert the class hierarchy, as you say
> >>> above, we
> >>> just want to add a new token/token type to the system. I believe
> >>> that the
> >>> type system is meant to be extensible (that is what I got from the
> >>> Chapter
> >>> above and from some of their other papers). Basically, if I want to
> >>> add a
> >>> class such as ExtendedDouble to Ptolemy's type system, I would subclass
> >>> BaseType, create an object of ExtendedDouble, and then add an edge to
> >>> the
> >>> new object in the lattice. I think what is desired in Chad's example
> >>> with
> >>> extending the set of values allowed in a Ptolemy token type, is to
> >>> augment
> >>> the edges in the lattice so that ExtendedDouble is a subclass of
> >>> Complex,
> >>> and Double is a subclass of ExtendedDouble.
> >>> Then, if I say that my port accepts the ExtendedDouble type, and I
> >>> receive a Double token, no big deal. If I say that my port accepts
> >>> Double, I am guaranteed (by Ptolemy type checking) that I will never
> >>> receive null, or missing, values.
> >>> In theory, doing this would not require augmenting any of Ptolemy's code
> >>> since we only need to pass around a different TypeLattice object,
> >>> which I
> >>> believe is the intention in Ptolemy. However, for some reason they
> >>> hard-coded the reference to a predefined Typelattice object, so it
> >>> doesn't quite work as they claim.
> >>>
> >>> Does that make sense?
> >>>
> >>> Shawn
> >>>
> >>>
> >>>
> >>>>
> >>>> -----Original Message-----
> >>>> From: Shawn Bowers [mailto:bowers at sdsc.edu]
> >>>> Sent: Fri 10/10/2003 9:28 AM
> >>>> To: Chad Berkley
> >>>> Cc: Amy Sundermier; Rich Williams
> >>>> Subject: Re: [seek-kr] Re: [seek-dev] data typing in ptolemy
> >>>>
> >>>>
> >>>> I am available on Monday, or pretty much any time next week for a
> >>>> conference call. Just let me know what you decide. I know the
> >>>> phones here can do three-way calling, but would need to figure out
> >>>> how to do it, which shouldn't be a problem (I think Ilkay has done
> >>>> it before).
> >>>>
> >>>> We can talk a bit about semantic typing as well, and some ideas for
> >>>> how to integrate semantic types in Ptolemy.
> >>>>
> >>>> Shawn
> >>>>
> >>>>
> >>>> On Fri, 10 Oct 2003, Chad Berkley wrote:
> >>>>
> >>>>
> >>>>> Hey Amy, Rich and Shawn (and others),
> >>>>>
> >>>>> Thanks for all your comments. Shawn and I had a pretty in-depth
> >>>>> discussion about this yesterday afternoon on irc and we hashed out
> >>>>> a few ideas. It seems we all have slightly different ideas about
> >>>>> what needs to be done so I'd like to propose having a conference
> >>>>> call on this issue. would sometime monday work? I'll propose 9:00
> >>>>> am PDT. If you have a problem with that time, propose a different
> >>>>> one. I can setup a vnc connection so we can have a whiteboard to
> >>>>> transfer ideas on. shawn or amy, can your phones do 3 way
> >>>>> calling? Mine does, do if one of your phones can do it, we can
> >>>>> chain together all 4 of us without having to spend
> >>>>> $.60/minute/participant on a conf. call. If anyone else wants to
> >>>>> engage in an extremely technical discussion about this issue, you
> >>>>> are more than welcome to join in (Bertram? Matt?).
> >>>>>
> >>>>> Amy, to answer your question about the Ptolemy source, we do have
> >>>>> the ptolemy source code, but I've been trying my best not to alter
> >>>>> it. We've realized that we may have to do so, but I'd like to avoid
> >>>>> that if it is at all possible. This would be a relatively simple
> >>>>> thing to fix if we just hacked the existing ptolemy source. (if
> >>>>> you want the source, by the way, it's freely available at
> >>>>> ptolemy.eecs.berkeley.edu).
> >>>>>
> >>>>> Rich, and Amy, you should look at the class
> >>>>> ptolemy.data.type.TypeLattice. Shawn was under the impression that
> >>>>> we could extend that class to subvert the entire class hierarchy
> >>>>> within ptolemy. we kind of went back and forth about this for a
> >>>>> while but it looks interresting. I'm going to try just hacking it
> >>>>> to see if changing the TypeLattice will indeed allow us to change
> >>>>> the hierarchy without changing the actual java class hierarchy
> >>>>> (something I'm still not convinced of....but Shawn found
> >>>>> documentation to that reguard).
> >>>>>
> >>>>> chad
> >>>>>
> >>>>> Amy Sundermier wrote:
> >>>>>
> >>>>>> Hi Chad (and mailing list recipients),
> >>>>>>
> >>>>>> Do you have the source for Ptolemy? It seems to me that one way
> >>>>>> to solve your "missing value" problem is to have a boolean in the
> >>>>>> base class of the hierarchy (Token) called "nullValue" that all
> >>>>>> subclasses inherit. Any token could then declare itself null by
> >>>>>> setting that boolean to true. Your code would have to check "if
> >>>>>> (token.isNullValue())" instead of "if (token == null)" but that's
> >>>>>> pretty self-documenting.
> >>>>>> As long as the Ptolemy infrastructure code is not using reflection
> >>>>>> to set and get values and making assumptions about what it will
> >>>>>> find, you should be able to modify a superclass in this way
> >>>>>> without breaking their code.
> >>>>>>
> >>>>>> This seems like an obvious solution so I wonder if you've made a
> >>>>>> policy decision not to modify the Ptolemy source?
> >>>>>> I'd be interested in learning more about the semantic typing
> >>>>>> issues alluded to by these emails. Looking forward to meeting you
> >>>>>> all.
> >>>>>>
> >>>>>> Amy Sundermier
> >>>>>> Arizona State University
> >>>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Chad Berkley [mailto:berkley at nceas.ucsb.edu]
> >>>>>> Sent: Thu 10/9/2003 10:46 AM
> >>>>>> To: seek-dev at ecoinformatics.org; seek-kr at ecoinformatics.org
> >>>>>> Cc:
> >>>>>> Subject: [seek-dev] data typing in ptolemy
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> Matt and I had a conversation on IRC the other day that we thought
> >>>>>> might be of interrest to those on these lists.
> >>>>>>
> >>>>>> Basically, I am now dealing with typing issues within ptolemy.
> >>>>>> Problems arise when you get missing values in the data. Ptolemy's
> >>>>>> type heirarchy does not allow missing values in a data tokens so
> >>>>>> Matt and I were talking about extending the ptolemy typing system
> >>>>>> to allow missing values. It occured to us that the typing system
> >>>>>> will need to be extended to allow for semantic typing in the future.
> >>>>>>
> >>>>>> The type class hierarchy currently looks like the following:
> >>>>>>
> >>>>>> Token
> >>>>>> |
> >>>>>> --------------------------
> >>>>>> | |
> >>>>>> ScalarToken AbstractConvertableToken
> >>>>>> | |
> >>>>>> ---------------...* -----------------
> >>>>>> | | | |
> >>>>>> DoubleToken IntToken BooleanToken StringToken
> >>>>>>
> >>>>>> *Note that ScalarToken also includes LongToken and ComplexToken.
> >>>>>>
> >>>>>> In addition to this Token hierarchy (Tokens are the means by which
> >>>>>> you pass data between actors over ports) there is also a port
> >>>>>> typing hierarchy implemented in the class BaseType. BaseType is
> >>>>>> the means by which you actually specify a port's type. It looks
> >>>>>> like this:
> >>>>>>
> >>>>>> BaseType
> >>>>>> |
> >>>>>> ---------------------------------------------------------....
> >>>>>> | | | | |
> >>>>>> BooleanType ComplexType GeneralType IntType DoubleType ....*
> >>>>>>
> >>>>>> * BaseType also includes EventType, LongType, NumericalType,
> >>>>>> ObjectType, SCalarType, StringType, UnknownType, UnsignedByteType
> >>>>>>
> >>>>>> Basically, in order to extend this typing system, we must extend
> >>>>>> both of these hierarchies since Tokens are the means by which data
> >>>>>> is transfered between ports and BaseTypes are the means by which
> >>>>>> you allow (or disallow) a port to accept different types of data.
> >>>>>>
> >>>>>> Extending the hierarchy
> >>>>>> -----------------------
> >>>>>> There are two different ways that I see to extend the hierarchy.
> >>>>>> The first is to extend the base class Token with our own tree of
> >>>>>> token types extending from the root of the tree. This will
> >>>>>> probably allow us the most flexibility in implementing types the
> >>>>>> way we need to, however, the main drawback I see to doing this is
> >>>>>> that we would not be able to use most existing actors because
> >>>>>> their ports are typed according to the current hierarchy. I think
> >>>>>> that one fact pretty much eliminates this approach from the options.
> >>>>>>
> >>>>>> The second approach I see is to extend each of the leaf token
> >>>>>> types. For example, extend DoubleToken to ExtendedDoubleToken and
> >>>>>> add our additional functionality there. This keeps our type
> >>>>>> system within the bounds of the current ptolemy hierarchy but
> >>>>>> limits our flexibility in extension. we are basically limited to
> >>>>>> the hierarchy that already exists. It is still unclear to me what
> >>>>>> the affects of doing this will be on existing actors. For
> >>>>>> instance, if we extend DoubleToken to allow missing values, and an
> >>>>>> actor with a port of BaseType.DoubleType gets an
> >>>>>> ExtendedDoubleToken, it would still need to be able to handle
> >>>>>> whatever value we assign as a missing value code. This is
> >>>>>> problematic, because we are then restricted to using an actual
> >>>>>> double value as a missing value code (i.e. -999.999) which we've
> >>>>>> always maintained was bad data practice. This could also cause
> >>>>>> problems because the actor cannot differentiate -999.999 from a
> >>>>>> normal value and will operate on it normally.
> >>>>>>
> >>>>>> This same problem comes up (but to a lesser extent) when you think
> >>>>>> about extending this system for semantics. What does an existing
> >>>>>> actor do with semantic information stored in the token? It can
> >>>>>> ignore it, but that may be detrimental to the analysis.
> >>>>>>
> >>>>>> Another possible option
> >>>>>> -----------------------
> >>>>>> The other possible solution to the missing value problem is to
> >>>>>> simply not send any data over the port when a missing value is
> >>>>>> encountered. I have modified the EML ingestion actor to
> >>>>>> dynamically create one typed port for each attribute in the data
> >>>>>> package. These ports can then be hooked up to other actors. The
> >>>>>> data is sent asyncronously and depends on the receiving ports to
> >>>>>> queue the data until all the input data is present to run the
> >>>>>> analysis.
> >>>>>>
> >>>>>> If I simply do not send a token when a missing value comes up, I
> >>>>>> forsee major timing problems. For instance, port A and port B are
> >>>>>> mapped to input ports X and Y (res.) of a plotter. port A sends a
> >>>>>> token to X, then B gets a missing value. It sends nothing. The
> >>>>>> plotter is then waiting for its second input. the next record is
> >>>>>> iterated into. port A sends another token to X. This causes an
> >>>>>> exception. The other scenario is, on the second iteration, A is a
> >>>>>> missing value but B is not. Then we are plotting two values from
> >>>>>> different records when Y recieves data from B in the second
> >>>>>> record. This would be a nightmare to deal with given the current
> >>>>>> directors.
> >>>>>>
> >>>>>> So, does anyone see something that I'm missing here? What are the
> >>>>>> needs of the semantic typing going to be as far as ptolemy goes?
> >>>>>> Anyone have a better solution than the three that I've layed out?
> >>>>>> This is a complex issue that I need to deal with before I can
> >>>>>> continue moving forward with AMS. I don't want to do anything
> >>>>>> that will hinder the future semantic extensions of ptolemy and
> >>>>>> this is just too much of a basic infrastructure item to try to
> >>>>>> hack. If anyone want to have an IRC chat about this, I'm on #seek.
> >>>>>>
> >>>>>> chad
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >>>
> >
> >
>
>
More information about the Seek-dev
mailing list