[kepler-dev] Bug 2240: add support for null values to data passing among ports

[Edward A. Lee]

This solution sounds reasonable to me...

Edward

At 06:36 PM 12/12/2005 -0800, Christopher Brooks wrote:
>Hi Edward,
>
>One of the Kepler bugs blocking the Kepler release is:
>http://bugzilla.ecoinformatics.org/show_bug.cgi?id=2240
>which is reproduced below.
>
>Basically, Kepler needs to handle missing data because not all
>ecological data sets are complete.
>
>The solution to this problem needs to work in PN and SDF.
>
>The Synchronous/Reactive (SR) domain has the notion of absent values.
>For example, domains/sr/lib/Absent.java has this class comment:
>
>  This actor outputs absent values.  That is, it produces no tokens,
>  and it calls the sendClear() method of the output port on each
>  firing.
>
>The sendClear() method is the IOPort.java sendClear() method.
>IOPort also has sendClearInside() and broadcastClear().
>
>Do you have any comments?
>
>Below is the text from the bug report:
>Below is the text from the bug report:
>--start--
>Currently ptolemy and kepler do not support passing null values
>(sometimes called missing values) among ports, even though this is
>common in analytical systems like R and SAS.  The concept of null is
>not even defined in the token types. This causes a real problem for
>data sources that are sparsely populated, as well as data streams that
>result from data integration operations that might produce null
>values.  We need to extend the underlying token representation to
>include a concept of null values and the actor framework to protect
>existing actors that might not know how to handle null values.
>Because nulls cannot currently be represented in Kepler, none of the
>existing actors support them.  An exception is thrown whenever a
>missing value is detected by the EML data source, and workflow
>execution ceases.
>
>Bowers and Jones discussed one possible partial solution to this on
>IRC, which is summarized here.
>
>1) Override the Token base class to support null values by providing
>    two new methods:
>
>    Token.null() sets the token's value to null
>    boolean Token.isNull() returns true if the token has been set to
>    null
>
>2) Override TypedIOPort to add a new method that takes a boolean
>    "dropNull"
>
>     a) by default this could be set to "true" then the existing get()
>    method would be reimplemented to call the new one with a default of
>    "true" because we can assume that no existing actor written can
>    handle null values, since there is no way to pass null values, and
>    so existing calls to get() will invisibly drop null values.
>
>     b) if an actor can handle null values, then it passes "false"
>     which indicates that the actor knows how to deal with nulls and
>     wants to receive them
>
>so the changes to IOPort (or maybe TypedIOPOrt) would be:
>
>IOPOrt.get(channelindex) becomes get(channelIndex, dropNullValues)
>and
>IOPort.get(channelIndex, int vectorLength) becomes get(channelIndex,
>vectorLength, dropNullValues)
>
>so, for example, the new implmenetation of get(channelIndex) would
>simply call get(channelIndex, true), so existing actors would not even
>notice the change.
>--end--
>
>_Christopher

------------
Edward A. Lee
Professor, Chair of the EE Division, Associate Chair of EECS
231 Cory Hall, UC Berkeley, Berkeley, CA 94720
phone: 510-642-0253 or 510-642-0455, fax: 510-642-2845
eal at eecs.Berkeley.EDU, http://ptolemy.eecs.berkeley.edu/~eal