[kepler-dev] [Ptango] Re: ObjectToRecord related questions

Tue Jul 24 06:05:19 PDT 2012

I agree with Marten that these problems have probably been solved.
Basically, the type system now infers types based on how the data
is used downstream, resolving to the least constrained type that
is consistent with the usage.

Thus, if downstream you extract a field from a record, then the
upstream type will be resolved to be a record with at least that
field...

Edward

On 7/23/12 5:41 PM, Marten Lohstroh wrote:
> I still had conversation marked for answering, but I think all the
> problems mentioned below are solved by the new JSONToToken and
> TokenToJSON actors, along with the changes to the type system that
> allow connecting these actors without declaring a type on the output
> port. Instead, now the type of the output port is inferred from the
> declared or resolved type of the connected receiver's input ports.
>
> Runtime errors can still occur if the JSON-data is insufficient to
> meet the requirements of the type constraints. However, when this
> happens, the error is produced on the sender's side, whereas before
> this would be on the receiver's side. This allows for more
> sophisticated error handling policies than just crash. The error
> handling however is still in development..,
>
> Marten
>
> On Thu, Apr 12, 2012 at 8:24 AM, Hogan, D. (GE Energy)<D.Hogan at ge.com>  wrote:
>> Beth,
>>
>>
>>
>> For your first example, I have a similar use case with some Fortran programs
>> that use namelists for I/O.
>>
>>
>>
>>> For other web data streams, the type signature changes.  Here we currently
>>> set the output port type to an empty record so at least the type solver
>>> knows it’s a record, but then we can’t connect this output to a
>>> RecordDisassembler – it would throw an exception when the model is executed
>>
>>
>>
>> Are you saying there’s no way to connect to a RecordDisassembler, or there’s
>> no way to connect to it without manually forcing the output port types
>> (which can lead to runtime errors)?  I have found the latter to be true, but
>> maybe we are talking about different things.
>>
>>> Also like you mentioned for objects, JSON is fairly close to a Ptolemy
>>> record, but like Java types, the JSON types are not exactly the same as
>>> Ptolemy types.  (For example, it looks possible that JSON arrays could
>>> contain mixed types in the same array).  We don’t have a good solution for
>>> that yet…  right now we are just extracting Ptolemy types and hoping for the
>>> best.
>>
>> I was thinking along the lines of a general tool to handle standard Java
>> data type conversions.  Take that tool and extend it to speak Ptolemy and
>> perhaps JSON types too.  For example, maybe we could teach
>> http://transmorph.sourceforge.net how to deal with Ptolemy tokens/types.  If
>> you want JSON support, you can initialize it with a different set of
>> converters than the default.  There would one place for all of the standard
>> Java data type conversions including generics (via TypeReferences to work
>> around type erasure) and convert to/from Ptolemy types.  For example, it
>> would be able to handle converting back and forth between a
>> Map<String,Map<String,Double>>  and a RecordToken or a List<List<Integer>>
>> and ArrayToken or IntMatrixToken.  It could be useful on simple cases too
>> like int[] to/from ArrayToken.
>>
>>
>>
>> That type of code is ugly, but it would help with quickly creating actors
>> that interface with existing code.
>>
>>
>>
>> From: Christopher Brooks [mailto:cxh at eecs.berkeley.edu]
>> Sent: Wednesday, April 04, 2012 12:05 PM
>> To: Hogan, D. (GE Energy)
>> Cc: kepler-dev at kepler-project.org; ptango at chess.eecs.berkeley.edu
>> Subject: Re: [Ptango] Re: [kepler-dev] ObjectToRecord related questions
>>
>>
>>
>>
>> Beth Latronico asked me to forward on the following about this issue.
>>
>> Beth writes:
>>
>> We ran into a similar problem trying to build an actor which reads
>> information in JSON from a web page and outputs a RecordToken.  Here’s a
>> summary of the discussion and a sample model in case this is helpful for
>> your situation!
>>
>> Actor:  ptolemy.actor.lib.conversions.JSONToRecord
>>
>> Test model:  ptolemy.actor.lib.conversions.test.auto.JSONToRecord1.xml
>>
>>
>>
>> Ideally we wanted to be able to set the type of the output to a RecordType
>> including all of the fields and types of those fields (vs. an empty Record),
>> so that we could use the RecordDisassembler.  However, since the type solver
>> only runs once and runs during preinitialize(), this means we need to get
>> the type information somehow ahead of time, and that the type signature
>> cannot change during execution.  (It’s OK if the data changes).
>>
>>
>>
>> For some web data streams, this seems OK – the type signature is always the
>> same – so we pull some sample data in preinitialize() to determine the type,
>> and then check the type signature in fire() to make sure it has not changed.
>> This assumes that we know the URL of the data source, the URL remains fixed
>> and that sample data is available before the model executes.
>>
>>
>>
>> For other web data streams, the type signature changes.  Here we currently
>> set the output port type to an empty record so at least the type solver
>> knows it’s a record, but then we can’t connect this output to a
>> RecordDisassembler – it would throw an exception when the model is executed.
>>
>> Token value = new RecordToken();
>>
>> output.setTypeEquals(value.getType());
>>
>>
>>
>> So, we’re debating on whether 1) these two cases should be split into
>> different actors, and 2) if it’s appropriate to use RecordDisassembler or
>> should we create some new actors that can handle loosely-typed records.
>>
>>
>>
>> Also like you mentioned for objects, JSON is fairly close to a Ptolemy
>> record, but like Java types, the JSON types are not exactly the same as
>> Ptolemy types.  (For example, it looks possible that JSON arrays could
>> contain mixed types in the same array).  We don’t have a good solution for
>> that yet…  right now we are just extracting Ptolemy types and hoping for the
>> best.
>>
>>
>>
>> Unfortunately there are more questions than answers at the moment, but hope
>> the discussion highlights some of the issues!
>>
>>
>>
>> Best,
>>
>> Beth
>>
>> Marten Lohstroh suggested that we could have a flag that when set required
>> strict typing.  Certain actors could record the type information in a manner
>> similar to how the Test actor works.  The Test actor has a shared parameter
>> called "trainingMode".  When trainingMode is true, the known good results of
>> the TestActor is recorded.  In a similar manner, certain actors could record
>> the type information when trainingMode was set to true.
>>
>>
>> _Christopher
>>
>> On 4/2/12 6:05 PM, Edward A. Lee wrote:
>>
>>
>> Ptolemy II is statically typed, so after a model is preinitialized,
>> the types of all ports must be known. The type system is described in
>> chapter 5 of this document:
>>
>> http://www.eecs.berkeley.edu/Pubs/TechRpts/2008/EECS-2008-29.html
>>
>> and also in this paper:
>>
>> http://chess.eecs.berkeley.edu/pubs/665.html
>>
>> For Record types, the most general record type is the empty record type.
>> That is, a record type {fieldName = double}, for example, is also
>> an instance of {}, the empty record type. Moreover, a record of
>> type {a = int, b = double} is also an instance of {a = int}.
>>
>> So, for example, if you know that your output record will contain
>> a field named "a" of type "int", then you can force the output type
>> to be {a = int} (right click on the actor, select Configure Ports,
>> and enter "{a = int}" in the type field). As long as there is actually
>> such a field, in the record, this will work. But if you run the model
>> and there is no such field, I believe you will get a run-time type
>> error (which is probably what you want).
>>
>> Edward
>>
>>
>> On 4/2/12 1:51 PM, Hogan, D. (GE Energy) wrote:
>>
>> I have an actor that uses the structure of a file generated at runtime
>> to create an output record.  The actor is working, but I have a few
>> questions on ObjectToRecord since it is similar.
>>
>> In ObjectToRecord, it does not change the type and lists it as fixme.
>> The documentation for RecordType mentions using setTypeAtMost(new
>> RecordType(new String[0], new Type[0])) when you want to specify a
>> record without specifying the fields.  Should I use that?  Is there a
>> better way to specify it is a record and the field names/types are
>> discovered at runtime?  Since unknown matches that type constraint, what
>> is the best way to handle this?
>>
>> ObjectToRecord doesn't try to convert between standard Java types and
>> Ptolemy types.  I noticed there is a
>> ptolemy.data.expr.ConversionUtilities, but it looks limited to the
>> expression language needs.  Is there another conversion in Kepler?
>>
>> _______________________________________________
>> Kepler-dev mailing list
>> Kepler-dev at kepler-project.org
>> http://lists.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-dev
>>
>>
>>
>> --
>>
>> Christopher Brooks, PMP                       University of California
>>
>> CHESS Executive Director                      US Mail: 337 Cory Hall
>>
>> Programmer/Analyst CHESS/Ptolemy/Trust        Berkeley, CA 94720-1774
>>
>> ph: 510.643.9841                                (Office: 545Q Cory)
>>
>> home: (F-Tu) 707.665.0131 cell: 707.332.0670
>
> _______________________________________________
> Ptango mailing list
> Ptango at chess.eecs.berkeley.edu
> http://chess.eecs.berkeley.edu/ptango/listinfo/ptango
-------------- next part --------------
A non-text attachment was scrubbed...
Name: eal.vcf
Type: text/x-vcard
Size: 342 bytes
Desc: not available
URL: <http://lists.nceas.ucsb.edu/kepler/pipermail/kepler-dev/attachments/20120724/577514b2/attachment.vcf>