[kepler-dev] [Bug 3586] New: - COMAD traces should be reproducible when there is limited concurrency

[bugzilla-daemon at ecoinformatics.org]

http://bugzilla.ecoinformatics.org/show_bug.cgi?id=3586

           Summary: COMAD traces should be reproducible when there is
                    limited concurrency
           Product: Kepler
           Version: 1.0.0
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: general
        AssignedTo: mcphillips at ecoinformatics.org
        ReportedBy: mcphillips at ecoinformatics.org
         QAContact: kepler-dev at ecoinformatics.org


The ComadTest actor is roughly the COMAD equivalent of Ptolemy's NonStrictTest.
 It records the stream of data it receives during a run.  In training mode it
stores an XML representation (similar to a trace file) of this stream in a
parameter value. In test mode it throws an exception if the received stream
doesn't match the previously stored value of this parameter.

This works fine currently if the showDetails parameter is set to false, hiding
all token and object IDs in the stored XML representation.  However, this makes
it impossible to verify that the provenance records are correct because the dep
attribute is hidden (the value of the dep attribute is a list of the ids of
tokens on which a new item depends).  However, if the IDs are recorded then the
ComadTest actor almost always reports a mismatch because, e.g., the IDs in the
dep attribute are not guaranteed to be in a particular order, IDs are assigned
asynchronously in different actors, etc.

We do not expect traces for multiple runs of COMAD workflows to be exactly
reproducible in general due to concurrency.  Each actor runs in its own thread
and token IDs are pulled from a common pool.  But in a simple, three-actor test
workflow (CollectionComposer -> ActorToBeTested -> ComadTest), I believe
reproducibility to be both possible and desirable.