[seek-dev] [Bug 1070] New: - propose a flexible system for transformation steps

[bugzilla-daemon@ecoinformatics.org]

http://bugzilla.ecoinformatics.org/show_bug.cgi?id=1070

           Summary: propose a flexible system for transformation steps
           Product: SEEK
           Version: unspecified
          Platform: Other
        OS/Version: other
            Status: NEW
          Severity: normal
          Priority: P2
         Component: seek-ams
        AssignedTo: jones at nceas.ucsb.edu
        ReportedBy: jones at nceas.ucsb.edu
         QAContact: seek-dev at ecoinformatics.org


In any workflow, linking any two steps is likely to require some sort of
transformation to get the outputs of one step to match the inputs of the next. 
This transformation is sometimes a simple cast (e.g., int -> float), sometimes a
more complex cast (e.g., string -> date), sometimes will require more complex
processing (e.g., statistical processing), and sometimes will require schema
transformations.  In all cases, the transformation between step A and step Z can
be represented as one or more intermediate tansformation steps:
  A -> T1 -> T2 -> TN -> Z

Our challenge includes:

1) use the SMS to locate candidate transformation steps T1..TN based on type
signature and ontologies
2) determine how to generate transformation steps automatically for simple
transforms such as unit conversions
3) create a simple GUI for creating transformation steps that map between two
existing steps
4) determine the pros and cons of having transformation steps be directly
associated with links (e.g., a link property) rather than simply introducing new
transform steps that do the same tasks directly into the pipeline

So, the objective of this bug is to:
1) propose a flexible system for discovering and generating transformation steps
2) identify the suite of complex issues involved in generating a mapping from
one step to another (especially when this might require several intermediate steps)
3) propose a plan of work for completing this plan