[kepler-dev] [Bug 5375] New: data not sent out after accumulating a large number of files in pending/

[bugzilla-daemon at ecoinformatics.org]

http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5375

           Summary: data not sent out after accumulating a large number of
                    files in pending/
           Product: Kepler
           Version: 2.1.0
          Platform: Other
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: sensor-view
        AssignedTo: barseghian at nceas.ucsb.edu
        ReportedBy: barseghian at nceas.ucsb.edu
         QAContact: kepler-dev at kepler-project.org
   Estimated Hours: 0.0


There seem to be a few problems associated with letting a large number of files
build up in spandata/pending/ (files buffer here until the next connection to
55056, at which point they should all be sent out).

1) The main issue is that after N files (hundreds?) have built up in pending,
connecting to 55056 does not cause the send to occur. At least this has
happened twice to me at span r24. Connecting to 55056 w/ spanToDT or telnet
makes no difference.

2) Errors in the log due to references to filenames that are simply the file
extension:
------
  50216 <<< Dequeue Data:[2011-04-05T11:40:51.670Z      CR800_sq311_2  
0.0000000000E+00
  50217 ]
  50218 current file will expired in 0 second(s)
  50219 waiting pending dir become available...
  50220 save data from [./spandata/current.csv] to
[./spandata/pending/4D9AFFC0.csv]
  50221 file: ./spandata/pending/.csv will be sent
  50222 cannot open file: ./spandata/pending/.csv
  50223 current file will expired in 25 second(s)
  50224 current file will expired in 20 second(s)
  50225 file: ./spandata/pending/.csv will be sent
  50226 cannot open file: ./spandata/pending/.csv
  50227 current file will expired in 15 second(s)
  50228 current file will expired in 10 second(s)
  50229 file: ./spandata/pending/.csv will be sent
  50230 cannot open file: ./spandata/pending/.csv
  50231 current file will expired in 5 second(s)
  50232 >>> Enqueue Data: 2011-04-05T11:41:21.670Z      CR800_Batt_Volt
1.3305263519E+01
----
Interestingly, pending/.csv never gets written afaict. Also, despite this, it
doesn't look like data is lost, e.g. 4D9AFFC0.csv does exist and contains data.

3) At a certain point expiration times go increasingly negative. E.g.:
cannot open file: ./spandata/pending/.csv
current file will expired in -29217 second(s)
current file will expired in -29222 second(s)
file: ./spandata/pending/.csv will be sent

-- 
Configure bugmail: http://bugzilla.ecoinformatics.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.