Commit Graph

12 Commits

Author SHA1 Message Date
89be6f5931 Add option to include interval start/end markup on extract
When enabled, lines like "# interval-start 1234567890123456" and "#
interval-end 1234567890123456" will be added to the data output.  Note
that there may be an "interval-end" timestamp followed by an identical
"interval-start" timestamp, if the response at the nilmdb level was
split up into multiple chunks.

In general, assume contiguous data if previous_interval_end ==
new_interval_start.
2013-03-19 14:23:33 -04:00
90b96799ac Bulk of the switch to int64 microsecond timestamps, including test data. 2013-03-15 15:08:58 -04:00
84e21ff467 Move ASCII data parsing from the server to the rocket interface.
The server buffers the string and passes it to nilmdb.  Nilmdb passes
the string to bulkdata.  Bulkdata uses the rocket interface to parse
it in chunks, as necessary.  Everything gets passed back up and
everyone is happy.

Currently, only pyrocket implements append_string.
2013-03-05 17:51:17 -05:00
11b228f77a Convert times to microsecond precision strings more consistently.
Use a new helper, nilmdb.utils.time.float_to_time_string().
This will help if we ever want to change representation (like using
uint64 microseconds since epoch, which saves us from having to
waste bits on the floating-point exponent)
2013-03-05 17:07:39 -05:00
0b75da7a8f Normalize the floating point formats to %.6e and %.16e
This is mostly a matter of taste, but it matches more closely with the
old way that prep did it, and it's more consistent.  It should roughly
match the available precision of floats and doubles.
2013-03-03 21:43:04 -05:00
9b6de6ecb7 Replace old layout strings everywhere 2013-03-03 13:37:02 -05:00
fad23ebb22 Add --timestamp-raw option to extract and list 2013-01-24 16:03:38 -05:00
9bf213707c Properly return an error if two timestamps are equal 2013-01-22 18:35:18 -05:00
e5d3deb6fe Removal support is complete.
`nrows` may change if you restart the server; documented why this is
the case in the design.md file.  It's not a problem.
2013-01-09 23:26:59 -05:00
27f8dcf06d Extract works now and we're back to 100% test coverage! It's slow,
though; need to figure out where the slowdown lies.

Add stream existence check to server's /intervals and /extract paths,
add tests for it.

Make start and end arguments optional for /extract, like /intervals

Move --quiet command line option to just the insert subcommand.
It's the only one that uses it right now, and otherwise it doesn't
show up in after a "nilmtool.py intervals --help".  Might revisit this
later if more commands start supporting --quiet.

Change cmdline/extract's write into a print, to keep the trailing
newline.

Fix lingering uses of Interval in nilmdb and change to DBInterval
instead.

Fix nilmdb interval bisection:
  - handle common case optimization correctly
  - db_endpos is always one after the last row, so use hi=db_endpos-1

Finish nlimdb stream_extract

Add a bunch of cmdline tests for extract, particularly testing border
cases around start/end.  Compares output to a set of files stored in
the tests/data dir.

Some more tests in test_client to get better coverage.



git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10893 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-05-30 18:24:36 +00:00
881b9a7bd1 add timestamped version
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10708 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-09 20:49:08 +00:00
09340d543d Add some test prep data
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10641 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-23 14:56:52 +00:00