230ec72609
Fix timestamp display issues with --annotate
2013-04-09 18:19:32 -04:00
e95142eabf
Huge update to support inserting in client.numpyclient, with tests
...
This includes both client.stream_insert_numpy and
client.stream_insert_numpy_context(). The test code is based on
similar test code for client.stream_insert_context, so it should be
fairly complete.
2013-04-08 18:51:45 -04:00
d21c3470bc
Client cleanups; fix tests to account for time epsilon = 1
2013-04-08 18:51:45 -04:00
cc211542f8
Add binary support to nilmdb.server; enforce content-type
2013-04-08 18:51:45 -04:00
8292dcf70b
Clean up stream/extract content-type and add a test for it
2013-04-08 18:51:45 -04:00
41ec13ee17
Rename bulkdata.append_string to bulkdata.append_data
2013-04-08 18:51:45 -04:00
c7f2df4abc
Add nilmdb.client.numpyclient.NumpyClient with stream_extract_numpy
...
This is a subclass of nilmdb.client.client.Client that adds numpy
specific routines, which should be a lot faster.
2013-04-07 17:43:52 -04:00
5b7409f802
Add binary extract to client, server, nilmdb, bulkdata, and rocket.
2013-04-07 16:06:52 -04:00
9ce0f69dff
Add "--delete" option to "nilmtool metadata" tool
...
This is the same as "--update" with an empty string as the value.
2013-04-02 16:07:28 -04:00
ebccfb3531
Fix stream renaming when the new path is a parent of the old
2013-04-01 19:25:17 -04:00
173121ca87
Switch URL to one that should definitely not resolve
2013-03-30 17:31:35 -04:00
26bab031bd
Add StreamInserter.send() to trigger intermediate block send
2013-03-30 17:30:43 -04:00
f8ca8d31e6
Remove Iteratorizer, as it's no longer needed
2013-03-24 21:31:03 -04:00
ed89d803f0
Remove aplotter code
2013-03-24 21:29:09 -04:00
3d24092cd2
Replace bare 'except:' with 'except: Exception'
...
Otherwise we might inadvertently catch SystemExit or KeyboardExit or
something we don't want to catch.
2013-03-24 21:28:01 -04:00
c0d450d39e
Add locking mechanism to avoid multiple servers on one DB
2013-03-24 21:20:20 -04:00
e790bb9e8a
Fix test failure when tests are run as root
2013-03-21 14:33:02 -04:00
89be6f5931
Add option to include interval start/end markup on extract
...
When enabled, lines like "# interval-start 1234567890123456" and "#
interval-end 1234567890123456" will be added to the data output. Note
that there may be an "interval-end" timestamp followed by an identical
"interval-start" timestamp, if the response at the nilmdb level was
split up into multiple chunks.
In general, assume contiguous data if previous_interval_end ==
new_interval_start.
2013-03-19 14:23:33 -04:00
4cdef3285d
Destroy now requires that all data has been previously removed.
...
Added new flag "-R" to command line to perform an automatic removal.
This should be the last of the ways in which a single command could
block the nilmdb thread for a long time.
2013-03-18 19:39:03 -04:00
bcd82c4d59
Limit the number of rows removed per call to nilmdb.stream_remove
...
Server class will retry as needed, as with stream_extract and
stream_intervals.
2013-03-18 18:22:45 -04:00
2d72891162
Accept "min" and "max" as timestamps on command line
2013-03-18 18:19:24 -04:00
0ef71c193b
Remove layout.pyx, since rocket replaced it
2013-03-15 22:32:40 -04:00
22274550ab
Test python version of Interval too
2013-03-15 21:37:03 -04:00
4f06d6ae68
Move Interval set_difference inside nilmdb.utils for clients
...
Clients might need to to Interval math too, so move a simple Interval
class and start putting helpers in there.
2013-03-15 21:37:03 -04:00
c54d8041c3
Update design docs
2013-03-15 21:07:01 -04:00
049375d30e
Fill out test coverage
2013-03-15 18:08:21 -04:00
88eb0123f5
Add test for Table.__getitem__ indexing
2013-03-15 18:08:21 -04:00
28e72fd53e
Remove Table.__getitem__; used only by tests
2013-03-15 18:08:21 -04:00
fb4f4519ff
Clean up and simplify Table.get_*, including __getitem__
2013-03-15 18:08:20 -04:00
90b96799ac
Bulk of the switch to int64 microsecond timestamps, including test data.
2013-03-15 15:08:58 -04:00
56679ad770
Move more datetime_tz calls into common code
2013-03-15 15:08:58 -04:00
b5541722c2
Continue moving time-handling code into nilmdb.utils.time
2013-03-15 15:08:58 -04:00
c7a712d8d8
Partial test for rounding issues
2013-03-15 15:08:57 -04:00
b9f0b35bbe
Stream renaming support, and comprehensive tests
...
Implemented in command line, client, server, nilmdb, bulkdata
2013-03-14 11:02:30 -04:00
b1b09f8cd0
Strengthen checks when creating paths, fix some bugs, and add tests
2013-03-13 17:45:47 -04:00
d467df7980
Add specific error for creating a path that already exists
2013-03-13 10:14:28 -04:00
b77f07a4cd
Fix reporting of parsing errors with malformed data
...
strtod() and friends will happily swallow newlines, so we need to skip
over spaces and look for that sort of thing manually.
2013-03-12 16:44:36 -04:00
83bc5bc775
Make rocket/bulkdata errors include column number and the bad data
...
The bad line is printed out on a new line, and a third line
with a ^ to indicate the position of the error.
2013-03-12 16:10:00 -04:00
7eca587fdf
Add 'nilmtool intervals' command, with --diff option
...
Can show the set-difference between the interval ranges in two
streams.
2013-03-11 17:07:26 -04:00
1d61d61a81
Add interval.set_difference function and associated tests
2013-03-11 15:40:50 -04:00
8e79998e95
Tune sqlite to use write-ahead-logging
...
Enable the following pragmas: synchronous=NORMAL, journal_mode=WAL.
This offers a significant speedup to INSERT times compared to
synchronous=FULL, and is roughly the same as synchronous=OFF
but should be a bit safer.
2013-03-11 15:13:43 -04:00
9f914598c2
Make /stream/list give some more extended info, like row count
...
Also changes the HTTP parameter from "extent" to "extended",
and the commandline parameter from "extent" to "ext".
2013-03-11 15:13:43 -04:00
232a3876c2
Clean up imports to separate client and server more.
...
"import nilmdb" doesn't do much; "import nilmdb.client" or "import
nilmdb.server" is now required.
2013-03-11 15:13:42 -04:00
1c27dd72d6
Fill out client tests and fix various bugs
...
Fixes various corner cases and other bugs regarding lines with
comments, having data but no endpoints, etc.
2013-03-08 12:36:17 -05:00
0fc092779d
Big rework of stream_insert_context and places that use it.
...
Things are now block-focused, rather than line-focused. This should
give a pretty big speedup to inserting client data, especially when
inserting preformatted data.
2013-03-07 20:30:11 -05:00
92724d10ba
Rework 'nilmtool insert' and some client stuff to speed up inserting data
...
Still needs work.
2013-03-06 20:49:14 -05:00
1d7acbf916
Remove null timestamper, speed up insert --none a tiny bit
2013-03-06 20:46:51 -05:00
ea3ea487bc
Merge branch 'rocket-insert'
...
Conflicts:
nilmdb/server/bulkdata.py
nilmdb/server/server.py
nilmdb/utils/__init__.py
2013-03-06 20:46:04 -05:00
69ad8c4842
Merge branch 'rocket'
2013-03-06 20:38:02 -05:00
65f09f793c
When re-raising exceptions in the server, preserve original tracebacks
2013-03-05 21:48:40 -05:00
