a547ddbbba
Change table.get_timestamp to table.__getitem__
...
This lets us use simple indexing to get timestamps from the table,
which allows us to use 'bisect' directly without needing a proxy class.
2013-03-15 18:08:21 -04:00
28e72fd53e
Remove Table.__getitem__; used only by tests
2013-03-15 18:08:21 -04:00
f63107b334
Add rocket.extract_timestamp to speed up bisections
2013-03-15 18:08:21 -04:00
955d7aa871
Remove floating port time support from nilmdb.utils.time
2013-03-15 18:08:21 -04:00
b8d2cf1b78
Consolidate rocket._extract_handle.params with extract_string
2013-03-15 18:08:21 -04:00
7c465730de
Remove rocket.extract_pyobject
2013-03-15 18:08:21 -04:00
aca130272d
Remove rocket.extract_list
2013-03-15 18:08:21 -04:00
76e5e9883f
Remove Table.append, rocket.append_iter
2013-03-15 18:08:20 -04:00
fb4f4519ff
Clean up and simplify Table.get_*, including __getitem__
2013-03-15 18:08:20 -04:00
30328714a7
Remove python implementation of rocket
2013-03-15 18:08:20 -04:00
759466de4a
Merge branch 'timestamp-integers'
2013-03-15 18:07:51 -04:00
d3efb829b5
Try to parse timestamps as double, if int64 parse fails
2013-03-15 15:19:41 -04:00
90b96799ac
Bulk of the switch to int64 microsecond timestamps, including test data.
2013-03-15 15:08:58 -04:00
56679ad770
Move more datetime_tz calls into common code
2013-03-15 15:08:58 -04:00
b5541722c2
Continue moving time-handling code into nilmdb.utils.time
2013-03-15 15:08:58 -04:00
aaea105861
Consolidate most timestamp <-> string conversions (outside of rocket)
2013-03-15 15:08:57 -04:00
e6a081d639
Consolidate timestamp constants into nilmdb.utils.time
2013-03-15 15:08:57 -04:00
1835d03412
Bump bulkdata database version to 3
2013-03-15 15:08:57 -04:00
c7a712d8d8
Partial test for rounding issues
2013-03-15 15:08:57 -04:00
20d315b4f7
Add documentation about upcoming timestamp changes
2013-03-15 15:08:57 -04:00
a44a5e3135
Merge branch 'argcomplete'
2013-03-15 15:08:42 -04:00
039b2a0557
Include nilmtool-bash-completion.sh script in .tar.gz
2013-03-15 15:08:28 -04:00
cd1dfe7dcd
Add completion functions to most commandline arguments
2013-03-15 14:26:38 -04:00
fb35517dfa
Add basic argument completion
2013-03-15 13:57:35 -04:00
b9f0b35bbe
Stream renaming support, and comprehensive tests
...
Implemented in command line, client, server, nilmdb, bulkdata
2013-03-14 11:02:30 -04:00
b1b09f8cd0
Strengthen checks when creating paths, fix some bugs, and add tests
2013-03-13 17:45:47 -04:00
d467df7980
Add specific error for creating a path that already exists
2013-03-13 10:14:28 -04:00
09bc7eb48c
Make StreamInserter.insert complain if data remains after send
...
Previously, we ignored problems when sending intermediate blocks,
since getting more data might make the next attempt succeed.
But in practice, malformed data would just build up, causing
problems. Raise an exception if there's too much data remaining
after trying to send an intermediate block.
2013-03-12 18:45:56 -04:00
b77f07a4cd
Fix reporting of parsing errors with malformed data
...
strtod() and friends will happily swallow newlines, so we need to skip
over spaces and look for that sort of thing manually.
2013-03-12 16:44:36 -04:00
59f0076306
Increase max layout count in rocket
2013-03-12 16:10:29 -04:00
83bc5bc775
Make rocket/bulkdata errors include column number and the bad data
...
The bad line is printed out on a new line, and a third line
with a ^ to indicate the position of the error.
2013-03-12 16:10:00 -04:00
6b1dfec828
In stream_list, return 0 instead of None for rows and seconds
...
For rows and seconds only. Extents still give None if they don't
exist.
2013-03-11 19:37:52 -04:00
d827f41fa5
Fix Makefile omission
2013-03-11 17:42:02 -04:00
7eca587fdf
Add 'nilmtool intervals' command, with --diff option
...
Can show the set-difference between the interval ranges in two
streams.
2013-03-11 17:07:26 -04:00
a351bc1b10
Add client, server, nilmdb support for listing interval differences
2013-03-11 17:07:08 -04:00
1d61d61a81
Add interval.set_difference function and associated tests
2013-03-11 15:40:50 -04:00
755255030b
Clean up interval __and__ function; we don't need to __and__ sets
2013-03-11 15:15:43 -04:00
8e79998e95
Tune sqlite to use write-ahead-logging
...
Enable the following pragmas: synchronous=NORMAL, journal_mode=WAL.
This offers a significant speedup to INSERT times compared to
synchronous=FULL, and is roughly the same as synchronous=OFF
but should be a bit safer.
2013-03-11 15:13:43 -04:00
9f914598c2
Make /stream/list give some more extended info, like row count
...
Also changes the HTTP parameter from "extent" to "extended",
and the commandline parameter from "extent" to "ext".
2013-03-11 15:13:43 -04:00
0468b04538
Fix pyrocket to handle comments better
2013-03-11 15:13:43 -04:00
232a3876c2
Clean up imports to separate client and server more.
...
"import nilmdb" doesn't do much; "import nilmdb.client" or "import
nilmdb.server" is now required.
2013-03-11 15:13:42 -04:00
1c27dd72d6
Fill out client tests and fix various bugs
...
Fixes various corner cases and other bugs regarding lines with
comments, having data but no endpoints, etc.
2013-03-08 12:36:17 -05:00
de5e474001
Update benchmarks in design.md
2013-03-07 20:33:30 -05:00
0fc092779d
Big rework of stream_insert_context and places that use it.
...
Things are now block-focused, rather than line-focused. This should
give a pretty big speedup to inserting client data, especially when
inserting preformatted data.
2013-03-07 20:30:11 -05:00
7abfdfbf3e
Add const qualifier to strings we get from Python
2013-03-07 16:27:07 -05:00
92724d10ba
Rework 'nilmtool insert' and some client stuff to speed up inserting data
...
Still needs work.
2013-03-06 20:49:14 -05:00
1d7acbf916
Remove null timestamper, speed up insert --none a tiny bit
2013-03-06 20:46:51 -05:00
ea3ea487bc
Merge branch 'rocket-insert'
...
Conflicts:
nilmdb/server/bulkdata.py
nilmdb/server/server.py
nilmdb/utils/__init__.py
2013-03-06 20:46:04 -05:00
69ad8c4842
Merge branch 'rocket'
2013-03-06 20:38:02 -05:00
0047e0360a
Implement Rocket.append_string() in C; misc cleanups along the way
...
This should more or less complete the rocket interface.
2013-03-06 15:50:00 -05:00
