Commit Graph

768 Commits

Author SHA1 Message Date
49d04db1d6 Allow start==end in stream_insert_context, if no data was provided. 2013-04-11 13:25:37 -04:00
ea838d05ae Warn against reused context managers, and fix broken tests 2013-04-11 13:25:00 -04:00
f2a48bdb2a Test binary extract; fix bugs 2013-04-11 13:24:11 -04:00
6d14e0b8aa Allow binary extract 2013-04-11 11:30:41 -04:00
b31b9327b9 Add tool to fix oversize files (the bug fixed by b98ff13) 2013-04-11 11:02:53 -04:00
b98ff1331a Fix bug where too much data was getting written to each file.
We were still calculating the maximum number of rows correctly,
so the extra data was really extra and would get re-written to the
beginning of the subsequent file.

The only case in which this would lead to database issues is if the
very last file was lengthened incorrectly, and the "nrows" calculation
would therefore be wrong when the database was reopened.  Still, even
in that case, it should just leave a small gap in the data, not cause
any errors.
2013-04-10 23:22:03 -04:00
00e6ba1124 Avoid ENOENT in nilmdb.utils.diskusage.du
ENOENT might show up if we're actively deleting files in the nilmdb
thread while trying to read available space from e.g. the server
thread.
2013-04-10 22:25:22 -04:00
01029230c9 Tweaks to sorting 2013-04-10 19:59:38 -04:00
ecc4e5ef9d Improve test coverage 2013-04-10 19:08:05 -04:00
23f31c472b Split sort_streams_nicely into separate file 2013-04-10 19:07:58 -04:00
a1e2746360 Fix bug in nilmdb.stream_remove with max_removals 2013-04-10 18:37:21 -04:00
1c40d59a52 server: use a generator in /stream/remove
Instead of returning a single number at the end of N nilmdb calls, we
now use a generator that returns one line of text every time there's a
new count of rows removed.  This ensures that the connection will stay
alive for very long removals.
2013-04-10 18:11:58 -04:00
bfb09a189f Fix coverage 2013-04-10 16:33:08 -04:00
416a499866 Support wildcards for destroy 2013-04-10 16:23:07 -04:00
637d193807 Fix unicode processing of command line arguments 2013-04-10 16:22:51 -04:00
b7fa5745ce nilmtool list: allow multiple paths to be supplied 2013-04-10 15:34:33 -04:00
0104c8edd9 nilmtool remove: allow wildcards and multiple paths 2013-04-10 15:27:46 -04:00
cf3b8e787d Add test for wrong number of fields in numpy insert 2013-04-10 15:06:50 -04:00
83d022016c nilmtool list: add new --layout option to show layouts 2013-04-10 14:58:44 -04:00
43b740ecaa nilmtool list: remove old -p parameter 2013-04-10 14:48:23 -04:00
4ce059b920 Give a slightly more clear error on bad array sizes 2013-04-09 19:56:58 -04:00
99a4228285 Set up default SIGPIPE handler
This lets you do something like "nilmtool extract | head" without
triggering backtraces.
2013-04-09 18:25:09 -04:00
230ec72609 Fix timestamp display issues with --annotate 2013-04-09 18:19:32 -04:00
d36ece3767 Fix up dependencies 2013-04-08 18:53:13 -04:00
231963538e Add some info about binary interface to design docs 2013-04-08 18:53:13 -04:00
b4d6aad6de Merge branch 'binary' 2013-04-08 18:52:52 -04:00
e95142eabf Huge update to support inserting in client.numpyclient, with tests
This includes both client.stream_insert_numpy and
client.stream_insert_numpy_context().  The test code is based on
similar test code for client.stream_insert_context, so it should be
fairly complete.
2013-04-08 18:51:45 -04:00
d21c3470bc Client cleanups; fix tests to account for time epsilon = 1 2013-04-08 18:51:45 -04:00
7576883f49 Add basic binary support to client, and restructure a bit 2013-04-08 18:51:45 -04:00
cc211542f8 Add binary support to nilmdb.server; enforce content-type 2013-04-08 18:51:45 -04:00
8292dcf70b Clean up stream/extract content-type and add a test for it 2013-04-08 18:51:45 -04:00
b362fd37f6 Add binary option to nilmdb.stream_insert 2013-04-08 18:51:45 -04:00
41ec13ee17 Rename bulkdata.append_string to bulkdata.append_data 2013-04-08 18:51:45 -04:00
efa9aa9097 Add binary option to bulkdata.append_string 2013-04-08 18:51:45 -04:00
d9afb48f45 Make append_binary signature look like append_string 2013-04-08 18:51:44 -04:00
d1140e0f16 Timestamps are int64, not uint64 2013-04-08 18:51:44 -04:00
6091e44561 Fix fread return value check 2013-04-08 18:51:44 -04:00
e233ba790f Add append_binary to rocket 2013-04-08 18:51:44 -04:00
f0304b4c00 Merge branch 'binary' into HEAD 2013-04-07 18:08:10 -04:00
60594ca58e Numpy is required for tests now, due to nilmdb.client.numpyclient
Still allow installation without it, though.
2013-04-07 18:05:43 -04:00
c7f2df4abc Add nilmdb.client.numpyclient.NumpyClient with stream_extract_numpy
This is a subclass of nilmdb.client.client.Client that adds numpy
specific routines, which should be a lot faster.
2013-04-07 17:43:52 -04:00
5b7409f802 Add binary extract to client, server, nilmdb, bulkdata, and rocket. 2013-04-07 16:06:52 -04:00
06038062a2 Fix error in time parsing 2013-04-06 19:12:17 -04:00
ae9fe89759 Parse timestamps with '@' before any other checks 2013-04-04 14:43:18 -04:00
04def60021 Include stream path in "no such stream" errors 2013-04-02 21:06:49 -04:00
9ce0f69dff Add "--delete" option to "nilmtool metadata" tool
This is the same as "--update" with an empty string as the value.
2013-04-02 16:07:28 -04:00
90c3be91c4 Natural sort for streams in client.stream_list 2013-04-02 14:37:32 -04:00
ebccfb3531 Fix stream renaming when the new path is a parent of the old 2013-04-01 19:25:17 -04:00
e006f1d02e Change default URL to http://localhost/nilmdb/ 2013-04-01 18:04:31 -04:00
5292319802 server: consolidate time processing and checks 2013-03-30 21:16:40 -04:00