Commit Graph

638 Commits

Author SHA1 Message Date
225003f412 Huge cleanup of namespaces, modules, packages, imports.
Now nilmdb.client, nilmdb.server, nilmdb.cmdline, and nilmdb.utils
are each their own modules, and there is a little bit more of a
logical separation between them.  Various changes scattered throughout
to fix naming (for example, nilmdb.nilmdb.NilmDBError is now
nilmdb.server.errors.NilmDBError).

Reduced usage of "from __future__ import absolute_import" as much
as possible.  It's still needed for the functions in the nilmdb/server
directory to be able to import the nilmdb module rather than the
nilmdb.py script.

This should hopefully ease future packaging a bit.
2013-01-28 19:04:52 -05:00
40b966aef2 Add pycurl-specific hack to Iteratorizer
Inside the pycurl callback, we can't raise exceptions, because the
pycurl extension module will unconditionally print the exception
itself, and not pass it up to the caller.  Instead, we have the
callback return a value that tells curl to abort.  (-1 would be best,
in case we were given 0 bytes, but the extension doesn't support
that either).

This resolves the 'Exception("should die")' problem when interrupting
a streaming generator like stream_extract.
2013-01-24 19:06:20 -05:00
294ec6988b Rewrite Iteratorizer as a context manager
Relying on __del__ to clean up the thread isn't as reliable.
2013-01-24 19:04:25 -05:00
fad23ebb22 Add --timestamp-raw option to extract and list 2013-01-24 16:03:38 -05:00
b226dc4337 Properly handle test case where server doesn't start 2013-01-24 16:03:38 -05:00
e7af863017 httpclient: make sure we error out quickly if nested calls are made
Curl will give an error if we call .setopt() while a .perform() is
in progress, for example if we try to do a stream_insert() while
in the middle of a stream_extract().  Move the setopt() to the
beginning of the get/put functions to ensure that we hit this
error before we mess with the URLs or anything else.
2013-01-24 15:36:10 -05:00
af6ce5b79c Remove superfluous from iteratorizor callback exception 2013-01-23 15:42:27 -05:00
0a6fc943e2 Add some better documentation of layout parameter to create.py 2013-01-22 18:47:39 -05:00
67c6e178e1 Documentation updates 2013-01-22 18:36:05 -05:00
9bf213707c Properly return an error if two timestamps are equal 2013-01-22 18:35:18 -05:00
5cd7899e98 Send a Access-Control-Allow-Origin (CORS) header with all responses 2013-01-22 14:42:03 -05:00
ceec5fb9b3 Force /stream/interval and /stream/extract responses to be text/plain 2013-01-22 12:47:06 -05:00
85be497edb Fix README 2013-01-21 17:30:01 -05:00
bd1b7107af Update TODO, clean up bulkdata error message 2013-01-21 11:43:28 -05:00
b8275f108d Make error message more helpful 2013-01-18 17:27:57 -05:00
2820ff9758 More fixes to mustclose decorator and argspecs 2013-01-18 17:21:30 -05:00
a015de893d Cleanup 2013-01-18 17:14:26 -05:00
b7f746e66d Fix lrucache decorator argspecs 2013-01-18 17:13:50 -05:00
40cf4941f0 Test that argspecs are maintained in lrucache 2013-01-18 17:01:46 -05:00
8a418ceb3e Fix issue where mustclose decorator doesn't maintain argspec 2013-01-18 16:57:15 -05:00
0312b6eb07 Test for issue where mustclose decorator didn't maintain argspec 2013-01-18 16:55:51 -05:00
077f197d24 Fix server returning 500 for bad HTTP parameters 2013-01-18 16:54:49 -05:00
62354b4dce Add test for bad-parameters-give-500-error 2013-01-17 19:58:48 -05:00
5970cd85cf Disable "ie-friendly" error message padding in CherryPy 2013-01-16 17:57:45 -05:00
4f6a742e6c Fix test failure 2013-01-16 17:31:31 -05:00
87b43e5d04 Command line errors cleaned up and made more consistent 2013-01-16 16:52:43 -05:00
f0c2a64ae3 Update doc formatting, .gitignore 2013-01-09 23:36:23 -05:00
e5d3deb6fe Removal support is complete.
`nrows` may change if you restart the server; documented why this is
the case in the design.md file.  It's not a problem.
2013-01-09 23:26:59 -05:00
d321058b48 Add basic versioning to bulkdata table format file. 2013-01-09 19:26:24 -05:00
cea83140c0 More work towards correctly removing rows. 2013-01-09 19:25:45 -05:00
7807d6caf0 Progress and tests for bulkdata.remove
Passes tests, but doesn't really handle nrows (and removing partially
full files) correctly, when deleting near the end of the data.
2013-01-09 17:39:29 -05:00
3d0fad3c2a Move some helper functions around 2013-01-09 17:39:29 -05:00
fe3b087435 Remove implemented in nilmdb; still needs bulkdata changes. 2013-01-08 21:07:52 -05:00
bcefe52298 nilmdb: Bring out range manipulating SQL so we can reuse it 2013-01-08 18:45:03 -05:00
f88c148ccc Interval removal work in progress. Needs NilmDB and BulkData work. 2013-01-08 18:37:01 -05:00
4a47b1d04a remove support: command line, client 2013-01-06 20:13:57 -05:00
80da937cb7 cmdline: return error when start > end (extract, list, remove) 2013-01-06 20:13:28 -05:00
c81972e66e Minor testsuite and commandline fixes.
Now supports "list /foo/bar" in addition to the older "list --path /foo/bar"
2013-01-06 19:25:07 -05:00
b09362fde1 Full coverage of nilmdb.utils.mustclose 2013-01-05 18:02:53 -05:00
b7688844fa Add a Nosetests plugin that lets me specify a test order within a directory. 2013-01-05 18:02:37 -05:00
3d212e7592 Move test helpers into subdirectory 2013-01-05 15:00:34 -05:00
7aedfdf9c3 Add lower level bulkdata test 2013-01-05 14:55:22 -05:00
ebd4f74959 Remove "pragma: no cover" from things that should get tested 2013-01-05 14:52:06 -05:00
ebe2fbab92 Add wrap_verify option to nilmdb.utils.must_close decorator 2013-01-05 14:51:41 -05:00
4831a0cae1 Small doc updates 2013-01-04 17:27:04 -05:00
07192c6ffb nilmdb.BulkData: Switch to nested subdir/filename layout
Use numbered subdirectories to avoid having too many files in one dir.
Add appropriate tests.

Also fix an issue where the mmap_open LRU cache could inappropriately
open a file twice because it was using the optional "newsize"
parameter as a key -- now lrucache can be given a slice object that
describes which arguments are important.
2013-01-04 16:51:05 -05:00
09d325e8ab Rename format -> _format in data dirs 2013-01-03 20:46:15 -05:00
11b0293d5f Clean up BulkData file size calculations, test more thoroughly
Now the goal is 128 MiB files, rather than a specific length.
2013-01-03 20:19:01 -05:00
493bbed82c More coverage and tests 2013-01-03 19:21:12 -05:00
3bc25daaab Trim urllib to get full coverage of the features in use 2013-01-03 17:10:07 -05:00