Commit Graph

152 Commits

Author SHA1 Message Date
f316026592 Move datetime_tz package under nilmdb.utils
datetime_tz isn't readily available, so it's a lot easier to just
package it within the nilmdb tree.
2013-01-30 19:03:42 -05:00
58c7c8f6ff Support "now" as a timestamp argument 2013-01-28 19:07:45 -05:00
225003f412 Huge cleanup of namespaces, modules, packages, imports.
Now nilmdb.client, nilmdb.server, nilmdb.cmdline, and nilmdb.utils
are each their own modules, and there is a little bit more of a
logical separation between them.  Various changes scattered throughout
to fix naming (for example, nilmdb.nilmdb.NilmDBError is now
nilmdb.server.errors.NilmDBError).

Reduced usage of "from __future__ import absolute_import" as much
as possible.  It's still needed for the functions in the nilmdb/server
directory to be able to import the nilmdb module rather than the
nilmdb.py script.

This should hopefully ease future packaging a bit.
2013-01-28 19:04:52 -05:00
40b966aef2 Add pycurl-specific hack to Iteratorizer
Inside the pycurl callback, we can't raise exceptions, because the
pycurl extension module will unconditionally print the exception
itself, and not pass it up to the caller.  Instead, we have the
callback return a value that tells curl to abort.  (-1 would be best,
in case we were given 0 bytes, but the extension doesn't support
that either).

This resolves the 'Exception("should die")' problem when interrupting
a streaming generator like stream_extract.
2013-01-24 19:06:20 -05:00
294ec6988b Rewrite Iteratorizer as a context manager
Relying on __del__ to clean up the thread isn't as reliable.
2013-01-24 19:04:25 -05:00
fad23ebb22 Add --timestamp-raw option to extract and list 2013-01-24 16:03:38 -05:00
b226dc4337 Properly handle test case where server doesn't start 2013-01-24 16:03:38 -05:00
9bf213707c Properly return an error if two timestamps are equal 2013-01-22 18:35:18 -05:00
5cd7899e98 Send a Access-Control-Allow-Origin (CORS) header with all responses 2013-01-22 14:42:03 -05:00
ceec5fb9b3 Force /stream/interval and /stream/extract responses to be text/plain 2013-01-22 12:47:06 -05:00
2820ff9758 More fixes to mustclose decorator and argspecs 2013-01-18 17:21:30 -05:00
b7f746e66d Fix lrucache decorator argspecs 2013-01-18 17:13:50 -05:00
40cf4941f0 Test that argspecs are maintained in lrucache 2013-01-18 17:01:46 -05:00
0312b6eb07 Test for issue where mustclose decorator didn't maintain argspec 2013-01-18 16:55:51 -05:00
62354b4dce Add test for bad-parameters-give-500-error 2013-01-17 19:58:48 -05:00
4f6a742e6c Fix test failure 2013-01-16 17:31:31 -05:00
87b43e5d04 Command line errors cleaned up and made more consistent 2013-01-16 16:52:43 -05:00
e5d3deb6fe Removal support is complete.
`nrows` may change if you restart the server; documented why this is
the case in the design.md file.  It's not a problem.
2013-01-09 23:26:59 -05:00
7807d6caf0 Progress and tests for bulkdata.remove
Passes tests, but doesn't really handle nrows (and removing partially
full files) correctly, when deleting near the end of the data.
2013-01-09 17:39:29 -05:00
3d0fad3c2a Move some helper functions around 2013-01-09 17:39:29 -05:00
fe3b087435 Remove implemented in nilmdb; still needs bulkdata changes. 2013-01-08 21:07:52 -05:00
f88c148ccc Interval removal work in progress. Needs NilmDB and BulkData work. 2013-01-08 18:37:01 -05:00
80da937cb7 cmdline: return error when start > end (extract, list, remove) 2013-01-06 20:13:28 -05:00
c81972e66e Minor testsuite and commandline fixes.
Now supports "list /foo/bar" in addition to the older "list --path /foo/bar"
2013-01-06 19:25:07 -05:00
b09362fde1 Full coverage of nilmdb.utils.mustclose 2013-01-05 18:02:53 -05:00
b7688844fa Add a Nosetests plugin that lets me specify a test order within a directory. 2013-01-05 18:02:37 -05:00
3d212e7592 Move test helpers into subdirectory 2013-01-05 15:00:34 -05:00
7aedfdf9c3 Add lower level bulkdata test 2013-01-05 14:55:22 -05:00
07192c6ffb nilmdb.BulkData: Switch to nested subdir/filename layout
Use numbered subdirectories to avoid having too many files in one dir.
Add appropriate tests.

Also fix an issue where the mmap_open LRU cache could inappropriately
open a file twice because it was using the optional "newsize"
parameter as a key -- now lrucache can be given a slice object that
describes which arguments are important.
2013-01-04 16:51:05 -05:00
11b0293d5f Clean up BulkData file size calculations, test more thoroughly
Now the goal is 128 MiB files, rather than a specific length.
2013-01-03 20:19:01 -05:00
493bbed82c More coverage and tests 2013-01-03 19:21:12 -05:00
c083d63c96 Tests for Unicode compliance 2013-01-03 17:03:52 -05:00
0221e3ea21 Update commandline test helpers to better handle Unicode
We replace cStringIO with StringIO subclass that forces UTF-8
encoding, and explicitly convert commandlines to UTF-8 before
shlex.  These changes will only affect tests, not normal commandline
operation.
2013-01-03 17:03:52 -05:00
06e91a6a98 Always use function version of print() 2013-01-03 17:02:38 -05:00
10d58f6a47 More test coverage 2013-01-02 00:00:05 -05:00
c7c65b6542 Work around CherryPy bug #1200; related cleanups
Spent way too long trying to track down a cryptic error that turned
out to be a CherryPy bug.  Now we catch this using a decorator in the
'extract' and 'intervals' generators that transforms exceptions that
trigger the bugs into one that does not.  fun!
2013-01-01 23:03:53 -05:00
83b937c720 More Pytables -> bulkdata conversion 2012-12-31 17:22:30 -05:00
6f1e6fe232 Isolate all PyTables stuff to a single file.
This will make migrating to my own data storage engine easier.
2012-12-31 17:22:29 -05:00
b0d76312d1 Add must_close() decorator, use it in nilmdb
Warns at runtime if a class's close() method wasn't called before the
object was destroyed.
2012-12-31 17:21:19 -05:00
f355c73209 Refactor utility classes into nilmdb.utils subdir/namespace
There's some bug with the testing harness where placing e.g.
  from du import du
in nilmdb/utils/__init__.py doesn't quite work -- sometimes the
module "du" replaces the function "du".  Not exactly sure why;
we work around that by just renaming files so they don't match
the imported names directly.
2012-12-31 15:55:36 -05:00
24d4752bc3 Add LRU cache memoizing decorator for functions 2012-12-31 14:39:16 -05:00
9082cc9f44 Merging adjacent intervals is working now!
Adjust test expectations accordingly, since the number of intervals
they print out will now be smaller.
2012-12-12 19:25:27 -05:00
bf64a40472 Some misc test additions, interval optimizations. Still need adjacency test 2012-12-11 23:31:55 -05:00
32dbeebc09 More insertion checks. Need to get interval concatenation working. 2012-12-11 18:08:00 -05:00
66ddc79b15 Inserting works again, with proper end/start for paired blocks.
timeit.sh script works too!
2012-12-07 20:30:39 -05:00
5fecec2a4c Support deleting streams with new 'destroy' command 2012-12-04 22:15:00 -05:00
17c329fd6d Start to be a little more strict about how intervals are half-open. 2012-11-29 15:35:11 -05:00
c0f87db3c1 Converted rbtree, interval to Cython. Serious speedups! 2012-11-29 15:13:09 -05:00
f39567b2bc Speed updates 2012-11-29 01:35:01 -05:00
99ec0f4946 Converted rbtree.py to Cython
About 3x faster
2012-11-29 01:25:51 -05:00