Commit Graph

237 Commits

Author SHA1 Message Date
c083d63c96 Tests for Unicode compliance 2013-01-03 17:03:52 -05:00
0221e3ea21 Update commandline test helpers to better handle Unicode
We replace cStringIO with StringIO subclass that forces UTF-8
encoding, and explicitly convert commandlines to UTF-8 before
shlex.  These changes will only affect tests, not normal commandline
operation.
2013-01-03 17:03:52 -05:00
f5fd2b064e Replace urllib.encode() with a version that encodes Unicode as UTF-8 instead 2013-01-03 17:02:38 -05:00
06e91a6a98 Always use function version of print() 2013-01-03 17:02:38 -05:00
41b3f3c018 Always use UTF-8 for filenames in nilmdb.bulkdata 2013-01-03 17:02:38 -05:00
842076fef4 Cleanup server error handling with decorator 2013-01-03 17:02:38 -05:00
10d58f6a47 More test coverage 2013-01-02 00:00:05 -05:00
e2464efc12 Test everything; remove debugging 2013-01-01 23:46:54 -05:00
1beae5024e Bulkdata extract works now. 2013-01-01 23:44:52 -05:00
c7c65b6542 Work around CherryPy bug #1200; related cleanups
Spent way too long trying to track down a cryptic error that turned
out to be a CherryPy bug.  Now we catch this using a decorator in the
'extract' and 'intervals' generators that transforms exceptions that
trigger the bugs into one that does not.  fun!
2013-01-01 23:03:53 -05:00
f41ff0a6e8 Inserting bulk data is essentially done, not tested 2013-01-01 21:04:35 -05:00
389c1d189f Make option to turn off chunked encoding for debugging more clear. 2013-01-01 21:03:33 -05:00
487298986e More work towards bulkdata 2012-12-31 18:44:57 -05:00
d4cd045c48 Fix path stuff, build packer in bulkdata.Table 2012-12-31 17:22:30 -05:00
3816645313 More work on BulkData 2012-12-31 17:22:30 -05:00
83b937c720 More Pytables -> bulkdata conversion 2012-12-31 17:22:30 -05:00
b3e6e8976f More work towards flat bulk data storage.
Cleaned up OS-specific path handling in nilmdb, bulkdata.
2012-12-31 17:22:30 -05:00
c890ea93cb WIP switching away from PyTables 2012-12-31 17:22:29 -05:00
84c68c6913 Better documentation, cache Tables 2012-12-31 17:22:29 -05:00
6f1e6fe232 Isolate all PyTables stuff to a single file.
This will make migrating to my own data storage engine easier.
2012-12-31 17:22:29 -05:00
b0d76312d1 Add must_close() decorator, use it in nilmdb
Warns at runtime if a class's close() method wasn't called before the
object was destroyed.
2012-12-31 17:21:19 -05:00
19c846c71c Remove outdated files 2012-12-31 15:55:43 -05:00
f355c73209 Refactor utility classes into nilmdb.utils subdir/namespace
There's some bug with the testing harness where placing e.g.
  from du import du
in nilmdb/utils/__init__.py doesn't quite work -- sometimes the
module "du" replaces the function "du".  Not exactly sure why;
we work around that by just renaming files so they don't match
the imported names directly.
2012-12-31 15:55:36 -05:00
173014ba19 Use nilmdb.lrucache for caching interval sets 2012-12-31 14:52:46 -05:00
24d4752bc3 Add LRU cache memoizing decorator for functions 2012-12-31 14:39:16 -05:00
a85b273e2e Remove compression.
Messes up extraction, since we random access for the timestamp binary
search.  In the future, maybe switching to multiple tables (one for
timestamp, one for compressed data) would be smart.
2012-12-14 17:19:23 -05:00
7f73b4b304 Use compression in pytables 2012-12-14 17:17:52 -05:00
f3eb6d1b79 Time it! 2012-12-14 16:57:02 -05:00
9082cc9f44 Merging adjacent intervals is working now!
Adjust test expectations accordingly, since the number of intervals
they print out will now be smaller.
2012-12-12 19:25:27 -05:00
bf64a40472 Some misc test additions, interval optimizations. Still need adjacency test 2012-12-11 23:31:55 -05:00
32dbeebc09 More insertion checks. Need to get interval concatenation working. 2012-12-11 18:08:00 -05:00
66ddc79b15 Inserting works again, with proper end/start for paired blocks.
timeit.sh script works too!
2012-12-07 20:30:39 -05:00
7a8bd0bf41 Don't include layout on client side 2012-12-07 16:24:15 -05:00
ee552de740 Start reworking/fixing insert timestamps 2012-12-06 20:25:24 -05:00
6d1fb61573 Use 'repr' instead of 'str' in Interval string representation.
Otherwise timestamps get truncated to 2 decimal places.
2012-12-05 17:47:48 -05:00
f094529e66 TODO update 2012-12-04 22:15:53 -05:00
5fecec2a4c Support deleting streams with new 'destroy' command 2012-12-04 22:15:00 -05:00
85bb46f45c Use pytable's createparents flag to avoid having to create group
structure manually.
2012-12-04 18:57:36 -05:00
17c329fd6d Start to be a little more strict about how intervals are half-open. 2012-11-29 15:35:11 -05:00
437e1b425a More speed tests, some whitespace cleanups 2012-11-29 15:22:47 -05:00
c0f87db3c1 Converted rbtree, interval to Cython. Serious speedups! 2012-11-29 15:13:09 -05:00
a9c5c19e30 Start converting interval.py to Cython. 2012-11-29 12:42:38 -05:00
f39567b2bc Speed updates 2012-11-29 01:35:01 -05:00
99ec0f4946 Converted rbtree.py to Cython
About 3x faster
2012-11-29 01:25:51 -05:00
f5c60f68dc Speed tests.
test_interval_speed is about O(n * log n), which is good -- but the
constants are high and it hits swap on a 4G machine for the 2**21
test.  Hopefully cython helps!
2012-11-29 01:00:54 -05:00
bdef0986d6 rbtree and interval tests fully pass now.
On to benchmarking...
2012-11-29 00:42:50 -05:00
c396c4dac8 rbtree tests complete 2012-11-29 00:07:49 -05:00
0b443f510b Filling out rbtree tests, search routines 2012-11-28 20:57:23 -05:00
66fa6f3824 Add rendering test 2012-11-28 18:34:51 -05:00
875fbe969f Some documentation and other cleanups in rbtree.py 2012-11-28 18:30:21 -05:00