7807d6caf0
Progress and tests for bulkdata.remove
...
Passes tests, but doesn't really handle nrows (and removing partially
full files) correctly, when deleting near the end of the data.
2013-01-09 17:39:29 -05:00
3d0fad3c2a
Move some helper functions around
2013-01-09 17:39:29 -05:00
fe3b087435
Remove implemented in nilmdb; still needs bulkdata changes.
2013-01-08 21:07:52 -05:00
f88c148ccc
Interval removal work in progress. Needs NilmDB and BulkData work.
2013-01-08 18:37:01 -05:00
80da937cb7
cmdline: return error when start > end (extract, list, remove)
2013-01-06 20:13:28 -05:00
c81972e66e
Minor testsuite and commandline fixes.
...
Now supports "list /foo/bar" in addition to the older "list --path /foo/bar"
2013-01-06 19:25:07 -05:00
b09362fde1
Full coverage of nilmdb.utils.mustclose
2013-01-05 18:02:53 -05:00
b7688844fa
Add a Nosetests plugin that lets me specify a test order within a directory.
2013-01-05 18:02:37 -05:00
3d212e7592
Move test helpers into subdirectory
2013-01-05 15:00:34 -05:00
7aedfdf9c3
Add lower level bulkdata test
2013-01-05 14:55:22 -05:00
07192c6ffb
nilmdb.BulkData: Switch to nested subdir/filename layout
...
Use numbered subdirectories to avoid having too many files in one dir.
Add appropriate tests.
Also fix an issue where the mmap_open LRU cache could inappropriately
open a file twice because it was using the optional "newsize"
parameter as a key -- now lrucache can be given a slice object that
describes which arguments are important.
2013-01-04 16:51:05 -05:00
11b0293d5f
Clean up BulkData file size calculations, test more thoroughly
...
Now the goal is 128 MiB files, rather than a specific length.
2013-01-03 20:19:01 -05:00
493bbed82c
More coverage and tests
2013-01-03 19:21:12 -05:00
c083d63c96
Tests for Unicode compliance
2013-01-03 17:03:52 -05:00
0221e3ea21
Update commandline test helpers to better handle Unicode
...
We replace cStringIO with StringIO subclass that forces UTF-8
encoding, and explicitly convert commandlines to UTF-8 before
shlex. These changes will only affect tests, not normal commandline
operation.
2013-01-03 17:03:52 -05:00
06e91a6a98
Always use function version of print()
2013-01-03 17:02:38 -05:00
10d58f6a47
More test coverage
2013-01-02 00:00:05 -05:00
c7c65b6542
Work around CherryPy bug #1200 ; related cleanups
...
Spent way too long trying to track down a cryptic error that turned
out to be a CherryPy bug. Now we catch this using a decorator in the
'extract' and 'intervals' generators that transforms exceptions that
trigger the bugs into one that does not. fun!
2013-01-01 23:03:53 -05:00
83b937c720
More Pytables -> bulkdata conversion
2012-12-31 17:22:30 -05:00
6f1e6fe232
Isolate all PyTables stuff to a single file.
...
This will make migrating to my own data storage engine easier.
2012-12-31 17:22:29 -05:00
b0d76312d1
Add must_close() decorator, use it in nilmdb
...
Warns at runtime if a class's close() method wasn't called before the
object was destroyed.
2012-12-31 17:21:19 -05:00
f355c73209
Refactor utility classes into nilmdb.utils subdir/namespace
...
There's some bug with the testing harness where placing e.g.
from du import du
in nilmdb/utils/__init__.py doesn't quite work -- sometimes the
module "du" replaces the function "du". Not exactly sure why;
we work around that by just renaming files so they don't match
the imported names directly.
2012-12-31 15:55:36 -05:00
24d4752bc3
Add LRU cache memoizing decorator for functions
2012-12-31 14:39:16 -05:00
9082cc9f44
Merging adjacent intervals is working now!
...
Adjust test expectations accordingly, since the number of intervals
they print out will now be smaller.
2012-12-12 19:25:27 -05:00
bf64a40472
Some misc test additions, interval optimizations. Still need adjacency test
2012-12-11 23:31:55 -05:00
32dbeebc09
More insertion checks. Need to get interval concatenation working.
2012-12-11 18:08:00 -05:00
66ddc79b15
Inserting works again, with proper end/start for paired blocks.
...
timeit.sh script works too!
2012-12-07 20:30:39 -05:00
5fecec2a4c
Support deleting streams with new 'destroy' command
2012-12-04 22:15:00 -05:00
17c329fd6d
Start to be a little more strict about how intervals are half-open.
2012-11-29 15:35:11 -05:00
c0f87db3c1
Converted rbtree, interval to Cython. Serious speedups!
2012-11-29 15:13:09 -05:00
f39567b2bc
Speed updates
2012-11-29 01:35:01 -05:00
99ec0f4946
Converted rbtree.py to Cython
...
About 3x faster
2012-11-29 01:25:51 -05:00
f5c60f68dc
Speed tests.
...
test_interval_speed is about O(n * log n), which is good -- but the
constants are high and it hits swap on a 4G machine for the 2**21
test. Hopefully cython helps!
2012-11-29 01:00:54 -05:00
bdef0986d6
rbtree and interval tests fully pass now.
...
On to benchmarking...
2012-11-29 00:42:50 -05:00
c396c4dac8
rbtree tests complete
2012-11-29 00:07:49 -05:00
0b443f510b
Filling out rbtree tests, search routines
2012-11-28 20:57:23 -05:00
66fa6f3824
Add rendering test
2012-11-28 18:34:51 -05:00
7211217f40
Working on getting the RBTree working. Intersections are busted.
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11380 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-11-15 18:55:56 +00:00
d34b980516
RBTree seems generally OK now
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11379 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-11-14 20:10:43 +00:00
6aee52d980
Deletion is still broken. F.
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11378 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-11-14 04:23:53 +00:00
090c8d5315
More progress
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11377 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-11-14 04:12:15 +00:00
1042ff9f4b
add RBtree C++ example that I based this on; update tests
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11376 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-11-14 03:55:37 +00:00
f9dea53c24
Randomize order for the insertion test
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11358 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-11-08 23:50:23 +00:00
6278d32f7d
Passes tests, but is slow
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11356 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-11-08 23:08:01 +00:00
991039903c
Partial implementation of Interval and IntervalSet with a red-black
...
tree.
This is currently hitting an issue where it's considering the
intersection of [0,1] and [1,2] to be [1,1]. It matches the
fully-closed definition of intervals, unlike before -- but might
cause issues. Need to consider whether test case is correct.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11355 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-11-08 22:56:05 +00:00
ea3e92be3f
Some prep work for merging adjacent insertions.
...
Doesn't actually merge them yet; need to change Interval
implementation to allow deletes.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11354 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-11-08 21:27:52 +00:00
407aedcd20
Can't represent 2112 on 32-bit systems
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10997 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-08-09 18:04:47 +00:00
bf8ff66c77
Fix timezone issues in cmdline test
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10996 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-08-09 18:03:18 +00:00
82f753efb1
disable TestLayoutSpeed for now
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10995 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-08-09 17:53:01 +00:00
e950794866
more info
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10991 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-08-08 22:00:54 +00:00
