nilm/nilmdb
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
209 Commits 3 Branches 64 Tags 3.2 MiB
insert-rework-wip
Commit Graph

209 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jim Paris
9082cc9f44 Merging adjacent intervals is working now! 
Adjust test expectations accordingly, since the number of intervals
they print out will now be smaller.
 2012-12-12 19:25:27 -05:00
Jim Paris
bf64a40472 Some misc test additions, interval optimizations. Still need adjacency test 2012-12-11 23:31:55 -05:00
Jim Paris
32dbeebc09 More insertion checks. Need to get interval concatenation working. 2012-12-11 18:08:00 -05:00
Jim Paris
66ddc79b15 Inserting works again, with proper end/start for paired blocks. 
timeit.sh script works too!
 2012-12-07 20:30:39 -05:00
Jim Paris
7a8bd0bf41 Don't include layout on client side 2012-12-07 16:24:15 -05:00
Jim Paris
ee552de740 Start reworking/fixing insert timestamps 2012-12-06 20:25:24 -05:00
Jim Paris
6d1fb61573 Use 'repr' instead of 'str' in Interval string representation. 
Otherwise timestamps get truncated to 2 decimal places.
 2012-12-05 17:47:48 -05:00
Jim Paris
f094529e66 TODO update 2012-12-04 22:15:53 -05:00
Jim Paris
5fecec2a4c Support deleting streams with new 'destroy' command 2012-12-04 22:15:00 -05:00
Jim Paris
85bb46f45c Use pytable's createparents flag to avoid having to create group 
structure manually.
 2012-12-04 18:57:36 -05:00
Jim Paris
17c329fd6d Start to be a little more strict about how intervals are half-open. 2012-11-29 15:35:11 -05:00
Jim Paris
437e1b425a More speed tests, some whitespace cleanups 2012-11-29 15:22:47 -05:00
Jim Paris
c0f87db3c1 Converted rbtree, interval to Cython. Serious speedups! 2012-11-29 15:13:09 -05:00
Jim Paris
a9c5c19e30 Start converting interval.py to Cython. 2012-11-29 12:42:38 -05:00
Jim Paris
f39567b2bc Speed updates 2012-11-29 01:35:01 -05:00
Jim Paris
99ec0f4946 Converted rbtree.py to Cython 
About 3x faster
 2012-11-29 01:25:51 -05:00
Jim Paris
f5c60f68dc Speed tests. 
test_interval_speed is about O(n * log n), which is good -- but the
constants are high and it hits swap on a 4G machine for the 2**21
test.  Hopefully cython helps!
 2012-11-29 01:00:54 -05:00
Jim Paris
bdef0986d6 rbtree and interval tests fully pass now. 
On to benchmarking...
 2012-11-29 00:42:50 -05:00
Jim Paris
c396c4dac8 rbtree tests complete 2012-11-29 00:07:49 -05:00
Jim Paris
0b443f510b Filling out rbtree tests, search routines 2012-11-28 20:57:23 -05:00
Jim Paris
66fa6f3824 Add rendering test 2012-11-28 18:34:51 -05:00
Jim Paris
875fbe969f Some documentation and other cleanups in rbtree.py 2012-11-28 18:30:21 -05:00
Jim Paris
e35e85886e add .gitignore 2012-11-28 17:21:51 -05:00
Jim Paris
7211217f40 Working on getting the RBTree working. Intersections are busted. 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11380 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-11-15 18:55:56 +00:00
Jim Paris
d34b980516 RBTree seems generally OK now 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11379 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-11-14 20:10:43 +00:00
Jim Paris
6aee52d980 Deletion is still broken. F. 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11378 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-11-14 04:23:53 +00:00
Jim Paris
090c8d5315 More progress 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11377 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-11-14 04:12:15 +00:00
Jim Paris
1042ff9f4b add RBtree C++ example that I based this on; update tests 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11376 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-11-14 03:55:37 +00:00
Jim Paris
bc687969c1 Work in progress switching to my own RBTree. Currently creates loops 
somewhere, need to figure out what's going on.


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11375 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-11-14 03:48:04 +00:00
Jim Paris
de27bd3f41 Attempt at using a sentinel instead of class instances for the leaf node.. doesnt quite work for deletion 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11361 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-11-10 02:12:01 +00:00
Jim Paris
4dcf713d0e Attempts at speeding up the RbTree implementation 
with cython.  Still quite a bit slower than the bxinterval
implementation, though.


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11360 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-11-09 21:06:04 +00:00
Jim Paris
f9dea53c24 Randomize order for the insertion test 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11358 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-11-08 23:50:23 +00:00
Jim Paris
6cedd7c327 fix 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11357 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-11-08 23:44:21 +00:00
Jim Paris
6278d32f7d Passes tests, but is slow 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11356 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-11-08 23:08:01 +00:00
Jim Paris
991039903c Partial implementation of Interval and IntervalSet with a red-black 
tree.

This is currently hitting an issue where it's considering the
intersection of [0,1] and [1,2] to be [1,1].  It matches the 
fully-closed definition of intervals, unlike before -- but might
cause issues.  Need to consider whether test case is correct.


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11355 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-11-08 22:56:05 +00:00
Jim Paris
ea3e92be3f Some prep work for merging adjacent insertions. 
Doesn't actually merge them yet; need to change Interval
implementation to allow deletes.


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11354 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-11-08 21:27:52 +00:00
Jim Paris
56c3306b38 Fix comments 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10999 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-08-09 19:11:25 +00:00
Jim Paris
cb6561c151 Fix some comments and text 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10998 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-08-09 18:26:45 +00:00
Jim Paris
407aedcd20 Can't represent 2112 on 32-bit systems 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10997 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-08-09 18:04:47 +00:00
Jim Paris
bf8ff66c77 Fix timezone issues in cmdline test 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10996 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-08-09 18:03:18 +00:00
Jim Paris
82f753efb1 disable TestLayoutSpeed for now 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10995 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-08-09 17:53:01 +00:00
Jim Paris
e950794866 more info 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10991 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-08-08 22:00:54 +00:00
Jim Paris
cb7c0cf83e Test roundtrip times for parser and formatter. Can optimize now 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10990 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-08-08 21:58:09 +00:00
Jim Paris
33cb7031a3 Test float32_8 type names in addition to PrepData aliases 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10989 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-08-08 19:23:28 +00:00
Jim Paris
33492fc851 Update TODO 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10983 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-08-06 22:06:29 +00:00
Jim Paris
5101522025 Tests pass with 100% coverage on non-cython modules 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10982 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-08-06 22:05:09 +00:00
Jim Paris
5130ab7e6a Start reworking the layout types. 
Current/old design has specific layouts: RawData, PrepData,
RawNotchedData.
Let's get rid of this entirely and switch to simpler data types that
are
just collections and counts of a single type.  We'll still use strings
to describe them, with format:

    type_count
  
where type is "uint16", "float32", or "float64", and count is an
integer.

nilmdb.layout.named() will parse these strings into the appropriate
handlers.  For compatibility:
  
    "RawData" == "uint16_6"
    "RawNotchedData" == "uint16_9"
    "PrepData" == "float32_8"


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10981 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-08-06 21:46:09 +00:00
Jim Paris
27024fb38e Clarify interval openness 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10969 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-07-25 21:03:11 +00:00
Jim Paris
ff4e934bef Add database size to "nilmtool.py info" output. 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10968 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-07-24 21:52:38 +00:00
Jim Paris
a1d09fb5fc TODO update 
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10967 ddd99763-3ecb-0310-9145-efcb8ce7c51f
 2012-07-24 21:18:01 +00:00