Commit Graph

40 Commits

Author SHA1 Message Date
bbd59c8b50 Add nilmdb.utils.interval.intersection by generalizing set_difference 2013-07-30 14:48:19 -04:00
fc43107307 Fill out test coverage 2013-07-09 19:06:26 -04:00
ed89d803f0 Remove aplotter code 2013-03-24 21:29:09 -04:00
22274550ab Test python version of Interval too 2013-03-15 21:37:03 -04:00
4f06d6ae68 Move Interval set_difference inside nilmdb.utils for clients
Clients might need to to Interval math too, so move a simple Interval
class and start putting helpers in there.
2013-03-15 21:37:03 -04:00
c54d8041c3 Update design docs 2013-03-15 21:07:01 -04:00
90b96799ac Bulk of the switch to int64 microsecond timestamps, including test data. 2013-03-15 15:08:58 -04:00
56679ad770 Move more datetime_tz calls into common code 2013-03-15 15:08:58 -04:00
1d61d61a81 Add interval.set_difference function and associated tests 2013-03-11 15:40:50 -04:00
11b228f77a Convert times to microsecond precision strings more consistently.
Use a new helper, nilmdb.utils.time.float_to_time_string().
This will help if we ever want to change representation (like using
uint64 microseconds since epoch, which saves us from having to
waste bits on the floating-point exponent)
2013-03-05 17:07:39 -05:00
a49c655816 Strictly enforce (start < end) for all intervals.
Previously, we allowed start == end, but this doesn't make sense with
half-open intervals.
2013-02-21 14:06:40 -05:00
f316026592 Move datetime_tz package under nilmdb.utils
datetime_tz isn't readily available, so it's a lot easier to just
package it within the nilmdb tree.
2013-01-30 19:03:42 -05:00
225003f412 Huge cleanup of namespaces, modules, packages, imports.
Now nilmdb.client, nilmdb.server, nilmdb.cmdline, and nilmdb.utils
are each their own modules, and there is a little bit more of a
logical separation between them.  Various changes scattered throughout
to fix naming (for example, nilmdb.nilmdb.NilmDBError is now
nilmdb.server.errors.NilmDBError).

Reduced usage of "from __future__ import absolute_import" as much
as possible.  It's still needed for the functions in the nilmdb/server
directory to be able to import the nilmdb module rather than the
nilmdb.py script.

This should hopefully ease future packaging a bit.
2013-01-28 19:04:52 -05:00
b09362fde1 Full coverage of nilmdb.utils.mustclose 2013-01-05 18:02:53 -05:00
3d212e7592 Move test helpers into subdirectory 2013-01-05 15:00:34 -05:00
f355c73209 Refactor utility classes into nilmdb.utils subdir/namespace
There's some bug with the testing harness where placing e.g.
  from du import du
in nilmdb/utils/__init__.py doesn't quite work -- sometimes the
module "du" replaces the function "du".  Not exactly sure why;
we work around that by just renaming files so they don't match
the imported names directly.
2012-12-31 15:55:36 -05:00
9082cc9f44 Merging adjacent intervals is working now!
Adjust test expectations accordingly, since the number of intervals
they print out will now be smaller.
2012-12-12 19:25:27 -05:00
bf64a40472 Some misc test additions, interval optimizations. Still need adjacency test 2012-12-11 23:31:55 -05:00
17c329fd6d Start to be a little more strict about how intervals are half-open. 2012-11-29 15:35:11 -05:00
c0f87db3c1 Converted rbtree, interval to Cython. Serious speedups! 2012-11-29 15:13:09 -05:00
f39567b2bc Speed updates 2012-11-29 01:35:01 -05:00
f5c60f68dc Speed tests.
test_interval_speed is about O(n * log n), which is good -- but the
constants are high and it hits swap on a 4G machine for the 2**21
test.  Hopefully cython helps!
2012-11-29 01:00:54 -05:00
bdef0986d6 rbtree and interval tests fully pass now.
On to benchmarking...
2012-11-29 00:42:50 -05:00
66fa6f3824 Add rendering test 2012-11-28 18:34:51 -05:00
7211217f40 Working on getting the RBTree working. Intersections are busted.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11380 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-11-15 18:55:56 +00:00
1042ff9f4b add RBtree C++ example that I based this on; update tests
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11376 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-11-14 03:55:37 +00:00
f9dea53c24 Randomize order for the insertion test
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11358 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-11-08 23:50:23 +00:00
6278d32f7d Passes tests, but is slow
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11356 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-11-08 23:08:01 +00:00
991039903c Partial implementation of Interval and IntervalSet with a red-black
tree.

This is currently hitting an issue where it's considering the
intersection of [0,1] and [1,2] to be [1,1].  It matches the 
fully-closed definition of intervals, unlike before -- but might
cause issues.  Need to consider whether test case is correct.


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11355 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-11-08 22:56:05 +00:00
ea3e92be3f Some prep work for merging adjacent insertions.
Doesn't actually merge them yet; need to change Interval
implementation to allow deletes.


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11354 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-11-08 21:27:52 +00:00
8a7b3b5f95 Add DBInterval and associated tests. Clean up other interval tests as
well, and fix missing coverage on IntervalSet.str()


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10890 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-05-25 21:45:05 +00:00
4501da6edc Can query intervals now.
On the big database, the server takes a few seconds to figure out the
interval intersections.  Need to think about how to improve that --
the real key might be to start reducing the number of intervals we're
storing by combining them, potentially as they're inserted.


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10838 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-05-07 22:32:02 +00:00
7eef39d5fd notes
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10832 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-05-04 16:08:32 +00:00
b84ffddd50 New bxinterval implementation is at least linear -- not much better in
the TestIntervalSpeed versus the previous bisect version, but should
be better in general.



git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10748 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-12 22:39:16 +00:00
1531114677 fix typos in test_interval; test big numbers and floats
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10745 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-12 21:05:54 +00:00
055cfa12b2 Various work on improving intervals...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10738 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-11 22:05:27 +00:00
cc4e3bdb76 Rename some nilmdb methods to be private
More work with nilmdb.stream_insert.  Checks for overlap between
parser output and database intervals, and actually inserts the
data into pytables.  Need to benchmark/figure out whether we can
use table.append rather than the row.append() nonsense -- it could
simplify things quite a bit.

Improve layout class and add tests to get more coverage.  Better
error handling on invalid inputs (reports the reason for the error)

Replace layout.fillrow with layout.filltable, but turns out that we
can probably just remove it anyway.

Add nilmdb.Timer for simple timing tests

Move some duplicated test suite helper functions into a new file,
test_helper.py

Add class to test_interval.py to match the others


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10661 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-29 21:43:05 +00:00
cf66eca42c Spent time trying to get Intervals and IntervalSets to handle a gap
tolerance in equality comparisions, but it's a real pain to get all
the edge cases right, and it's not clear we actually need that
functionality at the moment.  Skip it for now.


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10659 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-28 19:25:02 +00:00
1046d0c47b more tests
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10657 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-28 00:22:44 +00:00
d940aabf66 Incomplete, need to finish interval/intervalset tests
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10656 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-27 23:19:08 +00:00