de7f78a43b
which are not necessary anymore
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10831 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-05-03 21:14:58 +00:00
fab3567d74
instructions for blist
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10830 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-05-03 21:14:46 +00:00
6d6514d5c3
Go back to bxinterval. The blist version was slower.
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10829 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-05-03 21:14:24 +00:00
b67fe79e47
Use blist instead of bxtree
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10828 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-05-03 21:13:55 +00:00
064b4bf715
notes
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10827 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-05-03 19:18:07 +00:00
e08be4c2a8
misc
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10826 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-05-03 19:17:48 +00:00
0276810776
More timing tests
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10825 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-05-03 19:17:35 +00:00
565d0e98a9
Remove trailing spaces
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10824 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-05-03 19:17:23 +00:00
47245df9bd
nilmdb now caches the intervals the first time a particular stream's
...
intervals are accessed, so it doesn't need to keep rebuilding them as
long as it's running.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10800 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-24 21:59:33 +00:00
c07670ac3e
Make the whole nilmdb.layout parser Cython, and rewrite the parsing
...
bits so it's hopefully quite a bit faster now.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10798 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-24 21:00:26 +00:00
37b4376b4c
design notes
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10756 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-13 21:00:33 +00:00
3b52ecafa3
Clean up unused code
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10755 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-13 20:53:03 +00:00
ac32647fac
Remove PyTables indexing support; it's not necessary since we're just
...
using pytables as a row-indexed data store.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10754 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-13 20:51:18 +00:00
4e143dfa18
docs
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10753 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-13 20:47:56 +00:00
b84ffddd50
New bxinterval implementation is at least linear -- not much better in
...
the TestIntervalSpeed versus the previous bisect version, but should
be better in general.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10748 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-12 22:39:16 +00:00
1531114677
fix typos in test_interval; test big numbers and floats
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10745 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-12 21:05:54 +00:00
ab4c1f0925
Update bxintersect to use doubles, not ints. Cython is easy!
...
Haven't updated IntervalSet, so current performance still sucks:
32: 805.000000 μs
64: 1534.000000 μs
128: 3158.000000 μs
256: 6430.000000 μs
512: 12972.000000 μs
1024: 26760.000000 μs
2048: 64091.000000 μs
4096: 108568.000000 μs
8192: 219389.000000 μs
16384: 471753.000000 μs
32768: 923798.000000 μs
65536: 1929795.000000 μs
131072: 3873633.000000 μs
262144: 8639542.000000 μs
524288: 17132838.000000 μs
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10744 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-12 20:26:33 +00:00
f7149e48e8
Add intersection.pyx from bx-python
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10743 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-12 20:16:46 +00:00
055cfa12b2
Various work on improving intervals...
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10738 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-11 22:05:27 +00:00
5cb03cd6ef
Update design.md. Add commit after each insert
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10729 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-10 20:52:59 +00:00
70bcc6d2b3
design notes
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10728 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-10 19:58:16 +00:00
43d3daa840
Testing
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10712 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-09 22:46:04 +00:00
7dfa288270
Can now insert data using command line tool. Time to benchmark etc.
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10709 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-09 21:31:39 +00:00
881b9a7bd1
add timestamped version
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10708 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-09 20:49:08 +00:00
e85acdd20c
Fill out and fix parse_Time
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10705 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-09 19:35:14 +00:00
d725ed1771
date/time parsing, needs work
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10685 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-06 20:54:48 +00:00
d58a27e2bf
Better coverage
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10684 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-06 18:37:05 +00:00
f3b0dfabea
More work on commandline and commandline tests.
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10682 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-06 18:25:09 +00:00
cccaec326a
Need to figure out a better way to handle this
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10679 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-04 22:57:12 +00:00
a3f444eb25
A lot more command line testing.
...
There'es some issue with tons of requests getting slowly blocked,
though...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10678 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-04 22:34:01 +00:00
277b0c1d00
More command line tests.
...
Make nilmdb.cmdline a proper class
Fix various stuff
Add /dbpath command to get DB path
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10677 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-04 18:54:24 +00:00
7bba4a80d9
Work on command line client, and some improvements to server
...
handling of bad URLs
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10676 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-03 22:21:42 +00:00
f29d38d9d9
More work on nilmtool
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10674 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-03 19:43:37 +00:00
b4a0288a39
work on command line
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10673 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-03 18:01:56 +00:00
e63ab23d20
stream_insert puts data into the database now. Fill out tests to
...
cover overlapping data, missing data, etc.
Consolidate nilmdb.client.MyCurl.getjson() and putjson().
Add __str__ for more compact representation of IntervalSet
Make stream_id not NULL in SQL database.
Format NilmDBError with name of exception for more clarity
Work around CherryPy issue 1138 by removing HTML escaping before
returning JSON error responses from the server.
Reached 100% test coverage again (woohoo)
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10665 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-31 04:15:29 +00:00
776279a4e6
Go back to generic parsing for most stuff now. Need to rework what's
...
most efficient here.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10664 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-30 22:15:04 +00:00
7a9012c3e9
work on improving layout parsing
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10663 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-30 21:29:22 +00:00
8c619cfde5
Minor cleanups
...
Remove nilmdb.layout.filltable
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10662 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-29 21:44:57 +00:00
cc4e3bdb76
Rename some nilmdb methods to be private
...
More work with nilmdb.stream_insert. Checks for overlap between
parser output and database intervals, and actually inserts the
data into pytables. Need to benchmark/figure out whether we can
use table.append rather than the row.append() nonsense -- it could
simplify things quite a bit.
Improve layout class and add tests to get more coverage. Better
error handling on invalid inputs (reports the reason for the error)
Replace layout.fillrow with layout.filltable, but turns out that we
can probably just remove it anyway.
Add nilmdb.Timer for simple timing tests
Move some duplicated test suite helper functions into a new file,
test_helper.py
Add class to test_interval.py to match the others
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10661 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-29 21:43:05 +00:00
3dfd187710
Use instantiated classes for layouts.
...
Change expected_daily_rows into rate_hz just for simplicity
nilmdb.layout.Parser now validates that timestamps are monotonically
increasing, and remembers the min and max seen.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10660 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-29 16:46:34 +00:00
cf66eca42c
Spent time trying to get Intervals and IntervalSets to handle a gap
...
tolerance in equality comparisions, but it's a real pain to get all
the edge cases right, and it's not clear we actually need that
functionality at the moment. Skip it for now.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10659 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-28 19:25:02 +00:00
1046d0c47b
more tests
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10657 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-28 00:22:44 +00:00
d940aabf66
Incomplete, need to finish interval/intervalset tests
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10656 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-27 23:19:08 +00:00
e1bf680d13
pass data to server
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10655 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-26 22:28:33 +00:00
18720db594
Remove outdated chunk tests. Again, serves as a good example if we
...
need chunked streaming uploads in the future, but probably not
necessary right now.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10654 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-26 21:36:27 +00:00
19c70bf887
Remove the cherrypy ChunkedRFile fixup, and the disable_prb tool that
...
disables process_request_body. They'll be necessary if we want to
support streaming chunked uploads, but at this point it doesn't seem
like that's going to be necessary.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10653 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-26 21:35:33 +00:00
a672119dd0
Minor cleanups for test passage
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10652 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-26 21:31:37 +00:00
f721e41f2b
Minor cleanups for test passage
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10651 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-26 21:31:35 +00:00
853639f390
better error code for curl-generated problems
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10650 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-26 21:29:04 +00:00
ba11e4467f
Parsing happens before serialization on server... now need to put in database
...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10649 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-26 21:23:58 +00:00