There's some bug with the testing harness where placing e.g.
from du import du
in nilmdb/utils/__init__.py doesn't quite work -- sometimes the
module "du" replaces the function "du". Not exactly sure why;
we work around that by just renaming files so they don't match
the imported names directly.
Doesn't actually merge them yet; need to change Interval
implementation to allow deletes.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@11354 ddd99763-3ecb-0310-9145-efcb8ce7c51f
- Flesh out tests for the new nilmdb.layout.Formatter
Coverage doesn't handle the cython module, so this is just
functional stuff, not necessarily complete.
Still need to finish each Layout.format()
- Split out test_client_5_chunked from test_client_4_misc
so it's easier to skip while debugging. Turning off streaming
lets us see tracebacks from within the server's content()
functions.
- More work on stream/extract in cmdline, client, server, nilmdb.
Still needs work on server side, but should be complete in nilmdb.
- Start nilmdb.layout.Formatter class
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10888 ddd99763-3ecb-0310-9145-efcb8ce7c51f
to the database, not the client. The server now maintains the open
HTTP connection and sends a continuous streaming reply to the GET
request.
HTTP client side uses an Iteratorizer to turn the curl.perform()
callback into an interator, and returns an iterator that yields
individual lines to the caller rather than buffering up all the data
at once. Should still be able to handle errors etc.
Server changed to return a "streaming JSON" instance for the
/stream/interval requests. This is just a series of independent
JSON documents (one per interval), separated by newlines.
Adjust nilmdb's max_results a bit. Now, multiple requests only exist
between the server <-> nilmdb threads, and they exist just to avoid
blocking the nilmdb thread by any one server thread for too long.
So adjust the size accordingly to match the fact that this is non-json
encoded data.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10881 ddd99763-3ecb-0310-9145-efcb8ce7c51f
On the big database, the server takes a few seconds to figure out the
interval intersections. Need to think about how to improve that --
the real key might be to start reducing the number of intervals we're
storing by combining them, potentially as they're inserted.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10838 ddd99763-3ecb-0310-9145-efcb8ce7c51f
variable instead now. Adjust tests accordingly.
Start list --detail option, using stream/intervals request.
Frontend should be ready, backend needs implementation.
Put interval adding back into nilmdb:_add_interval so things work.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10833 ddd99763-3ecb-0310-9145-efcb8ce7c51f
Make nilmdb.cmdline a proper class
Fix various stuff
Add /dbpath command to get DB path
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10677 ddd99763-3ecb-0310-9145-efcb8ce7c51f
More work with nilmdb.stream_insert. Checks for overlap between
parser output and database intervals, and actually inserts the
data into pytables. Need to benchmark/figure out whether we can
use table.append rather than the row.append() nonsense -- it could
simplify things quite a bit.
Improve layout class and add tests to get more coverage. Better
error handling on invalid inputs (reports the reason for the error)
Replace layout.fillrow with layout.filltable, but turns out that we
can probably just remove it anyway.
Add nilmdb.Timer for simple timing tests
Move some duplicated test suite helper functions into a new file,
test_helper.py
Add class to test_interval.py to match the others
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10661 ddd99763-3ecb-0310-9145-efcb8ce7c51f
tolerance in equality comparisions, but it's a real pain to get all
the edge cases right, and it's not clear we actually need that
functionality at the moment. Skip it for now.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10659 ddd99763-3ecb-0310-9145-efcb8ce7c51f
Add "end" parameter to TimestamperRate, to force the file to end
before a specific time. Will be good for loading in old files and
knowing that things don't overlap.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10644 ddd99763-3ecb-0310-9145-efcb8ce7c51f
go away if it gets packaged in Debian etc.
Add nilmdb.timestamper classes
Start test_timestamper (needs work)
Number the functions in test_client so they run in the right order to
leave the database in a good state.
Fix some brokenness with module importing and namespaces
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10643 ddd99763-3ecb-0310-9145-efcb8ce7c51f
