Previous commits went back and forth a bit on whether the various APIs
should use bytes or strings, but bytes appears to be a better answer,
because actual data in streams will always be 7-bit ASCII or raw
binary. There's no reason to apply the performance penalty of
constantly converting between bytes and strings.
One drawback now is that lots of code now has to have "b" prefixes on
strings, especially in tests, which inflates this commit quite a bit.
Things are now block-focused, rather than line-focused. This should
give a pretty big speedup to inserting client data, especially when
inserting preformatted data.
Now nilmdb.client, nilmdb.server, nilmdb.cmdline, and nilmdb.utils
are each their own modules, and there is a little bit more of a
logical separation between them. Various changes scattered throughout
to fix naming (for example, nilmdb.nilmdb.NilmDBError is now
nilmdb.server.errors.NilmDBError).
Reduced usage of "from __future__ import absolute_import" as much
as possible. It's still needed for the functions in the nilmdb/server
directory to be able to import the nilmdb module rather than the
nilmdb.py script.
This should hopefully ease future packaging a bit.
There's some bug with the testing harness where placing e.g.
from du import du
in nilmdb/utils/__init__.py doesn't quite work -- sometimes the
module "du" replaces the function "du". Not exactly sure why;
we work around that by just renaming files so they don't match
the imported names directly.
Current/old design has specific layouts: RawData, PrepData,
RawNotchedData.
Let's get rid of this entirely and switch to simpler data types that
are
just collections and counts of a single type. We'll still use strings
to describe them, with format:
type_count
where type is "uint16", "float32", or "float64", and count is an
integer.
nilmdb.layout.named() will parse these strings into the appropriate
handlers. For compatibility:
"RawData" == "uint16_6"
"RawNotchedData" == "uint16_9"
"PrepData" == "float32_8"
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10981 ddd99763-3ecb-0310-9145-efcb8ce7c51f
though; need to figure out where the slowdown lies.
Add stream existence check to server's /intervals and /extract paths,
add tests for it.
Make start and end arguments optional for /extract, like /intervals
Move --quiet command line option to just the insert subcommand.
It's the only one that uses it right now, and otherwise it doesn't
show up in after a "nilmtool.py intervals --help". Might revisit this
later if more commands start supporting --quiet.
Change cmdline/extract's write into a print, to keep the trailing
newline.
Fix lingering uses of Interval in nilmdb and change to DBInterval
instead.
Fix nilmdb interval bisection:
- handle common case optimization correctly
- db_endpos is always one after the last row, so use hi=db_endpos-1
Finish nlimdb stream_extract
Add a bunch of cmdline tests for extract, particularly testing border
cases around start/end. Compares output to a set of files stored in
the tests/data dir.
Some more tests in test_client to get better coverage.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10893 ddd99763-3ecb-0310-9145-efcb8ce7c51f
individual subcommands are still treated as being part of the same
class (and use "self" to refer to the Cmdline class), but they're
different modules now.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10850 ddd99763-3ecb-0310-9145-efcb8ce7c51f
Jim Paris