Commit Graph

15 Commits

Author SHA1 Message Date
6f1e6fe232 Isolate all PyTables stuff to a single file.
This will make migrating to my own data storage engine easier.
2012-12-31 17:22:29 -05:00
f355c73209 Refactor utility classes into nilmdb.utils subdir/namespace
There's some bug with the testing harness where placing e.g.
  from du import du
in nilmdb/utils/__init__.py doesn't quite work -- sometimes the
module "du" replaces the function "du".  Not exactly sure why;
we work around that by just renaming files so they don't match
the imported names directly.
2012-12-31 15:55:36 -05:00
82f753efb1 disable TestLayoutSpeed for now
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10995 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-08-09 17:53:01 +00:00
e950794866 more info
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10991 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-08-08 22:00:54 +00:00
cb7c0cf83e Test roundtrip times for parser and formatter. Can optimize now
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10990 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-08-08 21:58:09 +00:00
33cb7031a3 Test float32_8 type names in addition to PrepData aliases
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10989 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-08-08 19:23:28 +00:00
5101522025 Tests pass with 100% coverage on non-cython modules
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10982 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-08-06 22:05:09 +00:00
5130ab7e6a Start reworking the layout types.
Current/old design has specific layouts: RawData, PrepData,
RawNotchedData.
Let's get rid of this entirely and switch to simpler data types that
are
just collections and counts of a single type.  We'll still use strings
to describe them, with format:

    type_count
  
where type is "uint16", "float32", or "float64", and count is an
integer.

nilmdb.layout.named() will parse these strings into the appropriate
handlers.  For compatibility:
  
    "RawData" == "uint16_6"
    "RawNotchedData" == "uint16_9"
    "PrepData" == "float32_8"


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10981 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-08-06 21:46:09 +00:00
11cc124019 Formatters done, not necessarily fast enough though
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10889 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-05-25 19:02:58 +00:00
e2daeb5e54 - Some updates to max_results handling on server side
- Flesh out tests for the new nilmdb.layout.Formatter
  Coverage doesn't handle the cython module, so this is just
  functional stuff, not necessarily complete.
  Still need to finish each Layout.format()

- Split out test_client_5_chunked from test_client_4_misc
  so it's easier to skip while debugging.  Turning off streaming
  lets us see tracebacks from within the server's content()
  functions.

- More work on stream/extract in cmdline, client, server, nilmdb.
  Still needs work on server side, but should be complete in nilmdb.

- Start nilmdb.layout.Formatter class


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10888 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-05-25 16:44:24 +00:00
3f436e6dfd Use simplejson instead of json for performance
Misc cleanups

More work on tests to get more coverage


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10869 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-05-16 22:19:00 +00:00
c07670ac3e Make the whole nilmdb.layout parser Cython, and rewrite the parsing
bits so it's hopefully quite a bit faster now.


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10798 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-04-24 21:00:26 +00:00
776279a4e6 Go back to generic parsing for most stuff now. Need to rework what's
most efficient here.


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10664 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-30 22:15:04 +00:00
7a9012c3e9 work on improving layout parsing
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10663 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-30 21:29:22 +00:00
cc4e3bdb76 Rename some nilmdb methods to be private
More work with nilmdb.stream_insert.  Checks for overlap between
parser output and database intervals, and actually inserts the
data into pytables.  Need to benchmark/figure out whether we can
use table.append rather than the row.append() nonsense -- it could
simplify things quite a bit.

Improve layout class and add tests to get more coverage.  Better
error handling on invalid inputs (reports the reason for the error)

Replace layout.fillrow with layout.filltable, but turns out that we
can probably just remove it anyway.

Add nilmdb.Timer for simple timing tests

Move some duplicated test suite helper functions into a new file,
test_helper.py

Add class to test_interval.py to match the others


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10661 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-29 21:43:05 +00:00