diff --git a/docs/design.md b/docs/design.md index 39bccb5..7f84e1f 100644 --- a/docs/design.md +++ b/docs/design.md @@ -266,3 +266,40 @@ Each table contains: from the end of the file will not shorten it; it will only be deleted when it has been fully filled and all of the data has been subsequently removed. + + +Rocket +------ + +Original design had the nilmdb.nilmdb thread (through bulkdata) +convert from on-disk layout to a Python list, and then the +nilmdb.server thread (from cherrypy) converts to ASCII. For at least +the extraction side of things, it's easy to pass the bulkdata a layout +name instead, and have it convert directly from on-disk to ASCII +format, because this conversion can then be shoved into a C module. +This module, which provides a means for converting directly from +on-disk format to ASCII or Python lists, is the "rocket" interface. +Python is still used to manage the files and figure out where the +data should go; rocket just puts binary data directly in or out of +those files at specified locations. + +Before rocket, testing speed with uint16_6 data, with an end-to-end +test (extracting data with nilmtool): + +- insert: 65 klines/sec +- extract: 120 klines/sec + +After switching to the rocket design, but using the Python version +(pyrocket): + +- insert: (hasn't yet been switched completely; more complicated due + to error handling when parsing ASCII) +- extract: 120 klines/sec + +After switching to a C extension module (rocket.c) + +- insert: +- extract: + + +