|
|
@@ -266,3 +266,40 @@ Each table contains: |
|
|
|
from the end of the file will not shorten it; it will only be |
|
|
|
deleted when it has been fully filled and all of the data has been |
|
|
|
subsequently removed. |
|
|
|
|
|
|
|
|
|
|
|
Rocket |
|
|
|
------ |
|
|
|
|
|
|
|
Original design had the nilmdb.nilmdb thread (through bulkdata) |
|
|
|
convert from on-disk layout to a Python list, and then the |
|
|
|
nilmdb.server thread (from cherrypy) converts to ASCII. For at least |
|
|
|
the extraction side of things, it's easy to pass the bulkdata a layout |
|
|
|
name instead, and have it convert directly from on-disk to ASCII |
|
|
|
format, because this conversion can then be shoved into a C module. |
|
|
|
This module, which provides a means for converting directly from |
|
|
|
on-disk format to ASCII or Python lists, is the "rocket" interface. |
|
|
|
Python is still used to manage the files and figure out where the |
|
|
|
data should go; rocket just puts binary data directly in or out of |
|
|
|
those files at specified locations. |
|
|
|
|
|
|
|
Before rocket, testing speed with uint16_6 data, with an end-to-end |
|
|
|
test (extracting data with nilmtool): |
|
|
|
|
|
|
|
- insert: 65 klines/sec |
|
|
|
- extract: 120 klines/sec |
|
|
|
|
|
|
|
After switching to the rocket design, but using the Python version |
|
|
|
(pyrocket): |
|
|
|
|
|
|
|
- insert: (hasn't yet been switched completely; more complicated due |
|
|
|
to error handling when parsing ASCII) |
|
|
|
- extract: 120 klines/sec |
|
|
|
|
|
|
|
After switching to a C extension module (rocket.c) |
|
|
|
|
|
|
|
- insert: |
|
|
|
- extract: |
|
|
|
|
|
|
|
|
|
|
|
|