|
|
@@ -389,3 +389,35 @@ Possible solutions: |
|
|
|
are always printed as int64 values, and a new format |
|
|
|
"@1234567890123456" is added to the parser for specifying them |
|
|
|
exactly. |
|
|
|
|
|
|
|
Binary interface |
|
|
|
---------------- |
|
|
|
|
|
|
|
The ASCII interface is too slow for high-bandwidth processing, like |
|
|
|
sinefits, prep, etc. A binary interface was added so that you can |
|
|
|
extract the raw binary out of the bulkdata storage. This binary is |
|
|
|
a little-endian format, e.g. in C a uint16_6 stream would be: |
|
|
|
|
|
|
|
#include <endian.h> |
|
|
|
#include <stdint.h> |
|
|
|
struct { |
|
|
|
int64_t timestamp_le; |
|
|
|
uint16_t data_le[6]; |
|
|
|
} __attribute__((packed)); |
|
|
|
|
|
|
|
Remember to byteswap (with e.g. `letoh` in C)! |
|
|
|
|
|
|
|
This interface is used by the new `nilmdb.client.numpyclient.NumpyClient` |
|
|
|
class, which is a subclass of the normal `nilmcb.client.client.Client` |
|
|
|
and has all of the same functions. It adds three new functions: |
|
|
|
|
|
|
|
- `stream_extract_numpy` to extract data as a Numpy array |
|
|
|
|
|
|
|
- `stream_insert_numpy` to insert data as a Numpy array |
|
|
|
|
|
|
|
- `stream_insert_numpy_context` is the context manager for |
|
|
|
incrementally inserting data |
|
|
|
|
|
|
|
It is significantly faster! It is about 20 times faster to decimate a |
|
|
|
stream with `nilm-decimate` when the filter code is using the new |
|
|
|
binary/numpy interface. |