Add some info about binary interface to design docs
This commit is contained in:
parent
b4d6aad6de
commit
231963538e
|
@ -389,3 +389,35 @@ Possible solutions:
|
|||
are always printed as int64 values, and a new format
|
||||
"@1234567890123456" is added to the parser for specifying them
|
||||
exactly.
|
||||
|
||||
Binary interface
|
||||
----------------
|
||||
|
||||
The ASCII interface is too slow for high-bandwidth processing, like
|
||||
sinefits, prep, etc. A binary interface was added so that you can
|
||||
extract the raw binary out of the bulkdata storage. This binary is
|
||||
a little-endian format, e.g. in C a uint16_6 stream would be:
|
||||
|
||||
#include <endian.h>
|
||||
#include <stdint.h>
|
||||
struct {
|
||||
int64_t timestamp_le;
|
||||
uint16_t data_le[6];
|
||||
} __attribute__((packed));
|
||||
|
||||
Remember to byteswap (with e.g. `letoh` in C)!
|
||||
|
||||
This interface is used by the new `nilmdb.client.numpyclient.NumpyClient`
|
||||
class, which is a subclass of the normal `nilmcb.client.client.Client`
|
||||
and has all of the same functions. It adds three new functions:
|
||||
|
||||
- `stream_extract_numpy` to extract data as a Numpy array
|
||||
|
||||
- `stream_insert_numpy` to insert data as a Numpy array
|
||||
|
||||
- `stream_insert_numpy_context` is the context manager for
|
||||
incrementally inserting data
|
||||
|
||||
It is significantly faster! It is about 20 times faster to decimate a
|
||||
stream with `nilm-decimate` when the filter code is using the new
|
||||
binary/numpy interface.
|
||||
|
|
Loading…
Reference in New Issue
Block a user