The following sequence could lead to this corruption:
  (1) Append new data to bulkdata
  (2) Update interval file positions in SQL
  (3) Flush (2)
  (4) Crash before flushing (1)
  (5) Reload database without running fsck
  (6) Start writing new data to end of bulkdata and introduce new interval

This is effectively the number of problems with the database that can
be fixed, since we restart fsck after each one.

This probably means that the file was written, and metadata was journaled,
but the system crashed before data was written.  If that happens, the
end of the file will be zeroed out.  We don't bother checking the entire
file here; if we see just one timestamp that is unexpectedly zero, let's
truncate the data there.

Since we process in chunks, we need to add "start" to the row number;
however, we don't need to use this when accessing data['timestamp']
(aka ts)

Some databases were seeing _format truncated to 0 bytes, after crashing
soon after a new stream was created (e.g. during decimation).

A bulkdata dir may get created for a new stream with an empty or
corrupted _format, before any data gets actually written.  In that
case, we can just delete the new stream; worst case, we lose some
metadata.

Note: The info in _format should really get moved into the database.
This was born when bulkdata switched from PyTables to a custom storage
system, and was probably stored this way to avoid tying the main DB
to specific implementation details while they were still in flux.

The called functions already raise RetryFsck.

Git doesn't save empty dirs, so put a placeholder there that is
ignored when the test data is copied to its final location during
testing.

This was never updated when timestamps were changed from floats to
ints.

This doesn't affect anything in nilmdb, but is needed by nilmrun.

Some pylint errors are worth fixing; many are not.

We now have full coverage of the main code.

This used to have problems with older CherryPy versions, but the
current one seems to handle it just fine.

With this solution, we can catch it cleanly in the standalone
nilmdb-server, and test the error paths in our normal test suite.

It works again now.

This is hard and finicky to test, so there's unfortunately not a clear
way to get 100% test coverage in cherrypy_start.  However, that
function is only used in the test suite and the standalone
nilmdb-server script, not in production (which goes through wsgi and
skips all this cherrypy server stuff entirely).

Non-embedded mode is not used in the test suite or wsgi server;
it was an option in the standalong nilmdb-server script, but it's
really not necessary, and removing it gets rid of some untested
code.