Commit Graph

231 Commits

Author SHA1 Message Date
ba11e4467f Parsing happens before serialization on server... now need to put in database
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10649 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-26 21:23:58 +00:00
17073905f7 Working on data parsing...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10648 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-26 19:54:04 +00:00
3b696e2a03 rework errors
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10647 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-26 18:06:15 +00:00
dd5658f987 Better handling on server side -- insert now reads the body.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10646 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-26 16:47:11 +00:00
54847b0710 Upload the data in the POST. Client side should be mostly good, for now
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10645 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-25 19:12:49 +00:00
b72e276eb8 More timestamper tests.
Add "end" parameter to TimestamperRate, to force the file to end
before a specific time.  Will be good for loading in old files and
knowing that things don't overlap.


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10644 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-24 21:15:24 +00:00
0808ed5bd8 Add datetime_tz module from python-datetime-tz repository. This could
go away if it gets packaged in Debian etc.

Add nilmdb.timestamper classes

Start test_timestamper (needs work)

Number the functions in test_client so they run in the right order to
leave the database in a good state.

Fix some brokenness with module importing and namespaces



git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10643 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-24 17:32:11 +00:00
ec25eac697 Misc cleanups, fix some pylint issues
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10642 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-23 15:50:33 +00:00
09340d543d Add some test prep data
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10641 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-23 14:56:52 +00:00
913883be3a Fix slow database stuff in testing by setting PRAGMA synchronous=OFF. Default is still FULL for production use, though
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10639 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-22 22:06:08 +00:00
8cc1dff0b8 Cleanup
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10637 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-22 21:00:01 +00:00
5b2d52b8bc Big rework of how errors are handled and propagated to the HTTP
response.  Now everything (status code, message, possibly tracebacks)
are passed to the client as JSON, so the client can display correct errors.
Nice!


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10630 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-22 00:14:13 +00:00
22ef82b59c More work on client tests
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10625 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-21 16:28:51 +00:00
a235c94c02 More work towards filling out the client...
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10624 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-20 22:21:16 +00:00
795d2ac7cf ignore
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10623 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-20 21:46:37 +00:00
1c4efb92c6 rename command line client nilmdb.py to nilmtool.py
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10622 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-20 21:46:18 +00:00
56a1770f45 client tests
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10611 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-20 14:26:29 +00:00
b766aef257 new client py
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10610 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-20 14:22:02 +00:00
3e5d0ef87d rename
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10609 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-20 14:21:55 +00:00
a26468c059 Rename client -> cmdline
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10608 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-20 14:19:45 +00:00
0099a41fd8 Work on a client implementation that can survive testing
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10604 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-20 01:18:30 +00:00
b1baacf272 cleanup
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10600 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-19 21:36:24 +00:00
8afaf8f329 Improve insert strutcure
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10562 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-14 20:29:30 +00:00
610860c379 Update
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10556 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-13 22:12:30 +00:00
c076af64af for reference
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10548 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-12 20:19:32 +00:00
d0435cbf91 for reference
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10547 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-12 20:19:29 +00:00
ace199ffa8 Working client-side pycurl code to send a streaming chunked PUT.
Working server-side cherrypy code to receive a streaming chunked PUT.


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10546 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-03-12 20:09:17 +00:00
5b89fa7ccf Fix style
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10446 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-02-21 21:30:53 +00:00
d8929467eb Cleanup
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10445 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-02-20 06:13:54 +00:00
ba66668fff playing with coverage a bit
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10444 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-02-20 06:09:42 +00:00
1e1c7fa9c4 Create StreamException class in NilmDB, which can get caught and
handled by the HTTP server.

Add /stream/get_metadata HTTP request.

Fill out test_stream_metadata

Add runserver.py script to just run the server from the command line,
outside of the testing environment.  Add necessary hooks within
nilmdb/server.py to allow this to happen.

Metadata operations get stream_id first and report error on missing
stream, instead of returning an empty dict.


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10443 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-02-20 05:52:33 +00:00
9b64a18daf Remove nilmdbsql; move functions into nilmdb.py instead
TODO                |    2 
 nilmdb/__init__.py  |    1 
 nilmdb/nilmdb.py    |  113 +++++++++++++++++++++++++++++++++++++++++-----
 nilmdb/nilmdbsql.py |  126 ----------------------------------------------------
 4 files changed, 102 insertions(+), 140 deletions(-)



git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10428 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-02-17 20:42:08 +00:00
398d382dac Stuff goes into the SQL database now.
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10427 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-02-17 00:32:36 +00:00
53e3c44186 Move schema updates to a dict
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10421 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-02-15 23:11:37 +00:00
4395f68662 Start SQL stuff
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10419 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-02-14 21:54:30 +00:00
c5ec07a661 Start working towards mixed sql/pytables db
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10373 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-02-08 19:28:16 +00:00
4ca726439b Branch
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10372 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-02-08 18:11:31 +00:00
72deddcd9b tags
git-svn-id: https://bucket.mit.edu/svn/nilm/tags/nilmdb-pytables@10371 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-02-08 18:07:32 +00:00
bc1d0260eb ignore
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10368 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-02-08 18:01:21 +00:00
ef2db81d11 Misc
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb@10367 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-02-08 18:01:05 +00:00
6812a28131 Make stream_list return number of rows.
Update layouts to include expected number of rows per day -- this is
used (as strongly suggested by the PyTables docs) to form a rough
estimate of the number of rows in a table during CreateTable.


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb-new@10340 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-01-30 20:25:11 +00:00
7933775462 TODO
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb-new@10339 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-01-30 18:09:05 +00:00
3b9d84d689 Adjust how layouts are managed: now we pass strings representing the
layouts, rather than direct dict objects.  This allows us to go
backwards and match the description of tables in the database with the
layouts defined within nilmdb.layout.

Fix some big issues with the nilmdb.server and startup errors like an
invalid port number.  We have to catch os._exit and report the error
ourselves.

Add some timeouts and otherwise clean up tests.

Make NilmDB track and complain about files that weren't closed before
the object is destroyed.

Update stream_list and /stream/list to return the layout corresponding
to each table.



git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb-new@10338 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-01-28 00:36:23 +00:00
b5e3b22558 Fixups, and add serializer to NilmDB access from CherryPy
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb-new@10336 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-01-27 19:17:09 +00:00
93d7088af3 update TODO
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb-new@10335 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-01-27 01:28:45 +00:00
9f4d88cc8f Added serialization wrapper. This will let us wrap the NilmDB object
and ensure that all function calls happen from within the context of a
single thread.


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb-new@10334 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-01-27 01:27:21 +00:00
8da7de817c Update TODO
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb-new@10332 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-01-26 00:14:16 +00:00
acdea5f00b Integrated CherryPy server into tree
Added testing framework for cherrypy server.
Added tests for basic server functionality.


git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb-new@10331 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-01-26 00:13:44 +00:00
6c9cf198d7 indexing
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb-new@10330 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-01-23 21:07:59 +00:00
5fa3ff9610 TODO, and ignore stuff
git-svn-id: https://bucket.mit.edu/svn/nilm/nilmdb-new@10329 ddd99763-3ecb-0310-9145-efcb8ce7c51f
2012-01-23 20:38:37 +00:00