nilm/nilmdb
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Commit Graph

  • cb6561c151 Fix some comments and text Jim Paris 2012-08-09 18:26:45 +00:00
  • 407aedcd20 Can't represent 2112 on 32-bit systems Jim Paris 2012-08-09 18:04:47 +00:00
  • bf8ff66c77 Fix timezone issues in cmdline test Jim Paris 2012-08-09 18:03:18 +00:00
  • 82f753efb1 disable TestLayoutSpeed for now Jim Paris 2012-08-09 17:53:01 +00:00
  • e950794866 more info Jim Paris 2012-08-08 22:00:54 +00:00
  • cb7c0cf83e Test roundtrip times for parser and formatter. Can optimize now Jim Paris 2012-08-08 21:58:09 +00:00
  • 33cb7031a3 Test float32_8 type names in addition to PrepData aliases Jim Paris 2012-08-08 19:23:28 +00:00
  • 33492fc851 Update TODO Jim Paris 2012-08-06 22:06:29 +00:00
  • 5101522025 Tests pass with 100% coverage on non-cython modules Jim Paris 2012-08-06 22:05:09 +00:00
  • 5130ab7e6a Start reworking the layout types. Jim Paris 2012-08-06 21:46:09 +00:00
  • 27024fb38e Clarify interval openness Jim Paris 2012-07-25 21:03:11 +00:00
  • ff4e934bef Add database size to "nilmtool.py info" output. Jim Paris 2012-07-24 21:52:38 +00:00
  • a1d09fb5fc TODO update Jim Paris 2012-07-24 21:18:01 +00:00
  • aefaac2405 Add "--count" option to extract Jim Paris 2012-06-25 18:52:50 +00:00
  • e96cff4fc5 update TODO Jim Paris 2012-06-25 18:14:39 +00:00
  • 97bec3b1ee Optimization that uses slices on the table rather than checking each row individually, when extracting data. Jim Paris 2012-06-04 23:46:33 +00:00
  • 27f8dcf06d Extract works now and we're back to 100% test coverage! It's slow, though; need to figure out where the slowdown lies. Jim Paris 2012-05-30 18:24:36 +00:00
  • cb97ad3d2c Interval bisection to find the correct database row seems to work in nilmdb now. Needs testing, and nilmdb's stream_extract needs to be finished Jim Paris 2012-05-29 23:49:08 +00:00
  • 8a7b3b5f95 Add DBInterval and associated tests. Clean up other interval tests as well, and fix missing coverage on IntervalSet.str() Jim Paris 2012-05-25 21:45:05 +00:00
  • 11cc124019 Formatters done, not necessarily fast enough though Jim Paris 2012-05-25 19:02:58 +00:00
  • e2daeb5e54 - Some updates to max_results handling on server side Jim Paris 2012-05-25 16:44:24 +00:00
  • cbc7c5125d WIP on getting stream extract to work. Jim Paris 2012-05-24 21:05:38 +00:00
  • 27fd9d54f9 Streaming responses from the server weren't actually streaming. Fix that. This requires a bit of restructuring of server.py:intervals() to allow us to properly report errors before beginning the stream. Jim Paris 2012-05-23 23:18:24 +00:00
  • e5e7ae9eda Fix bug that caused responses to always be truncated early Jim Paris 2012-05-23 20:19:06 +00:00
  • 315bc57ac3 More complete test coverage of nilmdb.httpclient, yay Jim Paris 2012-05-23 20:00:01 +00:00
  • 3b0b9175d6 For stream_interval, make the server handle sending multiple requests to the database, not the client. The server now maintains the open HTTP connection and sends a continuous streaming reply to the GET request. Jim Paris 2012-05-23 19:08:59 +00:00
  • e570800396 Add "iteratorizor", which transforms a function that uses callbacks into a generator. Needed to get pycurl's WRITEFUNCTION to give us a generator instead. It's a bit tricky and uses threads and queues, but seems to work OK. Jim Paris 2012-05-23 01:00:12 +00:00
  • c327378373 Split MyCurl into a HTTPClient class instead Jim Paris 2012-05-21 21:50:01 +00:00
  • aaffd61e4e rework MyCurl stuff Jim Paris 2012-05-21 21:43:30 +00:00
  • b32edb1ed6 Undo last commit Jim Paris 2012-05-18 21:49:52 +00:00
  • 624980a47b Started moving to python-requests so we can do a get() with an interator on the response, but it turns out that it's really crappy with regards to socket closing, so I'll revert this in a moment. Jim Paris 2012-05-18 21:49:29 +00:00
  • 3f436e6dfd Use simplejson instead of json for performance Jim Paris 2012-05-16 22:19:00 +00:00
  • d647ea7eee Cleanup handling of truncated intervals, more work on extract Jim Paris 2012-05-11 18:31:11 +00:00
  • 9ff30f8c1d ignore generated files Jim Paris 2012-05-11 15:20:52 +00:00
  • e3be1a1d8a Add IntervalSet.intersection(). This returns a generator that allows us to look at just some of the intervals without having to reconstruct an entire IntervalSet class -- which greatly reduces server load when handling requests that cover large interval ranges. Jim Paris 2012-05-10 21:54:29 +00:00
  • f63e58f2d9 Cleanup cmdline by splitting it into multiple files. Essentially the individual subcommands are still treated as being part of the same class (and use "self" to refer to the Cmdline class), but they're different modules now. Jim Paris 2012-05-10 20:35:51 +00:00
  • a05a026bc7 Make stream_intervals use a generator, so the client library handles re-issuing the request when the output data gets truncated. Works well. Jim Paris 2012-05-10 18:14:57 +00:00
  • 1d875b1f1f Start extract command in cmdline.py Jim Paris 2012-05-08 21:27:56 +00:00
  • f4f2493b59 Some small speed improvements. Intervals are really a pain; need to rework how this all works together, but will probably move on to extraction now. Jim Paris 2012-05-08 17:40:11 +00:00
  • 4501da6edc Can query intervals now. Jim Paris 2012-05-07 22:32:02 +00:00
  • 36045fe53b Remove --utc option from command line tool -- set TZ environment variable instead now. Adjust tests accordingly. Jim Paris 2012-05-04 22:36:27 +00:00
  • 7eef39d5fd notes Jim Paris 2012-05-04 16:08:32 +00:00
  • de7f78a43b which are not necessary anymore Jim Paris 2012-05-03 21:14:58 +00:00
  • fab3567d74 instructions for blist Jim Paris 2012-05-03 21:14:46 +00:00
  • 6d6514d5c3 Go back to bxinterval. The blist version was slower. Jim Paris 2012-05-03 21:14:24 +00:00
  • b67fe79e47 Use blist instead of bxtree Jim Paris 2012-05-03 21:13:55 +00:00
  • 064b4bf715 notes Jim Paris 2012-05-03 19:18:07 +00:00
  • e08be4c2a8 misc Jim Paris 2012-05-03 19:17:48 +00:00
  • 0276810776 More timing tests Jim Paris 2012-05-03 19:17:35 +00:00
  • 565d0e98a9 Remove trailing spaces Jim Paris 2012-05-03 19:17:23 +00:00
  • 47245df9bd nilmdb now caches the intervals the first time a particular stream's intervals are accessed, so it doesn't need to keep rebuilding them as long as it's running. Jim Paris 2012-04-24 21:59:33 +00:00
  • c07670ac3e Make the whole nilmdb.layout parser Cython, and rewrite the parsing bits so it's hopefully quite a bit faster now. Jim Paris 2012-04-24 21:00:26 +00:00
  • 37b4376b4c design notes Jim Paris 2012-04-13 21:00:33 +00:00
  • 3b52ecafa3 Clean up unused code Jim Paris 2012-04-13 20:53:03 +00:00
  • ac32647fac Remove PyTables indexing support; it's not necessary since we're just using pytables as a row-indexed data store. Jim Paris 2012-04-13 20:51:18 +00:00
  • 4e143dfa18 docs Jim Paris 2012-04-13 20:47:56 +00:00
  • b84ffddd50 New bxinterval implementation is at least linear -- not much better in the TestIntervalSpeed versus the previous bisect version, but should be better in general. Jim Paris 2012-04-12 22:39:16 +00:00
  • 1531114677 fix typos in test_interval; test big numbers and floats Jim Paris 2012-04-12 21:05:54 +00:00
  • ab4c1f0925 Update bxintersect to use doubles, not ints. Cython is easy! Jim Paris 2012-04-12 20:26:33 +00:00
  • f7149e48e8 Add intersection.pyx from bx-python Jim Paris 2012-04-12 20:16:46 +00:00
  • 055cfa12b2 Various work on improving intervals... Jim Paris 2012-04-11 22:05:27 +00:00
  • 5cb03cd6ef Update design.md. Add commit after each insert Jim Paris 2012-04-10 20:52:59 +00:00
  • 70bcc6d2b3 design notes Jim Paris 2012-04-10 19:58:16 +00:00
  • 43d3daa840 Testing Jim Paris 2012-04-09 22:46:04 +00:00
  • 7dfa288270 Can now insert data using command line tool. Time to benchmark etc. Jim Paris 2012-04-09 21:31:39 +00:00
  • 881b9a7bd1 add timestamped version Jim Paris 2012-04-09 20:49:08 +00:00
  • e85acdd20c Fill out and fix parse_Time Jim Paris 2012-04-09 19:35:14 +00:00
  • d725ed1771 date/time parsing, needs work Jim Paris 2012-04-06 20:54:48 +00:00
  • d58a27e2bf Better coverage Jim Paris 2012-04-06 18:37:05 +00:00
  • f3b0dfabea More work on commandline and commandline tests. Jim Paris 2012-04-06 18:25:09 +00:00
  • cccaec326a Need to figure out a better way to handle this Jim Paris 2012-04-04 22:57:12 +00:00
  • a3f444eb25 A lot more command line testing. There'es some issue with tons of requests getting slowly blocked, though... Jim Paris 2012-04-04 22:34:01 +00:00
  • 277b0c1d00 More command line tests. Make nilmdb.cmdline a proper class Fix various stuff Add /dbpath command to get DB path Jim Paris 2012-04-04 18:54:24 +00:00
  • 7bba4a80d9 Work on command line client, and some improvements to server handling of bad URLs Jim Paris 2012-04-03 22:21:42 +00:00
  • f29d38d9d9 More work on nilmtool Jim Paris 2012-04-03 19:43:37 +00:00
  • b4a0288a39 work on command line Jim Paris 2012-04-03 18:01:56 +00:00
  • e63ab23d20 stream_insert puts data into the database now. Fill out tests to cover overlapping data, missing data, etc. Jim Paris 2012-03-31 04:15:29 +00:00
  • 776279a4e6 Go back to generic parsing for most stuff now. Need to rework what's most efficient here. Jim Paris 2012-03-30 22:15:04 +00:00
  • 7a9012c3e9 work on improving layout parsing Jim Paris 2012-03-30 21:29:22 +00:00
  • 8c619cfde5 Minor cleanups Jim Paris 2012-03-29 21:44:57 +00:00
  • cc4e3bdb76 Rename some nilmdb methods to be private Jim Paris 2012-03-29 21:43:05 +00:00
  • 3dfd187710 Use instantiated classes for layouts. Jim Paris 2012-03-29 16:46:34 +00:00
  • cf66eca42c Spent time trying to get Intervals and IntervalSets to handle a gap tolerance in equality comparisions, but it's a real pain to get all the edge cases right, and it's not clear we actually need that functionality at the moment. Skip it for now. Jim Paris 2012-03-28 19:25:02 +00:00
  • 1046d0c47b more tests Jim Paris 2012-03-28 00:22:44 +00:00
  • d940aabf66 Incomplete, need to finish interval/intervalset tests Jim Paris 2012-03-27 23:19:08 +00:00
  • e1bf680d13 pass data to server Jim Paris 2012-03-26 22:28:33 +00:00
  • 18720db594 Remove outdated chunk tests. Again, serves as a good example if we need chunked streaming uploads in the future, but probably not necessary right now. Jim Paris 2012-03-26 21:36:27 +00:00
  • 19c70bf887 Remove the cherrypy ChunkedRFile fixup, and the disable_prb tool that disables process_request_body. They'll be necessary if we want to support streaming chunked uploads, but at this point it doesn't seem like that's going to be necessary. Jim Paris 2012-03-26 21:35:33 +00:00
  • a672119dd0 Minor cleanups for test passage Jim Paris 2012-03-26 21:31:37 +00:00
  • f721e41f2b Minor cleanups for test passage Jim Paris 2012-03-26 21:31:35 +00:00
  • 853639f390 better error code for curl-generated problems Jim Paris 2012-03-26 21:29:04 +00:00
  • ba11e4467f Parsing happens before serialization on server... now need to put in database Jim Paris 2012-03-26 21:23:58 +00:00
  • 17073905f7 Working on data parsing... Jim Paris 2012-03-26 19:54:04 +00:00
  • 3b696e2a03 rework errors Jim Paris 2012-03-26 18:06:15 +00:00
  • dd5658f987 Better handling on server side -- insert now reads the body. Jim Paris 2012-03-26 16:47:11 +00:00
  • 54847b0710 Upload the data in the POST. Client side should be mostly good, for now Jim Paris 2012-03-25 19:12:49 +00:00
  • b72e276eb8 More timestamper tests. Add "end" parameter to TimestamperRate, to force the file to end before a specific time. Will be good for loading in old files and knowing that things don't overlap. Jim Paris 2012-03-24 21:15:24 +00:00
  • 0808ed5bd8 Add datetime_tz module from python-datetime-tz repository. This could go away if it gets packaged in Debian etc. Jim Paris 2012-03-24 17:32:11 +00:00
  • ec25eac697 Misc cleanups, fix some pylint issues Jim Paris 2012-03-23 15:50:33 +00:00
  • 09340d543d Add some test prep data Jim Paris 2012-03-23 14:56:52 +00:00