Some processing thoughts: zcat log.gz | grep ^1 | tail -n +200000 > log-locked octave: a=load("log-locked"); len=size(a,1) t=1:10000; keith=a(:,3); calib=a(1,4) dac=a(:,5); adc=a(:,6); meas=-(dac - calib * adc); plot(keith(t),meas(t)) # should be linear p = polyfit(keith, meas, 1) plot(t,keith(t) * p(1) + p(2),t,meas(t)) # should match t=1:len; plot(t,keith(t) * p(1) + p(2) - meas(t)) # absolute error (units=dac) axis([0 len -1 1]) Range is something like +- 0.5 over the entire range (of "DAC values") This gives log(1024 / 1.0) / log(2) = 10 bits No good! Now let's convert the DAC values to their more-accurate value using the lookup table lookup=load("../../../firmware/lookup.inc"); size(lookup) meas=-(lookup(dac + 1)/64 - calib * adc); p = polyfit(keith, meas, 1) t=1:len; plot(t,keith(t) * p(1) + p(2) - meas(t)) # absolute error (units=dac) axis([0 len -0.3 0.3]) Now closer to +- 0.05 This gives log(1024 / 0.1) / log(2) = 13.3 bits