octave: a=load("log-cut"); len=size(a,1) t=1:10000; keith=a(:,3); calib=a(1,4) dac=a(:,5); adc=a(:,6); meas=-(dac - calib * adc); plot(keith(t),meas(t)) p = polyfit(keith, meas, 1) t=1:len; plot(t,keith(t) * p(1) + p(2) - meas(t)) axis([0 len -1 1]) Range is something like +- 0.5 over the entire range (of "DAC values") This gives log(1024 / 1.0) / log(2) = 10 bits No good! Now let's convert the DAC values to their more-accurate value using the lookup table lookup=load("../../../firmware/lookup.inc"); size(lookup) meas=-(lookup(dac + 1)/64 - calib * adc); p = polyfit(keith, meas, 1) t=1:len; plot(t,keith(t) * p(1) + p(2) - meas(t)) axis([0 len -0.3 0.3]) #Now closer to +- 0.1 #This gives log(1024 / 0.1) / log(2) = 13.3 bits