Tuesday 12 October 2021

More on FSD and resolution

 These two voltmeter readings show how changing the FSD on the scale alters the resolution of the instrument and thus the precision of the reading. In the first picture the FSD is 200V so the resolution is +-0.1V.

But a reading of 7.9V will also fit on the FSD 20V scale. This time the resolution is +-0.01V so the reading is more precise (to more decimal places). Interestingly 7.99V should surely round up to 8.0V. I'll need to consider if there is an obvious reason for this not working out.
The FSD 2000mV scale would give even more decimal places but 2000mV is 2V so we would be off the scale and get no reading.