Print

Instrument Accuracy

Dimension Instrument Instrument
Precision
Instrument Accuracy
(max. relative error)
Common
Error Limiting
Factor
Length meter stickCenco 73115 0.5 mm < 0.5% visual
Length Vernier
calipersCraftsman 40257
0.05 mm 0.1% misreading scale
Mass electronic
balanceOhaus GT 4000
0.1 g 0.1% to 1% calibration
Time digital
stopwatchPASCO SE-8755
0.01 s ~0.001% reaction time
(~0.2 s)
Time photogatePASCO ME-9206A 5 ms 0.50% data processing
Time Smart TimerPASCO ME-8930 0.1 ms 0.01% of full scale calibration
Frequency signal
generatorInstek FG-8016G
0.1 Hz 1% calibration
Freefall
Time
Tape TimerPASCO ME-9283 0.1% obstruction
Time/dist Spark
generatorDaedalon EA-50
0.1% wrinkled paper,
spin on
inelastic
Angle Digital LevelSpi-Tronic Pro 3600 0.05° 0.05° unstable level
Table 1: Instrument Accuracy - Mechanics
Dimension Instrument Instrument
Precision
Instrument Accuracy
(max. relative error)
Common
Error Limiting
Factor
V, I, R multimeterRadio Shack 22-163, Micronta 22-167 4.5 digits DCV: 1%     ACV: 1.5%
DCA (< 30 mA): 1%
DCA (0.3 to 10 A): 2%
R (< 300 kΩ): 1%
R (0.3 to 3 MΩ): 2%
extra resistance,
calibration
V, I, R multimeterFluke 77 Series II 4.5 digits 0.3% for 1 mV to 320 V
1.5% for 0.1 mA to 10 A
0.5% for 1Ω to 3.2 MΩ
calibration
V, I, R multimeterKeithley 178 DMM (see bottom of meter) extra resistance,
calibration
Capacitance capacitance
meterBK 810B
3.5 digits 0.5% for 200 pF to 0.2 mF
2% for 0.2 mF to 20 mF
calibration,
stray capac.
Inductance LCR meterBK 875 A 3.5 digits 3% for C < 0.2 mF
3% for L < 20 H
calibration
V, freq. oscilloscopeTektronix 2211, 2213 2 digits 3% for both gain and sweep visual resol.,
calibration
Resistance Decade
resistorGeneral Radio Co. 1432-N, 1432-A, 1432-J
10% calibration
Resistance Resistance
subst. boxHeath EUW-28
7 Ohms 10% calibration
Inductance Decade
inductorWide Range Electronics ALD-12
1 mH 10% calibration
Mag. Field Hall ProbeFW Bell 5070 0.01 mT ±2% DC, ±3.5% AC probe
alignment
Table 2: Instrument Accuracy - Electricity and Magnetism