Calibration of instruments is key to attain reproducibility!

Measurements that do not measure up

Researchers all over the world have been consistently populating the field of research with exciting new discoveries. Often, fellow researchers would wish to reproduce previous experimental results before proceeding. This is especially true for characterizations, where the original research provides a basis for comparison for subsequent perturbations to the system.

However, the difficulty in reproducing comparable experimental data as described in literature is a prominent issue. Many factors can potentially contribute to the discrepancies observed, of which the inherent difference between the instruments used is one of the most probable.

According to Dr. Yeoh Jing Wui, an expert in computational modelling, models constructed using parametric values (e.g. enzyme kinetics, protein degradation rates) obtained from one particular instrument are usually non-transferrable to another set of data collected using a different instrument. A reliable interpretation of two sets of experimental results thus requires the consideration of variation between instruments, but it is tedious to obtain the necessary data experimentally.

Our measured approach: Synchronized fluorescence measurement

Our team aimed to ensure that the parts we submitted can be easily utilized by others. On top of providing characterization data, we also want to evaluate the repeatability of our experimental design and reproducibility of our results by introducing a practical approach as described below.

In this particular experimental design, we were performing characterizations on the EL222 blue light repressible system (BBa_K2819103) to serve as part improvement. The measurements include the following parameters: cell density at 600nm (OD₆₀₀), as well as fluorescence intensity of RFP with an excitation and emission maxima of 535 and 600 nm. To mimic experimental trials whereby researchers aim to replicate results from previous literature, we perform synchronised characterization experiments in separate laboratories, where readings were taken using different microplate readers. Cell cultures and necessary reagents were provided independently, and a detailed protocol was followed to the letter in both labs.

After the experiments, we eagerly plotted the data (Figure 1).

Despite being mentally prepared, we could not help but feel disappointed at the results. As can be seen, the absolute values obtained from the two labs differed by a substantial margin. This had significant implications: the results were not reproducible, thus no meaningful interpretation of these two sets of data could be established. This also meant that subsequent modeling efforts could only take into considerations data acquired from one of the microplate readers, and other iGEMmers or researchers from all over the world looking to reproduce our characterization results and models would inevitably be let down.

Not all is lost, however. A quick glance at the graphs told us that the trend is largely preserved. We hypothesized that the discrepancies in the absolute values obtained was likely due to the inherent difference in the instruments, i.e. the microplate readers, used. To further evaluate this, we plotted data points obtained from Lab A against that from Lab B, complete with trendlines with equations and R² values (Figure 2).

Good linear relationships were established between the two data sets with an R² value of 0.9603. Making use of the linear equation derived, we calibrated data points obtained in Lab A and replotted them against those from Lab B (Figure 3). Indeed, the graphs now almost overlapped each other!

The immeasurable significance of good measurement

Why did we go through all these trouble?

In this measurement practice, we have demonstrated the inherent difference in instruments. The two microplate readers (Synergy H1 Hybrid Multi-Mode Reader) were manufactured by BioTek© in different years, which further proved that differences between instruments are inevitable, even when they are from the same supplier.

Our approach of evaluating the relationship between the experimental datas has allowed the subsequent calibration of raw datas. Hence, a reliable comparison could be established, where the calibrated values of one reader matched the original values of its counterpart. Reproducibility can thus be attained.

For us, this meant that we do not have to rely on a single microplate reader to obtain all our experimental data! Values obtained from the other reader can now be easily calibrated and subsequently used to fit our models.

For synthetic biologists all over the world, our approach suggested that there is a practice which can be followed such that reproducibility is more realizable. In fact, we propose that a routine of synchronized experiments and calibration of instruments be followed prior to data collection. This allows data to be compared across instruments, laboratories, and research.

What we achieved in our "IntraLab" study is hardly groundbreaking, but it is our hope that this simple procedure can eventually become the standard practice, with iGEM as the springboard.