Article Text
Abstract
Background Reticulocytes are the most sensitive index available to authorities who seek to sanction athletes for blood doping based on deviations beyond individual reference ranges. Because such data comprise longitudinal results that are generated by different laboratories, the comparability of reticulocyte counts from different instruments is of crucial importance.
Aims To enhance between-instrument comparability of reticulocyte counts reported by the Sysmex XT-2000i automated haematology analyser.
Methods We optimised recalibration of instruments towards assigned values of control material (e-CHECK) in tandem with fresh blood verification.
Results In terms of reticulocyte counts reported as a percentage of all cells in a fresh blood sample, it was possible to recalibrate all three test instruments so that the mean of 10 samples was within 0.1% of the comparative instrument's mean value.
Conclusions This approach provides a straightforward means of reducing between-instrument differences in reticulocyte counts generated by the Sysmex XT-2000i.
- Reticulocyte Counts
- Quality Control
- Haematology
Statistics from Altmetric.com
Introduction
Reticulocytes, which typically constitute around 1% of all red cells in circulation, are transient entities which can be detected for ∼24 h after they have been expelled from the bone marrow and before they transition into mature red blood cells (RBCs).1 Reticulocyte counts yield an accurate index of erythropoietic activity, yet show relatively modest within-subject variation over time, even in elite athletes participating in strenuous training and competition.2 However, blood doping is recognised to cause abnormally large changes in reticulocyte counts.3 Antidoping authorities attempting to detect athletes who may have blood doped collect venous blood samples and scrutinise reticulocyte values for evidence of atypical fluctuations above or below the athlete's individual reference range.4 This approach has become mainstream only since 2008 and is known generically as the Athlete Biological Passport.
Since international-calibre athletes travel extensively in order to train and compete on the global stage, a dedicated network of approximately 35 laboratories have been accredited around the world by the World Anti-Doping Agency (WADA) to conduct blood tests on behalf of antidoping authorities. Results from one laboratory are compared directly with results obtained from other facilities. Subsequently, because even subtle fluctuations within an athlete's longitudinal profile may carry great evidentiary weight that could ultimately lead to the athlete being sanctioned and banned from their sport, the comparability of reticulocyte results between laboratories is of paramount importance.
To minimise bias between instruments, laboratories are required to use analysers with comparable technical characteristics.4 In practice, this has led to more than 30 laboratories utilising the Sysmex XT-2000i haematology analyser, and the remainder to use the (more elaborate) Sysmex XE-2100. Satisfactory instrument performance is monitored via monthly proficiency tests overseen by the WADA, wherein the same quality control (QC) material is tested in multiple laboratories. However, these proficiency results are derived from the analysis of control materials, whose stabilised cell membranes react differently to the instrument's reagents compared with fresh blood cells. Such matrix effects among control materials are well known in haematology.5
In order to compensate for this physical difference between stabilised and fresh blood cells, Sysmex instruments utilise a different procedure (‘QC’ mode) when assaying control material compared to when fresh blood samples are run (‘Patient’ mode). Therefore the validity of proficiency test results, which interrogate only the QC-based mode of instrument performance, rests upon the assumption that the QC-derived results will satisfactorily reflect any between-instrument bias associated with the Patient mode of analysis.
Manufacturers such as Sysmex must validate instruments for their intended analytical application, and existing QC-based proficiency tests are deemed satisfactory in clinical settings. However, the Athlete Biological Passport benefits from exacting levels of instrument performance; moreover the Clinical and Laboratory Standards Institute guidelines explicitly encourage laboratories to instigate procedures based upon their specific operating conditions.5
Therefore the aim of this study was twofold. First, we sought to ascertain how precisely the Sysmex instrument's QC-derived results correlated with fresh blood results, and specifically whether biases evident in fresh blood results were satisfactorily reproduced in QC data. Second, we investigated whether it was possible to minimise reticulocyte bias between instruments via implementation of an optimised instrument calibration procedure.
Methods
Control materials
We sought to monitor instrument performance when measuring both stabilised and fresh blood samples. For stabilised cells we utilised e-CHECK, which is a stabilised whole blood matrix specifically designed for the Sysmex X series analysers developed by Streck Laboratories (Omaha, USA). The product is provided in three levels that vary in concentration by parameter (L1, L2 and L3). Although not technically a calibrator, L2 e-CHECK is routinely used by technicians to calibrate Sysmex instruments (personal communication). L2 reticulocyte levels (∼2.6%) are somewhat higher than what are normally found in athlete populations, and so our experiments also incorporated L3 e-CHECK (∼0.8% reticulocytes) where appropriate. For fresh blood we used aliquots of a single fresh blood specimen shared between laboratories. Specifically, prior to experiments one laboratory collected venous blood samples from each of 10 volunteers into K2EDTA tubes (Becton-Dickinson, Plymouth, UK). One tube from each volunteer was retained on-site while the additional tubes were shipped to the other laboratories. To minimise the likelihood of disparate sample degradation between collection and analysis, all batches of tubes were stored in identical storage devices whether the batch remained on-site or was transported by air to the other laboratories. Furthermore, the subsequent analyses of samples occurred simultaneously (ie, within 60 min) at each site to ensure that samples had been exposed for the same duration to those storage conditions.
Comparison of reticulocyte counts derived from fresh and stabilised blood samples
Ten laboratories, each operating an XT-2000i ,were recruited to participate in the first experiment. Ten K2EDTA tubes were consecutively drawn from each volunteer and numbered 1–10 based on the order in which they were drawn from the subject. To control for any effect of ‘draw order’ on blood variables (eg, extended tourniquet application) a Latin square design was used to allocate tubes to respective laboratories, so that for each laboratory one of the samples used had been drawn at each of the 10 ‘orders’. Fresh blood samples were distributed to each laboratory approximately 1 week in advance of the e-CHECK vials. We compared the mean value of 10 fresh blood samples analysed in singleton with the mean value from seven consecutive assays of the L3 e-CHECK vial. All laboratories measured fresh samples in ‘Patient’ mode and e-CHECK in ‘QC’ mode. Furthermore, six of the 10 laboratories also measured the e-CHECK material using the ‘Patient’ mode of sample analysis.
Calibration of reticulocyte counts
Prior to any recalibration, we established the instrument's baseline performance (‘before’), measuring stabilised as well as fresh blood samples by calculating the average reticulocyte percentage of fresh blood samples collected from 10 volunteers, as well the average of seven replicates of level 3 e-CHECK. Post-adjustment these same analyses were repeated (‘after’) in order to document the changes induced by the recalibration procedure.
Our calibration protocol focused on three discrete facets that determine the reticulocyte value reported by the XT-2000i instrument: laser alignment and performance; spatial positioning of the RBC cloud in relation to reticulocyte gates; and adjustment of the calibration factor.
The XT-2000i instrument differentiates and characterises cell types using fluorescence flow cytometry which generates forward scatter (FSC), side scatter (SSC) and side fluorescence (SFL) signals (for a detailed description and evaluation, readers are referred to Langford6).Therefore the protocol commenced with verifying that both the alignment (ie, Sysmex recommend a specific focal point for reliable results) and performance (ie, width of the beam at the critical focal point) of the laser in terms of both FSC and SSC adhered as close as possible to the relevant targets provided by Sysmex. This verification was performed using latex beads (FSC via Latex Cat no 4207A, Lot 37728, Duke Scientific Corporation, Palo Alto, California, USA; SSC and SFL via AlignFlow 2.5 um alignment beads, Invitrogen Molecular Probes, Eugene, Oregon, USA).
With respect to RBC characterisation, the instrument generates a ‘cloud’ of dotpoints which are dispersed on a scattergram according to each cell's FSC (y-axis) and fluorescence (x-axis) value. The manufacturer refers to the central position of the red cell cloud as having a specific ‘RBC-X’ and ‘RBC-Y’ coordinate. We initially conducted an experiment to familiarise ourselves with the impact of changing the RBC-X sensitivity gain on both QC and fresh blood results (ie, increasing RBC-X will shift the cloud to the right and therefore tend to increase the number of dotpoints falling within the ‘reticulocyte gated boundary’ and thus increase reported reticulocyte counts, see figure 1).
Adopting the initial position as 100%, the RBC-X was adjusted by increments (95% through to 104% of the initial value). At each increment we recorded the average reticulocyte count of three fresh blood samples that were assayed in triplicate. In a follow-up experiment, one fresh and one stabilised blood sample (L2 e-CHECK) were analysed in singleton at each of those same RBC-X settings. Based on the insights gleaned from these experiments, and given that Streck states that achievement of those targets using control material will in turn yield the most accurate reading on fresh human blood, our calibration protocol sought to align the RBC-X and RBC-Y values of L2 e-CHECK as closely as possible to the targets identified on the vial label.
Somewhat less emphasis was then given to calibration of the absolute reticulocyte counts (RETIC #), optical RBC count (RBC-O), RBC impedance (RBC-I) and finally the optical platelet count (PLT-O) which is also performed in the reticulocyte channel.
Only after satisfactory calibration of those parameters was the final step of adjustment of the calibration factor (CAL FACTOR) undertaken. This is a software-based adjustment which multiplies the number of dotpoints lying within the reticulocyte gated region. This allows the technician to numerically adjust up or down the reticulocyte counts independent of the position of the cell cloud on the graphical plot. We adjusted CAL FACTOR until the absolute reticulocyte count was as close as possible to the L2 e-CHECK target.
Results
Comparison of fresh and stabilised results from XT-2000i instruments
To illustrate the effect that using the ‘QC’ mode had on the number of cells in a given sample which are designated as reticulocytes, figure 2 plots the results from stabilised control material (L3 e-CHECK) measured in both ‘QC’ and ‘Patient’ modes by six of the laboratories. As clearly illustrated in relation to the line of identity, when the XT-2000i instrument uses the ‘QC’ mode to measure reticulocyte counts, values are lower than when the sample is measured in ‘Patient’ mode.
In addition, we found that QC-derived results did not correlate with fresh blood results, and specifically that biases evident between different instruments when analysing fresh blood samples were not reproduced in QC results. When the reticulocyte counts of 10 different instruments were compared for fresh blood samples and the L3 e-CHECK stabilised cells, the correlation between the mean values of fresh (n=10) and stabilised (n=7) blood was r=−0.19 (p=0.6, figure 3).
Sensitivity adjustments via RBC-X
There was a progressive increase in reported reticulocyte counts as RBC-X was increased from 95% through to 104% of the initial value (figure 4). Specifically, a 1% increase in RBC-X yielded on average an absolute increase of 0.05% in the reported percentage of reticulocytes in fresh blood samples. However, adjustment of the instrument's RBC-X setting had different impacts on fresh and stabilised blood samples. As depicted in figure 5, results from fresh samples were found to be more responsive to RBC-X changes than were stabilised samples (p=0.001 for test of equality of slopes).
Moreover, we found that despite maintaining the instrument's RBC-X setting within the accepted tolerances for RBC-X as specified on the L2 e-CHECK vial (ie, 15.3–30.9), the reticulocyte count reported when the same sample was reanalysed at each RBC-X increment varied between 0.7% and 1.17%—that is, a range of 0.47% for fresh blood despite the instrument operating within manufacturer's specifications.⇓
Recalibration and alignment of instruments
Our first step was to establish a point of reference, by recalibrating one analyser and designating this as our comparative instrument. For didactic purposes, pertinent findings during the subsequent recalibration of three different instruments are outlined below.
Test instrument #1
A comparison of the average reticulocyte count in 10 fresh blood samples ‘before’ alignment established a bias of 0.31% wherein the test instrument's mean of 0.840% was 27% lower than the comparative instrument's mean of 1.145%, while L3 e-CHECK was 25% lower (0.853% vs 1.130%, mean of n=7). Following adjustment to bring RBC-X and RBC-Y as close as possible to the L2 e-CHECK target, the 10 fresh samples were re-run but a virtually unchanged bias of 0.29% was still present. RETIC #, RBC-O and RBC-I were found to be not on target and were adjusted to the L2 e-CHECK target. ‘After’ adjustment, the bias in fresh blood results had been reduced to 0.08% (1.009% vs 1.091% or 8%, test and comparative, respectively) while L3 e-CHECK was 0.016% higher in the test than the comparative instrument (1.106% vs 1.090% or 1%).
Test instrument #2
A second analyser was found to have a bias of 0.24% compared to the comparative instrument, based on the average of 10 fresh blood samples (1.137% vs 1.372%, or 17% difference), while L3 e-CHECK was 19% lower (0.746% vs 0.925%, mean of n=7). RBC-X was increased to align with target values while RBC-Y was found to already align closely to target. Neither RBC-I, RBC-O or PLT-O required any adjustment since they were all on target. The CAL FACTOR was increased from 1010 up to 1170. Because these changes to the test instrument were made within 2 h of the ‘before’ measurements being conducted, we did not consider it necessary to conduct ‘after’ analyses in the comparative laboratory because we considered fresh blood samples to be stable over this period. Therefore any changes in the test laboratory could be attributed solely to instrument adjustments. Subsequently, ‘after’ adjustment (and compared to ‘before’ results from the comparative instrument), the average of 10 samples in the test instrument was 1.356% which demonstrated that the bias in fresh blood results had been reduced to 0.02% which was 2% lower than the comparative instrument, while L3 e-CHECK was 3% lower (0.917%) after the instrument had been aligned.
Test instrument #3
In contrast to the first two machines, fresh blood results from a third test instrument were found to already align closely to the comparative instrument ‘before’ any adjustments had been made (1.485% vs 1.545%, a bias of 0.06% or 4% lower). Both sensitivity (RBC-X, RBC-Y) and other parameters (RBC-I, RBC-O, PLT-O) were closely aligned to target and required no adjustment. The CAL FACTOR was increased from 1265 to 1299, which was expected to increase the reported value for reticulocytes. However when the 10 fresh samples were re-run there was no evident difference from ‘before’ (ie, 1.455% post-adjustment in the test instrument). An evening spent troubleshooting the instrument discovered no obvious anomalies. Subsequently the entire instrument including software was restarted next morning. Fresh samples were then re-run at both the test and comparative laboratory to compensate for any degradation of samples overnight. ‘After’ results from the test instrument showed that the bias had been reduced to 0.001% which was 0.1% lower (1.501% vs 1.502%) than the comparative instrument, while L3 e-CHECK was 2% lower (0.97% vs 0.99%).
Discussion
Our study has shown that a 0.31% bias in reticulocyte counts can exist between two XT-2000i instruments which are both operating within the manufacturer's specifications. We also demonstrated that recalibrating instruments towards the assigned value of control materials, instead of a generic tolerance, brought reticulocyte counts into close alignment.
When contemplating possible explanations as to how two of our test instruments demonstrated absolute biases of 0.24% and 0.31% when counting reticulocytes in fresh blood compared to our comparative instrument, we conclude that the most likely origin stems from how the separate instruments were calibrated during installation. Standard installation procedures do not require the technician to recover QC reticulocyte values from the Sysmex XT-2000i (other than to verify that in terms of precision the reticulocyte percentage and absolute numbers have a CV<15%). Instead, calibration materials are used to establish the other channels (eg, the 16-parameter haemogram plus 5-part white blood cell differential), then the technician merely confirms using 10 normal range fresh blood samples that the average reticulocyte value for those 10 samples lie within the instrument's reference interval (ie, an XT's typical reference range is approximately 0.64–1.65%). This would seem to provide relatively generous tolerances. For example, in the case of our first test instrument reporting 0.31% low based on the average of 10 fresh blood samples, that instrument would still yield a result that would fall within an acceptable range provided that the true average value of those 10 samples lay between 0.95% and 1.96% (ie, 0.64%+0.31% and 1.65%+0.31%, respectively). Under that hypothetical circumstance there would have been no basis for the installing technician to have refined the instrument's set-up during installation. Likewise, our data show that the manufacturer's tolerance for RBC-X sensitivity adjustments means that an instrument's fresh blood reticulocyte counts can span a range of 0.47% without failing the manufacturer's QC-based performance specifications. In other words, it seems tenable that a bias in the order of 0.3–0.5% could exist for the reticulocyte percentage reported by two properly calibrated XT-2000i instruments.
We have shown that it is tenable to remove bias between instruments down to at least one decimal place; in fact in our hands a test instrument replicated the comparative instrument's reticulocyte counts down to two decimal places. We consider either to be zero bias in the context of the Athlete Biological Passport. Our original hypothesis was that because the XT-2000i uses different approaches depending whether stabilised or fresh samples are tested, alignment of reticulocyte counts would necessarily require the comparison of fresh blood results between instruments. However, we found that excellent alignment could also be achieved merely by calibrating each instrument to the assigned value of control materials. This possibility has important implications for those WADA-accredited labs testing athlete samples, because an alignment protocol which utilised surrogate samples would avoid having to transport fresh blood to remote laboratories within the limited shelf life associated with this live tissue specimen. However, as proposed by the CLSI's standard on validation, verification and quality assurance of haematology analysers, when possible fresh blood should be part of an overall QC programme.5 A sensible compromise to enhance linkage between QC-derived data and reportable patient results might be to fortify a QC-based approach with localised fresh blood ring studies. For example, regional laboratories within close proximity may elect to optimise alignment by sharing fresh blood samples with their immediate neighbours. Without too much coordination one member could compare with a different regional cohort of laboratories and therefore propagate the confirmation beyond their localised region.
The benefit that improved between-laboratory comparability of reticulocyte counts brings to antidoping efforts is important but deceptively subtle. Currently, there is a two-step process followed before an athlete can be sanctioned via data derived from Complete Blood Count results. The first step entails a statistical program which flags abnormal blood values that lie beyond the athlete's individual reference range. These reference ranges are generated with a tolerance for both within- and between-subject components of variation, which far exceed the magnitude of between-laboratory variation. Decreasing these variance components by modest amounts has surprisingly little impact on the tolerance thresholds. Subsequently, because the between-laboratory error component is dwarfed by the within-subject component, reducing the variance has little impact on the statistical process.
However, regardless of the statistics, an athlete is not considered to have committed an antidoping rule violation until and unless during the second step an expert review of the haematological data concludes that the most likely cause of the abnormal blood result was doping (as opposed to, for example, a pathology or analytical issue). This expert review shares a common lineage with how clinical haematologists evaluate serial change of reported results in a given patient, inasmuch as both groups are obliged to factor into their considerations an allowance for between-laboratory differences. A subjective allowance of 0.2–0.3% is typical of the buffer afforded in the athlete's favour when blood is tested in different laboratories. Reducing the between-laboratory bias to within 0.1% or lower, as we have shown is possible to accomplish, would effectively mean that experts could interpret all results as if they had been collected on the same instrument. This would reduce the subjective tolerances made for potential between-laboratory bias, and thereby provide additional certainty to their opinions.
With regard to instrument-to-instrument correlation, although Sysmex Corporation publish expectations for basic haematology parameters (eg, white blood cell count ±5.0%, RBC count ±2.5%, haemoglobin ±2.0%, haematocrit ±2.5%, mean cell volume ±2.5%, platelet count ±7.0%) we could find no reference to the expected correlation of reticulocyte counts between instruments. Therefore there is no benchmark against which to interpret a 0.31% bias (ie, a 27% difference in results) in reticulocyte count obtained from instruments of the same make and model. However the post-adjustment differences of 8%, 2% and 0.1% in reticulocyte counts between each of our test instruments and the comparative instrument compare favourably with expectations for other haematology parameters.
Take-home message
-
Optimising the XT-2000i's calibration to quality control material effectively removed between-instrument differences in fresh blood reticulocyte counts that were otherwise tolerated by normal quality assurance processes.
Acknowledgments
This project has been carried out with the support of the World Anti-Doping Agency.
Footnotes
-
Contributors All authors contributed to the design of the experiment and generation of the manuscript.
-
Funding World Anti-Doping Agency.
-
Competing interests None.
-
Provenance and peer review Not commissioned; externally peer reviewed.