Topic Highlight Open Access
Copyright ©2014 Baishideng Publishing Group Inc. All rights reserved.
World J Clin Oncol. Dec 10, 2014; 5(5): 931-965
Published online Dec 10, 2014. doi: 10.5306/wjco.v5.i5.931
Clinical application of DNA ploidy to cervical cancer screening: A review
David Garner, Integrative Oncology Department, British Columbia Cancer Agency, Vancouver, V5Z 1L3, Canada
Author contributions: Garner D solely contributed to this paper.
Correspondence to: David Garner, PhD, Integrative Oncology Department, British Columbia Cancer Agency, 675 w10th Ave., Vancouver, V5Z 1L3, Canada. dmgarner@telus.net
Telephone: +1-604-6758081 Fax: +1-604-6758099
Received: December 30, 2013
Revised: April 21, 2014
Accepted: May 16, 2014
Published online: December 10, 2014
Processing time: 346 Days and 5 Hours

Abstract

Screening for cervical cancer with DNA ploidy assessment by automated quantitative image cytometry has spread throughout China over the past decade and now an estimated 1 million tests per year are done there. Compared to conventional liquid based cytology, DNA ploidy has competitive accuracy with much higher throughput per technician. DNA ploidy has the enormous advantage that it is an objective technology that can be taught in typically 2 or 3 wk, unlike qualitative cytology, and so it can enable screening in places that lack sufficient qualified cytotechnologists and cytopathologists for conventional cytology. Most papers on experience with application of the technology to cervical cancer screening over the past decade were published in the Chinese language. This review aims to provide a consistent framework for analysis of screening data and to summarize some of the work published from 2005 to the end of 2013. Of particular interest are a few studies comparing DNA ploidy with testing for high risk human papilloma virus (hrHPV) which suggest that DNA ploidy is at least equivalent, easier and less expensive than hrHPV testing. There may also be patient management benefits to combining hrHPV testing with DNA ploidy. Some knowledge gaps are identified and some suggestions are made for future research directions.

Key Words: Cervical cancer screening; DNA ploidy; Automated quantitative image cytometry; High risk HPV testing

Core tip: Although application of automated quantitative image cytometry to screen for cervical cancer was first developed in Canada, the United States and Europe, it is most widely used clinically in China where it is applied to about one million tests annually. Over sixty papers reporting the clinical results have been published in Chinese since 2005. As the first review of this topic in any language, in addition to the usual goals of a review, it has the opportunity to increase the awareness of the Chinese clinical experience for those outside of China and to increase awareness of the technology background for English readers in China.



INTRODUCTION

In a 2005 paper based on a study conducted in China[1], this reviewer and coworkers stated the study objective as: “To establish if measurements of DNA ploidy could be used to assist cytopathologists and cytotechnologists in population based cervical cancer screening programs in countries where manually reading the slides is impossible due to the lack of sufficient skilled cytotechnologists.” Based on the study results, we concluded that it could work. Since that time, DNA ploidy technology has been approved for cervical cancer screening, endorsed by several medical societies and fairly widely disseminated throughout China. The goal of this paper is to review the current status of automated quantitative image cytometry (AQIC) to measure DNA ploidy as applied to cervical cancer screening.

The structure of this review is: (1) A very general introduction to the technology, with some historical perspective and with discussion of some practical issues; (2) A detailed development of a framework for evaluation of screening tests, mostly as a suggestion for how published data analysis might be made more clinically relevant. This section also attempts to alert readers to various well known and less well known pitfalls these evaluations are prone to and to estimate them to discriminate those that are important from those that are not; (3) A summary of existing published results, mostly from China, including reworking some of the published data in terms of the proposed framework; and (4) A discussion of some research still to be done especially in light of the huge advances in cervical cancer control made in the past decade due to the developments of human papilloma virus (HPV) vaccines and HPV testing. Does the objective from the 2005 paper still apply? Does DNA ploidy still have a potential cervical cancer screening role to substitute for nonexistent cytologists in this world of HPV vaccines and HPV testing?

TECHNOLOGY BACKGROUND
DNA ploidy

“Ploidy” is the genetics term for the number of basic sets of chromosomes in the nucleus of a cell. Cells that have an integer multiple of the basic set of chromosomes are “euploid”. Most human cells are euploid and have 46 chromosomes or two times the basic set of 23 chromosomes (one set from the mother and one from the father), referred to as “diploid”. Some human heart, liver and other cells are euploid with 92 chromosomes or 4 times the basic set and are known as “tetraploid”[2]. Human gametes have one set of the 23 chromosomes, unpaired, and so are also “euploid”. Curiously, mature red blood cells in mammals have no chromosomes.

Cells which do not have an integer multiple of the basic set of chromosomes are “aneuploid”, simply meaning “not euploid”. Most human embryos that have an extra or missing single chromosome do not survive gestation, but some do[3]; for example, Down’s syndrome occurs when there are three instances of chromosome 21 in all cells.

Some degree of aneuploidy is observed in virtually all solid tissue cancers[4] in a “mosaic”; that is, the normal cells remain “euploid” but the cancer tumor cells are “aneuploid”. (This contrasts with “non-mosaic” aneuploidy like Down’s syndrome in which almost all cells are aneuploid.) Elucidation of the role of aneuploidy in cancer has a fascinating history, briefly sketched next, and remains an active area of study today. The key point is that aneuploidy is the hallmark of cancer cells in general and, in the case of cervical cancer, is present both in the early “pre-cancer” or “pre-invasive cancer” phases as well as in the later “invasive” phases. Generally, to detect aneuploid cells is to detect cancer cells.

Aneuploidy and cancer: A brief history

By the year 1890, chromosomes had been discovered and, although their function was not yet proven, it was known that the material responsible for heredity was contained in the cell nucleus where chromosomes are found. The German pathologist, Hansemann[5], published a paper in 1890 entitled “About asymmetric cell division in epithelial cancers and its biological significance” and another in 1891[6] entitled “About pathological mitoses”; both papers are available online and contain wonderful hand drawn illustrations of asymmetric mitoses, as well as other phenomena, such as what is now known as apoptosis. Cells in the process of division are more commonly seen in tumors than in normal tissue and the division in normal tissue is almost always symmetric, producing identical daughter cells, whereas in cancer, one daughter cell often has more chromosomes than the other. These observations had been noted earlier by others, but Hansemann suggested that the defining characteristic of cancer cells is that they lose their ability to divide symmetrically and, in the process, cease to have tissue specialization and increase their ability to live more autonomously, as cancer cells do in metastasis[7].

Although both cameras and light bulbs were invented before 1890, they were not commonly available for microscopy and the illustrations in the journals were usually hand drawn by the author, sometimes assisted by “Abbe’s drawing apparatus”, rather than printed photomicrographs. Electric light commercialization slowly started in the 1880s, but incandescent light bulbs were not very practical until the invention of the tungsten filament in 1904. Also, it was not until 1893 that Köhler[8] published the method that bears his name and is still used on most modern transmission microscopes to evenly illuminate a microscope slide without also seeing an image of the light source superimposed on the specimen image. Yet, with minimal technology and very limited knowledge of chromosomes, genetics and cancer, Hansemann was able to make accurate observations and to formulate valuable hypotheses on the cellular mechanism of cancer.

A German zoologist named Boveri[9] studied multicentric cell division in double fertilized sea urchin eggs and, when he learned of the work of Hansemann on asymmetric cell division in cancer, he proposed a chromosomal theory of cancer in 1902 (an English translation is available online[10]). In essence, the theory is that cancer cells come from normal cells that divide asymmetrically for various reasons and, although this is usually fatal for the daughter cells, sometimes a daughter cell will survive and become the progenitor for all the subsequent cancer cells. The Boveri theory of cancer is that aneuploidy, resulting from an error in the division of a normal cell, is the cause of cancer.

Boveri was jointly credited with the American, Walter Sutton (working independently), in 1903 with the discovery that chromosomes are the vectors for heredity.

Boveri[11] published a more complete theory of cancer in 1914 (an English translation is available online[12]) and he died a few months later. Partly due to World War 1, this work was largely ignored. A translation into English was published by his wife in 1929[13] but received only limited attention outside of Germany until the 1970s. This paper included 20 important insightful hypotheses about cancer, anticipating such concepts as the existence of oncogenes, tumor suppressor genes and cancer induced by infectious agents. Arguably, all 20 hypotheses were subsequently shown to be substantially correct, starting some 50 years after his death[14]! The zoologist Boveri never studied or experimented with tumors or tumor cells and it has been speculated that it was because he was an “outsider” that he was able to form intuitive, imaginative and correct insights, uninhibited by the medical orthodoxy of the day[14].

Boveri’s hypothesis that aneuploidy is the cause of cancer remains unresolved today, but is a topic of active study[3,15-17]. Recent work shows that aneuploidy can be both a promoter and an inhibitor of cancer, depending upon the degree of chromosomal instability[18].

Again, the key point for this review is that generally to detect aneuploid cells is to detect cancer cells.

It should be noted that “definitive diagnosis”, including tumor type, is usually determined by a diagnostic test, not by the screening test (Figure 1). In the case of cervical cancers, colposcopy directed biopsy is the usual diagnostic test. Screening by detecting DNA aneuploidy alone cannot determine what type of tumor is present, although advanced cytometry techniques, beyond the scope of those discussed here, could make such determination.

Figure 1
Figure 1 A simplified flow diagram of the relationship between screening and diagnosis showing that screening generally has not 2 but 3 outcomes: positive, negative and equivocal, which is managed by time and re-testing.
DNA Cytometry

“Cytometry” means “cell measurement” which comes in two basic technology flavors: “flow” and “image” (sometimes called “static”) cytometry. There are a broad range of techniques to measure chromosomes, both in flow and image cytometry, ranging from the simple measurement of the total DNA content of a cell nucleus to the complex and sophisticated enumeration of small segments of individual chromosomes.

This paper is limited to consideration of AQIC that measures the total DNA content of cell nuclei, along with features that describe the distribution of the DNA within the nucleus and the morphology of each nucleus. These cytometer systems compare the DNA content of each cell nucleus measured to the average DNA content of the measured normal cell population[19]. Cervical “Pap” samples are predominantly comprised of normal cells, even when taken from a woman with invasive cervical cancer. This comparison identifies aneuploid cells even though the technique does not actually identify or count individual chromosomes. A cell with a DNA content 2.5 times that of a normal cell is reliably determined to be aneuploid, even though it is not possible to say that cell has, e.g., 115 chromosomes. This technique is also known as “quantitative DNA Cytometry” or just “DNA Cytometry”.

When AQIC was first developed, it was common to apply the genetic language of “ploidy”. Unfortunately, this can be misleading, especially in today’s world of genomics, because one will be inclined to infer that individual chromosomes are identified and counted. The term “DNA ploidy” was coined to be distinct from simply “ploidy” or “chromosomal ploidy” in an attempt to correct this sloppy language, with only middling success because today’s young life science students will certainly have encountered the terms of genetics and genomics, but are much less likely to have encountered DNA Cytometry. So it is important to understand that the AQIC technology discussed here does detect aneuploidy by cell measurement, but without explicitly enumerating chromosomes as a geneticist might expect[20].

How DNA is measured

Automated quantitative image cytometers are comprised of a microscope fitted with a digital camera, motorized X-Y stage, robotic slide loader and automatic focus mechanism, all under computer control. The cytometers of this review operate on absorbance microscopy.

Most cytometry relies on staining particular molecules in the specimen and then quantifying the stain. For the automated quantitative image cytometry considered here, the Feulgen[21] reaction stains the nuclear DNA specifically and proportionately to the amount of DNA present, everything else remains clear and unstained[22]. For cervical samples, this greatly simplifies the scene on the slide because the cell cytoplasm is left unstained, as is any red blood (no DNA). Only the DNA contained in the nuclei of epithelial cells, white blood cells and, if present, sperm cell nuclei are stained; the possibility of “false aneuploidy” arising from epithelial cells being overlaid by other epithelial cells, white blood cells or sperm cells is discussed later. Several microorganisms are commonly found in cervix samples, especially Döderlein’s bacillus which can completely cover the cervix epithelial cells; the staining protocol does not stain any of these microorganisms either.

Figures 2 are liquid based cytology (LBC) cervical samples with the blue DNA stain thionin (Figure 2A) and with the Pap stain used by conventional cytology (Figure 2B). The images were taken with the same optics (magnification, etc.) on the same cytometer. Notice that there are many more cells in the 2A image than in the 2B image and, even so, the scene is much simpler. Notice also that not all cells are in focus at the same time in either image due to the 3-dimensional stacking of the cells. Figure 3 shows a cluster of cells that is very difficult to examine due to the overlapping stained cytoplasm; this would not be a problem with only the DNA stain. Figure 4 shows “low grade abnormality” in the 3 Pap stained photos (arrows), which is especially difficult to see in Figure 4D. The large blue nucleus (Figure 4A) is aneuploid with a DNA content about 2.7 times normal. The key message is that the DNA stained slides are much simpler to measure and interpret than the conventional Pap stained slides.

Figure 2
Figure 2 Liquid based cervical samples stained with A: Feulgen thionin and B: Pap stain.
Figure 3
Figure 3 Liquid based cytology cervical sample example of a cell cluster obscured by cytoplasm.
Figure 4
Figure 4 Abnormal cell nuclei. A: Feulgen thionin stained; B, C, D: Pap stained (arrows).

Measurement of the amount of DNA with a cytometer is identical to the measurement of a chemical with a spectrometer. The basic idea of a spectrometer is shown in Figure 5-light is selected by a slit into a beam which directs it onto a monochromator. A particular color is selected by another slit, the beam is passed through the chemical sample which absorbs some of the light and any remaining light is detected with a photosensor. The concentration of the chemical in the sample can be precisely measured by applying a rule of physics known as Beer’s Law.

Figure 5
Figure 5 Basic principle of a spectrometer.

A quantitative cytometer is a very simplified spectrometer (Figure 6). The optimum light color is selected with color filter and passes through the glass microscope slide. Some of any light that passes through stained DNA is absorbed and the remaining light is detected by the pixels of a digital camera. Beer’s law applies separately to every pixel of the camera-in effect, a cytometer is a “Mega-Micro-Spectrometer”-that is, millions of tiny, pixel-sized spectrometers. The total DNA present in a nucleus is obtained by adding up the DNA measured by each pixel of the image of that nucleus. A recent extensive review discusses all of the technical considerations for quantitative cytometry[23].

Figure 6
Figure 6 Basic principle of an image cytometer.
AUTOMATED QUANTITATIVE IMAGE CYTOMETRY: OVERVIEW
Scanner operation

This review is focused on AQICs that feature “walk away automation” where barcoded stained slides are placed in a slide loader and the operator initiates scanning on a supervisor computer and the cytometer operates without further intervention. Although such machines can scan smears, they are generally employed with LBC samples; indeed, LBC was invented by Cytyc Corp. (now Hologic) with the intended goal to simplify the task of automated imaging[24]. All of the operations of focusing, image capture, segmentation of the scene into foreground and background, feature measurement etc. are performed without operator intervention.

Modern AQICs scan a LBC cervical slide in typically 5-10 min and are constantly getting faster, especially due to improvements in computer and digital camera speeds. While the scan rate is typically between 5 and 10 samples per hour, the throughput is generally defined by the slide loader capacity for overnight scanning and so, depending on the particular vendor’s design, can range from 75 to 250 per cytometer per day or 15000 to more than 50000 slides per cytometer per year, based on 225 working days per year. If a large volume of samples is available, as is the case for tests in an organized screening program, automated quantitative image cytometry can be very efficient.

Scan review and report generation

Reporting with an automated quantitative image cytometer is done at the conclusion of an interactive review of the scan data for each slide which is comprised of stored images of the cell nuclei, counts of various cell types, and histograms and scatter plots of cell DNA content and other features of the cell nuclei. The reviewer follows a very simple checklist procedure to systematically examine the data, looking: (1) first to check that the DNA scale (normalization) is valid, then; (2) checking any images that could be of aneuploid cell nuclei, then; (3) looking for aneuploid “stemlines”, then; (4) assessing cell proliferation; and (5) if none of these are present, then the case is negative and an assessment of scanning adequacy is made.

The idea that a scan has examined enough cells to be deemed to be a “satisfactory” or “adequate” assessment only applies to slides that are free of abnormality because slides deemed to be positive for abnormality are considered to be satisfactory regardless of how many cells were measured.

More than 90% of slides are clearly negative or clearly positive and so working through this checklist for those slides typically takes 1 min or less, including generating the report, once the technician has several days of training and experience. The remaining < 10% of cases are more problematic because their result hinges on the detailed assessment of typically 5 or fewer images of what could be aneuploid cells. The central issue is not “exactly what kind of cell is this an image of?”, but the much simpler question of “is this the image of a single cell nucleus or an artifact caused by the overlap of two or more cell nuclei or white blood cells or sperm?”

Figure 7 shows a mini image gallery. The leftmost image is clearly a single nucleus and the next image is clearly of overlapping nuclei as evidenced by the distinct overlapping boundaries and dark overlap area. In fact, the scan computer has no difficulty identifying the second image as being of overlapping cells and automatically places it into a separate group, so in “real life” the reviewer would be unlikely to ever look at this second image.

Figure 7
Figure 7 Gallery of potentially aneuploid cell nuclei.

The last image in the red box (previously shown in Figure 4A) is possibly more difficult because the shape suggests two overlapping ovals but the distinct boundaries and dark overlapping areas are absent. This is actually a single nucleus and the reviewer has the option of clicking on the image and the slide is automagically moved to place that cell into the center of the field of view of the review microscope and it is easily determined to be a single nucleus by panning slightly through focus.

Even when some images must be revisited with the review microscope, the overall task is simple and fast. Most revisit cases take less than 5 min each, so a typical reviewer can work through 40 cases per hour on a sustained basis (4 slides requiring revisiting plus 36 slides without revisiting). Some vendors provide review/revisit microscopes that are separate from the cytometer, while for other vendors the cytometer serves both scan and review/revisit functions.

The report that is generated summarizes the data showing a gallery of images of the most significant cell nuclei, a histogram of the DNA content, a scatter plot of area vs DNA content and a summary of cell counts in different DNA ranges.

In conventional cytology of the Pap test, most of the world reports according to TBS 2001 (The Bethesda System, revision of 2001[25]) which is very rich, very nuanced, very complicated and with rather poor inter- and intra-observer agreement[26]. From the viewpoint of patient management, there are really only 4 basic recommendations: (1) Return for another test at the routine screening interval (typically 3-5 years), the test is negative (no abnormality found); (2) Go to colposcopy within a few weeks for a detailed gynecological examination and possible biopsy, the test is positive (abnormality clearly found); (3) Return for another test at an interval shorter than the routine screening interval (typically 6 mo), the test was equivocal (too atypical to call “negative” but not enough to call “positive”)-such cases are usually resolved by time; and (4) Return for another test at an interval shorter than the equivocal repeat interval (typically 1 mo), there was no valid test result due to some process failure.

The basic scheme for screening is shown in Figure 1 (recommendation #4 above is not shown). The key focus and goal of the DNA ploidy test is to provide the correct management recommendation from this set of 4 for each woman. A set of objective rules is applied to the data in each report to make the correct follow up patient management recommendation.

These rules can be adjusted to adapt to the reality of each local situation. For example, in a setting where a woman is unlikely to be tested more than once or twice in her lifetime, it may be decided to adjust the rules to make the “negative predictive value” (the chance that a woman who has a negative test result is actually free of cervical cancer or significant pre-cancer) higher than it would be where more testing is done per lifetime. Or, if there are very limited gynecology and colposcopy resources, it may be required that the “positive predictive value” (the chance that a woman who has a positive test result actually has cervical cancer or significant pre-cancer) be adjusted so as to not overwhelm the diagnostic and therapeutic resources available.

The key advantage of DNA Cytometry is that it takes typically only 2-4 wk to train a technician (good high school graduate with some work experience) to competently perform all tasks: cell deposition onto glass slides, staining, operation of the DNA scanner, scan data review and reporting. This contrasts with the 1-2 years of special training for cytotechnologist[27,28] and 3-6 years of specialization training following receipt of an MD degree for a cytopathologist[29-32]. This is not to suggest equivalence between a minimally trained technician and skill of well trained cytotechnologists and cytopathologists, but it is to suggest comparable cervical cancer screening test performance, as summarized later in this review. It is this key advantage that makes automated quantitative image cytometry a candidate for screening in low resource settings where the conventional cytology Pap test cannot be performed due to the lack of trained cytotechnologists or cytopathologists. In such settings, AQIC could mean the difference between screening and not screening.

AQIC DNA PLOIDY: CLINICAL CONSIDERATIONS
Scanner operation: What constitutes a scan?

Two different endpoints are used to define what constitutes an adequate scan for AQIC of liquid based cervix slides: (1) scan all cells deposited on the slide; and (2) scan a preset number of epithelial cells on the slide. Option (1) is the approach primarily motivated by litigation-if an abnormal cell is present on the slide there could be liability. Option (2) is based on science and is completely consistent with current international cervical cancer screening guidelines[33].

What minimum number of epithelial cells must be measured for a scan to be satisfactory? A starting point to answer this question is to consider the guidelines for conventional cytology. TBS 2001[25] stated “Minimal squamous cellularity requirements for a specimen to qualify as “satisfactory”…(is) 5000 squamous cells for liquid-based preparations.” It has been reported[34] that this guideline was based on “...personal communications from the authors of two (subsequently published) papers[35,36].” In fact, reference[36] showed a threshold behavior of a jump in sensitivity for ASCUS+, LSIL+ and HSIL+ on Surepath (Becton Dickinson) LBC samples with < 5000 epithelial cells compared to those with > 5000 epithelial cells, supporting this guideline. However, reference[35] concluded that “Cellularity does not provide assurance of adequacy. Any cellularity criterion should be based on measurement of the prevalence of abnormal cells on abnormal slides.” McQueen[34] seems to be the only publication so far to take this approach with ThinPrep slides (Hologic) and yet they also conclude: “We have demonstrated that the range of ratios of dyskaryotic to total squamous cells in ThinPrep® preparations is such that it is not feasible to set a minimum acceptable total squamous cellularity so that there is an acceptable probability that all specimen vials containing dyskaryotic cells will be identified. In the light of this, a pragmatic approach should be adopted by deciding on an arbitrary minimum acceptable total squamous cellularity which ensures a rate of detection of abnormality that is at least as good as that for (smears) and which does not impose undue burdens on the users and providers of the screening program.” The 2008 American Society for Colposcopy and Cervical Pathology (ASCCP) committee review[33] fell just short of endorsing this recommendation but noted that the TBS 2001 guidelines have been in wide use for many years and stated: “The Bethesda 2001 squamous cellularity criteria provide an acceptable threshold of unsatisfactory results for most patient populations and laboratory settings, although additional studies and data would be useful.”

McQueen[34] estimated that for ThinPrep slides there must be at least 16 “targets” (scenes of abnormality) and comprised of at least 87 dyskaryotic cells in total in order to reduce the probability of a false negative result to 2%. This is for Pap stained samples screened visually by experienced cytotechnologists. What is the situation for DNA Cytometry?

Automated quantitative image cytometry is measurement-based and so the detection of even a single aneuploid cell is done with a high degree of confidence, as discussed more in the next section. If we assume that LBC mixes the sample so that the subsample of both physiological cell clusters and isolated cells deposited on the slide is statistically randomized and representative of what is in the vial, then it is possible to calculate from the Poisson distribution, the probability of a real false negative case, defined as having no aneuploid epithelial cell present among the epithelial cells measured. Figure 8 shows the probability of a real false negative occurring given that 5000 epithelial cells are measured. Note that the figure is a log-log plot. Figure 8 shows that when the ratio of normal to abnormal cells is less than about 1000:1, the probability of a real false negative is less than about 1% when 5000 epithelial cells are measured. McQueen et al[34] counted the number of dyskaryotic cells and total squamous cells on 23 HSIL slides and found only 1 (4.3%) to have a normal:abnormal ratio of > 1000:1-that case was 4600:1.

Figure 8
Figure 8 The probability of a real false negative occurring given that 5000 epithelial cells are measured.

A more general way of looking at this is shown by the black line in Figure 9 (note the log-linear scale). The X-axis is the aneuploid:normal cell ratio times the number of epithelial cells measured, so it does not just apply to the case of 5000 cells as in Figure 8. The simple “rules of thumb” are: (1) if the aneuploid:normal cell ratio times the number of cells measured is about 5, then the probability of a real false negative is just under 1%; and (2) if this product is about 7, then the probability of a real false negative is approximately 0.1%, which is well into the realm of diminishing returns. That is, to get a ≤ 1% chance of a real false negative case when the ratio of normal to abnormal cells is 1000:1 requires measuring 5000 epithelial cells; if the normal to abnormal ratio is 5000:1, then 25000 epithelial cells must be measured and so on. Setting a goal of a real false negative rate of ≤ 1% is probably very aggressive for screening.

Figure 9
Figure 9 General relationship between number of cells measured and the probability of a real false negative. The red line is considered “equivocal”.

A final way to look at this is Figure 10 which shows how many cells must be measured for a 1% chance of a real false negative case (black line) or for a 95% chance of the being 3 or more aneuploid cells measured (blue line). Most publications on AQIC applied to cervical cancer screening report measuring 6000 or 8000 epithelial cells and so theoretically have a < 1% probability of having a real false negative when the normal:aneuploid cell ratio is less than 1300:1 and 1700:1, respectively.

Figure 10
Figure 10 Given an aneuploid cell fraction, how many cells must be measured to get 1% chance of a real false negative or a 95% chance of a positive test result.
SCAN REVIEW AND REPORT GENERATION
Normalization and DNA scale

Previously, it was mentioned that Beer’s law is applied to measure DNA for each pixel of the image and that the DNA content of each cell is determined by simply adding up the DNA content of each pixel of the image of that cell nucleus. This procedure quantifies the DNA on an arbitrary scale, but it is more useful to convert this to a relative DNA scale by normalizing to the normal diploid cell population. Again, cervical “Pap” samples are predominantly comprised of normal diploid cells, even when taken from a woman with invasive cervical cancer, and so for Pap samples such an internal reference cell population always exists. The normalized scale described here is called “DNA Index” or “DI” in which diploid cells have a DI = 1, tetraploid have DI = 2, etc. Figure 11 is an example of such a histogram measured for a pig liver “touch” preparation. The green peak represents purely single diploid hepatocytes, but the other peaks include mixtures of single tetraploid cells and clusters of diploid and/or tetraploid cells used to check the linearity of the DNA measurement. Arguably, the DNA Index is the natural scale to use for cancer screening which is primarily concerned with deviations from diploid for somatic cells.

Figure 11
Figure 11 DNA index for pig hepatocytes for DNA linearity calibration.

However, the original DNA scale is the “C-index” scale in which DI = 1 is designated as “2C”, DI = 2 is “4C”etc. This scale was first used in 1950 by the American, Swift[37], in a study employing DNA measurement by microphotometry of Feulgen stained cells of various plant species (basically the same technique as AQIC). Swift found that within the non-dividing cells of a species, the DNA content of cells tended to be in quanta that he labeled “C” for “classes”: “Non-Dividing Tissues: Photometric measurements made on tissues where mitoses were uncommon tended to fall in certain well marked classes. Means of these classes fit in the series 1:2:4:8:16:32.” On this scale, 1C is the quantum of DNA in a (plant or animal) sperm cell. The DNA amount corresponding to 1C varies from species to species. An earlier similar study of various animal species[38] noted the ratio 1:2:4 for DNA in cells from rat liver, but did not describe this in terms of the C-index, which would have been 2C:4C:8C. Arguably, the C-index is the natural scale for biology, in general, as opposed to the narrow field of cancer.

The first step in the data review checklist is to determine that the DNA normalization is valid, meaning that the normal diploid cell population was correctly identified and that the normalization was correctly applied. Figure 12 is a typical, correctly normalized DI histogram of the cervix sample taken from a healthy woman. The scatter plot 12 shows almost all diploid epithelial cells to be in a tight distribution except for a very few hypo-diploid epithelial cells highlighted by the blue oval. This hypo-diploid skewing is a general feature of cervix samples and is comprised of dead epithelial cells in which the DNA has degenerated. The amount of skewing increases with the presence of infection, particularly, but not limited to, trichomonas vaginalis. The skewing is much more obvious in Figure 13 where it does not affect the normalization. The last example (Figure 14) is of a case of trichomonas vaginalis and shows a peak of cells with degenerated DNA that is bigger than the normal diploid peak that should be used to normalize and define the DNA scale. It is beyond the scope of this review to discuss automatic normalization algorithms and technician training strategies to deal with this. However, this problem is minimized if the sample taker cleans the cervix of excess discharge and secretions prior to taking the sample, as is called for by most sampling protocols[33,39]. Some gynecologists believe this to compromise the sample, but an Randomized controlled trial (RCT) that involved taking two consecutive samples has shown the results to be independent of sampling sequence[40].

Figure 12
Figure 12 Typical DNA ploidy data for sample taken from a healthy cervix.
Figure 13
Figure 13 Typical DNA ploidy data for sample taken from a cervix without removing excess secretions or discharge.
Figure 14
Figure 14 DNA ploidy data for sample taken from a cervix infected with trichomonas vaginalis without removing excess secretions or discharge.
High DI aneuploid cells and DI > 2.5 (or > 5C)

The second step in the data review checklist is to examine any images that could be of aneuploid cell nuclei, most usually defined as having DI > 2.5 or “5C exceeding”. Why is this the practical definition of aneuploidy?

Since AQIC does not identify and count individual chromosomes, it is usually not possible to discriminate normal cells in cell cycle (mitosis), for which the DNA content ranges from 1 ≤ DI ≤ 2, from aneuploid cells with similar DI. Given that no measurements are exact, it is usual to apply a 10% uncertainty[20] meaning that normal cell cycle ranges from 0.9 ≤ DI ≤ 2.2 and hence the likelihood is that cells with DI > 2.2 are aneuploid. The convention of DI > 2.5 (> 5C) was probably a “padding” to the 10% DNA measurement uncertainty, for broad, conservative generalizability. However, Guillaud et al[41] looked directly at the DNA ploidy sensitivity and specificity for cervical cancer screening by AQIC as a function of DNA content of putative aneuploid cells. When 3 or more aneuploid cells were present, the threshold converged to DI ≥ 2.2 (4.4C), but when only 2 or 1 aneuploid cells were present, the threshold rose to DI ≥ 2.3 (4.6C) and DI ≥ 2.4 (4.8C), respectively. These results may be specific to their cytometer and their particular laboratory operations standards; other labs may want to calibrate their performance if they choose to adjust the definition of high DI aneuploidy.

Another potential ambiguity is due to polyploidy which refers to euploid cells with DI = 2, 4, 8, 16, etc. (4C, 8C, 16C, 32C…). As mentioned previously, this condition does occur normally in some tissues[2] and, even when abnormal, does not necessarily indicate cancer. Concern for polyploidy induced by koilocytosis led Chatelain et al[42] to propose setting the threshold for aneuploidy in cervical cancer to DI ≥ 4.5 (≥ 9C). However, for cervical cancer screening, such restrictively defined aneuploid cells are too rare to give the test reasonable sensitivity, even in high risk HPV (hrHPV) positive cases[41,43-45] and no evidence of threshold behavior at DI = 4.5 was observed by Guillaud et al[41]. For the purposes of cervical cancer screening, the effective definition of an aneuploid cell is one with DI ≥ 2.5 (5C). Although the 9C aneuploidy threshold is not useful for cervical cancer screening, it may have utility in diagnosis.

Figure 15 shows the histogram, scatter plot and gallery of the 8 aneuploid cells found in this typical positive case.

Figure 15
Figure 15 DNA ploidy data from a typical positive case with several aneuploid nuclei with DI > 2. 5, the aneuploid nucleus gallery is shown with measured DI indicated in red.
Aneuploid stemlines

The third step in the data review checklist is to check aneuploid stemlines which appear as a peak in the DNA histogram, but not at DI = 1 or 2 (2C or 4C) where normal cycling cell peaks appear[20]. The fact that stemline cells are in a DI peak means that they have more or less the same DNA content and so they must divide coherently and with a relatively minor amount of chromosomal instability. By definition, such stemlines are aneuploid. Stemline cells are usually in mitotic cell cycle and so a smaller G2/M peak is usually present at twice the DI of the stemline G1/G0, which is usually but not always found between 1 ≤ DI ≤ 2. Figure 16 is an example of a stemline histogram with G1/G0 at DI = 1.6 and with cells consistent with G2/M at DI = 3.2, highlighted in blue in the right histogram with the broken vertical scale. Figure 17 is the quite rare case where the stemline G1/G0 is at DI = 2.3 (≥ 2 is rare) and with cells consistent with G2/M at DI = 4.6 also highlighted in blue in the right histogram with the broken vertical scale.

Figure 16
Figure 16 DNA ploidy histograms of aneuploid stemline at DI = 1. 6. A: Full histogram; B: Histogram with broken vertical scale.
Figure 17
Figure 17 DNA ploidy histograms of a rare aneuploid stemline at DI = 2. 3. A: Full histogram; B: Histogram with broken vertical scale.

Although looking for aneuploid stemlines is a step in the review checklist, it is rare for a sample to have a stemline without there also being aneuploid cells with DI ≥ 2.5, so identifying stemlines is usually redundant.

The relative lack of chromosomal instability on an aneuploid stemline may mean that the cell morphology looks rather normal and a biopsy may be interpreted as displaying hyperplasia rather than neoplasia, causing the stemline to be seen as an apparent false positive. However, hyperplasia is genetically normal but stemlines identified by DNA Cytometry are genetically abnormal and so are neoplastic. Presumably, the presence of identifiable stemlines occurs early in the process of neoplastic transformation. This author is not aware of any publications on the malignant potential or aggressiveness of lesions of the uterine cervix that are primarily characterized by aneuploid stemlines.

Cell proliferation

The fourth step in the data review checklist is to determine if there is unusual cell proliferation. Epithelial cells in a properly taken sample from a healthy cervix should almost all be completely differentiated and so do not divide. As mentioned previously, in the range 0.9 ≤ DI ≤ 2.2 it is not possible by simple DNA Cytometry to discriminate aneuploid cells from normal cells in mitotic cell cycle. Li et al[46] added anti-Ki67 immunostain (marker of cells in mitosis) to Feulgen stained cervical samples, but found no improvement in sensitivity and specificity compared to DNA Cytometry alone.

Excessive cell proliferation could indicate, among other things: (1) repair of wounds or repair processes due to fungal, bacterial and other infections; (2) response to hormones, drugs or radiation, especially in cancer treatment; (3) response to viral infections like HPV; (4) hyperplasia as an epigenetic response to some chemical, mechanical or other stress; and (5) neoplasia.

One expects that applying this criterion alone for cancer screening will have a relatively high false positive rate with so many non-neoplastic causes for cell proliferation. This criterion is usually applied by comparing the number of epithelial cells with 1.25 ≤ DI ≤ 2.5 to the total epithelial cell count. Figure 18 is a typical example showing over 15% proliferation and, as is usually the case, there are 4 aneuploid cells also present, making the proliferation rate determination redundant. Figure 19 is a rare case without any measured aneuploid cells and has a proliferation fraction of 8.7%.

Figure 18
Figure 18 DNA ploidy data of a high proliferation case which also shows high DI aneuploid cells.
Figure 19
Figure 19 DNA ploidy data of a case with “equivocal” proliferation and no high DI aneuploid cells.

The original proliferation rule was derived from unpublished data of the author at the British Columbia Cancer Agency (BCCA) about 20 years ago (Figure 20), which set a threshold of 10% proliferation for calling a case positive, which is approximately the boundary between LSIL and HSIL. Most of the papers published in China in the past 10 years report using this value. A second “equivocal” threshold is set at 5% proliferation which is approximately the value for ASCUS and is used as an “equivocal” diagnosis, discussed later.

Figure 20
Figure 20 Original data from British Columbia Cancer Agency used to formulate the “proliferation” rules, based on measurement of > 500 samples.
No evidence of abnormality-sample adequacy

The fifth and last step in the data review checklist only applies when none of the preceding criteria for positivity are present. When no abnormality is found, the assessment of sample adequacy must be applied-were enough cells examined for a high probability that the case is actually negative? There is always some reluctance to declare a sample to be “unsatisfactory” with a recommendation of a return visit of the woman for another sample, which must balance the increased but unknown risk to the woman with an unsatisfactory test[33] against “… undue burdens on the users and providers of the screening program”[34], including costs. Although the background of this has already been discussed at length, none of the publications reviewed here have included a definition of what their laboratory uses for an adequacy standard. Guillaud et al[41] determined 2000 or more epithelial cells on ThinPrep slides to optimize the sensitivity and specificity in a research setting.

FRAMEWORK FOR EVALUATING SCREENING TESTS

Arbyn et al[47] and Adriaensen et al[48] reviewed how to evaluate cervical cancer screening technologies through their various phases, in 2009 and updated in 2013, with extensive references.

However, most of the papers available for examination for this review are cross-sectional, observational studies that compare automated quantitative image cytometry to liquid based conventional cytology and the analysis centers on 2 × 2 contingency tables and the extraction of comparative sensitivity, specificity, positive and negative predictive values, usually also reported in the paper abstract. The literature has similar analyses for screening for other cancers with other technologies[49]. In the opinion of this reviewer, these analyses are confusing at best and highly misleading at worst and most are singular in their absence of insight into genuine clinical utility of the tests. This section attempts to provide some suggestions to improve the data analysis with particular focus on the clinical meaning.

A typical study reviewed here is observational, applied to 10000 subjects screened with both Tests 1 and 2; the 600 who are positive by either test receive a recommendation to attend colposcopy; and the 250 available biopsy results become the exclusive focus of the analysis-the other 9750 cases of the study are ignored! In this analysis example there are “excluded” data (the 10000 - 600 = 9400 cases that test negative by both Tests 1 and 2) and “missing” data (the 600 - 250 = 350 cases who tested positive by at least one of Test 1 and 2 but with no follow-up result). Typically, there are 5 biopsy outcomes: negative (or cervicitis or inflammatory), CIN1, CIN2, CIN3 and invasive cancer. Tests 1 and 2 also typically have several diagnostic grades, such as NILM, ASCUS, LSIL, etc. The study applies some thresholds to collapse Test 1 vs histology and Test 2 vs histology to 2 × 2 contingency tables and the well-known formulae are applied to calculate sensitivity, specificity and predictive values. Typically, the reported specificities are very low as are the negative predictive values (NPVs), which is entirely an artifact of the failure to include the other 9400 negative cases in the analysis. Some statistics software even extracts the area under the receiver operator characteristics (ROC) curve from this single measured point, which is also often reported in the paper abstract. Finally, many papers also report the result of combining Tests 1 and 2 based on these same 2 × 2 tables, almost always as the logical “or” of positive results (the combined test is positive if either 1 or 2 is positive).

What is the interpretation of the results based solely on the 2 × 2 tables for the small subset of cases that have biopsy results available? For the moment, let us ignore the problems of verification bias, missing data and imperfect “gold standard” reference diagnosis-they are discussed later. Because the table is comprised only of cases testing positive by one or both screen tests, both the sensitivity and positive predictive value (PPV) are legitimate estimates of the screen test values and the PPV can be interpreted as the probability that a woman sent for biopsy will have a positive result. The specificity and negative predictive values are meaningless, however, because the “excluded” data is the vast majority of negative cases. In fact, the negative cases that are included in this 2 × 2 table are very strongly biased because the cases negative by Test 1 were positive by Test 2 and vice versa-otherwise colposcopy would not have been recommended (a few cases may be negative by both Test 1 and 2, but a biopsy resulted from other clinical evidence, which is not usually consistent with the notion of “screening”[50]). In this fictional example, the vast majority of the 9400 subjects who tested negative by both tests are indeed negative and with less doubt than the highly biased negative cases included in the 2 × 2 table of biopsy results. The only value this “specificity” and “NPV” have is to show them to be higher in one test than the other, but the real magnitude of this difference cannot be estimated from these results. This reviewer believes that authors who perform such analysis should not report these as “specificity” and “NPV” but should find new terms, such as “reduced” or “biopsy biased” specificity and NPV, or better yet, not report them at all as they have no clinical meaning. It is ironic that for this kind of observational study the specificity and NPV are so grossly miscalculated and reported when, in fact, they can usually be quite reliably determined, as discussed later.

Given a 2 × 2 contingency table, the mathematical definitions of sensitivity, specificity, positive and negative predictive values are crystal clear, but the same is not true for their meaning. It is very common to have different definitions of “sensitivity”, “specificity” or “predictive” values”, even though the formulae by which they are extracted for the 2 × 2 tables are invariant; that is, the meaning depends on the table content. For example, in the world of HPV testing, there is the “analytic sensitivity” which refers to the ability of the test to detect the presence of HPV as distinct from “clinical sensitivity” which refers to the ability of the test to predict the presence of CIN[51-53]; in general, too high analytic HPV sensitivity causes very low clinical specificity. As another example, in the screening mammography program of British Columbia[54], the positive predictive value of a single screening mammogram ranges from about 2% to 20%, depending on the age of the woman, averaging to about 6.5% across all ages. This is the “test PPV”. However, the screening program does not send women with a positive screen mammogram to biopsy but instead performs other tests, primarily diagnostic mammograms and/or ultrasound. Of the women actually sent for biopsy, about 33% are found to have cancer or DCIS-this is the “program PPV” and its clinical interpretation is “the probability that a woman sent for biopsy actually has breast cancer.”

Excluded data

While a screening test ideally should have both high sensitivity and specificity, high specificity is more critical. A screen test with a sensitivity of 50% and specificity of 95% would be inefficient but could be useful; the conventional Pap test has approximately this performance and it has reduced cervical cancer mortality by 70% in countries where it has been effectively applied. A screen test with a sensitivity of 95% and specificity of 50% would not be useful by itself because half of the screened population would have false positive test results. Depending on the follow-up consequences of a positive screen test result, most screen tests must have a specificity of at least 90% and probably many require specificity > 98%. In many of the papers reviewed here, the specificities are reported as “test” specificities in the abstract and are very low, often 40%-80% and even as low as 2%-3%[55], entirely an artifact of failing to include most of the negative test data. On seeing these low values, those who understand screening might dismiss both Tests 1 and 2 as being completely useless.

These considerations show that the excluded cases that tested negative by both Test 1 and 2 should be included in the contingency tables (the effect of verification bias is discussed later) in order to obtain reasonable estimates of the specificity and NPV.

Screening does not have binary results

A key problem with 2 × 2 tables is the clinical reality that there are not two but three screening test results[56]: positive, negative and equivocal, as shown in Figure 1, and defined as “too atypical to call ‘negative’ but not enough to call ‘positive’.’’ It is common to use time and re-testing to resolve equivocal cases, which is especially important for cervical cancer screening given that most “pre-cancers” resolve without medical intervention, especially in younger women[57-64]. A recent study estimates that < 2% of CIN2/3 lesions progress to cancer within 10 years[65]. At BCCA, ASCUS and LSIL are considered to be equivocal and the management recommendation is for repeat Pap testing at 6 mo intervals up to 4 times; if there is no resolution of the case by then, colposcopy is recommended[50]. The most recent ASCCP management guidelines[66] treat cytology ASCUS and histology CIN1 in all women and LSIL and CIN2 in young women as equivocal, with follow-up intermediate between how negative and positive cases are managed. While there is concern for loss of the patient to follow-up and undue burden on the patient, this must be balanced against the burdens and possible negative consequences of overtreatment, especially in younger women[67-75]. Various strategies are possible to mitigate the burden of repeat testing.

For DNA Cytometry, most authors follow the recommendation that cases with the detection of 1 or 2 high DI aneuploid cells should be considered “equivocal” while ≥ 3 are considered positive (Figures 9, 10). Some laboratories also call cases equivocal that have between 5 and 10% proliferation fraction, as shown in Figure 20.

Unfortunately, there are no standard formulae for managing 3 × 3 tables, so some consideration should be given to the clinical meaning of what is calculated and reported. In collapsing the screen Test 1 or 2 and histology “gold standard” 3 × 3 tables to 2 × 2 contingency tables, one possible clinically relevant approach is to set the thresholds between “negative” and “equivocal” for sensitivity, specificity and NPV as shown in Figure 21.

Figure 21
Figure 21 Mapping of 3 × 3 to 2 × 2 tables for sensitivity, specificity and negative predictive value.

Here the clinical interpretation of the screen test performance is: Sensitivity-the probability that positive gold standard cases had screen test results of positive or at least of equivocal. The notion is that equivocal screen cases cannot be called negative since they do carry a recommendation for follow-up; equivocal screen tests may represent delayed diagnosis cases but not missed diagnosis cases; Specificity-the probability that screen Test 1 negative cases are also screen Test 2 negative and biopsy negative (when biopsy is available) cases; and Negative Predictive Value-its complement, (1-NPV), is the probability that a woman who is told she is disease free is actually not.

For PPV, the collapse of the 3 × 3 to 2 × 2 table follows a different scheme (Figure 22) because its usual clinical interpretation is: Positive Predictive Value-the probability that a patient sent for biopsy will have a positive biopsy result. Women with equivocal screen test results will not generally be recommended for colposcopy and possible biopsy.

Figure 22
Figure 22 Mapping of 3 × 3 to 2 × 2 tables for positive predictive value.
A worked example

To illustrate the magnitude of the differences between these different approaches to analysis we work through an example based on data by Bao[76] from 2009, selected because it is one of the larger studies published and because it has reasonably complete reporting of the data. Table 1 shows the overall results for cytology and DNA Cytometry for almost 20000 cases.

Table 1 Cytology and DNA Cytometry results from Ref[76].
Cytology1DNA Cytometry
NILM18503Negative17855
ASCUS720 (0.40)Equivocal1395
LSIL296 (0.89)Positive371
HSIL59 (1.00)
Total19621Total19621

The screen test vs biopsy results are Table 2 for Cytology and Table 3 for DNA Cytometry.

Table 2 Biopsy results vs Cytology from Ref[76].
Cytology
HistologyTotalNILM/inflamASCUSLSILHSIL
Ca15654
CIN3435101414
CIN26612152316
CIN19842222113
CC/Neg401285861812
Total6233441398159
Table 3 Biopsy results vs DNA Cytometry from Ref[76].
DNA Cytometry
HistologyTotalNegEquivPos
Ca15213
CIN34311230
CIN26631944
CIN198155528
CC/Neg4018728034
Total623106368149

The 2 × 2 tables used in the publication are shown in Table 4 using CIN2+, LSIL+ and DNA Positive as the thresholds for histology, cytology and DNA Cytometry, respectively.

Table 4 2 × 2 Contingency Tables for Biopsy results vs Cytology and vs DNA Cytometry from Ref[76].
Cytology
DNA Cytometry
HistologyTotalLSIL+NILM/ASCUSPosNeg/Equiv
CIN2+12476488737
Neg/CIN14996443562437
Total623140483149474

The test performance results are given in Table 5. The authors did not report the NPV and PPV in their publication. Based on these numbers, the performance of the two tests is quite comparable, although cytology missed 6 of 15 invasive cancers using the LSIL+ threshold while DNA Cytometry missed 2.

Table 5 Cytology and Cytometry test performance based on data and analysis from Ref[76].
CytologyDNA Cytometry
Sensitivity61.3%70.2%
Specificity87.2%87.6%
PPV54.3%58.4%
NPV90.1%92.2%

Next are 3 × 3 tables that include the previously excluded negative cases, but also with the missing data still deleted (as with the author’s analysis above); biopsy results are compared with Cytology in Table 6 and with DNA Cytometry in Table 7. The definitions of Positive, Negative and Equivocal are arbitrary and arguable but used for illustration.

Table 6 3 × 3 Contingency Table for Biopsy results vs Cytology from Ref[76].
Cytology
HistologyTotalPositive (LSIL+)Equivocal (ASCUS)Negative
Positive (CIN2+)124763117
Equivocal (CIN1)98342242
Negative18560308618444
Total1878214013918503
Table 7 3 × 3 Contingency Table for Biopsy results vs DNA Cytometry from Ref[76].
DNA Cytometry
HistologyTotalPositiveEquivocalNegative
Positive (CIN2+)12487334
Equivocal (CIN1)98285515
Negative181503428017836
Total1837214936817855

Following the approach outlined above to collapse the 3 × 3 tables to 2 × 2 tables according to the clinical interpretation described above yields new test performance results shown in Table 8.

Table 8 Cytology and Cytometry test performance based on data from Ref[76] and revised analysis.
CytologyDNA Cytometry
Sensitivity73.4%91.4%
Specificity99.4%98.3%
PPV54.3%58.4%
NPV99.68%99.89%

The increase in sensitivities is due to considering equivocal cases as positive since they include a recommendation for follow-up. The PPVs are unchanged. The specificities and NPVs are much higher when the negative cases are included and are consistent with the requirements for screening test performance.

Many authors interpret the high NPVs as being almost equal and so to be uninteresting. In fact, the clinical significance of the NPV is seen from its complement (1-NPV), which relates to “false negative” cases. In this example of 10000 women who are advised that they are disease free, in fact, 32 or 11 are not, based on cytology or DNA Cytometry, respectively. In this regard, the performance is quite different between the two tests. This is especially important when routine screening is infrequent, say once or twice during a woman’s lifetime.

An alternative approach to compare the tests is to simply tabulate the results descriptively as in Table 9.

Table 9 Descriptive summary of the data of Ref [76] as re-analyzed n (%).
Clinical resultCytology number of casesDNA Cytometry number of casesCytometry-Cytology number (%)
1Number of women immediately returned to routine screening1850317855-648 (-3.5)
2Number of CIN1+ cases (false negative cases) per 10000 women returned to routine screening3211-21 (-66)
3Number of women referred to immediate colposcopy1401499 (6)
4Number of cases of invasive cancer immediately diagnosed9134 (44)
5Number of CIN3+ cases immediately diagnosed37436 (16)
6Number of clinically positive cases (CIN2+) immediately diagnosed by colposcopy768711 (15)
7Number of women requiring 6 mo follow-up due to equivocal result139368229 (165)
8Number of women with potentially delayed clinically positive (CIN2+) diagnosis31332 (6)
9Number of CIN3+ cases missed514 (80)
10Number of CIN2+ cases missed17413 (76)

This study was primarily of rural women who had not previously been screened.

The impact of uncorrected verification bias

Most observational screening studies suffer from verification bias because the “gold standard” of colposcopy guided biopsy is only applied to the cases that test positive by the screen test, so the true disease state of those who test negative is not verified. There are statistical methods to correct for this kind of bias (e.g.[77]). RCTs sometimes manage this by performing colposcopy and biopsy on a random sample of the negative cases, under proper informed consent and ethics board approval (e.g.[78]), although this also has its pitfalls[47].

It may be instructive to look at the impact when this kind of verification bias is left uncorrected. Figure 23 plots the “DNA Cytometry” data of the previous example (Table 8) as a function of the fraction of positive and equivocal cases detected in the study; the left axis is where all positive and equivocal cases were detected and it declines to where only 50% were detected at the right axis. PPV is completely unaffected because it relates only to ratio of positives that the screen correctly identified to the total that it tested positive; hence, any false negatives do not affect this ratio. NPV and specificity decline only slightly over the range plotted. It is the sensitivity, of course, that is potentially most impacted by this kind of verification bias, with an inverse proportional linear dependence; that is, if only half of the positive cases were detected in the study, then the measured sensitivity is double the correct value.

Figure 23
Figure 23 Effect of uncorrected verification bias on test performance indicators.

In summary, when this kind of verification bias is left uncorrected it has no impact or almost no impact on predictive values or specificity but it can significantly inflate the measured sensitivities. However, in studies such as those described above, it is likely that well over 90% of the true positive cases were detected by one or both of the screening tests[47,79], so the impact of uncorrected verification bias in this type of study is unlikely to be very large.

Unbiased comparisons of test performance

It has been shown that the ratio of (Test 2) : (Test 1) for true positive rate and for false positive rate determined just from the complete biopsy results (with no missing data) does not suffer verification bias of the type discussed above (not subjecting negative cases to the “gold standard” test)[80,81]. This may be enough to rank Test 1 vs Test 2.

Table 10[80] summarizes the biopsy data required and the relative true positive rate, relTPR, and relative false positive rate, relFPR, are simply[81]:

Table 10 Definitions of the biopsy data required for comparison of two screening tests independent of simple verification bias, Ref[80].
Gold standard positive
Gold standard negative
Test 1+Test 1-TotalTest 1+Test 1-Total
Test 2+aba + bABA + B
Test 2-c[d][c + d]C[D][C + D]
Totala + c[b + d][n]A + C[B + D][N]

relFPR (Test 2/Test 1) = {(a + b)/[n]}/{(a + c)/[n]} = (a + b)/(a + c)

relFPR (Test 2/Test 1) {(A + B)/[N]}/{(A + C)/[N]} = (A + B)/(A + C)

The key point is that the unknown [n] and [N] are eliminated, leaving only known values from the biopsy result table. If relTPR increases and relFPR decreases, then Test 2 is easily judged to be the better test, although as Arbyn et al[47] points out, such cross-sectional results do not necessarily translate into longitudinal improvement in the screening program. It is more generally the situation that both relTPR and relFPR increase at the same time, in which case it is necessary to consider the disease prevalence in the target screen population to determine if the cost of increased false positives justifies the benefit of the increased true positive detection. In the special case where the target population and the study population have the same prevalence (this is generally true for observational studies of routine clinical data), then the FP/TP ratio is given by: FP/TP = (B-C)/(b-c). This ratio is one “figure of merit” in deciding if Test 2 is better than Test 1 and comes down to the judgment of “how many extra false positive cases are acceptable for each additional true positive case detected?” The value of this ratio will depend upon the undiagnosed disease prevalence.

Exactly the same results are obtained by inspection of the appropriate 2 × 2 tables for each test vs the biopsy results; both calculation methods are illustrated next.

Tian et al[82], provided the case by case data for that paper to allow illustration of the method here. The first look at this is based on “positive” defined as ASCUS+, Equivocal+ and CIN1+ for cytology, DNA Cytometry (AQIC) and pathology, respectively, summarized in Table 11. The relTPR = 1.3 and relFPR = 1.8, both increasing as expected. However, the FP/TP ratio is just 0.64, meaning that for every extra true positive detected by AQIC, 2/3 additional false positive cases occur, which is probably an acceptable trade-off.

Table 11 Biopsy data from Ref[82] for simple verification bias free comparison of Cytology vs DNA cytometry.
Biopsy +
Biopsy -
Cytology +Cytology -TotalCytology +Cytology -Total
AQIC+13037167302353
AQIC-101000
Total13137168302353

The same result is obtained from the published[82] 2 × 2 tables, using the same thresholds for positive, as shown in Table 12. By inspection relTPR = 167/131 = 1.3 and relFPR = 53/30 = 1.8 and FP/TP = (53-30)/(167-131) = 23/36 = 0.64.

Table 12 Alternative presentation of Biopsy data from Ref[82] for simple verification bias free comparison of Cytology vs DNA Cytometry.
Cytology
DNA Cytometry
HistologyTotalASCUS+NILMHistologyTotalPos/equivNeg
CIN1+16813137CIN1+1271671
Neg533023Neg121530
Total22116160Total2212201

These ratios may provide a useful, unbiased “figure of merit” with which to compare one test to another. The key message is that these ratios-the relative true positive rate, relTPR, relative false positive rate, relFPR and FP/TP-are not subject to verification bias of the type due to failing to perform the “gold standard” reference test on the large number for cases that test negative by both Test 1 and Test 2. However, they may be subject to bias due to missing data, discussed next.

Missing data

But what about the missing data-the cases recommended for colposcopy for whom there is no follow-up information? This is a particular kind of verification bias and there are formal methods to correct for it (for example[77,83-87]) that involve using statistical methods to impute the missing data with minimal bias, but they are beyond the scope of this paper. The simplest method is to delete these cases from the analysis, as was done in the example above and in all of the DNA Cytometry papers reviewed here. This is approximately equivalent to assuming that the missing data is the same as the known data. Because the missing data is all from cases that tested positive by either or both Tests 1 and 2, this has no effect on the NPV, for reasons analogous to why the PPV was unaffected in Figure 23. There is no simple, general statement one can make about the impact of the missing data on the sensitivity, specificity or PPV except that the range of possible deviation between the uncorrected measurements and the true values is proportional to the amount of missing data. Each of the three performance measures could rise, fall or stay the same if the missing data were not missing. The key message is that it is extremely important to organize studies so as to minimize the amount of missing data. In the Bao paper, they did not report the total number of colposcopy follow up recommendations but, depending on the correlations between cytology and cytometry, at least 65% of the biopsy follow up data is missing, which is very typical for this series of papers from China.

Effects of an imperfect “gold standard” reference

The “gold standard” reference diagnosis for cervical cancer studies is usually colposcopy directed biopsy, which a number of studies over the past decade have shown to be imperfect[88-92]. More worrying were studies that showed that the measured sensitivity of Visual Inspection with Acetic Acid (VIA) were substantially inflated (by 1/6 to 1/3) due to use of colposcopy directed biopsy as the “gold standard’ reference[93-95]. Theoretical studies[96] of such use of imperfect references have shown that if the screen and reference test errors are statistically correlated, then the test accuracy (sensitivity and specificity) measurements are over-estimated; conversely, if the screen and reference test errors are statistically independent, then the test accuracy measurements are underestimated. VIA is essentially a variation of colposcopy and so is highly correlated.

It has also become clear over the past decade that colposcopy sensitivity improves the more biopsies that are taken[88,93,95,97,98], independent of the skill[99] or medical training[90] of the person performing the colposcopy, ranging from nurse-practitioner to gynecological oncologists. Many propose supplementing any lesion biopsies with a random biopsy from any quadrant without visible lesions[97,99]. However, while this increases the rate of detection of CIN2+, it does not appear to affect the measured sensitivity of cytology, although it may improve the specificity slightly[94]. It is unknown if taking more biopsies has any effect on the DNA Cytometry accuracy measurements but one might expect any effect to be small based on the cytology situation.

Combining binary test results

A great many papers, including several reviewed here, look at combining the results of two binary (or dichotomous) tests with the hope that the combined test result is an improvement over either of the component Tests 1 and 2. This is also applied to tests like Hybrid Capture 2 (HC2) HPV test that measures continuously valued viral load but which is made binary by applying a threshold, ≥ 1 RLU/Cutoff is positive, otherwise it is negative[100].

It can be proven that there are only two non-trivial ways to combine 2 binary tests: as the logical “and” of positive cases, in which the result is positive only if both Tests 1 and 2 are positive, and as the logical “or” of positive cases, in which the result is positive if either or both Tests 1 and 2 are positive. It is also possible to prove that the combined test results will conform to the limits in Table 13, where “Min” and “Max” are the minimum or maximum of the two entries in parenthesis and where Sp = specificity and Se = sensitivity.

Table 13 The test performance limits for the combination of two binary valued tests.
Sensitivity
Specificity
MinimumMaximumMinimumMaximum
Logical “AND”(Se1 + Se2) - 11Min (Se1, Se2)Max (Sp1, Sp2)(Sp1 + Sp2)3
Logical “OR”Max (Se1, Se2)(Se1 + Se2)4(Sp1 + Sp2) - 12Min (Sp1, Sp2)

For the logical “and” of positive results, the specificity of the combined test will be at least as high as the specificity of the most specific component test and could be 100%; however, the sensitivity will be no better than the sensitivity of the least sensitive component test and could be zero.

Conversely, for the logical “or” of positive results, the sensitivity of the combined test will be at least as high as the sensitivity of the most sensitive component test and could be 100%; however, the specificity will be no better than the specificity of the least specific component test and could be zero.

Examples of “and” and “or” combinations of binary tests and their compliance to these limits can be found in refs[101-103].

Furthermore, it can also be proven that the result of combining these tests does not depend on the test order (they are commutative).

In either combination, it is not necessary to perform both tests on all subjects[104]. For the logical “and” of positive test results, if the first test result is negative then the combined test result will also be negative, regardless of the second test result; similarly, for the logical “or” of positive test results, if the first test result is positive, it does not matter what the second test result is, the combined test result will be positive. This, plus the fact that the test order does not affect the results, means that a test strategy can be adapted to minimize costs in clinical practice (both must be done for comparative performance studies, of course).

There is one caveat with these rules: they apply for analytical tests that are independent in the sense that one test does not impact the other. For example, if one test compromised the sample for the other test, these rules might not apply. It is not so clear that cytology is a test that is independent of DNA Cytometry or HPV status, if the cytologist is aware of the DNA Cytometry or HPV results. Since most of the papers reviewed here are observational studies looking at both tests in routine use, it would be expected that the cytologist would, in general, be aware of the results of the other tests since the goal is to get the correct answer for the patient, not to do a strict clinical study.

It does not seem to be appreciated by many authors that the limits in Table 13 seriously constrain any potential benefits from combining binary tests. Since one of sensitivity and specificity will rise as the other falls when the tests are combined, it usually only makes sense to combine tests that have very similar and high values for the one that will fall, so that it does not fall much and stays in range of what is useful. For example, cytology has poor sensitivity, but excellent specificity; hrHPV testing has excellent sensitivity, but only good specificity. Combining these tests will result in either excellent sensitivity and less than good specificity, or excellent specificity and poor sensitivity. There is no simple combination of HPV testing and cytology that works well. The work around for this until a better test or test combination is found is in the patient management algorithms, which are becoming increasingly complex[66]. These algorithms manage “equivocal” cases with a combination of time and re-testing, in recognition that screen tests, even in combination, have 3 outcomes, not 2, as already discussed.

Impact of disease prevalence on estimates of test performance

A key issue in appreciating published study results is to determine their degree of generalizability. A basic concept of epidemiology is that while predictive values explicitly depend strongly on disease prevalence, sensitivity and specificity are completely independent of it[105]. Although this prevalence independence of sensitivity and specificity is certainly true, it is a fragile truth that may not survive test generalization. That is, if a published study demonstrates that Test A has sensitivity X and specificity Y with a particular disease prevalence Z, it does not mean that when applied to a different population with a disease prevalence of 1.5 x Z, Test A will have the same sensitivity and specificity found in the study, except if the mean, width and shape of the underlying distributions of positive and negative populations is also the same as in the study. Figure 24 illustrates ROC curves, which should be independent of disease prevalence since they essentially plot sensitivity vs specificity, for the case where the mean, width and shape of both positive and negative populations remains the same and only the positive distribution is multiplied by a scalar to reduce the distribution area without distortion or shifting of the peak position. As expected, the change in prevalence has no effect on the ROC curve or sensitivity and specificity, indicated by the blue line.

Figure 24
Figure 24 Sensitivity and specificity are independent of disease prevalence when the change in prevalence is only a scalar multiplier that does not change position, width or shape of the population distribution.

However, as a practical matter, “real life” changes in disease prevalence rarely occur this way. If the prevalence increases, and especially if it is a large increase, it also probably causes changes in the mean, width and overall shape of both positive and negative distributions as the whole population is somewhat sicker, as illustrated in Figure 25. In scanning the figure from left to right, the prevalence increases, but the positions (μ) and widths (σ) of both distributions also increases. In this example, both distribution means (μ) shifted to the right the same amount-this would not cause any change in the ROC curve itself-the change in the ROC curve is caused by the change in widths (σ) of the distributions. The magenta line represents the Test threshold and the red point on the ROC curve corresponds to test performance at that threshold. Unless the test threshold is adjusted by re-calibrating the test to the new populations, then the test performance moves from optimal, as indicated by the shift of this operating point. This figure is based on symmetric analytic distributions for ease of calculation and drawing-there is no requirement that the underlying distributions for ROC curves are symmetric, analytic or even single peaked; hence, “real life” shifts in disease prevalence may cause different shifts in test performance than the simple ones illustrated here.

Figure 25
Figure 25 “Real life” changes in disease prevalence are usually accompanied by changes in both positive and negative population distribution positions, widths and shapes which do change the sensitivity and specificity.

This result is technology independent and will apply not just to AQIC but also to hrHPV testing, for example. One situation where there is a large difference in undiagnosed disease prevalence is when a test is applied for screening (testing people without symptoms) vs diagnosis (testing people with positive screen test results or with symptoms). Several recent publications have experimentally verified this for colposcopy[106], hrHPV testing[107], the conventional Pap test[108], as well as a recent review of several meta-analyses of various diseases[109]. The previous discussion of BCCA screening mammography vs diagnostic mammography is another example. Many physicians erroneously think that test sensitivity and specificity are invariant properties of the test and therefore independent of whom the test is applied to; this is not necessarily the case.

Although it is well understood that predictive values explicitly depend upon undiagnosed disease prevalence, it is less widely appreciated that this imposes significant limits on the positive predictive value. Why is the PPV not 100%? For a low prevalence disease, such as cancer in a screening setting, PPV is determined mostly by the false positives (or 1- specificity); PPV is only weakly dependent on sensitivity. Figure 26 shows the maximum possible PPV (under the condition that sensitivity is 100%) as a function of undiagnosed disease prevalence for various very high values of specificity. Even at a disease prevalence of 1% (1000/100000 persons) and with a test specificity of 99%, the PPV will only be 50%; if the test specificity falls to 98%, the PPV drops to about 33%. This is another demonstration of the crucial importance of high specificity for screening tests. It also illustrates that cancer screening is more than a test-it is a process by which negative cases are progressively removed from the population under surveillance until the disease prevalence of the remainder is increased to levels that allow acceptably high PPVs.

Figure 26
Figure 26 Positive predictive value at 100% test sensitivity as a function of undiagnosed disease prevalence for various high values of test specificity.

In summary, undiagnosed disease prevalence has a big impact on screening test performance, firstly by affecting the real world generalizability of sensitivity and specificity and secondly by limiting the maximum possible test PPV.

The large Table 14 summarizes the observed crude disease prevalence in several automated quantitative image cytometry studies from China from 2005-2013. The studies were selected as having the word “screening” in the title, abstract or introduction; studies with only the word “diagnosis” were excluded. As previously discussed, there is often substantial “missing” biopsy result data in studies from China. The column “follow-up ratio” is an estimate made by this reviewer of the extent to which high grade cytology had corresponding biopsy data-the blank entries indicate that it was not possible to estimate this. With few exceptions, the listed disease prevalences are underestimated, in many cases by a factor of 2 or more. The heterogeneity of the results is quite alarming and certainly justifies the warnings just made concerning generalizing the study test results for sensitivity and specificity. The very high disease prevalence in some studies that are claimed to be “screening” is quite vexing.

Table 14 Crude cancer prevalence and other data from 36 published DNA Cytometry studies from China, 2005-2013.
Ref.Invasive cancer casesCIN2+ casesTotal screenedCrude cancer prevalence (per 100000 persons)Crude CIN2+ prevalence (%)Follow-up ratioCa/CIN2+ (× 10)
[110]3145006002.82.14
[82]71002369829.50.420.380.7
[111]48103402011942.560.784.66
[112]122120083.31.830.45
[113]217673297.22.531.18
[114]4123551112.60.343.33
[115]215316263.30.470.391.33
[116]14784109340.71.90.931.79
[117]484926143.20.910.360.48
[118]2302599771.150.67
[119]36512002505.420.46
[120]91116793132.51.6310.81
[121]016200300.80
[122]4872153185.84.040.46
[123]3561207924.80.460.54
[124]5263000166.70.871.92
[125]81425886135.92.4110.56
[126]111477735142.21.910.75
[127]141459821.70.890.24
[128]4341227832.60.281.18
[129]211358955.70.311.82
[130]3171806166.10.941.76
[131]010120600.830
[132]85136032221.4211.57
[133]10616823993441.80.76.31
[134]6361220491.82.951.67
[76]151241962176.40.631.21
[55]221307065.10.680.95
[135]1152000500.750.67
[136]05125600.40
[137]3017218097165.80.951.74
[138]17451221.71.551.43
[139]10534302325.612.331.89
[140]6122832211.90.425
[141]159586701731.11.58
[142]3018722169135.30.841.6
PUBLICATIONS ON THE APPLICATION OF DNA PLOIDY TO CERVICAL CANCER SCREENING

Since 2005, the use of AQIC testing for cervical and other cancers has continuously expanded to about 1 million tests per year in 2013. At least 60 publications on various comparisons for AQIC performance to other means of screening or diagnosing cervical cancer have appeared since 2005, mostly in Chinese language journals. This section reviews some of these papers.

Disease prevalence

Perhaps a good starting point is Table 14 on measured crude prevalence rates of invasive cervical cancer and CIN. As shown previously, disease prevalence is a very important epidemiological quantity that impacts the performance and deployment of tests, especially screening tests. However, prevalence is very difficult to measure and there are two very different flavors of prevalence that can cause confusion. Some cancer monitoring agencies report cancer prevalence meaning as “How many people diagnosed with cancer are alive today?” These are estimated either as “limited-duration” prevalence (limited to cases diagnosed over, say, the past 10 years)[143] or “complete” prevalence which is independent of when the cancer was diagnosed, as is done by SEER[144]. Neither of these is relevant to cancer screening which is focused on and influenced by “undiagnosed” cancer prevalence. From here on, the term “prevalence” will be understood to mean “undiagnosed prevalence,” unless explicitly modified.

If the cancer is undiagnosed, how do we know how much there is? In a well screened population, the disease prevalence is less important than it is in poorly screened populations. An effective screening program will detect the “prevalent” cases of cancer within several years of its introduction (depending on the screening program effectiveness and deployment details) and will remove them from the pool of undiagnosed cancers, leaving mostly what are known as new “incident” cases, approximately equal to the incident rate for that cancer among that population times the screening interval: P (cases/100000 people) is approximately equal to I (cases/100000 people/year) × Screen Interval (years). In an unscreened population, the introduction of screening is usually the time when the disease prevalence can best be estimated by measurement; otherwise, it can only be estimated from cancer natural history and population models. In the absence of screening, cancer is diagnosed either by investigation of symptoms or incidentally during some other medical investigation. There is usually a delay between the onset of symptoms and diagnosis because the patient, physician or both ignore the symptoms for some time, as they are rarely specific. A crude “back of the envelope” estimate of undiagnosed invasive cancer in an unscreened population would be the incidence rate times the delay between invasion and onset of threshold symptoms-that is, symptoms strong enough to lead to diagnostic investigation: P (cases/100000 people) is approximately equal to I (cases/100000 people/year) × Symptom Threshold Interval (years). What is the incidence rate of cervical cancer in China? The official rate from the 2004-2005 “Third National Survey” is 3.2/100000/year and the mortality is 2.4/100000/year, age standardized to the world population[145]. If it takes on average 10 years between the onset on invasive cervical cancer and the onset of symptoms significant enough to seek medical attention, then this would predict invasive cervical cancer prevalence of 30-60/100000, allowing for differences in age standardized rates. This is a factor 10-40 less than what is reported in several entries in Table 14.

Although the Chinese registries consistently report such cervical cancer incidence rates[146-150] and the international reports (based on the same registries) only slightly inflate them presumably due to different age standardization[151,152], there are good reasons for skepticism, including the fact that the registries span only 2%-6% of the population of China[145]. Pooled and population based studies have shown prevalence rates of CIN2+ and hrHPV infection to be similar to or higher than in other Asian and south Asian countries with much higher cervical cancer incidence rates[145,153-156]. Although the official mortality rate is slightly higher than in countries like the United States, United Kingdom or Canada with long established, effective cervical cancer screening programs, the official incidence rate in China is only 1/3-1/2 of that in these countries, even though a great many women in China have never been screened[145,157]. In Hong Kong, opportunistic cervical screening was introduced in 1970 at which time the age standardized incidence rate was measured to be 25-35/100000/year[158] and even today, with a comprehensive cervical cancer screening program, has a crude rate of 9.7 and age standardized rate 6.9/100000/year[159]. One of the requirements of screening is that the disease must be common enough to justify the expenditure of resources and it has been suggested that a crude rate of about 3/100000/year is a reasonable threshold[160]; below this incidence rate, screening is not justified. This is likely not the situation with cervical cancer in China, but only if cervical cancer is being seriously under-reported by the registries, putting the official incidence rate in serious error. There is a clear need to resolve if the surprisingly low official incidence rate for cervical cancer in China is correct or not.

Two pooled population based analyses[154,161] found crude prevalence for invasive cervical cancer of 170/100000 and for CIN2+ of 2.6% for combined rural and urban populations not screened within the past 5 years, including a significant proportion of women who had never been screened. These data are consistent with many of those in Table 14.

In Table 14, in which most of the prevalences are lower limits due to missing biopsy follow-up data, 8 of 36 studies found invasive cancer prevalence > 250/100000; these tended to be small hospital-based studies that were possibly contaminated by diagnosis cases. In the author’s experience in China, the Pap test is widely used for diagnosis rather than for screening, even though there is evidence that samples from symptomatic women are often unsatisfactory due to blood and other reasons and have a higher false negative rate than from asymptomatic women[39]. As previously discussed, test performance can vary considerably due to differences in disease prevalence in the population tested. Laboratories that routinely use the automated quantitative image cytometry method for differential diagnosis may require re-calibration of the operating point for their instrument.

Comparisons of AQIC with LBC

The diffusion of the automated quantitative image cytometry technology in China has primarily been through existing pathology laboratories which are generally hospital based. Although the cytometers of the various vendors have regulatory approvals[162,163] and endorsement by various medical societies and expert groups, it is natural that each lab would compare performance with existing conventional liquid-based cytology. Consequently, most of the publications from China are cross-sectional, observational studies from routine clinical practice in which both liquid based conventional cytology and AQIC are done on the same sample and the test performances compared, as detailed previously. Most of these studies found substantially increased sensitivity (up to a factor of 2)[76,82,111,112,117,119,120,125,126,129-131,133,134,136,137,140-142], although some found essentially equal[55,115,116,118,120,121,124,132,135,138,139] for AQIC at a slight loss of specificity compared to conventional LBC. These reports are not summarized in a table due to the great variability in analyses and reporting as outlined previously. There would be great benefit if future publications followed a more clinically meaningful standard analysis and reporting.

One RCT conducted in China spanning some 23000 women was published and claimed substantial superiority in sensitivity and PPV comparing AQIC to conventional LBC[133]. Unfortunately, this reviewer cannot understand the analysis done in this paper[164] and cannot even say with confidence how many cases of cancer were detected in the study, even following clarification[165]. The study also had some disturbing longitudinal results that were not explained. On face value, the claims made in the publication of this trial are consistent with those of the observational studies listed above.

These papers, to varying degrees, speak to three intertwined issues: (1) that AQIC is simple and effective and could be applied in low resource settings where skilled cytologists are in short supply; (2) that AQIC is a good “second opinion” or “adjunct” to add on to the conventional LBC Pap test; or (3) that the combined test (AQIC positive or LBC cytology positive = case positive) is best because of the increased sensitivity. To this reviewer’s knowledge, issue (1) has really not been deployed in practice in China, although there seems to be a consensus belief that it would work well; most AQIC in China is deployed in hospitals with trained cytologists. Issue (3) is connected to issue (2) but generally ignores the performance cost in combined test specificity and, as previously discussed in detail, most of these papers incorrectly calculate the test specificity. Also as discussed previously, combination tests have the potential to work best when both have very high and very similar values for, say, specificity, so that the combination will greatly improve the sensitivity but only negligibly reduce specificity, or vice versa. This is generally not the case for conventional cytology and AQIC, so it is more likely that it is optimal to only use the best test.

Is it necessary to do both conventional cytology and automated quantitative image cytometry? First, here is some evidence that it is not necessary to do both. The biopsy data of Tian et al[82] was previously used to illustrate the unbiased method to compare two tests[80,81] and demonstrated that AQIC compared to conventional LBC increased the true positive (TP) count at the cost of less than 1 false positive (FP) case each, which is excellent for screening. When both tests are combined as the “or” of positive results, depending on what thresholds are used, the number of TPs increased by at most 1 case (a CIN1) while the number of increased FPs range from 0-19, indicating that even the combination test provides no added value. Similarly, an earlier paper by Yu et al[126] spanning almost 8000 cases, found that conventional LBC found one case (CIN2) missed by AQIC, but AQIC found 29 cases of CIN2+ missed by cytology using the ASCUS+ threshold. As expected, combining the two tests also added nothing.

The results of other papers show that doing both tests is beneficial. For example, the paper by Bao[76], already discussed, spanned almost 20000 women and conventional LBC found 4 cases (1 CIN3 and 3 CIN2) missed by AQIC, a marginal improvement. This comes down to a question of resource availability. As mentioned in the introduction, the conclusion of our 2005 paper[1] was that AQIC made it possible to do large scale screening inexpensively and accurately, even when skilled cytologists are not available. In that situation, doing conventional LBC is not an option.

However, when conventional LBC is an option and both tests can be performed, then the choice comes down to a mixture of the medical consequences of marginally improved sensitivity at the cost of worse specificity, combined with the business consequences, such as customer perception of value, marketing, cost, price, profit, acceptability to the payer, and so on. One Chinese vendor[166] has developed a cytometer designed to scan counterstained slides, so that the blue thionin/counterstain amounts to a fake Papanicolaou stain. This cytometer conceptually scans the slides twice-once with the optics set to minimize the visibility and interference of the counterstain to allow reasonably good DNA measurement, and once in full color for qualitative visual assessment by the reviewer. This system, in principle, has significant advantages over, for example, Hologic’s ThinPrep Imager which uses a similar approach[167] to staining and imaging slides but fails to present any DNA quantitative information to the cytologist, operating instead only as qualitative assessment aid.

In summary, in most settings in China, automated quantitative image cytometry provides substantial increase in sensitivity and relatively slightly decreased specificity compared to conventional LBC. However, most labs that offer AQIC also perform conventional cytology on the same liquid based sample with the consequence of an incremental increase in sensitivity accompanied by an incremental decrease in specificity.

AQIC, cytology and hrHPV testing

Management of women with ASCUS, an equivocal cytology result and the most common non-negative result, has always been problematic. The ASCCP guidelines of 2006[168] for the management of women with ASCUS cervical cancer screening tests called for: (1) immediate colposcopy; or (2) repeat cytology testing at 6 mo intervals until two consecutive negative follow-up Pap tests; or (3) “reflex” hrHPV testing, in which case women with hrHPV positive result are managed as if they have LSIL, or with hrHPV negative result have repeat cytology testing in 1 year.

However, these guidelines were revised in 2012[66]. The new guidelines for management of women aged ≥ 25 with ASCUS cytology are: (1) immediate colposcopy is not recommended for women of any age; (2) hrHPV testing is preferred, in which case women: (a) with hrHPV positive result are managed as if they have LSIL, or (b) with hrHPV negative result are repeat co-tested (cytology and hrHPV) in 3 years; or (3) repeat cytology testing (not preferred) in one year, and (a) if the cytology is NILM, return to routine screening interval (3 years), or (b) if the repeat cytology is ASCUS+, attend colposcopy.

For women aged 21-24 with ASCUS, the management is much more conservative: (1) immediate colposcopy is not recommended for women of any age; (2) repeat cytology testing in one year is preferred, and (a) if the first repeat cytology is ASC-H+, attend colposcopy, but (b) if the first repeat cytology is LSIL, ASCUS or NILM, repeat cytology in 1 year, and (i) if the 2nd repeat cytology is ASCUS+, attend colposcopy, or (ii) return to routine screening interval (3 years) when 2 consecutive annual repeat cytology test results are NILM; (3) hrHPV testing is not preferred, but women; (a) with hrHPV positive, repeat cytology in 1 year-immediate colposcopy or repeat hrHPV testing are not recommended, and (b) with hrHPV negative result are returned to the routine screening (cytology only) interval of 3 years.

A recent cross-sectional study by Zhang et al[169] looked at DNA ploidy in 875 cases of ASCUS who had biopsies (of which 157 were CIN2+) and a subset of 294 of these cases (53 of which were CIN2+) were also tested for hrHPV. The raw results of the 294 cases are given in Table 15.

Table 15 Biopsy data for DNA Cytometry and hrHPV test results for 294 cases of Cytology ASCUS, from Ref[169].
hrHPVHistology
NegCIN1CIN2+
hrHPV Pos216
Ploidy Neg90101
Ploidy Equ22258
Ploidy Pos51243
hrHPV Neg78
Ploidy Neg3501
Ploidy Equ1200
Ploidy Pos14160

The performance indicators (prefaced with “b” to indicate “biopsy”) for DNA ploidy for the whole data set and for hrHPV, DNA ploidy and hrHPV combined with DNA ploidy as an “and” of positive test results are given in Table 16 (the combined test result was not included in the Zhang paper). In general, DNA ploidy had better specificity and hence better PPV than hrHPV testing and both had similar NPV complement. As the authors concluded, AQIC is as effective as hrHPV testing for managing ASCUS patients, while being cheaper and easier to use.

Table 16 Test performance indicators for DNA Cytometry, hrHPV tests and combined DNA Cytometry and hrHPV tests, for Cytology ASCUS cases, Ref[169].
875 Cases (157 CIN2+)
294 Cases (53 CIN2+)
Performance indicator for CIN2+ (%)DNA PloidyhrHPVDNA PloidyDNA Ploidy and hrHPV
bSensitivity98.898.196.296.2
bSpecificity47.532.056.073.4
bPPV29.124.132.544.3
bNPV99.498.798.598.9

As discussed previously, for the logical “and” of positive test results the specificity of the combined test will be at least as high as the specificity of the most specific component test and could be 100%; however, the sensitivity will be no better than the sensitivity of the least sensitive component test and could be zero. In fact, the combined test sensitivity did not fall below the least sensitive test which indicates that for detecting positive cases, DNA ploidy and HPV testing are very highly correlated. This is also an example of why it is generally the case that combining binary tests only works well if both have similarly high values for one performance indicator (either sensitivity or specificity); the idea is that this high valued performance indicator will drop only minimally when the tests are combined, while the lower valued other performance indicator will rise substantially. That is exactly what happened in this example. For reference, the PPV of cytology for all 875 ASCUS cases is only 18%.

If it is accepted that management of ASCUS will inevitably involve some equivocal cases who must be monitored over time, similar to the ASCCP guidelines above, then the 3 × 3 table analysis previously discussed could be applied. In this situation, only the "biopsy Positive Predictive Values" (bPPV) are revised upwards for each test, as shown in Table 17.

Table 17 Test performance indicators following re-analysis of the data of Table 16 from Ref[169].
875 Cases (157 CIN2+)
294 Cases (53 CIN2+)
Performance indicator for CIN2+ (%)DNA PloidyhrHPVDNA PloidyDNA Ploidy and hrHPV
bSensitivity98.898.196.296.2
bSpecificity47.532.056.073.4
bPPV40.824.147.871.7
bNPV99.498.798.598.9

In the case of the combined test: (1) 60 cases would be identified as positive of which 43 will have CIN2+, including 100% of the cancer cases, 88% of the CIN3 cases and 70% of the CIN2 cases, (2) 179 cases would be identified as negative, of which 2 would have CIN2+, and (3) 55 cases would be identified as being equivocal of which 8 would have CIN2+.

For comparison, if the hrHPV test was not done, the 3 state analysis outcomes would be: (1) 90 cases would be identified as positive of which 43 will have CIN2+, exactly as in the combined case above, (2) 137 cases would be identified as negative, of which 2 would have CIN2+, and (3) 67 cases would be identified as being equivocal of which 8 would have CIN2+.

The addition of the hrHPV test correctly removed 30 negative cases from the positive group and 12 from the equivocal group into the negative group.

The same research group more recently published a smaller prospective study[170] of 327 cases of ASCUS, 193 of whom had the AQIC test and the remaining 134 had the hrHPV test. The performance indicators were essentially the same as the previous study and no statistically significant difference was detected between AQIC and hrHPV. Again, if the patient management followed the 3 × 3 table guidelines discussed previously, the PPV of DNA ploidy would have been boosted from 32% to 41%.

A tiny study from France using interactive (as opposed to automated) image analysis found similar concordance between DNA ploidy and hrHPV for ASCUS cases[171].

A few additional studies comparing hrHPV with AQIC, but not focused on ASCUS, have been published in China[172-174] and generally they show reasonable concordance between the two techniques but are not large enough to draw more conclusions.

The previously mentioned study of Guillaud and coworkers[41] compared conventional LBC (ThinPrep) with DNA ploidy and with hrHPV testing by QiaGen HC2 on more than 1500 high grade samples and concluded: “DNA ploidy shows comparable sensitivity, specificity, PPV and NPV values to conventional cytology and HC 2.”

In summary, there is evidence that AQIC is at least as useful as hrHPV testing for managing ASCUS cases and probably for other abnormal cytology grades and is simpler and cheaper to perform.

CONCLUSION

Like all countries, China has low-, mid- and high-resource settings. Referring to the 2005 study objective recounted in introduction, the key resource under discussion here is the availability of experienced and highly trained cytotechnologists and cytopathologists-if they do not exist, then the conventional Pap test cannot be performed. The other key resource, of course, is money which constrains public health initiatives in all countries. This is not a discussion about the availability of electricity, water or similar resources. The objective cited in the introduction was related to this definition of low resource settings and not just in China. This review will look separately at the low resource and mid-/high-resource settings.

Training

A key element of AQIC technology is that it can be taught much more quickly than cytology and in the hands of the trainees, can perform comparably to conventional LBC, performed by experienced and highly trained cytologists. The record in China has demonstrated that it is routinely possible to teach the technology from slide preparation and staining, to operation of the cytometer, review of the DNA ploidy data and report generation in 10 working days, especially when dedicated training facilities are available, as is the case with vendors in China. It is somewhat more difficult and inefficient to teach the technology in the end-user facility because the trainees usually have other routine work to do and so their attention is divided. Even so, this reviewer did have the opportunity to teach this technology to technicians in Morocco about 1 year ago at their facility; competition with their routine work was a big challenge, as were language and other issues, but the training was successful in a relatively short period of time (3 wk total). In this case, the trainees were experienced pathology lab technicians, so they were quick studies for sample preparation and staining. However, they had absolutely no experience in microscopy which is only done by the pathologists there and so all aspects of data and image review had to be mastered by them.

Various AQIC products from different vendors are in wide use in China in the hands of many hundreds of different operators and yet the review of publications in the previous section indicates that reasonably consistent results are being obtained under widely disparate conditions. This speaks to the inherent robustness of the DNA ploidy technology.

Low resource setting deployment

In every country, public health programs are first and foremost political initiatives, a fact that strongly shapes what is done and how. China started a “Two Cancer” screening program (cervical and breast) targeted at rural women in central and western China in 2009 that will continue until at least 2015[175]. This is apparently a follow-on to a 2002-2010 cervical cancer screening program[150], possibly to enjoy the anticipated high popularity and expected efficiencies of combined vs single disease screening[176]. One curiosity with this initiative is that the target population is aged 30-59 for cervical cancer screening, which is appropriate, but 35-69 for breast screening by clinical breast exam[177]; I have direct knowledge of the target age being 25-59 for breast cancer screening in Ordos under this program. Few outside of China advocate screening for breast cancer before age 50[178] (although some, such as BCCA, initiate breast cancer screening starting at age 40[54]) which, combined with the fact that Chinese women both inside and outside China have 1/3-1/2 the breast cancer incidence rates as other women[179,180], has led a Hong Kong expert group, who have twice reviewed the evidence recently[181,182], to conclude that there is insufficient evidence to recommend for or against breast cancer screening.

This reviewer has personal experience that AQIC was implemented for this program in the city of Ordos (metropolitan population 600000) in Inner Mongolia but it was required that a cytometer and support staff be provided to each of the 8 or 10 participating hospitals rather than setting up a central lab with two technicians and only two cytometers, which would not only make QA easier to establish and maintain, but would enjoy the economy of scale that comes from a high volume, high throughput operation. The point is that the concepts of efficiency can be mixed and contradictory when public health programs are implemented because of many competing interests.

This is a point of great disappointment for this reviewer because the opportunity has so far been lost to learn in real practice how affordable cervical cancer screening could be in China. The 2012 study by Li et al[176] determined the affordability limit for rural women for the “two cancer” screening to be 50 RMB, which is probably achievable with an efficient program implementation. Money remains a major constraint to this Two Cancer public health initiative[177].

So far, China is the center of DNA ploidy application to cervical cancer screening. However, a pilot project is underway in Morocco. The first phase was to demonstrate that the technology could be taught and learned, the second phase was to compare the results with split samples and conventional LBC to determine if the technology had enough merit to move forward into phase three: the actual screening project, which is now underway. A similar project is being conducted in the Philippines[183] but no reports on it are yet available.

Mid-and high-resource setting deployment

Most of the preceding papers reviewed are from laboratories that also perform cytology and so do not lack the key resource of trained and experienced cytologists. However, there does not seem to be the profession of “cytotechnologist” in China, except in Hong Kong[184]. In many countries, screening by cytology is divided into 2 tasks: (1) the locator function, performed by a cytotechnologist who can sign out negative cases; and (2) the interpreter function performed by a cytopathologist who is the only one who can sign out positive cases. The locator skills of cytotechnologists are generally superior to those of cytopathologists[185]. In China, normally these two tasks are performed by a cytopathologist. Although it is very rapidly improving, as recently as 2005 only 1/3 of licensed doctors in China held the equivalent of a bachelor’s degree (only 1/8 in rural China)[186], so there is a case to be made for AQIC technology even where cytologists are available. This is confirmed by the wide acceptance of the technology there to date.

One frustration facing the authors of most published studies in China is the previously discussed “missing data”-the inability to “close the loop” and gain access to biopsy and treatment data necessary to gauge relative success of the screening. All medical databases suffer from some “persons lost to follow-up” but there is a big difference between 60% and 6% missing data. There is no effective coordination and cooperation between the various medical providers and the various professional medical societies and regulators seem to lack the clout to make such cooperation happen, even at the city district level, let alone the city, county, province or national level in China. There are undoubtedly many factors contributing to this situation. It is impossible to know how well any technology or program is doing if it cannot be measured. Missing data makes even cross-sectional studies, such as those reviewed here, very difficult. This situation will make longitudinal studies virtually impossible and longitudinal studies are a crucial element in the pathway to evaluating a screening strategy[48]. Other countries, even the United States, have found ways to measure and share outcome data while still respecting patient autonomy and confidentiality. A great opportunity is being lost in China due to this lack of coordination and cooperation.

The revised ASCCP patient management guidelines[66] were substantially informed by the remarkable database of 1.4 million women managed by Kaiser Permanente Northern California (KPNC) with both LBC and hrHPV testing from 2003 to 2010[187-194]. This substantial longitudinal database allowed the formulation of “risk adapted” patient management guidelines. KPNC is a not-for-profit, private Health Maintenance Organization-essentially an activist insurance company that is also involved in the medical service delivery. Screening for cancer is a very easy way to spend lots of money because it involves testing people, of whom 90+% are healthy. So KPNC is highly motivated to evaluate what it does and to consider what it could do by adapting to maximize the health of its clients and minimize the cost. Perhaps, this is generalizable-that the insurer is the most motivated to effect the kind of change required to measure screening on the required scale.

FUTURE WORK

Much remains to be done, according to the framework for evaluating screening strategies outlined by Arbyn and co-workers[48].

To this reviewer, the most exciting results to emerge from this review are the comparisons with hrHPV testing with DNA ploidy that seem to show that

DNA ploidy = LBC + hrHPV testing

The work on ASCUS needs to be repeated and expanded to all of the other categories of cytological abnormality. It would be especially interesting to conduct a study involving CareHPV[195], the low priced hrHPV test from QiaGen for low resource settings. Combinations of DNA ploidy with HC 2 or other widely used hrHPV tests in “non-low resource” settings would also be valuable.

Much of the world has started vaccinating girls and in some cases, boys, against HPV and this is expected to expand throughout the world, including low resource countries[196]. As the vaccinated cohorts reach screening age, existing well established cytology based screening programs will become stressed because the frequency of abnormality will drop significantly (nominally 70% with current vaccines) which will make it increasingly more difficult for cytologists to maintain both competence and vigilance. DNA Ploidy by automated quantitative image cytometry would greatly relieve this stress.

hrHPV testing requires an accompanying “triage” test to manage the 10%-15% of women who will test positive[197]. The ASCUS hrHPV/DNA ploidy data of Zhang et al[169], reviewed above, seemed to be the ideal case for combining tests because both have comparably high sensitivity. However, in this example, both tests were made into binary tests which are constrained to increasing either sensitivity or specificity, while decreasing the other, even though both are inherently continuously valued tests. It would also be interesting to look at what happens when hrHPV and DNA ploidy are combined, not as binary tests but as continuously valued tests, for example, using Bayes’ theorem. It may then be possible to find conditions where both sensitivity and specificity increase at the same time. Nothing is free in this world, so it is unrealistic to expect that the joint improvement would exceed that of the binary test, but it might be enough to optimize the test combination for some screening applications.

The value of cervical cancer screening has been essentially limited to squamous cell carcinoma and has done little to reduce the mortality and morbidity from adenocarcinoma, which is increasing in incidence[198-200]. It is unclear how much sampling contributes to the low sensitivity of cytology to adenocarcinoma, but studies using hrHPV testing suggest that many cases of adenocarcinoma are hrHPV positive and cytology negative, indicating that the samples are adequate[191,201,202]. It would be interesting to compare the sensitivity for detection of adenocarcinoma for DNA Cytometry with that of conventional LBC and hrHPV testing.

A number of other issues and opportunities for further research have been brushed against in this review. The important issue of official cervical cancer incidence rates needs to be reconciled with those of the rest of the world and with the very high prevalence rates seen in many studies in China. To this end, it would be beneficial if papers more clearly defined their test populations, especially if they are an admixture of screening and diagnosis cases.

The question of whether conventional LBC done in addition to AQIC provides a net benefit remains unresolved, although it may be both vendor product and laboratory staff skill dependent.

The problem of the imperfect “gold standard” test and the possible impact, if any, that has on DNA ploidy results would be useful to know. Is there a difference in measured DNA ploidy sensitivity when the gold standard is colposcopy directed biopsy vs random biopsy?

Much was written here about a definition of sample adequacy but the question is not resolved. What is the best number of cells to measure to get a reliable negative test result, where “best” is a trade-off between scan time and false negative rate? How much is this influenced by sample taking? If the adequacy number is increased to, say, 50000 epithelial cells, what does this do to the diagnostic rules-that is, will 1 or 2 aneuploid cells still be “equivocal” and 3 positive? In fact, at any epithelial cell count threshold, it would be good to review all of the diagnostic rules including the proliferation rules for positive, negative and equivocal. While they seem to be working reasonably well now over a huge range of undiagnosed disease prevalence, are they optimal?

In a related matter, the concern expressed by Chatelain et al[42] on HPV infection induced polyploidy may be worth looking at. That is, should cells with DI of 2, 4 or 8 be grouped separately from other DI > 2.5 cells to determine if they have different value in screening? This could be coupled to any HPV testing project.

Another similar issue is the malignant potential of aneuploid stemlines. This requires good longitudinal data and also poses possible problems with ethical approval. However, if such stemlines routinely result in negative colposcopy and/or biopsy (seen as hyperplasia), it would be interesting to follow such patients over 5 years to see how much it progresses.

Finally, one of the weaknesses in DNA ploidy by the methods described here comes from Feulgen staining which is very slow (3 to 4 h procedure) and can also be somewhat finicky. Guillaud et al[203] developed a modified Feulgen process for the dye Azure A (a thiazine like thionin, methylene blue, Azure B etc.) that takes 30 min but produces a slightly wider diploid peak. Another rapid Feulgen reaction was reported from China[204] but it seemed to be partly a re-discovery of acid hydrolysis at a temperature of 60C, which was used by Feulgen[22] originally and has been not recommended for routine work due to the need for precise control of all staining conditions. Claims are made from time to time that variations of the hematoxylin staining are quantitative, but this has been rejected experimentally by Biesterfeld[205]. The need for an improved (faster) staining protocol remains.

Footnotes

P- Reviewer: Chen P, Chen Z S- Editor: Wen LL L- Editor: Roemmele A E- Editor: Lu YJ

References
1.  Sun XR, Wang J, Garner D, Palcic B. Detection of cervical cancer and high grade neoplastic lesions by a combination of liquid-based sampling preparation and DNA measurements using automated image cytometry. Cell Oncol. 2005;27:33-41.  [PubMed]  [DOI]  [Cited in This Article: ]
2.  Biesterfeld S, Gerres K, Fischer-Wein G, Böcking A. Polyploidy in non-neoplastic tissues. J Clin Pathol. 1994;47:38-42.  [PubMed]  [DOI]  [Cited in This Article: ]
3.  Zasadil LM, Britigan EM, Weaver BA. 2n or not 2n: Aneuploidy, polyploidy and chromosomal instability in primary and tumor cells. Semin Cell Dev Biol. 2013;24:370-379.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 63]  [Cited by in F6Publishing: 74]  [Article Influence: 6.7]  [Reference Citation Analysis (0)]
4.  Weaver BA, Cleveland DW. Does aneuploidy cause cancer? Curr Opin Cell Biol. 2006;18:658-667.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 372]  [Cited by in F6Publishing: 398]  [Article Influence: 22.1]  [Reference Citation Analysis (0)]
5.  Hansemann D. Ueber asymmetrische Zelltheilung in Epithelkrebsen und deren biologische Bedeutung. Virchows Arch. 1890;119:299-326.  [PubMed]  [DOI]  [Cited in This Article: ]
6.  Hansemann D. Ueber pathologische Mitosen. Virchows Arch. 1891;123:356-370.  [PubMed]  [DOI]  [Cited in This Article: ]
7.  Bignold LP, Coghlan BL, Jersmann HP. David Paul Hansemann: chromosomes and the origin of the cancerous features of tumor cells. Cell Oncol. 2009;31:61.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in F6Publishing: 5]  [Reference Citation Analysis (0)]
8.  Köhler A. Ein neues Beleuchtungsverfahren für mikrophotographische Zwecke. Zeitschrift für wissenschaftliche Mikroskopie und für Mikroskopische Technik [Internet]. 1893;10:433-440 Available from: http://www.archive.org/stream/zeitschriftfrw10stut#page/432/mode/2up.  [PubMed]  [DOI]  [Cited in This Article: ]
9.  Boveri T. Über mehrpolige Mitosen als Mittel zur Analyse des Zellkerns. Verh Phys Med Gesellschaft Würzburg. 1902;35:67-90.  [PubMed]  [DOI]  [Cited in This Article: ]
10.  Boveri T. [On Multipolar Mitosis as a Means of Analysis of the Cell Nucleus] [Internet].  Available from: http://10e.devbio.com/article.php?id=74.  [PubMed]  [DOI]  [Cited in This Article: ]
11.  Boveri T Zur Frage der Entstehung maligner Tumoren. Fischer Verlag: Jena 1914; .  [PubMed]  [DOI]  [Cited in This Article: ]
12.  Boveri T. Concerning the origin of malignant tumours by Theodor Boveri. Translated and annotated by Henry Harris. J Cell Sci. 2008;121 Suppl 1:1-84.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 347]  [Cited by in F6Publishing: 373]  [Article Influence: 23.3]  [Reference Citation Analysis (0)]
13.  Boveri T The origin of malignant tumors. Baltimore: Williams and Wilkins 1929; .  [PubMed]  [DOI]  [Cited in This Article: ]
14.  Wunderlich V. JMM---past and present. Chromosomes and cancer: Theodor Boveri’s predictions 100 years later. J Mol Med (Berl). 2002;80:545-548.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 13]  [Cited by in F6Publishing: 11]  [Article Influence: 0.5]  [Reference Citation Analysis (1)]
15.  Duijf PH, Schultz N, Benezra R. Cancer cells preferentially lose small chromosomes. Int J Cancer. 2013;132:2316-2326.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 111]  [Cited by in F6Publishing: 123]  [Article Influence: 10.3]  [Reference Citation Analysis (0)]
16.  Krajcovic M, Overholtzer M. Mechanisms of ploidy increase in human cancers: a new role for cell cannibalism. Cancer Res. 2012;72:1596-1601.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 68]  [Cited by in F6Publishing: 72]  [Article Influence: 6.0]  [Reference Citation Analysis (0)]
17.  Ganem NJ, Godinho SA, Pellman D. A mechanism linking extra centrosomes to chromosomal instability. Nature. 2009;460:278-282.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1025]  [Cited by in F6Publishing: 1077]  [Article Influence: 71.8]  [Reference Citation Analysis (0)]
18.  Weaver BA, Cleveland DW. The role of aneuploidy in promoting and suppressing tumors. J Cell Biol. 2009;185:935-937.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 60]  [Cited by in F6Publishing: 62]  [Article Influence: 4.1]  [Reference Citation Analysis (0)]
19.  Böcking A, Giroud F, Reith A. Consensus report of the ESACP task force on standardization of diagnostic DNA image cytometry. European Society for Analytical Cellular Pathology. Anal Cell Pathol. 1995;8:67-74.  [PubMed]  [DOI]  [Cited in This Article: ]
20.  Haroske G, Baak JP, Danielsen H, Giroud F, Gschwendtner A, Oberholzer M, Reith A, Spieler P, Böcking A. Fourth updated ESACP consensus report on diagnostic DNA image cytometry. Anal Cell Pathol. 2001;23:89-95.  [PubMed]  [DOI]  [Cited in This Article: ]
21.  Feulgen R, Rossenbeck H. Mikroskopisch-chemischer Nachweis einer Nucleinsaure vom Typus der Thymonucleinsaure und die darauf beruhende elektive Farbung von Zellkernen in mikroskopischen Praparaten. Z Physiol Chem. 1924;135:203-248.  [PubMed]  [DOI]  [Cited in This Article: ]
22.  Chieco P, Derenzini M. The Feulgen reaction 75 years on. Histochem Cell Biol. 1999;111:345-358.  [PubMed]  [DOI]  [Cited in This Article: ]
23.  Chieco P, Jonker A, De Boer BA, Ruijter JM, Van Noorden CJ. Image cytometry: protocols for 2D and 3D quantification in microscopic images. Prog Histochem Cytochem. 2013;47:211-333.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 38]  [Cited by in F6Publishing: 32]  [Article Influence: 2.9]  [Reference Citation Analysis (0)]
24.  Lapidus SN. Founder, former President and CEO Cytyc Corp., personal communication .  [PubMed]  [DOI]  [Cited in This Article: ]
25.  Solomon D, Davey D, Kurman R, Moriarty A, O’Connor D, Prey M, Raab S, Sherman M, Wilbur D, Wright T. The 2001 Bethesda System: terminology for reporting results of cervical cytology. JAMA. 2002;287:2114-2119.  [PubMed]  [DOI]  [Cited in This Article: ]
26.  Stoler MH, Schiffman M. Interobserver reproducibility of cervical cytologic and histologic interpretations: realistic estimates from the ASCUS-LSIL Triage Study. JAMA. 2001;285:1500-1505.  [PubMed]  [DOI]  [Cited in This Article: ]
27.  CAAHEP Accredited Cytotechnology Programs. American Society of Cytopathology [Internet].  Available from: http://www.cytopathology.org/caahep-accredited-cytotechnology-programs/.  [PubMed]  [DOI]  [Cited in This Article: ]
28.  CAAHEP Accredited Cytotechnology Programs; NAIT. Cytotechnology [Internet].  Available from: http://www.nait.ca/program_home_76613.htm.  [PubMed]  [DOI]  [Cited in This Article: ]
29.  Anshu A, Cochand-Priollet B, Cross P, Desai M, Dina R, Duskova J, Evered A, Farnsworth A, Gray W, Gupta SS. Survey of medical training in cytopathology carried out by the journal Cytopathology. Cytopathology. 2010;21:147-156.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 18]  [Cited by in F6Publishing: 21]  [Article Influence: 1.5]  [Reference Citation Analysis (0)]
30.   Available from: http://www.abpath.org/BookletofInformation.pdf.  [PubMed]  [DOI]  [Cited in This Article: ]
31.   Available from: http://www.royalcollege.ca/cs/groups/public/documents/document/y2vk/mdaw/~edisp/tztest3rcpsced000604.pdf.  [PubMed]  [DOI]  [Cited in This Article: ]
32.   Available from: http://www.royalcollege.ca/cs/groups/public/documents/document/mdaw/mday/~edisp/002207.pdf.  [PubMed]  [DOI]  [Cited in This Article: ]
33.  Davey DD, Cox JT, Austin RM, Birdsong G, Colgan TJ, Howell LP, Husain M, Darragh TM. Cervical cytology specimen adequacy: patient management guidelines and optimizing specimen collection. J Low Genit Tract Dis. 2008;12:71-81.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 31]  [Cited by in F6Publishing: 34]  [Article Influence: 2.1]  [Reference Citation Analysis (0)]
34.  McQueen F, Duvall E. Using a quality control approach to define an ‘adequately cellular’ liquid-based cervical cytology specimen. Cytopathology. 2006;17:168-174.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 20]  [Cited by in F6Publishing: 20]  [Article Influence: 1.1]  [Reference Citation Analysis (0)]
35.  Bishop JW. Cellularity of liquid-based, thin-layer cervical cytology slides. Acta Cytol. 2002;46:633-636.  [PubMed]  [DOI]  [Cited in This Article: ]
36.  Studeman KD, Ioffe OB, Puszkiewicz J, Sauvegeot J, Henry MR. Effect of cellularity on the sensitivity of detecting squamous lesions in liquid-based cervical cytology. Acta Cytol. 2003;47:605-610.  [PubMed]  [DOI]  [Cited in This Article: ]
37.  Swift H. The constancy of desoxyribose nucleic acid in plant nuclei. Proc Natl Acad Sci USA. 1950;36:643-654.  [PubMed]  [DOI]  [Cited in This Article: ]
38.  Ris H, Mirsky AE. Quantitative cytochemical determination of desoxyribonucleic acid with the Feulgen nucleal reaction. J Gen Physiol. 1949;33:125-146.  [PubMed]  [DOI]  [Cited in This Article: ]
39.  Screening for Cancer of the Cervix-An Office Manual for Health Professionals [Internet]. 2013;.  Available from: http://www.screeningbc.ca/NR/rdonlyres/21BBF070-6504-4A37-A1BB-45563BF387C7/63243/CCSPmanual_Mar2013_Small1.pdf.  [PubMed]  [DOI]  [Cited in This Article: ]
40.  Mayrand MH, Duarte-Franco E, Rodrigues I, Walter SD, Hanley J, Ferenczy A, Ratnam S, Coutlée F, Franco EL. Human papillomavirus DNA vs Papanicolaou screening tests for cervical cancer. N Engl J Med. 2007;357:1579-1588.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 767]  [Cited by in F6Publishing: 714]  [Article Influence: 42.0]  [Reference Citation Analysis (0)]
41.  Guillaud M, Benedet JL, Cantor SB, Staerkel G, Follen M, MacAulay C. DNA ploidy compared with human papilloma virus testing (Hybrid Capture II) and conventional cervical cytology as a primary screening test for cervical high-grade lesions and cancer in 1555 patients with biopsy confirmation. Cancer. 2006;107:309-318.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 35]  [Cited by in F6Publishing: 37]  [Article Influence: 2.1]  [Reference Citation Analysis (0)]
42.  Chatelain R, Schunck T, Schindler EM, Schindler AE, Böcking A. Diagnosis of prospective malignancy in koilocytic dysplasias of the cervix with DNA cytometry. J Reprod Med. 1989;34:505-510.  [PubMed]  [DOI]  [Cited in This Article: ]
43.  Zhang Z, Liu Z. Clinical study of cellular DNA quantitative analysis method for cervical cancer screening. Xiandai Funv. 2010;9:121-126.  [PubMed]  [DOI]  [Cited in This Article: ]
44.  Bollmann R, Bankfalvi A, Griefingholt H, Trosic A, Speich N, Schmitt C, Bollmann M. Validity of combined cytology and human papillomavirus (HPV) genotyping with adjuvant DNA-cytometry in routine cervical screening: results from 31031 women from the Bonn-region in West Germany. Oncol Rep. 2005;13:915-922.  [PubMed]  [DOI]  [Cited in This Article: ]
45.  Li M, Liu F, Di Y, Deng Q, Zhong P. Value of DNA quantitative analysis combined with colposcopy for diagnosis of cervical cancer and high-grade cervical intraepithelial neoplasia. Zhongguo Fuyou Baojian. 2012;27:2841-2844.  [PubMed]  [DOI]  [Cited in This Article: ]
46.  Li G, Guillaud M, Follen M, MacAulay C. Double staining cytologic samples with quantitative Feulgen-thionin and anti-Ki-67 immunocytochemistry as a method of distinguishing cells with abnormal DNA content from normal cycling cells. Anal Quant Cytol Histol. 2012;34:273-284.  [PubMed]  [DOI]  [Cited in This Article: ]
47.  Arbyn M, Ronco G, Cuzick J, Wentzensen N, Castle PE. How to evaluate emerging technologies in cervical cancer screening? Int J Cancer. 2009;125:2489-2496.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 77]  [Cited by in F6Publishing: 75]  [Article Influence: 5.0]  [Reference Citation Analysis (0)]
48.  Adriaensen WJ, Matheï C, Buntinx FJ, Arbyn M. A framework provided an outline toward the proper evaluation of potential screening strategies. J Clin Epidemiol. 2013;66:639-647.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 14]  [Cited by in F6Publishing: 18]  [Article Influence: 1.6]  [Reference Citation Analysis (0)]
49.  van Hemel BM, Haarsma JG, Ruitenbeek T, Groen HJ, Suurmeijer AJ. Application of the ThinPrep imaging system in urine cytology: a prospective study. Cancer Cytopathol. 2013;121:410-414.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 10]  [Cited by in F6Publishing: 11]  [Article Influence: 1.0]  [Reference Citation Analysis (0)]
50.   Available from: http://www.screeningbc.ca/NR/rdonlyres/F2D89BB9-3B33-4DBD-BE76-975A220B0C7B/69596/CCSPAnnualReport2012PrintVersion.pdf.  [PubMed]  [DOI]  [Cited in This Article: ]
51.  Stoler MH, Castle PE, Solomon D, Schiffman M. The expanded use of HPV testing in gynecologic practice per ASCCP-guided management requires the use of well-validated assays. Am J Clin Pathol. 2007;127:335-337.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 116]  [Cited by in F6Publishing: 107]  [Article Influence: 6.3]  [Reference Citation Analysis (0)]
52.  Meijer CJ, Berkhof J, Castle PE, Hesselink AT, Franco EL, Ronco G, Arbyn M, Bosch FX, Cuzick J, Dillner J. Guidelines for human papillomavirus DNA test requirements for primary cervical cancer screening in women 30 years and older. Int J Cancer. 2009;124:516-520.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 500]  [Cited by in F6Publishing: 493]  [Article Influence: 32.9]  [Reference Citation Analysis (0)]
53.  Kinney W, Stoler MH, Castle PE. Special commentary: patient safety and the next generation of HPV DNA tests. Am J Clin Pathol. 2010;134:193-199.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 73]  [Cited by in F6Publishing: 56]  [Article Influence: 4.0]  [Reference Citation Analysis (0)]
54.   Available from: http://www.screeningbc.ca/NR/rdonlyres/D302DDFE-474D-48F2-912D-F5612AA8B204/68074/SMPAnnualReport2013Rev14January2014.pdf.  [PubMed]  [DOI]  [Cited in This Article: ]
55.  Tong H, Wang Z, Ming S, Yan K, Yan J, Li F. DNA ploidy analysis and routine cytologic diagnosis of cervical intraepithelial neoplasia compared diagnostic evaluation. Zhonghua Linchuang Yishi Zazhi (Dianzi Ban). 2009;3:47-55.  [PubMed]  [DOI]  [Cited in This Article: ]
56.  Lord SJ, Staub LP, Bossuyt PM, Irwig LM. Target practice: choosing target conditions for test accuracy studies that are relevant to clinical practice. BMJ. 2011;343:d4684.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 31]  [Cited by in F6Publishing: 35]  [Article Influence: 2.7]  [Reference Citation Analysis (0)]
57.  Castle PE, Schiffman M, Wheeler CM, Solomon D. Evidence for frequent regression of cervical intraepithelial neoplasia-grade 2. Obstet Gynecol. 2009;113:18-25.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 277]  [Cited by in F6Publishing: 283]  [Article Influence: 18.9]  [Reference Citation Analysis (0)]
58.  Wang SM, Colombara D, Shi JF, Zhao FH, Li J, Chen F, Chen W, Li SM, Zhang X, Pan QJ. Six-year regression and progression of cervical lesions of different human papillomavirus viral loads in varied histological diagnoses. Int J Gynecol Cancer. 2013;23:716-723.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 26]  [Cited by in F6Publishing: 27]  [Article Influence: 2.7]  [Reference Citation Analysis (0)]
59.  Sasieni P, Castanon A, Parkin DM. How many cervical cancers are prevented by treatment of screen-detected disease in young women? Int J Cancer. 2009;124:461-464.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 31]  [Cited by in F6Publishing: 33]  [Article Influence: 2.2]  [Reference Citation Analysis (0)]
60.  Sasieni PD, Castanon A, Parkin MD. Response to: Why young women should be screened for cervical cancer: The distinction between CIN2 and CIN3. Int J Cancer. 2009;126:2257-2258.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1]  [Cited by in F6Publishing: 1]  [Article Influence: 0.1]  [Reference Citation Analysis (0)]
61.  Morrison BJ, Coldman AJ, Boyes DA, Anderson GH. Forty years of repeated screening: the significance of carcinoma in situ. Br J Cancer. 1996;74:814-819.  [PubMed]  [DOI]  [Cited in This Article: ]
62.  Moscicki AB, Ma Y, Wibbelsman C, Darragh TM, Powers A, Farhat S, Shiboski S. Rate of and risks for regression of cervical intraepithelial neoplasia 2 in adolescents and young women. Obstet Gynecol. 2010;116:1373-1380.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 184]  [Cited by in F6Publishing: 179]  [Article Influence: 13.8]  [Reference Citation Analysis (0)]
63.  Moscicki AB, Shiboski S, Hills NK, Powell KJ, Jay N, Hanson EN, Miller S, Canjura-Clayton KL, Farhat S, Broering JM. Regression of low-grade squamous intra-epithelial lesions in young women. Lancet. 2004;364:1678-1683.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 295]  [Cited by in F6Publishing: 266]  [Article Influence: 13.3]  [Reference Citation Analysis (0)]
64.  Matsumoto K, Oki A, Furuta R, Maeda H, Yasugi T, Takatsuka N, Mitsuhashi A, Fujii T, Hirai Y, Iwasaka T. Predicting the progression of cervical precursor lesions by human papillomavirus genotyping: a prospective cohort study. Int J Cancer. 2011;128:2898-2910.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 80]  [Cited by in F6Publishing: 87]  [Article Influence: 6.2]  [Reference Citation Analysis (0)]
65.  Vink MA, Bogaards JA, van Kemenade FJ, de Melker HE, Meijer CJ, Berkhof J. Clinical progression of high-grade cervical intraepithelial neoplasia: estimating the time to preclinical cervical cancer from doubly censored national registry data. Am J Epidemiol. 2013;178:1161-1169.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 69]  [Cited by in F6Publishing: 87]  [Article Influence: 7.9]  [Reference Citation Analysis (0)]
66.  Massad LS, Einstein MH, Huh WK, Katki HA, Kinney WK, Schiffman M, Solomon D, Wentzensen N, Lawson HW. 2012 updated consensus guidelines for the management of abnormal cervical cancer screening tests and cancer precursors. J Low Genit Tract Dis. 2013;17:S1-S27.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 478]  [Cited by in F6Publishing: 494]  [Article Influence: 44.9]  [Reference Citation Analysis (0)]
67.  Wuntakal R, Castanon A, Sasieni PD, Hollingworth A. Pregnancy outcomes after treatment for cervical intraepithelial neoplasia in a single NHS hospital. Int J Gynecol Cancer. 2013;23:710-715.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 5]  [Cited by in F6Publishing: 6]  [Article Influence: 0.6]  [Reference Citation Analysis (0)]
68.  Castanon A, Brocklehurst P, Evans H, Peebles D, Singh N, Walker P, Patnick J, Sasieni P. Risk of preterm birth after treatment for cervical intraepithelial neoplasia among women attending colposcopy in England: retrospective-prospective cohort study. BMJ. 2012;345:e5174.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 83]  [Cited by in F6Publishing: 79]  [Article Influence: 6.6]  [Reference Citation Analysis (0)]
69.  Lima AF, Francisco C, Júlio C, Paula T, Vitorino A, Borrego J. Obstetric outcomes after treatment for cervical intraepithelial neoplasia: six years of experience. J Low Genit Tract Dis. 2011;15:276-279.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 9]  [Cited by in F6Publishing: 11]  [Article Influence: 0.9]  [Reference Citation Analysis (0)]
70.  Cotton S, Sharp L, Cochran C, Gray N, Cruickshank M, Smart L, Thornton A, Little J. After-effects reported by women having follow-up cervical cytology tests in primary care: a cohort study within the TOMBOLA trial. Br J Gen Pract. 2011;61:e333-e339.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 7]  [Cited by in F6Publishing: 7]  [Article Influence: 0.5]  [Reference Citation Analysis (0)]
71.  Sharp L, Cotton S, Cochran C, Gray N, Little J, Neal K, Cruickshank M. After-effects reported by women following colposcopy, cervical biopsies and LLETZ: results from the TOMBOLA trial. BJOG. 2009;116:1506-1514.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 67]  [Cited by in F6Publishing: 74]  [Article Influence: 4.9]  [Reference Citation Analysis (0)]
72.  Bevis KS, Biggio JR. Cervical conization and the risk of preterm delivery. Am J Obstet Gynecol. 2011;205:19-27.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 63]  [Cited by in F6Publishing: 77]  [Article Influence: 5.9]  [Reference Citation Analysis (0)]
73.  Sasieni PD, Castanon A, Cuzick J. NHSCSP No 31: The impact of cervical screening on young women: A critical review of the literature [Internet].  Available from: http://www.cancerscreening.nhs.uk/cervical/publications/nhscsp31.html.  [PubMed]  [DOI]  [Cited in This Article: ]
74.  Vesco K, Whitlock E, Eder M, Lin J. Screening for Cervical Cancer: A Systematic Evidence Review for the US Preventive Services Task Force [Internet].  Available from: http://www.ncbi.nlm.nih.gov/books/NBK66099/.  [PubMed]  [DOI]  [Cited in This Article: ]
75.  Kulasingam S, Havrilesky L, Ghebre R. Screening for Cervical Cancer: A Decision Analysis for the US Preventive Services Task Force [Internet].  Available from: http://www.ncbi.nlm.nih.gov/books/NBK92546/.  [PubMed]  [DOI]  [Cited in This Article: ]
76.  Bao L. Clinical Observation on the Diagnosis of High-grade Cervical Intraepithelial Neoplasia and Cervical Cancer using Automatic DNA Ploidy Analysis. Zhongguo Jihua Shengyuxue Zazhi. 2009;163:292-294.  [PubMed]  [DOI]  [Cited in This Article: ]
77.  Xue X, Kim MY, Castle PE, Strickler HD. A new method to address verification bias in studies of clinical screening tests: cervical cancer screening assays as an example. J Clin Epidemiol. 2014;67:343-353.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 12]  [Cited by in F6Publishing: 12]  [Article Influence: 1.1]  [Reference Citation Analysis (0)]
78.  Ogilvie GS, van Niekerk DJ, Krajden M, Martin RE, Ehlen TG, Ceballos K, Peacock SJ, Smith LW, Kan L, Cook DA. A randomized controlled trial of Human Papillomavirus (HPV) testing for cervical cancer screening: trial design and preliminary results (HPV FOCAL Trial). BMC Cancer. 2010;10:111.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 55]  [Cited by in F6Publishing: 60]  [Article Influence: 4.3]  [Reference Citation Analysis (0)]
79.  Zhao FH, Lin MJ, Chen F, Hu SY, Zhang R, Belinson JL, Sellors JW, Franceschi S, Qiao YL, Castle PE. Performance of high-risk human papillomavirus DNA testing as a primary screen for cervical cancer: a pooled analysis of individual patient data from 17 population-based studies from China. Lancet Oncol. 2010;11:1160-1171.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 105]  [Cited by in F6Publishing: 120]  [Article Influence: 8.6]  [Reference Citation Analysis (0)]
80.  Schatzkin A, Connor RJ, Taylor PR, Bunnag B. Comparing new and old screening tests when a reference procedure cannot be performed on all screenees. Example of automated cytometry for early detection of cervical cancer. Am J Epidemiol. 1987;125:672-678.  [PubMed]  [DOI]  [Cited in This Article: ]
81.  Chock C, Irwig L, Berry G, Glasziou P. Comparing dichotomous screening tests when individuals negative on both tests are not verified. J Clin Epidemiol. 1997;50:1211-1217.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 36]  [Cited by in F6Publishing: 42]  [Article Influence: 1.6]  [Reference Citation Analysis (0)]
82.  Tian YW, Liu G, Zhou J, Gao L, Zhu CY. Application of DNA image cytometry in the cervical intraepithelial neoplasia and cervical cancer screening. Zhenduan Binglixue Zazhi. 2013;20:425-428.  [PubMed]  [DOI]  [Cited in This Article: ]
83.  Rutjes AW, Reitsma JB, Coomarasamy A, Khan KS, Bossuyt PM. Evaluation of diagnostic tests when there is no gold standard. A review of methods. Health technology assessment (Winchester, England). 2007;11:iii-ix-51.  [PubMed]  [DOI]  [Cited in This Article: ]
84.  Stock EM, Stamey JD, Sankaranarayanan R, Young DM, Muwonge R, Arbyn M. Estimation of disease prevalence, true positive rate, and false positive rate of two screening tests when disease verification is applied on only screen-positives: a hierarchical model using multi-center data. Cancer Epidemiol. 2012;36:153-160.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 7]  [Cited by in F6Publishing: 8]  [Article Influence: 0.6]  [Reference Citation Analysis (0)]
85.  de Groot JA, Bossuyt PM, Reitsma JB, Rutjes AW, Dendukuri N, Janssen KJ, Moons KG. Verification problems in diagnostic accuracy studies: consequences and solutions. BMJ. 2011;343:d4770.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 55]  [Cited by in F6Publishing: 58]  [Article Influence: 4.5]  [Reference Citation Analysis (0)]
86.  Alonzo TA. Verification bias-corrected estimators of the relative true and false positive rates of two binary screening tests. Stat Med. 2005;24:403-417.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 15]  [Cited by in F6Publishing: 15]  [Article Influence: 0.8]  [Reference Citation Analysis (0)]
87.  Fluss R, Reiser B, Faraggi D, Rotnitzky A. Estimation of the ROC curve under verification bias. Biom J. 2009;51:475-490.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 26]  [Cited by in F6Publishing: 25]  [Article Influence: 1.7]  [Reference Citation Analysis (0)]
88.  Pretorius RG, Zhang WH, Belinson JL, Huang MN, Wu LY, Zhang X, Qiao YL. Colposcopically directed biopsy, random cervical biopsy, and endocervical curettage in the diagnosis of cervical intraepithelial neoplasia II or worse. Am J Obstet Gynecol. 2004;191:430-434.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 222]  [Cited by in F6Publishing: 219]  [Article Influence: 11.0]  [Reference Citation Analysis (0)]
89.  Ferris DG, Litaker MS. Prediction of cervical histologic results using an abbreviated Reid Colposcopic Index during ALTS. Am J Obstet Gynecol. 2006;194:704-710.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 61]  [Cited by in F6Publishing: 58]  [Article Influence: 3.2]  [Reference Citation Analysis (0)]
90.  Gage JC, Hanson VW, Abbey K, Dippery S, Gardner S, Kubota J, Schiffman M, Solomon D, Jeronimo J. Number of cervical biopsies and sensitivity of colposcopy. Obstet Gynecol. 2006;108:264-272.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 235]  [Cited by in F6Publishing: 250]  [Article Influence: 13.9]  [Reference Citation Analysis (0)]
91.  Hearp ML, Locante AM, Ben-Rubin M, Dietrich R, David O. Validity of sampling error as a cause of noncorrelation. Cancer. 2007;111:275-279.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 11]  [Cited by in F6Publishing: 11]  [Article Influence: 0.6]  [Reference Citation Analysis (0)]
92.  Moss EL, Moran A, Douce G, Parkes J, Todd RW, Redman CW. Cervical cytology/histology discrepancy: a 4-year review of patient outcome. Cytopathology. 2010;21:389-394.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 8]  [Cited by in F6Publishing: 8]  [Article Influence: 0.6]  [Reference Citation Analysis (0)]
93.  Pretorius RG, Kim RJ, Belinson JL, Elson P, Qiao YL. Inflation of sensitivity of cervical cancer screening tests secondary to correlated error in colposcopy. J Low Genit Tract Dis. 2006;10:5-9.  [PubMed]  [DOI]  [Cited in This Article: ]
94.  Cagle AJ, Hu SY, Sellors JW, Bao YP, Lim JM, Li SM, Lewis K, Song Y, Ma JF, Pan QJ. Use of an expanded gold standard to estimate the accuracy of colposcopy and visual inspection with acetic acid. Int J Cancer. 2010;126:156-161.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 27]  [Cited by in F6Publishing: 28]  [Article Influence: 1.9]  [Reference Citation Analysis (0)]
95.  Pretorius RG, Bao YP, Belinson JL, Burchette RJ, Smith JS, Qiao YL. Inappropriate gold standard bias in cervical cancer screening studies. Int J Cancer. 2007;121:2218-2224.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 57]  [Cited by in F6Publishing: 61]  [Article Influence: 3.6]  [Reference Citation Analysis (0)]
96.  Phelps CE, Hutson A. Estimating diagnostic test accuracy using a “fuzzy gold standard”. Med Decis Making. 1995;15:44-57.  [PubMed]  [DOI]  [Cited in This Article: ]
97.  Pretorius RG, Belinson JL, Azizi F, Peterson PC, Belinson S. Utility of random cervical biopsy and endocervical curettage in a low-risk population. J Low Genit Tract Dis. 2012;16:333-338.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 30]  [Cited by in F6Publishing: 34]  [Article Influence: 2.8]  [Reference Citation Analysis (0)]
98.  Stoler MH, Vichnin MD, Ferenczy A, Ferris DG, Perez G, Paavonen J, Joura EA, Djursing H, Sigurdsson K, Jefferson L. The accuracy of colposcopic biopsy: analyses from the placebo arm of the Gardasil clinical trials. Int J Cancer. 2011;128:1354-1362.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 106]  [Cited by in F6Publishing: 116]  [Article Influence: 8.9]  [Reference Citation Analysis (0)]
99.  Pretorius RG, Belinson JL, Burchette RJ, Hu S, Zhang X, Qiao YL. Regardless of skill, performing more biopsies increases the sensitivity of colposcopy. J Low Genit Tract Dis. 2011;15:180-188.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 75]  [Cited by in F6Publishing: 84]  [Article Influence: 7.0]  [Reference Citation Analysis (0)]
100.   Available from: http://www.accessdata.fda.gov/cdrh_docs/pdf/P890064S009c.pdf.  [PubMed]  [DOI]  [Cited in This Article: ]
101.  Allison JE, Sakoda LC, Levin TR, Tucker JP, Tekawa IS, Cuff T, Pauly MP, Shlager L, Palitz AM, Zhao WK. Screening for colorectal neoplasms with new fecal occult blood tests: update on performance characteristics. J Natl Cancer Inst. 2007;99:1462-1470.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 249]  [Cited by in F6Publishing: 248]  [Article Influence: 14.6]  [Reference Citation Analysis (0)]
102.  Pfister C, Chautard D, Devonec M, Perrin P, Chopin D, Rischmann P, Bouchot O, Beurton D, Coulange C, Rambeaud JJ. Immunocyt test improves the diagnostic accuracy of urinary cytology: results of a French multicenter study. J Urol. 2003;169:921-924.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 90]  [Cited by in F6Publishing: 87]  [Article Influence: 4.1]  [Reference Citation Analysis (0)]
103.  Castle PE, Stoler MH, Wright TC, Sharma A, Wright TL, Behrens CM. Performance of carcinogenic human papillomavirus (HPV) testing and HPV16 or HPV18 genotyping for cervical cancer screening of women aged 25 years and older: a subanalysis of the ATHENA study. Lancet Oncol. 2011;12:880-890.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 359]  [Cited by in F6Publishing: 374]  [Article Influence: 28.8]  [Reference Citation Analysis (0)]
104.  Hayen A, Macaskill P, Irwig L, Bossuyt P. Appropriate statistical methods are required to assess diagnostic tests for replacement, add-on, and triage. J Clin Epidemiol. 2010;63:883-891.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 51]  [Cited by in F6Publishing: 61]  [Article Influence: 4.4]  [Reference Citation Analysis (0)]
105.  Sedgwick P. Screening tests and indices of performance: effects of prevalence. BMJ. 2011;343:d6483.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 2]  [Cited by in F6Publishing: 2]  [Article Influence: 0.2]  [Reference Citation Analysis (0)]
106.  Cantor SB, Cárdenas-Turanzas M, Cox DD, Atkinson EN, Nogueras-Gonzalez GM, Beck JR, Follen M, Benedet JL. Accuracy of colposcopy in the diagnostic setting compared with the screening setting. Obstet Gynecol. 2008;111:7-14.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 55]  [Cited by in F6Publishing: 58]  [Article Influence: 3.6]  [Reference Citation Analysis (0)]
107.  Cárdenas-Turanzas M, Nogueras-Gonzalez GM, Scheurer ME, Adler-Storthz K, Benedet JL, Beck JR, Follen M, Cantor SB. The performance of human papillomavirus high-risk DNA testing in the screening and diagnostic settings. Cancer Epidemiol Biomarkers Prev. 2008;17:2865-2871.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 20]  [Cited by in F6Publishing: 19]  [Article Influence: 1.2]  [Reference Citation Analysis (0)]
108.  Cárdenas-Turanzas M, Follen M, Nogueras-Gonzalez GM, Benedet JL, Beck JR, Cantor SB. The accuracy of the Papanicolaou smear in the screening and diagnostic settings. J Low Genit Tract Dis. 2008;12:269-275.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 8]  [Cited by in F6Publishing: 10]  [Article Influence: 0.7]  [Reference Citation Analysis (0)]
109.  Leeflang MM, Rutjes AW, Reitsma JB, Hooft L, Bossuyt PM. Variation of a test’s sensitivity and specificity with disease prevalence. CMAJ. 2013;185:E537-E544.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 288]  [Cited by in F6Publishing: 354]  [Article Influence: 32.2]  [Reference Citation Analysis (0)]
110.  Liu GZ, Yang Q, Guang SF. The clinical value of DNA ploidy pattern analysis in screening for cervical cancer. Anhui Weisheng Zhiye Jishu Xueyuan Xuebao. 2013;12:82-83.  [PubMed]  [DOI]  [Cited in This Article: ]
111.  Xu S, Mei JH, Han YL, Wang SS, Xiong YF. Quantitative analysis of cervical cell DNA and liquid-based cytology in cervical screening of early lesions Comparison. Guangdong Yixue. 2013;34:1387-1390.  [PubMed]  [DOI]  [Cited in This Article: ]
112.  Chen CY, Yi WJ. Liquid-based cervical cytology TBS combined quantitative analysis of cervical cell DNA in cervical cancer screening examination application value. Guangxi Yixue. 2013;35:594-596.  [PubMed]  [DOI]  [Cited in This Article: ]
113.  Zhang XH. Quantitative analysis of DNA ploidy advantage in the ultra-early stage cervical cancer screening. Jilin Yixue. 2013;34:2679-2680.  [PubMed]  [DOI]  [Cited in This Article: ]
114.  Kang YQ, OuYang XH, He D, Jiang P, Yan ZX. Super early screening of cervical cell DNA Clinical analysis of cervical lesions. Zhongguo Yixue Chuangxin. 2013;10:134-135.  [PubMed]  [DOI]  [Cited in This Article: ]
115.  Ceng Y, Zuo J, Ceng Y, Sun X, Li Z. Quantitative detection of DNA in cervical cancer screening. Guoji Jianyan Yixue Zazhi. 2013;34:199-200-219.  [PubMed]  [DOI]  [Cited in This Article: ]
116.  Wang G, Li W, Shi S, Cao Y, Tang T. Clinical Applications of DNA Ploidy Analysis in Screening Cervical Carcinoma. Zhongguo Zhongliu Linchuang. 2012;21:1639-1642.  [PubMed]  [DOI]  [Cited in This Article: ]
117.  Sun X. DNA Image Cytometry in Cervical Squamous Lesions. JASC. 2012;1:S31.  [PubMed]  [DOI]  [Cited in This Article: ]
118.  Hou A, Zhang Y, Ly X, Shao X, Wang X. A evaluation on cervical intraepithelial lesion and cervical cancer screening by DNA quantitative analysis and liquid-based monolayer cytology. Zhongguo Shiyan Zhenduanxue. 2012;4:653-655.  [PubMed]  [DOI]  [Cited in This Article: ]
119.  Wei LF, Lin HM, Qin YAF. Clinical research of cervical cell DNA quantitative analysis in opportunistic screening of cervical diseases among women in cities of Zhuang national minority regions. Huaxia Yixue. 2012;25:181-184.  [PubMed]  [DOI]  [Cited in This Article: ]
120.  Zhang YB. Quantitative analysis of DNA and liquid-based cytology in cervical lesion screening clinical observation. Guangxi Yike Daxue Xuebao. 2012;253-255.  [PubMed]  [DOI]  [Cited in This Article: ]
121.  Lan M, Cheng X, Zhu C, Liu T. Value of automatic DNA image cytometry in cervical intraepithelial neoplasia and cervical cancer screening. Zhenduan Binglixue Zazhi. 2012;19:15-18.  [PubMed]  [DOI]  [Cited in This Article: ]
122.  Jia Y. Application of quantitative analysis of DNA ploidy combined HR-HPV screening for cervical cancer detection. Hebei Lianhe Daxue Xuebao (Yixueban). 2012;14:18-19.  [PubMed]  [DOI]  [Cited in This Article: ]
123.  Hu Y. Quantitative analysis of DNA ploidy screening for cervical cancer detection technology feasibility and precancerous lesions in disease screening women. Yixue Xinxi. 2011;24:3582-3583.  [PubMed]  [DOI]  [Cited in This Article: ]
124.  Qin Z. Quantitative analysis of DNA TBS reporting methods combined clinical analysis of cervical cancer screening. Weisheng Zhiye Jiaoyu. 2011;29:144-145.  [PubMed]  [DOI]  [Cited in This Article: ]
125.  Zhong P, Li M. The clinical significance of quantitative analysis of cervical cellular DNA in cervical cancer screening. Zhongguo Yiyao Daobao. 2011;8:75-78.  [PubMed]  [DOI]  [Cited in This Article: ]
126.  Yu XR, Liu Y, Wang X, Kuang QL, Li XF, Kuang CY, Li XR, Liu Y, Wang CY, Zhu CY. Cell quantitative analysis of DNA ploidy in cervical cancer screening. Zhenduan Binglixue Zazhi. 2011;18:289-292.  [PubMed]  [DOI]  [Cited in This Article: ]
127.  Shen N, Wang Q, He Z, Liu M, Xu T, Chen DM, Zhang H, Li P, Huang L, Guan YX. The Study of Application of DNA Image Cytometry in Cervical Cancer Screening and Analysis on the Misunderstanding. Zhongguo Xiandai Yisheng. 2011;49:75-76.  [PubMed]  [DOI]  [Cited in This Article: ]
128.  Yao J. Screening for cervical cancer in women in Kaifeng City. Linchuang Yixue. 2011;31:77-78.  [PubMed]  [DOI]  [Cited in This Article: ]
129.  Chang K. Cervical TBS DNA quantitative cytology in cervical cancer screening cervical biopsy combined with the application. Zhongguo Fuyou Baojian. 2011;26:5854-5855.  [PubMed]  [DOI]  [Cited in This Article: ]
130.  Mei H, Wei X, Fang Y. Quantitative analysis of DNA for early screening of cervical cancer and precancerous lesions of clinical observation. Dangdai Yixue. 2010;16:2-3.  [PubMed]  [DOI]  [Cited in This Article: ]
131.  Huang H, Huang W, Huang Y. Application of DNA ploidy analysis of cervical screening. Jianyan Yixue Yu Linchuang. 2010;7:724-725.  [PubMed]  [DOI]  [Cited in This Article: ]
132.  Li Z, Zhang M, Li H. Improved Detection of Cervical Cancer and High Grade Neoplastic Lesions by a Combination of Conventional Cytology and DNA Automated Image Cytometer. J Cancer Ther. 2010;1:47-51.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1]  [Cited by in F6Publishing: 1]  [Article Influence: 0.1]  [Reference Citation Analysis (0)]
133.  Tong H, Shen R, Wang Z, Kan Y, Wang Y, Li F, Wang F, Yang J, Guo X. DNA ploidy cytometry testing for cervical cancer screening in China (DNACIC Trial): a prospective randomized, controlled trial. Clin Cancer Res. 2009;15:6438-6445.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 20]  [Cited by in F6Publishing: 21]  [Article Influence: 1.4]  [Reference Citation Analysis (0)]
134.  Liang Y, Huang W, Huang Y, Wei F, Qu D. The clinical significance of DNA quantitative analysis method for cervical cancer screening. Yixue Xinxi. 2009;1:219-220.  [PubMed]  [DOI]  [Cited in This Article: ]
135.  Li LJ, Wang K, Yang F, Cao Y. Clinic studies of carrying out cervical cancer general investigation using cell DNA quantitative analysis. Zhongguo Fuyou Baojian. 2008;23:2319-2320.  [PubMed]  [DOI]  [Cited in This Article: ]
136.  Ye X, Xiao J, Zhao X. DNA ploidy analysis in the application of early screening of cervical lesions. Shiyong Zhenduan yu Zhiliao Zazhi. 2006;20:370-371.  [PubMed]  [DOI]  [Cited in This Article: ]
137.  Sun X, Li Y, Che D, Yan X, Ling H, Hong Z. Cervical cancer screening by using DNA quantitative cytology. Zhenduan Binglixue Zazhi. 2005;12:12-16.  [PubMed]  [DOI]  [Cited in This Article: ]
138.  Yang Y, Zhang F, Zeng L, Rao J, Zhong M, Tu Z. Retrospective study on application of DNA image cytometer in the cervical cancer screening. Zhongguo Fuyou Baojian. 2013;28:2304-2306.  [PubMed]  [DOI]  [Cited in This Article: ]
139.  Liu J, Xie J, Chang Z, Ping J, Wang C, Wang J. Diagnostic significance of TCT combined with DNA quantitative analysis in cervical lesions screening of zhuang nationality in Guangxi province TCT. Jiepou Kexue Jinzhan. 2012;18:269-271.  [PubMed]  [DOI]  [Cited in This Article: ]
140.  Li M, Liu F, Zhong P. Quantitative analysis of DNA in cervical cancer and high-grade cervical intraepithelial neoplasia diagnosis. Guangdong Yixueyuan Xuebao. 2011;29:153-155.  [PubMed]  [DOI]  [Cited in This Article: ]
141.  Kan YJ, Shen Y, Tong H, Hua Q. Diagnostic Value of DNA Ploid Analysis for Cervical Lesions. Jilin Yixue. 2011;32:430-432.  [PubMed]  [DOI]  [Cited in This Article: ]
142.  Tu Q, Wang J, Che D, Tu H, Sun X. Determination of significance of aneuploid cells in cervical intraepithelial neoplasia grade. Linchuang Yu Shiyan Binglixue Zazhi. 2007;23:235-236.  [PubMed]  [DOI]  [Cited in This Article: ]
143.   Available from: http://www.cancer.ca/~/media/cancer.ca/CW/cancer information/cancer 101/Canadian cancer statistics/canadian-cancer-statistics-2013-EN.pdf.  [PubMed]  [DOI]  [Cited in This Article: ]
144.   Available from: http://seer.cancer.gov/faststats/selections.php?run=runit&output=3&data=5&statistic=9&race=1&sex=3&age=1&series=cancer&cancer=57.  [PubMed]  [DOI]  [Cited in This Article: ]
145.  Shi JF, Canfell K, Lew JB, Qiao YL. The burden of cervical cancer in China: synthesis of the evidence. Int J Cancer. 2012;130:641-652.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 100]  [Cited by in F6Publishing: 112]  [Article Influence: 8.6]  [Reference Citation Analysis (0)]
146.  Lei T, Mao WM, Lei TH, Dai LQ, Fang L, Chen WQ, Zhang SW. Incidence and mortality trend of cervical cancer in 11 cancer registries of China. Chin J Cancer Res. 2011;23:10-14.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 10]  [Cited by in F6Publishing: 12]  [Article Influence: 1.1]  [Reference Citation Analysis (0)]
147.  Zhao FH, Hu SY, Zhang SW, Chen WQ, Qiao YL. Cervical cancer mortality in 2004-2005 and changes during last 30 years in China. Zhonghua Yufang Yixue Zazhi. 2010;44:408-412.  [PubMed]  [DOI]  [Cited in This Article: ]
148.  Lei T, Mao WM, Yang HJ, Chen XZ, Lei TH, Wang X. Study on cancer incidence through the Cancer Registry Program in 11 Cities and Counties, China. Zhonghua Liuxingbingxue Zazhi. 2009;30:1165-1170.  [PubMed]  [DOI]  [Cited in This Article: ]
149.  Chen JG, Zhu J, Parkin DM, Zhang YH, Lu JH, Zhu YR, Chen TY. Trends in the incidence of cancer in Qidong, China, 1978-2002. Int J Cancer. 2006;119:1447-1454.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 81]  [Cited by in F6Publishing: 88]  [Article Influence: 4.9]  [Reference Citation Analysis (0)]
150.  Li J, Kang LN, Qiao YL. Review of the cervical cancer disease burden in mainland China. Asian Pac J Cancer Prev. 2011;12:1149-1153.  [PubMed]  [DOI]  [Cited in This Article: ]
151.  Arbyn M, Castellsagué X, de Sanjosé S, Bruni L, Saraiya M, Bray F, Ferlay J. Worldwide burden of cervical cancer in 2008. Ann Oncol. 2011;22:2675-2686.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 689]  [Cited by in F6Publishing: 711]  [Article Influence: 54.7]  [Reference Citation Analysis (0)]
152.  Yang L, Parkin DM, Ferlay J, Li L, Chen Y. Estimates of cancer incidence in China for 2000 and projections for 2005. Cancer Epidemiol Biomarkers Prev. 2005;14:243-250.  [PubMed]  [DOI]  [Cited in This Article: ]
153.  Zhang R, Velicer C, Chen W, Liaw KL, Wu EQ, Liu B, Cui JF, Belinson JL, Zhang X, Shen GH. Human papillomavirus genotype distribution in cervical intraepithelial neoplasia grades 1 or worse among 4215 Chinese women in a population-based study. Cancer Epidemiol. 2013;37:939-945.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 13]  [Cited by in F6Publishing: 14]  [Article Influence: 1.3]  [Reference Citation Analysis (0)]
154.  Zhao FH, Lewkowitz AK, Hu SY, Chen F, Li LY, Zhang QM, Wu RF, Li CQ, Wei LH, Xu AD. Prevalence of human papillomavirus and cervical intraepithelial neoplasia in China: a pooled analysis of 17 population-based studies. Int J Cancer. 2012;131:2929-2938.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 111]  [Cited by in F6Publishing: 147]  [Article Influence: 12.3]  [Reference Citation Analysis (0)]
155.  Wu EQ, Liu B, Cui JF, Chen W, Wang JB, Lu L, Niyazi M, Zhao C, Ren SD, Li CQ. Prevalence of type-specific human papillomavirus and pap results in Chinese women: a multi-center, population-based cross-sectional study. Cancer Causes Control. 2013;24:795-803.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 44]  [Cited by in F6Publishing: 46]  [Article Influence: 4.2]  [Reference Citation Analysis (0)]
156.  Liu SS, Chan KY, Leung RC, Chan KK, Tam KF, Luk MH, Lo SS, Fong DY, Cheung AN, Lin ZQ. Prevalence and risk factors of Human Papillomavirus (HPV) infection in southern Chinese women-a population-based study. PLoS One. 2011;6:e19244.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 38]  [Cited by in F6Publishing: 53]  [Article Influence: 4.1]  [Reference Citation Analysis (0)]
157.  Meng RL, Ma WJ, Xu YJ, Sond XL, Nie SP, Xu HF, Xu XJ, Cai QM, Xia Le. Screening rates of cervical cancer, breast cancer and their influential factors in Guangdong Province. Huanan Yufang Yixue. 2010;36:8-14.  [PubMed]  [DOI]  [Cited in This Article: ]
158.  Health DO. Topical Health Report No. 4 Prevention and Screening of Cervical Cancer [Internet].  Available from: http://www.chp.gov.hk/files/pdf/grp-thr-report4-en-20041209.pdf.  [PubMed]  [DOI]  [Cited in This Article: ]
159.   Available from: http://www.cervicalscreening.gov.hk/english/sr/sr_statistics_cc.html.  [PubMed]  [DOI]  [Cited in This Article: ]
160.  Katki HA, Kinney WK, Fetterman B, Lorey T, Poitras NE, Cheung L, Demuth F, Schiffman M, Wacholder S, Castle PE. Cervical cancer risk for women undergoing concurrent testing for human papillomavirus and cervical cytology: a population-based study in routine clinical practice. Lancet Oncol. 2011;12:663-672.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 440]  [Cited by in F6Publishing: 400]  [Article Influence: 30.8]  [Reference Citation Analysis (0)]
161.  Pan QJ, Hu SY, Zhang X, Ci PW, Zhang WH, Guo HQ, Cao J, Zhao FH, Lytwyn A, Qiao YL. Pooled analysis of the performance of liquid-based cytology in population-based cervical cancer screening studies in China. Cancer Cytopathol. 2013;121:473-482.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 30]  [Cited by in F6Publishing: 34]  [Article Influence: 3.1]  [Reference Citation Analysis (0)]
162.   Available from: http://www.med.motic.com/Page.aspx?PageCategoryId=7&PageContentId=4&LangCode=zhcn.  [PubMed]  [DOI]  [Cited in This Article: ]
163.   Available from: http://landingmed.com/cn/.  [PubMed]  [DOI]  [Cited in This Article: ]
164.  Garner DM, Guillaud MD, MacAulay CE. DNA ploidy cytometry testing for cervical cancer screening in China-letter. Clin Cancer Res. 2010;16:3517; author reply 3517-3519.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 4]  [Cited by in F6Publishing: 4]  [Article Influence: 0.3]  [Reference Citation Analysis (0)]
165.  Tong H, Shen R, Kan Y, Wang Z, Wang Y, Li F. DNA Cytometry Testing for Cervical Cancer Screening-Response. Clin Cancer Res. 2010;16:3517-3519.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1]  [Cited by in F6Publishing: 1]  [Article Influence: 0.1]  [Reference Citation Analysis (0)]
166.   Available from: http://landingmed.com/wp-content/uploads/2013/04/cytoptic.pdf.  [PubMed]  [DOI]  [Cited in This Article: ]
167.   Available from: http://www.thinprep.com/hcp/imaging_system/benefits_dual_review.html.  [PubMed]  [DOI]  [Cited in This Article: ]
168.  Wright TC, Massad LS, Dunton CJ, Spitzer M, Wilkinson EJ, Solomon D. 2006 consensus guidelines for the management of women with abnormal cervical cancer screening tests. Am J Obstet Gynecol. 2007;197:346-355.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 574]  [Cited by in F6Publishing: 478]  [Article Influence: 28.1]  [Reference Citation Analysis (0)]
169.  Zhang DL, Yang Y, Zhou LM. Significance of DNA ploidy analysis in diagnosis of ASCUS. Zhonghua Fuchanke Zazhi. 2012;47:259-262.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in F6Publishing: 1]  [Reference Citation Analysis (0)]
170.  Yang Y, Wu X, Zhang D. Significance of DNA Quantitative Cytology and High Risk HPV-DNA Test in ASCUS Triage Diagnosis. Zhongliu Fangzhi Yanjiu. 2013;40:671-674.  [PubMed]  [DOI]  [Cited in This Article: ]
171.  Lorenzato M, Caudroy S, Nou JM, Dalstein V, Joseph K, Bellefqih S, Durlach A, Thil C, Dez F, Bouttens D. Contribution of DNA ploidy image cytometry to the management of ASC cervical lesions. Cancer. 2008;114:263-269.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 18]  [Cited by in F6Publishing: 19]  [Article Influence: 1.2]  [Reference Citation Analysis (0)]
172.  Chen G, Xu Y, Li L, Yang L. Clinical application values of DNA ploidy analysis of cervical cells and Thinprep cytologic test combined with HPV detection for diagnosis of cervical precancerous lesion. Zhongguo Fuyou Baojian. 2013;28:1179-1181.  [PubMed]  [DOI]  [Cited in This Article: ]
173.  Jiao H, Ye Y, Zhang J, Li Y, Xia C. Functions of DNA ploidy analysis combined with high-risk human papillomavirus detection in screening of cervical cancer. Zhongguo Fuyou Baojian. 2012;25:3399-3401.  [PubMed]  [DOI]  [Cited in This Article: ]
174.  Feng S, Feng S. Investigation and analysis on HPV infection status of 1 026 women of child-bearing age. Zhongguo Fuyou Baojian. 2011;26:1374-1377.  [PubMed]  [DOI]  [Cited in This Article: ]
175.   Available from: http://english.peopledaily.com.cn/90782/90880/8437954.html.  [PubMed]  [DOI]  [Cited in This Article: ]
176.  Li ZF, Wang SM, Shi JF, Zhao FH, Ma JF, Qiao YL, Feng XX. Combined screening of cervical cancer, breast cancer and reproductive tract infections in rural China. Asian Pac J Cancer Prev. 2012;13:3529-3533.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 5]  [Cited by in F6Publishing: 6]  [Article Influence: 0.5]  [Reference Citation Analysis (0)]
177.  Yu ZX. Preliminary Report on Screening for Cervical Cancer and Breast Cancer in Shaanxi Province. Zhongguo Zhongliu. 2010;19:377-379.  [PubMed]  [DOI]  [Cited in This Article: ]
178.  Dowling EC, Klabunde C, Patnick J, Ballard-Barbash R. Breast and cervical cancer screening programme implementation in 16 countries. J Med Screen. 2010;17:139-146.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 76]  [Cited by in F6Publishing: 77]  [Article Influence: 5.5]  [Reference Citation Analysis (0)]
179.  Leung GM, Lam TH, Thach TQ, Woo PP, Wong IO, Hedley AJ. Mass breast screening is highly inefficient. Hong Kong Med J. 2007;13:332; author reply 332-333.  [PubMed]  [DOI]  [Cited in This Article: ]
180.  Wong IO, Tsang JW, Cowling BJ, Leung GM. Optimizing resource allocation for breast cancer prevention and care among Hong Kong Chinese women. Cancer. 2012;118:4394-4403.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 7]  [Cited by in F6Publishing: 8]  [Article Influence: 0.7]  [Reference Citation Analysis (0)]
181.   Available from: http://www.chp.gov.hk/files/pdf/grp-cancer-cewg-en-20050121.pdf.  [PubMed]  [DOI]  [Cited in This Article: ]
182.  Cancer Experts Working Group. Cancer Expert Working Group On Cancer Prevention and Screening Recommendations on Breast Cancer Screening [Internet].  Available from: http://www.chp.gov.hk/files/pdf/recommendations_on_breast_cancer_screening_2010.pdf.  [PubMed]  [DOI]  [Cited in This Article: ]
183.  Turic B. President and CEO, Perceptronix Medical Imaging Corp., personal communication .  [PubMed]  [DOI]  [Cited in This Article: ]
184.   Available from: http://www.cytology.org.hk/Download/Final Draft2.pdf.  [PubMed]  [DOI]  [Cited in This Article: ]
185.  Young NA, Moriarty AT, Walsh MK, Wang E, Wilbur DC. The potential for failure in gynecologic regulatory proficiency testing with current slide validation criteria: results from the College of American Pathologists Interlaboratory Comparison in Gynecologic Cytology Program. Arch Pathol Lab Med. 2006;130:1114-1118.  [PubMed]  [DOI]  [Cited in This Article: ]
186.  Anand S, Fan VY, Zhang J, Zhang L, Ke Y, Dong Z, Chen LC. China’s human resources for health: quantity, quality, and distribution. Lancet. 2008;372:1774-1781.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 210]  [Cited by in F6Publishing: 228]  [Article Influence: 14.3]  [Reference Citation Analysis (0)]
187.  Katki HA, Schiffman M, Castle PE, Fetterman B, Poitras NE, Lorey T, Cheung LC, Raine-Bennett T, Gage JC, Kinney WK. Benchmarking CIN 3+ risk as the basis for incorporating HPV and Pap cotesting into cervical screening and management guidelines. J Low Genit Tract Dis. 2013;17:S28-S35.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 133]  [Cited by in F6Publishing: 147]  [Article Influence: 13.4]  [Reference Citation Analysis (0)]
188.  Katki HA, Schiffman M, Castle PE, Fetterman B, Poitras NE, Lorey T, Cheung LC, Raine-Bennett T, Gage JC, Kinney WK. Five-year risk of CIN 3+ to guide the management of women aged 21 to 24 years. J Low Genit Tract Dis. 2013;17:S64-S68.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 22]  [Cited by in F6Publishing: 24]  [Article Influence: 2.2]  [Reference Citation Analysis (0)]
189.  Katki HA, Schiffman M, Castle PE, Fetterman B, Poitras NE, Lorey T, Cheung LC, Raine-Bennett T, Gage JC, Kinney WK. Five-year risk of recurrence after treatment of CIN 2, CIN 3, or AIS: performance of HPV and Pap cotesting in posttreatment management. J Low Genit Tract Dis. 2013;17:S78-S84.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 59]  [Cited by in F6Publishing: 67]  [Article Influence: 6.1]  [Reference Citation Analysis (0)]
190.  Katki HA, Schiffman M, Castle PE, Fetterman B, Poitras NE, Lorey T, Cheung LC, Raine-Bennett T, Gage JC, Kinney WK. Five-year risks of CIN 2+ and CIN 3+ among women with HPV-positive and HPV-negative LSIL Pap results. J Low Genit Tract Dis. 2013;17:S43-S49.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 36]  [Cited by in F6Publishing: 37]  [Article Influence: 3.4]  [Reference Citation Analysis (0)]
191.  Katki HA, Schiffman M, Castle PE, Fetterman B, Poitras NE, Lorey T, Cheung LC, Raine-Bennett T, Gage JC, Kinney WK. Five-year risks of CIN 3+ and cervical cancer among women who test Pap-negative but are HPV-positive. J Low Genit Tract Dis. 2013;17:S56-S63.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 59]  [Cited by in F6Publishing: 67]  [Article Influence: 6.1]  [Reference Citation Analysis (0)]
192.  Katki HA, Schiffman M, Castle PE, Fetterman B, Poitras NE, Lorey T, Cheung LC, Raine-Bennett T, Gage JC, Kinney WK. Five-year risks of CIN 3+ and cervical cancer among women with HPV-positive and HPV-negative high-grade Pap results. J Low Genit Tract Dis. 2013;17:S50-S55.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 43]  [Cited by in F6Publishing: 44]  [Article Influence: 4.0]  [Reference Citation Analysis (0)]
193.  Katki HA, Schiffman M, Castle PE, Fetterman B, Poitras NE, Lorey T, Cheung LC, Raine-Bennett T, Gage JC, Kinney WK. Five-year risks of CIN 3+ and cervical cancer among women with HPV testing of ASC-US Pap results. J Low Genit Tract Dis. 2013;17:S36-S42.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 68]  [Cited by in F6Publishing: 75]  [Article Influence: 6.8]  [Reference Citation Analysis (0)]
194.  Katki HA, Gage JC, Schiffman M, Castle PE, Fetterman B, Poitras NE, Lorey T, Cheung LC, Raine-Bennett T, Kinney WK. Follow-up testing after colposcopy: five-year risk of CIN 2+ after a colposcopic diagnosis of CIN 1 or less. J Low Genit Tract Dis. 2013;17:S69-S77.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 45]  [Cited by in F6Publishing: 50]  [Article Influence: 4.5]  [Reference Citation Analysis (0)]
195.   Available from: http://www.evaluategroup.com/Universal/View.aspx?type=Story&id=420248.  [PubMed]  [DOI]  [Cited in This Article: ]
196.   Available from: http://www.gavialliance.org/support/nvs/human-papillomavirus-vaccine-support/.  [PubMed]  [DOI]  [Cited in This Article: ]
197.  Rijkaart DC, Berkhof J, van Kemenade FJ, Coupe VM, Hesselink AT, Rozendaal L, Heideman DA, Verheijen RH, Bulk S, Verweij WM. Evaluation of 14 triage strategies for HPV DNA-positive women in population-based cervical screening. Int J Cancer. 2012;130:602-610.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 151]  [Cited by in F6Publishing: 162]  [Article Influence: 12.5]  [Reference Citation Analysis (0)]
198.  Bray F, Carstensen B, Møller H, Zappa M, Zakelj MP, Lawrence G, Hakama M, Weiderpass E. Incidence trends of adenocarcinoma of the cervix in 13 European countries. Cancer Epidemiol Biomarkers Prev. 2005;14:2191-2199.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 239]  [Cited by in F6Publishing: 258]  [Article Influence: 13.6]  [Reference Citation Analysis (0)]
199.  Adegoke O, Kulasingam S, Virnig B. Cervical cancer trends in the United States: a 35-year population-based analysis. J Womens Health (Larchmt). 2012;21:1031-1037.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 237]  [Cited by in F6Publishing: 227]  [Article Influence: 18.9]  [Reference Citation Analysis (0)]
200.  Lönnberg S, Nieminen P, Luostarinen T, Anttila A. Mortality audit of the Finnish cervical cancer screening program. Int J Cancer. 2013;132:2134-2140.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 41]  [Cited by in F6Publishing: 42]  [Article Influence: 3.5]  [Reference Citation Analysis (0)]
201.  Ronco G, Dillner J, Elfström KM, Tunesi S, Snijders PJ, Arbyn M, Kitchener H, Segnan N, Gilham C, Giorgi-Rossi P. Efficacy of HPV-based screening for prevention of invasive cervical cancer: follow-up of four European randomised controlled trials. Lancet. 2014;383:524-532.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1041]  [Cited by in F6Publishing: 1131]  [Article Influence: 113.1]  [Reference Citation Analysis (0)]
202.  Kinney W, Fetterman B, Cox JT, Lorey T, Flanagan T, Castle PE. Characteristics of 44 cervical cancers diagnosed following Pap-negative, high risk HPV-positive screening in routine clinical practice. Gynecol Oncol. 2011;121:309-313.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 34]  [Cited by in F6Publishing: 31]  [Article Influence: 2.4]  [Reference Citation Analysis (0)]
203.  Guillaud M, Benedet JL, Follen M, Crain BT, MacAulay C. Scan-and-treat methodology using Azure A fast stain as a cost-effective cervical cancer screening alternative to visual inspection with acetic acid. Gynecol Oncol. 2007;107:S256-S259.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 2]  [Cited by in F6Publishing: 2]  [Article Influence: 0.1]  [Reference Citation Analysis (0)]
204.  Jiao H, Ye Y, Xia C. Application and comparison of two kinds of staining methods for measurement of nuclear DNA contents. Zhongguo Tishixue yu Tuxiang Fenxi. 2007;12:180-184.  [PubMed]  [DOI]  [Cited in This Article: ]
205.  Biesterfeld S, Beckers S, Del Carmen Villa Cadenas M, Schramm M. Feulgen staining remains the gold standard for precise DNA image cytometry. Anticancer Res. 2011;31:53-58.  [PubMed]  [DOI]  [Cited in This Article: ]