Browsing by Subject "Observer Variation"
Now showing 1 - 20 of 29
Results Per Page
Sort Options
Item Open Access An automated method for comparing motion artifacts in cine four-dimensional computed tomography images.(Journal of applied clinical medical physics, 2012-11-08) Cui, Guoqiang; Jew, Brian; Hong, Julian C; Johnston, Eric W; Loo, Billy W; Maxim, Peter GThe aim of this study is to develop an automated method to objectively compare motion artifacts in two four-dimensional computed tomography (4D CT) image sets, and identify the one that would appear to human observers with fewer or smaller artifacts. Our proposed method is based on the difference of the normalized correlation coefficients between edge slices at couch transitions, which we hypothesize may be a suitable metric to identify motion artifacts. We evaluated our method using ten pairs of 4D CT image sets that showed subtle differences in artifacts between images in a pair, which were identifiable by human observers. One set of 4D CT images was sorted using breathing traces in which our clinically implemented 4D CT sorting software miscalculated the respiratory phase, which expectedly led to artifacts in the images. The other set of images consisted of the same images; however, these were sorted using the same breathing traces but with corrected phases. Next we calculated the normalized correlation coefficients between edge slices at all couch transitions for all respiratory phases in both image sets to evaluate for motion artifacts. For nine image set pairs, our method identified the 4D CT sets sorted using the breathing traces with the corrected respiratory phase to result in images with fewer or smaller artifacts, whereas for one image pair, no difference was noted. Two observers independently assessed the accuracy of our method. Both observers identified 9 image sets that were sorted using the breathing traces with corrected respiratory phase as having fewer or smaller artifacts. In summary, using the 4D CT data of ten pairs of 4D CT image sets, we have demonstrated proof of principle that our method is able to replicate the results of two human observers in identifying the image set with fewer or smaller artifacts.Item Open Access Automatic segmentation of seven retinal layers in SDOCT images congruent with expert manual segmentation.(Opt Express, 2010-08-30) Chiu, SJ; Li, XT; Nicholas, P; Toth, CA; Izatt, JA; Farsiu, SSegmentation of anatomical and pathological structures in ophthalmic images is crucial for the diagnosis and study of ocular diseases. However, manual segmentation is often a time-consuming and subjective process. This paper presents an automatic approach for segmenting retinal layers in Spectral Domain Optical Coherence Tomography images using graph theory and dynamic programming. Results show that this method accurately segments eight retinal layer boundaries in normal adult eyes more closely to an expert grader as compared to a second expert grader.Item Open Access Development and implementation of a proficiency testing program for Luminex bead-based cytokine assays.(Journal of Immunological Methods, 2014-07) Lynch, Heather E; Sanchez, Ana M; D'Souza, M Patricia; Rountree, Wes; Denny, Thomas N; Kalos, Michael; Sempowski, Gregory DLuminex bead array assays are widely used for rapid biomarker quantification due to the ability to measure up to 100 unique analytes in a single well of a 96-well plate. There has been, however, no comprehensive analysis of variables impacting assay performance, nor development of a standardized proficiency testing program for laboratories performing these assays. To meet this need, the NIH/NIAID and the Cancer Immunotherapy Consortium of the Cancer Research Institute collaborated to develop and implement a Luminex assay proficiency testing program as part of the NIH/NIAID-sponsored External Quality Assurance Program Oversight Laboratory (EQAPOL) at Duke University. The program currently monitors 25 domestic and international sites with two external proficiency panels per year. Each panel includes a de-identified commercial Luminex assay kit with standards to quantify human IFNγ, TNFα, IL-6, IL-10 and IL-2, and a series of recombinant cytokine-spiked human serum samples. All aspects of panel development, testing and shipping are performed under GCLP by EQAPOL support teams. Following development testing, a comprehensive site proficiency scoring system comprised of timeliness, protocol adherence, accuracy and precision was implemented. The overall mean proficiency score across three rounds of testing has remained stable (EP3: 76%, EP4: 75%, EP5: 77%); however, a more detailed analysis of site reported results indicates a significant improvement of intra- (within) and inter- (between) site variation, suggesting that training and remediation for poor performing sites may be having a positive impact on proficiency. Through continued proficiency testing, identification of variables affecting Luminex assay outcomes will strengthen efforts to bring standardization to the field.Item Open Access Development of a contemporary globally diverse HIV viral panel by the EQAPOL program.(J Immunol Methods, 2014-07) Sanchez, Ana M; DeMarco, C Todd; Hora, Bhavna; Keinonen, Sarah; Chen, Yue; Brinkley, Christie; Stone, Mars; Tobler, Leslie; Keating, Sheila; Schito, Marco; Busch, Michael P; Gao, Feng; Denny, Thomas NThe significant diversity among HIV-1 variants poses serious challenges for vaccine development and for developing sensitive assays for screening, surveillance, diagnosis, and clinical management. Recognizing a need to develop a panel of HIV representing the current genetic and geographic diversity NIH/NIAID contracted the External Quality Assurance Program Oversight Laboratory (EQAPOL) to isolate, characterize and establish panels of HIV-1 strains representing global diverse subtypes and circulating recombinant forms (CRFs), and to make them available to the research community. HIV-positive plasma specimens and previously established isolates were collected through a variety of collaborations with a preference for samples from acutely/recently infected persons. Source specimens were cultured to high-titer/high-volume using well-characterized cryopreserved PBMCs from National y donors. Panel samples were stored as neat culture supernatant or diluted into defibrinated plasma. Characterization for the final expanded virus stocks included viral load, p24 antigen, infectivity (TCID), sterility, coreceptor usage, and near full-length genome sequencing. Viruses are made available to approved, interested laboratories using an online ordering application. The current EQAPOL Viral Diversity panel includes 100 viral specimens representing 6 subtypes (A, B, C, D, F, and G), 2 sub-subtypes (F1 and F2), 7 CRFs (01, 02, 04, 14, 22, 24, and 47), 19 URFs and 3 group O viruses from 22 countries. The EQAPOL Viral Diversity panel is an invaluable collection of well-characterized reagents that are available to the scientific community, including researchers, epidemiologists, and commercial manufacturers of diagnostics and pharmaceuticals to support HIV research, as well as diagnostic and vaccine development.Item Open Access Establishment and maintenance of a PBMC repository for functional cellular studies in support of clinical vaccine trials.(J Immunol Methods, 2014-07) Sambor, Anna; Garcia, Ambrosia; Berrong, Mark; Pickeral, Joy; Brown, Sara; Rountree, Wes; Sanchez, Ana; Pollara, Justin; Frahm, Nicole; Keinonen, Sarah; Kijak, Gustavo H; Roederer, Mario; Levine, Gail; D'Souza, M Patricia; Jaimes, Maria; Koup, Richard; Denny, Thomas; Cox, Josephine; Ferrari, GuidoA large repository of cryopreserved peripheral blood mononuclear cells (PBMCs) samples was created to provide laboratories testing the specimens from human immunodeficiency virus-1 (HIV-1) vaccine clinical trials the material for assay development, optimization, and validation. One hundred thirty-one PBMC samples were collected using leukapheresis procedure between 2007 and 2013 by the Comprehensive T cell Vaccine Immune Monitoring Consortium core repository. The donors included 83 human immunodeficiency virus-1 (HIV-1) seronegative and 32 HIV-1 seropositive subjects. The samples were extensively characterized for the ability of T cell subsets to respond to recall viral antigens including cytomegalovirus, Epstein-Barr virus, influenza virus, and HIV-1 using Interferon-gamma (IFN-γ) enzyme linked immunospot (ELISpot) and IFN-γ/interleukin 2 (IL-2) intracellular cytokine staining (ICS) assays. A subset of samples was evaluated over time to determine the integrity of the cryopreserved samples in relation to recovery, viability, and functionality. The principal results of our study demonstrate that viable and functional cells were consistently recovered from the cryopreserved samples. Therefore, we determined that this repository of large size cryopreserved cellular samples constitutes a unique resource for laboratories that are involved in optimization and validation of assays to evaluate T, B, and NK cellular functions in the context of clinical trials.Item Open Access Impact of beta-blockade premedication on image quality of ECG-gated thoracic aorta CT angiography.(Acta radiologica (Stockholm, Sweden : 1987), 2014-12) Entezari, Pegah; Collins, Jeremy; Chalian, Hamid; Tore, Huseyin Gurkan; Carr, James; Yaghmai, VahidBACKGROUND: Thoracic aortic aneurysm is one of the most common aorta pathologies worldwide, which is commonly evaluated by computed tomography angiography (CTA). One of the routine methods to improve the image quality of CTA is heart rate reduction prior to study by beta-blockade administration. PURPOSE: To assess the effect of beta-blockade on image quality of the ascending aorta in electrocardiography (ECG)-gated dual-source CTA (DSCTA) images. MATERIAL AND METHODS: In this retrospective study, ECG-gated thoracic aorta CTA images of 40 patients without beta-blocker administration were compared with ECG-gated images of 40 patients with beta-blockade. Images of the aorta were analyzed objectively and subjectively at three levels: sinus of Valsalva (sinus), sinotubular junction (STJ), and mid ascending aorta (MAA). Quantitative sharpness index (SI) and signal-to-noise ratio (SNR) were calculated and two radiologists evaluated the image quality using a 3-point scale. RESULTS: Mean heart rate in beta-blocker and non-beta-blocker groups was 61.7 beats per minute (bpm) (range, 58.1-63.9 bpm) and 72.9 bpm (range, 69.3-84.1 bpm), respectively (P < 0.05). Aorta wall SI, SNR, and subjective grading were comparable between the two groups at all three levels (P > 0.05). CONCLUSION: Beta-blocker premedication may not be necessary for imaging of ascending aorta with ECG-gated DSCTA.Item Open Access Implementing a Continuous Quality Improvement Program in a High-Volume Clinical Echocardiography Laboratory: Improving Care for Patients With Aortic Stenosis.(Circ Cardiovasc Imaging, 2016-03) Samad, Zainab; Minter, Stephanie; Armour, Alicia; Tinnemore, Amanda; Sivak, Joseph A; Sedberry, Brenda; Strub, Karen; Horan, Seanna M; Harrison, J Kevin; Kisslo, Joseph; Douglas, Pamela S; Velazquez, Eric JBACKGROUND: The management of aortic stenosis rests on accurate echocardiographic diagnosis. Hence, it was chosen as a test case to examine the utility of continuous quality improvement (CQI) approaches to increase echocardiographic data accuracy and reliability. A novel, multistep CQI program was designed and prospectively used to investigate whether it could minimize the difference in aortic valve mean gradients reported by echocardiography when compared with cardiac catheterization. METHODS AND RESULTS: The Duke Echo Laboratory compiled a multidisciplinary CQI team including 4 senior sonographers and MD faculty to develop a mapped CQI process that incorporated Intersocietal Accreditation Commission standards. Quarterly, the CQI team reviewed all moderate- or greater-severity aortic stenosis echocardiography studies with concomitant catheterization data, and deidentified individual and group results were shared at meetings attended by cardiologists and sonographers. After review of 2011 data, the CQI team proposed specific amendments implemented over 2012: the use of nontraditional imaging and Doppler windows as well as evaluation of aortic gradients by a second sonographer. The primary outcome measure was agreement between catheterization- and echocardiography-derived mean gradients calculated by using the coverage probability index with a prespecified acceptable echocardiography-catheterization difference of <10 mm Hg in mean gradient. Between January 2011 and January 2014, 2093 echocardiograms reported moderate or greater aortic stenosis. Among cases with available catheterization data pre- and post-CQI, the coverage probability index increased from 54% to 70% (P=0.03; 98 cases, year 2011; 70 cases, year 2013). The proportion of patients referred for invasive valve hemodynamics decreased from 47% pre-CQI to 19% post-CQI (P<0.001). CONCLUSIONS: A laboratory practice pattern that was amenable to reform was identified, and a multistep modification was designed and implemented that produced clinically valuable performance improvements. The new protocol improved aortic stenosis mean gradient agreement between echocardiography and catheterization and was associated with a measurable decrease in referrals of patients for invasive studies.Item Open Access Inter- and Intra-rater Reliability of the Hart-ISSG Proximal Junctional Failure Severity Scale.(Spine, 2018-04) Hart, Robert A; Rastegar, Farbod; Contag, Alec; Kane, Marie; Daniels, Alan; Klineberg, Eric; Eastlack, Robert; Smith, Justin S; Hostin, Richard; Hamilton, D Kojo; Gum, Jeffrey L; Burton, Douglas C; Sheer, Justin K; Ames, Christopher; Schwab, Frank; Lafage, Virginie; Bess, Shay; Shaffrey, Christopher; Kebaish, Khaled; and the International Spine Study GroupStudy design
Reliability/external validation study.Objective
Investigate inter- and intrarater reliability of the Hart-International Spine Study Group (ISSG) Proximal Junctional Failure Severity Scale (PJFSS) and its correlation with operative revision in patients with proximal junctional failure (PJF).Summary of background data
The Hart-ISSG PJFSS is a validated classification system for PJF. Reliability of the PJFSS has not been assessed.Methods
Sixteen detailed clinical scenarios were assessed using the ISSG PJFSS classification in six categories: neurologic status, axial pain, instrumentation issue, proximal kyphotic angle, level of upper instrumented vertebrae (UIV), and severity of UIV/UIV+1 fracture. Eleven spine surgeons evaluated each case in all six categories during two different assessments, and provided recommendations regarding operative revision or observation for each case. Inter- and intrarater reliability were calculated based on intraclass correlation coefficients.Results
All intraclass correlation coefficients demonstrated "almost perfect"' (0.817-0.988) inter-rater agreement for both assessments, except UIV/UIV+1 fracture severity during the second assessment, which demonstrated "substantial" agreement' (0.692). Five of six categories had "almost perfect" mean intrarater reliability (0.805-0.981), while "instrumentation issue" demonstrated "substantial" mean agreement (0.757). Inter-rater reliability for recommendation of surgical intervention was "almost perfect" during both assessments (0.911 and 0.922, respectively). Mean PJFSS scores between the two assessments were significantly higher for cases recommended for operative revision (8.43 ± 0.90) versus cases recommended for observation (P < 0.0001).Conclusion
The ISSG PJFSS is a reliable and repeatable classification system for assessing patients with PJF. Higher PJFSS scales correlate with recommendation for operative revision, extending prior external validation of the PJFSS.Level of evidence
3.Item Open Access Intraobserver variability in fibroid size measurements: estimated effects on assessing fibroid growth.(Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine, 2014-07) Moshesh, Malana; Peddada, Shyamal D; Cooper, Tracy; Baird, DonnaTo evaluate intraobserver variability of fibroid sonographic measurements and apply this factor to fibroid growth assessment.Study participants were African American women aged 23 to 34 years who had never had a diagnosis of uterine fibroids. All participants underwent transvaginal sonography to screen for the presence of previously undiagnosed fibroids (≥0.5 cm in diameter). The diameters of up to 6 fibroids were measured in 3 perpendicular planes at 3 separate times during the examinations by experienced sonographers. Intraobserver variability as measured by the coefficient of variation (CV) for fibroid diameter and volume was calculated for each fibroid, and factors associated with the CV were assessed by regression models. The impact of variability on growth assessment was determined.Ninety-six of 300 women screened were found to have at least 1 fibroid, yielding a total of 174 fibroids for this analysis. The mean CV for the 3 measurements of fibroid maximum diameter was 5.9%. The mean CV for fibroid volume was 12.7%. Fibroid size contributed significantly to intraobserver variability (P = .04), with greater variability for smaller fibroids. Fibroid type (submucosal, intramural, or subserosal) was not important. Fibroids from the same woman tended to have similar measurement variability when assessed for volume but not for maximum diameter. Calculations showed that when following up fibroids, as much as a 20% increase in diameter could be due to measurement error, not "true growth."A small fibroid must have a greater change in size than a large fibroid to conclude that it is growing, but even for small fibroids an increase in diameter of greater than 20% is likely to indicate true growth, not measurement variability.Item Open Access Introduction to a Special Issue of the Journal of Immunological Methods: Building global resource programs to support HIV/AIDS clinical trial studies.(J Immunol Methods, 2014-07) Sanchez, Ana M; Denny, Thomas N; O'Gorman, MauriceThis Special Issue of the Journal of Immunological Methods includes 16 manuscripts describing quality assurance activities related to virologic and immunologic monitoring of six global laboratory resource programs that support international HIV/AIDS clinical trial studies: Collaboration for AIDS Vaccine Discovery (CAVD); Center for HIV/AIDS Vaccine Immunology (CHAVI); External Quality Assurance Program Oversight Laboratory (EQAPOL); HIV Vaccine Trial Network (HVTN); International AIDS Vaccine Initiative (IAVI); and Immunology Quality Assessment (IQA). The reports from these programs address the many components required to develop comprehensive quality control activities and subsequent quality assurance programs for immune monitoring in global clinical trials including: all aspects of processing, storing, and quality assessment of PBMC preparations used ubiquitously in HIV clinical trials, the development and optimization of assays for CD8 HIV responses and HIV neutralization, a comprehensive global HIV virus repository, and reports on the development and execution of novel external proficiency testing programs for immunophenotyping, intracellular cytokine staining, ELISPOT and luminex based cytokine measurements. In addition, there are articles describing the implementation of Good Clinical Laboratory Practices (GCLP) in a large quality assurance laboratory, the development of statistical methods specific for external proficiency testing assessment, a discussion on the ability to set objective thresholds for measuring rare events by flow cytometry, and finally, a manuscript which addresses a framework for the structured reporting of T cell immune function based assays. It is anticipated that this series of manuscripts covering a wide range of quality assurance activities associated with the conduct of global clinical trials will provide a resource for individuals and programs involved in improving the harmonization, standardization, accuracy, and sensitivity of virologic and immunologic testing.Item Open Access Leukopak PBMC sample processing for preparing quality control material to support proficiency testing programs.(Journal of Immunological Methods, 2014-07) Garcia, Ambrosia; Keinonen, Sarah; Sanchez, Ana M; Ferrari, Guido; Denny, Thomas N; Moody, M AnthonyExternal proficiency testing programs designed to evaluate the performance of end-point laboratories involved in vaccine and therapeutic clinical trials form an important part of clinical trial quality assurance. Good clinical laboratory practice (GCLP) guidelines recommend both assay validation and proficiency testing for assays being used in clinical trials, and such testing is facilitated by the availability of large numbers of well-characterized test samples. These samples can be distributed to laboratories participating in these programs and allow monitoring of laboratory performance over time and among participating sites when results are obtained with samples derived from a large master set. The leukapheresis procedure provides an ideal way to collect samples from participants that can meet the required number of cells to support these activities. The collection and processing of leukapheresis samples require tight coordination between the clinical and laboratory teams to collect, process, and cryopreserve large number of samples within the established ideal time of ≤8 hours. Here, we describe our experience with a leukapheresis cryopreseration program that has been able to preserve the functionality of cellular subsets and that provides the sample numbers necessary to run an external proficiency testing program.Item Open Access Objective assessment of vaginal surgical skills.(Am J Obstet Gynecol, 2010-07) Chen, Chi Chiung Grace; Korn, Abner; Klingele, Christopher; Barber, Matthew D; Paraiso, Marie Fidela R; Walters, Mark D; Jelovsek, J EricOBJECTIVE: To develop and validate an instrument to assess surgical skills during vaginal surgery. STUDY DESIGN: Trainees from 2 institutions were directly assessed in the operating room by supervising surgeons while performing a vaginal hysterectomy using the new Vaginal Surgical Skills Index, global rating scale, and visual analogue scale. Trainees were assessed again by the same surgeons 4 weeks after the live surgery and by a blinded outside reviewer using a videotape of the case. Internal consistency, interrater and intrarater reliability, and construct validity were evaluated. RESULTS: Two hundred twelve evaluations were analyzed on 76 surgeries from 27 trainees. There was good internal consistency, interrater, and intrarater reliability. Vaginal Surgical Skills Index scores correlated with global rating score and visual analog scale scores. Increasing Vaginal Surgical Skills Index scores significantly correlated with year of training and surgical volume with an estimated increase in score of 0.3 per hysterectomy performed. CONCLUSION: The Vaginal Surgical Skills Index is a feasible, reliable, and valid instrument to assess vaginal surgical skills.Item Open Access On the correlation between second order texture features and human observer detection performance in digital images.(Scientific reports, 2020-08) Nisbett, William H; Kavuri, Amar; Das, MiniImage texture, the relative spatial arrangement of intensity values in an image, encodes valuable information about the scene. As it stands, much of this potential information remains untapped. Understanding how to decipher textural details would afford another method of extracting knowledge of the physical world from images. In this work, we attempt to bridge the gap in research between quantitative texture analysis and the visual perception of textures. The impact of changes in image texture on human observer's ability to perform signal detection and localization tasks in complex digital images is not understood. We examine this critical question by studying task-based human observer performance in detecting and localizing signals in tomographic breast images. We have also investigated how these changes impact the formation of second-order image texture. We used digital breast tomosynthesis (DBT) an FDA approved tomographic X-ray breast imaging method as the modality of choice to show our preliminary results. Our human observer studies involve localization ROC (LROC) studies for low contrast mass detection in DBT. Simulated images are used as they offer the benefit of known ground truth. Our results prove that changes in system geometry or processing leads to changes in image texture magnitudes. We show that the variations in several well-known texture features estimated in digital images correlate with human observer detection-localization performance for signals embedded in them. This insight can allow efficient and practical techniques to identify the best imaging system design and algorithms or filtering tools by examining the changes in these texture features. This concept linking texture feature estimates and task based image quality assessment can be extended to several other imaging modalities and applications as well. It can also offer feedback in system and algorithm designs with a goal to improve perceptual benefits. Broader impact can be in wide array of areas including imaging system design, image processing, data science, machine learning, computer vision, perceptual and vision science. Our results also point to the caution that must be exercised in using these texture features as image-based radiomic features or as predictive markers for risk assessment as they are sensitive to system or image processing changes.Item Open Access Optimization and validation of a neutralizing antibody assay for HIV-1 in A3R5 cells.(Journal of immunological methods, 2014-07) Sarzotti-Kelsoe, Marcella; Daniell, Xiaoju; Todd, Christopher A; Bilska, Miroslawa; Martelli, Amanda; LaBranche, Celia; Perez, Lautaro G; Ochsenbauer, Christina; Kappes, John C; Rountree, Wes; Denny, Thomas N; Montefiori, David CA3R5 is a human CD4(+) lymphoblastoid cell line that was engineered to express CCR5 and is useful for the detection of weak neutralizing antibody responses against tier 2 strains of HIV-1. Here we describe the optimization and validation of the HIV-1 neutralizing antibody assay that utilizes A3R5 cells, performed in compliance with Good Clinical Laboratory Practice (GCLP) guidelines. The assay utilizes Renilla luciferase-expressing replication competent infectious molecular clones (IMC) encoding heterologous env genes from different HIV-1 clades. Key assay validation parameters tested included specificity, accuracy, precision, limit of detection and quantitation, specificity, linearity and range, and robustness. Plasma samples demonstrated higher non-specific activity than serum samples in the A3R5 assay. This assay can tolerate a wide range of virus input but is more sensitive to cell concentration. The higher sensitivity of the A3R5 assay in neutralization responses to tier 2 strains of HIV-1 makes it complementary to, but not a substitute for the TZM-bl assay. The validated A3R5 assay is employed as an endpoint immunogenicity test for vaccine-elicited neutralizing antibodies against tier 2 strains of HIV-1, and to identify correlates of protection in HIV-1 vaccine trials conducted globally.Item Open Access Optimization and validation of a neutralizing antibody assay for HIV-1 in A3R5 cells.(J Immunol Methods, 2014-07) Sarzotti-Kelsoe, M; Daniell, X; Todd, CA; Bilska, M; Martelli, A; LaBranche, C; Perez, LG; Ochsenbauer, C; Kappes, JC; Rountree, W; Denny, TN; Montefiori, DCA3R5 is a human CD4(+) lymphoblastoid cell line that was engineered to express CCR5 and is useful for the detection of weak neutralizing antibody responses against tier 2 strains of HIV-1. Here we describe the optimization and validation of the HIV-1 neutralizing antibody assay that utilizes A3R5 cells, performed in compliance with Good Clinical Laboratory Practice (GCLP) guidelines. The assay utilizes Renilla luciferase-expressing replication competent infectious molecular clones (IMC) encoding heterologous env genes from different HIV-1 clades. Key assay validation parameters tested included specificity, accuracy, precision, limit of detection and quantitation, specificity, linearity and range, and robustness. Plasma samples demonstrated higher non-specific activity than serum samples in the A3R5 assay. This assay can tolerate a wide range of virus input but is more sensitive to cell concentration. The higher sensitivity of the A3R5 assay in neutralization responses to tier 2 strains of HIV-1 makes it complementary to, but not a substitute for the TZM-bl assay. The validated A3R5 assay is employed as an endpoint immunogenicity test for vaccine-elicited neutralizing antibodies against tier 2 strains of HIV-1, and to identify correlates of protection in HIV-1 vaccine trials conducted globally.Item Open Access Reliability assessment of a novel cervical spine deformity classification system.(Journal of neurosurgery. Spine, 2015-12) Ames, Christopher P; Smith, Justin S; Eastlack, Robert; Blaskiewicz, Donald J; Shaffrey, Christopher I; Schwab, Frank; Bess, Shay; Kim, Han Jo; Mundis, Gregory M; Klineberg, Eric; Gupta, Munish; O'Brien, Michael; Hostin, Richard; Scheer, Justin K; Protopsaltis, Themistocles S; Fu, Kai-Ming G; Hart, Robert; Albert, Todd J; Riew, K Daniel; Fehlings, Michael G; Deviren, Vedat; Lafage, Virginie; International Spine Study GroupObject
Despite the complexity of cervical spine deformity (CSD) and its significant impact on patient quality of life, there exists no comprehensive classification system. The objective of this study was to develop a novel classification system based on a modified Delphi approach and to characterize the intra- and interobserver reliability of this classification.Methods
Based on an extensive literature review and a modified Delphi approach with an expert panel, a CSD classification system was generated. The classification system included a deformity descriptor and 5 modifiers that incorporated sagittal, regional, and global spinopelvic alignment and neurological status. The descriptors included: "C," "CT," and "T" for primary cervical kyphotic deformities with an apex in the cervical spine, cervicothoracic junction, or thoracic spine, respectively; "S" for primary coronal deformity with a coronal Cobb angle ≥ 15°; and "CVJ" for primary craniovertebral junction deformity. The modifiers included C2-7 sagittal vertical axis (SVA), horizontal gaze (chin-brow to vertical angle [CBVA]), T1 slope (TS) minus C2-7 lordosis (TS-CL), myelopathy (modified Japanese Orthopaedic Association [mJOA] scale score), and the Scoliosis Research Society (SRS)-Schwab classification for thoracolumbar deformity. Application of the classification system requires the following: 1) full-length standing posteroanterior (PA) and lateral spine radiographs that include the cervical spine and femoral heads; 2) standing PA and lateral cervical spine radiographs; 3) completed and scored mJOA questionnaire; and 4) a clinical photograph or radiograph that includes the skull for measurement of the CBVA. A series of 10 CSD cases, broadly representative of the classification system, were selected and sufficient radiographic and clinical history to enable classification were assembled. A panel of spinal deformity surgeons was queried to classify each case twice, with a minimum of 1 intervening week. Inter- and intrarater reliability measures were based on calculations of Fleiss k coefficient values.Results
Twenty spinal deformity surgeons participated in this study. Interrater reliability (Fleiss k coefficients) for the deformity descriptor rounds 1 and 2 were 0.489 and 0.280, respectively, and mean intrarater reliability was 0.584. For the modifiers, including the SRS-Schwab components, the interrater (round 1/round 2) and intrarater reliabilities (Fleiss k coefficients) were: C2-7 SVA (0.338/0.412, 0.584), horizontal gaze (0.779/0.430, 0.768), TS-CL (0.721/0.567, 0.720), myelopathy (0.602/0.477, 0.746), SRS-Schwab curve type (0.590/0.433, 0.564), pelvic incidence-lumbar lordosis (0.554/0.386, 0.826), pelvic tilt (0.714/0.627, 0.633), and C7-S1 SVA (0.071/0.064, 0.233), respectively. The parameter with the poorest reliability was the C7-S1 SVA, which may have resulted from differences in interpretation of positive and negative measurements.Conclusions
The proposed classification provides a mechanism to assess CSD within the framework of global spinopelvic malalignment and clinically relevant parameters. The intra- and interobserver reliabilities suggest moderate agreement and serve as the basis for subsequent improvement and study of the proposed classification.Item Open Access Right coronary wall CMR in the older asymptomatic advance cohort: positive remodeling and associations with type 2 diabetes and coronary calcium.(J Cardiovasc Magn Reson, 2010-12-30) Terashima, Masahiro; Nguyen, Patricia K; Rubin, Geoffrey D; Meyer, Craig H; Shimakawa, Ann; Nishimura, Dwight G; Ehara, Shoichi; Iribarren, Carlos; Courtney, Brian K; Go, Alan S; Hlatky, Mark A; Fortmann, Stephen P; McConnell, Michael VBACKGROUND: Coronary wall cardiovascular magnetic resonance (CMR) is a promising noninvasive approach to assess subclinical atherosclerosis, but data are limited in subjects over 60 years old, who are at increased risk. The purpose of the study was to evaluate coronary wall CMR in an asymptomatic older cohort. RESULTS: Cross-sectional images of the proximal right coronary artery (RCA) were acquired using spiral black-blood coronary CMR (0.7 mm resolution) in 223 older, community-based patients without a history of cardiovascular disease (age 60-72 years old, 38% female). Coronary measurements (total vessel area, lumen area, wall area, and wall thickness) had small intra- and inter-observer variabilities (r = 0.93~0.99, all p < 0.0001), though one-third of these older subjects had suboptimal image quality. Increased coronary wall thickness correlated with increased coronary vessel area (p < 0.0001), consistent with positive remodeling. On multivariate analysis, type 2 diabetes was the only risk factor associated with increased coronary wall area and thickness (p = 0.03 and p = 0.007, respectively). Coronary wall CMR measures were also associated with coronary calcification (p = 0.01-0.03). CONCLUSIONS: Right coronary wall CMR in asymptomatic older subjects showed increased coronary atherosclerosis in subjects with type 2 diabetes as well as coronary calcification. Coronary wall CMR may contribute to the noninvasive assessment of subclinical coronary atherosclerosis in older, at-risk patient groups.Item Open Access Scoliosis Research Society-Schwab adult spinal deformity classification: a validation study.(Spine, 2012-05) Schwab, Frank; Ungar, Benjamin; Blondel, Benjamin; Buchowski, Jacob; Coe, Jeffrey; Deinlein, Donald; DeWald, Christopher; Mehdian, Hossein; Shaffrey, Christopher; Tribus, Clifford; Lafage, VirginieStudy design
Inter- and intra-rater variability study.Objective
On the basis of a Scoliosis Research Society effort, this study seeks to determine whether the new adult spinal deformity (ASD) classification system is clear and reliable.Summary of background data
A classification of adult ASD can serve several purposes, including consistent characterization of a clinical entity, a basis for comparing different treatments, and recommended treatments. Although pediatric scoliosis classifications are well established, an ASD classification is still being developed. A previous classification developed by Schwab et al has met with clinical relevance but did not include pelvic parameters, which have shown substantial correlation with health-related quality of life measures in recent studies.Methods
Initiated by the Scoliosis Research Society Adult Deformity Committee, this study revised a previously published classification to include pelvic parameters. Modifier cutoffs were determined using health-related quality of life analysis from a multicenter database of adult deformity patients. Nine readers graded 21 premarked cases twice each, approximately 1 week apart. Inter- and intra-rater variability and agreement were determined for curve type and each modifier separately. Fleiss' kappa was used for reliability measures, with values of 0.00 to 0.20 considered slight, 0.21 to 0.40 fair, 0.41 to 0.60 moderate, 0.61 to 0.80 substantial, and 0.81 to 1.00 almost perfect agreement.Results
Inter-rater kappa for curve type was 0.80 and 0.87 for the 2 readings, respectively, with modifier kappas of 0.75 and 0.86, 0.97 and 0.98, and 0.96 and 0.96 for pelvic incidence minus lumbar lordosis (PI-LL), pelvic tilt (PT), and sagittal vertical axis (SVA), respectively. By the second reading, curve type was identified by all readers consistently in 66.7%, PI-LL in 71.4%, PT in 95.2%, and SVA in 90.5% of cases. Intra-rater kappa averaged 0.94 for curve type, 0.88 for PI-LL, 0.97 for PT, and 0.97 for SVA across all readers.Conclusion
Data from this study show that there is excellent inter- and intra-rater reliability and inter-rater agreement for curve type and each modifier. The high degree of reliability demonstrates that applying the classification system is easy and consistent.Item Open Access Setting objective thresholds for rare event detection in flow cytometry.(J Immunol Methods, 2014-07) Richards, Adam J; Staats, Janet; Enzor, Jennifer; McKinnon, Katherine; Frelinger, Jacob; Denny, Thomas N; Weinhold, Kent J; Chan, CliburnThe accurate identification of rare antigen-specific cytokine positive cells from peripheral blood mononuclear cells (PBMC) after antigenic stimulation in an intracellular staining (ICS) flow cytometry assay is challenging, as cytokine positive events may be fairly diffusely distributed and lack an obvious separation from the negative population. Traditionally, the approach by flow operators has been to manually set a positivity threshold to partition events into cytokine-positive and cytokine-negative. This approach suffers from subjectivity and inconsistency across different flow operators. The use of statistical clustering methods does not remove the need to find an objective threshold between between positive and negative events since consistent identification of rare event subsets is highly challenging for automated algorithms, especially when there is distributional overlap between the positive and negative events ("smear"). We present a new approach, based on the Fβ measure, that is similar to manual thresholding in providing a hard cutoff, but has the advantage of being determined objectively. The performance of this algorithm is compared with results obtained by expert visual gating. Several ICS data sets from the External Quality Assurance Program Oversight Laboratory (EQAPOL) proficiency program were used to make the comparisons. We first show that visually determined thresholds are difficult to reproduce and pose a problem when comparing results across operators or laboratories, as well as problems that occur with the use of commonly employed clustering algorithms. In contrast, a single parameterization for the Fβ method performs consistently across different centers, samples, and instruments because it optimizes the precision/recall tradeoff by using both negative and positive controls.Item Open Access Spinal instability neoplastic score: an analysis of reliability and validity from the spine oncology study group.(Journal of clinical oncology : official journal of the American Society of Clinical Oncology, 2011-08) Fourney, Daryl R; Frangou, Evan M; Ryken, Timothy C; Dipaola, Christian P; Shaffrey, Christopher I; Berven, Sigurd H; Bilsky, Mark H; Harrop, James S; Fehlings, Michael G; Boriani, Stefano; Chou, Dean; Schmidt, Meic H; Polly, David W; Biagini, Roberto; Burch, Shane; Dekutoski, Mark B; Ganju, Aruna; Gerszten, Peter C; Gokaslan, Ziya L; Groff, Michael W; Liebsch, Norbert J; Mendel, Ehud; Okuno, Scott H; Patel, Shreyaskumar; Rhines, Laurence D; Rose, Peter S; Sciubba, Daniel M; Sundaresan, Narayan; Tomita, Katsuro; Varga, Peter P; Vialle, Luiz R; Vrionis, Frank D; Yamada, Yoshiya; Fisher, Charles GPurpose
Standardized indications for treatment of tumor-related spinal instability are hampered by the lack of a valid and reliable classification system. The objective of this study was to determine the interobserver reliability, intraobserver reliability, and predictive validity of the Spinal Instability Neoplastic Score (SINS).Methods
Clinical and radiographic data from 30 patients with spinal tumors were classified as stable, potentially unstable, and unstable by members of the Spine Oncology Study Group. The median category for each patient case (consensus opinion) was used as the gold standard for predictive validity testing. On two occasions at least 6 weeks apart, each rater also scored each patient using SINS. Each total score was converted into a three-category data field, with 0 to 6 as stable, 7 to 12 as potentially unstable, and 13 to 18 as unstable.Results
The κ statistics for interobserver reliability were 0.790, 0.841, 0.244, 0.456, 0.462, and 0.492 for the fields of location, pain, bone quality, alignment, vertebral body collapse, and posterolateral involvement, respectively. The κ statistics for intraobserver reliability were 0.806, 0.859, 0.528, 0.614, 0.590, and 0.662 for the same respective fields. Intraclass correlation coefficients for inter- and intraobserver reliability of total SINS score were 0.846 (95% CI, 0.773 to 0.911) and 0.886 (95% CI, 0.868 to 0.902), respectively. The κ statistic for predictive validity was 0.712 (95% CI, 0.676 to 0.766).Conclusion
SINS demonstrated near-perfect inter- and intraobserver reliability in determining three clinically relevant categories of stability. The sensitivity and specificity of SINS for potentially unstable or unstable lesions were 95.7% and 79.5%, respectively.