Figures
Abstract
Background
The aim of this study was to evaluate the diagnostic utility of PAI at detecting thyroid microcalcifications at 700 nm laser wavelengths.
Methods
This study included 36 resected samples in 18 patients. To evaluate the PA manifestation of microcalcifications in PAI, gray level histogram and co-occurrence matrix (COM) texture parameters were extracted from the 3 fixed ROI US and PA images, respectively, per sample. We compared the textural parameters obtained from specimen PAIs between samples with punctate microcalcifications on specimen radiography and those without microcalcifications.
Results
On specimen US, the mean value (2748.4±862.5) of samples with microcalcifications on specimen radiography was higher than that (1961.9±780.2) of those without microcalcifications (P = 0.007). However, there were no significant differences in textural parameters obtained from specimen PAIs between samples with punctate microcalcifications on specimen radiography and those without when applying both the mean value of the three slices of thyroid specimens and the value of the thyroid specimen slice which had the highest value of the mean values in specimen US.
Conclusion
PAI did not show significant PA contrast on thyroid microcalcifications indicating that the experimental setup and protocols should be enhanced, e.g., method of complete blood rejection from ex vivo specimens, the multi-wavelength spectroscopic PA imaging method which can solely extract the PA signal from microcalcifications even with high spectral interferences, or PA imaging with narrower slice thickness using 2-dimensional array transducer, etc.
Citation: Kang J, Chung WY, Kang S-W, Kwon HJ, Yoo J, Kim E-K, et al. (2014) Ex Vivo Estimation of Photoacoustic Imaging for Detecting Thyroid Microcalcifications. PLoS ONE 9(11): e113358. https://doi.org/10.1371/journal.pone.0113358
Editor: Xiaoyuan Chen, NIH, United States of America
Received: May 9, 2014; Accepted: October 24, 2014; Published: November 21, 2014
Copyright: © 2014 Kang et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The authors confirm that, for approved reasons, some access restrictions apply to the data underlying the findings. Human thyroid tissues were obtained following the approval of the Institutional Review Board (IRB) of Severance Hospital (Seoul, Korea). Informed written consent was obtained from all patients and the study was performed in accordance with the ethical guidelines of the Helsinki Declaration.
Funding: This study was supported by a faculty research grant from Yonsei University College of Medicine for 2012 (No. 6-2012-0140). This research was also supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) by the Ministry of Education (2013R1A1A2058817) and by International Collaborative R&D Program (M0000053, Development of a Novel Multi-modal In vivo Imaging System for Animal-to-Human Use) funded by the Ministry of Trade, Industry and Energy (MOTIE), Korea. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
High-frequency ultrasonography (US) is a highly sensitive diagnostic tool that detects thyroid calcifications as well as thyroid nodules. On US, hyperechoic foci with or without acoustic shadowing have been interpreted as calcifications with the sensitivity and specificity of US for diagnosing a malignancy reported to be 26.1–59.1% and 85.8–95.0%, respectively [1]. However, US has shown some drawbacks in the detection of thyroid microcalcifications which are related to thyroid malignancies [2]. In the field of breast cancer diagnosis, mammography has been widely used on lesions which are not visible on US to document breast calcifications related to breast cancers [3]. Although thyroid calcifications can only be seen through specimen radiography, there have been no preoperative clinical tools that detect thyroid calcifications like mammography does with breast calcifications until now [2].
Photoacoustic imaging (PAI) is a raising imaging modality that uses the different optical absorption properties of tissues and it provides high spatial and contrast resolutions [4], [5]. When pulsed laser light is emitted into targets, the targets absorb energy. Then, a transient temperature rise is converted to a pressure rise by thermoelastic expansion and subsequently photoacoustic signals are generated. The signals are received by US transducers and then imaged via several reconstructive techniques [6]–[8]. Since the absorption coefficients of oxygenated- and deoxygenated-hemoglobin in blood are higher than that of surrounding tissues and the optical absorption is different according to the type of molecule as well as tissue, PAI provides functional information about tissue oxygenation and vascularization [9]. Also, integrated US and PAI have been introduced in a variety of clinical fields such as diagnoses of ovarian cancer [10], [11], breast cancer [12], and inflammatory joint diseases [13], and the detection of lipid deposits [14] and breast microcalcifications [15].
Through previous ex vivo experiments, we have found that the optimal laser wavelength of PAI to detect breast microcalcifications which were not detected on US is in the 690–700 nm range [15]. Although thyroid glands have different tissue characteristics from breast tissue, we assume that microcalcifications have the same absorption coefficients regardless of tissue type. In this study, we evaluate the diagnostic utility of PAI at detecting thyroid microcalcifications at 700 nm laser wavelengths.
Subjects and Methods
Human thyroid tissues were obtained following the approval of the Institutional Review Board (IRB) of Severance Hospital (Seoul, Korea). Informed written consent was obtained from all patients and the study was performed in accordance with the ethical guidelines of the Helsinki Declaration. The study period was from Dec 2012 to Mar 2014. All patients scheduled for thyroid cancer surgery at the surgery department of our hospital were evaluated for inclusion. For this study, specimen radiographs were immediately obtained after surgery. Specimen radiographs were evaluated in comparison to staging ultrasound (US) images by one radiologist who decided the study candidates. After reviewing the two images, two thyroid tissue samples (about 20×5×5 mm, length × width × depth) from different lobes were obtained from the specimens, with one containing compact punctate calcifications and the other not containing any calcifications. This sample collection did not interfere with the pathologic diagnosis process. To verify calcifications, specimen radiographs were also obtained at resected thyroid tissues. Initially, 54 samples from 27 patients were obtained for this study. Eighteen samples in 9 patients were excluded because they had blood coagulations on the samples, which were formed after the resection and would possibly introduce a big bias on the result. Therefore, of the initial 54 samples, 36 resected samples from 18 patients were included as the final subjects of this study. The selected specimens were secondly washed out by being steeped in saline solution for 6 hours.
Imaging by ultrasonography and specimen radiography [2]
Thyroid US was performed with a 5–12-MHz linear-array transducer (iU22; Philips Medical Systems). Compound imaging was performed in all cases. Real-time ultrasonography was performed by one of 7 radiologists specializing in thyroid imaging. The scanning protocol in all patients included both transverse and longitudinal real-time imaging of the thyroid, with the use of a picture archiving and communications system to review all patient data. Microcalcifications revealed by US were defined as hyperechoic punctate foci with or without acoustic shadowing, excluding dense round calcifications or condensed colloids showing a comet tail artifact.
After surgery, specimen radiographs were obtained using the Lorad/Hologic Selenia FFDM system (Lorad/Hologic, Danbury, Connecticut, USA), which was a dedicated mammography unit during the study period. The system, based on a detector with amorphous selenium, used a direct-capture, 70-µm pixel device and yielded a 2560×3328 matrix image with an 18×24-cm paddle. The system was set to allocate 16-bit images and store them at 12 bits. Routine views of thyroid specimens were obtained (focal spot size 0.3 mm). These images were displayed on a pair of high-resolution, 5-megapixel LCD monitors (MFGD 5621HD; Barco, Buluth, Georgia, USA) that were part of the review workstation (Selenia Softcopy WorkstationTM; Lorad/Hologic) with soft-copy reading software (MeVis BreastCare; MeVis Medical Solutions, Bremen, Germany).
Ex vivo experimental setup for photoacoustic imaging
After acquiring X-ray specimen radiographs, the ex vivo experiments were conducted with the PA/US imaging system shown in Figure 1. For the experiments, the respective thyroid specimens were set on a scatter-free gel pad (Aquaflex, Parker Lab, Inc., Farfield, NJ, USA), which was immersed in a 0.9% saline-filled container. The temperature in the container was stably maintained at 24°C. For the experiments, radio-frequency (RF) echo data for USI and PAI were captured with a commercial US scanner equipped with a SonixTouch research package (Ultrasonix Corp., Vancouver, BC, Canada) and a 5–14-MHz linear array transducer connected to a 128-channel SonixDAQ parallel system. The Q-switch trigger of a Nd:YAG laser-pumped OPO system (Surelite III-10 and Surelite OPO Plus, Continuum Inc., Santa Clara, CA, USA) was generated whenever the laser pulse was excited with a 10 Hz rate and then it was sent to a US scanner. By referring the Q-switch trigger, the dedicated data acquisition system stored a scanline and a frame RF channel data for PAI and USI, respectively. The laser pulse duration was 7 ns and its wavelength was fixed at 700 nm by a software program in the workstation. The distance between the array transducer and thyroid specimen was fixed at 30 mm, which was the geometrical focusing point of a custom bifurcated optic fiber bundle (Fiberoptic Systems, Inc., Simi Valley, CA, USA). The energy density was maintained at 10 mJ/cm2, which is under the ANSI radiative safety standards [17]. The energy density was ascertained by averaging 500 sequential pulses with an energy meter and sensor (MAESTRO and QE25, Gentec Electro-Optics Inc., Quebec, QC, Canada). From the experimental setup of PAI, three PA image slices were acquired from the respective resected thyroid tissues, which seemed to contain the calcification-suspicious foci on the corresponding US image slices regardless of the existence of microcalcifications. Afterwards, for the selected slices, regions-of-interest (ROIs) were chosen within an area the size of 0.5 cm and 0.5 cm in the axial and lateral directions, respectively. Note that the slices and ROIs were carefully chosen by a graduate researcher (K.J.) not to include the hyperreactive artifact regions in PA images, which are suspected to be from the partially-distributed blood remained despite of the 6-hour wash-out procedure with saline solution. Also, it should be noted that the selection of all the image slices and ROIs were conducted without specimen radiographs, thereby the statistical texture analysis parameters could be blindly computed for the respective thyroid specimens.
Note that the FB stands for the optical fiber bundle which delivers the laser energy to the thyroid specimen and the x, y, and z direction indicate the elevation, lateral, and axial direction from the PA/US combined probe, respectively.
Computation of texture analysis parameters
To evaluate the PA manifestation of microcalcifications in PAI, gray level histogram and co-occurrence matrix (COM) texture parameters [16] were extracted from the 3 fixed ROI US and PA images, respectively, per sample by using in-house software developed with Matlab R2010a (MathWorks, Natick, Massachusetts). Gray level histogram parameters consisted of mean, mean variance, skewness, kurtosis, and entropy, which presented a graphical distribution of the echogenic intensity values of pixels in the ROI. The COM parameters consisted of contrast, correlation, uniformity, homogeneity, and entropy, which described the joint probability of pixel pairs along all directions at different distances in the ROI. The co-occurrence matrix represented the spatial relationship of two neighbor pixels based on intensity level. First, ROI data were converted into P level images where P is a positive integer. P, the number of the level, has a relationship with SNR (Signal-to-Noise Ratio). Usually, a low level number corresponds to the robustness of algorithm to noise but it has limitations such as a lack of spatial information. In order to define the spatial relationship between two pixels, co-occurrence distance and direction should be defined in advance. Classification performance is related with co-occurrence distance and direction parameters because of the character of target texture. Therefore, to acquire optimal results, several co-occurrence matrices were calculated by adjusting the level number, co-occurrence distance and direction. In this study, ROI data were converted into 4, 6, 8, 10 level images. From these converted images, co-occurrence matrices were calculated in the orthogonal or diagonal direction and a co-occurrence matrix distance of 1 and 3. COM parameter values were generated from the co-occurrence matrix. Contrast, correlation, uniformity, homogeneity and entropy defined by Haralick et al were used as COM parameters [16].
Specimen pathology
After conducting PAI, all resected specimens were fixed in 10 percent formalin and cut at 3-mm intervals. Surgical specimens were embedded in paraffin wax and stained with haematoxylin and eosin for histological examination. A pathologist, who did not have any prior information about the US and specimen radiographs, evaluated the presence of psammoma bodies (PBs) in the specimens.
Statistical Analysis
To compare continuous variables, the independent two-sample t-test or the Mann-Whitney U test was used according to the normality from the Kolmogorov-Smirnov test. Continuous variables were presented as means ± standard deviation (SD) or as medians with interquartile range. Statistical significance was assumed when the P value was less than or equal to.05. All reported P values are two-sided. All analyses were performed using SAS statistical software (SAS system for Windows, version 9.1.3; SAS Institute, Cary, NC).
Results
During the study period, there were 36 resected thyroid tissues. While 18 tissues contained compact punctate calcifications on specimen radiography, the other 18 tissues did not. Among the 18 resected samples with punctate microcalcifications on specimen radiography, 14 had PBs on pathology. In contrast, 5 samples of the tissues without punctate microcalcifications on specimen radiology had PBs on pathology.
On specimen US, the mean value (2748.4±862.5) of samples with microcalcifications on specimen radiography was higher than that (1961.9±780.2) of those without microcalcifications (P = 0.007). However, there were no significant differences in textural parameters obtained from specimen PAIs between samples with punctate microcalcifications on specimen radiography and those without when applying both the mean value of the three slices of thyroid specimens and the value of the thyroid specimen slice which had the highest value of the mean values in specimen US (Tables 1 and 2, Figures 2 and 3). When we compared textural parameters obtained from specimen PAIs between the samples with PBs on pathology and those without after applying the mean value of the three slices of thyroid specimens, there were also no significant differences between the tissues with and without PBs (Table 3).
A specimen radiograph (a) showed a mass with calcifications in the upper and mid pole of the right thyroid gland and punctate microcalcifications in the lower pole of the right thyroid gland which were not seen on US. There were no discernible calcifications in the left thyroid gland. To verify calcifications on PAI, two thyroid tissues from different lobes were obtained from thyroid specimens, with tissue from the right lobe containing compact punctate calcifications and tissue from the left lobe not containing any calcifications (two square boxes). The tissues were retrieved from the thyroid specimen without disturbing pathologic diagnosis. A specimen radiograph (b) showed punctate microcalcifications (arrows) acquired from the right thyroid gland and no calcifications acquired from the left thyroid gland. The texture analysis was conducted for the selected region-of-interests (ROIs) indicated by the white boxes in PA images (c) at 700-nm lasing wavelength. Note that the clustered hyperreactive PA signals suspected from the blood coagulations (white arrows) were excluded in the texture analysis, which can also be found on the corresponding photographs (d). In visual assessment, the ROI of the right resected tissue with microcalcifications contains more PA signals than that of the left resected tissue without microcalcifications. Correspondingly, the mean values of both US and PA images calculated from the right thyroid gland were higher than that from the left thyroid gland. There were psammoma bodies from a right resected tissue and no psammoma bodies (arrows) (H&E; X100) from a left resected tissue on pathology (e).
A specimen radiograph (a) showed a mass with diffuse punctate calcifications in the left thyroid gland and no discernable calcifications in the right thyroid gland. To verify calcifications on photoacoustic imaging, two thyroid tissues from different lobes were obtained from thyroid specimens, with tissue from the left lobe containing compact punctate calcifications and tissue from the right lobe not containing any calcifications (two square boxes). The tissues were retrieved from the specimen without disturbing pathologic diagnosis. A specimen radiograph (b) showed punctate microcalcifications (arrows) acquired from the left thyroid gland and no calcifications acquired from the right thyroid gland. The texture analysis was conducted for the selected region-of-interests (ROIs) indicated by the white boxes in PA images (c) at 700-nm lasing wavelength. Note that the clustered hyperreactive PA signals suspected from the blood coagulations (white arrows) were excluded in the texture analysis, which can also be found on the corresponding photographs (d). In visual assessment of PA image, the ROI of the right resected tissue with microcalcifications contains more PA signals than that of the left resected tissue without microcalcifications. Although the mean value of samples from the left thyroid gland was higher than that of those from the right thyroid gland on specimen US, the mean value of samples from the right thyroid gland was higher than that of the left thyroid gland on both photoacoustic images. There were psammoma bodies from a left resected tissue and no psammoma bodies (arrows) (H&E; X100) from a right resected tissue on pathology (e).
There were 8 patients who had higher mean values from samples with microcalcifications than those without microcalcifications on specimen PAIs (Figure 2). The remaining patients showed opposite results (Figure 3).
Discussion
Microcalcifications within a thyroid nodule have been highly associated with thyroid cancer [17]–[19]. Therefore, several guidelines have accepted microcalcifications as a useful US feature in the diagnosis of thyroid cancer or in the selection of a thyroid nodule for fine needle aspiration [1], [20], [21]. However, there are two limitations in applying this US feature at a thyroid nodule in the clinical field. First, hyperechoic spots are not always related to microcalcifications. Second, assessing microcalcifications is subjective like the assessment of other US features. Reported interobserver variability shows a kappa value of 0.44–0.63 [22]–[25], which reflects the subjective nature of calcification assessment on US. Although US is the highest sensitive preoperative diagnostic tool for detecting a focal thyroid lesion, it does not show all microcalcifications which may be related to PBs [2]. Therefore, there is an increasing demand for a more sensitive and objective diagnostic tool in thyroid nodule assessment.
Photoacoustic imaging has been widely studied fused with variable medical fields such as US and microscopy [26], [27]. Recently, Kang et al. investigated PAI-visualized microcalcifications in breast tissues in a study which revealed the absorbance differences between microcalcifications and surrounding tissues [15], [28]. They demonstrated that a wavelength between 700 nm and 800 nm was the distinguishing absorption spectrum of microcalcifications in breast tissue [15]. Therefore, we investigated the diagnostic utility of PAI for thyroid microcalcifications at 700 nm laser wavelengths on the assumption that microcalcifications have the same absorption coefficients as breast tissue. In our study, we only included thyroid tissues with diffuse scattered punctate microcalcifications on specimen radiography and corresponding specimens without microcalcifications extracted from the same patient to minimize some factors such as underlying tissue changes which can impact the results of this study. We compared the parameters of texture analysis on PAIs in 3 different situations. First, we compared texture analysis obtained from the mean values of 3 specimen PAIs between tissues with punctate microcalcifications on specimen radiography and those without. As the mean value of histograms on US was higher in samples with microcalcifications on specimen radiography than those without, we can confirm that representative PAIs can show microcalcifications. However, the mean value of histograms on PAIs was lower in samples with microcalcifications on specimen radiography than ones without microcalcifications without statistical significance. Other parameters from texture analysis did not show any differences between them. Second, we selected the values of texture analysis from the slice of a thyroid specimen which had the highest value of the mean values in specimen US. However, there were no statistical significant parameters that differentiated samples with and without microcalcifications. Third, we compared textural parameters in specimens with PBs on pathology and those without because not all microcalcifications on specimen radiography correlate with PBs which can be related to a malignancy [2]. However, no significant differences were found between them as well.
From these results, it can be concluded that the textural analysis on PAI of thyroid microcalcifications with a linear array transducer does not indicate statistical significance in the spatial domain unlike that of breast microcalcifications. There can be several reasons that explain the different results between these similar studies on thyroid and breast tissues: blood coagulation on thyroid tissue samples may be the main artifact of our study because the thyroid gland is a highly vascular organ compared to breast tissue. From this structural difference between breast and thyroid tissue, we can assume that thyroid specimens may contain more blood that can coagulate between time durations for excision, resection and delivery to the imaging room for ex vivo experiments. Therefore, the relatively small endogenous PA signal from thyroid microcalcifications can be interfered with the oxy- and deoxy-hemoglobins which are the most significant light absorbers within the human body. In our study, the clustered hyperreactive PA signals were correspondingly shown on the reconstructed PA images. Those suspected to be from the residual blood coagulations are indicated by white arrows in Figures 2 and 3, respectively, despite the wash-out procedure with saline solution. Also, a technical issue arose in the scanning process for blood-free regions of the thyroid specimens. Although, a graduate researcher (K. J.) carefully selected slice positions to exclude blood coagulations, hyperreactive interference could not be completely excluded due to the broad slice thickness in the elevation direction of the US linear array transducer as simulated by Kang et al. (e.g., approximately 2 mm of slice thickness at 30-mm depth) [29]. Also, 3 dimensional images were not used in this study as microcalcifications on specimen radiography could not be correlated with US images. However, we assume that this limitation is negligible because the mean value of US images from samples with microcalcifications on specimen radiography was higher than that of US images from samples without microcalcifications.
In conclusion, PAI did not show significant difference in textural analysis on thyroid tissue with microcalcifications. This might be because PAI is more sensitive to tissue absorption changes rather than scattering changes, like microcalcifications, which can affect the result of our study. Thus, to be useful for clinical practice, the subtle light absorption change caused by microcalcifications should be differentiable in PA imaging. This requires an improvement in experimental setup and protocols and in turn, spatial resolution must be improved, especially in the elevation direction. For this, the 2-dimensional array transducer (e.g., xMATRIX transducer, Philips Electronics, Amsterdam, Netherlands) can be used which can provide narrower and uniformed elevation slice thickness throughout the imaging depth. Secondly, the complete blood rejection method for excised ex vivo specimens should be employed to solely evaluate the PA signals generated from thyroid microcalcifications. However, the ultimate solution should be the development of a multiwavelength spectroscopic PA imaging technique which can differentiate the microcalcifications from blood in vivo even in a highly interfered environment.
Author Contributions
Conceived and designed the experiments: JK JYK. Performed the experiments: JK HJK JHC T-KS JYK. Analyzed the data: JK JY JYK. Contributed reagents/materials/analysis tools: JK WYC S-WK HJK JHC T-KS JYK. Wrote the paper: JK E-KK JYK SL. Designed the software used in analysis: JY.
References
- 1. Frates MC, Benson CB, Charboneau JW, Cibas ES, Clark OH, et al. (2005) Management of thyroid nodules detected at US: Society of Radiologists in Ultrasound consensus conference statement. Radiology 237: 794–800.
- 2. Kwak JY, Kim EK, Hong SW, Kim MJ, Moon HJ, et al. (2010) Value of specimen radiographs in diagnosing multifocality of thyroid cancer. Br J Surg 97: 517–524.
- 3.
American College of Radiology (2003) Breast imaging reporting and data system, breast imaging atlas. Reston: American College of Radiology.
- 4. Zhang HF, Maslov K, Stoica G, Wang LV (2006) Functional photoacoustic microscopy for high-resolution and noninvasive in vivo imaging. Nat Biotechnol 24: 848–851.
- 5. Siphanto RI, Thumma KK, Kolkman RG, van Leeuwen TG, de Mul FF, et al. (2005) Serial noninvasive photoacoustic imaging of neovascularization in tumor angiogenesis. Opt Express 13: 89–95.
- 6. Erpelding TN, Kim C, Pramanik M, Jankovic L, Maslov K, et al. (2010) Sentinel lymph nodes in the rat: noninvasive photoacoustic and US imaging with a clinical US system. Radiology 256: 102–110.
- 7. Wang X, Fowlkes JB, Cannata JM, Hu C, Carson PL (2011) Photoacoustic imaging with a commercial ultrasound system and a custom probe. Ultrasound Med Biol 37: 484–492.
- 8.
Xu M, Wang LV (2006) Photoacoustic imaging in biomedicine. Review of Scientific Instruments 77.
- 9. Su JL, Wang B, Wilson KE, Bayer CL, Chen YS, et al. (2010) Advances in Clinical and Biomedical Applications of Photoacoustic Imaging. Expert Opin Med Diagn 4: 497–510.
- 10. Aguirre A, Guo P, Gamelin J, Yan S, Sanders MM, et al. (2009) Coregistered three-dimensional ultrasound and photoacoustic imaging system for ovarian tissue characterization. J Biomed Opt 14: 054014.
- 11. Aguirre A, Ardeshirpour Y, Sanders MM, Brewer M, Zhu Q (2011) Potential role of coregistered photoacoustic and ultrasound imaging in ovarian cancer detection and characterization. Transl Oncol 4: 29–37.
- 12.
Kitai T, Torii M, Sugie T, Kanao S, Mikami Y, et al.. (2012) Photoacoustic mammography: initial clinical results. Breast Cancer.
- 13. Xu G, Rajian JR, Girish G, Kaplan MJ, Fowlkes JB, et al. (2013) Photoacoustic and ultrasound dual-modality imaging of human peripheral joints. J Biomed Opt 18: 10502.
- 14. Wang B, Karpiouk A, Yeager D, Amirian J, Litovsky S, et al. (2012) In vivo intravascular ultrasound-guided photoacoustic imaging of lipid in plaques using an animal model of atherosclerosis. Ultrasound Med Biol 38: 2098–2103.
- 15.
Kang J, Kim EK, Young Kwak J, Yoo Y, Song TK, et al.. (2011) Optimal laser wavelength for photoacoustic imaging of breast microcalcifications. Applied Physics Letters 99.
- 16. Haralick RM, Shanmugam K, Dinstein I (1973) Textural features for image classification. IEEE Transactions on Systems, Man and Cybernetics smc 3: 610–621.
- 17. Kwak JY, Jung I, Baek JH, Baek SM, Choi N, et al. (2013) Image reporting and characterization system for ultrasound features of thyroid nodules: multicentric Korean retrospective study. Korean J Radiol 14: 110–117.
- 18. Kwak JY, Han KH, Yoon JH, Moon HJ, Son EJ, et al. (2011) Thyroid imaging reporting and data system for US features of nodules: a step in establishing better stratification of cancer risk. Radiology 260: 892–899.
- 19. Moon HJ, Sung JM, Kim EK, Yoon JH, Youk JH, et al. (2012) Diagnostic performance of gray-scale US and elastography in solid thyroid nodules. Radiology 262: 1002–1013.
- 20. Cooper DS, Doherty GM, Haugen BR, Kloos RT, Lee SL, et al. (2009) Revised American Thyroid Association management guidelines for patients with thyroid nodules and differentiated thyroid cancer. Thyroid 19: 1167–1214.
- 21. Gharib H, Papini E, Paschke R, Duick DS, Valcavi R, et al. (2010) American Association of Clinical Endocrinologists, Associazione Medici Endocrinologi, and European Thyroid Association medical guidelines for clinical practice for the diagnosis and management of thyroid nodules: executive summary of recommendations. J Endocrinol Invest 33: 51–56.
- 22. Choi SH, Kim EK, Kwak JY, Kim MJ, Son EJ (2010) Interobserver and intraobserver variations in ultrasound assessment of thyroid nodules. Thyroid 20: 167–172.
- 23. Park SJ, Park SH, Choi YJ, Kim DW, Son EJ, et al. (2012) Interobserver variability and diagnostic performance in US assessment of thyroid nodule according to size. Ultraschall Med 33: E186–190.
- 24. Park SH, Kim SJ, Kim EK, Kim MJ, Son EJ, et al. (2009) Interobserver agreement in assessing the sonographic and elastographic features of malignant thyroid nodules. AJR Am J Roentgenol 193: W416–423.
- 25. Kim SH, Park CS, Jung SL, Kang BJ, Kim JY, et al. (2010) Observer Variability and the Performance between Faculties and Residents: US Criteria for Benign and Malignant Thyroid Nodules. Korean J Radiol 11: 149–155.
- 26. Rao B, Soto F, Kerschensteiner D, Wang LV (2014) Integrated photoacoustic, confocal, and two-photon microscope. J Biomed Opt 19: 36002.
- 27. Bouchard R, Sahin O, Emelianov S (2014) Ultrasound-guided photoacoustic imaging: current state and future development. IEEE Trans Ultrason Ferroelectr Freq Control 61: 450–466.
- 28.
Kang J, Kim EK, Kim GR, Yoon C, Song TK, et al.. (2013) Photoacoustic imaging of breast microcalcifications: A validation study with 3-dimensional ex vivo data and spectrophotometric measurement. J Biophotonics.
- 29.
Kang J, Kim EK, Song TK, Chang JH. Photoacoustic Imaging of Breast Microcalcifications: A Validation Study with 3-dimensional ex vivo Data; 2012; Dresden, Germany.