This paper presents an adaptive imaging technique run on a mobile service system for endoscopic image enhancement by using color transform and Gray Level Co-occurrence Matrices (GLCM) for a single input endoscopy image. The method is simply deal with the color image channels combination which chose the maximum scalar values of red, green and blue channel images, respectively. The GLCM subsequently applied for selecting the highest contrast and entropy images of the expanding image series. The enhanced endoscopy image is generated by fusing of the color, contrast and entropy images. We also proposed a service system with medical image retrieval application via quick response code authentication based on the Android operating system, which helps clinicians convenient in using mobile phone and reviewing images of the patient with cost efficiency. For the mobile technologies are growing rapidly, the mobile service system is installed to connect a Picture Archive and Communication Systems (PACS) system in hospital and applied for automatic evaluation of colon images screening. The experimental results show the proposed system is efficient for observing gastrointestinal tract polyp. The performance is evaluated and compared with Fujinon intelligent chromo endoscopy enhanced method.
The image enhance-based endoscopies for tumours and polyps diagnosis become popular and influential in recent decades (Gross et al., 2011). Gastrointestinal tract (GI) cancer is a common malignant tumor seen in the clinic and the incidence rate of GI lesion is increasing in most countries. The focal point to improve the survival rate is the diagnosis and treatment at an early stage, such as regular endoscopy examination. The traditional endoscopy only detects an apart of characteristics of the lesion, even apply magnifying techniques and it is not enough for observing the mucosal blood and pit pattern capillaries (Tanaka et al., 2006). Therefore, the narrow-band imaging (NBI) (Sharma et al., 2006) and Fujinon intelligent chromo endoscopy (FICE) (Chung et al., 2010) are developed to enhance endoscopy images for diagnosis.
NBI is a developed technology that uses optical filters for colors sequential illumination and narrows the bandwidth of spectral transmittance (Tanaka et al., 2006). It allows enhancing the contrast of vascular patterns on the mucosal surface. NBI enables the observation of the fine capillaries in the superficial mucosa of the gastrointestinal tract. In contrast, FICE, invented by Yoichi et al. (Shiobara, Zhou, Haneishi, Tsumura, & Miyake, 1996), is based on a spectral estimation technology without optical filters. It is a diagnosis device for an easier observation of tissue characterization on surface parts, and capillary orientations become clearer. FICE system used the pure image processing technique and makes the possible to select the most suitable wavelengths for observing vessels and small tumor. However, NBI was performed with highly cost of optical lens, and studies shown that FICE is little improvement for clinical observation (Gupta, Ibrahim, Deviere, & Van Gossum, 2011; Murino et al., 2011).
The aim of this work is to develop an efficiently enhancing of the endoscopy image while the clinical screening, especially for the texture of microvascular and microsurface features. The work is encouraged by the Hirakawa and Simon (2011) to develop a convenient digital color filter for computing enhanced endoscopy image. In Hirakawa and Simon (2011), they combine photographic filter placed over the lens and the color filter array on image sensor induces differences in red, green, and blue channel sensitivities. In contrast to their approach, we replace real filters with pure digital method to generate the corresponding channel images.
We applied the color shifting method (Wang et al., 2013) to generate a series of images. Our eyes are sensitive to light which lies in a very small region of the electromagnetic spectrum labeled visible light. The human eye is not capable of seeing radiation with wavelengths outside the visible spectrum. The rendered endoscopy images are definitely lying within the visible light and each image pixel corresponds to a wavelength within the range of 400-700 nm. The series of shifting color images is generated by modifying the values of chrominance and chroma, but keeps the values of the luminance fixed for each image pixel. This says, the acquired image pixels could shift to many new colors by modifying the corresponding values of chrominance and chroma within the visible light. To shift the chrominance and chroma of the image pixels is the key idea of this work to generate a series of images from a single input endoscopy image.
To select the enhanced image, we take the color in three channel images and contrast based on GLCM analysis into account (the theoretical aspects were best approached in Akono, Tankam, Nyoungui Tonye, & Dipanda, 2006) and we apply a second order GLCM on the endoscopy image to transform the gray level image to statistics computing. The GLCM is adopted for computing and selecting the highest contrast value from the generated series of an endoscopy image. We demonstrate that to apply the second GLCM for shifting images over the different wavelength of the endoscopy images allows, and their variations can lead to discrimination in the texture and details of polys and types. This work is both to increase the efficiency and clearly of polyp observation for fixable endoscopy in gastrointestinal tract.
PACS are comprehensive management systems for diagnostic imaging studies that are increasingly used in hospitals and health care systems. It is an essential system for a hospital to share and explore electronic information inside (Hood & Scott, 2006). The PACS application has changed with advanced technology, and it not only used in hospital but also make medical images explore on mobile devices (Maglogiannis, Doukas, Kormentzas, & Pliakas, 2009).
In this paper, we investigate an enhanced method to observe the endoscopy images for vessels, subcutaneous or polyp of the GI tract. The major contribution of this study is to design a fast image processing method which processes on separated color image channels and selects the channel images with highest scalar values. The images are then applied to clinical GI endoscopy for efficient observing polyp. The technique is simple and fast for the physician to the diagnose and measuring the tumors and non-tumor area.
2Related worksGI endoscopy is an important diagnostic and therapeutic tool in our daily medical practice. The evolution of endoscopic technologies from simple tubes to the flexible scopes of today was achieved by cooperation between physicians, surgeons, technicians, and manufactures (Bai & Li, 2009). The quality, texture and resolution of the color image acquired from the gastrointestinal endoscopy infuse the physicians to make the decision of abnormal lesions. Images enhancement then became a key component of the modern endoscopic technology.
NBI is one of the popular methods that use the blue-green color in the 415 nm and 540 nm illumination bands (Uraoka, Saito, Ikematsu, Yamamoto, & Sano, 2011). It is a developed technology that uses optical filters for colors sequential illumination and narrows the bandwidth of the spectral transmittance (Tanaka et al., 2006). NBI enables the observation of the fine capillaries in the superficial mucosa of the gastrointestinal tract. The drawback of NBI is the lacks of a normal light source; the bandwidth of the frequency is narrowed by imaging narrow the scope, and cannot be quickly converted to the other band.
In contrast to NBI, FICE is a computed virtual chromoendoscopy system. It decomposes images by wavelength, then directly produces reconstructed images with enhanced mucosal surface contrast (Nakayoshi et al., 2004) or contour (López-Juárez, Castelán, Castro-Martínez, Peña-Cabrera, & Osorio-Comparan, 2013). The method is efficient and a bit cost-effective to develop; however, Wang et al. (2013) mentions about the FICE system not really enhanced endoscopy images for small, abnormal polyp observation. In this study, we would like to develop an image enhancement technique based on the concept of virtual chromoendoscopy system for contrast enhancing the images captured by GI fixable endoscopy.
For the color enhancement, Jean-Luc et al. (Starck, Murtagh, Candes, & Donoho, 2003) present a method for contrast enhancement based on the curvelet transform. Their findings are that the image enhancement outperforms other filter-based enhancement methods on noisy images. Our enhanced technique is based on the contrast value extended line-scan method (Wang, Lin, County, & Yan, 2012) to develop an adaptive image enhancement technique. This paper was inspired by Masahito et al. (2009) and Wang et al. (2012) to develop a convenient enhanced method for adaptive and quick image contrast enhancement of the mucosa and polyp area.
The higher order GLCM was applied in Huber et al. (2009) for the analysis of the trabecular bones within proximal femur radiographs. Ameling, Wirth, Paulus, Lacey, & Vilarino (2009) proposed methods for automatic detection of polyps in colonoscopy images to support surgeons during endoscopy examination. They are based on Grey-Level-Co-occurrence and Local-Binary-Patterns to develop a texture feature extraction method for examination.
The color shifting originally is an inherent property of a LCD and the visible light corresponds to a wavelength range of 400-700 nm, and a color range of violet through red. The white light is a mixture of the colors of the visible spectrum and black is a total absence of light. In LCD display, the angular color uniformity of a display affects the perceived image quality significantly. It is because the human eye is more sensitive to color changes than luminance or contrast changes (Teunissen, Zhong, Chen, & Heynderickx, 2009). Several previous researchers efforts to improve the color uniformity have been based on the compensation of retardation in the wavelength range. They focused on the angular shifts of the primary colors in the visible wavelength range. This paper follows the color shifting method (Wang et al., 2013) to generate a series of images. This study aims to enhance the endoscopy images for better screening rate of GI tract diagnosis.
Cloud computing and data storage is a solution to solve problems for telecommunications of medical image archives (Liu, Cao, Zhou, Mogel, & Documet, 2003; Aquino-Santos et al., 2008). Teng, Green, Johnson, Jones, & Treasure (2012) proposed medical image interchange and management framework based on industry standards and leading cloud computing platform, which developed for mobile medical imaging devices and applications to securely communicate with a cloud-based image storage and management service using standard DICOM protocol. The mobile application is developed using Google's Android OS and provides management of patient health records and medical images (Doukas, Pliakas, & Maglogiannis, 2010). In this paper, we use a standalone medical image retrieval application with Quick Response Code (QR-Code) (Rouillard, 2008) authentication based on the Android operating system, which helps doctors to conveniently assess the medical images via mobile phone.
3Mobile service systemWe designed an application based on Android operating system (Shabtai et al., 2010), which can allow doctors to retrieve medical images of patient after doctor authenticates identity (the proposed system architecture is shown in Fig. 1). The identical procedure has two ways to authenticate, that are typing identity information and scanning the QR-code of identity. The QR-code is not only used to authenticate, but also used in the searching patient number — every patient has their own QR-information security reason, and the code is encrypted by Advanced Encryption Standard (AES) (Bogdanov, Khovratovich, & Rechberger, 2011).
On the server, there is an application program interface (API) which is used to communicate application with the server. It handles up all requesting, responding information and encoding it to JSON (Crockford, 2006) format between mobile applications and server. The API is coding in PHP (Pleva), which is a popular general-purpose scripting language that is especially suited for web development. With processing, medical image (DICOM), this paper uses the DCMTK tools (Eichelberg et al., 2004) for extracts tags and image. DCMTK is a collection of libraries and applications implementing large parts the DICOM standard. It includes software for examining, constructing and converting DICOM image files, handling offline media, sending and receiving images over a network connection, as well as demonstrative image storage and work list servers. DCMTK is written in a mixture of ANSI C and C++, and it comes with complete source code and is made available as open source software.
3.1Android applicationTheir functions are authenticating identity; authenticate identity by QR-code, searching identity of the patient, searching identity of the patient by QR-code and daily logging. The connection between application and server is used in transport layer security (TLS) (Josefsson, 2006), which is currently used to protect data during transportation. The flow chart of Android application is shown in Figure 2.
3.2Server and database
To build and simulate data on own private cloud based on Apache server (Mockus, Fielding, & Herbsleb, 2000). DCMTK tools will be executed and extract information when user requests information by API. Figure 3 is data flow chart of API, which is explaining for requesting and responding information. For example, the API will respond medical image and tags of DICOM from database and DCMTK tools when user is searching identity of patient.
3.3Back end management interface
The manager of organization can create user, delete user, build the QR-code of identity with AES encrypt, load medical images of patient to database and check out daily logging. The daily logging contains following schemas: user, date, time, and type of access and International Mobile Equipment Identity (IMEI) number. IMEI is a unique 15 digit code upon production on mobile devices and it can checked all known information regarding manufacturer, model type, and country of approval of a handset.
There is a problem during the system encodes cipher to QR-code with AES encrypt, that the cipher cannot be decrypted after decode the QR-code, because the cipher contains the character which is out of code list, and we used Base64 encoding (Josefsson, 2006) to solve the problem. Base64 encoding is a group of similar binary-to-text encoding schemes that represent binary data in an ASCII string format by translating it into a radix-64 representation. Figure 4 compares Base64 with Non-Base64.
4UV shifting
In this section, we first briefly introduce the approach for image expansion of single endoscopy in UV space (Wang et al., 2013). The RGB color image first transforms to YUV color space. Moreover multiply a varying matrix A3×3 to generate a set of color images with the same Y channel. Then, we assemble an enhanced color endoscopy image via the ranking method to choice highest scalar values in red, green and blue channels, respectively.
Scalar values are quantity defined only by its magnitude within a processing mask. The polyp area of endoscopy image can be evidently distinguished in the proposed ranking method.
Let f(x) ∈ X is a cropped candidate image area to be enhanced and X is the single input endoscopy image. Suppose A ∈ R3×3 that maps the sensor YUV image to the linear Y U’V’ shifting space and
All formulae should be numbered with consecutive Arabic numbers within parenthesis.
where A is to expand multiple images by multiplying the matrix in UV space and δ1-δ4 are linear varying values from 0.001 to 1.000 with 0.001 resolution. In other words, the formula is to change the values of chrominance and chroma using a linear transform matrix.
Then the ranking values are computed rely on the scalar values, which are the edge energy of a set transformed image and can usually be discerned from the gray scale representation of the area alone, which represents the luminance band of the YUV color space.
5Contrast computingThis section details the selection of a highest contrast endoscopy image from a series of images generated by image expanding in section IV. The visual effect is more clearly proportional to the higher contrast value. The contrast is the reflecting of the clarity and the degree of texture of an image. We crop an area of the input endoscopy image and then apply the gradient computing for the crop image. The crop area can be used to apply the second order of GLCM method and the contrast values are calculated for the expanded crop images. The GLCM matrix is calculated from the intensity values of the selected crop area. The matrix capture properties of the selected crop area, but they are not directly useful for further analysis. It is then further used to compute the contrast value.
6Enhanced image fusionAn approach to image fusion in color includes performing image fusion separately and independently on each color channel, then providing the resulting three color channels and a contrast image as a single color image. In practice, this works in the following steps: we assemble an enhanced endoscopy image via the ranking method to choice highest scalar values in red, green, blue and a highest contrast images, respectively. The scalar values of red, green and blue are the quantity defined only by its magnitude within a processing mask (section IV) and the highest contrast computed by GLCM in section V.
To assemble an enhanced endoscopy image, four adaptive parameters are used for computing the enhanced image, which is defined as
where i is the selected crop area of an input endoscopy image and the adaptive process, labeled σ1 to σ4, expresses a family of parameters that operated by the scalar values and contrast values. The is the enhanced crop area of an endoscopy image.
7Experimental results and performance evaluationThis paper proposed an adaptive imaging method for enhancing endoscopy image to discover the tiny vessels and GI polyps in an early stage. Figure 5 illustrates the procedure of the proposed adaptive imaging method. The input endoscopy color image is transformed to YUV color space. We are therefore based on the edge energy to compute the scalar and contrast values of the expanding images for their red, green, blue and gradient channels. The enhanced image is obtained using the corresponding scalar and contrast values.
The flowchart demonstrated the adaptive imaging method. The input endoscopy color image is transformed to YUV color space firstly. We therefore adopted edge energy to count each expanding in image for their red, green and blue channels. The gradient image is used to compute and select highest contrast. The enhanced image is combined using the scalar and contrast values.
We setup a QR-code first of identity, which is shown in Figure 6, and the application screens are shown in Figure 7. The application is named “Ds Viewer”. The personal information of patient has replaced QR-code, and it only can be read by application because it has encrypted by AES. The cipher is encoded in Base64 before encoding QR-code. It makes sure to protect privacy of patient in the ward, and the key of AES is set on the server. It are presented authentication, searching patient and showing list of result that screens in Figure 5, and user can use button of QR-code for application both in first and second screen. The third screen is showing a list of results that contains medical image, name of patient and date. After tapping one of third screen item, then application will show up the fourth screen and it shows medical image of patient which is selected.
Privacy and information security are very important, we possibly make system accord the Health Insurance Portability and Accountability Act (HIPAA) (Huber et al., 2009). For above problems already concern in our system that using TLS between application server of connection, encrypted QR-code and daily logging.
7.2DatasetThere is 53 endoscopy image dataset used to evaluate our approach created by Division of Gastroenterology in Chang Bing Show Chwan Memorial Hospital. The dataset is recorded by FICE EPX-4400 machine to videos with normal white light and the generated FICE Channel 4 during clinical screening. The videos are then used for cropping images for evaluating the proposed endoscopy enhancing method. The dataset is a collection of 21 normal, 20 tubular adenoma, 10 hyperplastic polyp and 2 adenocarcinoma endoscopy videos for testing the image enhancement method.
7.3Adaptive imaging resultsIn this paper, we develop an image enhancement method based on UV shifting in YUV color space and edge energy for a cropped processed image area extracted from endoscopy image. We have implemented the UV shifting and contrast computing procedures for assembling a single enhanced endoscopy image. The proposed method is a pure image processing technique based on color expanding of a cropped candidate polyp area. The image resolution is 720 p recorded by FICE EPX-4400.
In Figure 8A, the input GI endoscopy image is with tiny vessels of a normal subject and Figure 8B is enhanced by FICE Channel 4. Channel 4 was selected because it is generally deemed to the best wavelength for this assessment (Curvers, Kiesslich, & Bergman, 2008), and a study (Coriat et al., 2008) reports that the FICE Channel 4 images were significantly better than conventional ones. Computed virtual chromoendoscopy enabled better analysis of the pit pattern and the normal-pathological mucosal junction. Figure 8C shows the enhanced crop image significantly better than conventional white light and FICE images, and it is suitable for clinical screen. Figure 9 is the enhancement and comparison of hyperplastic polyp, and the results show the very significantly improvement, better than conventional and FICE images even observed by human natural eye. The experiment demonstrates the proposed adaptive imaging method is efficient and with better contrast for observing the polyp for a colonoscopy.
The experiment demonstrates the proposed adaptive imaging method applied to observe the vascular polyp of the image pair five images in our dataset. A: the traditional gastrointestinal endoscopy image of a normal patient. B: the endoscopy image with enhanced area by FICE Channel 4. C: the endoscopy image with enhanced area by the proposed contrast adaptive imaging method.
The experiment demonstrates the proposed adaptive imaging method applied to observe the hyperplastic polyp of the image pair 31 image in our dataset. A: traditional gastrintestinal endoscopy image of a normal patient. B: endoscopy image with enhanced area by FICE Channel 4. C: the endoscopy image with enhanced area by the proposed contrast adaptive imaging method.
The dataset is used to evaluate and compare the performance of the proposed enhancing endoscopy image. The contrast values are adopted for comparison of image enhancement and the values are computed from the crop area of the normal, FICE Channel 4, and the proposed image enhancement method. Figure 10 shows the mean and standard deviation comparison of the 53 image pair of the dataset, and on Figure 11 are the contrast values of the normal, FICE channel 4 and the proposed image enhancement method, respectively. The entropy evaluation of white light, FICE Channel 4 and the proposed adaptive imaging are shown in Figure 12.
This paper proposed an image enhancement method to better the regular GI tract diagnosis. The proposed adaptive imaging method is a practical application in the early diagnosis of cancerous area. The adaptive imaging method has shown promising performance of evaluating on FICE images for real patients colonoscopy videos. For the actual patient testing, the institutional review board (IRB 1010202) is proved for evaluating our method in advance (colon images). The results show the proposed method as useful and efficient for colonoscopy diagnosis, especially for small tumors. The proposed adaptive imaging method can be extended to develop a tiny tumor search, and an alarm system will be considered in the future. The results demonstrate the proposed system works well for both PACS system and DICOM image processing in colon cancer evaluation. For more real testing and connection of PACS with colon images, we need to submit the other article of IRB agreement and evaluate the usability of the proposed system.
In this paper the prototype, a service system of medical image retrieval application with QR-code authentication based on Android OS and private cloud, can help personal clinics and clinicians in the remote area develop their own mobile retrieval system, and it will also improve proprietary mobile devices with existing clinical systems. Integration of mobile and cloud systems will lead to better clinical decision. In the future, we will continue to improve this prototype, and make it can read 2D/3D medical image, or add tools for image adjust in the application that will be similar to fully medical image workstation.
The main limitation of the present method is that it is unable to process images for real-time diagnosis in endoscopic room. This is because the current development focused on image enhancement algorithms, and this issue will be considered in the future.
The support of this work was given in part by the National Science Council of Taiwan, R.O.C. under Grant NSC-101-2218-E-758-001 and NSC-2221-E-442-001-MY2 and is gratefully acknowledged.