An Efficient Content Based Image Retrieval using EI Classification and Color Features

Abstract

An efficient method for image search and retrieval has been proposed in this study. For this purpose images are decomposed in equal squares of minimum 24 × 16 size and then edge detection is applied to those decomposed parts. Pixels classification is done on the basis of edge pixels and inner pixels. Features are selected from edge pixels for populating the database. Moreover, color differences are used to cluster same color retrieved results. Precision and recall rates have been used as quantification measures. It can be seen from the results that proposed method shows a very good balance of precision and recall in minimum retrieval time, achieved results are comprised of 66%-100% rate for precision and 68%-80% for recall.

1

Introduction

Digital images are everywhere. The immense use of digital camera has led to millions of image databases. Digital images are used in countless applications like in education, agriculture, medical sciences, defense, and entertainment and so on. Images features color, texture, shape, spatial layout etc are used as content to search and match the images [1-3]. Therefore, there is an intense need of image searching, matching and retrieval at different steps of applications. A prominent example of this is in agriculture, an application that is classification of healthy and ill seeds which needs image matching with a standard healthy seed. An application of image retrieval named comic image retrieval system; introduced in [4] utilized the image features for image retrieval. Content based image retrieval known as CBIR has been getting great attention of researchers since the last decades because of its usage in a vast number of applications as one is discussed in [5].

An efficient method for image content detection has been presented in [6] that used pixel value histogram and found the correlation coefficient between those values. Corner matching is also a powerful technique in context of image match estimations [7]. A powerful descriptor based on angle edge and histograms is also presented for retrieval of images [8]. Application of Pythagorean theory to image yielded very valuable characteristics for the matching, searching and retrieval of images. Another efficient method for image retrieval is discussed in [9] by combining the multiple features of color, shape and relevance feedback. Shape descriptors are stronger than other features like texture and color in semantically describing the image [10]. However, the challenge is the extraction and representation of shape information accurately [11]. Performance of shape descriptors is categorized as qualitative and quantitative descriptions. Qualitative description involves the degree of accuracy in representing the true shape information and quantitative description involves the amount of data needs to be indexed for searching the image [12].

CBIR has also been extended up to 3D image shape retrieval [13] but still 2D shape retrieval is practical because of its superior quantitative characteristics in many tasks like pattern recognition, character recognition and image matching etc. [14]. Shape descriptors are broadly categorized into two groups namely contour based descriptors and region based descriptors. Contour based descriptors mainly use the extracted boundary information of image whereas region based descriptors use all information existing in the image region. Both are beneficial and necessary according to nature of task.

The challenging task in designing a descriptor is coping with geometric transformations and semantic gap. In [15] color distances are used as color features but both [15, 16] lack the handling of geometric transformations. An effort has been put for reducing the semantic gap in [17]. Color features have been used to propose an improved CBIR system [18].

It has been noted that only the color features are not sufficient for efficient and effective image retrieval. If color features are combined with shape features, it can become a very strong descriptor for image search. In [19] the shape and color features are combined to achieve an effective retrieval system. In this paper image edge features are combined with color features in order to achieve the desired results. The organization structure of rest of the paper is as follows: in Section 2 proposed work has been discussed in detail. Section 3 presents results and discussion and Section 4 concludes the overall method. References are included in Section 5.

2

Proposed work

The typical steps involved in CBIR system are feature extraction, populating the database with those features, extracting the same features from query image, comparing the features of query image and database features and on the basis of this comparison the images are finally retrieved. The proposed solution also follows the above mentioned steps. Feature extraction involves image decomposition and EI classification. Clustering is then used for grouping the retrieved results on the basis of color. The structure of proposed method is shown in Fig.1.

Figure 1.

Generalized Structure of Proposed Method.

2.1

Image decomposition

Let I denotes the image with dimensions m × n Image is decomposed into equal squares known as cells C_ij; such that m,n,i, j ∈ N.

m/i = 0 and n/j = 0 means i, j are one of multiples of m,n but the minimum size of C_ij will be 24×16 The main reason of this decomposition is to control the orientation of image edges. If image has been passed under any orientation like rotation, translation or scaling, it does not affect the descriptor. By collecting local features in this sense, decomposition plays a very important role in the solution going to be proposed. An example of decomposed image is shown in Fig.2.

Figure 2.

Example of decomposed image.

2.2

Ei (edge & inner) pixels classification

EI classification is done to classify the image pixels into two classes namely edge pixels and inner pixels. The class of edge pixels is used to extract required edge related features. Fig.3 depicts the EI classification.

Figure 3.

Pixels Classification.

For each C_ij gradient g_ij = ∇C_ij is computed using canny edge detector.

Now C_ij is a combination of black & white pixels (0, 1), where c_ij is 1 that is classified as edge pixel and where c_ij is 0 that is classified as inner pixel. In this way the background is separated from foreground pixels. This leads to the simplicity of features extraction process.

For the whole image I_mn for each C_ij, i&j are recorded for edge pixel. To populate the database of search images sum of i & j is entered into the database as feature.

2.3

Image search, clustering and database

Let Q_mn be the query image and q_ij are decomposed parts of query image. Decomposition of query image is done in the same way as done with database images. Same features are computed for query image and compared using Euclidean distance formula, the smaller the difference the similar the images.

Disk folder structure has been used to store the downloaded images. All images have been stored in JPEG format. Inner Pixels are used for clustering the images before presenting to users. For this color distances are used by selecting inner pixels. The positive distances Red-Green, Green-Blue and Red-Blue are used for this purpose.

Then overall distance is calculated as follows:

These features are also stored in database with search images and retrieved results are clustered in a group of same color images on the basis of value of D, where the users have an opportunity of selecting the color of their own choice.

3

Experimental results and discussion

The COREL database [20, 21] is a huge collection of color images of different categories consisting of 10,000 images. The dataset selected for experimental results of the proposed technique contains 2000 images. Dataset has been divided into 20 categories with 100 images in each category. Experimental images consist of flowers, animals, landscapes, buildings, cars and plants etc. System is CORE i3, 2.8 GHz, 2 GB RAM with Windows 7 operating system. System has been designed in such a manner that it retrieves 80 images against a query image. The tool used is MATLAB 7.0.

Widely accepted Precision and Recall rates are used as quantification measures.

Precision=relevant retrieved images/all retrieved images

Recall=relevant retrieved images/all relevant images in database

Fig.4, Fig.5 and Fig.6 show the results of retrieved images against a query image given to the system. Top 10 to 15 results have been shown here only due to the space limitations.

Figure 4.

Query Image and Top 15 Retrieved Results for Image Parrot, Results are Clustered on Basis of Inner Pixels’ Colors.

Figure 5.

Query Image and Top 10 Retrieved Results for Image Waterfall, Results are Clustered on Basis of Inner Pixels’ Colors.

Figure 6.

Query Image and Top 15 Retrieved Results for Image Red fort, Results are Clustered on Basis of Inner Pixels’ Colors.

First a query image of parrot is given to search. and retrieve the related images.

A query image of Red fort is given to system to search and retrieve the related images.

Implementing a CBIR system has many challenges to be faced. These include Size of images, Angle of orientation, Color descriptors and Selection of unique identifier for deciding the class of object and distance of acquisition instrument from the image which are of great importance. It is a very challenging task to design a robust algorithm for image search. In the proposed solution best possible effort has been made to make the retrieval process efficient. As one can easily see from the results that proposed solution provides a good ratio of relevance and a good balance between precision and recall, the angle of orientation can be one of 30, 45, 90, and 135. It has been set to 90 for all images before populating the database and each time for query image also. It really helps in accurate retrieval process. Allowed dimensions of images are between 640×480 pixels and 1024×840 pixels. In case 60% or above similarity between query image decomposed parts and database images is found, the query image is considered to be a relevant image. Table 1 and Table 2, Fig.7 and Fig.8 provide the results of Precision and Recall rates and computation time respectively.

Table 1.

Image Retrieval Results for Different Query Images.

Query Image	Total Retrieved Images	Relevant Retrieved Images	Irrelevant Retrieved Images	Precision	Recall
Parrot	80	76	4	0.95	0.76
Waterfall	80	68	12	0.85	0.68
Redfort	80	80	0	1.00	0.80
Rose	80	72	8	0.90	0.72
Tiger	80	53	27	0.66	0.27
Sky	80	77	3	0.96	0.77
LandScape	80	68	12	0.85	0.68
Fruit	80	79	1	0.99	0.79

Table 2.

Retrieval Time for Different Query Images.

Query Image	Total Relevant Images	Total Retrieved Images	Computation Time (sec)
Parrot	100	80	0.755
Waterfall	100	80	0.756
Redfort	100	80	0.787
Rose	100	80	0.982
Tiger	100	80	0.734
Sky	100	80	0.798
LandScape	100	80	0.765
Fruit	100	80	0.458

Figure 7.

Relationship between Precion and Recall.

Figure 8.

Comparison of Precision and Recall Rates.

There were 100 images for each category of images from parrot to fruit and there were 2000 images in database. The algorithm searches on the basis of defined features and compares the query image after extracting the same features from query image. Proposed solution is designed to retrieve 80 images per search. One can easily observe from the results that it extracts the most relevant images, higher the degree of relevance, greater the balance between precision and recall rates. Precision defines the ratio between relevant retrieved images and total retrieved images; recall is actual relevant retrieved images and total relevant images in database. Table 1 describes these ratios for all categories in database. Many experiments have been performed by querying different images of each category and the results of experiments have proven the consistency of proposed method for each type of query image.

Fig.7 shows the results of Table 1 graphically for analysis. Table 2 shows the maximum time for retrieval that is 0.7 sec and practically implementable.

For fruit category it is 0.4. Because of simple edge regions, it takes less time for searching and retrieving the images. Experiments have also been performed to record the behavior of retrieval process using color distance only as described in proposed solution. Table 3 and Fig.8 clearly show the difference of precision and recall rates for color and edge features combined and only color features. One can easily see that when color and edge features are combined the retrieval performance is better. Hence when shape features are combined with color features they make a stronger descriptor for image matching, searching and retrieval.

Table 3.

Comparison of Proposed Method with Retrieval Results of Color Features.

Query Image	Based on Color+Edge Features		Based on Color Features
	Precision	Recall	Precision	Recall
Parrot	0.95	0.76	0.45	0.26
Waterfall	0.85	0.68	0.35	0.18
Redfort	1.00	0.80	0.50	0.20
Rose	0.90	0.72	0.60	0.32
Tiger	0.66	0.27	0.36	0.20
Sky	0.96	0.77	0.56	0.12
LandScape	0.85	0.68	0.45	0.23
Fruit	0.99	0.79	0.79	0.39

4

Conclusion

Proposed solution has been successfully implemented and tested and obtained results are quite satisfactory. System accepts a color image about which no information exists in the system. It then analyzes it and produces similar results. This is a prominent feature of CBIR and of the proposed solution as well. Due to its computation simplicity proposed solution will definitely add an effective elucidation to the research area of CBIR. It has been proved that by combining shape features with color features a strong descriptor for image matching is obtained.