Activity Recognition From IR Images Using Fuzzy Clustering Techniques

INTERNATIONAL JOURNAL FOR TRENDS IN ENGINEERING & TECHNOLOGY
VOLUME 5 ISSUE 1 – MAY 2015 - ISSN: 2349 - 9303
6
Activity Recognition From IR Images Using Fuzzy
Clustering Techniques
Savitha Suman
Karunya University, Department of EIE,
sonia446.savitha@gmail.com
D. Pamela
Karunya University, Assistant Professor,
Department of EIE
pamela@karunya.edu
Abstract— Infrared sensors ensures that activity recognition is possible in the day and night times. It is used especially
for activity monitoring of older adults as falls are more prevalent at night than the day. This paper focus on an application
of fuzzy set techniques and it is capable of accurately detecting several different activity states related to fall detection and
fall risk assessment and it also includes sitting, standing and being on the floor to ensure that elderly residents gets the
help they need quickly in case of emergencies. Fall detection and fall risk assessment is used for an aging in place facility
for the elderly people. It describes the silhouette extraction process, the image features , and the fuzzy clustering
technique.
Index Terms— Activity labeling, Fuzzy clustering, Image moments, Infrared camera .
——————————  ——————————
1 INTRODUCTION
Activity recognition is done on vision sensors under normal
illumination and low lightning conditions that indicate the severe fall
risk of older adults .Since nocturnal activities are an important aspect
of an independent lifestyle it will create a potential problem. This
shows the need for surveillance techniques that can be implemented in
the absence of light or under negligible lighting conditions. Fall
detection and fall risk assessment has given much more importance
and dynamic infrared sensors are also involved.
Fuzzy clustering techniques are mostly unsupervised methods that
can be used to organize data into groups based on similarities among
the different data items . All the clustering algorithms do not rely on
assumptions common to conventional mean, median, average,
standard deviation methods etc., and it undergoes statistical sharing of
data, and therefore they are useful in situations where little prior
understanding exists. The potential of clustering algorithms to reveal
the underlying structures in data can be exploited in a wide avariety of
applications, including classification of an activities, processing of an
image, recognizing of a pattern, modeling and identification.
2 OVERVIEW
In this system, background subtraction techniques using mixture of
Gaussian models with texture features are used on the raw image
data to separate the foreground from the background and the resulting
silhouette are taken as inputs to the automatic activity segmentation
system. It is to build an automated video surveillance system to
continuously monitor elderly persons as they perform their day-to-
day activities, maintaining their privacy by using silhouettes instead
of raw images for further analysis. It has been shown previously that
silhouettes addresses the privacy concerns of elderly persons
participating ,and increases their willingness to accept video
monitoring systems in their households [1]. From these silhouettes,
image moments are extracted, which are then clustered to produce
fuzzy labels in the basic activity categories.
Clustering itself can be concluded as a fuzzy concept[2].
Depending on the implemented clustering algorithm the criterion
function to be optimized changes, and the nature and shape of the
clusters vary.
While clustering was employed in some of the above mentioned
techniques and silhouettes were extracted in others, combination to
segment activities are used nowhere. By using fuzzy clustering
techniques in identifying sit-to-stand frames using image moments
on visible light data has inspired the work.
Fig .1 Block diagram of an algorithm
The paper is organized as follows. Silhouette extraction and
moment description is used for clustering present in section 3.
Section 4 describes the fuzzy clustering techniques used for activity
————————————————
 Author name is currently pursuing masters degree program in electric
power engineering in University, Country, PH-01123456789. E-mail:
author_name@mail.com
 Co-Author name is currently pursuing masters degree program in electric
power engineering in University, Country, PH-01123456789. E-mail:
author_name@mail.com
(This information is optional; change it according to your need.)
Images (from
any source)
Pre-processing
&
Silhouette
Extraction
Extraction Of
Image Moments
Activity State
Identification
Fuzzy Clustering
Of
Image Moments

7
analysis and Section 5 briefly describes the experimental setup and
results using the standard web cameras under normal illumination.
The conclusions and future work present in Section V6.
3 METHODOLOGY
3.1 Silhouette Extraction
Silhouette extraction is a background change detection technique
whose accuracy depends on how well the background is modeled.
The background subtraction method implemented using color and
texture features employs shadow removal for greater accuracy binary
morphological operations are used to fill up holes and remove noise
from the extracted silhouettes [3]. After obtaining the silhouettes
from the image steps, the next step in the algorithm is extracting
image moments as shown in the block diagram. Image moments are
applicable in a wide range of applications such as pattern recognition
and image encoding.
One of the most important and popular set of moments is the set
of Hu moments [4]. This Hu moments consists of seven central
moments taken around the weighted image center. In particular, the
first three Hu moments are more robust than the other Hu moments
in the presence of noise and were used in this analysis. scale and
rotation invariant in Hu moments, makes them extremely robust
and applicable in different scenarios [5]. However, they are non-
orthogonal in nature; i.e., their basis functions are correlated, making
the information captured redundant. In contrast, the Zernike
orthogonal moments comprise image moments with higher
performance in terms of noise resilience, information redundancy
and capability of reconstruction.
3.2 Image Moments For Activity Classification
The image moments using here for the classification of an activities
is Zernike moments [6]. These moments are briefly described below:
Zernike Moments: Zernike moments are the mappings of an image
onto a set of complex Zernike polynomials. Since Zernike
polynomials are orthogonal to each other, these moments can
represent the properties of an image with no redundancy or overlap
of information between the moments. These moments are
significantly dependent on the scaling and translation of the object in
an Region Of Interest.
4 FUZZY CLUSTERING
Fuzzy clustering techniques are used of partition data on the basis of
their closeness or similarity using fuzzy methods. As opposed to the
hard clustering, each element can belong to a certain cluster with
varying degrees of membership.
The Gustafson-Kessel [7] fuzzy clustering technique was
implemented on the Zernike image moments . With applications in
several fields such as image processing, recognizing the patterns,
identification of systems, and classificaion it has become very
popular clustering algorithm [8]. The Gustafson-Kessel clustering
technique is that it is well suited for the ellipsoidal clusters produced
by the moments . This clustering technique is an extension of the
fuzzy c-means algorithm in which each cluster has its own unique
co-variance matrix which makes it robust and it is more applicable
for various data sets which contains ellipsoidal clusters of different
orientations and sizes [9]. As the basic approach for clustering is well
known it is summarized for completeness.
A membership function indicates the membership degree of a
particular element regarding as an event. The level of influence of an
element is indicated between 0 and 1 and it is the membership
range. In order to reduce the number of samples and thus to
minimize the amount of input signal, used for fuzzy clustering.
Algorithm:
1. Fix c = number of clusters &initialize the iteration counter t
= 1.
2. For all the data points and for each of the clusters we have to
initialize the membership matrix U. (The initialization is
explained further in this section.)
3. Do.
4. Compute the cluster centers using
5. Compute the covariance matrices for each of the clusters as
in
6. Update the partition matrix
7. Increment the iteration counter t.
8. Until || μ (t) −μ (t − 1) || < Є or t > where Є is the
maximum permissible error and is the maximum
number of iterations specified.
Here, μ(t) is the vector of all centers, and the distance norm
employed to determine convergence is the standard Euclidean
distance. An important point to be noted is that it is essential to
initialize the membership values to random values but with the
means equal to 1/c (where c is the number of clusters) and standard
deviation equal to 1 so that the algorithm converges in a faster rate.
For ensuring equal importance to each of the moments Standard
essential is used. Otherwise, the algorithm would focus on the
moments with the highest range. It is worth noting that since we
constrain the determinant of the co-variance matrix to be 1, we
impose restrictions on the size of the clusters, and as a consequence,
the identified ellipsoidal clusters have to be of similar size [10].
Parameters of the Gustafson–Kessel Algorithm:
As for the FCM algorithm (except for the norm including matrix A,
which is automatically adapted) same parameters must be specified
to the number of clusters c, termination tolerance and fuzzy exponent
m. Cluster volume 𝛑 is the additional parameter. Without any basic
information, for each cluster 𝛑 is simply fixed at 1. The main
disadvantage of this setting is that due to the constraint Gustafson -
Kessel algorithms, clusters of equal volumes can be found.
5 EXPERIMENTAL SETUP AND RESULTS – WEB
CAMERAS WITH INFRARED LIGHT
5.1 Web Cameras Using Infrared Illumination:

8
An image sequence database was at a resolution of 640 × 480 with
IR camera . We can recognize activities even with the degradation in
silhouette quality. In the visible spectrum, these images are
completely black. IR lighting was used with a wavelength of 850nm;
in total, there were 216 individual IR LED's distributed between the
two lamps with a total power draw of approximately 20w. There is
no IR filter on the camera lens.
The possible activities practiced at night and included them in our
data collection: walking, standing without hand motions, a hand
motion, sitting down and standing up, sitting on a sofa, going to bed,
and getting up from bed. The data collection also contained the
following situations on the bed: sleeping (lying on the bed), being
sleepless (flipping with some movements, i.e., with a little toss and
turns), sitting on the bed, and transitioning from sitting to lying on
the bed. In addition, four abnormal activities (falling) were included:
walking in the room and falling to the ground due to loss of balance,
slipping when trying to get up from a chair, falling when trying to get
up from a bed, and falling out of the bed when sleeping. The frame
rate is 3 frames/s.
5.2 Experimental Results:
Preliminary experiments were conducted to establish the input
parameters and best features are used for these data. Several
participants performed from a variety of activities. Silhouettes are
extracted from the raw image sequences, and the moment features
were computed. The GK clustering technique requires the number of
clusters to be specified as an input parameter. By demonstrating the
clustering of the Zernike moments using the GK algorithm with the
number of clusters initialized to the number of activities will be
yielded [6]. Since single camera images are used here, the activities
of walking and standing cannot be differentiated in general; thus,
they are grouped together as ―upright‖ frames for the purpose of
activity recognition.
The clustering results of one data sequence using an input of
three clusters. The clustering results with the X-axis that indicates
the frame number in the sequence and the Y -axis that indicates the
cluster number after hardening the membership matrix. These results
have been color coded for display purposes and black-colored points
indicate the areas where the points are cowardly clustered and it is
evident that the two clusters obtained represent the ―sit‖ and ―on the
floor‖ kind of activities, without any prior information, we couldn't
identify which cluster indicates which activity. This scenario had
involved a participant who performs several actions in an unlit room.
The three activities performed in the scenario below were night
time activities of moving around in the room (upright), sleeping on
the bed, and then falling onto the floor. After fuzzy clustering and
partition hardening activities are well separated . However, in this
scenario, the activity ―fall‖ is equivalent to ―on the floor‖ since no
other parameter has been taken into consideration which could
differentiate between the two activities. The detection of the
transition frames as well as identifying the activity state using
prototype matching. Using prototype matching, the activity states
were identified and the sample results are taken. In the raw images,
the person has been circled in yellow to distinguish the person from
the background for visualization. When compared with the standard
illuminated data silhouettes are fairly noisy and their shapes are quite
different. However, the results show strong clustering of the image
moments obtained from the same activity states. This makes activity
analysis possible, even in the dark using fuzzy clustering. Some of
the color-labeled image frames with blue indicating an upright
frame, red signifying on the bed activity label, and pink representing
on the floor activity label.
All the analyzed pixels can be given to the Fuzzy Clustering
algorithm . In this the images will be processed in four steps:
(1) Pre-processing
(2) Silhouette extraction
(3) Activity classification
(4) Fuzzy clustering
(1)Pre-processing: Pre-processing is used to find the foreground
detection and it is done in three process.
(i) Gray image: Gray image is also known as infrared images.
(ii) Binary image: It is a digital image that has only two possible
values for each pixel. Typically two colors used for a binary images
are black and white though any kind of colors were used. The color
used for the object in the image is the foreground color while the
rest of the image is the background color.
(iii) Edge Detection: It is used to detect a wide ranges of edges in
images.
(2) Silhouette Extraction: Silhouette extraction is used for detecting
the subtraction of the background. The dark shape and outline of
someone will be visible against in lighter background and especially
in dim light.
(i) Image Dilation: It is developed by binary images. Binary image
is to gradually enlarge the boundaries of regions of foreground
pixels.
(ii) Image Filling: It is used for closing the regions while keeping
Fig.2 Gray Image
Fig.3 Binary Image
Fig.4 Edge Detection

9
initial region sizes.
(3) Activity Classification: Activity Classification is done by
extracting the image moments by Hu moments . Hu moments is
used for activity classification.
Hu Moments: Feature moments and normalized central
moments can be applied on an image to calculate the seven invariant
moments defined by Hu moments in terms of centralized moments
for purpose of shape recognition.
The function to be used directly by the user ―A‖ is a 2D matrix
representing an image. Inside this function another function cent
moments (p, q, A) is called normalized moments. Based on
normalized central moments, Hu moments will give seven moment
invariants:
• M1=n20+n02
• M2=(n20-n02)^2+4*n11^2
• M3=(n30-3*n12)^2+(3*n21-n03)^2
• M4=(n30+n12)^2+(n21+n03)^2
• M5=(n30-3*n21)*(n30+n12)*[(n30+n12)^2
3*(n21+n03)^2]+(3*n21-n03)*(n21+n03)*[3*(n30
+n12)^2-(n21+n03)^2]
• M6=(n20-n02)*[(n30+n12)^2-(n21+n03)^2] +4 *n11
*(n30+n12)*(n21+n03)
• M7=(3*n21-n03)*(n30+n12)*[(n30+n12)^2-3*(n21
+n03)^2]-(n30+3*n12)*(n21+n03)*[3*(n30+n12)^2-
(n21+n03)^2]
Output of Hu moments of different activities:
 First image of Hu moments:
M =
1.8648
2.8531
0.0248
0.0289
0.0008
0.0487
-0.0000
 Second image of Hu moments:
M =
0.4353
0.1240
0.0065
0.0030
0.0000
0.0000
0.0011
 Third image Hu moments:
M =
0.1946
0.0041
0.0002
0.0001
-0.0000
0.0000
0.0000
 Fourth image Hu moments:
M =
0.3562
0.0824
0.0053
0.0009
0.0000
0.0002
0.0000
 Fifth image Hu moments:
M =
0.4206
0.1203
0.0108
0.0014
0.0000
0.0004
0.0000
 Sixth image Hu moments:
M =
0.3802
0.1113
0.0027
0.0006
0.0000
0.0001
0.0000
 Seventh image Hu moments:
M =
0.3600
0.0993
0.0016
0.0007
0.0000
0.0002
0.0000
 Eighth image Hu moments:
M =
0.3829
0.0804
0.0091
0.0028
0.0000
-0.0006
0.0000
Fig.5 Image Dilation
Fig.6 Image Filling

10
 Ninth image Hu moments:
M =
0.2601
0.0324
0.0020
0.0002
0.0000
0.0000
0.0000
 Tenth image Hu moments:
M =
0.2424
0.0241
0.0013
0.0001
0.0000
0.0000
-0.0000
 Eleventh image Hu moments:
M =
0.2986
0.0456
0.0021
0.0012
0.0000
0.0002
0.0000
(4) Fuzzy Clustering: Fuzzy cluster algorithm is used to identify
and to classify the Zernike moments. The fuzzy sampling algorithm
extracts the most relevant aspects of the Zernike moments , without
loss of important information. In this work, the clustering processes
is used to indicate the value and location of each cluster, without
requiring the generation of functions or rules corresponding to each
cluster. It is a great advantage since it requires less computational
effort. It applies the fuzzy clustering algorithm to the GK on Zernike
moments, that are clustered to 4 clusters.
The GK clustering technique requires the number of clusters to be
specified as an input parameter. Using the GK algorithm it is shown
that clustering the Zernike moments with the number of clusters
initialized to the number of activities outputs gives best results. Since
only images of one camera are used here, the activities of walking
and standing cannot be differentiated in general so, they are grouped
together as ―upright‖ frames for the purpose of activity recognition.
It shows the clustering results of one data sequence using an input of
2 clusters.
Figure 8 shows the clustering results with the X axis
indicating the frame number in the sequence and the Y axis
indicating the cluster number after hardening the membership
matrix. The results have been color coded for display purposes. In
both the figures, the points colored red represent frames of a person
sitting on the couch and the blue colored points represent the image
frames indicating the person on the floor. Black colored points
indicate the areas where the points are densely clustered.
6 CONCLUSION AND FUTURE WORK
Fuzzy clustering methods have been employed in detecting activity
frames in different environments, both controlled as well as
uncontrolled . A classifier was constructed and the Zernike Moments
were obtained by using clustering results and membership results by
frame number.
In the future work , the simulation and implementation of
detecting the different activities i.e., sitting, standing, on the floor
etc., is displayed on the LCD in processors.
REFERENCES
[1] J. Keller and I. Sledge (2007), ―A cluster by any other name,‖ in
Proc. IEEE Proc. Fuzzy Inf. Process,427–432.
[2] G. Demiris, D. Parker, J. Giger, M. Skubic, and M. Rantz (2009),
―Older adults’ privacy considerations for vision based recognition
methods of eldercare applications,‖ Technol. Health Care, 17(1),
41–48.
[3] A.K. Jain (1989), Fundamentals of Digital Image Processing.
Englewood Cliffs, NJ, USA: Prentice-Hall.
[4] A.El Maadi and X. Maldague (2007), ―Outdoor infrared video
surveillance: A novel 3dynamic technique for the subtraction of a
changing background of IR images,‖ Elsevier,Infrared Physics
Technology 49, 261–265.
[5] M. K. Hu (1962), ―Visual pattern recognition by moment
invariants,‖ IRE trans.Inf.Theory, 8,179-187.
[6] T. Banerjee, J. M. Keller, M. Skubic, and C. C.Abbott(2010), ―Sit-
to-stand detection using fuzzy clustering techniques,‖ in Proc. IEEE
transaction on Fuzzy Systems / Computer Intel, 13(4), 1–8.
[7] D. E. Gustafson and W.C. Kessel (1979) , ―Fuzzy clustering with a
fuzzy covariance matrix,‖ in proc. IEEE Annu. Conf. Decision
Control, San Diego, CA, USA, 761-766.
[8] M. J. Lesot and R. Kruse (2006), ―Gustafson-Kessel-like clustering
algorithm based on typicality degrees,‖ presented at the Int. Conf.
Inf. Process. Manag. Uncertainity, Paris, France.
Fig.7 GK on Zernike Moments and clustering results into 2
clusters.
Fig.8 GK on Zernike Moments and clustering results into
membership results by frame number.

11
[9] R. Babuska, P.J. van der Veen, and U.Kaymak (2002), ―Improved
covariance estimation for gustafson- Kessel clustering,‖ in Proc.
IEEE Int. Conf.Fuzzy Syst.,1081-1085.
[10] S. Theodoridis and K. Koutroumbas (1999), Pattern Recognition.
San Diego, CA, USA.

Activity Recognition From IR Images Using Fuzzy Clustering Techniques

Recommended

More Related Content

What's hot (15)

Viewers also liked (19)

Similar to Activity Recognition From IR Images Using Fuzzy Clustering Techniques (20)

More from IJTET Journal (20)

Recently uploaded (20)

Activity Recognition From IR Images Using Fuzzy Clustering Techniques