International Journals of Advanced Research in
Computer Science and Software Engineering
ISSN: 2277-128X (Volume-7, Issue-6)
Research Article
June
2017
A Guesstimate on Web Usage Mining Algorithms and
Techniques
Prabha.K*, Suganya.T
Department of Computer Science, Periyar University PG Extension Centre,
India
Abstract— The World Wide Web (WWW) is becoming one of the most preferred and widespread mediums of
learning. Unfortunately, most of the current Web-based learning systems are still delivering the same educational
resources in the same way to learners with different profiles. Web usage mining is defined as the application of data
mining technologies to online usage patterns as a way to better understand and serve the needs of web-based
applications. The performance of web information retrievals and web based data warehousing are boosted with the
extraction of information from the web using web mining tools.
Index Terms—Preprocessing, Classification, Clustering, Data cleaning, Supervised Learning, Unsupervised learning.
I. INTRODUCTION
Web mining aims to discover useful information or knowledge from the Web hyperlink structure, page content and usage
data. Web is taking an important place in human’s life and day by day it increases the number of information based on
the expectations of the customers using it. Daily updating is needed to fulfill the needs of the users. Web mining is used
to extract the web information that is needed by the users so that the necessary details can be fetched and utilized.
Automation is everywhere and in every field to avoid the human work in creation of anything. Web mining utilizes the
automatic way of information extraction from the World Wide Web according to the preferences [4]. Web mining can be
divided into three separate categories depending on the type of data to explore.Web Content Mining, Web Structure
Mining and Web Usage Mining.
The significant web mining applications are website design, web search, search engines, information retrieval,
network management, Ecommerce, business and artificial intelligence, web market places and web communities. Online
business breaks the barrier of time and space as compared to the physical office business. Big companies around the
world are realizing that e-commerce is not just buying and selling over Internet, rather it improves the efficiency to
compete with other giants in the market. This application includes the temporal issues for the users. According to
analysis targets, web mining can be divided into three different types, which are Web usage mining, Web content mining
and Web structure mining. Similar to Etzioni [7], suggest decomposing web mining into these subtasks, namely
 Resource Finding: It is used to extract the data from online text resources available on web.
 Information selection and pre-processing: This process transforms the original retrieved data into information.
 Generalization: Individual web sites as well as across multiple sites.
 Analysis: It involves the validation and interpretation of the mined patterns. A human plays an important role in
information on knowledge discovery process on web [4].
A. Web Content Mining
Web content mining is the mining, extraction and integration of useful data, information and knowledge from
Web page content. It describes the discovery of useful information from the web documents. In web content mining the
content may be text, image, audio, video, metadata and hyperlinks etc. Web content mining also distinguishes personal
home pages with other web pages. Research in web content mining encompasses resource discovery from the web,
document categorization and clustering, and information extraction from web pages [10].
Web Content Mining is the process of extracting information from the contents of Web documents. It examines
content of the web pages as well and web searching. Content data corresponds to the collection of facts a Web page was
designed to convey to the users. Web content may be unstructured (plain text), semi- structured (HTML documents), or
structured (extracted from databases into dynamic Web pages). Such dynamic data cannot be indexed and consist what is
called “the hidden Web”. A research area closely related to content mining is text mining.
B. Web Structure Mining
This kind of mining emphasizes on the data which describes the structure of the content. It is classified into two
types namely intra-page structure and inter-page structure. Intra-page structure means the existence of links within a
page. No separate page will be opened in this case. Inter-page structure involves the connection of one page with the
other page.
© www.ijarcsse.com, All Rights Reserved
Page | 518
Prabha et al., International Journals of Advanced Research in Computer Science and Software Engineering
ISSN: 2277-128X (Volume-7, Issue-6)
C. Web Usage Mining
Web servers record and accumulate data about user interactions whenever requests for resources are received.
Analyzing the web access logs of different web sites can help understand the user behavior and the web structure, thereby
improving the design of this colossal collection of resources. There are two main tendencies in Web Usage Mining
driven by the applications of the discoveries: General Access Pattern Tracking and Customized Usage Tracking. The
general access pattern tracking analyzes the web logs to understand access patterns and trends. These analyses can shed
light on better structure and grouping of resource providers. Many web analysis tools existed but they are limited and
usually unsatisfactory. We have designed a web log data mining tool, Weblog Miner, and proposed techniques for using
data mining and On-line Analytical Processing (OLAP) on treated and transformed web access files.
Applying data mining techniques on access logs unveils interesting access patterns that can be used to
restructure sites in a more efficient grouping, pinpoint effective advertising locations, and target specific users for
specific selling ads. Customized usage tracking analyzes individual trends [9, 3]. Its purpose is to customize web sites to
users. The information displayed the depth of the site structure and the format of the resources can all be dynamically
customized for each user over time based on their access patterns. While it is encouraging and exciting to see the various
potential applications of web log file analysis, it is important to know that the success of such applications depends on
what and how much valid and reliable knowledge one can discover from the large raw log data. Current web servers store
limited information about the accesses. However, for an effective web usage mining, an important cleaning and data
transformation step before analysis may be needed.
II. WEB USAGE MINING ALGORITHMS
A. Association Rules
Association rules are used for finding the correlations among web pages that frequently appear together in a user
browsing session. Apriori algorithm [1] is the most popular algorithm that expresses the frequent co-occurrence of web
pages together. These rules help in providing the recommendations to user as they visit the sites and web designers to
restructure web sites. These rules show that if page M is accessed by a user then it is very likely that he visits page N also
in same session. Algorithms used for association rules include maxi-mal forward references, Markov Chains, FP growth
and prefix span.
B. Clustering
The main purpose of clustering in web usage mining is to aggregate the similar sessions together [5], [11]. Self
organized maps, graph partitioning, ant based technique, K-means with genetic algorithms, EM- CFuzzy means
algorithm are the algorithms used for clustering the sessions. Mainly clustering is of two types usage clustering and page
clustering. Usage clusters discover user that have same browsing pattern where page cluster gathers the content related
pages together. This is per-formed by transforming the session into vectors of n elements where n may be no of pages or
page views. Afterwards distance measure is used to find out the similarity between sessions. Clustering helps in
personalizing the web site as it identifies the users with similar behavior.
C. Classification
Classification is supervised way of learning which maps the data items to one of many predefined classes [2].
Various supervised learning algorithms used for doing classification are decision trees, naïve Bayesian
classifiers, k-nearest neighbor classifiers and support vector machine. Classification mainly performs the automatic
categorization of documents. In web usage mining, application of classification algorithm on server logs may lead to
detection of interesting patterns of users who visits news site.
D. Sequential Patterns
In web usage mining, sequential pattern is used to discover the sessions that are found in a sequence. They
include the sequence of items that frequently occur in a particular order. MIDAS (Mining Internet Data for Association
Sequences) algorithm is most commonly used for finding sequential patterns which provide marketing intelligent
behavior for e-commerce scenario [8].
III. WEB USAGE MINING TECHNIQUES
The web usage mining generally includes the following several steps: data collection, data pretreatment [6] and
knowledge discovery and pattern analysis.
A. Data Collection
Data collection is the first step of web usage mining, the data authenticity and integrity will directly affect the
following works smoothly carrying on and the final recommendation of characteristic service’s quality. Therefore it must
use scientific, reasonable and advanced technology to gather various data. At present, towards web usage mining
technology, the main data origin has three kinds: server data, client data and middle data (agent server data and package
detecting).
B. Data Preprocessing
Some databases are insufficient, inconsistent and including noise. The data pre-treatment is to carry on a
unification transformation to those databases. The result is that the database will to become integrate and consistent, thus
© www.ijarcsse.com, All Rights Reserved
Page | 519
Prabha et al., International Journals of Advanced Research in Computer Science and Software Engineering
ISSN: 2277-128X (Volume-7, Issue-6)
establish the database which may mine. In the data pre-treatment work, mainly include data cleaning, user identification,
session identification and path completion.
1) Data Cleaning
The purpose of data cleaning is to eliminate irrelevant items, and these kinds of techniques are of importance for
any type of web log analysis not only data mining. According to the purposes of different mining applications, irrelevant
records in web access log will be eliminated during data cleaning. Since the target of Web Usage Mining is to get the
user’s travel patterns, following two kinds of records are unnecessary and should be removed
 The records of graphics, videos and the format information. The records have filename suffixes of GIF,
JPEG, CSS, and so on, which can found in the URI field of the every record.
 The records with the failed HTTP status code. By examining the Status field of every record in the web
access log, the records with status codes over 299 or under 200 are removed. It should be pointed out that
different from most other researches, records having value of POST or HEAD in the Method field are reserved
in present study for acquiring more accurate referrer information.
2) User and Session Identification
The task of user and session identification is to find out the different user sessions from the original web access
log. User’s identification is, to identify who access web site and which pages are accessed. The goal of session
identification is to divide the page accesses of each user at a time into individual sessions. A session is a series of web
pages user browse in a single access. The difficulties to accomplish this step are introduced by using proxy
servers e.g. different users may have same IP address in the log. A referrer-based method is proposed to solve these
problems in this study. The rules adopted to distinguish user sessions can be described as follows:
a) The different IP addresses distinguish different users;
b) If the IP addresses are same, the different browsers and operation systems indicate different users;
c) If all of the IP address, browsers and operating systems are same, the referrer information should be taken into
account. The Refer URI field is checked, and a new user session is identified if the URL in the Refer URI field hasn’t
been accessed previously, or there is a large interval (usually more than 10 seconds) between the accessing time of this
record and the previous one if the Refer URI field is empty;
d) The session identified by rule 3 may contains more than one visit by the same user at different time, the time-oriented
heuristics is then used to divide the different visits into different user sessions. After grouping the records in web logs
into user sessions, the path completion algorithm should be used for acquiring the complete user access path.
3) Path Completion
Another critical step in data preprocessing is path completion. There are some reasons that result in path’s
incompletion, for instance, local cache, agent cache, “post” technique and browser’s “back” button can result in some
important accesses not recorded in the access log file, and the number of Uniform Resource Locators (URL) recorded in
log may be less than the real one. Using the local caching and proxy servers also produces the difficulties for path
completion because users can access the pages in the local caching or the proxy servers caching without leaving any
record in server’s access log. As a result, the user access paths are incompletely preserved. Web access log is to discover
user’s travel pattern, the missing pages in the user access path should be appended. The purpose of the path completion is
to accomplish this task. The better results of data preprocessing, we will improve the mined patterns' quality and save
algorithm's running time. It is especially important to web log files, in respect that the structure of web log files are not
the same as the data in database or data warehouse. They are not structured and complete due to various causations. So it
is especially necessary to pre-process web log files in web usage mining. Through data pre-processing, web log can be
transformed into another data structure, which is easy to be mined.
C. Pattern Discovery
Use statistical method to carry on the analysis and mine the pre-treated data. We may discover the user or the
user community's interests then construct interest model. At present the usually used machine learning methods mainly
have clustering, classifying, the relation discovery and the order model discovery. Each method has its own excellence
and shortcomings, but the quite effective method mainly is classifying and clustering at the present.
D. Pattern Analysis
A challenge of Pattern Analysis is to filter uninteresting information and to visualize and interpret the interesting
patterns to the user. First delete the less significance rules or models from the interested model storehouse. Next use
technology of OLAP and so on to carry on the comprehensive mining and analysis; Once more, let discovered data or
knowledge be visible. Finally, provide the characteristic service to the electronic commerce website.
IV. CONCLUSION
Web usage mining is a kind of mining to server logs. Web usage mining plays an important role in realizing,
enhancing the usability of the website design, the improvement of customer’s relations and improving the requirement of
system performance and so on. Web usage mining provides the support for the website design, providing personalization
server and other business making decision, etc. Its main aim is getting useful to users for easy access information in logs
to make sites perfect with effectual. Data preprocessing is the data pretreatment work. The future work is to implement
the algorithms in the techniques of Web usage mining.
© www.ijarcsse.com, All Rights Reserved
Page | 520
Prabha et al., International Journals of Advanced Research in Computer Science and Software Engineering
ISSN: 2277-128X (Volume-7, Issue-6)
REFERENCES
[1]
A. Banerjeee and J. Ghosh,”Click stream Clustering using Weighted Longest Common Subsequences”, In
Proceedings of the Web Mining Workshop at the 1st SIAM Conference on Data Mining, 2001.
[2]
A.G.Buchner, M. Baumgarten, S.S.Anand MD.Mulvenna and J.G. Hughes. “Navigation Pattern Discovery from
Internet Data”, In WEBKDD, San Die-go, CA 1999.
[3]
Bharanipriya, V., Prasad, V.K., Web Content Mining Tools: A comparative Study, International Journal of
Information Technology and Knowledge Management. Vol. 4, No 1, pp 211-215 (2011).
[4]
Jaideep Srivastava, Prasanna Desikan, Vipin Kumar, ―Web Mining - Concepts, Applications & Research
Directions and Web Mining‖ – Accomplishments & Future Directions.
[5]
J. Heer and Ed H. Chi.,”Mining the Structure of User Activity Using Cluster stability”, In Proceedings of the
Workshop on Web Analysis, Second SIAM Conference on Data Mining ACM Press, 2002.
[6]
Margaret H. Dunham “Data Mining Introductory and Advanced Topics ISBN 978-81-7758-785-2 pp 205-218
[7]
O.Etzioni, The World Wide Web: Quagmire or gold mine. Communication of the ACM, 39(11): 65-68, 1996
[8]
R. Cooley,”Web Usage Mining: discovery and Applications of Interesting Patterns from Web Data”, PhD thesis,
University of Minnesota, 2000.
[9]
Sravan Kumar, D. and Naveena Devi, B. ―Learner’s Centric Approach for Web Mining ‖ et al. (IJCSIT)
International Journal of Computer Science and Information Technologies, Vol. 1(2), 2010.
[10]
S.Balan, “A Study of Various Techniques of Web Content Mining Research Issues and Tools”,
[11]
U. Fayyad G. Piatetsky-Shapiro and P. Smyth.,”From Data Mining to Knowledge Discovery: An Overview”, In
Proc. ACM KDD 1994.
© www.ijarcsse.com, All Rights Reserved
Page | 521