ew Machine Learning Crawling Algorithm For Web Forums
Journal Title: International Journal for Research in Applied Science and Engineering Technology (IJRASET) - Year 2015, Vol 3, Issue 8
Abstract
In this paper, we present FoCUS (Forum Crawler Under Supervision), a supervised web-scale forum crawler. The goal of FoCUS is to only trawl relevant forum content from the web with minimal overhead. Forum threads contain information content that is the target of forum crawlers. Although forums have different layouts or styles and are powered by different forum software packages, they always have similar implicit navigation paths connected by specific URL types to lead users from entry pages to thread pages. Based on this observation, we reduce the web forum crawling problem to a URL type recognition problem and show how to learn accurate and effective regular expression patterns of implicit navigation paths from an automatically created training set using aggregated results from weak page type classifiers. Robust page type classifiers can be trained from as few as 5 annotated forums and applied to a large set of unseen forums. Our test results show that FoCUS achieved over 98% effectiveness and 97% coverage on a large set of test forums powered by over 150 different forum software packages.
Authors and Affiliations
M. Arjun, B. Bharath Kumar
Keywords
Related Articles
- EP ID EP21177
- DOI -
- Views 187
- Downloads 4
Web Server Based Agriculture Automation Using Sensor Network
automatic irrigation scheme helps the farmer to irrigate the land in well-organized method can be explained in these thesis. This system helps to reduce water usage in agricultural field. Remote sensor unit gives inform...
Application of Artificial Neural Networks in Civil Engineering
The use of artificial neural networks (ANNs) has increased in many areas of engineering for over the last few years. The ANNs have been applied to many geotechnical engineering problems and have demonstrated some degree...
Utilization of Waste Adhesives in Bituminous Concrete
In India due to rapid growth in urbanization leads to increased number of vehicles day by day on the road pavements which results in repeated loading and higher riding quality roads. Surface course is the top most layer...
Diesel Engine with Hydrogen in Dual Fuel Mode: A Review
Depleting fossil fuel resources and increased energy demand forced automobile companies to search clean alternative. There are also Concerns for global warming and tightened emission norms. For this there are different...
Study on Compressive Strength of M30 Grade Concrete with Partial Replacement Of C.A with Electrical ARC Furnace Slag
In this research we have replace different proportion percentage of normal aggregate with Electric Arc Furnace Slag aggregate and compared with conventional concrete. The compressive strength and tensile strength test i...