ew Machine Learning Crawling Algorithm For Web Forums
Journal Title: International Journal for Research in Applied Science and Engineering Technology (IJRASET) - Year 2015, Vol 3, Issue 8
Abstract
In this paper, we present FoCUS (Forum Crawler Under Supervision), a supervised web-scale forum crawler. The goal of FoCUS is to only trawl relevant forum content from the web with minimal overhead. Forum threads contain information content that is the target of forum crawlers. Although forums have different layouts or styles and are powered by different forum software packages, they always have similar implicit navigation paths connected by specific URL types to lead users from entry pages to thread pages. Based on this observation, we reduce the web forum crawling problem to a URL type recognition problem and show how to learn accurate and effective regular expression patterns of implicit navigation paths from an automatically created training set using aggregated results from weak page type classifiers. Robust page type classifiers can be trained from as few as 5 annotated forums and applied to a large set of unseen forums. Our test results show that FoCUS achieved over 98% effectiveness and 97% coverage on a large set of test forums powered by over 150 different forum software packages.
Authors and Affiliations
M. Arjun, B. Bharath Kumar
Keywords
Related Articles
- EP ID EP21177
- DOI -
- Views 205
- Downloads 4
Research Paper on Online Bookshop Management System
Today it is becoming very difficult to maintain records manually. Software system easily does the job of maintaining daily records as well as the transaction according to the user requirements. Only basic knowledge of c...
Detection of Built-up Areas from High Resolution Satellite Images
With the advancement in remote sensing technology, high resolution remote sensing have become a key source of information in many application platforms such as urban development, agriculture monitoring, military, intell...
Analysis of Compressive Strength of Low Volume Fly Ash Concrete
Fly ash is a major by-product of thermal power plant, the alternative option for its disposal was to use it in building materials. Incorporation of fly ash in concrete specimen has shown enhancement in properties of con...
Implementation of Sub-Threshold Source Coupled Logic for Ultra-Low Power Application
This thesis work generally focuses on the use of sub-threshold source combined logic (STSCL) for building digital circuits and systems working at very low voltage and promise to provide desirable performance with excell...
A Prototype for Cell Phone Signal Isolator for GSM Network with Preschedule Time Duration Using Arduino
The prototype developed here is mainly intended to prevent the usage of mobile phones in place inside the range, thus providing an effective and reliable device for blocking mobile communication in required restricted s...