Mutual Exclusion Principle for Multithreaded Web Crawlers
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2012, Vol 3, Issue 9
Abstract
This paper describes mutual exclusion principle for multithreaded web crawlers. The existing web crawlers use data structures to hold frontier set in local address space. This space could be used to run more crawler threads for faster operation. All crawler threads fetch the URL to crawl from the centralized frontier. The mutual exclusion principle is used to provide access to frontier for each crawler thread in synchronized manner to avoid deadlock. The approach to utilize the waiting time on mutual exclusion lock in efficient manner has been discussed in detail.
Authors and Affiliations
Kartik Perisetla
Keywords
Related Articles
- EP ID EP108868
- DOI -
- Views 75
- Downloads 0
Representation Modeling Persona by using Ontologies: Vocabulary Persona
Semantic Web is then to add to all these resources semantics that allow computer systems to "understand" the meaning by accessing structured collections of information and inference rules that can be used to drive reason...
Short-Term Load Forecasting for Electrical Dispatcher of Baghdad City based on SVM-FA
The improvement of load forecasting accuracy is an important issue in the scientific optimization of power systems. The availability of accurate statistical data and a suitable scientific method are necessary for a perfe...
Intelligent Security for Phishing Online using Adaptive Neuro Fuzzy Systems
Anti-phishing detection solutions employed in industry use blacklist-based approaches to achieve low false-positive rates, but blacklist approaches utilizes website URLs only. This study analyses and combines phishing em...
Personalized Semantic Retrieval and Summarization of Web Based Documents
The current retrieval methods are essentially based on the string-matching approach lacking of semantic information and can’t understand the user's query intent and interest very well. These methods do regard as the pers...
An agent based approach for simulating complex systems with spatial dynamics application in the land use planning
In this research a new agent based approach for simulating complex systems with spatial dynamics is presented. We propose an architecture based on coupling between two systems: multi-agent systems and geographic informat...