Web Crawler Design Issues: A Review
Journal Title: International Journal of Management, IT and Engineering - Year 2012, Vol 2, Issue 8
Abstract
The large size and the dynamic nature of the Web increase the need for updating Web based information retrieval systems. Crawlers facilitate the process by following the hyperlinks in Web pages to automatically download a partial snapshot of the Web. While some systems rely on crawlers that exhaustively crawl the Web, others focus on topic specific collections. In present paper the various types of crawlers are discussed. The paper also discusses several web crawler design issues along with their solutions.
Authors and Affiliations
Deepika And Dr Ashutosh Dixit
Keywords
Related Articles
- EP ID EP18526
- DOI -
- Views 326
- Downloads 13
What do we need in the future – Effect based control strategies or death of species ?
The critical load approach is a methodology according to which critical loads are used as a criterion to assess whether emission reduction strategies are sufficient. It is now more than a decade since the critical load...
Impact of Information communication technology on Entrepreneurial Traits among exporters in Coimbatore region
An entrepreneur is an originator, a challenger and a driver. Creating new innovation with innovative ideas or thoughts is main character of an entrepreneur in business or a company. The entrepreneur is a leader and a p...
Office Romance: Legal Challenges and Strategic Implications
Office romance is a reality of the modern workplace as more and more employees meet their best friends and life-partners through their professional connections. Over 40% of today’s employees have had personal and roman...
slugPOWER QUALITY IMPROVEMENT USING PWM VOLTAGE REGULATOR
In this present paper, stress has been laid upon the present scenario of power quality in every grid. With more and more use of non linear electrical loads instead of linear loads, we get increased efficiency with redu...
slugRole of Foreign Direct Investment in Indian Small Scale Industries
FDI plays an important role in any country‟s economical activities. Companies big or small can not solely depend on the conventional source of finance all together. For financial and technological support they have to...