Web Scraper Revealing Trends of Target Products and New Insights in Online Shopping Websites
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2018, Vol 9, Issue 6
Abstract
Trillions of posts from Facebook, tweets in Twitter, photos on Instagram and e-mails on exchange servers are overwhelming the Internet with big data. This necessitates the development of such tools that can detect the frequent updates and select the required information instantly. This research work aims to implement scraper software that is capable of collecting the updated information from the target products hosted in fabulous online e-commerce websites. The software is implemented using Scrapy and Django frameworks. The software is configured and evaluated across different e-commerce websites. Individual website generates a greater amount of data about the products that need to be scraped. The proposed software provides the ability to search a target product in a single consolidated place instead of searching across various websites, such as amazon.com, alibaba.com and daraz.pk. Furthermore, the scheduling mechanism enables the scraper to execute at a required frequency within a specified time frame.
Authors and Affiliations
Habib Ullah, Zahid Ullah, Shahid Maqsood, Abdul Hafeez
Keywords
Related Articles
- EP ID EP324798
- DOI 10.14569/IJACSA.2018.090658
- Views 88
- Downloads 0
Detection of Soft Atheroscelarotic Plaques in Cardiac Computed Tomography Angiography
Computed tomography angiography (CTA) has turned non-invasive diagnosis of cardiovascular anomalies into a reality as state-of-the-art imaging equipment is capable of recording sub-millimeter details. Based on high inten...
Skew Detection/Correction and Local Minima/Maxima Techniques for Extracting a New Arabic Benchmark Database
We propose a set of techniques for extracting a new standard benchmark database for Arabic handwritten scripts. Thresholding, filtering, and skew detection/correction techniques are developed as a pre-processing step of...
An Empirical Investigation of the Correlation between Package-Level Cohesion and Maintenance Effort
The quality of the software design has a considerable impact on software maintainability. Improving software quality can reduce costs and efforts of software maintenance. Cohesion, as one of software quality characterist...
Optimized Routing Information Exchange in Hybrid IPv4-IPv6 Network using OSPFV3 & EIGRPv6
IPv6 is the next generation internet protocol which is gradually replacing the IPv4. IPv6 offers larger address space, simpler header format, efficient routing, better QoS and built-in security mechanisms. The migration...
Growing Cloud Computing Efficiency
Cloud computing is basically altering the expectation for how and when computing, storage and networking assets should be allocated, managed and devoted. End-users are progressively more sensitive in response time...