How Does Data Scraping Work?
A summary of how data scraping works on the Axiom platform.
In today's world, data is the currency of success. Businesses are continuously collecting and analysing data to gain insights, make informed decisions and better serve their customers. Marketplaces are no exception. They heavily rely on product data to showcase products to customers and provide them with an excellent shopping experience. However, collecting and updating product data can be time-consuming and challenging.
The Axiom platform offers a range of insights and features to help suppliers enhance their product data. One such feature is data scraping. But what exactly is data scraping, and how does it work on the Axiom platform?
Data Scraping
Data scraping is a process of extracting data from web pages. It involves automated software that visits a website, reads the content, and extracts relevant information. The Axiom platform uses data scraping techniques to extract product information from supplier websites and help suppliers to enrich their product data.
It is important to note that Axiom's web scraping practices comply with ethical data scraping practices and relevant data privacy regulations to protect the personal data of users and website owners.
Scraping on the Axiom platform
1. Supplier uploads product data
To begin, the supplier agrees to the T&Cs of the Axiom marketplace platform and uploads their product data. This data includes product page URLs for the product listings on the supplier's website or e-commerce platform.
Please ensure that the provided product URLs are valid and publically accessible in a web browser.
Some e-commerce websites use methods such as CAPTCHAs, robots.txt files and session-based protections to control access to specific parts of their website and to prevent certain pages or sections of their website from being scraped. Please ensure that the relevant parts are accessible if you wish to use the scraping functionality to help improve your product data.
2. Axiom scraper visits each URL
After uploading the product data, the scraper will visit each URL to extract publicly available product information with a high degree of accuracy. This information comprises product names, descriptions, images, and categories, among other fields.
The scraping process is limited to scrape a maximum of 5 pages per second per supplier to respect the website's bandwidth and server resources, so as to not cause any harm. We will also not re-scrape the same URL for at least 30 days even if a product is reuploaded.
3. Translation (if required)
If the product information is in a different language than the one required, the scraper will translate the relevant fields to ensure that the supplier and their customers can fully understand the data.
4. Supplier views examples of scraped data
The supplier can then view examples of the scraped data to see which fields they would like to systematically use to enrich their product data. This ensures that the supplier has control over what data is used to enrich their product listings.
5. Enrichment of product data
Finally, the supplier's product data is enriched with the scraped data. This ultimately improves the potential for the right products to be served to the right customers in the shop, and boosts the user experience.
Summing up
The Axiom Marketplace's data scraping feature is a powerful tool for suppliers looking to improve their product listings. By utilising this feature, suppliers can enrich their product data, ultimately leading to a better user experience and increased sales.