Web scraping and data extraction

Web scraping automatically collects information published on websites, instead of copying it item by item for hours. Competitor prices, listings, business directories or customer reviews land in a clean file, ready to use. On BeFreelancr, you hand this work to a freelancer who builds the extraction tool suited to your need and delivers the data in CSV or Excel. An online store monitoring its competitors or a researcher building a corpus saves a lot of time. Before ordering, specify the target websites, the information to extract and how often, and check that you're allowed to reuse this data.

Are you a freelance developer?

Sign up on BeFreelancr, create your profile and services, and find clients. Solo freelancers, collectives and agencies are all welcome!

Need a developer?

Tell us what you need and we'll find the right freelancer for your project. For more complex projects, we can also suggest collectives or agencies.

Web scraping and data extraction: FAQ

How much does it cost to hire a freelancer for web scraping?

Between €50 and €1,000 for a typical job, delivered in 2 to 7 days. A list collected from a single simple website, then delivered as a CSV, sits at the low end of the range. The budget grows with the number of sources, pages loaded with JavaScript, the volume, and removing duplicates or sending the data to a database.

Web scraping, API or open data: what should you ask a developer for?

First ask whether an official source exists. An API or an open dataset is more stable than scraping, and its terms of use are written down. Many national business registers, for example, publish their data openly or through an API.

Scraping makes sense when the information isn't available anywhere else, such as prices or listings displayed on a website. A connector to an API is often a job for workflow automation, a more lasting solution.

Is web scraping legal?

It isn't banned in principle, but that doesn't mean anything goes. Several rules add up:

  • Personal data: in the European Union, data that's visible online is still covered by the GDPR. You need a legal basis, and you must inform the people concerned.
  • Databases: under the EU Database Directive, the maker of a database that required a substantial investment can prohibit the extraction or reuse of a substantial part of it.
  • Website terms: read its terms of use, which may exclude any automated collection.
  • The robots.txt file: it shows what the site accepts, without granting permission to access it.

This answer stays general, and rules differ outside the EU. The developer carries out the extraction, but it's up to you to check the intended use, with a lawyer if personal data or a large volume is involved.

Can you scrape LinkedIn or directories for email prospecting?

It's the riskiest case. In December 2024, France's data protection authority (CNIL) fined KASPR €240,000 under the GDPR for harvesting contact details on LinkedIn: this data was only visible to some of its owners' connections, and they didn't expect it to be collected this way.

A prospecting contact list is therefore better built from authorized sources. A lead generation provider should also tell you where the contacts come from.

What should you prepare to order data extraction?

  • The sources: the exact URLs of the target pages.
  • The expected file: one column per piece of information, with an example filled in by hand.
  • Volume and frequency: a thousand rows once or every week is not the same job.
  • The format: CSV or Excel for simple use, JSON or a database for technical use.
  • The intended use: it determines the useful fields, and what's better left uncollected.

Ask for a sample before the full collection, then check for duplicates and empty values. To draw conclusions from the file, data analysis can take over.

One-off extraction or recurring collection: what should you order?

For a study or a single file, delivering the data is enough. Tracking a competitor's prices or monitoring listings requires a collection that runs on its own, and that's a different service.

A script breaks as soon as the website changes its layout. Before ordering, ask where it runs, who fixes it, and whether its code and documentation are handed over to you.

Can a JavaScript website or a site protected by a CAPTCHA be scraped?

A page loaded with JavaScript is read with an automated browser, such as Playwright or Selenium. It takes longer to build and to run, so it costs more.

A CAPTCHA or a password-protected area is another matter: the site is showing that it refuses this kind of access. In its guidance on AI development, France's data protection authority (CNIL) asks for websites that clearly object to scraping, notably by this means, to be excluded. Don't order a workaround: ask the data publisher for access or a license instead.