Andrewnolan13 / DoneDeal

retreive car listing data from DoneDeal.ie, Ireland's largest Motor Marketplace

DoneDeal is an Irish online marketplace focused on helping buyers and sellers of cars in Ireland. The site lists more cars for sale in Ireland than any other website.

This module provides an easy way to scrape Motor listing data from the website.

Usage

import donedeal as dd

# Initialize scraping object
scraper = dd.CarScraper()

# Search for Mercedes E-Class and BMW 3 Series in Ireland with mileage under 140,000km and price under 20,000
# This is just formatting the search URL
scraper.set_make_model(make='Mercedes-Benz', model='E-Class') 
scraper.set_make_model(make='BMW', model='3-Series')
scraper.set_mileage(mileage_to=140_000)
scraper.set_price(price_to=20_000)

# Start scraping
scraper.scrape(batch_size=500)

# Access scraped data
df = scraper.DataFrame  # pandas.DataFrame

Install

To clone the repository and run the test script, follow these steps:

1. Open the command prompt.
2. Navigate to the directory where you want to clone the repository.

cd C:\path\to\desired\location

3. Clone the repo.

git clone https://github.com/Andrewnolan13/DoneDeal.git

4. Install module on path (so you can import it from anywhere)

cd Donedeal
python setup.py install

Tests

After the repository and installed, you can perform the following tests in your command line:

donedeal

which will return 90593 cars in Ireland, or something. You can also run the command:

donedeal_merc_test

This will execute the Mercedes.py script and display a plot in your browser.

alternatively, if you don't want to install the module on path: After the repository is cloned, navigate to the DoneDeal/tests directory and run Mercedes.py:

cd DoneDeal/tests
python Mercedes.py

Built-in Data cleaning functions

#cleaning/formatting + features
df:pd.DataFrame = scraper.DataFrame.copy()
df=df[dd.constants.RECOMMENDED_COLUMNS] #alot of data gets returned, RECOMMENDED_COLUMNS is an optional subset of columns 

df=(df.pipe(dd.data_cleaning.assign_lat_lon) # turns string 'x,y' into (float(x),float(y))
      .pipe(dd.data_cleaning.assign_mileage) # turns string 'x km' or 'x mi' into float(x) in km (turns mi to km)
      .pipe(dd.data_cleaning.price_to_float)) # turns string 'abc,def' into float(abcdef)
df = (df.loc[lambda self:~self.price.apply(dd.data_cleaning.isBsPrice)] # drops prices like 1234,123456,123456789,111111111 etc 
        .loc[lambda self:~self.kilometers.apply(dd.data_cleaning.isBsPrice)])# sometimes the mileage is bs too. sequential digits like 123456 are just so unlikely 

>>> df.head()

	make	model	price	fuelType	trim	transmission	engine	enginePower	acceleration	seats	numDoors	colour	country	owners	roadTax	NCTExpiry	bodyType	year	currency	sellerType	county	countyTown	header	friendlyUrl	lat	lon	kilometers
0	Mercedes-Benz	500	17500.0	Petrol	NaN		5.0 L	NaN	NaN	4		Grey	Ireland	3		May 2025	Coupe	1987	EUR	PRIVATE	Co. Dublin	Dublin 16	1987 Mercedes-Benz 500	https://www.donedeal.ie/cars-for-sale/1987-mercedes-benz-500/35499157	53.255728	-6.296347	99000.000
1	Mercedes-Benz	SLK-Class	19995.0	Diesel	SPORT	Automatic	2.1 L	204 hp	6.7 sec	2	2	Blue	Ireland	2	280	Jul 2025	Convertible	2014	EUR	PRO	Co. Mayo	Ballina	141 MERCEDES SLK 250CDI AUTO AMG - LOW MILES -	https://www.donedeal.ie/cars-for-sale/141-mercedes-slk-250cdi-auto-amg-low-miles-/36950512	54.114950	-9.155069	111688.196
2	Mercedes-Benz	S-Class	11999.0	Diesel	NaN	Automatic	3.0 L	NaN	NaN	5	4	Silver	Ireland	9	1250	Jan 2025	Saloon	2010	EUR	PRIVATE	Co. Dublin	South County	Mercedes-Benz S-Class	https://www.donedeal.ie/cars-for-sale/mercedes-benz-s-class/33603970	53.269971	-6.224923	110000.000
3	Mercedes-Benz	E-Class	10999.0	Diesel	Classic	Automatic	2.1 L	136 hp	9.5 sec	5	4	Black	Ireland	5	280	Nov 2024	Saloon	2013	EUR	PRO	Co. Dublin	Dublin 15	Mercedes-Benz E-Class 200 CDI Blue Efficiency ECO	https://www.donedeal.ie/cars-for-sale/mercedes-benz-e-class-200-cdi-blue-efficiency-eco/37552808	53.390700	-6.381690	135999.000
4	Mercedes-Benz	E-Class	17500.0	Diesel	CLASSIC	Automatic	2.1 L	136 hp	9.5 sec	5	4	Black	Ireland	3	200	Mar 2025	Saloon	2015	EUR	PRIVATE	Co. Dublin	Clondalkin	Mercedes Benz E220 2015	https://www.donedeal.ie/cars-for-sale/mercedes-benz-e220-2015/37024503	53.327529	-6.400042	206000.000