This work is inspired by https://github.com/digital-engineering/airbnb-scraper, which is a very excellent work and helped me a lot. However, it can only search for properties based on keywords and can only return limited number of search results (for each search, Airbnb only return about 300 results). So it's inconvenient to scrap large number of properties (for example, you want you scrap all the available properties in New York). This modified tool allows large scale Airbnb scrapping given the listing ID of each property. Text information of properties are saved to CSV or xlsx. The tool can also save photo links (in CSV), reviews (in CSV) and the API response (in txt).
scrapy crawl airbnb -a query="test.csv" -o new_york.csv
You can find the values for these by first doing a search manually on the Airbnb site.
path: The path to the csv file containing the listing ids of properties (column 'id'). (required)output: Path of output file.
- Python 3.9+
- Scrapy
- openpyxl
- ElasticSearch 7+ if using elasticsearch pipeline
- see requirements.txt for details
- This work is modified based on https://github.com/digital-engineering/airbnb-scraper, which is a very excellent work and helped me a lot.
For Notes, Installation and Configuration, please see https://github.com/digital-engineering/airbnb-scraper#readme.