Welcome to the "TamuDatathon_Data-Synthesis" repository. This project focuses on creating a database through web scraping, specifically targeting "Internships and Job Opportunities" data. The repository utilizes web scraping techniques with tools like Selenium and Beautiful Soup to extract information from popular job-related websites such as Indeed.com, Levels.fyi, and Glassdoor.com.
Within this repository, you will find the following files and folders:
-
Glassdoor_Files: This folder contains files related to web scraping and data extraction from Glassdoor.com. It includes code and datasets.
-
Indeed_Files: Here, you can find the initial files and code related to web scraping and data extraction from Indeed.com.
-
Level.fyi_Files: This folder contains files and code for web scraping and data extraction from Levels.fyi, a platform known for salary data.
-
Glassdoor_data.csv: This file holds data extracted from Glassdoor.com and may serve as a dataset for analysis or further processing.
-
HackathonDataSet.py: This Python script likely contains code related to data synthesis and preparation for the hackathon.
-
Read_Json_to_csv.ipynb: An IPython Notebook file that might be used for converting JSON data to CSV format for easier analysis and manipulation.
-
webscraper.ipynb: Another IPython Notebook file that could contain web scraping code, showcasing the techniques used in the project.
-
README.md: The initial README file (the one you are currently reading) providing an overview of the repository.
The primary goal of this project is to collect internship and job opportunity data from popular job-related websites. It involves using web scraping techniques to gather information from these websites and potentially synthesize the data into a structured database for analysis.
This repository was created as part of a project for a data-related hackathon, and it showcases the skills and tools used for web scraping and data synthesis.
If you have any questions or need further information about this project, please feel free to reach out.
Anish Karthik