The goal was to build a web application that scrapes various websites for data related to the Mission to Mars and displays the information in a single HTML page.
An initial scraping using Jupyter Notebook, BeautifulSoup, Pandas, and Requests/Splinter.
- Scrape the (https://mars.nasa.gov/news/) and collect the latest News Title and Paragragh Text.
-
Visit the url for JPL's Featured Space Image (https://www.jpl.nasa.gov/spaceimages/?search=&category=Mars).
-
Used splinter to navigate the site and find the image url for the current Featured Mars Image and assign the url string to a variable.
- Visits the Mars Weather twitter account (https://twitter.com/marswxreport?lang=en) and scrapes the latest Mars weather tweet from the page.
-
Visits the Mars Facts webpage (http://space-facts.com/mars/) and uses Pandas to scrape the table containing facts about the planet including Diameter, Mass, etc.
-
Uses Pandas to convert the data to a HTML table string.
- Visits the USGS Astrogeology site (https://astrogeology.usgs.gov/search/results?q=hemisphere+enhanced&k1=target&v1=Mars) to obtain high resolution images for each of Mar's hemispheres.
Uses MongoDB with Flask templating to create a new HTML page that displays all of the information that was scraped from the URLs above.