This repository provides all codes, data, and notes covered in my YouTube playlist on Data Wrangling with Python. I added links to YouTube videos. You find all videos on our Channel YUNIKARN
This is a detailed step-by-step guide that develops a Python code to extract information from pdf files. This is very useful if you have to handle a large number of files. The Python code returns the number of all search term occurrences in the document and identifies the page numbers. I introduce the PyPDF2 package, which we need to install.
Installation on Anaconda: conda install -c conda-forge pypdf2
Installation using the pip installer: pip install PyPDF2
I show you how to create and activate a virtual environment (which is optional – but useful to do). Then we develop the code step-by-step. This will enable you to learn how to modify the code to suit your specific requirements. Please leave a comment if you have any questions.
Finally, we will refactor the code. We define a function that takes a search term and filename and returns a tuple containing the total number of occurrences and the number of pages that contain the search term at least once.
Chapters
- 0:00 Welcome
- 0:15 Return all occurrences & page numbers
- 0:44 Example pdf
- 2:23 Python setup
- 3:55 Virtual environment
- 6:16 Coding fun
- 28:05 Refactoring
This tutorial covers the Python HTTP library, Requests, which can be used to obtain data from the web. We write a Python code that downloads climate data from http://berkeleyearth.org/. The same principle can be used to download files from other urls. I show you how to download the data into a text file. Then we use NumPy to load the text file into a NumPy array. Finally, we convert the data into a Pandas DataFrame. If this sounds like crazy data fun for you, please join us!
I show you how to create and activate a virtual environment (which is optional – but useful to do). We use the pip installer.
If you do not have the pip installer, download get-pip.py. The download needs to be in same path as your Python installation - then change the directory into the folder. Using the command line, type: python get-pip.py, and finally check the installation: pip -V.
To install virtual environments, use: pip install virtualenv
We develop the code step-by-step. This will enable you to learn how to modify the code to suit your specific requirements. Please leave a comment if you have any questions. Python is the way!
Chapters
- 0:00 Data from the Web
- 0:36 Virtual Environment
- 2:26 Using Requests
- 4:02 Save Data to File
- 5:03 DataFrame
This tutorial covers the Python package yfinance, which enables us to connect to Yahoo Finance (API). We download data for the Bitcoin-USD trade and work with the Pandas DataFrame. Finally, we visualise the data using Matplotlib.
This is a detailed step-by-step guide that develops a Python code to read PDF files and determine the most common words. This is very useful if you want to get an idea about the content of PDF files without reading them yourself. Applications include systematic literature reviews or selecting newspaper articles.
All material is on GitHub.
I show you how to create and activate a virtual environment (which is optional – but useful). Then we develop the code step-by-step. This will enable you to learn how to modify the code to suit your specific requirements. Please leave a comment if you have any questions.
Chapters
- 0:00 Common Words in PDF Files
- 0:48 Virtual Environment
- 1:56 Main.py & Module
- 2:44 The word_rank Function
- 8:07 Counter Class