Found Issue, Going Next Course 'name'
zerohope opened this issue · comments
Try to down the course learn git the hard way, it doesn't work
this is the address in the urls.txt https://www.educative.io/courses/learn-git-hard-way/JYOGn0Bkv5J
Hi @anilabhadatta , I am having the same issue, how do we fix it?
Alright, I am not a Python expert but I debugged this problem and found the problem was with the below code
print("Getting File Name")
meta_script_selector = "script[type='application/ld+json']"
metadata = driver.find_elements( By.CSS_SELECTOR, meta_script_selector)[0].get_attribute('innerHTML')
metadata = json.loads(metadata)
if course_folder:
file_name = metadata['name']
Just note this line
metadata = driver.find_elements( By.CSS_SELECTOR, meta_script_selector)[0].get_attribute('innerHTML')
It uses the 0th element but in my case, it was 1st element( Found out after looking through the sources)
Changing the previous line to the below line fixes the problem
metadata = driver.find_elements( By.CSS_SELECTOR, meta_script_selector)[1].get_attribute('innerHTML')
I am not sure it's always one, but the one solution I could think of is to look through all the array elements if there is no name found.
by the way, @anilabhadatta , Thank you for this tool.
@sumant4ssm thank you for pointing out the issue.
Currently i am not well so i will try to fix it in few days.
If possible comment the latest get file name function and uncomment the previous one present just above.
thanks @anilabhadatta . It was an easy fix and was able to fix it.
@sumant4ssm thank you for pointing out the issue. Currently i am not well so i will try to fix it in few days. If possible comment the latest get file name function and uncomment the previous one present just above.
after commenting getting below error any help is greatly appreciated
ad Webpage Function
Checking Login Function
Checking for captcha Function...
Create Course Folder Function
Getting File Name
File Name Found
Found Issue, Going Next Course slugify() got an unexpected keyword argument 'replacements'
Script Execution Complete
@zerohope @sumant4ssm @paletikoushik Test it now, should be fixed. Clone the repo again in a fresh location.
cloned the repo again in fresh location still getting below error, any help is greatly appreciated sir.
Driver Loaded
[Selected config: 0] Starting Scraping: 0, https://www.educative.io/courses/decode-coding-interview-cpp/qABQKQqWmmk
Load Webpage Function
Checking Login Function
Checking for captcha Function...
Create Course Folder Function
Getting File Name
File Name Found
Found Issue, Going Next Course slugify() got an unexpected keyword argument 'replacements'
@paletikoushik In my testing, i am not receiving this error.
Please answer the following?
Have you created a virtual env?
If yes, then did you install the modules from requirements.txt in your virtual env?
In my tool, I am using python-slugify instead of slugify. Maybe that is causing some issues.
One more note I should add, there is a bug in file naming as well. I will fix it today.
Ok sir, so I made mistake
after install requirements.txt I got selenium missing.
I did pip3 install selenium and after that got slugify missing and I used pip3 install slugify, so that is creating the problem sir now how to solve this issue sir. I tried to uninstall it by pip3 uninstall slugify and installing pip3 install python-slugify still getting same error. I tested it on mac and its working fine (as that was first time i got same error like selenium missing and slugify missing and I installed pip3 install python-slugify and its working. What should I do on my windows laptop any help is greatly appreciated.
@paletikoushik can you share your screen? Send me a meet link and ping me up
running the commands with
python chromedriver.py
python educative_scraper.py instead of python3 chromedriver.py..... solved my issue it asked for selenium and slugify and i used python-slugify it worked like charm thanks for the scraper and many many thanks for your support.
Welcome.