[Solved] getSongs() doesn't return all songs from an artist
brandhsu opened this issue · comments
Describe the bug
getSongs() fails to retrieve all songs from an artist
To Reproduce
from azapi import AZlyrics
import pprint
API = AZlyrics('google', accuracy=0.5)
API.artist = 'bangtanboys'
songs = API.getSongs()
pprint.pprint(songs, indent=2)
printed json
{
"24/7=Heaven": {
"album": "Dark & Wild",
"type": "album",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/247heaven.html",
"year": "2014"
},
"2nd Grade (2학년)": {
"album": "Dark & Wild",
"type": "album",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/2ndgrade2.html",
"year": "2014"
},
"Attack On Bangtan (진격의 방탄; Jingyeogui Bangtan)": {
"album": "O!RUL8,2?",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/attackonbangtanjingyeoguibangtan.html",
"year": "2013"
},
"BTS Cypher Pt. 1": {
"album": "O!RUL8,2?",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/btscypherpt1.html",
"year": "2013"
},
"BTS Cypher Pt. 2: Triptych": {
"album": "Skool Luv Affair",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/btscypherpt2triptych.html",
"year": "2014"
},
"BTS Cypher Pt. 3: Killer": {
"album": "Dark & Wild",
"type": "album",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/btscypherpt3killer.html",
"year": "2014"
},
"Blanket Kick (이불킥)": {
"album": "Dark & Wild",
"type": "album",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/blanketkick.html",
"year": "2014"
},
"Boy In Luv (상남자; Sang-Namja)": {
"album": "Skool Luv Affair",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/boyinluvsangnamja.html",
"year": "2014"
},
"Can You Turn Off Your Cellphone? (핸드폰 좀 꺼줄래)": {
"album": "Dark & Wild",
"type": "album",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/canyouturnoffyourcellphone.html",
"year": "2014"
},
"Coffee": {
"album": "O!RUL8,2?",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/coffee.html",
"year": "2013"
},
"Danger": {
"album": "Dark & Wild",
"type": "album",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/danger.html",
"year": "2014"
},
"Does That Make Sense? (Outro) (그게 말이 돼? (Outro))": {
"album": "Dark & Wild",
"type": "album",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/doesthatmakesenseoutrooutro.html",
"year": "2014"
},
"Hip Hop Lover (힙합성애자)": {
"album": "Dark & Wild",
"type": "album",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/hiphoplover.html",
"year": "2014"
},
"I Like It (Slow Jam Remix) (좋아요; Joh-Ayo)": {
"album": "Skool Luv Affair",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/ilikeitslowjamremixjohayo.html",
"year": "2014"
},
"I Like It (좋아요; Joayo)": {
"album": "2 Cool 4 Skool",
"type": "single",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/ilikeitjoayo.html",
"year": "2013"
},
"If I Ruled The World": {
"album": "O!RUL8,2?",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/ifiruledtheworld.html",
"year": "2013"
},
"Interlude: What Are You Doing? (Interlude:뭐해; Interlude: Mwohae)": {
"album": "Dark & Wild",
"type": "album",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/interludewhatareyoudoinginterludeinterludemwohae.html",
"year": "2014"
},
"Intro: 2 Cool 4 Skool": {
"album": "2 Cool 4 Skool",
"type": "single",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/intro2cool4skool.html",
"year": "2013"
},
"Intro: O!RUL8,2": {
"album": "O!RUL8,2?",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/introorul82.html",
"year": "2013"
},
"Intro: Skool Luv Affair": {
"album": "Skool Luv Affair",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/introskoolluvaffair.html",
"year": "2014"
},
"Jump": {
"album": "Skool Luv Affair",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/jump.html",
"year": "2014"
},
"Just One Day (하루만; Haruman)": {
"album": "Skool Luv Affair",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/justonedayharuman.html",
"year": "2014"
},
"Let Me Know": {
"album": "Dark & Wild",
"type": "album",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/letmeknow.html",
"year": "2014"
},
"Look Here (여기 봐)": {
"album": "Dark & Wild",
"type": "album",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/lookhere.html",
"year": "2014"
},
"Miss Right": {
"album": "Skool Luv Affair",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/missright.html",
"year": "2014"
},
"N.O": {
"album": "O!RUL8,
2?",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/no.html",
"year": "2013"
},
"No More Dream": {
"album": "2 Cool 4 Skool",
"type": "single",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/nomoredream.html",
"year": "2013"
},
"Outro: Luv In Skool": {
"album": "O!RUL8,
2?",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/outroluvinskool.html",
"year": "2013"
},
"Outro: Propose": {
"album": "Skool Luv Affair",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/outropropose.html",
"year": "2014"
},
"Path (길; Gil)": {
"album": "2 Cool 4 Skool",
"type": "single",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/pathgil.html",
"year": "2013"
},
"Rain": {
"album": "Dark & Wild",
"type": "album",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/rain.html",
"year": "2014"
},
"Satoori Rap (팔도강산; Paldogangsan)": {
"album": "O!RUL8,2?",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/satoorirappaldogangsan.html",
"year": "2013"
},
"Skit: Soulmate": {
"album": "Skool Luv Affair",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/skitsoulmate.html",
"year": "2014"
},
"Spine Breaker (등골브레이커; Deung-Gol Beuleikeo)": {
"album": "Skool Luv Affair",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/spinebreakerdeunggolbeuleikeo.html",
"year": "2014"
},
"Tomorrow": {
"album": "Skool Luv Affair",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/tomorrow.html",
"year": "2014"
},
"War Of Hormone (호르몬 전쟁)": {
"album": "Dark & Wild",
"type": "album",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/warofhormone.html",
"year": "2014"
},
"We Are Bulletproof Pt. 2": {
"album": "2 Cool 4 Skool",
"type": "single",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/wearebulletproofpt2.html",
"year": "2013"
},
"We On": {
"album": "O!RUL8,2?",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/weon.html",
"year": "2013"
},
"What Am I To You (Intro)": {
"album": "Dark & Wild",
"type": "album",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/whatamitoyouintro.html",
"year": "2014"
},
"Where Did You Come From? (어디에서 왔는지; Eodieseo Wassneunji)": {
"album": "Skool Luv Affair",
"type": "EP",
"url": "http: //www.azlyrics.com/lyrics/bangtanboys/wheredidyoucomefromeodieseowassneunji.html",
"year": "2014"
}
}
Expected behavior
When going to the url: https://www.azlyrics.com/b/bangtanboys.html
we can see how songs have been chopped off in the print statement when compared to the actual list of songs on AZlyrics.com.
I believe this issue has to do the the ParseSongs() method in tools.py.
AZlyrics seems to include google adds in one under a certain div tag and I think this is causing early termination possibly.
Screenshots
Desktop (please complete the following information):
Using macOS Majove
Additional context
Add any other context about the problem here.
Thanks @brandhsu,
I will do my best to solve this issue ASAP.
I tested azapi and I found that in tools.py inside ParseSongs function, I wrote this within the main loop over listAlbum items:
...
if elmnt.name == 'script':
break
...And because AZLyrics.com updated their website with new Google Ads mechanism which now exists between songs under listAlbum dev and uses a script tag, the loop terminates itself when facing the first Google Ad causing many songs not to be included.
Problem was fixed when this code is removed.
I will update azapi to v3.0.3 ASAP
Now pip install azapi==3.0.3 to fix your issue :)