Giters

Kagami / tistore

:camera: Tistory photo grabber

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Recent site changes cause program to not work

Silenciuse opened this issue · comments

commented

Hey, they changed their URLs again, this time to something completely different.

Here's some examples, same deal as last time where "?original" can be added to the end:
https://k.kakaocdn.net/dn/bcF4AQ/btqtUkQPhad/g6Sl50hXCukzakdKrm3K4k/img.jpg
https://boxgame.tistory.com/2475

Apparently they can have download buttons now too, although this is the only blog I've seen with it enabled:
https://farbutnear0305.tistory.com/64

Also every file now downloads with the same name, which is really annoying.

Thanks for creating and maintaining this program.

commented

Thanks, I will look into it.

commented

Thanks for the second link, it shows how make it return Content-Disposition header:

curl -I 'https://k.kakaocdn.net/dn/uhnRH/btqtUjkd9re/P18KTYzrsiiS7qimPPui2K/img.jpg?attach=1&nm=HappyIreneDay201.jpg'

Funny that it will return anything you pass in nm parameter. Should work fine with current architecture of tistore though.

Also seems like old blogs haven't migrated to the new link scheme, so we need to support both options.

commented

Seems like kakaocdn servers violate RFC and return Content-Disposition in the wrong format so aria2c doesn't accept that.

Compare:

$ curl -sI 'https://t1.daumcdn.net/cfile/tistory/99F063505B1ABCFE22?original' | grep -i disposition
content-disposition: inline; filename="DSC_8301.jpg"; filename*=UTF-8''DSC_8301.jpg

$ curl -sI 'https://k.kakaocdn.net/dn/uhnRH/btqtUjkd9re/P18KTYzrsiiS7qimPPui2K/img.jpg?attach=1&nm=HappyIreneDay2019.jpg' | grep -i disposition
Content-Disposition: attachment; filename="HappyIreneDay2019.jpg"; filename*="UTF-8''HappyIreneDay2019.jpg"

Note the extra quotes in UTF-8 filename in second case.

I can make patched version of aria2c binary but that feels very hackish. Or maybe I should write to Kakao devs and ask them to fix the header…

commented

Wow, I can't believe it, but they actually gave pretty useful workaround that quickly. Instead of

https://k.kakaocdn.net/dn/uhnRH/btqtUjkd9re/P18KTYzrsiiS7qimPPui2K/img.jpg?attach=1&nm=HappyIreneDay2019.jpg

you can use

https://k.kakaocdn.net/dn/uhnRH/btqtUjkd9re/P18KTYzrsiiS7qimPPui2K/HappyIreneDay2019.jpg?knm=img.jpg

which works perfectly fine with aria2c.

I suppose it's not documented anywhere but should work perfectly fine for our needs. Just need to rewrite URL a bit. Hope it won't have issues with escaping/Unicode.

commented

Please try 0.5.2 release.
It works with your examples but escaping/unicode handling might be not 100% correct, so I need some feedback.