Ladder malforms https links
hyferg opened this issue · comments
To reproduce:
- Download the v0.0.20 ladder_darwin_arm64 release
- Run ladder with no args
- Go to "http://localhost:8080/raw/https://techcrunch.com/2023/11/30/anduril-unveils-roadrunner-a-fighter-jet-weapon-that-lands-like-a-falcon-9/"
You will notice that links on the page to techcrunch are malformed like below with '/' placed before the scheme and after the top level domain.
<a href="/https://techcrunch.com//author/aria-alamalhodaei/">
Ok it looks like line the rewriteHtml function at line 252 in handlers/proxy.go is doing this, but why?
body = strings.ReplaceAll(body, "href=\"https://"+u.Host, "href=\"/https://"+u.Host+"/")
@hyferg Hi Cameron, this issue should be fixed in the next v2 release.
Great, I'm guessing that this is done so that when you're in the ui, ladder can intercept links to the host and keep you in ladder when you click them?
Yes, the next version is much more reliable at doing that.
You're welcome to try it out, the branch is origin/proxy_v2. For the techcrunch site, try the new localhost:8080/outline/<site> path.
Nice to hear!
I had this error on the origin/proxy_v2
ProxyChain error for '': invalid referer URL: '' on request '....'
Do you have any rough idea of when the proxy_v2 will be stable or merged to main?
I think for now I will just make a custom build of v0.0.20 with the href rewriting commented out. My use case only needs to get properly formed page data via api reliably.
Oh, then try the /api/content endpoint, perhaps that's what you need. Are you looking for something in particular?
ProxyChain error for '': invalid referer URL: '' on request '....'
This usually happens when the URL is malformed somehow. Still ironing out the bugs.
Not sure when v2 will be stable, I haven't implemented ruleset support yet.
Hey I patched v0.0.20 for my personal use by commenting out most of the links in the rewriteHtml function. How that behaves is what I'm looking for in particular!