Whats the issue stopping me from using this on other websites?

[Jordan Coeyman]

Love the package thanks for the work :)

I'm trying to get https://github.com/GuyPaddock/harmon-binary/blob/master/examples/doge.js to work, but simply with another website. The example runs great, but once I switch out nodejs.org for anything else (ie: google.com), the requests mess up.

Any tips? (I'm trying to run this from a wildcard - *.example.com)

[Guy Elsmore-Paddock]

Hi Jordan,

First, it looks like the Harmon-Binary package pull request was accepted upstream in harmon itself, so you might be better off using harmon directly. I haven't tested it myself just yet, but once I do, I'll be dropping this separate project from Github.

Secondly, when you say the the requests "mess up", what do you mean? Are multiple requests running over each other, is the output garbled, or are you getting an error message?

One other important thing to pay attention to is how you end up replacing or removing content in the stream. I tend to use the "through" module with NodeJS for all of my edits or content removals, like so:

func: function(htmlNode, req, res) {
// "outer: true" => modify the matching element.
var stream = htmlNode.createStream({outer: true});

stream.pipe(through(function (buf) {
// Pipe nothing through to the document; just eliminate the element.
})).pipe(stream);
}

The above would completely remove the target element from the output.

Thanks!

Sincerely,

Guy A. Paddock

Chief Executive Officer
RedBottle Design

[phone]
----- Original Message -----

From: "Jordan Coeyman" [e-mail]
To: "GuyPaddock/harmon-binary" [e-mail]
Sent: Thursday, December 18, 2014 10:35:16 AM
Subject: [harmon-binary] Whats the issue stopping me from using this on other websites? (#1)

Love the package thanks for the work :)
I'm trying to get https://github.com/GuyPaddock/harmon-binary/blob/master/examples/doge.js to work, but simply with another website. The example runs great, but once I switch out nodejs.org for anything else (ie: google.com ), the requests mess up.
Any tips? (I'm trying to run this from a wildcard - *.example.com)
—
Reply to this email directly or view it on GitHub .

[Jordan Coeyman]

Guy, thanks for the super quick response..

I'm not even trying to modify anything (yet) on the server I am attempting to proxy to. I just wanted to get the proxy up and running, then I was going to do the modifications I needed.

I can't seem to understand what exactly is going on.. No errors in the server console.

I get a different browser error based on the server I am trying to proxy to upon request.

For example I see an nginx 502 bad gateway on my own site, when I point the proxy to Google I see a custom Google "Not Found" error, and when I point it to clickhole.com (random site, satire news) it spits a custom 503 service unavail. The example of nodejs.org works perfectly though.

[Guy Elsmore-Paddock]

Hmm, it sounds like the request headers aren't being passed through properly. You may need to rewrite the "host" header that is being passed through the proxy to make sure that the site you're proxying knows which site to serve-up.

Typically, the browser sends the site a request header that says "Ok, I'm vising google.com, so send me the content for that site" and then the server knows which site its serving up (in case multiple sites are hosted on the same IP / machine).

In another project, we use this snippet of code to create the proxy:

var proxy = httpProxy.createProxyServer({
target: targetProto + '://' + targetTld,
secure: false,
headers: {
// Force the host to match the target.
host: targetTld,
}
});

Where httpProxy is the http-proxy module (created via require() ), targetProto is either "http" or "https", targetTld is the target domain name (like "google.com" or "facebook.com", etc).

----- Original Message -----

From: "Jordan Coeyman" [e-mail]
To: "GuyPaddock/harmon-binary" [e-mail]
Cc: "Guy Paddock" [e-mail]
Sent: Thursday, December 18, 2014 10:49:34 AM
Subject: Re: [harmon-binary] Whats the issue stopping me from using this on other websites? (#1)

Guy, thanks for the super quick response..
I'm not even trying to modify anything (yet) on the server I am attempting to proxy to. I just wanted to get the proxy up and running, then I was going to do the modifications I needed.
I can't seem to understand what exactly is going on.. No errors in the server console.
I get a different error based on the server I am trying to proxy to.
For example I see an nginx 502 bad gateway, when I point the proxy to Google I see a custom Google "Not Found" error, and when I point it to clickhole.com (random site, satire news) it spits a custom 503 service unavail. The example of nodejs.org works perfectly though.
—
Reply to this email directly or view it on GitHub .

[Jordan Coeyman]

Guy you rock, thank you for your help. Got it working with your help 👍