Installing httptoolkit-server for use as a library

[Temm]

Since scripting in the UI is not a thing yet, I wanted to install this package to script responses that way (since, from my understanding, I can use it as a library to accomplish this).
Unfortunately it seems like the npm release is deprecated

HTTP Toolkit Server is no longer published to npm. Latest releases are available from https://github.com/httptoolkit/httptoolkit-server/releases

Although I couldn't find any reason for this or discussion about it.
Trying to npm install it from github (since npm supports github as a package source) using
npm i httptoolkit/httptoolkit-server#v1.14.1
installs the package from github, but seems to only end up with the overrides and the bin folder, which doesn't match up with the content of the windows release download for 1.14.1, and seems to be missing core files required in order to use the package.

How do I use httptoolkit-server as a library (without going the c-style way of just cloning it into a folder in my project)?
Why was the npm download deprecated?

[Tim Perry]

The official release source is GH releases: https://github.com/httptoolkit/httptoolkit-server/releases.

Just to document some of the reasoning: it's not published to npm because the releases aren't really a source package, and so the npm published version that used the sources was very odd, and risked ending up with different behaviour unexpectedly to the actually tested release builds. For example, the full release bundles & minifies all the JS code (which hugely shrinks things & speeds up startup), it includes cross-builds automatically to include prebuilt native libraries for each of the binary dependencies we use for the appropriate platform in each case, and includes the officially supported & tested version of node (there are quite a few cases where different node versions will do unexpected things, since we're really into the HTTP internals in some places). There's also other details like the overrides/js folder, which has its own package.json and dependencies that need installing separately, and the resulting package was very large for an npm package (6MB excluding dependencies) etc etc.

It's certainly possible to distribute that via npm, but it's not really how npm was intended to be used.

How do I use httptoolkit-server as a library?

In short - that's not officially supported. Of course, you're still very welcome to do so, but it's a very different form of distribution, that download stats suggested effectively nobody was using, so it was unnecessary effort to maintain while being hard to formally guarantee as working.

Since scripting in the UI is not a thing yet, I wanted to install this package to script responses that way (since, from my understanding, I can use it as a library to accomplish this).

In more positive news though, based on this, I don't think you want to run this as a library anyway. For starters, doing so won't make it much easier to script responses.

I think you're looking for https://github.com/httptoolkit/mockttp/ which absolutely is intended to be used as a standalone library, and is on npm. That's the HTTP intercepting proxy & engine & rules as a standalone JS library you can directly script.

You can use that standalone as a headless proxy, manually configuring clients elsewhere to use it, or you can use HTTP Toolkit and all the interception setup/UI/etc alongside your custom proxy, and configure this Mockttp proxy as your upstream proxy on the Settings page, so its scripting is effectively applies to all traffic that's passed upstream from HTTP Toolkit.

Does that make sense?

[Temm]

Yeah, that reasoning does make sense.

or you can use HTTP Toolkit and all the interception setup/UI/etc alongside your custom proxy, and configure this Mockttp proxy as your upstream proxy on the Settings page, so its scripting is effectively applies to all traffic that's passed upstream from HTTP Toolkit.

That's a great idea, I will test that later! I was interested in using this as a library for the auto-setup interceptors, but if simply chaining the proxy behind HTTP toolkit works, that will work just as well for manual use.

Thanks!

[Temm]

Putting my basic boilerplate for this here, just in case anyone else will attempt this in the future.

mockttp server code

import mock from "mockttp";
import fs from "fs"
import { resolve } from "path"

try {
    // Check if ./certificate.pem and ./key.pem exist
    fs.accessSync('./certificate.pem', fs.constants.R_OK);
} catch {
    console.log("Generating CA certificate")
    mock.generateCACertificate().then((ca) => {
        fs.writeFileSync('./certificate.pem', ca.cert)
        fs.writeFileSync('./key.pem', ca.key)
    })

    let absolutePath = resolve("./certificate.pem")

    console.log("Generated and saved at", absolutePath, "& .../key.pem")
    console.log("Make sure to import certificate.pem as a CA cert in the HTTP Toolkit proxy options");
}

const server = mock.getLocal({
    https: {
        certPath: "./certificate.pem",
        keyPath: "./key.pem"
    }
});

async function main() {
    await server.forUnmatchedRequest().thenPassThrough({
        beforeRequest: (req) => {
            console.log("Got request", req.url)
        }
    })
    
    console.log("Started mockttp")
    
    await server.start(8085);
}

main();