sc-forks / solidity-coverage

I am in the process of migrating an existing Truffle codebase over to Hardhat. Many of our tests use packages which rely on inter-process RPC calls to an external Ethereum node. This limits our ability to use Hardhat's in-process node for the time being.

When trying to set up solidity-coverage, it seems as though support exists for external nodes only if they are Ganache clients. This is indicated in both the docs and the code.

Is it possible to run solidity-coverage against a generic RPC node? In our case, an instance of npx hardhat node, but potentially anything.

@kronosapiens Unfortunately the ganache support in the code is potemkin and exists because the test suite continues to use it (scheduled for removal in #839). The docs are out of date - solidity-coverage stopped working with the most recent major version of ganache and now it's deprecated.

Is it possible to run solidity-coverage against a generic RPC node? In our case, an instance of npx hardhat node, but potentially anything.

Could you give an example in the Colony code where you were doing this so I can understand how it might be replicated?

We are essentially doing this:

npx hardhat node & npx hardhat test --network localhost

I'd like to be able to do something similar with coverage, along the lines of

npx hardhat node --port 8555 & npx hardhat coverage --network coverage

Assuming the coverage network is properly defined in hardhat.config.js. Looking at the coverage code, it seems like the branch only considers in-process hardhat nodes or external Ganache clients. Seems like the culprit is right around here, since the external node returns false for that flag (or at least that's what I think is going on).

@kronosapiens Ok. Ganache was also always launched "in-process" here but it listens on a port like a server

solidity-coverage/lib/api.js

Line 185 in fbed395

await pify(this.server.listen)(this.port);

Am going to have to look at the Hardhat code more closely to what can be done here to support this. If you could show me an example of an "inter-process RPC call to an external Ethereum node" it would help to build a test case here.

Here's an example of pointing an accessory service to the RPC node.

It seems like in this case, generalizing the handling of external nodes would be a sufficient solution. Mimicking how Hardhat itself handles the --network flag might sort it, using that to pull up the hardhat.config.js network information and setting up a provider based off of that.

Great, that's perfect. Will look into this....

@kronosapiens I've made a sample project which shows how you can have two blockchains available in the unit tests here:

https://github.com/cgewecke/multi-provider-hardhat-example

These are two chains in the same process, instead of IPC. Hardhat's default network client (the one solidity-coverage hijacks) doesn't listen on a port and accept outside calls.

It's also conceivable that you could roll your own coverage-enabled standalone hardhat node by:

using the solidity-coverage API and
overloading the task defined in hardhat's built-in node task

@cgewecke thanks for looking into this! I can't view your repo, is it private?

@kronosapiens Yes ....it was, really sorry. It's public now.

There's an example of using the second blockchain in the final test in the test file.

@cgewecke thanks for that. Looks like the solution of having two in-process providers isn't quite what we need, but your idea of implementing a coverage-compliant external node is something I'll keep looking into. Talking to Alex, it seems like the previous in-process Ganache provider accepted external RPC calls, so it might not be as simple as I'd hoped to migrate that behavior over to Hardhat.

@cgewecke turns out it wasn't as hard as we'd feared to make the in-process node available externally. Alex came up with this workaround which has been serving us just fine:

task("coverage", "Run coverage with an open port").setAction(async () => {
  const app = express();
  const port = 8555;

  app.use(bodyParser.json());
  app.post("/", async function (req, res) {
    const response = await hre.network.provider.request(req.body);
    res.send({ jsonrpc: "2.0", result: response, id: req.body.id });
  });
  app.listen(port, function () {
    console.log(`Exposing the provider on port ${port}!`);
  });

  await runSuper();
});

@kronosapiens Wow, nice! I thought it would be a lot harder than that 😄

Running solidity-coverage against a generic RPC node