Proposal: integrate MSW with Pact (pact.io)
mefellows opened this issue · comments
Is your feature request related to a problem? Please describe.
Mocking APIs makes is preferable to end-to-end integration testing, because mocking enables:
- Fast feedback
- Requires fewer dependencies
- Does not need a dedicated test environment
- Reliable (not flakey) tests
- Simple debugging
You create sets of unit tests on either side of a boundary:
The problem with mocking APIs, is that you don't have confidence that your mocks are genuine representations of reality, and can lead to integration issues elsewhere (e.g. in production):
Describe the solution you'd like
Pact is a tool that uses a process known as consumer-driven contract-testing to capture the expectations from the API consumer, and replay them against the API provider to ensure these mocks are kept in sync. By using MSW as the capture mechanism (replacing the existing consumer test process) and serialising them into a Pact compatible contract (as defined by the Pact specification), we could use MSW with confidence knowing the mocks don't drift from reality.
Describe alternatives you've considered
n/a
Additional context
As discussed at the recent TestJS Summit
Opening up for conversation here.
I could see this being valuable to allow a front end team to work in parallel or advance of a service team's development effort. If there is interest in the integration, would the next step be a proof of concept in translating the MSW mock format to pact's format?
Yes! I don't know enough about the internal workings of MSW at the moment. So a rough sketch of what a solution might look like would be useful.
From my understanding of MSW, it really is designed to work completely within the browser. I don't think it's possible, but establishing a tunnel from the service worker to a mock on the host would be the most ideal.
Assuming it's not, here are a few pieces of the puzzle I think that are needed.
- A general lib that is able to translate abstract definitions into a Pact format, following the pact specification (this is something I could definitely contribute to/guide). Potentially, we could use WASM from our Rust core for this purpose
- A plugin for MSW that allows configuration of the Pact bits, and extracts the mocks. Either by reading any serialised versions of the MSW mocks or by hooking into the request/response lifecycle via the events that MSW emits that dispatches to the library to capture the interactions.
- Somehow, a Pact file needs to be written or made available to the user to download - this is the crucial artifact that is used for the purposes of contract testing
From (3) standard Pact tooling should be able to work.
How far off reality am I here?
Hey, @mefellows. Thank you for opening this proposal, I think the integration with Pact would be beneficial to the end-users.
From my understanding of MSW, it really is designed to work completely within the browser.
Partially. MSW can run in both browser and Node. When it runs in a browser it has 2 counterparts: a worker and a client. While the worker is detached, the client is the mock definitions you write that are in your code. The worker messages the client about the request/response events and so the client may use those events to establish a channel with Pact, given that's possible from the client-side code.
Something I wish to understand better is what would be the relation between Pact and MSW, specifically:
- Would Pact's user-defined contract act like an input to MSW?
- Would the requests/responses intercepted/produced by MSW act as an input to Pact?
- What setup would be required from the developer to achieve this integration?
A diagram of some sort would be useful to show what happens to a request made on the page.
hi @mefellows @kettanaito ,
I'd hoped to start looking at this next week. I've been slammed this week.
The way I picture it, is providing a way to generate the pact contract on commit. A developer can work and generate the necessary msw mocks as normal. Once they are ready to push their changes, they could run this plugin to convert the msw mock into an updated contract in the pact format. We would be trying to keep the consumer contract up to date with any changes to the mocks that the client is generating. If there is a diff, then we would know to update the pact broker. Does that flow make sense or am I over simplifying what happens on either end?
Thank you for that diagram, @krulletc! It makes things much more clear now.
Is it correct to state this is the workflow for the end user:
write request handlers (MSW)
-> git:commit
-> createPactContractFrom(handlers)
-> (?) run Pact in CI against the committed contract
I think we should be agnostic about how the contract is generated from MSW (e.g. Husky is just one trigger for it), but what you say sounds about right.
I think it's still worth exploring the possibility of Pact to define the MSW mocks (and therefore it will be able to generate the pact contract quite easily). There are some limitations to generating a Pact contract from other mocks - namely, you lose out on some of the scenario naming and things called Matchers (that we would need to infer). Here as a screenshot from an example contract
Without the scenario name and state, we'd need to autogenerate it.
Not a problem, just worth considering.
I think preserving the MSW experience as much as possible is still a principle we'd like to follow, because it's what makes MSW so popular!
Yes, definitely be tool agnostic on how the pact would be generated.
Based on what you are saying, it sounds like I could explore generating the MSW mock from the pact contract first
Sounds great. Did you want to take a crack at spiking the approaches to get a sense of what might be best?
As an aside, I spiked a Cypress plugin for Pact a while back (and we now use a variant of this at Pactflow): https://github.com/pactflow/example-consumer-cypress/tree/master/cypress
It overrides the old cy.route command, but a newer version would use the cy.route2 which can proxy arbitrary requests.
It works quite well and it might help with this project that has similar goals.
FYI the Pact maintainers and community hang out at slack.pact.io if you wanted to reach out. The #pact-js-development channel is probably the most appropriate if you wanted to open a thread on this.
Based on what you are saying, it sounds like I could explore generating the MSW mock from the pact contract first
That sounds like a great starting point. Our team has been thinking about the idea of a "sources" package that would provide the developers with various ways to generate request handlers. I can see how handlers based on Pact contracts may be an amazing illustration and a practical use-case for such a package.
I can only encourage to cross-share our findings so each side learns more about the other. For instance, you can prepare a minimal meaningful contract definition for Pact and our team can experiment with turning it into request handlers (or vice-versa).
Technically speaking, the rest.* API should be sufficient to create a basic mapping based on the Pact's contract.
{
"interactions": [
{
"request": {
"method": "GET",
"path": "/mallory",
"query": "name=ron&status=good"
},
"response": {
"status": 200,
"headers": {
"Content-Type": "text/html"
},
"body": "That is some good Mallory."
}
}
]
}The above contract representation in MSW's request handlers:
rest.get('/mallory', (req, res, ctx) => {
return res(
ctx.set({ 'Content-Type': 'text/html' }),
ctx.body('That is some good Mallory.')
)
})Note that
200is the default response status code in MSW.
This kind of transformation can be roughly implemented like this:
import { rest, setupWorker, response, context } from 'msw'
function contractToHandlers(contract) {
return contract.interactions.map((interaction) => {
const { method, path } = interaction.request
return rest[method](path, () => createResponse(interaction.response))
})
}
function createResponse(expectedResponse) {
const { status, headers, body } = expectedResponse
const transformers = [
context.set(status),
headers && context.set(headers),
body && context.body(body)
].filter(Boolean)
return response(...transformers)
}
setupWorker(contractToHandlers({ interactions: [/* ... */] }))There may be some gotchas along the way, like MSW ignoring query parameters in a request URL, from which we can only learn and find the best solution.
That "sources" idea looks great @kettanaito! Using the Cypress plugin as a guide, that should be fairly straightforward to implement.
One thing we may want to consider, is that one of the principals of Pact is that any mock that is setup must be used (i.e. it's a mock and not a stub), otherwise a contract may be written with expectations in it that may not have actually been required/supported by the client code. This is bad, because we would force the API provider to ensure it can support it, even if not actually needed.
So one side-effect of this, is that we need to be cautious when using Pact as a source in combination with other sources - because we may get a false sense of security (e.g. "our Pact tests are passing, so we must be safe to release"). But if there are other routes served by MSW that aren't covered by the contract, this statement is not true.
I can see some other challenges (on the Pact side), but they'll just be things we'll need to consider as we move forward through it.
But if there are other routes served by MSW that aren't covered by the contract, this statement is not true.
As long as MSW handlers are derived from the Pact contract that should never happen, shouldn't it? Or perhaps you take into account some additional handlers that a developer may define alongside those generated from the contract? I see, then it'd breach the exclusive nature of the contract-based assertions.
We haven't defined strict boundaries as to whether multiple sources can be combined to compose a set of handlers. However, it's likely to be the case, as I don't see MSW being strict when it comes to the choice of a developer on what to based the mocks on. This kind of discussion is invaluable in shaping the future API of such "sources".
As long as MSW handlers are derived from the Pact contract that should never happen, shouldn't it? Or perhaps you take into account some additional handlers that a developer may define alongside those generated from the contract? I see, then it'd breach the exclusive nature of the contract-based assertions.
That's correct. I'd say this is more of a documentation problem (perhaps also the Pact plugin could detect such cases and warn the user?).
We haven't defined strict boundaries as to whether multiple sources can be combined to compose a set of handlers. However, it's likely to be the case, as I don't see MSW being strict when it comes to the choice of a developer on what to based the mocks on. This kind of discussion is invaluable in shaping the future API of such "sources".
Yep. I don't think MSW should restrict the number / type of sources (unless there is a technical reason of course). But just something to be aware of from a Pact perspective.
So I thought I should mention a third use case that is probably the easiest to support, which is to take an existing pact file (the generated artifact that usually produced by a separate test case). Taking an existing already generated pact files should be almost trivial with the API described above.
I'm not sure how useful it is as I still need to fully wrap my head around the best use cases here, but felt best to mention for posterity. But I think the use case is this:
- User has already used Pact for unit testing the API client/layer (and thus has generated a pact file previously - locally or hosted via a Pact Broker)
- User wants to re-use the pact file as the stub for integrations at the web tier (e.g. a React test using React Testing Library), but wants to guarantee that any stubs used for those tests don't drift from the actual provider implementation
- User only uses the pact file as a source
For (2) you could choose how to implement the stub:
a) start a pact stub server with the pact file as an input, and redirect requests to it
b) parse the pact file, and convert into MSW compatible stubs (I think this is probably preferable for speed etc.)
Hey hey, this piqued my interest as I came across msw yesterday reading a video.
Anyway based on the work done by @kettanaito, msw mocks setup from a pact file
https://github.com/YOU54F/msw-pact/blob/main/consumer/src/setupMswFromPact.js
which when used in a test looks like this. https://github.com/YOU54F/msw-pact/blob/main/consumer/src/mswFromPact.msw.spec.js
Probably more useful is this
https://github.com/YOU54F/msw-pact/blob/main/consumer/src/pactFromMsw.msw.spec.js
which will generate a pact file object from a matched msw req/res
import { rest } from "msw";
import { setupServer } from "msw/node";
const server = setupServer();
server.listen();
const requestMatch = new Promise((resolve) => {
server.on("request:match", resolve);
});
const responseMocked = new Promise((resolve) => {
server.on("response:mocked", resolve);
});
Promise.all([requestMatch, responseMocked]).then((data) => {
console.log("Request matched and response mocked");
const request = data[0]; // MockedRequest<DefaultRequestBody>
const response = data[1]; // IsomorphicResponse;
const pactFile = convertMswMatchToPact(request, response);
});
server.on("request:unhandled", (unhandled) => {
const { url } = unhandled;
console.log("This request was unhandled by msw: " + url);
});
The msw res/req is mapped to a pact object here https://github.com/YOU54F/msw-pact/blob/main/consumer/src/convertMswMatchToPact.js
Hey hey, so I’ve released msw-pact which will intercept a mock-service-worker request/response and transform it into a pact
https://github.com/YOU54F/msw-pact
wow, this is awesome. I've been stretched thin on multiple work projects so never dug in. Glad you did @YOU54F !
Amazing Yousef! I'll give this a test run over the next little while.
I like the idea of the plugin supporting multiple modes (generating pacts and also using Pacts as a source). This is a similar model we're looking to do for Cypress.
The other things on my mind are how we configure multiple mocks with different providers, e.g. it's possible for a consumer to have multiple API providers, do we need a way to configure that or is that already covered?
Hey MSW team, thanks for an awesome package, I only discovered it recently and it's a joy to use.
I'm currently setting up a new project and was hoping to find a way to integrate pact and MSW, so was delighted to find this thread.
I was wondering how things are going with this as the conversation has stopped - are people successfully using the msw-pact package from YOU54F?
Thanks!
How do @MerlinMason , I've just been chatting to @IJuanI who have been doing some really cool stuff with msw-pact with a fork, over at his company.
Ahh nice, thanks @YOU54F, I'll get started working with his fork and maybe in the future we'll see this built into MSW :)
Hey @kettanaito,
This is now released https://github.com/pactflow/pact-msw-adapter
I wrote a quick start guide for our website https://docs.pactflow.io/docs/bi-directional-contract-testing/tools/msw
and have a blog post due to go out soon, would be great to connect with you about reviewing and syncing up
Hey, @YOU54F. Sorry for such a late reply! That looks absolutely stunning! Thank you for your work on this. Excited to see people improving their products with proper contract-driven testing and MSW in the same box.
I will close the issue then as I believe the initially proposed functionality is achieved by pact-msw-adapter. I'm open to any discussions in regards to how make our integration easier from the MSW's side. I'm sorry to admit I didn't have time to look into this before, I've got overloaded with what seems an infinite number of other tasks. I hope for your understanding on that.
Of course - totally get it @kettanaito! Thanks for your support and keep up the awesome work on MSW (and your other "side" projects 😆 )