This repo was forked from nubank. I've uploaded from my local machine when it was not public anymore.
A simplistic example of how Nubank organizes and tests microservices.
This is a really basic savings account microservice that doesn't really do much other than serve as an example.
At Nubank we organize our microservices using the "ports and adapters architecture", also known as "hexagonal architecture".
With this architecture, code is organized into several layers: logic,
controllers, adapters, and ports.
Deals with pure business logic and shouldn't have side-effects or throw exceptions.
The "glue" between all the other layers, orchestrating calls between pure business logic, adapters, and ports.
The layer that converts external data representations into internal ones, and vice-versa. Acts as buffer to protect the service from changes in the outside world; when a data representation changes, you only need to change how the adapters deal with it.
The layer that communicates with the outside world, such as http, kafka, and datomic.
We use the components abstraction
to organize our ports (e.g. HTTP client, datomic client, redis client) and any
other logic that needs to track mutable state or encode dependencies between
stateful components. For every environment (e.g. test, e2e, prod, staging...) we
have a different version of our component systems, enabling us to easily inject
mocks or different implementations for different contexts.
We make components available to incoming http and kafka handlers. For instance, the pedestal http handlers have access to things like the datomic or HTTP components, and pass them down to the controller level for general use.
Our http client logic is split into two components:
http: this component defines serialization and error handling logic. In this example repository the this logic is basically non-existent due to the overhead making the code useful to the general public.http-impl; this component defines the http client library we use. We started withhttp-kitbut have recently migrated away from this tofinagledue to stability issues.
In the case of this example service, we define a rudimentary in-memory storage component. In our actual services we generally use a datomic component.
We use pedestal for or http serving layer, but we deconstruct pedestal logic into several different components, deviating from the structure you would see in the pedestal starter template.
Encapsulates the pedestal http routes. This example project doesn't make use of this abstraction, but in Nubank's internal microservices we use the routes component to give us the ability to create bookmarks for urls and reference them in various contexts, like our http client component. In addition, we can extend the routes programmatically with operational routes related to other components, for instance providing http routes for starting and stopping the topic consumer in our kafka component.
Builds the pedestal service conifguration. Since it defines the interceptors for
the http handlers, this service component needs to depend on all components we
want to be available in those handlers, such as the http client and storage
client.
Contains logic to start the servlet
For instance the difference between the dev and mock servlets is that the mock servlet, used in integration tests, creates the server but doesn't actually start it.
At Nubank we use Midje as our test framework.
We've structured our integration tests to follow a world-transition system that
is encoded in the selvage flow macro.
Lastly, to check the form of nested data-structures during testing we employ
matcher-combinators.
Straw-man examples of what our unit tests may look like can be found in
controller-test
A straw-man example of how we do integration testing can be found in
account-flow
More of an explanation of how selvage flow tests work can be in the selvage
repository.
lein midje :autotest
- Start the application:
lein run - Go to localhost:8080 to see:
Hello World!
- Start a new REPL:
lein repl - Start your service in dev-mode:
(def dev-serv (run-dev)) - Connect your editor to the running REPL session. Re-evaluated code will be seen immediately in the service.
Since this is a simple example of how Nubank's microservices are structured, many aspects are missing:
- endpoint schemas: our service http and kafka endpoints are always annotated with schemas.
- better adapter examples: since endpoint schemas aren't a part of this example, our adapters from external to internal data representations aren't very interesting or representative.
- kafka component: we make heavy use of kafka and have wrapped producer and consumer logic in components and also developed mocks for them.
- and much more: datomic component, proper config component that does a waterfall of overriding, etc.