question: why move data access away from business/data layer?
mrkagelui opened this issue · comments
I joined your class quite some time ago when the data access still resided at business/data layer, and business/core layer depended upon that to e.g., retrieve all users. the same functionality now is in business/core/user/store, i.e., instead of a layer, it's a "sub package" of the user package. if I'm not wrong, the intention is that no core package other than user should depend on usercache or userdb.
what's the motivation behind this? what if different core packages need the same data? one hypothetical use case is some "configuration".
for the sake of discussion, imagine a "multiplier" service with two main use cases: 1. CRUD an arbitrary number of numbers that serve as multipliers; 2. receive a number and respond with the result of multiplying the number with all the multipliers.
in this service, the use cases of listing all multipliers and the actual multiplication would both need to fetch all multipliers from persistence, and I don't think they belong in the same package. how would this architecture fit?
Answers below
I joined your class quite some time ago when the data access still resided at business/data layer, and business/core layer depended upon that to e.g., retrieve all users. the same functionality now is in business/core/user/store, i.e., instead of a layer, it's a "sub package" of the user package. if I'm not wrong, the intention is that no package other than user should depend on usercache or userdb.
Correct, and you have can different storage layers for the core business package.
what's the motivation behind this? what if different core packages need the same data? one hypothetical use case is some "configuration".
The core package need to maintain a domain specific purpose. So no two packages should be accessing the same data. The App layer allows you to merge data from different domains together.
for the sake of discussion, imagine a "multiplier" service with two main use cases: 1. CRUD an arbitrary number of numbers that serve as multipliers; 2. receive a number and respond with the result of multiplying the number with all the multipliers.
in this service, the use cases of listing all multipliers and the actual multiplication would both need to fetch all multipliers from persistence, and I don't think they belong in the same package. how would this architecture fit?
With the little information you have provided.
I would have a core package for the multiply which provides a single API for this feature functionality.
I would have a storer implementation for the CRUD related needs
I would have an app layer multiplegrp to provide the endpoints
If there are potential N number of multipler implementations, then I would add a layer under the core package called multipliers/ and implement the different multiplier packages. Declare an interface in the core package for this. The same pattern you see with the storers.
— Bill
so one endpoint directly depends on the "storer" to retrieve all multipliers, and then another actual "multiply" endpoint depends on the core "multiply" package which in turn depends on the "storer".
yup, that's what I thought as well. and naturally these "data access" packages are organized based on their use cases, not "tables" or data sources right? e.g., using a relational DB, it's possible for two core packages to need to query the same table, and even so, they should reside in different "sub packages" of the two core packages.
If you don't understand the data, you don't understand the problem.
The endpoints rely on the core business packages. Ignore any idea of a storer when you are in the app layer.
Actually, we organize the business packages on domains which are directly tied to the main data we need to store and work with. Many times that is a 1 to 1 with the main data tables we define. It's nice to have that relationship. This relationship doesn't need to carry into the App layer however, though most of the time it does.
The data models drive the domains at the business core layer. This works really well to keep code within the scope of what it should be providing. Start at the business core layer models and then work down and up from there.
In service we have user data and product data. We defined models in business/core for that data and eventually defined a storage implementation. If the app layer needs user data, it must always go through the user core business package. Same with product. If I saw either package trying to access data from storage that isn't part of it's domain, that would be a problem.
If there is a relationship between these two models. A product references a user, which we see because the Product model has a UserID field. Then we will allow the Product core business package to use the User Core business package internally. But you can't have the opposite. Following the data model prevents cross import problems. If you end up with a cross import problem, you pretty much have a problem with your data model.
I see. yes the dependency between core business packages could be hard to manage. but i remember you said that packages in the same layer shouldn't import one another?
We want to avoid that 100% in the foundation layer.
You can't avoid that in the business layer, so you need guidelines for when it is safe. I've given you a guideline here.
I see. I was thinking to make the "storer" dedicated to each domain like you suggested. using this service as an example, if there's a need to check if the user ID is valid when creating a product, my productdb package would make that query as well. this way product core business package doesn't need to depend on user core business package. what do you think of that?
if there's a need to check if the user ID is valid when creating a product, my productdb package would make that query as well. this way product core business package doesn't need to depend on user core business package. what do you think of that?
NO!
If there is a need to check the userID when creating a product, that can be done inside of the core.Product.Create function using the user core package API. This is allowed because the relationship exists.
You could also do that check in the handler and return a 400 error for a bad data model. I like moving as much of the data validation to the App layer so when there are business to business calls, we are not rechecking things like if the userID is valid.
That being said, since this is a business rule when creating a Product, the UserID check should be done inside the call to Create at the core layer.
I see, thank you for sharing, I understand the idea now. one probably last question: what do you think is wrong with that approach?
You don't want to have any cross domain code happening at the storage level. That breaks the "firewalls" completely. Then you don't know what package is doing what. It's really important that the Core packages manage a domain and no other package touches anything related to that domain. The storage packages are isolated to the core they belong to. It's why we put them under.
Understand your data model, understand the relationships in the model, and keep those boundaries in the code.
I see. I think I get the whole idea now, thank you Bill!