#Microservice #Database #Anti-#Pattern

Published on February 24, 2019

By definition, Microservices (MS) are supposed to possess such service-oriented aspects as being

• independent (from what: from each other, or from environment, or both, or this is a signal of different ownership of the MS in the same Application),
•  autonomous (can perform its business tasks on its own),
• specialised.

While Principles of Service Orientation define each of these aspects, it is unclear what has been meant for Microservices. For example, what Microservices are independent from – from each other, or from environment, or both, or this is an indicator of that Microservices in the same Application may have different ownership ?… Also, does autonomous mean Microservice should perform its business tasks on its own only or it may invoke other Microservices as suppliers?

 Anyway, these aspects have led somehow to a believe that Microservices should own the data and related databases. Yes, they could because in theory each of Microservice has the freedom to own data and data store, but this is not a mandatory quality, especially when recall that Microservices are not the centre of universe and there are other corporate interests exist.

 In this post we will review the phenomena of ‘each MS has its own database’ from the corporate view point.

The first is first – what a Microservice is. There are two opinions: 1) it is a finer-grained stateless component with a standardised interface that in composition with other components constitute an Application, which before appeared as a monolith; 2) it is a small stateless SOA Service that provides the simplest and the most fine-grained business functionality with a single RESTfull interface and which can/may support only a subset of Principles of Service Orientation.

Here is a series of questions:

• Why a stateless Microservice should own any data and database?
• May a Microservice own any data and database within enterprise?
• Is Microservice about performing particular functionality or it is in the business of sharing data?
• What another Microservice has to do if it needs only a subset of data owned the Microservice?
• How to manage consistency of data ownership among independent Microservices created and owned by different teams/divisions in the enterprise?

Let’s assume we have accepted the “Microservice owns its data and database” pattern and review the consequences of this.

Scenario 1. A Consumer invokes an Microservice and sends some data to it for processing. The stateless Microservice executes its functionality autonomously and returns the result. All is done in memory -no database is needed.

 Scenario 2. A Consumer invokes an autonomous and sends some data to it for processing. However, the stateless Microservice cannot execute its functionality in full based on the given data and it cannot ask Consumer to give it more data – this will be the unacceptable overhead for the Consumer to obtain this additional data. What Microservice has to do? Yes, it has to invoke another Microservice that promises to deliver this data in return. Hold on, please, is this possible for a Microservice, doesn’t this violate an idea of being autonomous? We relax autonomy rule and allow such invocation. So, our Microservice has stored its data in its database and waits for the response from the invoked Microservice, synchronously or asynchronously (via messages or events). You have to understand that such chain of invocations may be quite long and every Microservice will persist its data.

What each of invoked stateless Microservice will do with its data obtained in the scope of this chain when the final result is passed back to the initiator of the chain? I do not know, probably, these data sets should be eliminated in the databases together with the instances of the Microservices.

So, the Microservice might need databases. However, whether using a distributed cache is not more efficient instead of databases. If the system crashes when the chain of invocations is not complete, would the surviving databases allow to restore this “transaction”? Since database is not shared between Microservice instances, i.e. we work with pairs of Microservice-database instances, probably, in the containers, databases do not add a value to Microservice performance.

Scenario 3. A Microservice is responsible for representing a Consumer Entity, which has multiple Consumer attributes like names, addresses, e-mails, telephone numbers, etc., and stores it in its own database. For this case, it is irrelevant if this is a relational or document database. The important part is that this Microservice owns one or several Consumer Entity sets of data and any other Microservice should invoke our Microservice if the Consumer data is needed.

In a short while, a new task comes up and a new Microservice needs to know whether a person with particular name lives at certain address. This new Microservice is supposed to call our Microservice, but it does not have such API. This new Microservice is owned by another division and may only request a change for our Microservice, but it is a long operational process that totally mismatch the spirit of ‘Microservice Architecture’. Our Microservice can provide the new Microservice with Consumer records including all data to allow the new Microservice to answer its question itself. We suggest that this will be the solution by many. Well, how about security – the new Microservice is entitled to know about the names and addresses only, it may not access any other data about the Consumer.

We have 3 options:

1. design our Microservice as a data servant for all possible future needs for any combinations of Consumer data
2. re-develop our Microservice continuously to satisfy new future needs
3. strip off our Microservice from the ownership of Consumer data and database.

The practicality of the option 1) is doubtful. The option 2) is unacceptable – the Microservice should work rather than being re-developed. We are left with the option 3).

So far, we have established the fact that Microservices might need temporary storages and that databases are not the lightest solution for this.

In the next step, we will analyse if the “Microservice owns its data and database” pattern can sustain enterprise needs at the level of Application or Product (containing one or several Applications).

When we have decomposed a monolith Application into components first time, we end-up with relatively complex components whose functionality, which cannot be treated as Microservices by definition. So, we continue decomposing further until we get to the finest-grained primitive functions/actions. These very functions/actions are represented by Microservices.

Unfortunately, just a collection of Microservices cannot solve the business task that the monolith Application solved – the Application has a value based on inter-operations between primitive functions/actions and their intermediary compositions, which individual Microservices do not have. To restore initial functionality and business outcome, we need to creat less and less grained compositions of Microservices or we have to return to smaller monoliths, which is undesirable.

The easiest way for creating compositions orchestrating the Microservices as is; otherwise, we will need to use choreography where each Microservice should know what events, from whom it has to listen for to act as well as what events it should to fire in response to each incoming event. In essence, this is the inter-Microservices coupling. If events are not-addressed to particular Microservices and just broadcasted to all. Then each Microservice and each Microservice composition has to decide if the it is its event, for it. In any case, both orchestration and choreography compositions are stateful – the state should be maintained until the composition completes its work. It is obvious that nobody would need to temper the state data, nobody cares about it and there is no need to hide it in the individual database. It is enough having private-record-per-service-instance in the shared data store.

With a freedom given to developers for choosing the programming language for Microservices and databases and “with over 300 databases available in the market”, consider the task for the Support of Microservice-based digital ‘applications’ your company would require. You can imagine what company management would say about the pattern in discussion. However, it seems that IT does not share such information with business… until the first dramatic failure.

If the company delegates the ownership of its data to Microservices, who and how at the end of the day would know what data the company has? How a Microservice would know which another Microservice to call to get needed data when the Microservice landscape changes every day/week? There may be much more questions and all of them are cause by just one thing – an idea that Microservice may encapsulate corporate’s data together with a small function. Is this a right model, in general? Isn’t it cutting the branch (compony) it sits on?

An entire idea of coupling functionality and data is pathologic. It anchors function/processing to particular data while processing power is in that it can process any data that adhere to the meta-data model owned by the function. If the concern is only about where to keep temporary session-scoped data, the “Microservice owns its data and database” pattern seems a heavy overhead.

If we deal with cases that are a bit more complex than UI & Microservice, where UI provides data to dedicated Microservice working in isolation as a data set guard, the company should have its data stores independently from Microservices. If the latter cannot work with shared data stores, it is not good and has to be fixed.

SOA Ecosystem has defined so called Data Access Services (DAS) that decouple business services/ Microservices from the data store/databases. The Microservice is no longer a POJO tied with the database (because it is convenient for developers) and it can be used freely in many different service re-compositions for solving emerging business tasks. In each composition, different sets of data and data stores may be used; data is a different concerns than functionality and have to be kept separately according to the Best Practice. DAS shields business Microservice from arbitrary data store and performs mapping/data transformation between the data schema of independent Microservice and independent data store. The latter know well how to deal with concurrency, if this is a concern.

We can conclude, the “Microservice owns its data and database” pattern is suitable for only one type of business operations – exchange of data while who and how processes data and obtains them remains hidden. RESTful API and JSON are very good for data exchange. Are they good for other business tasks for Microservice? We do doubt. Well, we cannot rely on that business applications will be limited by data exchange only, this is simply not enough. Thus, we need Microservice to work effectively with shared data stores.