Published on April 5, 2018
A Problem
An emerging EU regulation – GDPR – defines a requirement that asks developers to create business and technology solutions with default data privacy protection by design. This requirement conquers the worlds of DevOps and API/Microservice Economy first of all. The result of this is that traditional perimeter protection inherited from the old time of monolith application and provided by API Gateways, as shown in Figure 1, is not enough anymore for the regulation compliance.
Why is this so? Because regardless macro or micro applications we develop, the major threat to the data privacy remains the same – it is an internal intruder in about 70% of cases (more or less depending on the particular research). Believe it or not, the DevOps operational model and Microservices only increase this risk. The technology level allows now just one person to execute the full development cycle and deploy a Microservice into production as a part of an application. The same person supports and maintains the Microservice/API. In other words, particular Microservice can be viewed as a representation of the person in the production area of the company.
If a company’s governance control is a bit weaker, the Microservice can camouflage unauthorised access to other Microservices (see Figure 1) in the same application or even in another application in the production environment, i.e. can get access to real personal or sensitive data. Who controls this? – Nobody, because security/data privacy is seen in development at the level of application as a whole, not at the level of the individual Microservice / API. We can have hundreds Microservices operating in an application with no operational data protection whatsoever.
Let us see what we can do to really provide data privacy protection by design when working with Microservices.
Available Solutions
At the simplest level, we have Microservice-provider and Microservice-consumer (MS-provider and MS-consumer respectively). When a MS-provider receives an API request, it has:
1) to find out if comes from a legitimate requestor, i.e. authenticate the requestor, and
2) to find whether this requestor has rights to access the resource provided by the MS-provider, i.e. validate the authorisation/permission of the requestor.
We make a few assumptions here: we follow the OASIS RA and RAF for SOA standards and consider that a MS-provider represents the execution results of a particular capability to those who are interested in these results. In the RESTful terminology, these results are a resource. The provider offers the service, but it is not obliged to serve everyone who requests the service. The service-consumer relationships are not the same object-object relationships.
That is, a MS-provider can accept the request and execute certain activity on the resource or can deny the request. In other words, the MS-provider may know or may don’t know the requestor, but should be sure that this one has authorised right to request certain action over the resource.
In this case we apply a simplification and talk about a single resource. It is not a fundamental limitation; the same can be done for several resources, but Microservice theory requires a ratio of 1:1 between a Microservice and the resource. Actually, we can have several different Microservices that work with the same resource, but this is a quite complex case and it is outside of the scope of this article.
The fist and, possibly, one of a few most popular solutions used by people for authentication and authorisation at the level of Microservices is a JSON Web Token (JWT). Many claim that JWT is quite efficient and scalable solution. The authentication with JWT is provided by the digital signature of the token based on PKI, i.e. the receiver can validate the signature of the token issuer. However, the signer is an Authority, not a sender of the token. How the sender is authenticated? An identification of the assumed sender may be enclosed in the token’s payload together with sender’s rights to require certain action on the resource, but there are no ways to verify this while the sender and “authorised” actor can be different entities.
A JWT is generated in response to the request and returned to the requestor, the MS-consumer in our case. The token generator follows the information provided by the token requester with no validation of this information – is it really about the one who requested the token or not. This is the first security gap in JWT. The second gap is in that the token, when returned to the requestor, can be handed over to anybody within the specified validity time, as shown in Figure 2. This can take place even after the access rights specified in the token changed or eliminated. Thus, the MS-provider does not have real abilities to control who requests access to the resources and on which ground. In our opinion, this is the negative inertia of thinking inherited from the OO development and full control over all objects in the monolith application.
So, the MS-consumer sends the request to the MS-provider together with the received JWT. The MS-provider decides to either execute the requested action (if it suggests that the requestor is legal and has the required action right) or not. Based on the implementation, the MS-provider can make this decision by itself or forward the request to one of the Authorisation Servers within the environment to try to determine the permissions. However, there is still no way to uncover a matching or difference between the one who obtained the token in the first place and the one who used it in the request.
People say that a JWT-based solution is “much more scalable than the traditional centralized session management: it allows every individual Microservice to handle the security of its own resources”. Well, I do not see any differences in the JWT model and “traditional centralized session management” except that an amateur in security developer now can make crucial security decision. If a MS-consumer caches the JWT and reuse it the next time instead of acquiring the new (current) one and the MS-provider does not validate the JWT, it is the third security gap because the access right can change for this time. Noting in JWT enforces consumers and providers to act in the secure manner, especially when all these validations are a hit for performance.
As a result, a high scalability of the JWT brings serious security risks that should be mitigated. When the risks mitigated the scalability and performance might appear not that high.
The next top-positioned solution is OAuth/OAuth 2.0[1], which uses a Delegated Authorisation protocol. Many people have started using the OAuth 2.0 for an authentication as well, but this is a misconception. In reality, the authentication is out of the scope of the OAuth 2.0[2]. A totally different process should run for the authentication via a 3rd party (its Authentication server) and its results, then have to be used together with the OAuth 2.0 token. A use of OAuth on its own is referred to as pseudo-authentication.
Yes, the OAuth 2.0 does not require a consumer (the MS-consumer) to provide its identity credentials, but we also cannot assume that each Microservices in the application will re-direct the login to a commonly trusted 3rd party like Facebook or Google for authentication. Following the basic OAuth flow and believing that the OAuth 2.0 is good for authentication leads people to do what turns out to be very bad security decisions. Instead of OAuth, an OpenID Connect (OIDC) – an authentication layer built on top of OAuth 2.0 – should be used for authentication. We have counted up to 7 back-and-forth handshake interactions for OpenID involving the owner of the resources, the Authentication server/service and the requestor for the resource. It does not seem to be good for performance as well.
We expect that many would ask – why we need to authenticate and authorise each Microservice in an application at all? There may be a few answers and one of them, the trivial one, is that the composition/collection of Microservices in the application changes all the time, possibly several times per hour. Some Microservices are deployed in new versions, an implementation of other Microservices changes, new Microservices appear and want to access existing ones, etc. Unless each Microservice is meticulously reviewed against the security regulations/policies before deployment, we cannot expect that each Microservices is secured by default and by design. A Microservice is not a simple piece of code – it is an implementation of business functionality and it has to run on the basis of business rules, which demand a business trust between a consumer and a provider (as specific requirement of service-oriented ecosystem) where an authentication and authorisation are the minimal means for establishing this trust.
Propagation of the End-User ID – Is It Really Needed?
A notion of propagating an end-user identity through the chain of application invocations was very popular in IT. This was based on the security model where 1) each end-use was granted particular access to certain system or application (to a resource), and 2) all systems and applications were known at each moment of time. When we get into the dynamic world of services when dozens or even hundreds new Microservices come up, we simply do not have time to grant the access rights to the end-user, including each MS-consumer. So, the traditional IT security model gets broken.
Nonetheless, there are frequent attempts to repeat the user ID propagation through the service chains, including Microservices. The reasons for this, as people say, is to know who used the services. All right, assume you know; so, what then? The maximum information we can get from this knowledge is about an initial access of the end-user to the first Microservice. All other Microservices in the invocation chain will be invoked not because the end-user has an access right to them, but because the initial Microservice needs those Microservice-supplements regardless that end-user.
This situation was recognised in SOA several years ago. Its resolution was formulated in the so-called Knight Rules of Service Ownership . For example, from the MS-provider’s stand-point, one rule states, “A consumer of my consumer is not my consumer”, as shown in Figure 3. That is, if a MS-consumer A invokes (somehow) a MS-provider EU, the latter does not need to know about the identities of MS-consumers B, C and D because it deals with and trusts to the MS-consumer A only. The MS-consumer A, in the role of “MS-provider”, can have trustful relationships with the MS-consumers B, C and D and know their identities.
How the trust is established is a different story – in the simplest case, the Microservice EU, the provider, sets a contract with the Microservice A, the consumer, based on the authentication and authorisation. Now the Microservice EU is interested in the identity of the Microservice A. However, the Microservice EU does not have contracts with MS-consumers B, C and D and do not need their identities because it isn’t obliged for providing services to them.
The conclusion we make from this observation is simple: if a MS-provider validates authorization of its direct MS-consumers, it needs to know only their identities. If the MS-provider does not have explicit contracts the direct MS-consumers, it may rely on implicit contracts agreed on its behalf by a trusted entity, e.g. AAS. So, we return to the issue of cross-Microservice authorisation, that has to be well secured, scalable and well performing.
A Few Ideas about Delegated Authorisation
We assume that each application built of Microservices has access to an Authorisation Authority service (AAS). Each Microservice during the deployment procedure registers with the AAS (or a federation of AAS). The AAS takes care of authenticating each registering Microservice.
A number of Microservices in the company (as well as in the application) is finite. For each existing Microservice, we can know all Microservices that communicate with it directly (“my consumers”). Usually, there are a few of them. We can create a shared Dependency Catalogue service that would be aware of every Microservice and all other Microservices that can call it. The list of the letter forms an Access List for the former. All Microservices with their needs in interaction with other Microservices are carefully reviewed at the code level and validated.
Every Microservice (its version and instance), upon getting online, has to obtain its individual Access List from the Dependency Catalogue service (as a part of initialisation or bootstrap). The Microservice can cache the Access List and/or may persist it in the managed data store, as shown in Figure 4. The integrity of the list should be protected (an encryption, or a digest, or a signature, or whatever solution can be used here). This is an additional security precaution in case of an intruder would try to alter this list.
If a Microservice provides for more than one action on the resource, the action enumeration (entitlement) has to be included in the Access List as well. If a new version of the MS-provider comes up, it has to register itself, its resource and its possible actions with the Dependency Catalogue service. If a new MS-consumer comes up, it has to register itself and its interests (data, meta-data and actions) to other Microservices.
At the run-time, when a request from a MS-consumer comes to the MS-provider (within or from outside of the application), the MS-provider validates the requestor and required actions against the preliminary obtained Access List (a “Should I know you?” pattern). If the match is found, the request is executed and responded. Otherwise, the MS-provider pauses the response, accesses the Dependency Catalogue service again, obtains a new fresh Access List and validates the request the second time. If the match is still not found, the request is denied; otherwise it is executed.
The MS-provider delegates the authentication and authorisation verification control to the AAS and saves a lot of time on this. Since for the time the MS-provider keeps the initial Access List, a new MS-consumer can appear in the Dependency Catalogue. The MS-provider executes a “lazy-refresh” of the Access List, i.e. downloads it only when the new MS-consumer invokes the MS-provider the first time, and gains additional performance of this. In the well organised MS development, a refresh can happen about each 10 minutes; in the majority of cases, such refreshment is rarer.
The overall solution can scale vertically and horizontally on the assumption that the company controls its Microservice development. The performance has to be better than in the cases with JWT and OAuth if they are used securely. However, if a Microservice’s API is exposed outside of the corporate boundaries, i.e. we do not have any chances to know who contacts us and on what authorisation grounds, the mechanism of Service Contract in sense of OASIS SOA RAF should be applied because it is the single current solution realising the concept of business trust. Otherwise, if a Microservice just responses to an external call without a business awareness of the requestor, it can result in the business risks for its own company. If a Microservice uses/engages an external API without a business awareness of the provider, it can result in the security and business risks for the company, but this is another story.
Conclusion
The article argues that the GDPR requires the code, which realises business functionality and operates with personal and private data, to be fully secured by design. This demands Microservices within the application to conduct full authentication and authorisation control over their inter-communications. We have demonstrated that popular JWT- and OAuth-based solutions, as they are used today, are not secured enough and cannot be considered compliant with GDPR.
The article offers an alternative model of delegated authentication and authorisation control for Microservice interactions within a protected perimeter, e.g., by API Gateways. The model is secured, open, scales horizontally and vertically, and performs better than if a full authentication and authorization had to be done in a traditional way for each Microservice invocation.
______________________________________________
[1] Access Tokens are issued via Auth0’s OAuth 2.0 endpoints: /authorize and /oauth/token. You can use any OAuth 2.0-compatible library to obtain Access Tokens.
[2] RFC 6749, 3.1. Authorization Endpoint explicitly says as follows: “The authorization endpoint is used to interact with the resource owner and obtain an authorization grant. The authorization server MUST first verify the identity of the resource owner. The way in which the authorization server authenticates the resource owner (e.g., username and password login, session cookies) is beyond the scope of this specification”.
HOW to ARCHITECT for DEVELOPERS 