Jamming #ISO #20022 into #Open #Banking #API makes them #REST-ish, not #RESTful

This case is a perfect example where a business value requires an adequate technical instrument, but when the instrument is driven by technology convenience, nobody cares about the business values. We are talking about a well-known problem that UK Open Banking (OB) and EU PSD2 face with financial transactions.

It seems that the technology considerations for PSD2 and, especially for Open Banking, were set around RESTful JSON-based API with minimal or none business and security concerns.  Particularly, they have forgotten about the business-centric financial standard – ISO 20022 – that was and currently is the universal vendor-agnostic definition of financial information that financial institutions exchange. They also have omitted and now try to ignore the EU GDPR – one of the fundamental OB “Read/Write” API specifications 3.1 mentions GDPR only 1 time and does not talk about personal data protection in the structure of API’s URI.

In short, here is what the problem is about. The ISO 20022 has been accepted as mandatory for OB and PSD2 already, but it comprises 15+ categories with 1 to 50 subcategories of XML Schemas. The OB and PSD2, as the technology “babies” triggered by FinTech organisations, define only REST and JSON for the API implementation, no XML.

It is also well-known fact that XML Schema is richer (and more verbose) than the JSON Schema. The fundamental concept of XML is that the data semantics/ inter-relationship, so important for people reading and comprehension, is embedded in the data itself – this is why XML is human and machine readable vendor-agnostic format of exchanging financial information. JSON, in contrast, moves the data semantics into the structure of REST URL exposing the data organisational structure to the entire world and forgetting that this structure is one of the most vulnerable and volatile in business (it depends on the context that always changes).

For modern developers, simplified and object-friendly JSON is much easier to use than XML though the previous generation of developers was fine with XML. (Not all areas of software should be optimised for scalability to the level of Social Media; there are and will be areas where the content and consistency are equally important). So, one of the top priority task for OB and PSD2 is how to force ISO 20022 XML documents into JSON.

We can find a few attempts to solve this problem. There was an attempt to develop bi-directional transformers from XML to JSON and from JSON to XML like Trace Financial’s Transformer. Apparently, we can transform XML to JSON with no information lost, but the opposite is incorrect – JSON Schema does not support all constructs available in the XML Schema, i.e. such information will be inevitably lost.

The authors of OB “Read/Write” API specifications 3.1 admit that they cannot make the exact map from JSON to XML and have to “modify ISO 20022 elements where the existing standard does not cater for an API context (such as filtering, pagination etc.). An example is having latitude and longitude in decimal format, as this is how developers will work with latitude and longitude; or using simple types (e.g., a single date-time field) instead of a complex type (e.g., a choice field with a nesting of date and time)”. We can agree with latitude and longitude, but we believe that the complex type for date/time has it value proved for years and truncating date-time structure is not acceptable.

In the special section, the OB “Read/Write” API specifications 3.1 states:

“A previous draft of the EBA RTS required that the interface “shall use ISO 20022 elements, components or approved message definitions”. In keeping with that requirement, the API payloads are designed using the ISO 20022 message elements and components where available. [M.P.: this means that there may be some APIs what violate the requirement of EBA RTS. Does it make sense to create a national standard incompatible with related standards in other countries?]

The principles we have applied to re-use of ISO message elements and components are:

• Where relevant, the API payloads have been flattened so that they are more developer friendly. [This is ridiculous – the ISO 20022 messages  are not for developers but for business people, first of all, and common business values are sacrificed for the sake of developer’s convenience. If it goes further this way, developers will dictate what customers may want and then will require payments just for appearance at work, as it was in socialism in USSR].This has been a request from the developer community, and the stakeholders involved in the design workshop.
• Only elements that are required for the functioning of the API endpoint will be included in the API payload. [M.P.: the elements of the message aim particular business value and goal; they are irrelevant to any technical mechanisms for their transitions. This means that the endpoints should be built for particular messages with all their elements, not the other way around. ] API endpoints are defined for specific use-cases (not to be generically extensible for all use-cases).”

We have found quoted text creepy but it constitute a cornerstone shift in the OB Specification – the stubborn RESTful technocratic approach has been moved toward much more reasonable REST-ish approach where JSON may be replaced by XML (this continues the REST-ish modernisation set around Command Pattern [https://howto-architect.com/online-presentations/)  .

An entire approach of substituting ISO 20022 XML Schemas with JSON Schemas is erroneous. The fundamental value of ISO 20022 is not in a convenience for developers but in the business values inserted into the XML Schemas. In other words, financial institutions or FinTech dealing with financial messages (defined in the OB and PSD2) should be able to send the XML and receive the XML for reading and processing while the transition of the information may be in aby format suitable for supporting technology, plus, no data or business value may be lost in the transition.

We need to outline that the popularity of JSON in significant portion relates to that JSON in the API allowed simple direct transition into the JavaScript objects used intensively in the rich user interface. In the case of ISO 20022, user interface is not at the high priority (if at all)..

Let’s look at a few XML constructs that require not that straightforward transformation to JSON. Here are a few of them:

XML	JSON
xs: float	{
xs: double	“type” : “number”
xs: decimal	}
	{
	type : “integer”,
xs:positiveInteger	“minimum”: 0,
	“exclusiveMinimum”: true
	}
	{
	type : “integer”,
xs:negativeInteger	“maximum”: 0,
	“exclusiveMaximum”: true
	}

This means that if a financial message would be originally written in JSON, we cannot be sure that its text described “xs:positiveInteger” or “xs:negativeInteger” as specified above and its conversion into XML can dramatically change the sense of the message. Obviously, it is inacceptable regardless potential FinTech’s benefits.

There are two more syntax topics that have to be specially discussed in the case of XML to JSON transformation:

1. XML Schema defines “minOccurs” and “maxOccur” notions that indicate that particular XML elements can be repeated and how many times in the XML document. JSON Schema has attributes such as “minIten” and “maxItem” that can be used in the similar sense, but XML has maxOccur=” unbounded”. This does not have an analogue in JSON, but is widely used in the SO 20022’s XML Schemas.
2. The SO 20022’s XML Schemas intensively use a construct of “complexType”, which leads us to the JSON’s “array” (a pure technical concept). For example [ISO 20022, message “CreditorPaymentActivationRequestV02”, pain.013.001.02 ]:

<xs:complexType name=”CorporateActionAdditionalInformation1“>

    <xs:sequence>

      <xs:element type=”BeneficialOwner1” name=”BnfclOwnrDtls” minOccurs=”0” maxOccurs=”unbounded“/>

     <xs:element type=”Max350Text” name=”RegnDtls” minOccurs=”0” maxOccurs=”1“/>

     <xs:element type=”PartyIdentification2Choice” name=”RcvrId” minOccurs=”0” maxOccurs=”1“/>

     <xs:element type=”YesNoIndicator” name=”CertfctnInd” minOccurs=”0” maxOccurs=”1“/>

     <xs:element type=”BeneficiaryCertificationType1FormatChoice” name=”CertfctnTp” minOccurs=”0” maxOccurs=”1“/>

     <xs:element type=”ProceedsDelivery1” name=”DlvryDtls” minOccurs=”0” maxOccurs=”unbounded“/>

     <xs:element type=”Max350Text” name=”AddtlInstr” minOccurs=”0” maxOccurs=”1“/>

</xs:sequence>

             </xs:complexType

This complexType of “CorporateActionAdditionalInformation1″can be presented as a JSON “array” of elements such as “ BnfclOwnrDtls “, =” RegnDtls “,  “ RcvrId “, etc.  However, since “ BnfclOwnrDtls ” and “ DlvryDtls ” are defined as maxOccurs=”unbounded”, these elements represent JSON “arrays” themselves. That is, we have an array of arrays. Also, if the schema would included  <xs:choice> instead of <xs:sequence>, we were needed to use JSON’s “oneOf” where each item in the list had to be an array of arrays. These comments show that XML transformation into JSON is not that simple and we do not know what would be more inconvenient for developers to write in XML or to debug cumbersome JSON expressions.

As mentioned before, the OB “Read/Write” API specifications 3.1 accepts that “the API payloads are designed using the ISO 20022 message elements and components where available”. And where is this available?

The HTML methods such as PUT, POST, PATCH and DELETE (HTTP 1.1 specification – RFC 7231 – explicitly permits the payload for DELETE) can carry the data payload in the XML format though REST does not consider this. The GET method does not support any payload in the request and the return has to be in JSON according REST. Thus, there is a hope for using XML for all REST methods instead of JSON but for GET requests. The examples of using an XML document in the request messages for Service are well known from the WSDL and WS-* interfaces.

Now, we review the challenge that RESTful GET faces in front of ISO20022 vocabulary. The syntax of GET requires:

1. the resource representation/information to be uniquely specified in the GET’s URI construct
2. the attributes of the requested entity of the specified have to uniquely identified; otherwise, the request asks for a collection of all entities of the type specified in the URI construct.

When RESTful GET is used within the same business organisations, there is a single ownership on information exists in the company. This might help a requestor to have a default access to the list of entities though it is not guaranteed. When the GET is issued between business organisations, as it is assumed in ISO standard, the security principles of “minimal access privilege” and “no trust” are preserved by financial organisations. That is, returning a list of all entities of certain type in response to GET is a quire exceptional, abnormal situation.

For example, if the requester needs a contact phone number of an entity located at an incomplete address, nobody will respond to the GET with all “possible” entities at the incomplete address. That is, to avoid a denial, the requester has to obtain either an exact address of particular unique identifier of the entity (if this is applicable). To illustrate this case, we have a typical description of an entity named “PartyIdentification43” [ISO 20022, message “CreditorPaymentActivationRequestV02”, pain.013.001.02 ]:

<xs:complexType name=”PartyIdentification43“>

   <xs:sequence>

<xs:element type=”Max140Text” name=”Nm” minOccurs=”0” maxOccurs=”1“/>

<xs:element type=”PostalAddress6” name=”PstlAdr” minOccurs=”0” maxOccurs=”1“/>

<xs:element type=”Party11Choice” name=”Id” minOccurs=”0” maxOccurs=”1“/>

<xs:element type=”CountryCode” name=”CtryOfRes” minOccurs=”0” maxOccurs=”1“/>

<xs:element type=”ContactDetails2” name=”CtctDtls” minOccurs=”0” maxOccurs=”1“/>

   </xs:sequence>

</xs:complexType>

and the requester needs just a telephone contact for this party. Actually, the telephone contact is available in only in the data-structure “CtctDtls”[ISO 20022, message “CreditorPaymentActivationRequestV02”, pain.013.001.02]:

<xs:complexType name=”ContactDetails2“>

   <xs:sequence>

<xs:element type=”NamePrefix1Code” name=”NmPrfx” minOccurs=”0” maxOccurs=”1“/>

 <xs:element type=”Max140Text” name=”Nm” minOccurs=”0” maxOccurs=”1“/>

<xs:element type=”PhoneNumber” name=”PhneNb” minOccurs=”0” maxOccurs=”1“/>

<xs:element type=”PhoneNumber” name=”MobNb” minOccurs=”0” maxOccurs=”1“/>

<xs:element type=”PhoneNumber” name=”FaxNb” minOccurs=”0” maxOccurs=”1“/>

 <xs:element type=”Max2048Text” name=”EmailAdr” minOccurs=”0” maxOccurs=”1“/>

                 <xs:element type=”Max35Text” name=”Othr” minOccurs=”0” maxOccurs=”1“/>

    </xs:sequence>

</xs:complexType>

Thus, to get access to the particular “CtctDtls”, the requester should request concrete “PartyIdentification43”, which is possible only if its “PostalAddress6”, “CountryCode” and “Party11Choice/Id” have concrete specifications also.  However, the “PstlAdr” includes many optional elements as well:

<xs:complexType name=”PostalAddress6“>

    <xs:sequence>

<xs:element type=”AddressType2Code” name=”AdrTp” minOccurs=”0” maxOccurs=”1“/>

<xs:element type=”Max70Text” name=”Dept” minOccurs=”0” maxOccurs=”1“/>

<xs:element type=”Max70Text” name=”SubDept” minOccurs=”0” maxOccurs=”1“/>

<xs:element type=”Max70Text” name=”StrtNm” minOccurs=”0” maxOccurs=”1“/>

<xs:element type=”Max16Text” name=”BldgNb” minOccurs=”0” maxOccurs=”1“/>

<xs:element type=”Max16Text” name=”PstCd” minOccurs=”0” maxOccurs=”1“/

><xs:element type=”Max35Text” name=”TwnNm” minOccurs=”0” maxOccurs=”1“/>

<xs:element type=”Max35Text” name=”CtrySubDvsn” minOccurs=”0” maxOccurs=”1“/>

<xs:element type=”CountryCode” name=”Ctry” minOccurs=”0” maxOccurs=”1“/>

<xs:element type=”Max70Text” name=”AdrLine” minOccurs=”0” maxOccurs=”7“/>

    </xs:sequence>

</xs:complexType>

Since here are so many optional elements that may be simply not specified in the provider’s records, there may be several “PartyIdentification43”, but the requester is granted access to only one. The “ISO 20022 and JSON: An Implementation Best Practices Whitepaper” from Jan,2018, tries to promote a use of GET in its technology pattern, e.g. first execute “GET /orders” that is supposed to return a list of all orders and then run “GET /orders/000235698741” where 000235698741 is the order’s ID. Since we deal with business messages outside of the technology realm, we very much doubt that anyone but an Authoritative Audit organisation would be given permission use GET in the described manner. That is, mentioned Best Practices for GET are simply not practical, not realistic.

We can conclude that using GET method in its traditional programmatic manner is impossible with ISO 20022 data model. Nonetheless, the solution may be found in the operational sphere:

1. the institution-requester registers with the institution-provider up-front and via different means than ISO 20022 messaging
2. during the registration, the institution-requester is granted a “read” permission against particular data/entity
3. all unique identifiers of permitted data/entity are disclosed/discovered and made available to the institution-requester during registration
4. the institution-requester uses provided identifiers in the standard GET syntax
5. if the institution-provider considers the disclosed identifiers as personal data according to GDPR, e.g. a passport number or a personal postal address provided together with the person’s name, this data should be encrypted based on the preliminary agreed encryptions mechanism.

Returning to our example, in order to obtain a telephone number of the contact of the PartyIdentification43, the registration has to include:

• at least, the PartyIdentification43’s “Id”
• if the counties and/or regions in the country may have more than one PartyIdentification43, the “CtryOfRes” and the full postal address “PstlAdr” have to be identified as well.

Only presence of mentioned detailed information allows to preserve strong security and related permission to “read”. In this case, the GET method may look like:

GET …/PartyIdentification43/{Id}/country/{CtryOfRes}/CtrySubDvsn/{[name]}/StrtNm  /{[name]}/BldgNb/{[number]}/PstCd/{}/TwnNm /{[name]}/phone? PhoneNumber=”MobNb”, “PhneNb”

where all data in {…} is obtained from the registration.

Even this detailed specification may return wrong result because all parts of the URI structure are optional in the ISO 20022 and needed telephone numbers may be associated on the institution-provider side with the entry that does not specify, e.g.,  StrtNm , BldgNb, PstCd or TwnNm. In any case, working with ISO 20022 in PSD2 and Open Banking, breaks rules for the REST APIs and appears more as an “XML over HTTP/HTPM” than a REST.