COMPLETE GOVERNANCE WITH GREG, APIM & BAM

Introduction

Service oriented architecture (SOA) is best way to develop softwares (services) to achieve any business use case. But SOA requires higher level of coordination and collaboration between lot of teams within the particular enterprise, from business teams to IT (information technology) teams and as well as among other teams and departments.This coordination and collaboration can be achieved by implementing a proper SOA governance model which deals with task, processes and people for defining and managing how services are created, supported and managed.

Governance is somewhat a political issue than a technical issue. While technology focuses on the interfaces, protocols and specifications, the business worry about method of serving customer. But both the technical and business emphasis on requirements to satisfy customers. Governance involves in all those aspects even those are separate efforts and processes. Governance conforms that everyone involved in those aspects are working together which are not contradicting with each other to ensure the financial side of the business is achieved as well as customer is satisfied.

What is governance

Exercising governance in SOA (Service oriented architecture) means implementing a process to ensure that everything is done according to protocols, defined guidelines, best practises, controls, management, rules, responsibilities and other related factors. Effective SOA governance must consider people, processes, technologies, deliverables, QOS (quality of services) in the entire lifecycle from identifying the business use case of a service to implementing, testing, delivering and up to reuse and until service is retired or no longer usable.

SOA governance consists two phases.

1. Design time governance
   This includes the process from identifying a business use case to developing and implementing it as service.
2. Run time governance
   This includes the process from delivering the service to end users to consume and enforcing policies to manage and control who can access the service and what they can do with it.

Reason for having two phases in governance is enforcing policies and SLAs from the service itself is expensive and unmanageable. This will be discussed more in runtime governance section.

Why Governance

There are many reasons why proper governance is needed not only for SOA but also everything we do. If there is not proper planning involved with the project, there is a higher chance that project end up in chaos.

Let’s imagine of a situations where there is no SOA governance involved with service development. Suppose X is working in ABC financial institute. And he is in loan branch and he has implemented a service where customers can view their current loan balance. Someone (Let’s say Y) in the saving branch implements another service which can be used to view customer’s current account balance. Account balance should show the loan balance too. And Y does not know that there is an existing service which he can use to get the loan balance. So Y too implements a new service which shows loan balance without using the existing service that X created.

Someone (Lets say Z) in another department in ABC company start using X loan balance service in his application. That applications start sending heavy traffic to the X’s loan service and crashes the server. Now X starts getting calls from people he does not know of, and complains that loan balance service is not working. Now X finds out that there are a lot of people other than Z in ABC company who use his loan balance service in their applications. Now X is in big trouble because his service is not working. This causes ABC company to lose their revenue and also it’s credibility to sustain their business.

This is a very simple scenario which could happen in any company or enterprise in the world right now. When there is a lack of effective SOA governance, repetition of services, unknown people using services, over usage of resource and services can happen. This could lead to major financial losses to a company.

But when there is proper SOA governance process is involved in the service lifecycle, it increases the possibility of achieving business goals and objectives.

Some reasons for having proper SOA governance in place

1. Shows well structured responsibility management to empower people
2. Can easily measure the effectiveness
3. Well defined rules, protocols and policies to meet the business goals
4. Avoid the repetitions  and reuse of existing service.
5. Mechanism to ensure specifications are followed to the details.

Design Time Governance

What is Design Time Governance

SOA design time governance is the process of enforcing rules and protocols from identifying the business and implementing it as service. This is related the design time service cycle. As you can see in the following diagram illustrate the design time life cycle.

SOA governance starts with identifying the business use case. Most probably this is done by business analyst or somebody who has domain knowledge in business. They analyze the modern business trends and find out business opportunities. They bridge the business problems to technology solutions.

Next stage of SOA governance lifecycle implementing the business requirement as services. This is typically done by the software developers. They should follow SOA architecture and specifications to develop those services. They must ensure that defined protocols, guidelines and rules are followed to the point.

Then implemented service(s) should tested by developers, they should create unit test, functional test and etc… to complete the testing.

Next phase of design time life cycle is QA testing. This is the stage which services are tested for validations (whether developed services fulfill the business requirement), performance test (checks how the load is handled by the service, maximum load before it crashes)

Next stage of the service is sending on production for consume. This is a critical moment where runtime governance starts come in to picture and there some decisions has to be taken such as

• who can invoke the service ?
• what they can do with it ?
• How long the service will be available for consumption ?
• How many request will be served in a given moment ?

Once the service(s) is in production, it belongs to runtime governance which will be discussed it the runtime governance section.

Final state of service lifecycle is retiring service(s) where it has come to a point that current implementation of the service is no longer valid  for current business requirements. Retirement can happen due to new versions of same service is implemented.

Use of tools in Design time Governance

To enforce proper design time governance we must use tools designed for that purpose. WSO2 GREG is a specially designed tool to cater these requirement. We capture the meta data of the service as everything we discussed so far. Such as

Runtime Governance

Why do we have a run time governance phase in the SOA governance. Answer is simple. It is very inefficient and ineffective to build runtime governance capabilities to service implementation itself. Let’s us examine what are the requirement in runtime governance phase. So we may better understand why it requires to separate runtime and design time governance

1. Access control
1. Authentication  - (Who can use the service)
2. Authorization  - (What they can do with it)
2. Logging
3. Enforcing policies
4. Versioning
5. Statistics and Monitoring
1. Response time
2. Success and failure rates
3. Per user usage
4. Per service usage
5. etc..
6. etc… (This list is open ended)

As show in the above image service implementation must be separated from policy enforcement (runtime governance). If not runtime governance requirements have to be implemented with service itself. Then it becomes a nightmare to manage both service and runtime governance requirements. So separation of service and policy enforcement is the most suitable way to achieve runtime governance.

Monitoring and Statistic

This is the one of the most critical requirement in SOA runtime governance. This gives the service provider ability to measure the effectiveness of the services provided.

Designtime and Runtime Governance Together

Please consider the following image.

As you can see I have aggregated both the design time governance and run time governance together in a single image. A strangest thing to notice here is that runtime governance is used in four stages of design time governance.

The reason for this is runtime governance should not be implemented when the service is in production. It should be tested and verified in from the service implementation state to production state. So the policy enforcing should be done as soon as implementation of the service started. If it is not done in those states of the design time life cycle, there will be a lot of complication when applying them on the production state. And ramifications will bring a nightmare to devOps to correct the issues in SLA policies. Even those policies must go through the testing and validating criterias.

Use case

Let’s imagine a use case where a company needs a complete governance process. Company X is financial company, who are mainly focused on giving it’s customers, a full financial solution from saving, loans, managing funds and brokering. This company has hundreds of employees from business analyst to software engineers to devops. They do various tasks in service life cycle. Business analysts are responsible for analysing business and proposing solutions. Software engineers are responsible for implementing the proposed business solutions as services. Devops are responsible for delivering the services in efficient and reliable way.

So let’s imagine a case where there is a need of a service to view the current account balance. Following activities will be done and questions will be answered to provide this view account balance functionality as a service.

1. Identify the use case -
1. Why it is needed ?
2. What is the gap this service will be filling ?
3. How this will benefit the company revenue ?
4. Is this a urgent requirement ?
5. Is there any other services identical to this which fulfils the requirement ?
2. Implement the requirement as a service -
1. What is the language used to implement the service ?
2. Is there a web service documentation ?
3. Who is responsible for developing the service ?
4. What is the service name and the version ?
3. Developer test -
1. Does the service works without breaking anything ?
2. Has checked with a code compatibility tool ?
3. Is error handling properly done ?
4. QA Testing -
1. Does the implemented service fulfil the business requirement ?  (acceptance test)
2. Does the service works smoothly ? (functional testing)
3. Is seamless integration with other services possible ? (integration test)
4. How does the service reacts to a high load ? (Load test)
5. Deploy on public server for public usage
1. Enforcing of rules and policies. This will be discussed later .
6. Retiring the service
1. Is there anyone using this service or has everyone migrated to new version ?

Now we have an idea what are the details we need to capture in every state of the service life cycle. So we need some kind tool to record all of this data. This is the place for WSO2 product stack to play a role in this situation.

GREG is for design-time governance and APIM for run-time governance.

Let’s see how we can use both WSO2 GREG and WSO2 APIM for the above use case.

GREG

WSO2 GREG is product specially designed to provide the right level of governance for SOA. You can find more information about this in following url.

https://docs.wso2.com/display/Governance520/WSO2+Governance+Registry+Documentation

You can download  and try WSO2 GREG from this location

http://wso2.com/products/governance-registry/

APIM

WSO2 APIM is a complete solution for managing apis, routing traffic, specially for managing the runtime governance.

The documentation about the recent release is available here

https://docs.wso2.com/display/AM1100/WSO2+API+Manager+Documentation

Latest WSO2 APIM product can be downloaded from this url

http://wso2.com/products/api-manager/

Integrated Solution

Please look at the following diagram. The deployment of both GREG and APIM will be as follows according to what we discussed in the article.

As shown in left side of the image there is a GREG cluster which is responsible for Design time governance. And there is a clusters of API Managers which are responsible for runtime governance.

Let me explain what is rationale behind the image.

GREG cluster used as the tool for managing meta data related to design time governance. It will store meta data related schemas, wsdls, apis, owners, urls, policies etc… and will be responsible for migrating the metadata between different design time life cycle status.

As seen in the image there are separate clusters of APIM in each environment. Those are acting as the runtime governance enforcers. Those will help the enforce SLAs to service consumers.
BAM (DAS) will be used as the monitoring tool which will keep track of usage related data.

As middleware company, WSO2 has full stack of products to support both design and runtime governance effectively.