Posts Tagged 'SOA'

Why not Petri Nets?

I have read the excellent article by Johan den Haan “The Process Centric vs. Information Centric approach to SOA”. It strengthen my idea that BPM must go beyond its current boundaries and explore new languages that better fit business process modeling.

I am not sure that an event-driven model can fit every requirement for modeling, in some case (see my previous post) when the process is driven by a complex data (i.e. product structure) some hierarchical model could be more effective and complementary to an event driven model.

Turning to event-driven I have tried to model a process using Petri Nets. I started from a process that I found hard to model with BPMN. When you have different asynchronous user data entry BPMN does not allow to have a simple vision of the end-to-end process.

Petri Nets allow to design a more flexible model taking into account exceptions in a seamless way. Moreover CEP configuration languages are very close to Petri Nets concepts and allow to quite straightforwardly translates Petri Nets into execution models.

Example of Petri Net (order to production process)

The flavours of SOA

SOA has two “flavours”. Both are valid and fit specific business needs.

1. Vertical (multichannel):

  • several channels share the same data and business logic (services) on a common layer
  • communication is usually synchronous – request/reply
  • the business analysis that brings to services starts from use cases (UML style: use case -> sequence diagram -> class diagram -> services)

2. Horizontal (event-driven / long-running transactions):

  • An applications triggers a business event (i.e. purchase order) and calls an asynchronous service to sink the event. The service is actually a long running process (i.e. order to delivery) implemented on a BPM engine. The BPM engine calls several services in order to execute the process
  • communication is both asynchronous/reliable/one way and synchronous request/reply
  • the business analysis that brings to services starts from a process model (BPMN -> BPEL -> services).

      The choice of which flavour to use depends on the business model of the organization.

      Often Financial services need to implement short running transactions with high interactions with customers (i.e. Internet Banking) and prefer the multichannel flavour.

      Manufacturing and logistics are more concerned in long running transactions and prefer the second flavour.

      Often Telco companies stay in the middle and need both.

      I think that every organization should explore the use of both styles, putting in discussion the internal culture (transaction-oriented or process-oriented) and evaluating the benefits coming from a different approach.

      BPM is a too tight dress on SOA

      In my previous posts I stressed the fact that BPM and SOA together brings many benefits in terms of business-IT alignment, agility, time-to-market and reuse of business capabilities.

      BPM and SOA cooperate in a symbiotic way:

      • BPM design-time tools model the business process to implement
      • BPM run-time tools (engines) execute the process instances calling SOA services
      • SOA offers the “execution environment” for BPM engines

      SOA services (together with user interaction forms in case of human workflow) are the building blocks for modeling and executing Business Processes.

      With this approach you can change your processes just changing the graphical model.

      Of course sometime you will need to develop extra SOA services. In other words you will need to expand your process execution environment. If no extra service is needed the time required for your change is minimal.

      This is a perfect scenario. If every Business Process could be implemented this way the promised benefits would be easily achieved.

      Unfortunately the world is more complex.

      Let’s take an example from a Telco company. One of the typical processes for a Telco is the “Order-to-Delivery” one. When a new purchase order has been issued the company has to provision all the components related to that order.

      You can model the O2D process using a BPMN or BPEL model. However the O2D process is strictly related to the purchased product and its components (ADSL line, cellular phone, modem, IP TV). You have two choices:

      1. Model a generic O2D process that fits every product.
      2. Model a very detailed process in which to specify every dependence among the components to provision (“B must be provisioned after A”) and the exceptions to handle. A complex model, “flow-chart like” is obtained.

      In the former case the model is quite useless. Clearly the logic about the provisioning of a specific product is outside the BPM model.

      In the latter case you get a complex and algorithmic model just less complex to maintain than a Java program. BPMN and BPEL reminds too much the procedural languages, it’s very easy to trespass the limit and use them as old fashioned flow-charts diagrams.

      If you modeled other aspects, like billing or pricing you could have the confirmation that BPMN and BPEL are not able to represent every aspect of a business process. They are languages specific for high level business processes and for simple orchestrations.

      Let’s imagine to have different formalisms that help to model different aspects of a business domain.

      For example the provisioning dependencies could be easily described in a non-procedural way using a Pert model.  Billing rules could be described with some hierarchical model derived from the product’s bill of material. Specific engines, similar to BPM engines shall be developed in order to interpret the run time instances derived from these non-BPM models. Of course the SOA environment is still valid also for these specialized engines.

      I do not propose a classic Model Driven Architecture (MDA) approach since the MDA brings to the automatic generation of programming code while I think to a non-procedural graphic model directly interpreted by specialized SOA based engines. Model-Driven SOA is a better term.

      I believe that this “Domain-Specific, Model-Driven, SOA-Based” (DOMOSO) approach is a pragmatic and promising solution in many scenarios. Particularly when business domains are stable in time and changes effects specific aspects of business processes only.

      SOA is not enough: we need a Business Process Execution Platform

      SOA and BPM aim to break the silos built by applications in order to allow the seamless execution of Business Processes. I covered this theme in some previous posts.

      Unfortunately, SOA and BPM alone are not enough for this aim. Let’s take a very simple example to demonstrate this statement. In next picture we can see an end-to-end process that creates a new customer, X, calling a service in one of its first steps. After some other steps, the same process call another service in order to bill the customer.

       

      a service can fail for data replication issues
      a service can fail for data replication issues

      The second service call can succeed or fail depending on the fact that the new customer record has already been replicated in the billing application database.

      This happens because SOA allows to design a conceptual model of processes and services but does not cover what happens “under the hood” of applications. SOA is concerned with business logic (orchestrations, business rules) not with data management.

      If SOA has boundaries, the principles that are the foundation of SOA don’t.

      Which are SOA principles? In my opinion they are four:

      1. “Seamless processes”: end-to-end processes must be executed in a seamless way
      2. “Business shape”: IT must be modelled with the shape of Business, ideally every IT item should have a Business counterpart
      3. “Model Driven”: IT should be designed as much as possible with a model-driven approach; consequently manual software coding should be used only when absolutely necessary in order to diminish development and maintenance costs and enhance business agility.
      4. “Global Assets Reuse”: the reuse of existing assets at enterprise level is another important factor for gaining savings, flexibility and time-to-market

      We can try to apply these principals outside the SOA boundaries.

      As a first step we can imagine a unique application that covers all the enterprise needs. This ideal, virtual application is a conceptual tool that allows us to forget for a moment the presence of application silos and think at a holistic, enterprise level. Our application implements, by definition, all business processes our company needs. We can name our application a Business Process Execution Platform (BPEP).

      Being an ideal application, BPEP must be a multi-tier application. In other words Presentation, Business Logic and Data tiers should be well distinguishable when inspecting the software code.

       

      the four tiers of BPEP

      the four tiers of BPEP

      A fourth tier as been added, it represents the IT Infrastructure tier (hardware and basic software).

      The second tier is the Business Logic tier, it contains all the well known solutions for SOA, BPM and Event-Driven Architecture (EDA). There are no doubts (I hope) regarding the possibility to apply our four SOA Principles to this tier. Anyway I will cover all four tiers:

      v     Interaction Tier represent the solutions used by our BPEP application to interact with users (both internal and external) and with external applications (B2B). Here is some example of Four Principles application

      1.      “Seamless processes”: Users should be able to execute every task remaining in the same presentation environment. Only a sign-on required. No more than one entry of every data.

      2.      “Business shape”: every interaction is business meaningful and represents a use-case for a business analysts.

      3.      “Model Driven”: the navigation among presentation forms / web services must be well modelled and configured at design time.

      4.      “Global Assets Reuse”: only one presentation form / web service should be used at enterprise level in order to execute a business function. For example a single presentation component (i. e. mash-up), possibly offering different user experiences, should be used for a customer data entry, including formal checks. The same glossary must be used in all user interactions in order to guarantee homogeneity in naming conventions.

      v     Business Logic Tier is the SOA/BPM kingdom

      1.      “Seamless processes”: end-to-end processes running on BPM engines can cross silos using SOA services exposed by applications.

      2.      “Business shape”: processes, events and services are both business and technical assets.

      3.      “Model Driven”: use of graphical languages like BPEL and of Rules Engines.

      4.      “Global Assets Reuse”: SOA services are reusable, sub-processes are SOA services (that obviously can be reused).

      v     Data Tier represent the tier in which Data are managed. Data are information assets crucial for enabling Business decisions. This tier represents what Gartner calls Enterprise Information Management (EIM):

      1.      “Seamless processes”: the same information should be provided to all qualified users, regardless the application or channel they are using. There must be a “single version of the truth”.

      2.      “Business shape”: Business is concerned with Information Assets (Master Data) not with physical RDBMS columns. An enterprise Common Data Model, including a Glossary, should be defined and managed in order to maintain a common understanding of Master Data. Every business rule (i.e. access control, consistency)  must  be defined on Master Data and then mapped to physical data. However real-time data freshness is not a must for all applications: a right-time approach is realistic and less expensive,

      3.      “Model Driven”: the Common Data Model (CDM) must be formally represented using E-R or Class Diagrams. Data replications among different RDBMS must be configured using graphical tools. Data transformations applied on replication data flows should use the CDM as reference for the semantic reconciliation.

      4.      “Global Assets Reuse”: Master Data are global assets that can be exposed as Data Services in a SOA environment. The concept of Data Service brings to the necessity of a physical or logical consolidation of data sources. This is the battle filed for technologies like master Data Management, ETL, Data Federation,

      v     Infrastructure Tier represent hardware and basic software that enable application execution:

      1.      “Seamless processes”: applications should acquire or release system resources seamlessly, depending on real-time business needs. Applications and resources must be decoupled in order to achieve flexibility.

      2.      “Business shape”: the IT resources must be mapped to “Business Services” (ITIL definition), In this way Business priorities can be used to manage data centres.

      3.      “Model Driven”: the mapping business service-enabling resources should be modelled and configured using a CMDB

      4.      “Global Assets Reuse”: IT resources should be global shared assets. The enabling technologies are: server virtualization / partitioning, grid or cloud computing,

      I hope I could demonstrate that SOA principles are more general than SOA itself.

      SOA is not dead at all but it does evolve, it is like a river that flows up into a more general discipline. I would call this discipline BPEP, but any suggestion regarding a more brilliant term is well accepted.

      SOA has a bad friend: EAI

      One of the typical mistakes regarding SOA is to consider this approach just a new kind of Enterprise Application Integration (EAI) solution.

      I will try to analyze the main differences between SOA and EAI.

      • Scope: EAI is typically an IT only project, SOA should involve business people and IT specialists
      • Goal: the goal of an EAI initiative is to solve the application integration problems. The  goal of a SOA initiative is to better execute, track and monitor business processes that use business services.
      • Means: EAI employs a bus (integration broker) in order to integrate the existing application silos – that remain at their places. SOA aims to break the walls of applications silos that hinder the flow of business processes and to create composite applications.
      • Benefits: EAI unifies the message transport middleware and brings simplification and reduction of technical latencies. SOA brings to agility in the creation, execution and the modification of business processes through the reuse of functional capabilities (services),
      • Topology: EAI has – conceptually – an hub & spoke topology (it aims the unification of the integration tools, although some integration brokers have a distributed technology). SOA has a distributed, enterprise-wide conceptual model (although many vendors push to a centralized, proprietary solution).

      Unfortunately many managers treat SOA as a technical project, this bring to EAI solutions based on the SOAP protocol and do not gain the business benefits of SOA.

      SOA projects should be considered strategic initiatives, obtain some senior management buy-in and involve business people, as well as middleware experts, software analysts and data administrators. Their success should be measured by business KPI, like shorter time-to-market for new products or shorter execution time for end-to-end processes.

       

      Bridging the gap between Business and IT

      In many organizations a “communicative gap” exists between Business and IT. The two teams use different languages and conceptual models. This brings to latency times due to the need to translate Business concepts and requirements in the IT language and vice versa. Moreover, often the translations are poor and bring to misunderstandings and loss of information.

      Sometime Business people learn the IT language and express their requirements in terms of applications and file transfers. Perhaps this is the worst scenario in which the real business needs are totally hidden and IT looses opportunities to enhance its solutions.

      The communication latency prevents an immediate information exchange between Business and IT and slows down the overall reactivity of the organization in its operations in the market.

      In a figurative language we can think to Business as a car driver and to IT as its vehicle that respond some second late to turn, accelerate or brake commands.

      the gap between Business and IT

      the gap between Business and IT

      Moreover business processes “disappear” (get implicit) when passed to IT. Often, important business rules have their IT counterpart in a bunch of Java code lines hidden in millions of other code lines that manage web pages or DB accesses. The changes of business rules require time and money because they require to modify the software analysis documents, to search the IT counterparts, modify the code and test the whole application.

      SOA represents a way to bridge this communicative gap. If services have a real meaning for Business you can create a one-to-one mapping between business processes and their “electronic images” that can be explicitly executed in IT systems by BPM or workflow engines. No semantic intermediations between the two models are needed

       

       

       


      SOA+BPM bridge the gap

      SOA+BPM bridge the gap


      The new processes are modelled by Business people composing functional capabilities (Business Services). From the IT side the development of the new processes will be obtained by composing the correspondent technological capabilities ((Web Services, IT Services).

      Business processes must be able to execute the activities offered by IT regardless the applications that actually have in charge those activities. The Business Services must have a clear meaning for the business experts that design the process and hide to them all the technical details that could blur a sharp representation of business processes.

      Business process representations should make use of very few components:

      • Business services – actions executed by IT systems that have a clear business meaning (i.e. “check the client’s credit”, “create a new order”)
      • Human interactions – solicited or unsolicited man-machine conversations (i.e. “ask the manager for the authorization”, “enter the order data”)
      • Flow controls, like conditional routing or termination.

      Somebody talks of a Process-View-Service (PVS) pattern that will progressively substitute the popular Model-View-Control (MVC) in the development of new applications, actually the proposed representation follows that pattern. BPMN models, as well as BPEL models enriched with human workflow, can be candidate formalisms.

      A hidden component in this view are the data that flows inside the process being used as input and output data for the invoked services.

      I believe that the vision described above represents the real nature of a Service Oriented Architecture and unify the disciplines of service orientation and Business Process Management (BPM). The technical aspects of SOA, like ESB or WS-* standards should be covered by another discipline, i.e. Web Services technologies.

      SOA Governance: SOA requires changes in IT organization

      Silos are not an accident, they exist for at least two reasons:

      1. The organization units (i.e. departments) managers tend to defend the ownership on the software applications that cover the processes falling under their competence. Unfortunately many E2E processes trespass the organizational borders of a single unit.
      2. In software development projects, the project manager (PM) success is measured with two simple KPI: time-to-delivery and development costs. This brings the PM to aprioristically refuse everything that is not under its direct control. Even the most open minded manager will try to avoid the use of shared facilities (like virtual servers or SOA services) because he perceives those facilities as potential threat to the achievement of his goals.

      The behaviour of both kinds of managers are justifiable and often forced. However it brings to the building of silos to the detriment to E2E processes execution.

      time-to-delivery and development costs. This brings the PM to aprioristically refuse everything that is not under its direct control

      The project manager success is measured with two simple KPI: time-to-delivery and development costs. This brings the PM to aprioristically refuse everything that is not under its direct control

       

      Enhancements in E2E process executions, reuse of common assets (like SOA services), more agility, cannot be achieved if “vertical drivers” that bring to silos are not balanced by “horizontal drivers” that aim the reuse of shared assets.
      The enterprises gain savings and agility in  the mid-long term when a reuse policy is enforced

      The enterprises gain savings and agility in the mid-long term when a reuse policy is enforced

      Organizations must commit a different organizational unit, with no vertical interest (i.e. SOA centre of excellence, the Enterprise Architecture team) to pursue an enterprise-wide reuse strategy. This unit must have the power to contrast vertical interests when they are counterproductive for the whole organization.

      The role of SOA Governance

      The role of SOA Governance

      The life-cycle of software initiatives must be re-examined in order to allow the “horizontal unit” to look for reuse opportunities, to suggest best practices and to check for the rise of new silos.