Why not Petri Nets?

I have read the excellent article by Johan den Haan “The Process Centric vs. Information Centric approach to SOA”. It strengthen my idea that BPM must go beyond its current boundaries and explore new languages that better fit business process modeling.

I am not sure that an event-driven model can fit every requirement for modeling, in some case (see my previous post) when the process is driven by a complex data (i.e. product structure) some hierarchical model could be more effective and complementary to an event driven model.

Turning to event-driven I have tried to model a process using Petri Nets. I started from a process that I found hard to model with BPMN. When you have different asynchronous user data entry BPMN does not allow to have a simple vision of the end-to-end process.

Petri Nets allow to design a more flexible model taking into account exceptions in a seamless way. Moreover CEP configuration languages are very close to Petri Nets concepts and allow to quite straightforwardly translates Petri Nets into execution models.

Example of Petri Net (order to production process)

The flavours of SOA

SOA has two “flavours”. Both are valid and fit specific business needs.

1. Vertical (multichannel):

  • several channels share the same data and business logic (services) on a common layer
  • communication is usually synchronous – request/reply
  • the business analysis that brings to services starts from use cases (UML style: use case -> sequence diagram -> class diagram -> services)

2. Horizontal (event-driven / long-running transactions):

  • An applications triggers a business event (i.e. purchase order) and calls an asynchronous service to sink the event. The service is actually a long running process (i.e. order to delivery) implemented on a BPM engine. The BPM engine calls several services in order to execute the process
  • communication is both asynchronous/reliable/one way and synchronous request/reply
  • the business analysis that brings to services starts from a process model (BPMN -> BPEL -> services).

      The choice of which flavour to use depends on the business model of the organization.

      Often Financial services need to implement short running transactions with high interactions with customers (i.e. Internet Banking) and prefer the multichannel flavour.

      Manufacturing and logistics are more concerned in long running transactions and prefer the second flavour.

      Often Telco companies stay in the middle and need both.

      I think that every organization should explore the use of both styles, putting in discussion the internal culture (transaction-oriented or process-oriented) and evaluating the benefits coming from a different approach.

      BPM is a too tight dress on SOA

      In my previous posts I stressed the fact that BPM and SOA together brings many benefits in terms of business-IT alignment, agility, time-to-market and reuse of business capabilities.

      BPM and SOA cooperate in a symbiotic way:

      • BPM design-time tools model the business process to implement
      • BPM run-time tools (engines) execute the process instances calling SOA services
      • SOA offers the “execution environment” for BPM engines

      SOA services (together with user interaction forms in case of human workflow) are the building blocks for modeling and executing Business Processes.

      With this approach you can change your processes just changing the graphical model.

      Of course sometime you will need to develop extra SOA services. In other words you will need to expand your process execution environment. If no extra service is needed the time required for your change is minimal.

      This is a perfect scenario. If every Business Process could be implemented this way the promised benefits would be easily achieved.

      Unfortunately the world is more complex.

      Let’s take an example from a Telco company. One of the typical processes for a Telco is the “Order-to-Delivery” one. When a new purchase order has been issued the company has to provision all the components related to that order.

      You can model the O2D process using a BPMN or BPEL model. However the O2D process is strictly related to the purchased product and its components (ADSL line, cellular phone, modem, IP TV). You have two choices:

      1. Model a generic O2D process that fits every product.
      2. Model a very detailed process in which to specify every dependence among the components to provision (“B must be provisioned after A”) and the exceptions to handle. A complex model, “flow-chart like” is obtained.

      In the former case the model is quite useless. Clearly the logic about the provisioning of a specific product is outside the BPM model.

      In the latter case you get a complex and algorithmic model just less complex to maintain than a Java program. BPMN and BPEL reminds too much the procedural languages, it’s very easy to trespass the limit and use them as old fashioned flow-charts diagrams.

      If you modeled other aspects, like billing or pricing you could have the confirmation that BPMN and BPEL are not able to represent every aspect of a business process. They are languages specific for high level business processes and for simple orchestrations.

      Let’s imagine to have different formalisms that help to model different aspects of a business domain.

      For example the provisioning dependencies could be easily described in a non-procedural way using a Pert model.  Billing rules could be described with some hierarchical model derived from the product’s bill of material. Specific engines, similar to BPM engines shall be developed in order to interpret the run time instances derived from these non-BPM models. Of course the SOA environment is still valid also for these specialized engines.

      I do not propose a classic Model Driven Architecture (MDA) approach since the MDA brings to the automatic generation of programming code while I think to a non-procedural graphic model directly interpreted by specialized SOA based engines. Model-Driven SOA is a better term.

      I believe that this “Domain-Specific, Model-Driven, SOA-Based” (DOMOSO) approach is a pragmatic and promising solution in many scenarios. Particularly when business domains are stable in time and changes effects specific aspects of business processes only.

      SOA is not enough: we need a Business Process Execution Platform

      SOA and BPM aim to break the silos built by applications in order to allow the seamless execution of Business Processes. I covered this theme in some previous posts.

      Unfortunately, SOA and BPM alone are not enough for this aim. Let’s take a very simple example to demonstrate this statement. In next picture we can see an end-to-end process that creates a new customer, X, calling a service in one of its first steps. After some other steps, the same process call another service in order to bill the customer.

       

      a service can fail for data replication issues
      a service can fail for data replication issues

      The second service call can succeed or fail depending on the fact that the new customer record has already been replicated in the billing application database.

      This happens because SOA allows to design a conceptual model of processes and services but does not cover what happens “under the hood” of applications. SOA is concerned with business logic (orchestrations, business rules) not with data management.

      If SOA has boundaries, the principles that are the foundation of SOA don’t.

      Which are SOA principles? In my opinion they are four:

      1. “Seamless processes”: end-to-end processes must be executed in a seamless way
      2. “Business shape”: IT must be modelled with the shape of Business, ideally every IT item should have a Business counterpart
      3. “Model Driven”: IT should be designed as much as possible with a model-driven approach; consequently manual software coding should be used only when absolutely necessary in order to diminish development and maintenance costs and enhance business agility.
      4. “Global Assets Reuse”: the reuse of existing assets at enterprise level is another important factor for gaining savings, flexibility and time-to-market

      We can try to apply these principals outside the SOA boundaries.

      As a first step we can imagine a unique application that covers all the enterprise needs. This ideal, virtual application is a conceptual tool that allows us to forget for a moment the presence of application silos and think at a holistic, enterprise level. Our application implements, by definition, all business processes our company needs. We can name our application a Business Process Execution Platform (BPEP).

      Being an ideal application, BPEP must be a multi-tier application. In other words Presentation, Business Logic and Data tiers should be well distinguishable when inspecting the software code.

       

      the four tiers of BPEP

      the four tiers of BPEP

      A fourth tier as been added, it represents the IT Infrastructure tier (hardware and basic software).

      The second tier is the Business Logic tier, it contains all the well known solutions for SOA, BPM and Event-Driven Architecture (EDA). There are no doubts (I hope) regarding the possibility to apply our four SOA Principles to this tier. Anyway I will cover all four tiers:

      v     Interaction Tier represent the solutions used by our BPEP application to interact with users (both internal and external) and with external applications (B2B). Here is some example of Four Principles application

      1.      “Seamless processes”: Users should be able to execute every task remaining in the same presentation environment. Only a sign-on required. No more than one entry of every data.

      2.      “Business shape”: every interaction is business meaningful and represents a use-case for a business analysts.

      3.      “Model Driven”: the navigation among presentation forms / web services must be well modelled and configured at design time.

      4.      “Global Assets Reuse”: only one presentation form / web service should be used at enterprise level in order to execute a business function. For example a single presentation component (i. e. mash-up), possibly offering different user experiences, should be used for a customer data entry, including formal checks. The same glossary must be used in all user interactions in order to guarantee homogeneity in naming conventions.

      v     Business Logic Tier is the SOA/BPM kingdom

      1.      “Seamless processes”: end-to-end processes running on BPM engines can cross silos using SOA services exposed by applications.

      2.      “Business shape”: processes, events and services are both business and technical assets.

      3.      “Model Driven”: use of graphical languages like BPEL and of Rules Engines.

      4.      “Global Assets Reuse”: SOA services are reusable, sub-processes are SOA services (that obviously can be reused).

      v     Data Tier represent the tier in which Data are managed. Data are information assets crucial for enabling Business decisions. This tier represents what Gartner calls Enterprise Information Management (EIM):

      1.      “Seamless processes”: the same information should be provided to all qualified users, regardless the application or channel they are using. There must be a “single version of the truth”.

      2.      “Business shape”: Business is concerned with Information Assets (Master Data) not with physical RDBMS columns. An enterprise Common Data Model, including a Glossary, should be defined and managed in order to maintain a common understanding of Master Data. Every business rule (i.e. access control, consistency)  must  be defined on Master Data and then mapped to physical data. However real-time data freshness is not a must for all applications: a right-time approach is realistic and less expensive,

      3.      “Model Driven”: the Common Data Model (CDM) must be formally represented using E-R or Class Diagrams. Data replications among different RDBMS must be configured using graphical tools. Data transformations applied on replication data flows should use the CDM as reference for the semantic reconciliation.

      4.      “Global Assets Reuse”: Master Data are global assets that can be exposed as Data Services in a SOA environment. The concept of Data Service brings to the necessity of a physical or logical consolidation of data sources. This is the battle filed for technologies like master Data Management, ETL, Data Federation,

      v     Infrastructure Tier represent hardware and basic software that enable application execution:

      1.      “Seamless processes”: applications should acquire or release system resources seamlessly, depending on real-time business needs. Applications and resources must be decoupled in order to achieve flexibility.

      2.      “Business shape”: the IT resources must be mapped to “Business Services” (ITIL definition), In this way Business priorities can be used to manage data centres.

      3.      “Model Driven”: the mapping business service-enabling resources should be modelled and configured using a CMDB

      4.      “Global Assets Reuse”: IT resources should be global shared assets. The enabling technologies are: server virtualization / partitioning, grid or cloud computing,

      I hope I could demonstrate that SOA principles are more general than SOA itself.

      SOA is not dead at all but it does evolve, it is like a river that flows up into a more general discipline. I would call this discipline BPEP, but any suggestion regarding a more brilliant term is well accepted.

      ESB: a misconception brings to wrong architectures

      Few terms in IT are more unfit than the ESB one.
      ESB means
      • Enterprise (only one for the whole enterprise)
      • Service (brings web services calls)
      • Bus (a common facility, like Ethernet or electric system).
      Is it a correct definition? Lets start with “enterprise”. How many ESB will enterprises use in their future? There are many reasons why the majority of organizations will use more than one brand of ESB:
      1. some ESBs come with COTS packages, i.e. SAP Netweaver or Oracle Fusion
      2. appliances can be used instead of an ESB in presence of critical performance constraints
      3. Merges & Acquisitions brings new ESBs in your IT department
      4. sometimes you don’t have much budget an open source ESB could let you save money
      5. a partner wants you to use its B2B solution, that includes an ESB
      6. your preferred vendor has been acquired and your preferred ESB is no longer viable
      7. …
      Even when you succeed to install only a brand of ESB you could be forced to deploy more than one instance. Some applications will have specific throughput requirements that require separate ESB instances. Moreover your Kazak and Bolivian subsidiaries will unlikely use your centralized ESB instance for their local communications.
      It is important to anticipate the case of having many instances of ESBs, possibly coming from different vendors.
      The concept of “Enterprise Service Bus” (only one in your organization) is unrealistic and risky because it can bring to bottlenecks and hinder the adoption of the most adequate and efficient choices to be made case by case, for example:
      • the choice of the most suited package, regardless the internal middleware solution
      • the possibility to bypass the central “Enterprise Service Bus” when it does not offer added value; the number of not necessary hops can quickly grow, particularly in case of orchestrations and compositions

      an ESB can by bypassed when it does not bring value

      an ESB can by bypassed when it does not bring value

      • a world-wide distribution of a SOA connectivity can bring to an unfeasible or expensive centralized solution
      • some applications require features than the selected ESB products does not provide

       

      How to gain SOA Governance in presence of more than one ESB?

      Actually several Governance functions must be centralized, if possible at Enterprise level:

                Registry/Repository,

                Security,

                Web Service Management (WSM) / Policy Management (PM),

                Semantic reconciliation,

                Late binding

      If, as seen before, you cannot have only one ESB, you must entrust this task to other tools that must be

      4     unique at enterprise level and

      4     independent from the technologies of your ESBs

       

      The “WS-Fabric”

      If you use Web Services standards you actually have a point of centralization, this is not the ESB but the place in which SOAP messages are moved over a standard transport layer, I use a term caught from Jim Webber: WS Fabric.

      the ESBs implement intermediaries

      the ESBs implement intermediaries

       If the WS-Fabric is the center of the SOA communication which is the role of an ESB? In my opinion the ESBs are tools that have an important role for the generation of intermediaries and can be placed both on consumer and in provider sides. Anyway, a piece of WS-Fabric must be placed in front of every service end-point.

      Intermediaries can be functional (i.e. can offer message transformations, service compositions, publishing, routing, adapters) or technical (access control, monitoring,  policy enforcers…).

      SOA has a bad friend: EAI

      One of the typical mistakes regarding SOA is to consider this approach just a new kind of Enterprise Application Integration (EAI) solution.

      I will try to analyze the main differences between SOA and EAI.

      • Scope: EAI is typically an IT only project, SOA should involve business people and IT specialists
      • Goal: the goal of an EAI initiative is to solve the application integration problems. The  goal of a SOA initiative is to better execute, track and monitor business processes that use business services.
      • Means: EAI employs a bus (integration broker) in order to integrate the existing application silos – that remain at their places. SOA aims to break the walls of applications silos that hinder the flow of business processes and to create composite applications.
      • Benefits: EAI unifies the message transport middleware and brings simplification and reduction of technical latencies. SOA brings to agility in the creation, execution and the modification of business processes through the reuse of functional capabilities (services),
      • Topology: EAI has – conceptually – an hub & spoke topology (it aims the unification of the integration tools, although some integration brokers have a distributed technology). SOA has a distributed, enterprise-wide conceptual model (although many vendors push to a centralized, proprietary solution).

      Unfortunately many managers treat SOA as a technical project, this bring to EAI solutions based on the SOAP protocol and do not gain the business benefits of SOA.

      SOA projects should be considered strategic initiatives, obtain some senior management buy-in and involve business people, as well as middleware experts, software analysts and data administrators. Their success should be measured by business KPI, like shorter time-to-market for new products or shorter execution time for end-to-end processes.

       

      Bridging the gap between Business and IT

      In many organizations a “communicative gap” exists between Business and IT. The two teams use different languages and conceptual models. This brings to latency times due to the need to translate Business concepts and requirements in the IT language and vice versa. Moreover, often the translations are poor and bring to misunderstandings and loss of information.

      Sometime Business people learn the IT language and express their requirements in terms of applications and file transfers. Perhaps this is the worst scenario in which the real business needs are totally hidden and IT looses opportunities to enhance its solutions.

      The communication latency prevents an immediate information exchange between Business and IT and slows down the overall reactivity of the organization in its operations in the market.

      In a figurative language we can think to Business as a car driver and to IT as its vehicle that respond some second late to turn, accelerate or brake commands.

      the gap between Business and IT

      the gap between Business and IT

      Moreover business processes “disappear” (get implicit) when passed to IT. Often, important business rules have their IT counterpart in a bunch of Java code lines hidden in millions of other code lines that manage web pages or DB accesses. The changes of business rules require time and money because they require to modify the software analysis documents, to search the IT counterparts, modify the code and test the whole application.

      SOA represents a way to bridge this communicative gap. If services have a real meaning for Business you can create a one-to-one mapping between business processes and their “electronic images” that can be explicitly executed in IT systems by BPM or workflow engines. No semantic intermediations between the two models are needed

       

       

       


      SOA+BPM bridge the gap

      SOA+BPM bridge the gap


      The new processes are modelled by Business people composing functional capabilities (Business Services). From the IT side the development of the new processes will be obtained by composing the correspondent technological capabilities ((Web Services, IT Services).

      Business processes must be able to execute the activities offered by IT regardless the applications that actually have in charge those activities. The Business Services must have a clear meaning for the business experts that design the process and hide to them all the technical details that could blur a sharp representation of business processes.

      Business process representations should make use of very few components:

      • Business services – actions executed by IT systems that have a clear business meaning (i.e. “check the client’s credit”, “create a new order”)
      • Human interactions – solicited or unsolicited man-machine conversations (i.e. “ask the manager for the authorization”, “enter the order data”)
      • Flow controls, like conditional routing or termination.

      Somebody talks of a Process-View-Service (PVS) pattern that will progressively substitute the popular Model-View-Control (MVC) in the development of new applications, actually the proposed representation follows that pattern. BPMN models, as well as BPEL models enriched with human workflow, can be candidate formalisms.

      A hidden component in this view are the data that flows inside the process being used as input and output data for the invoked services.

      I believe that the vision described above represents the real nature of a Service Oriented Architecture and unify the disciplines of service orientation and Business Process Management (BPM). The technical aspects of SOA, like ESB or WS-* standards should be covered by another discipline, i.e. Web Services technologies.