Cordys BOP4.1 to be released

Ok, we have been waiting for it for quite some time now...
However -at last- on Cordial 2011 Cordys announced that Fix Pack 1 for BOP4 (commercial name: BOP4.1) is to be released before the end of this month!

Those who are familiar with my blogs will understand that I was very eager to find out all about the new “goodies” packed in this new release.

At Cordial11 we were presented a brief overview of the new features of BOP4.1 and the roadmap for future capabilities. Though it was not completely clear to me what exactly is in scope for 4.1 and what not I would like to share the global picture of my perception of the new features of BOP4.1.

BAM KPI-s simplified
In BOP4 implementing BAM is fairly cumbersome. To have an idea: it involves five different kinds artifacts in BOP4 to report on KPI-s.
In BOP4.1 Cordys allows to define a KPI right where it belongs: in the BPM itself. This KPI is directly usable in dashboards. This is a big improvement for productivity, maintainability and transparency.

Document support in BPM-s
In BOP4 extensive document handling was only offered in Case Management.
BOP4.1 supports this now for BPM-s as well by allowing users to attach documents to process instances. Design time It will be possible to control (per activity) the types of documents that are allowed and the according access rights for the documents for different roles and users.

Multi language support
BOP4 offered very limited language support.
This should drastically improve in BOP4.1. Every language sensitive element in Cordys will be candidate subject to multi language support. Integration of BOP4.1 with Google Translate ensures that only a minimum of effort is required to create the required language tables.

Social Collaboration
BOP4.1 allows multiple developers on remote locations to simultaneously alter or review artifacts, like BPM-s and User Screens. Changes will be immediately visible available to all participants. During this process they can interact with each other, e.g. through real time messaging.

Document Generation
BOP4.1 supports documentation generation on BPM-s.
To be honest the demonstrated documentation looked rather plain and technical in my eyes, so I expect the intended audience is development. I am not really convinced if this will add much value. In my opinion models and structure should be “self explanatory” anyway (according to the agile paradigm), and all information is available in a structured way in the Cordys repository.

Multi browser support
Cordys claims that BOP4.1. will be supported by all widely accepted web browsers, like Safire, Google Chrome and Firefox. This is simply good news, as we already have seen customer requirements on browser support!

MDM modeling
BOP4.1 has improved MDM modeling capabilities.
In BOP4 I have used MDM hub and spoke only because it was required for BAM. Therefore it is hard to value this enhancement.

Process Instance Management (PIM) improvements
BOP4.1 will offer enhancements in PIM. One of the demonstrated capabilities is that it is possible to have BAM-like overviews on the status for specific users: What is their current work load, what is their productivity etc.
In the roadmap session I saw a demo of social capabilities as well, like being able to obtain all kind of information of participants in work lists and the possibility to interact with them though different social media. It is unclear to me to which extent this functionality will be in scope for BOP4.1.

Integration CPF-BOP4.1
BOP4.1 will provide integration with Cordys’ PaaS solution “Cordys Process Factory” (CPF): A logical step to improve the cloud capabilities of BOP4.1. Unfortunately we did not see a demo of this, so I cannot value this enhancement at this moment.

Deployment features
BOP4.1 will offer deployment enhancements by changing the ISV Package structure.
This should allow for easier, safer and better controlled deployments. It should improve productivity for administration and reduce the risk of errors.

Other enhancements and improvements:
A number of enhancements in BOP4.1 will likely increase development productivity to some extend:
- Context sensitive options for constructs in the BPM models (like I have already seen in e.g. SAP BPM). This means that when selecting e.g. an activity the developer can immediately see what the possible connectable constructs are and the required one can be selected.
- Zoom functionality (Handy for larger models. On the other hand: Do we want models to be large?)
- Improved process messaging (In BOP4 this was a bit cumbersome -having to use process instance ids-, as I wrote on in my earlier Blogs)
- E-Mail integration (E.g. easier to make mail templates)

Overall BOP4.1 looks like indeed it will offer some significant enhancements.
Currently I am personally most happy with multi browser support, multi language support, the new BAM and document management. I would really like to lay my hands on those.

I will address my next blog to the features that I am still missing in BOP4.1 and the roadmap of Cordys.

Regards, Harald van der Weel

Top 10 wish list for BOP4 fix pack 1.

2011 will be an exciting year!

One of the reasons is the expected major release for Cordys BOP4: Fix Pack 1! (Maybe already in Q1?)

Though already am a “fan” of the current product (See my blog: Why Cordys BOP4) I have my top 10 wish list ready:


1. BPMN 2.0
As the formal release of BPMN 2.0 is due soon, and 99 percent of the standard is stable I hope for a solid support of this standard in FP1.

Especially extension of the event- and signaling mechanisms, and the addition of compensation constructs would be valuable features!

2. Data modeling
Currently it is possible to define data structures in BOP4. Additional UML class diagram-like modeling would greatly support the process of defining and communicating the business object model in a graphical way. Obviously MDM would profit too.

3. BAM Usability
As I already wrote in an earlier BLOG BAM Imperfections there are several possibilities to improve usability and flexibility of BAM. The good news is that Cordys already committed to these improvements, so expectations are high!

4. Document support BPM
Case Management already supports a kind of out-of-the-box document management. It seems little investment to allow documents to be attached to BPM Instances as well. This would prevent the need for a custom solution based on an external DMS or database solution.

5. Simulation
A fair simulation environment would be a welcome extension to the BOP4 stack. It should allow for defining the expected loads and spread, and the characteristics of the resource capacity. Reports over the simulation runs should allow for proper analysis and process optimization.

6. Complex Event Processing (CEP)
In automated environments where decision making is depending on the combination of several (types of) events Complex Event Processing (CEP) is a must-have. Addition of a CEP module would greatly increase the applicability of BOP4.

7. “Hot” business rules
BOP4 offers great facilities of defining business rules and decision tables. The feature of allowing business to “hot modify” the (values/business parameters of) rules without having to republish or redeploy them would increase the flexibility of the concerned business processes.

8. User interface: modeling versus scripting
For some fairly basic UI functions it takes java scripting in BOP4 (Example: the invocation of a business process). It would improve development efficiency to allow developers to model such behavior instead of scripting it.

9. BPM Complex Mapping
The BPM data mapping possibilities in BOP4 are quiet sophisticated. However certain types of mappings are not yet supported in the BPM mapping tool, where the underlying technology XSLT would allow so. Example: the mapping of two lists onto each other. One of the work arounds is to model these mappings in the BPM itself, obscuring the intended business functionality. Extension of the mapping functionality would prevent this necessity.

10. Reporting
Where BOP4 offers fairly advanced BAM functionality, plain reporting functionality is missing.
As the Cordys Process Factory (CPF) offers reporting functionality it should also not be rocket science to get it available for BOP4 too.

As Cordys claims the BOP4 product itself is the most valuable marketing instrument I have great expectations and I hope that (a late Santa) will generously fill my sock!


Harald van der Weel

Getting started with BOP4: Cordys Academy self study Training

 
I have received several requests to inform people on “how to start off with BOP4”:
Where to begin, how to learn about the features and how to get hands-on experience?

The good news: Cordys has recognized this need!
One of the reasons I have become eager to adopt Cordys BOP4 is the simple fact that Cordys Academy offers high quality training facilities.

Not just the conventional classroom courses: Also a great computer based BOP4 fundamentals! For free, and it can be taken anytime, anywhere!

The Cordys Academy is offering basically a complete package, consisting of the following:

-          Self study environment

Link: https://wiki.cordys.com/display/acorg/Cordys+BOP-4+Self+Study+Training+Environment

-          Fundamentals Tutorial

Link: https://wiki.cordys.com/display/acorg/Cordys+Fundamentals+BOP-4

-          Several advanced topics (“Training extensions”)

Link: https://wiki.cordys.com/display/acorg/Training%20extensions

Access to these sites only requires registration.
This can be arranged instantly through any of the links above. Each will result in the login screen below:



Selecting “New User? Register here” will show the form below.

 
Complete the form and follow further instructions to register.

Once done… you are ready to “play”!
 
Self study environment

The self study environment (I’ll call it “sandbox”) is nothing but a single downloadable VM-ware image!
It is configured as a complete Cordys server the provides all main functionality.
Apart from allowing “students” to do all practical exercises in the tutorials, it even would allow you to do a complete project (for the 3 months license period at least)

Provided you have a VM Ware player (free downloadable at http://www.vmware.com/ ), setting it up simply requires the following steps:

1.       Download the zipped VM Ware environment (It is rather big: Zipped about 2Gb, unzipped about 6Gb)

2.       Unzip the VM environment

3.       Run the VM image

4.       Start a browser session (for the moment IE7 or IE8)

5.       Go to the startup URL (Specified in the Setup and User guide)

6.       Start using Cordys BOP4, provided a default Student1

(If still more info is required: The steps above are well explained in a “Setup and User guide”, that can be downloaded from the “self study environment” page)

This sandbox allows to manage users and provide them with the necessary roles to develop, test, deploy, execute and monitor almost any Cordys work product. The sandbox even allows to use an external SVN version control system to store and version all work products externally from the VM Environment.
The sandbox even provides an example MySQL database with content and web services.

The link to the Self Study Environment is: https://wiki.cordys.com/display/acorg/Cordys+BOP-4+Self+Study+Training+Environment, which invokes the page below.


Fundamentals tutorial

The fundaments tutorial teaches novice Cordys BOP4 users everything to know about the BOP4 environment.

Multiple tutorial “chapters” take the user sequentially through all necessary subjects, in order to make a start with Cordys BOP4.
Each chapter contains theoretical information (documents and presentations) and hands-on exercises to practice the learned skills. The exercises are very concise and even suitable for people with very limited technical skills.

Subjects that are covered are:

-          Overview and introduction to the Cordys environment

-          Managing users and setting up a new project

-          Developing business processes (BPMN models)

-          Developing and using services

-          Developing user interfaces

-          Developing workflows

The exercises of each module should be executed sequentially, because there are certain dependencies between them.
They will provide a starting developer with the basic skills to build and run simple BPM solutions.

Note that doing all tutorials will take between about 3 and 5 days to finish, depending on your experience in similar environments.

The link https://wiki.cordys.com/display/acorg/Cordys+Fundamentals+BOP-4 takes you to the page below. The order of the modules is Top-Down


Training extensions

A number of advanced BOP4 topics are offered in “Training Extensions”.

This concerns topics that elaborate certain topics in the fundamentals, but also subjects that have not been covered yet before (like Business Activity Monitoring (BAM) or Case Management).

In case of requiring more in-depth knowledge or considering certification these training extensions are a valuable source of information.

The link https://wiki.cordys.com/display/acorg/Training%20extensions will take you to the following page:


Conclusion

Cordys provides great means for organizations and users to gain knowledge about the BOP4 product from scratch at no (direct) cost.

Please bear in mind:
A computer based training may provide you a good starting point. Effective skills are mostly built up by actually applying them in a practical situation.
I would advise starting teams to involve an experienced developer/solution architect, for a number of reasons, like:

-          It provides a direct productivity gain

-          It helps to properly setting up the project and guidelines

-          It will (cross)-train your inexperienced team

-          It will solve “blocking issues” faster

-          You may profit of the network of the experienced developer

Good luck to all new Cordys BOP4 adopters.

Hope to meet you in a future project!

My road to BOP4 certification

I have been a satisfied Cordys consultant for some time. This may be no surprise for the ones having read my BLOGs the past few months. Though I am still looking at “other fish in the sea” I felt I was ready for a new step in my relationship with Cordys BOP4 and I wanted to take things to the next level: Certification!

As I was confident about my knowledge and experience I decided to subscribe for certification, even before knowing the exact procedure. Remembering earlier certifications like RUP, PMBok, Prince2, ITIL and Java, I was expecting an on-site computer based theory exam. So I blocked a day to travel to Cordys in the beautiful surroundings of Putten. At the meantime I chose a nice piece of wall in my office to proudly present my future Cordys certificate!

I read the Cordys certification site and spoke to two primer Cordys employees who had taken the exam as well. How (happily) surprised I was to found out the complete exam could be taken from my own office! (Good news in an overcrowded country like the Netherlands, where distances are short, but travel times sometimes long!).

I was less thrilled to find out it wasn’t just a theoretical exam I had to take, like the ones of several other BPM vendors. It would not as simple as that! Apart from an 80 question theory exam, I would be exposed to a practical exam and an web cam interview before a commission!

It was getting clear to me that Cordys does not hand out a certification to just anyone who is able to spell out “BOP4”.

I felt obliged to do a little more of preparation than initially planned, which was a bit of a pain in these busy times. On the other hand: The value of the future certificate would be a lot higher because this higher threshold!

I ordered the certification exam via https://certification.cordys.com, and selected the date I already blocked.

The learning path for the theory requirements showed – though I already had considerable hands on experience with BOP4 - there were still a lot of details to know, as I (appearently) hadn’t touched every dark corner of BOP4 during the projects.

The good thing was that the windy road through the BOP4 documentation provided me with a number new insights. Cool! I was even increasing my knowledge, instead of just proofing it.

Then the moment was there!

On the date the exam was scheduled I could pick my own time to take the theory exam, within the timeframe of 24 hours. I just needed to visit the certification dashboard and press the “take exam” button.

The game was on!

In 60 minutes 80 various multiple-choice questions were fired at me, of which at least 75 percent had to be answered correctly. The exam tool allowed to mark questions to review them later (provided there is still time left!). 80 questions in an hour is a fair amount, but it is easy to spend too much time in rethinking the alternatives. I needed my time so I just had about 10 minutes left to reconsider the questions I was not sure about. I did not have to use the “end exam” button: After 60 minutes the exam ended automatically.

The certification site then asked me politely for some patience to determine the outcome… And… after a short while I was congratulated! I had passed the theoretical part of the exam! And – more important - the prerequisite to take the second, practical part.

The certification site presented me with 6 practical exercises to solve. All should be finished within a week.

I have the disposition of a local BOP4 environment, however I decided to use the VM-ware environment provided by Cordys: It provides the North-Wind demo, and likely some of the artifacts or database tables could be reused in the exam exercises.

The exercises concerned mainly (combined) UI, BPM and business rules tasks. Of some it took some time to interpret the exact purpose of each assignment, but in the end it was rather obvious what was expected. I decided to start right away, and managed to fulfill the questions in roughly an hour each, including interpretation and testing. Indeed I was able to reuse some NorthWind database tables and webservices, so that was a little advantage.

After reviewing the solutions again the next day I was sure enough to submit the results. This was just a matter of synchronizing the workspace and zipping and sending the synchronized folder to the certification commission.

After submitting the certification site stated that it would maximally take 3 weeks before I would hear whether I would be conducted to an interview.

It appeared to take only a couple of days before I received (and confirmed) an invitation. As required I purchased a webcam and prepared my google environment for the video conference.

The interview was conducted by the certification program manager and the product manager. Basically I had to explain each solution, and answer additional questions about them. My experience and knowledge proved to be sound: as I was capable of coming up with all the answers.

After more than an hour we finished discussing all exercises and the interview was over.

For a third time I was kindly requested to wait for a verdict.

Confident I waited the result. Only after a quarter of an hour I was informed by a colleague who learned via Cordys Twitter that I was certified! Not long after that I received a formal confirmation mail as well. It was official: I had entered the world of Certified Cordys consultants!

Though I had some expectations I felt pretty relieved after the three-stage-procedure that lasted over a week! What made me especially proud was the score of 98% in the practical part.

And now?

Now we’re even more ready for all those fine BPM projects that will use BOP4!

Best Regards and a fruitful 2011,

Harald van der Weel

Why Cordys BOP4

 

Since the latest “rebrand” of BOP4 is this Cordys BPM -stack divided in three suites: CAF, BPMS and SOA grid.

They each map to a specific level in the 5-tier model:

·         CAF (Composite Application Framework) implements the Presentation-tier. It includes  User Interface + Task Management + Composite Applications.

·         BPMS (BPM Suite) embodies the “Process” tier. It consists of BPM + BAM + Business Rules + Case Management.

·         The SOA Grid represents the Services/Integration tier. It contains of MDM, SOA /  ESB and Business Services.

 

BPMS can be considered as the core of BOP4, containing the primary BPM artifacts. CAF and SOA complete the stack, to allow the development of complete solutions, including user interaction and services integration.

 

Other vendors like Tibco and Oracle do package and sell their BPM solutions as complete suites as well. Under the hood these stacks consist of multiple separate products (glued together), each with a specific purpose and (development / runtime) environments.

 

The Tibco stack alone contains e.g. iProcess (BPM execution and design), Business Studio (BPMN design), Business Works (BW - develop services), iDecisions (Business Rules), Active Matrix (ESB), Rendevous and EMS (Messaging), General Interface (GI - Presentation) and iAnalytics and Spotfire (BAM).

BW (as successor of IM) and the Messaging products EMS and Rendezvous can be considered as the core products that made Tibco big. The other products were either developed or bought (E.g. iProcess, iAnalytics and iDecisions are originally Staffware products) to complete the Tibco stack and respond to the market, that has shifted from EAI towards BPM over the past years. Integrating all these products (or better: allowing them to corporate) has resulted in a sub-optimal solution.

For example: Using only iProcess/Business Studio (BPM) en BW (EAI) together already involves 3 different modeling environments, 2 runtime environments, and 2 different interfaces between BW and iProcess (one requiring EMS). It is not hard to image the complexity of administrating the stack, and developing, testing, deploying and maintaining the cross-product solutions built on it, and obviously the resulting additional cost and lead times.

 

For Oracle goes a similar story. When EAI became “hot” the database giant added the Fusion stack to their data-oriented and ERP portfolio. Once Oracle recognized that their BPEL engine was not sufficient to serve the BPM market, they took over BEA to fill the gap with AquaLogic BPM stack.

Apart from the integration issues of all these components another issue was introduced: In the 11g version it is still hard for Oracle to explain the coexistence of two process oriented products like Oracle BPEL and BPM in the same suite. Though Oracle states that BPEL should be used for short-lived processes where BPM is intended for long-lived processes this is a rather artificial distinction, as the BPM product can do both!

 

In order to prevent stepping in these pitfalls Cordys has deliberately chosen for an alternative approach. The Baan company invested considerable time (2 years!) and money to hide inside a lab developing the BPM product stack “Business Operations Platform 4” (or simply BOP4). This suite is effectively one solid integrated stack. The same web application may be used by Business Analysts, UI-, Service- or BPM developers, testers, administrators and users to do “their job”.

 

The so called “Composite Application Framework”-environment allows to  design, implement, test, deploy and run any BPM related artifact. The BOP4 authentication and authorization mechanisms supports role based security for any user role, so every user will only “see” the functionality he/she requires. Artifacts can be seamlessly combined: Web Services, roles, data structures, business rules, UI-screens, Case Models, other BPM flows… All can be added to e.g. BPM model simply by dragging and dropping it on the canvas. A button click allows to validate, publish, run, test or debug the same BPM-process.

 

Cordys promotes the philosophy “eat your own dog food”. That implies that the same approach is used to setup the BOP4 environment itself: The stack completely web service based, just like the custom applications developed upon it. This implies that all tools and functionality provided by the BOP4 environment are available to use as a web service and used in the custom application! (E.g. user management).

Deploying applications itself is a simple 2 step process, that allows for a complete project to be packaged and deployed on a different environment.

 

It is not hard to imagine that facilitates quick time-to-market solutions, Something that has been proved by our projects. Combined with the claim of Cordys that the product is linear scalable it BOP4 is surely a strong proposition. As Cordys BOP4 is completely based upon open standards (e.g. BPMN, SOAP, JMS, XForms, SAML and XML / XPath / XSLT), the risks for vendor buy-in are limited.

 

Does this all sound too much as a sales pitch? Please forgive an BPM-consultant for being enthusiastic about a product that does what he really needs!

Surely: We had some issues with the first versions of BOP4, due to immaturity (Since CU4 stability has increased significantly!). And yes, not all components are completely “ready” (like BAM).

 

However due to BOP4-s solid architecture and the eagerness of Cordys to offer a top-of-the-bill BPM stack these drawbacks will be a matter of time. And they should, as Cordys claims: The strongest marketing tool of BOP4 is the product itself.

Case Management combined with BPM

The Cordys BOP4 BPMS suite offers -amongst other features- Case Modelling and -obviously- Business Process Modelling (BPM)

 

As most of you will know BPM is based on the OMG standard BPMN and is intended for relatively well defined business processes that follow a certain sequence of steps in order to meet a certain business goal. Examples: account payable, handling purchase orders, new hires etc.

 

Case Modelling is intended for processes or activities that in general may not be squeezed into a rigid flow. Each instance of a case model (case) has unique characteristics and (free format) information attached and requires a specific approach. The so-called "case workers" are in general professionals with specialized skills, who should decide per case what action to take and at what time, and what information is relevant to the case. This requires that case workers are not restricted by rigid sequences and data structures.

The generic aspects that apply to each case instance (e.g. state transitions or constraints in the order of activities) can be defined using Case Modelling, and still may leave a lot of freedom to the case worker. Case modelling is therefor more suitable in case of complex processes like insurance claims, all kind of disputes and complex loan applications.

 

Can't this Case behaviour be accomplished in BPM? Obviously BPMN supports parallelism and loops, and steps may be defined as "optional". Hence it may be possible to model similar functionality in BPM, but only with additional effort. Besides that BPM is lacking specific functionality like attaching documents, work assignments and state transitions.

 

There is however also some functionality that Cordys Case Management lacks in comparison to BPM, e.g. the possibility to add web-service invocation steps and the execution of business rules. 

 

The description and examples above may suggest that BPM and Case Modelling are mutual exclusive. I however am convinced that each process may potentially contain both BPM and Case Model characteristics.

 

An example: A visit of a patient in a hospital may require an well defined administrative sign-up process (which could be implemented in a BPM), followed by a more more agile diagnosis or treatment (which could be a Case model).

 

Cordys BOP4 has recognized this need, and allows to use BPM and Case Modelling together:

A BPM can be added as an activity to a Case model, and a case model can be incorporated as a step in a BPM.

 

This provides us with the following options:

 - Decide for each process separately whether to make it a BPM or Case Model (selecting the most appropriate)

 - Make each process a BPM process that may include Case Model steps.

 - Make each process a Case, that may contain BPM sequences.

  

I advise to pick option 2 and make every process a BPM, and include Case Models if required. (Note that this may even be a just an start and end activity, and just a single Case Model invocation)

 

There are two technical arguments to favor this option:

 - Cordys BOP4 allows to generate a Web Service on a BPM (and hence invoke it from externally)

 - BPM in Cordys BOP4 allows to use BAM. Selecting BPM as top-level process makes it easier to monitor the overall process state.

  

Obviously future features in both BPM and/or Case Management in may influence the advise above.  
 

 

Business Objects in BOP4

 

Introduction

Maintenance of Business Objects is required for most business processes.

MDM has an important role in enterprise data management and quality. It provides a “single point of truth” in the enterprise and allows connected  processes and service to use “the latest and greatest” and clean business data. MDM therefore is the preferred way to manage business data.

 

If MDM is not in place, and business objects are not maintained in backend-applications there are mainly two ways how to maintain and persist business objects:

1. A dedicated solution database
2. Object Templates

 

This blog will briefly zoom in into these options

 

Options

 

1: Dedicated database

 

This option can be compared to the traditional application development approach. From the logical data/object model a technical data model is developed. This model describes the database tables and the constraints like primary keys and relations. Based on this a database can be created, preferably separate from the Cordys database instance.

Cordys BOP4 provides the feature to query the metadata of a database, and to generate standard or customized web-services to access the database tabels. These web services should be deployed on a WS-AppServer container, before they can be used e.g. in XForms or BPM-s

 

Cordys allows developers to add business rules to these web servcies in order to perform additional validations and control the lifecycle of these business objects, represented by the database tables.

 

2. Object Templates

 

Object Templates are a construct that is offered by Cordys BOP4.

 

It allows to define data objects based on Schema fragments. The developer may specify constraints and validations, attach (execution-) rules en specify whether or not changes to the the object are subject to audits. Besides that it is possible to specify if the object should be persisted by Cordys. If that option is selected the instances of the Object Template will be stored inside the CoBOC database (the database used to store information about the BPM instances).

 

Object templates can be accessed using the following CoBOC WebServices:

 - UpdateXMLObject to insert, update or delete an object

 - GetObjectsByAttribute  to retrieve a object

 

Obviously these can could be embedded in a dedicated Webservice per separate Object

 

 

Best pick?

Both approaches above have pro-s and con-s. The "best" choice depends on project or architectural requirements:

 

Dedicated database

Pro-s

• Open database format (Data can be accessed easily from outside)
• More control on data (Separate database, custimizable WS services for data access)

Con-s

• Data layer should be maintained manually
• Separate WebServices are required

 

Object templates

Pro-s

• The data-tier remains hidden
• More productive because of less required steps and objects
• Audit possibilities

Con-s

• Cordys specific data format and Web Services to access data

• Data stored in CoBOC, so it may impact BPM execution performance

 

Depending on requirements (e.g. portability, testability, traceability, maintainability) one should decide on the way to go.

 

 Regards,

 

Harald van der weel

 

Testing Cordys BOP-4 using SOAPUI

 

SOAP-UI

Intro

In the previous project I have experienced soapUI as a low cost but very effective test tool for Cordys BOP-4 solutions. The abilities to invoke web services and test the result from outside the Cordys environment are very convenient. The features to define test scripts make it a very powerful tool to run automatic tests (e.g. for regression tests)

SOAP UI is build around uses an hierarchy of test objects: Each level contains 0 or more of the lower level objects:

1. Project

    2. Test-suite

        3. Test-case

            4. Test step.

Test steps

In the Test Steps the effective test actions are defined. Test actions may for instance be:

• 
  
  invoking a SOAP Service
• 
  
  validating the result (e.g. by checking database content)
• 
  
  Set properties
• 
  
  Pausing for an amount of time.
• 
  
  Executing an script

Properties

Properties contain values that can be used in the tests by a reference to the property. There are system and user defined properties. Properties are always related to one of the test-object types mentioned above and available to all object beneath it in the hierarchy.

A convenient application of properties are end-points –URLs for the web services that are tested. This way it is possible to test web services from another (DTAP) environment simply be changing this property.

Cordys Case

It was my intention to use SOAP UI to test my state-transition model, as described in my earlier BLOG-s. Testing it involves injecting numerous messages via SOAP, and check whether the correct state transition (or error) was invoked.

Initial approach

May first approach was to build test cases that consisted of multiples series of 3 steps:

• 
  
  A SOAP call to send a message
• 
  
  A wait step to allow Cordys to process the request
• 
  
  A validation step.

This approached worked: I was able to work out a number of scenario’s that covered most of the functionality that I wanted to test.

As my state-diagram was pretty extended (consisting of 17 states) I experiences a number of disadvantages:

• 
  
  Building was toilsome, defining similar steps for each state transition to test
• 
  
  The test scenario is hard to read, as only the title of each step is visible, and the vision is blurred because of sequences of 3 steps.
• 
  
  The scenarios were relatively hard to maintain, every time replacing 3 steps in case of a change.

Optimization 1

This led to a need of being able to define the sequence of the test offline, in a  file, were only the vital data would have to be specified.

This approach enables to maintain and store the test sequence in a text, making the scenario much more visible and way easier to modify (e.g. simply by inserting  some lines)

The solution I developed used the following capabilities of SOAP UI:

• 
  
  Run “Groovy” scripts (See http://groovy.codehaus.org/)
• 
  
  Invoke test cases from groovy
• 
  
  Read files from Groovy
• 
  
  Set property variables in Groovy

The main idea is simple:

• 
  
  Recognize and separate the repeating steps in the test case. Extract one instance of the repeating steps and isolate them in a separate test case (“TC1”)
• 
  
  Replace all required hard-coded values by a property
• 
  
  Define an external file having sets of lines containing values that may on the defined parameters
• 
  
  Build a "groovy" script that reads each line, sets the according parameters and executes  TC1

The file now consisted for each state-transition-to-test of a line of multiple properties in a fixed order. Each line was processed by the groovy script and assigned to the properties, before invoking the 3-step test sequence in TC1.

This approach worked fine.

The only disadvantage was that in the sequence certain properties remain the same for a long period of time (e.g. the correlation id of messages, which  is the same for all steps in the same sequence).

Optimization 2

The perfectionist in me decided to make the test script more fancy and general purpose, so I added the following options:

• 
  
   The option to specify the properties that are mapped upon the value lists.

            #plist,<property-1>,<property-2>, … <property-N>

• 
  
  The option to specify a property that is assigned with a unique value each time it is set

            #property,<property>,<value>

• 
  
  The option to specific a specific property until it is set again.

            #unique, <property>

• 
  
  The option to add line-comments

         //<Any comment>

An example of an input file is:

// MessageId is always unique

#unique, messageId

// For the first sequence we always use the same sequence id

#property,correlationId,cor01

// Each value line contains values for the properties messageType,Sender,Receiver

#plist,messageType,Sender,Receiver

MT01,Party1,Party2

MT02,Party2,Party1

MT01,Party1,Party3

MT02,Party3,Party1

MT03,Party3,Party4

MT04,Party4,Party3

// Now we addone property to the value lines: The correlation Id.

// The pre-set value is overruled now

#plist,correlationId,messageType,Sender,Receiver

Cor02, MT01,Party1,Party2

Cor02,MT02,Party2,Party1

Cor03,MT01,Party1,Party3

Cor03,MT02,Party3,Party1

To provide an idea what the code looked like I add a screendump of the code.

(As a Groovy novice I won't accept any liaibility ;) ) 

Regards,

Harald van der Weel

Haraldvanderweel@gmail.com

BOP-4 BAM imperfections

  

Where the BOP-4 product stack generally looks very complete, mature and well designed, I have to admit I am still a little less enthusiastic about the Business Activity Monitoring (BAM) part.

In our latest project we have implemented a quiet extensive BAM implementation. We managed to implement a pretty useable dashboard, that satisfied the project and customer needs. However the BAM functionality did not feel quiet as “ready” as e.g. BPM.

BAM intends to monitor KPI-s that map upon a values used in a BPM. To get a graphical representation of these values in you dashboard a number of steps (and Cordys objects) is required.

1. 
   
   First the BPM values to reflect in BAM have to be explicitly exposed using the “Message filter” tab of the BPM. A selection is to be made of all values in the message map in the BPM.
2. 
   
   Next a Process Monitoring Object (PMO) must be created. This object defines the actual BAM data to collect in the BAM database, and the events when they are collected. It takes the BPM process as the input of the PMO. A selection of exposed data values and possible events must be made, and it is possible to process or enrich the data e.g. by using XPath or web services.
3. 
   
   Business Measure (BM) objects are intended to define the actual information from the PMO-s in the BAM database, to reflect in the report. A Business Measure is actually a SQL query that results in a aggregated measurements (averages, totals, counts) reported over a specific “dimension” (e.g. a Location, Person or Unit). This dimension ends up in a GROUP BY clause and it is possible to use a number of parameters in the WHERE clause. A Business Measure will result in a UI dashboard component and a related web service to get the specific data out of the BAM database. The web service can be used autonomously as well.
4. 
   
   The BAM dashboard is effectively an XForm, to which one or more BM -graphical components are added. There they can be linked to each other or to standard BAM components like “Time Selectors”. This way the output from one component  can be used in a filter of another. This way some kind of down drilling can be implemented. For a little advanced functionality scripting is required.

Though the functionality described above behaves correctly, there are a number of inconveniences and hence potential enhancements. I will name a number that come to mind:

1. 
   
   The number of fields to use in a PMO is limited. A PMO just offers a limited amount of fields for each data type (15x String, 10x Numeric, 10x decimal and 10x Date-Time). Exceeding one of these amounts results in an error message.
2. 
   
   Unlike other Cordys objects a PMO object must be manually unpublished before a new version can be published. There is no proper warning or error system available to prevent mistakes.  
3. 
   
   The possibilities to manipulate the looks of the dashboard graphs are very limited.
4. 
   
   Not types of graphical components are standard supported by BAM (like the gauge component). It requires intensive scripting.
5. 
   
   It is cumbersome to implement drill down features. Relations between data is now specified on graph level, and the business measures should already be prepared for the specific need. It would be more convenient to define these relations on data level.
6. 
   
   BAM does not support runtime packages; only staging packages
7. 
   
   BAM does not offer any standard reporting options (unlike Cordys Process Factory – CPF)
8. 
   
   In an ideal world PMO-s and BM shouldn’t be required. It would be nice to be able to specify data to expose on BPM level, and the relations too. This should be sufficient to define and populate a BAM database. When designing the dashboard standard graphical components should allow to select any data that should be displayed.  

At the recent Cordial event I was able to ask the development team after expected developments of BAM. They confirmed these imperfections and announced a number of enhancements that will be implemented with FP1 (Optimistic planning: end of 2010).

It is not hard to imagine I am looking forward very much to FP1, which I expect to solve a number of this issues!

Personal highlights Cordial 2010

Event: Cordial 2010 event

Date: 14 and 15 September 2010

Location: Cordys, Putten, Netherlands

This year I had the luck to attend Cordial 2010.

This is a quick impression and listing of personal highlights of the event:

• A massive crowd of both customers and partners of Cordys attended the events from all over the world, showing that Cordys is very much alive.
• Being introduced to the new CEO and Jan Baan’s successor Per Johnsson
• A very dynamic and inspiring presentation by Manfred Kets de Vries about Leadership
• The opportunity to meet the product development team face-2-face to get an insight in the expected enhancements in both BOP-4 and Cordys Process Process Factory, that will be shipped in fp1. (FP1 ships “when ready”, expected before the end of 2010)

I.m.o. the most interesting:

• Multi browser support
• BAM enhancements (KPI-s in processes, Alignment of BAM)
• MDM
• Connections between CPF and BOP-4
• CPF mobile support
• Extended security features for CPF(e.g. SAML2.0)
• The insights provided by Forrester in the trends in Cloud and BPM, where the most imlessons are:
• There is a bright future for combined on-premises / cloud solutions, mostly depending on the level of confidentiality and commodity
• Data remains the biggest asset
• New business and payment models will arise based on the options that the cloud offers.
• Collaboration using the social networks
• 2 thrilling demonstrations by Matt Davies and Chris Hyde to realize a Cloud and a hybrid BOP-4 / Cloud solution in 9 minutes
• The statement that Cordys shifts from a product demonstration towards a marketing organization, where the product “the most important marketing tool”
• Meeting a lot of old “acquaintances” and many new interesting cordys related professionals.

More detailed info can be found on http://cordial.cordys.com

A more extensive report can be found on:

http://rogervdkimmenade.blogspot.com/2010/09/cordial-2010.html

Looking forward to Cordial 2011!

Event-state modeling in Cordys

 

Case

Recently we have been working on a solution that can be considered as a message broker / hub:

It’s functionality is mainly to receive, validate and route messages between parties in a specific business segment over different channels.

 

All messages have a specific type, syntax and semantics. Messages may be interrelated; e.g. one message may be a response to another message. A business wide protocol-standard describes the complete interrelation between message types, and specifies the possible sequence of messages in a flow: A so called "conversation". In a conversation any message of a specific type can be only followed by a message of one or more other specific types.

In case we want to monitor conversations we need to model the messaging protocol. This protocol can be considered as an event-state diagram: The last received (and expected!) message in the message sequence reflects the state, and the reception of a new message reflects an event.

 

Multiple concurrent conversations may be active at a time. This requires a correlation id to link the reception of a new message (see  the previous blog “Events in Cordys”) to an active conversation (or to start a new one, if there is no flow active for that particular correlation id).

 

Implementation

What does it take to implement such an event-state diagram in Cordys BOP-4?

 

The core of the solution consists of three processes, a decision table and a database-table (with access web services):

• Monitor-process: A process that describes the state model
• HandleMessage-process: A process that is started on new receives new messages and that triggers events.
• WaitForEvent-process: A process that actually waits for an event to be triggered by HandleMessage, where the message is passed
• StateTransitions-decision table: A decision table containing all valid state transions that can be triggered by a message. 
• InstanceCorrelation- database table: A database table for storage of the instance id and correlation id

 

Monitor-process

Monitor is the main process. It describes the overal state model. At runtimes each instance reflects a conversation and it effectively keeps the current state of the “conversation”. A Correlation Id is required to correlate the messages to this conversation. The Monitor process simply consists of a number of states that are interrelated: From a particular state an occuring event may cause a transition to one of one or more possible new states. A state is in fact nothing more than a sub-process call to WaitForEvent. WaitForEvent waits for an event and after a correct (expected) message is received it returns the resulting (new) state to the monitoring process. Only in case a state may have more than 1 possible resulting states the monitoring process should use a decision to effectuate the correct transition. All other logic is handles in the WaitForEvent (by invoking the StateTransitions-decision table)

 

Because the Monitor-process is started with the first message of a new conversation it has an input message, having the same format as the messages received by WaitForEvent.

 

     Example of a monitor process

 

WaitForEvent-process

Monitor specifies two main parameters to the WaitForEvent-sub process: The current state of Monitor and the Correlation Id that belongs to the conversation.

The first step in the WaitForEvent is to store its Instance Id in a “InstanceCorrelation” table, together with the Correlation Id before invoking the “Receive Message” step. (See also my previous blog “Events in Cordys”).

Once an event has been triggeredby the HandleMessage-process the WaitForEvent event checks whether it is an event that it expected, depending on the current state. The logic for this check is implemented in the decision table "StateTransitions", that I will explain below.

 

If a message is received that is not an expected successor of the current state, an exception occurs that ought to be handled. (In our case we simply log an “Unexpected Message” error). After this the process waits for a next event to be triggered.

If the message that is received is expected however, the decision table returns the “new state”, which is returned to the monitoring process. Before returning to the monitor however first the InstanceCorrelation table is cleaned up by removing the record that was inserted at the beginning of the process.

 

    "WaitForEvent"-process  

 

HandleMessage-process

HandleMessage is triggered at reception of a new message. It first uses the correlation id in the message to look up a corresponding Instance Id of a waiting process in the InstanceCorrelation table. If no such record is found, we use the decision table to check if the event is an start-event for a new conversation. If so, a new process instance of Monitor is started -via the invokation of a generated Web Service on Monitor-  and passing the message.

In case the Instance Id is found, a notification to WaitForEvent is sent using ExecuteProcess, passing Instance Id and the message (as explained in my previous blog “Events in Cordys”).

 

    "HandleMessage"-process

 

StateTransitions-decision table

The StateTransitions decision table contains all valid "current"state - message combinations, and the resulting new state. (Similar logic may also be implemented e.g. in a matrix, implemented in a database table, see also my blog "logic in Cordys").

 

Note that actually all logic on state transitions is effectively implemented in this decision table, and not in the Monitor process: The Monitor process is only intended to keep the current state of a conversation: Not to execute any transition logic, apart from navigating to the correct new state based on the out-come of the StateTransitions decision table!

 

    "StateTransitions"-decission table

 

Later I will zoom in on the reporting over the monitoring process.

 

 

Regards,

 

Harald van der Weel,

e-mail: haraldvanderweel@gmail.com