-- Posted by Carole-Ann
The more I read on Complex Event processing (CEP), the more I believe that people are confused or people are trying to confuse end users. It reminds me of the Business Process Management (BPM) / Business Rules Management Systems (BRMS) market a few years ago: BPM vendors said that the rules could be modeled in the process flow and BRMS vendors said that processes could be implemented as ruleflows... It took a few years to realize that each technology had a sweet spot and a real purpose. They were complementary.
Now looking at the CEP Blog from Tim Bass, comments posted here (look at the comments) and there on the blogosphere, it is obvious that the role of CEP is unclear. I am not disputing the value of the technology but merely begging the industry to keep things simple and straight-forward.
I will provide a simplistic prospective here. You may argue that it is too simplistic but the more we discuss it, the stronger I feel about it.
Simple CEP / BPM / BRMS model
In our BPM / BRMS world, we came to the conclusion that we needed two types of technology: BRMS tools are used to make business decisions that are owned by the business users, BPM tools are used to carry out a process, executing on the decisions made by the BRMS tool. Can you make a decision as part of a flow? Yes. Is it good practice to maintain them there? No. Can you automate a flow in a rules service? Yes. Is it good practice to do so? No. I assume we are all in agreement so far.
When CEP comes into the picture, we feel compelled to question this model. Is CEP the right technology for processes? Is it the right technology for decisions? Some go as far as questioning whether CEP should replace Business Intelligence (BI) and/or Business Activity Monitoring (BAM).
At that point in time, being a product manager by trade, I get back to the problem definition. what are we trying to accomplish? Complex Event Processing... This is all about processing events. When is it valuable? When you receive a flurry of events that cannot be handled efficiently using other traditional methods. This is actually what happens most of the time in the real world: lots of things happen all the time. Having a module that processes these events real-time would have tremendous value. It would help dictate what kind of processes need to be executed and what types of decisions need to be made.
Do we have to worry about an event overload during the execution of a process? Typically no. A single "document" is being treated by the process. It may access more data when needed but it would not typically get flooded by other external events. Do we have to worry about an event overload during the execution of a decision step (business rules)? Typically no. Business rules apply to a set of business objects. Additional data may be accessed as needed of course. I qualified my answers with "typically" because there may be times when many events could happens and influence the processing of a flow or a decision. Would all those "exceptional paths" belong to the process or business rules. If limited they may but if we are talking about tons of events we may have to deal with, it is more likely we would want to use a module for processing those events, correlating and filtering them. Isn't that the forte of CEP?
The CEP / BPM / BRMS world seems fairly simple after all:
- CEP module receives or intercepts a flurry of events and processes them with the objective of figuring out what those events are relevant for; it triggers the appropriate business processes or decision services
- BPM module receives the request for a given process to be applied to a higher level entity (an application, a document...); it automates the steps defined in the business process
- BRMS module is invoked with a given context to apply business rules; it makes a business decision
Carlos Serrano-Morales wrote a great post on that subject.
Compared to the Human Body
My background is in life sciences, a source of inspiration. Please indulge me here while I digress in a biological comparison.
I picture CEP as being the sensors. The skin for example receives tons of inputs (events) on temperature, moisture level, etc. It is not responsible for making a business decision although it happens that it makes "some level" of decision. Typically raised hair and goosebumps may result from the sensors detecting cold weather.
BPM would be the nerves carrying the information to the brain and executing on its orders.
As you probably guessed, BRMS would be represented here by the brain, capable of making a higher level decision based on the contextual information correlated by the CEP module (it is cold) but also based on business strategies (use a blanket at night or turn up the heater if that's not enough; get a sweater during the day).
Assuming that we decide here to use a blanket, the BPM system will be responsible for carrying out this order, get the complete system to get up, walk to the linen closet and grab the most appropriate blanket.
CEP may intervene any time to provide input that may alter the process or the decision making. If the skin feels cold *and* the eye sees that an ice cube was dropped on the skin, the information would be provided again using the appropriate process / decision service.
I want to emphasize that, although CEP was making some level of decision, it is not built or informed to make the business decision. There is a clear separation of role.
What makes this so complicated?
I think we are all guilty within the industry. We are used to use our hammer on any kind of nail or screw in order to feel as ubiquitous as possible. We hate to see technology used for a specific problem, by fear that it loses appeal. I actually think that success comes from the acknowledgment that a tool has a purpose, and learning how to use it appropriately. If we were still trying to convince the world that BRMS is a credible alternative to BPM (and vice versa), we may still be involved in half baked projects (product management recommended book: Angel Customers and Demon Customers). We would not have been able to grow into a recognized component of the agile architecture. CEP needs to follow this lead.
I am afraid that we are also getting confused by our own algorithms. Some Business Rules Engines (BRE) are toying with the usage of Rete to CEP problems. This should be irrelevant to the role of CEP. If in the end we use Rete, Parallel Rete or any other algorithm to implement CEP, it should not matter to the role of CEP in the first place. We must admit that our own Charles Forgy has presented his findings at the latest October Rules Fest. The intention was not to create confusion at the usage level - mea culpa if we did. We believe that it would be a different Decision Automation module, part of Decision Management of course but not necessarily part of BRMS.
Complication comes from other sources as well. Because CEP deals with events in time, we sometimes associate it with Business Intelligence (BI), thinking that because it can process events real-time, it should be able to process time-aware query (how many A events happened after a B event). When combining both aspects (real-time and BI), you could build a Business Activity Monitoring (BAM) system but this does not mean that CEP is BAM either...
Well, as passionate as everyone is on that subject, I am expecting a few heated comments here... This is healthy as it will help the whole industry as well as the end users to come to a conclusion on what CEP is and what it is good for. Go ahead!
