We needed a really light weight ESB and we were evaluating Mule, ServiceMix and finally decided to write our own Spring based service bus. And it ended up working great for us. At that time, the Spring Integration was just being developed. I was looking into the Spring Integration and looks like it is very endpoint / channel based. We wrote a component based service bus, that would just intercept, transform and invoke a service. Spring Integration is neat and flexible, but I’m not sure how the polling channel model would perform. Also we have services that communicate synchronously, but I’m not sure if there is a way to define a request/response type of architecture with Spring Integration. Looks like it is necessary to implement an input and output channel for every messaging pattern like transformer, router, etc.

• Don’t want to learn ant scripts
• Tired of writing ant scripts
• Don’t want to maintain XML files

Here’s a pure Java build API. You just have to download the JAR, add it to your projects and write the entire build configuration in Java.

ProtoJ is a pure java build, installation and deployment library that hands control over to you. There’s no third-party ProtoJ application directory to maintain, no environment variables to set up and absolutely no other required software. Simply add the ProtoJ library to your project and get ready to experience the joy of a friction-free development cycle.

More information at – http://code.google.com/p/protoj/

So what do you really look for in an interface engine? There is more to an IE than just being a pass-through of messages.

Some good features of an interface engine –

1) Store and forward capability
2) Auto-acknowledgement
3) Guaranteed message redelivery
4) Rollback and retry capabilities
5) Reliable messaging, Once only delivery
6) Message Ordering, Error-free, Sequential
7) Disconnected Operation
Message Logging
9) Message Security
10) Consistent Routing
11) Message Monitoring
12) Exception/Error Escalation
13) Uptime Quantification
14) Persistent Messaging
15) Timely Delivery
16) Failover
17) Fault Tolerance
18) Load Balancing
19) MTTF (Mean time to Repair), MTBF (Mean time between Failure) management
20) Disk I/O performance monitoring
21) Continuous Availability
22) Transaction-aware distribution
23) Loosely coupled asynchronous messaging framework
24) Dead letter / exception mechanism
25) Messaging logging and auditing
26) Message Parallel Processing
27) Exception & Error Handling / Escalation
28) Deployment Infrastructure
29) Message Correlation
30) Message Routing Mechanism
31) Test messaging (ability to check if services are operational)
32) Reliable messaging
33) Store and Forward Configuration
34) Concurrency Management
35) Aggregation Locking Mechanisms
36) Response Timeout Configurations
37) Message Payload Optimizations
38) Conversational Capability
39) Component Failover Management
40) Cache management
41) Database Availability
42) Redundancy
43) Error escalation
44) Protocol independence
45) Adapter extensions
46) Best practices & reference architectures, implementation of enterprise integration patterns

Most of the interface engines range from 50,000 – 150,000$ / CPU license. HL7 IE’s are Vitria Businessware, Interfaceware, eLink, Oracle SOA Suite, Orion Health Rhapsody, Cloverleaf, OPENLink, Sybase e-Biz Impact, ConnectMate, BizTalk etc. Others are generic IE’s where we would need to buy/develop a HL7 adapter which include BEA, Websphere integration, Tibco, WebMethods and Ensemble.

Some IE’s require development licenses as well. Open source HL7 IE’s are SeeBeyond eGate and Mirth. Mirth is a great open source product as a HL7 interface engine used by various government entities.

We have had numerous arguments with your DBA’s about normalization.

We had the following issues

1. Normalizing 2 different tables used by entirely different services (Audit, Business) because they have the same field name
2. Normalizing immutable audit table

The following links might help you in defending your architecture -

Read the rest of this entry »

We have a cache synchronizer which uses the SCN_TO_TIMESTAMP(ORW_ROWSCN) in one of our queries to check for the recently modified records and updates the cache. The problem with the SCN_TO_TIMESTAMP(ORA_ROWSCN) is that the SCN’s are only valid for 5 days.

So a query like SELECT SCN_TO_TIMESTAMP(MAX(ORA_ROWSCN)) from SOME_TEST_TABLE; will return correctly if the recent ORA_ROWSCN was created in the last 5 days. Or you would get the following error.

java.sql.SQLException: [BEA][Oracle JDBC Driver][Oracle]ORA-08181: specified number is not a valid system change number
ORA-06512: at "SYS.SCN_TO_TIMESTAMP

Luckily we had a MODIFIED_DATE column that our application updates whenever a record is modified. So we ended up using that.

We ended up using the JDK timer, since

• it was easy to use
• calling an EJB
• instantiated once (quartz instantiates the job everytime)

One nice thing about the JDK timer, even if task is scheduled at a certain time, it would not execute if an existing task is running. Also its thread-safe, since only one task is executing at a time.

More information at -
Task Scheduling – Spring Framework Documentation

Do you have programmers complaining that the code is too complex or Spring makes everything hard to understand? When you look at the XML and code, it might be all two dimensional. It takes time for developers who are new to Spring to understand the framework. But once they do, its like a completely different three dimensional view of a stereogram. It makes programming much easier and results in simple and maintainable code. Dependency Injection is probably one of the best patterns I’ve seen in the last few years. It’s just an amazing peice of technology and I’m glad that I spent the time working on it. The understanding just happens like a flash, all of a sudden you start understanding the way things work. So if you are not getting the technology, be patient and write a few sample applications and you would see it too.

More information on Spring framework at -
http://www.springframework.org

Unfortunately our budget for a load balancer hardware never got approved. So we had to think of ways to do software load balancing. We are using the HttpClusterServlet for our development environments. BEA recommends that it is best not to use this in production. We decided to use Apache proxy for proxying requests to the server.

More information here –
Weblogic Apache HTTP Plugin – http://download.oracle.com/docs/cd/E11035_01/wls100/plugins/apache.html
Apache mod_proxy – http://httpd.apache.org/docs/2.0/mod/mod_proxy.html

We use Weblogic JMS for processing requests asynchronously. We also have an audit log, which writes the request/responses to the database. On a rollback scenario, we would like to audit the exception information. With the container managed transaction scenario, the JMS and JDBC transactions participate in a two phase commit. If the database is down, then the JMS message is rolled back and retried at some later point of time, as configured in the redelivery delay parameters. Now the exception information is logged in the console, but we could not get this into our audit logs because if we catch the exception then the database rollback would not happen. If we catch and re-throw then we could do a non-transactional write of the audit log or our exception handling could be part of a new transaction. But if the exception handling logic writes to the database, and we wish to rollback and retry both transactions, it would not be possible. The getJMSRedelivered() in the JMS message can help figure out if the message was redelivered. If you are using Weblogic, JMS_BEA_RedeliveryLimit and JMS_BEA_DeliveryTime message headers can help determine some information about the redelivery of the message. But it still would not solve the redelivery audit issue.

We use Ehcache and Hibernate in our product. We also use the custom cache implementations for loading frequently used objects into memory. One problem with the cache implementation is that it does not support transcational caching. We tried using OsCache, but it was quite limiting in its configuration. We decided to extend EhCache to support a transactional distributed cache. Since our application runs within a J2EE server, we decided to use JMS. We faced a problem initially, when a transaction would get rolled back, but the element would be inserted into the cache. We were using the cache.put() api, which updated the local cache even on a rollback. Instead of using cache.put(), we started manually notifying the cache listeners. We also created our own Jms cache peers, peer providers,  peer replicators and peer provider factories.
    public static void notifyCacheListenerRemove(final Cache cache,
            final Object key) {
        if (ValidationUtil.isEmpty(cache)) {
            return;
        }

        if (ValidationUtil.isNotEmpty(cache.getCacheEventNotificationService())
                && ValidationUtil.isNotEmpty(cache
                        .getCacheEventNotificationService()
                        .getCacheEventListeners())
                && ValidationUtil.isNotEmpty(cache
                        .getCacheEventNotificationService()
                        .getCacheEventListeners().size() > 0)) {
            final Iterator<CacheReplicator> it = cache
                    .getCacheEventNotificationService()
                    .getCacheEventListeners().iterator();
            while (it.hasNext()) {
                final CacheReplicator replicator = it.next();
                if (ValidationUtil.isEmpty(replicator)) {
                    continue;
                }
                replicator.notifyElementPut(cache, element);
            }
        } else {
            cache.put(element);
        }
    }