Archive for the ‘rants’ Category

Learn by Errors : Java + OSGi

Recently I worked on getting Apache Hive work inside an OSGi environment. While
not proving to be a proverbial piece of cake (software right?.. Why am I not
surprised? :)), it led me through an assortment of Java and OSGi errors. Here I
am listing some of them that bit me bit hard (no pun intended) so that I
thought of making a blog out them just for my own satisfaction.


I got this nastiness during initialization of one of OSGi service components.
The culprit was not immediately identifiable since the offending bundle was in
ACTIVE state. On the surface everything looked fine except for the fact the
Hive server which was supposed to start during the initialization of the
service component present in the bundle was not up and running. A quick ‘ls’ in
the OSGi console revealed the service component is in ‘unsatisfied’ state.
Finally a ‘comp’ revealed the root cause, the VerifyError.

The VerifyError can occur if the runtime dependency of a class is different to that
of the dependency that was used at compilation time. For example if the method
signatures have changed between the dependencies then this error would result.
This is nicely explained at [1] in the accepted answer. As it turned out
slightly different versions of a package had been exposed in two bundles causing
the Hive bundle to pick up a different version over the version that was in the
compilation environment. Proper OSGi versioning turned out to be the solution.


This error also cropped up under a similar circumstance where two packages were
present in the system. As [2] clearly explains, the reason for this in my case
was an interface being changed to an abstract class between the conflicting
package versions. Again the versioning helped to save the day.

java.lang.LinkageError : loader constraint violation in xxxx – blah …

Now this seems to be a famous error specially in OSGi enviornments. Main root
cause seems to be two classes loaded by different ClassLoaders coming in to
contact in a certain way. For example say Class A object accept a Class B object
as a method parameter. Class B is loaded by ClassLoader-A which also loads Class
A. But at the method invocation time how ever an object of Class B which has
been loaded by ClassLoader-B is passed as an argument to an object of Class A
which has been loaded by ClassLoader-A. Now the result would be a big fat
LinkageError with a very verbose error message.

The graph based class loadingstructure in OSGi makes it specially conducive to these kind of errors. In my case the culprit was a package which had been duplicated in two different
bundles and a particular class in that package loaded by the separate
ClassLoaders of each of the bundles coming in to contact via a third bundle
present in the system during a method call. So this was a case of not following
“import what you export” best practice [3] in OSGi. Doing so would help to
reduce the exposure of duplicated packages across bundles and help to maintain a
consistent class space for a given package. And so this turned out to be the
resolution for that in this case.

Package uses conflict: Import-Package: yyy; version=”x.x.x”

I had my fair share of this inconvenience thrown at my face every so often
during the exercise. There are two excellent posts [4],[5] exactly on this issue
at SpringSource which helped a lot. However let me summarize my learning on this
issue. Simply if a bundle is being exposed to two versions of the same package
through a direct import and via a uses constraint this error would come up. The
diagram best illustrates this situation.

The bundle A imports org.foo version 1.0.0 directly. However it also imports
bundle org.bar from bundle B. However as it turns out package org.bar also uses
org.foo package albeit it’s a different version (2.0.0) than that of the version
imported by bundle A. Now bundle A is directly wired to version 1.0.0 of org.foo
and also being exposed to the version 2.0.0 of org.foo due to the
import of org.bar which is using version 2.0.0 of org.foo. Now since a bundle
cannot be wired to different versions of the same package, a uses conflict would
come up with offending import org.bar as the root cause. (e.g: Package uses
conflict: Import-Package: org.bar; version=”0.0.0″). The solution would be to
change package import versions of org.bar in either bundle A or bundle B so that
both would be pointing to the same package version. Another excellent blog by
Neil Bartlett on this can be found at [6].


One of my friends at work came across this while trying to incorporate another
third party library in to our OSGi enviornment. JavaDocs goes on to say that
this gets “Thrown if the Java Virtual Machine cannot find an appropriate
native-language definition of a method declared native”. The offending library
was a linux .so (dynamically linked library) file which was not visible to
bundle ClassLoader at runtime. We were able to get it working by directly
including the library resource to the bundle ClassLoader. An earlier attempt on
setting this resource on TCCL (Thread Context ClassLoader) failed and this let
us to the realization that the TCCL is typically not the bundle class loader. A
good reading on TCCL under Equinox OSGi enviornment can be found at [7].


[1] http://stackoverflow.com/questions/100107/reasons-of-getting-a-java-lang-verifyerror
[2] http://stackoverflow.com/questions/1980452/what-causes-java-lang-incompatibleclasschangeerror
[3] http://blog.osgi.org/2007/04/importance-of-exporting-nd-importing.html
[4] http://blog.springsource.org/2008/10/20/understanding-the-osgi-uses-directive/
[5] http://blog.springsource.org/2008/11/22/diagnosing-osgi-uses-conflicts/

[6] http://njbartlett.name/2011/02/09/uses-constraints.html
[7] http://wiki.eclipse.org/ContextClassLoader_Enhancements

Read Full Post »

Nowadays we are constantly reminded of the virtues of being proactive or more

colloquially put “Being one step ahead of the game” when it comes to handling

our businesses whether it be a SME or a multi-national cooperation. Quickly

detecting or in some cases even predicting, trends in activities originating

within and outside the organization and streamlining business activities

accordingly may decide between death or life, of the business it’s said. The

often touted solution for this problem is implementing a proper monitoring

solution which would give the decision makers relevant information at correct

time. However most businesses are at a loss where to begin or how to properly

implement means of obtaining such insights. This is not surprising given that even

the buzzwords surrounding the monitoring concepts tend to be fuzzy.


Whoa.. That’s some pretty serious language (OK it is, at least to me :). I

consider my self linguistically challenged when it comes to English.). Well I

wanted to start with a serious note since we are dealing with a serious subject

here right??. :). Anyway this says a part of the story when it comes to

business monitoring domain. Sometimes the monitoring solutions forced on

businesses are just like this. Some serious mumbo jumbo with hundreds of bells

and whistles which most of us don’t care to understand. And of course some

times not capturing what really needs to be monitored in the business as well.

On top of that there is a buzz word soup surrounding the monitoring products

which each vendor come up with different interpretations according to their

implementations. Anyway let’s get some perspective on some business monitoring

key words according to the way I see it.


Let’s monitor some activities


“Business Activity Monitoring” is a term coined by Gartner Inc. which is

defined as the “The aggregation, analysis and presentation of real-time

information about activities inside organizations and involving customers and

partners”. However it can be seen the term is used in different contexts

meaning different things to different people specially when it comes vendor

solutions. The confusion tends be mostly around the fact on what can be

considered a business activity. For example for a business executive a sale of

a product will be a perfectly valid business activity which need to be

monitored while for tech op guy would need monitoring on the load of the server

hosting the sales application. I have heard some people say the latter does not

really falls under the term “Business Activity” since that level of monitoring

is of no importance to strategic decision-making of the business. But as far as

I believe it is no less important and should be a part of a comprehensive

monitoring solution since any high level decisions made would depend on the

smooth functioning of daily operations supported by a proper functioning

infrastructure (If servers are out sales numbers are going to get hurt. So will

the sales projections. Simple as that). It’s a matter of providing a suitable

view to each intended user group according to the type of monitoring

information they are interested in.


Anyway latter kind of monitoring may better fit under “Operational Intelligence”

category of which I will be talking about in a bit. In that sense we can think of

“Business Activity Monitoring” as a subset of “Business Monitoring” so that

this fulfills a part of the holistic view on approaching the monitoring problem

where all of what needs to be monitored in the business would come under

a comprehensive monitoring solution. This is one major point where the

vendors differ in their solutions. Some monitoring solutions focus on

a mixture of monitoring aspects and so their definition of BAM varies



BPM – A side kick??


Another difference between various BAM solutions is in the way they are

implemented. Some assume the presence of an existence of a Business Process

Management(BPM) solution, mostly from the same vendor and so the monitoring

solution is tightly coupled to that. While these kinds of solutions may provide

better integration in terms of the products in my opinion they lack the

flexibility to monitor most business scenarios where no business process

management solutions are in place. If the monitoring framework is generic

enough it’s a matter of putting required data capturing agents at points of

interest to capture and send data to the BAM solution which should be able to

correlate events from incoming events. However if there is a BPM solution

already present from the same vendor it should also be able to leverage that as

well. This way it would provide most flexibility in terms of monitoring



Key to success – KPI


Another term mentioned side by side with BAM is key performance

indicators(KPI). A BAM solution would monitor a set of predefined KPIs and make

sure that necessary actions are taken (it may be firing some alerts to relevant

parties or even automatically triggering some corrective action if possible)

when KPIs are not met with respect to their desired values. A good definition that

I found on what constitute a KPI is as follows.

Key Performance Indicators are quantifiable measurements that reflect the critical success factors of an organization. They will differ depending on the organization

So these are highly specific to the organization. Let me give a couple of simple examples on KPIs.

  1. For a retail store a valid KPI would be the percentage of days where daily sales revenue target was not met.
  2. For a delivery service a KPI would monitor the number of deliveries that went 10% overtime than their expected delivery times.
  3. A KPI for a call center would monitor the number of calls which took less than 2 minutes to resolve the problem.

Here we can identify the importance of the ability to customize the KPI

definitions according to the nature of the business. While properly identifying

the necessary KPIs should be done with involvement of the business management,

the BAM solution should facilitate defining business specific KPI definitions.


Intelligence in Operations – OI


Next comes the “Operational Intelligence” aspect of the business monitoring. It

is more or less similar to “Business Activity Monitoring” except that

“Operational Intelligence” is more oriented towards monitoring day today

business activities and geared to find issues in the system in real-time in

order for taking corrective actions. I believe technical operations monitoring

fits under this description since it involves the day-to-day aspect and the

required response times for any found issue should be more real-time. But

business matrices requiring close monitoring may well be included as part of

“Operational Intelligence” aspects as well. So here comes another word (“Real

time”) in to the mix which means different things to different people. There

are several levels of real-timeness as per products we see in the market. Some

position them as real-time monitoring solutions while others support near real

time monitoring and the boundary between these are blurry at best. As with any

thing else when it comes to monitoring, the required response time of the

solution depends on the context. A solution monitoring a business critical

application server may require response times within several seconds while a

low volume internal application server may not need such real-time monitoring.

A good rule of thumb should be that if it’s real-time expect a sub minute

response time while if it’s near real-time a couple of minutes lag at times may

be acceptable. Of course the vendors can stretch these either way according to

their implementations. So always try to read between the lines of marketing

terms to really see whether the solution a vendor is proposing really matches

what is required.


CEP to the rescue


Often the response times required by “Operational Intelligence” monitoring

necessitates the usage of a Complex Event Processing(CEP) solution underneath

which would monitor incoming event streams upon entry and trigger certain

actions when anomalies are detected. So the real-timeness of the product will

directly depend upon the performance characteristics and scalability of the CEP

solution used underneath.


Another type of Intelligence – BI


Next type of “Intelligence” a business want is “Business Intelligence”. Yeah I

know there are so many types of “Intelligences” floating around and this is one

of the important ones. This is geared towards finding trends in business

operations and market environment and coming up with predictions on the

business conditions. This is basically a historical data analysis which may

pull out data from a data ware house do some ETL operations and run some data

mining operations on data to gain new insights on business operations. So these

jobs are not real-time rather batch jobs which are scheduled at suitable



Ok. I think that’s enough for a day. Hope I made some sense out of the

monitoring buzz word fiesta. Hopefully this post would be good base for a next

post I plan to write some time soon in which I would outline some practical

experiences me and our team had while implementing a business monitoring
solution ourselves.

Read Full Post »

Been there. Done that. And suffered for that…

Programming is fun. But there are some other associated stuff we programmers blissfully skip or procrastinate because they are not so cool.

End result?…

Somebody is going to get hurt at the end of the day and that somebody may very well be a ourselves. So here are some stuff I have experienced and some of the stuff I my self have been guilty of doing and insights I gotten from them.

Good ol’ docs

It’s a well documented fact that documentation is.. hmm well.. let me think.. Good to have. Or is it important? Yep I know the feeling :). But it’s as things turn out, is some thing that needs to be done at the end of day. Who said we programmers do not have to toil for our food 🙂 right?. From a user’s perspective a feature without proper documentation is a near close to a feature which is not there at all. Say you developed a really neat feature and obviously you want people to try it out right? But what if they are not able to wrap their head around how to use it or they have to play a guessing game to get for trying to get it to work and in the process failing miserably? Now not only you have wasted their time but also have earned some bad karma. And yes, an intuitive user interface can go a long way to ease user’s pain but a good, to the point documentation sprinkled on top makes up a recipe that users can’t get enough of.

The Extra Mile

Say you developed this new cool feature. But in the hurry of pushing it off you cut some corners and left some manual step in the usage flow which better would have been done behind the curtains unbeknownst to the user. Now the user has to do this manual step every time he uses your feature which quickly becomes a pain specially if it turns out to be a heavily used feature. Optimize the UX. Cut unnecessary stuff from the user flow. Go that extra mile and users will thank you for that.

Mind your cycle

Go easy on your self. Make your development cycle quicker. Say you have some repetitive process to do in order to make the code you wrote to run in the test environment in order to check whether your feature/ fix is working correctly. Invest some time on automating this process, may be writing a handy script and it will help you to finish your work early and then go play :).

Let’s configure it

What if user want to fine tune the size of foo queue holding tasks for the bar thread pool of your program? Oh ok let’s make it configurable via UI then right? Or should we?? Too much configurability thrown at user’s face kills user experience. Do not force your users to fill in stuff which are better left with some sensible defaults every time they use your stuff. It may be that there is no need to configure every nook and corner of your program to make it work the way you want. Decide what should be in and what should be out. Better yet the middle ground to come would be to provide those configurations in an optional advanced configuration section with some sensible defaults which if user sees fit will go and change. And also remember to document them clearly as well so that user knows better when configuring those.

Nasty API docs

Wrong API docs are worse than having no API docs at all. It really happened to me once with a JMS API not working as published in its API docs. And I thought my thread programming was causing it. Took some considerable amount of hairs pulled to figure out the fault is with the API. Since my assumptions of the API derived from the docs were wrong, so was my program. Specially be mindful when you are changing an existing API implementation whether the assumptions and results returned in certain conditions specified in API docs still holds. If not change the docs accordingly.

Carpenters wanted..

Manage your broken windows. You may have to cut some corners and pull out some hacks due to time or release pressures. It’s OK as long as you know what your broken windows are and you intend to repair them the first chance you get. Leave some reminders and attend to them when you get the chance.

Love thy code.

Show that you care so that others will care. If you maintain your code in a good condition the other people taking over or contributing to your code will tend to care about maintaining it the same way. This is specially important in open source settings where you will not be the only one mending a piece of code written by you at the end of the day.

So there goes my list of tidbits on programming for the better. Pretty much regulation and common sense stuff which does not warrant a mention you might say. But as mentioned in the beginning I have been there. Done that. And have paid for that :).  And we keep doing that as well. So I hope this will post serve as a reminder for me at least, when I am on verge of doing some nasty thing next time around :). Anyway this is just my 2 cents. Please holler if you beg to differ.

Read Full Post »