Archive for March, 2012

Recently I wanted to set up a remote desktop sharing session from home pc to my laptop. While going through the set up guide I came across ssh tunneling. Even though there are many articles on the subject still it took me a considerable amount of googling, some experimenting and couple of Wireshark sessions to grasp what’s going under the hood. Most of the guides were incomplete in terms of explaining the concept which left me desiring for a good article on the subject with some explanatory illustrations. So I decided to write it my self. So here goes…


A SSH tunnel consists of an encrypted tunnel created through a SSH protocol
connection. A SSH tunnel can be used to transfer unencrypted traffic over a
network through an encrypted channel. For example we can use a ssh tunnel to
securely transfer files between a FTP server and a client even though the FTP
protocol itself is not encrypted. SSH tunnels also provide a means to bypass firewalls that prohibits or filter certain internet services. For example an organization will block certain sites using their proxy filter. But users may not wish to have their web traffic
monitored or blocked by the organization proxy filter. If users can connect to
an external SSH server, they can create a SSH tunnel to forward a given port on
their local machine to port 80 on remote web-server via the external SSH
server. I will describe this scenario in detail in a little while.

To set up a SSH tunnel a given port of one machine needs to be forwarded (of
which I am going to talk about in a little while) to a port in the other
machine which will be the other end of the tunnel. Once the SSH tunnel has been
established, the user can connect to earlier specified port at first machine to
access the network service.

Port Forwarding

SSH tunnels can be created in several ways using different kinds of port forwarding
mechanisms. Ports can be forwarded in three ways.

  1. Local port forwarding
  2. Remote port forwarding
  3. Dynamic port forwarding

I didn’t explain what port forwarding is. I found Wikipedia’s definition more explanatory.

Port forwarding or port mapping is a name given to the combined technique of

  1. translating the address and/or port number of a packet to a new destination
  2. possibly accepting such packet(s) in a packet filter(firewall)
  3. forwarding the packet according to the routing table.

Here the first technique will be used in creating an SSH tunnel. When a client application connects to the local port (local endpoint) of the SSH tunnel and transfer data these data will be forwarded to the remote end by translating the host and port values to that of the remote end of the channel.

So with that let’s see how SSH tunnels can be created using forwarded ports with an examples.

Tunnelling with Local port forwarding

Let’s say that yahoo.com is being blocked using a proxy filter in the University.
(For the sake of this example. :). Cannot think any valid reason why yahoo would be blocked). A SSH tunnel can be used to bypass this restriction. Let’s name my machine at the university as ‘work’ and my home machine as ‘home’. ‘home’ needs to have a public IP for this to work. And I am running a SSH server on my home machine. Following diagram illustrates the scenario.

To create the SSH tunnel execute following from ‘work’ machine.

ssh -L 9001:yahoo.com:80 home

The ‘L’ switch indicates that a local port forward is need to be created. The switch syntax is as follows.

-L <local-port-to-listen>:<remote-host>:<remote-port>

Now the SSH client at ‘work’ will connect to SSH server running at ‘home’ (usually running at port 22) binding port 9001 of ‘work’ to listen for local requests thus creating a SSH tunnel between ‘home’ and ‘work’. At the ‘home’ end it will create a connection to ‘yahoo.com’ at port 80. So ‘work’ doesn’t need to know how to connect to yahoo.com. Only ‘home’ needs to worry about that. The channel between ‘work’ and ‘home’ will be encrypted while the connection between ‘home’ and ‘yahoo.com’ will be unencrypted.

Now it is possible to browse yahoo.com by visiting http://localhost:9001 in the web browser at ‘work’ computer. The ‘home’ computer will act as a gateway which would accept requests from ‘work’ machine and fetch data and tunnelling it back. So the syntax of the full command would be as follows.

ssh -L <local-port-to-listen>:<remote-host>:<remote-port> <gateway>

The image below describes the scenario.

Here the ‘host’ to ‘yahoo.com’ connection is only made when browser makes the
request not at the tunnel setup time.

It is also possible to specify a port in the ‘home’ computer itself instead of
connecting to an external host. This is useful if I were to set up a VNC session
between ‘work’ and ‘home’. Then the command line would be as follows.

ssh -L 5900:localhost:5900 home (Executed from 'work')

So here what does localhost refer to? Is it the ‘work’ since the command line is executed from ‘work’? Turns out that it is not. As explained earlier is relative to the gateway (‘home’ in this case) , not the machine from where the tunnel is initiated. So this will make a connection to port 5900 of the ‘home’ computer where the VNC client would be listening in.

The created tunnel can be used to transfer all kinds of data not limited to web browsing sessions. We can also tunnel SSH sessions from this as well. Let’s assume there is another computer (‘banned’) to which we need to SSH from within University but the SSH access is being blocked. It is possible to tunnel a SSH session to this host using a local port forward. The setup would look like this.

As can be seen now the transferred data between ‘work’ and ‘banned’ are encrypted end to end. For this we need to create a local port forward as follows.

ssh -L 9001:banned:22 home

Now we need to create a SSH session to local port 9001 from where the session
will get tunneled to ‘banned’ via ‘home’ computer.

ssh -p 9001 localhost

With that let’s move on to next type of SSH tunnelling method, reverse tunnelling.

Reverse Tunnelling with remote port forwarding

Let’s say it is required to connect to an internal university website from home.
The university firewall is blocking all incoming traffic. How can we connect from ‘home’ to internal network so that we can browse the internal site? A VPN setup is a good candidate here. However for this example let’s assume we don’t have this facility. Enter SSH reverse tunnelling..

As in the earlier case we will initiate the tunnel from ‘work’ computer behind the firewall. This is possible since only incoming traffic is blocking and outgoing traffic is allowed. However instead of the earlier case the client will now be at the ‘home’ computer. Instead of -L option we now define -R which specifies
a reverse tunnel need to be created.

ssh -R 9001:intra-site.com:80 home (Executed from 'work')

Once executed the SSH client at ‘work’ will connect to SSH server running at home creating a SSH channel. Then the server will bind port 9001 on ‘home’ machine to listen for incoming requests which would subsequently be routed through the created SSH channel between ‘home’ and ‘work’. Now it’s possible to browse the internal site
by visiting http://localhost:9001 in ‘home’ web browser. The ‘work’ will then create a connection to intra-site and relay back the response to ‘home’ via the created SSH channel.

As nice all of these would be still you need to create another tunnel if you need to connect to another site in both cases. Wouldn’t it be nice if it is possible to proxy traffic to any site using the SSH channel created? That’s what dynamic port forwarding is all about.

Dynamic Port Forwarding

Dynamic port forwarding allows to configure one local port for tunnelling data to all remote destinations. However to utilize this the client application connecting to local port should send their traffic using the SOCKS protocol. At the client side of the tunnel a SOCKS proxy would be created and the application (eg. browser) uses the SOCKS protocol to specify where the traffic should be sent when it leaves the other end of the ssh tunnel.

ssh -D 9001 home (Executed from 'work')

Here SSH will create a SOCKS proxy listening in for connections at local port
9001 and upon receiving a request would route the traffic via SSH channel
created between ‘work’ and ‘home’. For this it is required to configure the
browser to point to the SOCKS proxy at port 9001 at localhost.

Read Full Post »

Nowadays we are constantly reminded of the virtues of being proactive or more

colloquially put “Being one step ahead of the game” when it comes to handling

our businesses whether it be a SME or a multi-national cooperation. Quickly

detecting or in some cases even predicting, trends in activities originating

within and outside the organization and streamlining business activities

accordingly may decide between death or life, of the business it’s said. The

often touted solution for this problem is implementing a proper monitoring

solution which would give the decision makers relevant information at correct

time. However most businesses are at a loss where to begin or how to properly

implement means of obtaining such insights. This is not surprising given that even

the buzzwords surrounding the monitoring concepts tend to be fuzzy.


Whoa.. That’s some pretty serious language (OK it is, at least to me :). I

consider my self linguistically challenged when it comes to English.). Well I

wanted to start with a serious note since we are dealing with a serious subject

here right??. :). Anyway this says a part of the story when it comes to

business monitoring domain. Sometimes the monitoring solutions forced on

businesses are just like this. Some serious mumbo jumbo with hundreds of bells

and whistles which most of us don’t care to understand. And of course some

times not capturing what really needs to be monitored in the business as well.

On top of that there is a buzz word soup surrounding the monitoring products

which each vendor come up with different interpretations according to their

implementations. Anyway let’s get some perspective on some business monitoring

key words according to the way I see it.


Let’s monitor some activities


“Business Activity Monitoring” is a term coined by Gartner Inc. which is

defined as the “The aggregation, analysis and presentation of real-time

information about activities inside organizations and involving customers and

partners”. However it can be seen the term is used in different contexts

meaning different things to different people specially when it comes vendor

solutions. The confusion tends be mostly around the fact on what can be

considered a business activity. For example for a business executive a sale of

a product will be a perfectly valid business activity which need to be

monitored while for tech op guy would need monitoring on the load of the server

hosting the sales application. I have heard some people say the latter does not

really falls under the term “Business Activity” since that level of monitoring

is of no importance to strategic decision-making of the business. But as far as

I believe it is no less important and should be a part of a comprehensive

monitoring solution since any high level decisions made would depend on the

smooth functioning of daily operations supported by a proper functioning

infrastructure (If servers are out sales numbers are going to get hurt. So will

the sales projections. Simple as that). It’s a matter of providing a suitable

view to each intended user group according to the type of monitoring

information they are interested in.


Anyway latter kind of monitoring may better fit under “Operational Intelligence”

category of which I will be talking about in a bit. In that sense we can think of

“Business Activity Monitoring” as a subset of “Business Monitoring” so that

this fulfills a part of the holistic view on approaching the monitoring problem

where all of what needs to be monitored in the business would come under

a comprehensive monitoring solution. This is one major point where the

vendors differ in their solutions. Some monitoring solutions focus on

a mixture of monitoring aspects and so their definition of BAM varies



BPM – A side kick??


Another difference between various BAM solutions is in the way they are

implemented. Some assume the presence of an existence of a Business Process

Management(BPM) solution, mostly from the same vendor and so the monitoring

solution is tightly coupled to that. While these kinds of solutions may provide

better integration in terms of the products in my opinion they lack the

flexibility to monitor most business scenarios where no business process

management solutions are in place. If the monitoring framework is generic

enough it’s a matter of putting required data capturing agents at points of

interest to capture and send data to the BAM solution which should be able to

correlate events from incoming events. However if there is a BPM solution

already present from the same vendor it should also be able to leverage that as

well. This way it would provide most flexibility in terms of monitoring



Key to success – KPI


Another term mentioned side by side with BAM is key performance

indicators(KPI). A BAM solution would monitor a set of predefined KPIs and make

sure that necessary actions are taken (it may be firing some alerts to relevant

parties or even automatically triggering some corrective action if possible)

when KPIs are not met with respect to their desired values. A good definition that

I found on what constitute a KPI is as follows.

Key Performance Indicators are quantifiable measurements that reflect the critical success factors of an organization. They will differ depending on the organization

So these are highly specific to the organization. Let me give a couple of simple examples on KPIs.

  1. For a retail store a valid KPI would be the percentage of days where daily sales revenue target was not met.
  2. For a delivery service a KPI would monitor the number of deliveries that went 10% overtime than their expected delivery times.
  3. A KPI for a call center would monitor the number of calls which took less than 2 minutes to resolve the problem.

Here we can identify the importance of the ability to customize the KPI

definitions according to the nature of the business. While properly identifying

the necessary KPIs should be done with involvement of the business management,

the BAM solution should facilitate defining business specific KPI definitions.


Intelligence in Operations – OI


Next comes the “Operational Intelligence” aspect of the business monitoring. It

is more or less similar to “Business Activity Monitoring” except that

“Operational Intelligence” is more oriented towards monitoring day today

business activities and geared to find issues in the system in real-time in

order for taking corrective actions. I believe technical operations monitoring

fits under this description since it involves the day-to-day aspect and the

required response times for any found issue should be more real-time. But

business matrices requiring close monitoring may well be included as part of

“Operational Intelligence” aspects as well. So here comes another word (“Real

time”) in to the mix which means different things to different people. There

are several levels of real-timeness as per products we see in the market. Some

position them as real-time monitoring solutions while others support near real

time monitoring and the boundary between these are blurry at best. As with any

thing else when it comes to monitoring, the required response time of the

solution depends on the context. A solution monitoring a business critical

application server may require response times within several seconds while a

low volume internal application server may not need such real-time monitoring.

A good rule of thumb should be that if it’s real-time expect a sub minute

response time while if it’s near real-time a couple of minutes lag at times may

be acceptable. Of course the vendors can stretch these either way according to

their implementations. So always try to read between the lines of marketing

terms to really see whether the solution a vendor is proposing really matches

what is required.


CEP to the rescue


Often the response times required by “Operational Intelligence” monitoring

necessitates the usage of a Complex Event Processing(CEP) solution underneath

which would monitor incoming event streams upon entry and trigger certain

actions when anomalies are detected. So the real-timeness of the product will

directly depend upon the performance characteristics and scalability of the CEP

solution used underneath.


Another type of Intelligence – BI


Next type of “Intelligence” a business want is “Business Intelligence”. Yeah I

know there are so many types of “Intelligences” floating around and this is one

of the important ones. This is geared towards finding trends in business

operations and market environment and coming up with predictions on the

business conditions. This is basically a historical data analysis which may

pull out data from a data ware house do some ETL operations and run some data

mining operations on data to gain new insights on business operations. So these

jobs are not real-time rather batch jobs which are scheduled at suitable



Ok. I think that’s enough for a day. Hope I made some sense out of the

monitoring buzz word fiesta. Hopefully this post would be good base for a next

post I plan to write some time soon in which I would outline some practical

experiences me and our team had while implementing a business monitoring
solution ourselves.

Read Full Post »