Software Architecture – How to approach ( A simplistic view) – Part 1

While architecting software systems all we have to do is what will change and what will be static. What can be enhanced and what can be removed. I find this principle at the core of Design Patterns.

Whatever can change (and this question needs to be answered on regular basis) one needs to put it on a process space. If these processes are well defined then you can scale them, maintain them, enhance them independently. Each component exposes some functionality what we can call “Service”. Most of the time developers these days relate Service with S in SOA and when it is done they immediately assume it is a “Web Service”. This is wrong understanding from my point of view.

Any piece of software that exposes some functionality can be termed as a Service. It is a generic term and Web Service is just one way of creating and consuming services. More about this in later articles.

Once we answer above questions (which will require deep domain thinking) we can just start developing small systems (Divide and conquer eh h?) and wire them together. Once this wiring infrastructure is already in place a new developer can create a new module when a new functionality is needed and you can be at peace that even if he messes up you can just stop his system.

Easier said then done. Software Architecture had to be developed scientifically and bigger picture needs to be kept in mind. Though there can be thousands of factors which go in determining how architecture for application should be designed but in this blog we will focus on how things can be simplified.

Before we start looking at these principles we need to accept that today lot of design decisions are being influenced by what Twitter/Facebook/LinkedIn are doing.

This is not wrong but it influences our approach. These websites are not built in a day and it takes years to reach that stage. When that time comes you will automatically know what to do.

For now we focus on systematic approach.

Key guidelines

  1. Separation of concerns
  2. Single Responsibility principle.
  3. Don’t repeat yourself (DRY)

There can be many more but these 3 rules in my opinion are the guiding light for a maintainable application.

1 & 2 are complimentary. 3rd principle is made more popular by frameworks like ROR but concept is simple “do not write same kind of code again and again”.

Use shared library, create a web service, deploy in different container instance etc. etc.

We have to remember that discipline is the most important key factor in engineering and software development is know different.

If we follow basics there is very little room for error we can leave. In case of any confusion we fall back to basics again. In practice you will find many people preaching about this but very few following these principles as it comes by practice.

1 & 2 are self descriptive for a seasoned programmer. You try to think as small as possible and expose that functionality. e.g. In an E-commerce site Payment Gateway module has got nothing to do with OrderProcessing module. Though there can be dependency but their logic is totally independent.

So there is a strong reason for separating them out.

If we think small there can be many instances when we can make small modules. Don’t be afraid how modules will work together for now. We will talk about that later.

Software Architecture – How to approach ( A simplistic view) – Part 1

Apache Camel Sample Project

In this very simple project we will show how camel actually works. We will try to download tweets from our own account and print them in Console.

We can write them to a file if we like. We will also show how polling works. I have created a sample maven project which you can download here

https://github.com/singhmarut/camelSample

Camel’s fundamental unit is route. You define routes and once correctly configured they start doing their work of integration.

A route can only exist between two end points. So what we want to do here is download tweets from “Twitter Account” and “show them in console”. We need a route which actually does this.

So we define a route by extending RouteBuilder class

package org.marut.camel;

import org.apache.camel.builder.RouteBuilder;

//Create a route in Camel Context. A route is a new thread. A Java DSL defines route

//Our objective is to create a route which reads tweets from your twitter account every 1 minute interval and

//sends them to console

public class TwitterRoute extends RouteBuilder {

static final String consumerKey  = “<Your Key>”;

static final String consumerSecret = “<Your secret>”;

static final String accessToken =”<Your access token>”;

static final String accessSecret =”<Your access secret>”;

@Override

public void configure() throws Exception {

//Camel’s Java DSL to define a global Exception handler..

onException(Exception.class).logStackTrace(true).handled(true);

//Time interval of polling

int delay = 60; //in seconds

String twitterUrl = String.format(“twitter://timeline/home?type=polling” +

“&delay=60&consumerKey=%s&consumerSecret=%s&accessToken=%s&accessTokenSecret=%s”,

consumerKey, consumerSecret, accessToken, accessSecret);

//You can redirect tweets to file or to a bean

from(twitterUrl)//.to(“file://home/mytweets.txt”);

//Bean instantition done by Type, Specify method name which gets called when

.bean(TwitterStore.class, “storeTweet”);

}

}

 

Here is TwitterStore class which actually prints message on console. Needless to say you can have your own implementation here.

public class TwitterStore {

 

public void storeTweet(String tweet){

System.out.println(tweet);

}

}

 

That’s it. Once your route is defined you need to register it with Camel Context and start Camel Context

 

TwitterRoute twitterRoute = new TwitterRoute();

try {

CamelContext myCamelContext = new DefaultCamelContext();

myCamelContext.addRoutes(twitterRoute);

myCamelContext.start();

System.in.read();

} catch (Exception e) {

// TODO Auto-generated catch block

e.printStackTrace();

}

Above code goes into main.

You can download maven project from above mentioned link. I have hosted project on github. Just drop me a comment if you need more help. In future we will see more sample projects on apache camel

Apache Camel Sample Project

Software architecture and it’s importance

This post is about understanding architecture of an application and it’s role in web application development.

I have worked in various domains writing softwares. All of us know that although developing software for each business domain throws new challenges and their needs differ vastly but there are many tools and systems which are used commonly everywhere.

So, most of the web applications are following this approach what we can call 3-tier applications.

You have got a presentation layer, business layer and a database layer. When you deploy you should be able to deploy on 3 boxes. Surprisingly lot of times you will end up deploying only on 2 boxes. Reason why some company might do this is that they did not segregate between physical and logical layers. One with all the logic and other one with just RDBMS running in background. Even sites like twitter and GroupOn had one big chunk of monolithic softwares where everybody was coding. Later on they decided to dismantle it into smaller blocks as it was impossible to scale this big piece.

Most of seasoned developers understand the meaning of layers but one important thing that they do not understand that a layer can be logical as well as physical.

A logical layer (separation of concerns, modular programming ..whatever you call them) sits in the same process space as other layers and just segregates logic.

You create a layer and expose it’s various functionality via interface. Client of your layer injects them using some fancy library like Guice, Spring or does a “new”.

Many start-up companies at least in India are taking this approach. They start small develop code in one big chunk and when they grow they start dismantling their code into various modules.

It might have worked for some companies but it puts a huge pressure on ROI and dependency increases on the existing developers. If one of them leaves you are doomed. And If I am not wrong you went with no documentation at all by following practices of agile programming (in wrong way). So all you can do is further increase the costs by offering higher wages :-).

On the other hand new people are feeling bad because they are not able to participate much as multiple activities are going on in parallel. Refactoring of current architecture, new feature development, pressure of management to develop new feature and bug fixing/maintenance of existing systems.

For refactoring you have probably cut a new branch and start working there but by the time you are finished your existing branch in production and your re-factored branch have been diverged so much that it becomes another exercise to merge them.

People do it though but almost on daily basis you will get regressions and all this mess will lead to another mess and increase the costs significantly. It will hamper the growth of the company.

There is no one solution to it. Different situations will require different solutions but I have also seen systems which have been managed pretty well over the years and thousands of new developers come and join the new team yet things remain in control.

So how come these two sides of same coin exists. When technology stack is same aren’t we supposed to have same kind of maintainable system?

What different one company did then others. As per me answer lies in Architecture of the systems.

One company was able to get it’s application’s architecture spot on right from the beginning  Another company was doubtful about it’s growth and quickly wanted to put something on dashboard of users and praying that when they grow they will think about it.

This is also not a bad approach it works in many cases but getting the design right in first place does not take so much effort as it looks like.

Web frameworks like Ruby on Rails, Grails and Java script library JQuery are built on the very concept of plugins. It keeps things under control and these small piece of softwares can be maintained easily. If some component starts behaving badly you just stop using it by unloading that particular component.

It is well known practice adopted by experts and computer scientists that one should write code as if he is writing a library. It automatically brings modularity in system and maintenance become very easy (comparatively). and same is true about architecture also. One should develop modules to be consumed by others. Modules or components are supposed to expose a certain functionality. Others are just consuming it.

Great..looks like this is holy grail for solving our problems. Not yet. Once we have created different components and decided to deploy them on different machines we are actually facing host of other issues Deployment, Inter process communication, Fault Tolerance, Centralized logging to name a few.

We will try to solve these problems one by one in upcoming articles

Software architecture and it’s importance

Apache Camel – Why you should learn?

Recently I encountered a problem related to integration with some third party APIs. These days it is pretty common to outsource some activity for a company, integrating with social networking website, integration with third party tools installed on company premises.

These tools can be built on various technologies and integration between different component becomes an issue when people start writing code for individual component and do not target integration as one cohesive process.

Result is cluttered tightly coupled development based on many assumptions. Necessity is to take a comprehensive uniform approach to tackle this problem.

Various open source ESB and commercial solutions are available for this but in my experience I have found that using an ESB right away is a difficult thing for developers to accept as every ESB faces a certain style of development.

So approach should be slowly introduce the concept involved in an ESB during development process.

One such core ingredient in ESB is called  Enterprise Integration Patterns into core modules and let people learn the concept and admire them.

Enterprise Integration Patterns (EIP) are patterns just like GOF patterns but context is “integration” between disparate systems.

These patterns are well documented in the book by Gregor Hohpe and Bobby Woolf.

If you want to have a quick look at what these patterns are then go to http://camel.apache.org/enterprise-integration-patterns.html

Apache Camel is an out of the box solution which implements all patterns described in EIP and provides integration with many more components which can help developers by avoiding pain of writing boilerplate code. It also gives many useful implementations which will bring a consistency in your large code base that everyone will be able to easily understand.

Patterns are a proven way of building robust scalable and easy maintenance software and following them religiously can make a developer’s life easier and make organization save huge amount of money in long term.

Camel not only provides implementation of these EIP but more importantly it’s syntax is so fluent that it brings a paradigm shift the way one writes integration code.

It is an important framework to learn for a Java developer and I do not find any direct alternative in Microsoft world.

Just to give you an idea consider a simple scenario. You want to integrate to a third party API like a social networking, Courier Service, Text to Speech service or something else. It may so happen that while calling this API network glitch occurs and your call does not succeed. What do you want to do?

You guessed it right..All we can do is retry. So probably you will write a while loop which will check the HTTP status of this API and then if it’s not 200 then just retry. OK you can do that but problem is very common to be faced by every developer in your team so all of them are writing same type of code.

If you are a well managed team then you will probably create a shared library and publish it to your local maven repo and inform about it.

Great..you did that. Let’s add another twist to the tale..You want to retry but after some time. What do you do? Put sleep somewhere in your while loop. Again enhance your library and roll out a new version. Over the period you will come up with many such problems and end up writing same code which is there in camel (only better implemented).

Camel on the other hand provides a nice fluent interface which promotes consistent coding style across the board and you have a tool in your hand which solves lot of your problems out of the box.

In the next article we will try to develop a small application for building a fault tolerant application using Apache Camel

Apache Camel – Why you should learn?