Vert.x Cluster

Vert.x is an extremely simple event based, non blocking,  library for distributed computing that can be easily embedded in any Java framework of your choice.

For sometime now I have been exploring Vert.x. I was looking for a vert.x cluster sample but I did not find any decent example so I decided to share that with community.

I modified one of the sample available from Vert.x examples and I will explain core parts of it here.

First the cluster has to be configured.

Config hazelcastConfig = new Config();

Config hazelcastConfig = new Config();
hazelcastConfig.getNetworkConfig().getJoin().getTcpIpConfig().addMember("127.0.0.1").setEnabled(true);
hazelcastConfig.getNetworkConfig().getJoin().getMulticastConfig().setEnabled(false);

ClusterManager mgr = new HazelcastClusterManager(hazelcastConfig);
VertxOptions options = new VertxOptions().setClusterManager(mgr);
Vertx.clusteredVertx(options, res -> {
    if (res.succeeded()) {
        vertx = res.result();
        deploy(vertx,context,mode);
    }
});

This code can be part of your main function. You can also create a cluster config file called “cluster.xml” rather than creating configurations programatically. This should be on classpath of your application so it can be put inside src/main/resources I am going to test this application on my local machine so I am using TCP discovery rather than Multicast for my application. Vert.x underlying uses hazelcast (default) for all it’s clustering capabilities. Hazelcast also comes up with a special configuration for AWS. So if you are going to deploy your application on AWS then that is the configuration you should use.

<?xml version="1.0" encoding="UTF-8"?>
<hazelcast xsi:schemaLocation="http://www.hazelcast.com/schema/config hazelcast-config-3.2.xsd"
           xmlns="http://www.hazelcast.com/schema/config"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <properties>
       .....
        <property name="hazelcast.wait.seconds.before.join">0</property>
    </properties>

    <group>
        <name>dev</name>
        <password>dev-pass</password>
    </group>
    <management-center enabled="false">http://localhost:8080/mancenter</management-center>
    <network>
        <port auto-increment="true" port-count="10000">5701</port>
        <outbound-ports>
            <!--
            Allowed port range when connecting to other nodes.
            0 or * means use system provided port.
            -->
            <ports>0</ports>
        </outbound-ports>
        <join>
            <!--<multicast enabled="false">-->
            <!--<multicast-group>224.2.2.3</multicast-group>-->
            <!--<multicast-port>54327</multicast-port>-->
            <!--</multicast>-->
            <tcp-ip enabled="true">
                <interface>192.168.1.8</interface>
            </tcp-ip>
            <aws enabled="false">
                .....
            </aws>
        </join>
        <interfaces enabled="false">
            <interface>10.10.1.*</interface>
        </interfaces>        
    </network>
    <partition-group enabled="false"/>
    <executor-service name="default">
        <pool-size>16</pool-size>
        <!--Queue capacity. 0 means Integer.MAX_VALUE.-->
        <queue-capacity>0</queue-capacity>
    </executor-service>
    <map name="__vertx.subs">

        <!--
            Number of backups. If 1 is set as the backup-count for example,
            then all entries of the map will be copied to another JVM for
            fail-safety. 0 means no backup.
        -->
        <backup-count>1</backup-count>
      
        <time-to-live-seconds>0</time-to-live-seconds>
        <max-idle-seconds>0</max-idle-seconds>
        <!--
            Valid values are:
            NONE (no eviction),
            LRU (Least Recently Used),
            LFU (Least Frequently Used).
            NONE is the default.
        -->
        <eviction-policy>NONE</eviction-policy>
        <!--
            Maximum size of the map. When max size is reached,
            map is evicted based on the policy defined.
            Any integer between 0 and Integer.MAX_VALUE. 0 means
            Integer.MAX_VALUE. Default is 0.
        -->
        <max-size policy="PER_NODE">0</max-size>
        <!--
            When max. size is reached, specified percentage of
            the map will be evicted. Any integer between 0 and 100.
            If 25 is set for example, 25% of the entries will
            get evicted.
        -->
        <eviction-percentage>25</eviction-percentage>
        <merge-policy>
      com.hazelcast.map.merge.LatestUpdateMapMergePolicy
</merge-policy>
    </map>

    <!-- Used internally in Vert.x to implement async locks -->
    <semaphore name="__vertx.*">
        <initial-permits>1</initial-permits>
    </semaphore>

</hazelcast>

I have included only important parts and left others from configuration Once we are done with cluster configuration all we have to do is create a verticle and deploy it.

public class ServerVerticle extends AbstractVerticle {

    int port;
    public ServerVerticle(int port){
        super();
        this.port = port;
    }
    @Override
    public void start() throws Exception {
        super.start();
        HttpServer server = vertx.createHttpServer();
        server.requestHandler(req -> {
            if (req.method() == HttpMethod.GET) {
                req.response().setChunked(true);

                if (req.path().equals("/products")) {
                    vertx.eventBus().<String>send(SpringDemoVerticle.ALL_PRODUCTS_ADDRESS, "", result -> {
                        if (result.succeeded()) {
                            req.response().setStatusCode(200).write(result.result().body()).end();
                        } else {
                            req.response().setStatusCode(500).write(result.cause().toString()).end();
                        }
                    });
                } else {
                    req.response().setStatusCode(200).write("Hello from vert.x").end();
                }

            } else {
                // We only support GET for now
                req.response().setStatusCode(405).end();
            }
        });

        server.listen(port);
    }
}

Great…let’s deploy this verticle.

Once configuration has been done you need to deploy verticles in cluster

Vertx.clusteredVertx(options, res -> {
    if (res.succeeded()) {
        Vertx vertx = res.result();
       //You should deploy verticles only when cluster has been initialized
        vertx.deployVerticle(new ServerVerticle(Integer.parseInt(args[0])));
    } else {
    }
});

 

You start the application by giving a port let’s say at port 9000 and start another instance at different port let’s say 9005 and Voila can see both of them start communicating. Complete source code can be found at https://github.com/singhmarut/vertx-cluster.git Happy Coding !!

Vert.x Cluster

Why Java is my default choice?

Recently in my company this never ending debate of language has come up.
Needless to say everybody loves the language/stack they have experience with. So I can not really debate about why not other languages but I can tell why Java is my default language.

Before I start let me tell you how many languages I have command today as far server side is considered

a. C++, Java,C#.Net, Scala, Groovy, Javascript (Node.js)  and ready to explore more always

I started my career 12 long years back as a rookie and started working on a project called Rule Engine developed in VC++.

Liked them as well but C++ remained the first love..I was of the view that compared to other programmers if I learn the language deep enough to be a master (yes cause I was immensely impressed with the likes of Scott Mayers, Herb Sutter, Alexndrascu (creator of D), Don Box “the com guy” and many others. They were all C++ programmers.

Fast forward I joined a start a “web start-up” which was creating a web app. and I was asked to lead. We rearchiteted whole server side code in Java but again somebody else took responsibility for front end cause that was “web” according to C++ programmer inside me.

It’s performance is only a tad bit slow compared to C++ and is negligible for a web application where other aspects of the application architecture come into play but there is no way Java can be slower than PHP, Python, Ruby etc

So you should Java cause

  • It is high performance
  • It has huge ecosystem..So many web frameworks to choose from
  • It is enterprise grade
  • It is the only language that runs on 6 billion devices
  • It is stable and will remain a popular language in coming future
  • It’s ecosystem is well managed by apache
  • No dearth of programmers
  • JVM opens new opportunities everyday
  • Monitoring tools available aplenty
  • It keeps evolving (Functional programming in Java8)

Very surprising to see people working in languages like PHP/Python rarely do memory profiling may be because they simply do not have a tool.

To me Java is a language you can do anything with. You want web programming you got it.

You want to write high performance server? Bring it on..So many projects like Storm and Hadoop if they can rely on Java/JVM then you can too.

Whatever you can achieve in Pythons and Rubys you can do with Java. May be you wont like the syntax but vast ecosystem around Java makes sure you are never at loss while selecting a library or a tool.

and if you know Java then you know JVM and if you know JVM you can choose from any language from your choice like Scala..Don’t like Scala? Try Kotlin..Don’t like that either try groovy,Ceylon and so son.

So yes I guess you are in safe spot with Java which extremely well understood with infinite documentation and remains one of the top choice of programmers for last 2 decades and I do not see its glory fading away anytime soon.

So working with Java is like working with a world class MNC…bit old school but still better then many start ups and yet deliver what it promises. Very little surprises here.

Happy coding !!

Why Java is my default choice?

Not so many reasons Why C++ sucks..

Coded in C++ for 5 years so would apologize in advance. But this is just my opinion. Do I want to start a language war? You bet.

– Pointers suck
– Memory management sucks
– Function Pointers yeah they do
– goto statements sucks
– Friend Functions sucks
– static initialization sucks

– Macros Suck
– headers files sucks? If you don’t know then don’t bother..you can live without knowing them
– Multiple Inheritance sucks
– Diamonds suck
– Interfaces? Sorry C++ doesn’t have them so they can’t…
– reinterpret_cast I think they do suck
– Compilation suck
– Makefile suck


– Threading? sorry what? oh yeah use PThreads or boost..What is boost? some other day man..long story

– Templates? so complex they deserve 10 posts like this as they just don’t suck they suck bigtime.You think you know them? solve a simple puzzle and see if you knew the answer already

C++ Inheritance Puzzle

Did I mention it was a simple one..people interested in knowing them in detail read
alexandrescu modern design

Correction – By the way C++ did get great support for threads now simplifies the job so much
http://en.cppreference.com/w/cpp/thread  Boy, I love this language

Not so many reasons Why C++ sucks..

Why foo?

Why foo?
Somebody asked in Neal Ford presentation at thought works why we name everything Foo? Class Foo, Function Foo

He could not answer so here is my take on what other names could be adopted?

Aoo – Can’t even pronounce
Boo- Pretty romantic doesn’t look like a geeky word
Coo – Don’t know how to pronounce
Doo – Looks like a children word Doo Doo
Eoo – Can’t even pronounce

Goo – sorry doesn’t work for Indians
Hoo – Spreads negativity looks like hooting
Ioo – Can’t even pronounce
Joo – May be let’s keep it for now
Koo – Doesn’t sound good Koo – coo
Loo – Don’t want to go there during a presentation
Moo – Looks like straight from nursery rhyme
Noo – May be let’s keep it
Ooo – What is this? Doesn’t work
Poo – Hmm..who wants to utter it while coding?
Qoo – Can’t even pronounce
Roo – May be let’s keep it for now
Soo – Doesn’t work for Indians at least and that’s all I am bothered
Too – looks like counting
Uoo – Can’t even pronounce
Voo – Seems romantic again…need a geeky word
Woo – Same as above
Xoo – Can’t even pronounce
Zoo – Sorry will go there later let me code for now

So we are left with

Foo – Joo – Noo

Out of this Foo comes first and it definitely sounds better than other 2..So Foo wins Yayyy…

Why foo?

Web Application Scalability Part-3 ES vs Redis

Its 3rd article in Web Scalability..Today I am looking at various projects going on in Indian Start ups and I am thrilled by variety of technologies being used.

While this sounds good at the same time I feel we are unable to utilize core strengths of technologies used.

Their usage in architecture will always be dependent on person to person and their experience and comfort level. But some simple things if adopted early stage can give immense benefits.

One such technology is our simple cache which is being used everywhere but alas not in the right way.

It’s called redis. Yes the kind of benefit this humble software can bring to create a highly scalable websites is awesome.

I have seen many startups are using Elastic Search as a “Cache”. Yes, I can not imagine someone doing this until you are total nuts. ES is an indexing technology and not a “Cache”. It will never update in real time which is what is expected from cache…It will never be in sync with underlying storage and hence almost always have stale data.

Using it as primary data source for anything is simply insane and shows the immaturity of the architect who chose it in the first place. It will not work except for most trivial projects.

Anyways ES has it niche but it’s not a cache but Redis is. It provides many advanced data structures and can do hell lot of thing in caching layer itself.

One needs to take a hard look at various data structures provided and how they can be used to create a “real” caching layer on top of it and get same output which one was hoping to get from ES out of the box.

It has sets/hashes/sorted sets/lists etc. etc.

Keep putting data at the rate your site might never experience and retrieve it at blazingly fast speed.

Most importantly its single thread model brings the consistency one might need for creating distributed counters.

Think about it and if you are confused how it can be used for your use case just like always drop an email..who knows I may like your use case interesting enough and help you

Web Application Scalability Part-3 ES vs Redis