Vert.x Cluster

Vert.x is an extremely simple event based, non blocking,  library for distributed computing that can be easily embedded in any Java framework of your choice.

For sometime now I have been exploring Vert.x. I was looking for a vert.x cluster sample but I did not find any decent example so I decided to share that with community.

I modified one of the sample available from Vert.x examples and I will explain core parts of it here.

First the cluster has to be configured.

Config hazelcastConfig = new Config();

Config hazelcastConfig = new Config();
hazelcastConfig.getNetworkConfig().getJoin().getTcpIpConfig().addMember("127.0.0.1").setEnabled(true);
hazelcastConfig.getNetworkConfig().getJoin().getMulticastConfig().setEnabled(false);

ClusterManager mgr = new HazelcastClusterManager(hazelcastConfig);
VertxOptions options = new VertxOptions().setClusterManager(mgr);
Vertx.clusteredVertx(options, res -> {
    if (res.succeeded()) {
        vertx = res.result();
        deploy(vertx,context,mode);
    }
});

This code can be part of your main function. You can also create a cluster config file called “cluster.xml” rather than creating configurations programatically. This should be on classpath of your application so it can be put inside src/main/resources I am going to test this application on my local machine so I am using TCP discovery rather than Multicast for my application. Vert.x underlying uses hazelcast (default) for all it’s clustering capabilities. Hazelcast also comes up with a special configuration for AWS. So if you are going to deploy your application on AWS then that is the configuration you should use.

<?xml version="1.0" encoding="UTF-8"?>
<hazelcast xsi:schemaLocation="http://www.hazelcast.com/schema/config hazelcast-config-3.2.xsd"
           xmlns="http://www.hazelcast.com/schema/config"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <properties>
       .....
        <property name="hazelcast.wait.seconds.before.join">0</property>
    </properties>

    <group>
        <name>dev</name>
        <password>dev-pass</password>
    </group>
    <management-center enabled="false">http://localhost:8080/mancenter</management-center>
    <network>
        <port auto-increment="true" port-count="10000">5701</port>
        <outbound-ports>
            <!--
            Allowed port range when connecting to other nodes.
            0 or * means use system provided port.
            -->
            <ports>0</ports>
        </outbound-ports>
        <join>
            <!--<multicast enabled="false">-->
            <!--<multicast-group>224.2.2.3</multicast-group>-->
            <!--<multicast-port>54327</multicast-port>-->
            <!--</multicast>-->
            <tcp-ip enabled="true">
                <interface>192.168.1.8</interface>
            </tcp-ip>
            <aws enabled="false">
                .....
            </aws>
        </join>
        <interfaces enabled="false">
            <interface>10.10.1.*</interface>
        </interfaces>        
    </network>
    <partition-group enabled="false"/>
    <executor-service name="default">
        <pool-size>16</pool-size>
        <!--Queue capacity. 0 means Integer.MAX_VALUE.-->
        <queue-capacity>0</queue-capacity>
    </executor-service>
    <map name="__vertx.subs">

        <!--
            Number of backups. If 1 is set as the backup-count for example,
            then all entries of the map will be copied to another JVM for
            fail-safety. 0 means no backup.
        -->
        <backup-count>1</backup-count>
      
        <time-to-live-seconds>0</time-to-live-seconds>
        <max-idle-seconds>0</max-idle-seconds>
        <!--
            Valid values are:
            NONE (no eviction),
            LRU (Least Recently Used),
            LFU (Least Frequently Used).
            NONE is the default.
        -->
        <eviction-policy>NONE</eviction-policy>
        <!--
            Maximum size of the map. When max size is reached,
            map is evicted based on the policy defined.
            Any integer between 0 and Integer.MAX_VALUE. 0 means
            Integer.MAX_VALUE. Default is 0.
        -->
        <max-size policy="PER_NODE">0</max-size>
        <!--
            When max. size is reached, specified percentage of
            the map will be evicted. Any integer between 0 and 100.
            If 25 is set for example, 25% of the entries will
            get evicted.
        -->
        <eviction-percentage>25</eviction-percentage>
        <merge-policy>
      com.hazelcast.map.merge.LatestUpdateMapMergePolicy
</merge-policy>
    </map>

    <!-- Used internally in Vert.x to implement async locks -->
    <semaphore name="__vertx.*">
        <initial-permits>1</initial-permits>
    </semaphore>

</hazelcast>

I have included only important parts and left others from configuration Once we are done with cluster configuration all we have to do is create a verticle and deploy it.

public class ServerVerticle extends AbstractVerticle {

    int port;
    public ServerVerticle(int port){
        super();
        this.port = port;
    }
    @Override
    public void start() throws Exception {
        super.start();
        HttpServer server = vertx.createHttpServer();
        server.requestHandler(req -> {
            if (req.method() == HttpMethod.GET) {
                req.response().setChunked(true);

                if (req.path().equals("/products")) {
                    vertx.eventBus().<String>send(SpringDemoVerticle.ALL_PRODUCTS_ADDRESS, "", result -> {
                        if (result.succeeded()) {
                            req.response().setStatusCode(200).write(result.result().body()).end();
                        } else {
                            req.response().setStatusCode(500).write(result.cause().toString()).end();
                        }
                    });
                } else {
                    req.response().setStatusCode(200).write("Hello from vert.x").end();
                }

            } else {
                // We only support GET for now
                req.response().setStatusCode(405).end();
            }
        });

        server.listen(port);
    }
}

Great…let’s deploy this verticle.

Once configuration has been done you need to deploy verticles in cluster

Vertx.clusteredVertx(options, res -> {
    if (res.succeeded()) {
        Vertx vertx = res.result();
       //You should deploy verticles only when cluster has been initialized
        vertx.deployVerticle(new ServerVerticle(Integer.parseInt(args[0])));
    } else {
    }
});

 

You start the application by giving a port let’s say at port 9000 and start another instance at different port let’s say 9005 and Voila can see both of them start communicating. Complete source code can be found at https://github.com/singhmarut/vertx-cluster.git Happy Coding !!

Vert.x Cluster

8 thoughts on “Vert.x Cluster

  1. gsagrawal says:

    i was trying to setup something similar on aws machines. I am facing issues with event bus (though all data is getting shared). To reproduce the same ,i forked one normal ping/pong example and did some changes (my use cases). https://github.com/gsagrawal/vertx3-ping-pong (i didnt checked in the cluster.xml but this is same as you mentioned) . Can you help me out here ?
    **though above example works perfectly well if i run these without my custom main (i dont wanna do that, i want to do some other configurations as well ) or in main if i set clusterHost as the pong server ip (but i cant hardcode the ip)

    Like

  2. I’m curious, what are the benefits of clustering VertX that you can’t get with a load balancer?

    Example, if I have a VertX cluster of 3 machines and shutdown one of the 3 machines, what happens to existing processes that’s happening on that machine that just got shut down?

    Like

    1. Well that’s a relevant question…Benefits of clustering via Vertx will give you flexibility of creating a cluster dynamically, you can add or remove node without changing any configuration with load balancer. Load balancer has a cost and also for middleware clustering “load balancer” is not the perfect solution. Imagine having 20 micro-services in the system. So load balancer compared to Vert.x
      – is not dynamic
      – has significant cost
      – has an overhead.
      – has dependency on “devops”

      To answer your other question behavior is same as it will be without Vert.x Anything running on that machine will be killed.
      Assuming you are talking about verticles then those verticles will be killed with the JVM. On the other hand if you are using Akka which is similar but more rich framework you have an hierarchy of actors so actor on some other machine will be notified in case this happens then you can take corrective action.
      http://doc.akka.io/

      Like

      1. I’ve been reading a bit more on VertX clustering; you are right about load balancers for backend services, we simply use a RabbitMQ cluster to “load-balance” backend services, frontend services however you still need a load-balancer to do SSL offloading, etc.
        Clustering gives you a shared variable space if I understand it correctly, specifically a map that can be synced between instances / verticles. What would be the typical use-case for such a shared map (for every reason I can think of using it, I can think of another way of doing it without clustering) and is that shared map “hosted” on an external Hazelcast instance or does each verticle have their own embedded Hazelcast?

        Like

      2. shared map is functionality provided by Hazelcast…I have encountered this scenarios and I think clustered map can be useful for
        a. In memory-data grid as a replacement for Redis/Memcache. Benefit that you get is that Data become part of your application so you can access data locally.
        You do not have to cross network boundaries or install a new software just for this purpose and yes you can keep adding machines which will give you infinite memory.
        You also have clustered AtomicLong which can be really useful in some scenarios.

        b. For session replication. It’s a common problem for which you generally rely on containers.

        Like

  3. One more thing you need to think that RabbitMQ cluster will never achieve the scalability of Vert.x Cluster and for RabbitMQ clustering you will need to manage another cluster which can come out of the box using Vert.x. So Vert.x is more suitable for microservices compared to a separate middleware like RabbitMQ or ActiveMQ

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s