Develop Reference

Topic created in all kafka port

server.propereties setup:
listeners=PLAINTEXT://:29092, SSL://:29093
SSL related set too done.
so that we can connect 29092 for plaintext and 29093 along with SSL setup.
Here am trying to produce data into port 29093 as below
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, System.getProperty("kafkaPort", "localhost:29093"));
//SSL related setup too done in props
Producer<Long, String> producer = new KafkaProducer<>(props, new LongSerializer(), new KafkaSerializer());
final ProducerRecord<Long, String> record = new ProducerRecord<Long, String>(System.getProperty("kafkaTopic", "dqerror"),
content);
RecordMetadata metadata = producer.send(record).get();
After publishing dqerror topic created in both also data get published in both
Data is published into two topic.
Actually, am trying to find is any possible to restrict to drop data into a specific port ?

Data is not published in "both" ports. There is only one Kafka cluster that is listening on two ports. There is one set of disks that the data is written into on your one broker.
Also, from what I can tell, there is only one topic used in your code.
If you want to restrict TCP traffic on any port, that would be a firewall rule from the OS, rather than any Kafka settings or Java code.

How to set read request timeout for cassandra

I try to create new endpoints for cassandra with different read request timeout. The endpoint with big timeout for requests with big data responds.
I found Scala code with com.datastax.cassandra driver and cassandra-default.yaml with read_request_timeout parameter. How to set read_request_timeout in Cluster builder or in other places in code ?
Cluster
.builder
.addContactPoints(cassandraHost.split(","): _*)
.withPort(cassandraPort)
.withRetryPolicy(DefaultRetryPolicy.INSTANCE)
.withLoadBalancingPolicy(
new TokenAwarePolicy(DCAwareRoundRobinPolicy.builder().build())).build
# How long the coordinator should wait for read operations to complete
read_request_timeout_in_ms: 5000

Set at query level using :
session.execute(
new SimpleStatement("CQL HERE").setReadTimeoutMillis(65000));
If you want to set while cluster bulding use :
Cluster cluster = Cluster.builder()
.addContactPoint("127.0.0.1")
.withSocketOptions(
new SocketOptions()
.setConnectTimeoutMillis(2000))
.build();
Socket Options

Multiple Endpoints in Cassandra cluster Connection

I want to give multiple Cassandra endpoints from the config file to my Java application.
Ex:
cassandra host: "host1, host2"
I tried addContactPoints(host), but it did not work. If one of the Cassandra node goes down, I don't want my application to go down.
cluster = Cluster.builder()
.withClusterName(cassandraConfig.getClusterName())
.addContactPoints(cassandraConfig.getHostName())
.withSocketOptions(new SocketOptions().setConnectTimeoutMillis(30000).setReadTimeoutMillis(30000))
.withPoolingOptions(poolingOptions).build();

The java driver is resilient to one of the contact points provided not being available. Contact points are used for establishing an initial connection [*]. As long as the driver is able to communicate with one contact point, it should be able to query the system.peers and system.local table to discover the rest of the nodes in the cluster.
* They are also added to a list of initial hosts in the cluster, but typically the contact points provided map to a node in the system.peers table.

Usage of the LOCAL_QUORUM consistency level in Datastax driver

For some reasons I need to query a particular datacenter within my cassandra cluster. According to the documentation, I can use the LOCAL_QUORUM consistency level:
Returns the record after a quorum of replicas in the current
datacenter as the coordinator has reported. Avoids latency of
inter-datacenter communication.
Do I correctly understand, that in order to specify a particular datacenter for the current query, I have to build a cluster on the given endpoint belonging to this particular DC?
Say, I have two DC's with the following nodes:
DC1: 172.0.1.1, 172.0.1.2
DC1: 172.0.2.1, 172.0.2.2
So, to work with DC1, I build a cluster as:
Cluster cluster = Cluster.builder().addContactPoint("172.0.1.1").build();
Session session = cluster.connect();
Statement statement = session.prepare("select * from ...").bind().setConsistencyLevel(ConsistencyLevel.LOCAL_QUORUM);
ResultSet resultSet = session.execute(session);
Is it a proper way to do that?

By itself, DCAwwareRoundRobinPolicy will pick the data center that it finds with the "least network distance" algorithm. To ensure it connects where you want, you should specify the DC as a parameter.
Here is how I tell our dev teams to do it:
Builder builder = Cluster.builder()
.addContactPoints(nodes)
.withQueryOptions(new QueryOptions()
.setConsistencyLevel(ConsistencyLevel.LOCAL_ONE))
.withLoadBalancingPolicy(new TokenAwarePolicy(
new DCAwareRoundRobinPolicy.Builder()
.withLocalDc("DC1").build()))
.withPoolingOptions(options);
Note: this may or may not be applicable to your situation, but do I recommend using the TokenAwarePolicy with the DCAwareRoundRobin nested inside it (specifying the local DC). That way any operation specifying the partition key will automatically route to the correct node, skipping the need for an extra hop required with a coordinator node.

According to the Cluster class documentation:
A cluster object maintains a permanent connection to one of the
cluster nodes which it uses solely to maintain information on the
state and current topology of the cluster
Also, because a default load balancing policy is DCAwareRoundRobinPolicy this approach should work fine as expected.

Can I have more than 1 'mongos' instance?

I am using Java to insert data into mongodb cluster.
Can I have more than 1 mongos instance so that I have a backup when 1 of my mongos is down?
Here is my java code to connect to mongos.
MongoClient mongoClient = new MongoClient("10.4.0.121",6001);
DB db = mongoClient.getDB("qbClientDB");
DBCollection collection = db.getCollection("clientInfo");
How can I specify my second mongos instance in my Java code? (If possible).
Thanks in advance.

The MongoClient docs say that you can, something similar to (the dummy addresses);
MongoClient mongoClient = new MongoClient(Arrays.asList(
new ServerAddress("10.4.0.121",6001),
new ServerAddress("10.4.0.122",6001),
new ServerAddress("10.4.0.123",6001)));
MongoClient will auto-detect whether the servers are a list of replica set members or a list of mongos servers.

public MongoClient(List<ServerAddress> seeds,
MongoClientOptions options)
//Creates a Mongo based on a list of replica set members or a list of mongos. It will find all members (the master will be used by default). If you pass in a single server in the list, the driver will still function as if it is a replica set. If you have a standalone server, use the Mongo(ServerAddress) constructor.
//If this is a list of mongos servers, it will pick the closest (lowest ping time) one to send all requests to, and automatically fail over to the next server if the closest is down.
MongoClient mongoClient = new MongoClient(Arrays.asList(
new ServerAddress("10.4.0.121",6001),
new ServerAddress("10.4.0.122",6001),
new ServerAddress("10.4.0.123",6001)));