Develop Reference

Final 2D-Array data getting altered without implicitly changing its value using Java

I am currently facing an issue regarding this method getSurroundingSumGrid() which is supposed to take data from an earlier grid that was built based off of text file data and use it to determine new values within the array sumGrid. The STATICGRID array gets built at first with the correct values but then as the for loop continues on, the STATICGRID values change to what i have set sumGrid to change to. I don't have any defined code where STATICGRID is ever set to equal another value and if I did it should give an error.
public double[][] getSurroundingSumGrid() {
this.sumGrid = getBaseGrid();
for (int rowNum = 0; rowNum < sumGrid.length; rowNum++) {
final double[][] STATICGRID = this.getBaseGrid();
double topNum = 0, botNum = 0, rightNum = 0, leftNum = 0;
for (int colNum = 0; colNum < sumGrid[0].length; colNum++) {
try {
topNum = STATICGRID[rowNum - 1][colNum];
System.out.println("TOPNUM : (" + (rowNum-1) + "," + colNum + ") " + STATICGRID[rowNum-1][colNum]);
} catch (Exception e) {
topNum = STATICGRID[rowNum][colNum];
System.out.println("Top IndexOutOfBoundsException: " + STATICGRID[rowNum][colNum] + " used instead.");
}
try {
botNum = STATICGRID[rowNum + 1][colNum];
System.out.println("BOTNUM : (" + (rowNum+1) + "," + colNum + ") " + STATICGRID[rowNum+1][colNum]);
} catch (Exception e) {
botNum = STATICGRID[rowNum][colNum];
System.out.println("Bot IndexOutOfBoundsException: " + STATICGRID[rowNum][colNum] + " used instead.");
}
try {
leftNum = STATICGRID[rowNum][colNum - 1];
System.out.println("LEFTNUM : (" + rowNum + "," + (colNum-1) + ") " + STATICGRID[rowNum][colNum-1]);
} catch (Exception e) {
leftNum = STATICGRID[rowNum][colNum];
System.out.println("Left IndexOutOfBoundsException: " + STATICGRID[rowNum][colNum] + " used instead.");
}
try {
rightNum = STATICGRID[rowNum][colNum + 1];
System.out.println("RIGHTNUM : (" + rowNum + "," + (colNum+1) + ") " + STATICGRID[rowNum][colNum+1]);
} catch (Exception e) {
rightNum = STATICGRID[rowNum][colNum];
System.out.println("Right IndexOutOfBoundsException: " + STATICGRID[rowNum][colNum] + " used instead.");
}
this.sumGrid[rowNum][colNum] = topNum + botNum + rightNum + leftNum;
System.out.println("STATICGRID NEW NUM : " + STATICGRID[rowNum][colNum]);
System.out.println("SUMGRID NEW NUM : " + sumGrid[rowNum][colNum]);
}
}
return this.sumGrid;
}
When doing these tests with the code I can see very clearly that the data in both arrays are changing overtime, and in turn giving me wrong results. I've tried for about 2 hours just moving things around and can't seem to figure out how to get this to work properly.
As you can even see, I even attempted rebuilding the STATICGRID array every single time the for loop completed and it wouldn't even hinder the result. It does the same thing regardless of where you put the STATICGRID at (either outside or inside at the top-most level of the for loop, and it doesn't matter whether it's final or not), it does the same thing. After looking at it for so long I'm beyond confused on why my code isn't working and I have a slight feeling that it is the try-catch statement but I wouldn't at all know why. I don't know a ton about the statement and what it does entirely but the reason it is there is because the data can get an IndexOutOfBoundsException so instead of getting that it would instead count itself for each IndexOutOfBoundsException it got as per the assignment instructions.
Thanks and I hope this makes sense.

Alright, thanks to FredK's suggestion at using a deepCopy, I did some research and used this method to get better results. This is the unoptomizedDeepCopy by Philip Isehour. I don't exactly understand it but I'm going to just use it for now and spend some time learning more about this and how they work. I'm currently a CS221 student and we haven't gone over deepCopy yet.
public static Object deepCopy(Object orig) {
Object obj = null;
try {
// Write the object out to a byte array
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream out = new ObjectOutputStream(bos);
out.writeObject(orig);
out.flush();
out.close();
// Make an input stream from the byte array and read
// a copy of the object back in.
ObjectInputStream in = new ObjectInputStream(
new ByteArrayInputStream(bos.toByteArray()));
obj = in.readObject();
}
catch(IOException e) {
e.printStackTrace();
}
catch(ClassNotFoundException cnfe) {
cnfe.printStackTrace();
}
return obj;
}
After this method, I then was able to stop the array from changing value by using this in front of the getBaseGrid() method.
public double[][] getSurroundingSumGrid() {
this.sumGrid = (double[][]) GridMonitor.deepCopy(this.getBaseGrid());
double[][] staticGrid = (double[][]) GridMonitor.deepCopy(this.getBaseGrid());
double topNum, botNum, rightNum, leftNum;
for (int rowNum = 0; rowNum < sumGrid.length; rowNum++) {
for (int colNum = 0; colNum < sumGrid[0].length; colNum++) {
try {
topNum = staticGrid[rowNum - 1][colNum];
} catch (Exception e) {
topNum = staticGrid[rowNum][colNum];
}
try {
botNum = staticGrid[rowNum + 1][colNum];
} catch (Exception e) {
botNum = staticGrid[rowNum][colNum];
}
try {
leftNum = staticGrid[rowNum][colNum - 1];
} catch (Exception e) {
leftNum = staticGrid[rowNum][colNum];
}
try {
rightNum = staticGrid[rowNum][colNum + 1];
} catch (Exception e) {
rightNum = staticGrid[rowNum][colNum];
}
this.sumGrid[rowNum][colNum] = topNum + botNum + rightNum + leftNum;
}
}
return this.sumGrid;
}
Thanks!

what in kafka DefaultRecord value field

Recently, I review the kafka code and test. I found a strange case:
I print the bytebuffer on the entry of SocketServer processCompletedReceives, as well as print the value on the point of Log sotre as follows:
the entry of SocketServer
private def processCompletedReceives() {
selector.completedReceives.asScala.foreach { receive =>
try {
openOrClosingChannel(receive.source) match {
case Some(channel) =>
val header = RequestHeader.parse(receive.payload)
val connectionId = receive.source
val context = new RequestContext(header, connectionId, channel.socketAddress,
channel.principal, listenerName, securityProtocol)
val req = new RequestChannel.Request(processor = id, context = context,
startTimeNanos = time.nanoseconds, memoryPool, receive.payload, requestChannel.metrics)
if(header.apiKey() == ApiKeys.PRODUCE){
LogHelper.log("produce request: %v" + java.util.Arrays.toString(receive.payload.array()))
}
...
the point of Log
validRecords.records().asScala.foreach { record =>
LogHelper.log("buffer info: value " + java.util.Arrays.toString(record.value().array()))
}
but, the result of print is different. and record.value() is not what I passed in client value like this:
public void run() {
int messageNo = 1;
while (true) {
String messageStr = "Message_" + messageNo;
long startTime = System.currentTimeMillis();
if (isAsync) { // Send asynchronously
producer.send(new ProducerRecord<>(topic,
messageNo,
messageStr), new DemoCallBack(startTime, messageNo, messageStr));
} else { // Send synchronously
try {
producer.send(new ProducerRecord<>(topic,
messageNo,
messageStr)).get();
System.out.println("Sent message: (" + messageNo + ", " + messageStr + ")");
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
}
++messageNo;
}
}
the print result is not the not String messageStr = "Message_" + messageNo;
so what happend in the case.

done. I write the code as follows:
public class KVExtractor {
private static final Logger logger = LoggerFactory.getLogger(KVExtractor.class);
public static Map.Entry<byte[], byte[]> extract(Record record) {
if (record.hasKey() && record.hasValue()) {
byte[] key = new byte[record.key().limit()];
record.key().get(key);
byte[] value = new byte[record.value().limit()];
record.value().get(value);
System.out.println("key : " + new String(key) + " value: " + new String(value));
return new AbstractMap.SimpleEntry<byte[], byte[]>(key, value);
}else if(record.hasValue()){
// illegal impl
byte[] data = new byte[record.value().limit()];
record.value().get(data);
System.out.println("no key but with value : " + new String(data));
}
return null;
}
}

twitter4j search for full 7 day range

I try to save tweets with keywords, I know that free API gives only 7 days of the result, but it never gets any set of a timeline greater than few hours, sometimes it even gives me a range of an hour. I did set since() and until() to the searching query. The maximum number of the tweets I've got from a single run was less than 400. And can anyone tell me why it stopped automatically with such few results? Thanks.
public static void main(String[] args) throws TwitterException {
String KEY_word;
String Exclude;
String Since;
String Until;
String OPT_dir;
String time;
int x;
Propertyloader confg = new Propertyloader();
KEY_word = confg.getProperty("KEY_word");
Exclude = confg.getProperty("Exclude");
Since = confg.getProperty("Since");
Until = confg.getProperty("Until");
OPT_dir = confg.getProperty("OPT_dir");
Twitter twitter = new TwitterFactory().getInstance();
try {
time = new SimpleDateFormat("yyyyMMddHHmm'.txt'").format(new Date());
x = 1;
Query query = new Query(KEY_word + Exclude);
query.since(Since);
query.until(Until);
QueryResult result;
do {
result = twitter.search(query);
List<Status> tweets = result.getTweets();
for (Status tweet : tweets) {
try {
String filedir = OPT_dir + KEY_word + time;
writeStringToFile(filedir, x + ". " + "#" + tweet.getUser().getScreenName() + ", At: " + tweet.getCreatedAt() + ", Rt= " + tweet.getRetweetCount() + ", Text: " + tweet.getText());
x += 1;
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
} while ((query = result.nextQuery()) != null);
System.exit(0);
} catch (TwitterException te) {
te.printStackTrace();
System.out.println("Failed to search tweets: " + te.getMessage());
System.exit(-1);
}
}
public static void writeStringToFile(String filePathAndName, String stringToBeWritten) throws IOException{
try
{
String filename= filePathAndName;
boolean append = true;
FileWriter fw = new FileWriter(filename,append);
fw.write(stringToBeWritten);//appends the string to the file
fw.write("\n" +"\n");
fw.close();
}
catch(IOException ioe)
{
System.err.println("IOException: " + ioe.getMessage());
}
}

You can get more tweets by using setMaxId. Here is an example :
long lowestTweetId = Long.MAX_VALUE;
x = 1;
Query query = new Query("stackoverflow");
query.since("2018-08-10");
query.until("2018-08-16");
query.setCount(100); //The number of tweets to return per page, up to a maximum of 100. Defaults to 15. https://developer.twitter.com/en/docs/tweets/search/api-reference/get-search-tweets.html
query.setResultType(Query.RECENT); // to get an order
int searchResultCount=100;
QueryResult result;
do {
result = twitter.search(query);
List<Status> tweets = result.getTweets();
for (Status tweet : tweets) {
try {
System.out.println( "#" + tweet.getUser().getScreenName() + ", At: " + tweet.getCreatedAt() );
x += 1;
if (tweet.getId() < lowestTweetId) {
lowestTweetId = tweet.getId();
query.setMaxId(lowestTweetId-1);
}
else {// each new maxid should be smaller than the other one so break here
//do whatever you want to handle it ex: break from two loops
}
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
} while (searchResultCount != 0 );

100 records at a time to udf

I have to pass record to an UDF which calls an API but as we want to do it parallely,we are using spark and thats why UDF is being developed, the problem here is that that UDF needs to take only 100 records at a time not more than that, it can't handle more than 100 records parallely, so how to ensure that only 100 record pass to it in one go please note we don't want to use count() function on whole record.
I am attaching the UDF code here,it's a generic UDF which returns array of struct.moreover if we pass 100 records in batchsize variable each time then,if suppose there are 198 records then if as we dont want to use count() we will not be knowing that its last batchsize is going to be 98.so how to handle that thing.
Guys... I have a generic UDF in which call is made for an API but before calling it creates batch of 100 firstly then only call restapi.. So the argument UDF takes are x1:string, x2:string, batchsize:integer(currently the batchsize is 100)..so in UDF until and unless the batchsize is not 100 the call will not happen.. And for each record it will return null.
So till 99th record it will return. Null but at 100th record the call will happen
[So, now the problem part:as we are taking batchsize 100 and call will take place only at 100th record. So, in condition like if we have suppose 198 record in file then 100 record will get the output but, other 98 will only return null as they will not get processed..
So please help a way around, and UDF take one record at a time, but it keep on collecting till 100th record.. I hope this clears up
public class Standardize_Address extends GenericUDF {
private static final Logger logger = LoggerFactory.getLogger(Standardize_Address.class);
private int counter = 0;
Client client = null;
private Batch batch = new Batch();
public Standardize_Address() {
client = new ClientBuilder().withUrl("https://ss-staging-public.beringmedia.com/street-address").build();
}
// StringObjectInspector streeti;
PrimitiveObjectInspector streeti;
PrimitiveObjectInspector cityi;
PrimitiveObjectInspector zipi;
PrimitiveObjectInspector statei;
PrimitiveObjectInspector batchsizei;
private ArrayList ret;
#Override
public String getDisplayString(String[] argument) {
return "My display string";
}
#Override
public ObjectInspector initialize(ObjectInspector[] args) throws UDFArgumentException {
System.out.println("under initialize");
if (args[0] == null) {
throw new UDFArgumentTypeException(0, "NO Street is mentioned");
}
if (args[1] == null) {
throw new UDFArgumentTypeException(0, "No Zip is mentioned");
}
if (args[2] == null) {
throw new UDFArgumentTypeException(0, "No city is mentioned");
}
if (args[3] == null) {
throw new UDFArgumentTypeException(0, "No State is mentioned");
}
if (args[4] == null) {
throw new UDFArgumentTypeException(0, "No batch size is mentioned");
}
/// streeti =args[0];
streeti = (PrimitiveObjectInspector)args[0];
// this.streetvalue = (StringObjectInspector) streeti;
cityi = (PrimitiveObjectInspector)args[1];
zipi = (PrimitiveObjectInspector)args[2];
statei = (PrimitiveObjectInspector)args[3];
batchsizei = (PrimitiveObjectInspector)args[4];
ret = new ArrayList();
ArrayList structFieldNames = new ArrayList();
ArrayList structFieldObjectInspectors = new ArrayList();
structFieldNames.add("Street");
structFieldNames.add("city");
structFieldNames.add("zip");
structFieldNames.add("state");
structFieldObjectInspectors.add(PrimitiveObjectInspectorFactory.writableStringObjectInspector);
structFieldObjectInspectors.add(PrimitiveObjectInspectorFactory.writableStringObjectInspector);
structFieldObjectInspectors.add(PrimitiveObjectInspectorFactory.writableStringObjectInspector);
structFieldObjectInspectors.add(PrimitiveObjectInspectorFactory.writableStringObjectInspector);
StructObjectInspector si2 = ObjectInspectorFactory.getStandardStructObjectInspector(structFieldNames,
structFieldObjectInspectors);
ListObjectInspector li2;
li2 = ObjectInspectorFactory.getStandardListObjectInspector(si2);
return li2;
}
#Override
public Object evaluate(DeferredObject[] args) throws HiveException {
ret.clear();
System.out.println("under evaluate");
// String street1 = streetvalue.getPrimitiveJavaObject(args[0].get());
Object oin = args[4].get();
System.out.println("under typecasting");
int batchsize = (Integer) batchsizei.getPrimitiveJavaObject(oin);
System.out.println("batchsize");
Object oin1 = args[0].get();
String street1 = (String) streeti.getPrimitiveJavaObject(oin1);
Object oin2 = args[1].get();
String zip1 = (String) zipi.getPrimitiveJavaObject(oin2);
Object oin3 = args[2].get();
String city1 = (String) cityi.getPrimitiveJavaObject(oin3);
Object oin4 = args[3].get();
String state1 = (String) statei.getPrimitiveJavaObject(oin4);
logger.info("address passed, street=" + street1 + ",zip=" + zip1 + ",city=" + city1 + ",state=" + state1);
counter++;
try {
System.out.println("under try");
Lookup lookup = new Lookup();
lookup.setStreet(street1);
lookup.setCity(city1);
lookup.setState(state1);
lookup.setZipCode(zip1);
lookup.setMaxCandidates(1);
batch.add(lookup);
} catch (BatchFullException ex) {
logger.error(ex.getMessage(), ex);
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
/* batch.add(lookup); */
if (counter == batchsize) {
System.out.println("under if");
try {
logger.info("batch input street " + batch.get(0).getStreet());
try {
client.send(batch);
} catch (Exception e) {
logger.error(e.getMessage(), e);
logger.warn("skipping current batch, continuing with the next batch");
batch.clear();
counter = 0;
return null;
}
Vector<Lookup> lookups = batch.getAllLookups();
for (int i = 0; i < batch.size(); i++) {
// ListObjectInspector candidates;
ArrayList<Candidate> candidates = lookups.get(i).getResult();
if (candidates.isEmpty()) {
logger.warn("Address " + i + " is invalid.\n");
continue;
}
logger.info("Address " + i + " is valid. (There is at least one candidate)");
for (Candidate candidate : candidates) {
final Components components = candidate.getComponents();
final Metadata metadata = candidate.getMetadata();
logger.info("\nCandidate " + candidate.getCandidateIndex() + ":");
logger.info("Delivery line 1: " + candidate.getDeliveryLine1());
logger.info("Last line: " + candidate.getLastLine());
logger.info("ZIP Code: " + components.getZipCode() + "-" + components.getPlus4Code());
logger.info("County: " + metadata.getCountyName());
logger.info("Latitude: " + metadata.getLatitude());
logger.info("Longitude: " + metadata.getLongitude());
}
Object[] e;
e = new Object[4];
e[0] = new Text(candidates.get(i).getComponents().getStreetName());
e[1] = new Text(candidates.get(i).getComponents().getCityName());
e[2] = new Text(candidates.get(i).getComponents().getZipCode());
e[3] = new Text(candidates.get(i).getComponents().getState());
ret.add(e);
}
counter = 0;
batch.clear();
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
return ret;
} else {
return null;
}
}
}

Kafka Java consumer works only for localhost and fails for remote server

I've been working with Kafka for two months, and I used this code to consume messages locally. I recently decided to distribute Zookeeper and Kafka and everything seems to work just fine. My issue started when I tried to use the consumer's code from a remote IP; Once I change seeds.add("127.0.0.1"); to seeds.add("104.131.40.xxx"); I get this error message:
run:
Error communicating with Broker [104.131.40.xxx] to find Leader for [temperature, 0] Reason:
java.net.ConnectException: Connection refused Can't find metadata for Topic and Partition. Exiting
BUILD SUCCESSFUL (total time: 21 seconds)r code here
this is the code that I currently use:
/*
Kafka API consumer reads 10 readings from the "temperature" topic
*/
package simpleexample;
import kafka.api.FetchRequest;
import kafka.api.FetchRequestBuilder;
import kafka.api.PartitionOffsetRequestInfo;
import kafka.common.ErrorMapping;
import kafka.common.TopicAndPartition;
import kafka.javaapi.*;
import kafka.javaapi.consumer.SimpleConsumer;
import kafka.message.MessageAndOffset;
import java.nio.ByteBuffer;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class SimpleExample {
public static void main(String args[]) {
SimpleExample example = new SimpleExample();
//long maxReads = Long.parseLong(args[0]);
long maxReads = 10;
//String topic = args[1];
String topic = "temperature";
//int partition = Integer.parseInt(args[2]);
int partition =0;
List<String> seeds = new ArrayList<String>();
//seeds.add(args[3]);
seeds.add("104.131.40.xxx");
//int port = Integer.parseInt(args[4]);
int port =9092;
try {
example.run(maxReads, topic, partition, seeds, port);
} catch (Exception e) {
System.out.println("Oops:" + e);
e.printStackTrace();
}
}
private List<String> m_replicaBrokers = new ArrayList<String>();
public SimpleExample() {
m_replicaBrokers = new ArrayList<String>();
}
public void run(long a_maxReads, String a_topic, int a_partition, List<String> a_seedBrokers, int a_port) throws Exception {
// find the meta data about the topic and partition we are interested in
//
PartitionMetadata metadata = findLeader(a_seedBrokers, a_port, a_topic, a_partition);
if (metadata == null) {
System.out.println("Can't find metadata for Topic and Partition. Exiting");
return;
}
if (metadata.leader() == null) {
System.out.println("Can't find Leader for Topic and Partition. Exiting");
return;
}
String leadBroker = metadata.leader().host();
String clientName = "Client_" + a_topic + "_" + a_partition;
SimpleConsumer consumer = new SimpleConsumer(leadBroker, a_port, 100000, 64 * 1024, clientName);
long readOffset = getLastOffset(consumer,a_topic, a_partition, kafka.api.OffsetRequest.EarliestTime(), clientName);
int numErrors = 0;
while (a_maxReads > 0) {
if (consumer == null) {
consumer = new SimpleConsumer(leadBroker, a_port, 100000, 64 * 1024, clientName);
}
FetchRequest req = new FetchRequestBuilder()
.clientId(clientName)
.addFetch(a_topic, a_partition, readOffset, 100000) // Note: this fetchSize of 100000 might need to be increased if large batches are written to Kafka
.build();
FetchResponse fetchResponse = consumer.fetch(req);
if (fetchResponse.hasError()) {
numErrors++;
// Something went wrong!
short code = fetchResponse.errorCode(a_topic, a_partition);
System.out.println("Error fetching data from the Broker:" + leadBroker + " Reason: " + code);
if (numErrors > 5) break;
if (code == ErrorMapping.OffsetOutOfRangeCode()) {
// We asked for an invalid offset. For simple case ask for the last element to reset
readOffset = getLastOffset(consumer,a_topic, a_partition, kafka.api.OffsetRequest.LatestTime(), clientName);
continue;
}
consumer.close();
consumer = null;
leadBroker = findNewLeader(leadBroker, a_topic, a_partition, a_port);
continue;
}
numErrors = 0;
long numRead = 0;
for (MessageAndOffset messageAndOffset : fetchResponse.messageSet(a_topic, a_partition)) {
long currentOffset = messageAndOffset.offset();
if (currentOffset < readOffset) {
System.out.println("Found an old offset: " + currentOffset + " Expecting: " + readOffset);
continue;
}
readOffset = messageAndOffset.nextOffset();
ByteBuffer payload = messageAndOffset.message().payload();
byte[] bytes = new byte[payload.limit()];
payload.get(bytes);
System.out.println(String.valueOf(messageAndOffset.offset()) + ": " + new String(bytes, "UTF-8"));
numRead++;
a_maxReads--;
}
if (numRead == 0) {
try {
Thread.sleep(1000);
} catch (InterruptedException ie) {
}
}
}
if (consumer != null) consumer.close();
}
public static long getLastOffset(SimpleConsumer consumer, String topic, int partition,
long whichTime, String clientName) {
TopicAndPartition topicAndPartition = new TopicAndPartition(topic, partition);
Map<TopicAndPartition, PartitionOffsetRequestInfo> requestInfo = new HashMap<TopicAndPartition, PartitionOffsetRequestInfo>();
requestInfo.put(topicAndPartition, new PartitionOffsetRequestInfo(whichTime, 1));
kafka.javaapi.OffsetRequest request = new kafka.javaapi.OffsetRequest(
requestInfo, kafka.api.OffsetRequest.CurrentVersion(), clientName);
OffsetResponse response = consumer.getOffsetsBefore(request);
if (response.hasError()) {
System.out.println("Error fetching data Offset Data the Broker. Reason: " + response.errorCode(topic, partition) );
return 0;
}
long[] offsets = response.offsets(topic, partition);
return offsets[0];
}
private String findNewLeader(String a_oldLeader, String a_topic, int a_partition, int a_port) throws Exception {
for (int i = 0; i < 3; i++) {
boolean goToSleep = false;
PartitionMetadata metadata = findLeader(m_replicaBrokers, a_port, a_topic, a_partition);
if (metadata == null) {
goToSleep = true;
} else if (metadata.leader() == null) {
goToSleep = true;
} else if (a_oldLeader.equalsIgnoreCase(metadata.leader().host()) && i == 0) {
// first time through if the leader hasn't changed give ZooKeeper a second to recover
// second time, assume the broker did recover before failover, or it was a non-Broker issue
//
goToSleep = true;
} else {
return metadata.leader().host();
}
if (goToSleep) {
try {
Thread.sleep(1000);
} catch (InterruptedException ie) {
}
}
}
System.out.println("Unable to find new leader after Broker failure. Exiting");
throw new Exception("Unable to find new leader after Broker failure. Exiting");
}
private PartitionMetadata findLeader(List<String> a_seedBrokers, int a_port, String a_topic, int a_partition) {
PartitionMetadata returnMetaData = null;
loop:
for (String seed : a_seedBrokers) {
SimpleConsumer consumer = null;
try {
consumer = new SimpleConsumer(seed, a_port, 100000, 64 * 1024, "leaderLookup");
List<String> topics = Collections.singletonList(a_topic);
TopicMetadataRequest req = new TopicMetadataRequest(topics);
kafka.javaapi.TopicMetadataResponse resp = consumer.send(req);
List<TopicMetadata> metaData = resp.topicsMetadata();
for (TopicMetadata item : metaData) {
for (PartitionMetadata part : item.partitionsMetadata()) {
if (part.partitionId() == a_partition) {
returnMetaData = part;
break loop;
}
}
}
} catch (Exception e) {
System.out.println("Error communicating with Broker [" + seed + "] to find Leader for [" + a_topic
+ ", " + a_partition + "] Reason: " + e);
} finally {
if (consumer != null) consumer.close();
}
}
if (returnMetaData != null) {
m_replicaBrokers.clear();
for (kafka.cluster.Broker replica : returnMetaData.replicas()) {
m_replicaBrokers.add(replica.host());
}
}
return returnMetaData;
}
}

You need to set the advertised.host.name instead of host.name in the kafka server.properties configuration file.