This question already has an answer here:
Java variables overwriting
(1 answer)
Closed 2 years ago.
I have task implementing Runnable, which need to populate ArrayList with custom classes extending TimerTask.
My code:
public class ClusterSessionTask implements Runnable {
#Override
public void run() {
CheckTask checkTask = null;
Map<String, String> keyspaces = getKeyspaces(cluster_name);
for (Map.Entry<String, String> key_to_colfam : keyspaces.entrySet()) {
String k = key_to_colfam.getKey();
String c = key_to_colfam.getValue();
checkTask = new CheckTask(k, c, session);
tasks.add(checkTask);
}
}
}
You can see I am initialising new CheckTask every iteration in the for loop, but I am still getting the previous elements in the list being overwritten.
See this debug info:
on the 1st iteration you can see the task a:b
But on the second one a:b is being overwritten by the new task c:d:
CheckTask is a class extending TimerTask:
class CheckTask extends TimerTask {
private static int i = 0;
private static String keyspace;
private static String colfam;
private static CqlSession session;
CheckTask(String k, String c, CqlSession s) {
keyspace = k;
colfam = c;
session = s;
}
public void run() {
...
}
#Override
public String toString() {
return keyspace + ":" + colfam;
}
}
What am I doing wrong? it is somehow passing the object by reference and the new commend being ignored?
Thanks
Problem is the static keyword for keyspace,colfam,session in the constructor.
Even if you create 2 instances , they would share same value
Related
I have an array of int with size 4, only one thread can access an array cell at a time.
I thought about using Semaphore but I don't know how or if there is a way to get the acquired index
I build a code example to explain butter:
public class Temp {
private ExecutorService executeService;
private Semaphore semaphore;
private int[] syncArray; // only one thread can access an array cell at the same time
public Temp() {
syncArray = new int[]{1,2,3,4};
executeService = Executors.newFixedThreadPool(10);
semaphore = new Semaphore(syncArray.length, true);
for(int i = 0;i < 100; i++) {
executeService.submit(new Runnable() {
#Override
public void run() {
semaphore.acquire();
// here I want to access one of the array cell
// dose not matter witch one as long as no other thread is currently use it
int syncArrayIndex = semaphore.getAcquiredIndex(); // is something like this possible?
syncArray[syncArrayIndex] += ...;
semaphore.release();
}
});
}
}
}
Edit:
this is a piece of code that looks closer the my real problem:
public class Temp {
private ExecutorService executeService;
private Semaphore semaphore;
private static ChromeDriver driver;
public Temp() {
executeService = Executors.newFixedThreadPool(10);
}
public Future<WikiPage> getWikiPage(String url) {
executeService.submit(new PageRequest(url) {
});
}
private static class PageRequest implements Callable<WikiPage> {
String url;
public PageRequest(String url) {
this.url = url;
}
#Override
public WikiPage call() throws Exception {
String html = "";
synchronized (driver) {
html = ...// get the wiki page, this part takes a log time
};
WikiPage ret = ...// parse the data to the WikiPage class
// this part takes less time but depend on the sync block above
return ret;
}
}
}
#Kayaman I'm not sure I understand your comment, the problem is that I return a future. Do you have a any suggestions on how to improve my code to run faster?
No, semaphore isn't useful here. It only knows about how many permits it has, there are no "indices" in a semaphore.
You can use AtomicIntegerArray instead, although if you explain your root problem, there may be a more suitable class to use.
I'm Trying to read an Avro file and perform some operations on it, everything works fine but the aggregation functions, when I use them it get the below exception :
aggregating on field positions is only possible on tuple data types
then I change my class to implement Tuple4 (as I have 4 fields) but then when I want to collect the results get AvroTypeException Unknown Type : T0
Here are my data and job classes :
public class Nation{
public Integer N_NATIONKEY;
public String N_NAME;
public Integer N_REGIONKEY;
public String N_COMMENT;
public Integer getN_NATIONKEY() {
return N_NATIONKEY;
}
public void setN_NATIONKEY(Integer n_NATIONKEY) {
N_NATIONKEY = n_NATIONKEY;
}
public String getN_NAME() {
return N_NAME;
}
public void setN_NAME(String n_NAME) {
N_NAME = n_NAME;
}
public Integer getN_REGIONKEY() {
return N_REGIONKEY;
}
public void setN_REGIONKEY(Integer n_REGIONKEY) {
N_REGIONKEY = n_REGIONKEY;
}
public String getN_COMMENT() {
return N_COMMENT;
}
public void setN_COMMENT(String n_COMMENT) {
N_COMMENT = n_COMMENT;
}
public Nation() {
}
public static void main(String[] args) throws Exception {
Configuration parameters = new Configuration();
final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
Path path2 = new Path("/Users/violet/Desktop/nation.avro");
AvroInputFormat<Nation> format = new AvroInputFormat<Nation>(path2,Nation.class);
format.configure(parameters);
DataSet<Nation> nation = env.createInput(format);
nation.aggregate(Aggregations.SUM,0);
JobExecutionResult res = env.execute();
}
and here's the tuple class and the same code for the job as above:
public class NationTuple extends Tuple4<Integer,String,Integer,String> {
Integer N_NATIONKEY(){ return this.f0;}
String N_NAME(){return this.f1;}
Integer N_REGIONKEY(){ return this.f2;}
String N_COMMENT(){ return this.f3;}
}
I tried with this class and got the TypeException (Used NationTuple everywhere instead of Nation)
I don't think having your class implementing Tuple4 is right way to go. Instead you should add to your topology a MapFunction that converts your NationTuple to Tuple4.
static Tuple4<Integer, String, Integer, String> toTuple(Nation nation) {
return Tuple4.of(nation.N_NATIONKEY, ...);
}
And then in your topology call:
inputData.map(p -> toTuple(p)).returns(new TypeHint<Tuple4<Integer, String, Integer, String>(){});
The only subtle part is that you need to provide a type hint so flink can figure out what kind of tuple your function returns.
Another solution is to use field names instead of tuple field indices when doing your aggregation. For example:
groupBy("N_NATIONKEY", "N_REGIONKEY")
This is all explained here: https://ci.apache.org/projects/flink/flink-docs-stable/dev/api_concepts.html#specifying-keys
Basically, i have a class where i have my arrays in, which is like this
public final class DepotDatabase {
private Driver[] arrayDrivers;
public DepotDatabase() {
arrayDrivers = new Driver[4];
arrayDrivers[0] = new Driver(1234, 1234, 0); // sample driver
arrayDrivers[1] = new Driver(4444, 4444, 0); // sample driver
arrayDrivers[2] = new Driver(1337, 1337, 1); // sample manager
arrayDrivers[3] = new Driver(1234, 1234, 0); // sample driver
}
and i want to print this array in another class, i did set up the array in another class
public Driver(int username, int password, int managerCheck) {
this.username = username;
this.password = password;
this.managerCheck = managerCheck;
}
but now i want to be able to print out all the drivers, but in another class which will be called ViewDrivers or something similar
You can create a method inside DepotDatabase to print the array, then create an object from and call print method.
public final class DepotDatabase {
private Driver[] arrayDrivers;
public void printArray() {
for (int i = 0; i < arrayDrivers.length; i++) {
Driver d = arrayDrivers[i];
System.out.println("Username : " + d.getUsername());
System.out.println("Password : " + d.getPassword());
System.out.println(" Manager Check: " + d.getManagerCheck());
}
}
the from the test class you can do:
public void execute() {
DepotDatabase ddb = new DepotDatabase();
ddb.printArray();
}
That's why you'll need to have getters and setters. You should have:
public Driver[] getDrivers() {
return arrayDrivers;
}
and in the other class, you simply call it (and print it or whatever).
Read this tutorial.
If you plan to print your array in another class you show create an accessor to it.
The common pattern for Java is to use "get plus name off attribute", getDrivers() you should also avoid the class name in such geter as it may changed due to application life.
public final class DepotDatabase {
//your code
public Driver[] getDrivers() {
return this.arrayDrivers;
}
}
Next question to answer is a returning the whole array is good idea. When you return it as above you loose control on it. And every one that call that method will be able to change the content of it.
To prevent this you should use so called Defensive copying
public Driver[] getDrivers() {
return Arrays.copyOf(arrayDrivers, arrayDrivers.length);
}
Then person will get an copy of it an will not harm your class.
The issue with this is that consumer of your class will have to call this method every time to get fresh list of cars.
To solve this issue you may want to user the [collection framework] where instead of array you cold define:
List<Driver> drivers new ArrayList<>();
and provide the drivers as [immutable] list
public Iterable<Driver> getDrivers() {
return java.util.Collections.unmodifiableList(drivers);
}
Iterable is an interface, that allow you to obtain an interator the the list consumer of class wold have possibility to traverse it. IF you wan to allow him to check that list contains some driver you can set the return type as Collection
class Storage {
private String items[] = new String[10];
public String[] getItems() {
return Arrays.copyOf(items, items.length);
}
}
class Store {
Storage storage = new Storage();
private void printStorage() {
String[] items = storage.getItems();
for (String item : items) {
}
}
}
I have facing thread issue in the below code.When then thread executes the Run method of the runnable object,it doesnt print the data that I expect it to be.
code 1--calling code
Map<String,Object> logData = CPEMethodData.getLogDataMap();
CatalogUpdaterLogger.getLogger().info("6 before new splunk logger log data =" + logData);
CatalogrLogger writer = new CatalogLogger(LogType.INFO,logData,LoggerType.CATALOGUPDATER);
LogPool.INSTANCE.submitTask(writer);//submitting writer which is a runnable object to the queue
//add one more task/writer to the queue in the same method
logData = CPEMethodData.getLogDataMap();
CatalogUpdaterLogger.getLogger().info("11 before 3rd writer=logData "+logData);
CatalogLogger writer2 = new CatalogLogger(LogType.INFO,logData,LoggerType.CATALOGUPDATER);
LogPool.INSTANCE.submitTask(writer2);
In the above code,I have checked that logData Returned by the CPEMethodData.getLogDataMap()is different which I expected.But still when the runnable object actually executes,it runs with same data...
code 2--creating thread pool with 5 threads...
public enum LogPool {
INSTANCE;
private static final int nThreads = 5;
final ExecutorService executor = Executors.newFixedThreadPool(nThreads);
public synchronized void submitTask(Runnable task) {
executor.execute(task);
}
Code 3--runnable code
public class CatalogLogger implements Runnable {
protected LogType logType;
protected LoggerType loggerType;
protected Map<String, Object> logData;
public CatalogLogger(LogType logType, Map<String, Object> logData,
LoggerType loggerType) {
this.logType = logType;
this.logData = logData;
this.loggerType = loggerType;
}
public void run() {
System.out.println("running with logData " + logData);
System.out.println(" Thread.currentThread().hashCode() " +Thread.currentThread().hashCode());
switch (loggerType) {
case ORDERPROCESSING:
logData(Logger.getLogger(ORDER_LOG));
break;
case CATALOGUPDATER:
logData(Logger.getLogger(CATALOGUPDATER_LOG));
break;
}
}
Below is the CPEmethoddata.getLogData
public class CPEMethodData {
private static ThreadLocal<Map<String, Object>> logDataMap = new ThreadLocal<Map<String, Object>>();
public static Map<String,Object> getLogDataMap() {
return logDataMap.get();
}
public static void setOppParameters(Map<String, Object> inputParams) {
Map<String, Object> oppStatus = logDataMap.get();
if (oppStatus == null) {
oppStatus = new HashMap<String, Object>();
logDataMap.set(oppStatus);
}
oppStatus.put(INPUT_PARAMS, inputParams);
}
#SuppressWarnings("unchecked")
public static Map<String, Object> getOperationParameters() {
Map<String, Object> oppStatus = logDataMap.get();
if (oppStatus != null)
return (Map<String, Object>) oppStatus.get(INPUT_PARAMS);
return null;
}
}
when I run the code 1 which submits two runnable to the queue,I expect to see different logData content in the sysout of the run method but as i have debugged it I see that data is same in both the executions...seems that 2nd runnable is interfering with the first one....Can anyone please help me to understand what is the problem here.I thought I am passing 2 different instances of CatalogLogger and shouldnt cause any problem..Also can anyone please suggest any solution for this ?
As written by the #ReneLink in the comment to my question ,CPEMethodData.getLogDataMap was returning same instance of the hashmap...So by the time thread's run method was getting executed hashmap's content were getting modified.I created deep copy of the hashmap using Cloner facility and passed the same to the thread.
Thanks #ReneLink for pointing out this to me.
I had posted somewhat similar question before also. I got clarification for my doubts as well. But still I need something more. The Hashmap will be initialized with the enum object as the key and a threadpool instance as the value. I am confused as of how to initialize the HashMap for every object been called by some other process ..To make clear :
My program, MyThreadpoolExcecutorPgm.java initializes a HashMap
My Progran AdditionHandler.java requests a thread from the HashMap by passing ThreadpoolName (enum). I am getting "No thread available from HashMap" message. Please do help me.
Below given is my code:
public class MyThreadpoolExcecutorPgm {
enum ThreadpoolName {
DR, BR, SV, MISCELLENEOUS;
}
private static String threadName;
private static HashMap<ThreadpoolName, ThreadPoolExecutor>
threadpoolExecutorHash;
public MyThreadpoolExcecutorPgm(String p_threadName) {
threadName = p_threadName;
}
public static void fillthreadpoolExecutorHash() {
int poolsize = 3;
int maxpoolsize = 3;
long keepAliveTime = 10;
ThreadPoolExecutor tp = null;
threadpoolExecutorHash = new HashMap<ThreadpoolName, ThreadPoolExecutor>();
for (ThreadpoolName poolName : ThreadpoolName.) // failing to implement
{
tp = new ThreadPoolExecutor(poolsize, maxpoolsize, keepAliveTime,
TimeUnit.SECONDS, new ArrayBlockingQueue<Runnable>(5));
threadpoolExecutorHash.put(poolName, tp);
}
}
public static ThreadPoolExecutor getThreadpoolExcecutor(
ThreadpoolName poolName) {
ThreadPoolExecutor thread = null;
if (threadpoolExecutorHash != null && poolName != null) {
thread = threadpoolExecutorHash.get(poolName);
} else {
System.out.println("No thread available from HashMap");
}
return thread;
}
}
AdditionHandler.java
public class AdditionHandler{
public void handle() {
AddProcess setObj = new AddProcess(5, 20);
ThreadPoolExecutor tpe = null;
ThreadpoolName poolName =ThreadpoolName.DR; //i am using my enum
tpe = MyThreadpoolExcecutorPgm.getThreadpoolExcecutor(poolName);
tpe.execute(setObj);
}
public static void main(String[] args) {
AdditionHandler obj = new AdditionHandler();
obj.handle();
}
}
I suspect you're just looking for the static values() method which is added to every enum:
for (ThreadpoolName poolName : ThreadpoolName.getValues())
Alternatively, you can use EnumSet.allOf():
for (ThreadpoolName poolName : EnumSet.allOf(ThreadpoolName.class))
(As Bozho says, EnumMap is a good alternative here. You still need to loop through the enum values.)
First, you'd better use EnumMap. Then make sure you have filled the map before you invoked the method.
You can iterate through enum values by one of (in descending order of preference)
for(Enum value : Enum.values())
for(Enum value : EnumSet.allOf(Enum.class))
for(Enum value : Enum.class.getEnumConstants())
But you should also be using an EnumMap.