Regarding time consuming calculations in java - java

I am trying to write a code for reading 120 files from a folder and performing some calculations on it. When i debug the code, it works fine, however, execution time is more than 20 mins, I am aware that this might be due to bug in the code. However, can someone look into it and suggest possible methods to reduce the execution time. Kindly let me know if I should provide further information. Thank You.
import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
public class myclass {
static int total = 1;
static int r = 0;
public static void main(String[] args) {
ArrayList<Double> mysignal = new ArrayList<Double>();
ArrayList<Double> mylist = new ArrayList<Double>();
double x;
double a;
myclass obj = new myclass();
String target_dir = "path for folder";
File dir = new File(target_dir);
File[] files = dir.listFiles();
for (File f : files) {
if (f.isFile()) {
BufferedReader inputStream = null;
try {
inputStream = new BufferedReader(new FileReader(f));
String line;
while ((line = inputStream.readLine()) != null) {
System.out.println(line);
mysignal.add(Double.valueOf(line));
total++;
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
a = obj.funtioneg(mysignal, total);
mylist.add(r, a);
System.out.println(mylist.get(r));
r++;
}
}
}
public double functioneg(ArrayList<Double> s, int N) {
ArrayList<Double> y = new ArrayList<Double>();
double sum = 0, a1 = 0;
double[] o1 = new double[N - 1];// processed signal
for (int n = 0; n < counter_main - 1; n++) {
for (int k = 0; k < 40; k++) {
if (n - k >= 0) {
a1 = s.get(n - k);
sum = sum + (a1 * a1);// energy
} else
sum = sum + 0;
}
o1[n] = sum;
sum = 0;
}
double sum1 = 0;
double avg;
for (int t = 0; t < counter_main - 1; t++) {
sum1 = sum1 + o1[t];
}
avg = sum1 / N - 1;
return (avg);
}
}

You need to close your InputStream
After reading each file in the directory (after your try - catch block) write the statement:
inputStream.close();

As andrewdleach pointed out, you should close your input stream.
Additionally you might want to try out the Java 8 function Files#walk (see this question) for more efficiently walking through the files.

First try to comment out the line:
System.out.println(line);
The output to console is slow (and I mean really slow), this line is basically duplicating the contents of each processed file to the console.
Other than that, you can as well try to accumulate time spent in the functioneq() method and/or parts of it (for example using System.nanoTime()) to find the most time consuming parts (or run under debugger and use profiling by sampling, which is the easiest profiling method and surprisingly effective - just pause the program repeatedly and see where it paused most frequently).

Related

Fastest way to find all cycles in a big directed graph

I have to find all the cycles in a directed graph where every node has to only go out to 1 node but it can have more than one come in towards it and print all the nodes that are in a cycle.
Is there anyway I can make [my code][1] run faster?
right now it runs 100k nodes at about 4s but the time-limit is 1.5s
import java.io.*;
import java.util.*;
public class Main {
public static void main (String[] args) throws IOException {
long startTime = 0;
BufferedReader input = new BufferedReader(new InputStreamReader(System.in));
BufferedWriter output = new BufferedWriter(new OutputStreamWriter(System.out));
int numOfPeople = Integer.parseInt(input.readLine());
StringTokenizer following = new StringTokenizer(input.readLine(), " ");
startTime = System.nanoTime();
int[] people = new int[numOfPeople], // index -> personID, value -> personID's friend
winningPotentials = new int[numOfPeople]; // index -> personID, value -> personID's winningPotential
Arrays.fill(winningPotentials, 50);
// adding followings of people
for (int i = 0 ; i < numOfPeople ; i++) {
people[i] = Integer.parseInt(following.nextToken()) - 1;
}
/*
SETTING WINNER POTENTIALS
*/
int numOfWinners = 0;
for (int person : people) {
if (winningPotentials[person] == 50) {
Deque<Integer> path = new ArrayDeque<>();
path.addLast(person);
while (true) {
int friend = people[person];
if (path.contains(friend)) {
// all those in a friend group are winningPot = 100
while (path.getLast() != friend) {
if (winningPotentials[path.peekLast()] != 100) {
numOfWinners++;
winningPotentials[path.peekLast()] = 100;
}
path.removeLast();
}
if (winningPotentials[path.peekLast()] != 100) {
numOfWinners++;
winningPotentials[path.peekLast()] = 100;
}
path.removeLast();
break;
}
// if friend hasn't been checked before, repeat
else {
path.addLast(friend);
person = friend;
}
}
// anyone in the path that wasnt in a friend group is winnerPot=0
for (int person2 : path)
winningPotentials[person2] = 0;
}
}
/*
PRINTING THE RESULTS
*/
StringBuilder sb = new StringBuilder();
sb.append(numOfWinners + "\n");
// print each winner
for (int i = 0 ; i < winningPotentials.length ; i++)
if (winningPotentials[i] == 100)
sb.append((i + 1) + " ");
sb.append("\nExecution Time ->\t" + ((System.nanoTime() - startTime) / 1000000) + "ms");
output.write(sb.toString());
output.flush();
output.close();
}
}
Why do you need a BufferedWriter ? Can you just not do a System.out.println(sb.toString()) ?
This can be implemented as a modified BFS algorithm.
Difference is what whenever you see a point that has already been added to the queue, and isn't the point before the one you were just at, you have found a cycle. So when you add points to the queue you add the current path to that point instead of just the point, but you also add the adjacent point (the last on the path) to a list of already found points.
I would probably wait to calculate the winning Potentials until you have found all the cycles.

Exception in thread "main" java.lang.NoClassDefFoundError on ssj-2.5.jar library

I have a problem running this java code that simulates the work of a CPU.It serves processes using round robin method . I got the code from a reference and it is exactly what i need but unfortunately its not running and throws this exception :
Exception in thread "main" java.lang.NoClassDefFoundError: optimization/Lmder_fcn
at umontreal.iro.lecuyer.probdist.StudentDist.inverseF(StudentDist.java:278)
at umontreal.iro.lecuyer.stat.Tally.confidenceIntervalStudent(Tally.java:294)
at umontreal.iro.lecuyer.stat.Tally.formatCIStudent(Tally.java:359)
at umontreal.iro.lecuyer.stat.Tally.report(Tally.java:497)
here is the code :
package timeshared;
import umontreal.iro.lecuyer.simevents.*;
import umontreal.iro.lecuyer.simprocs.*;
import umontreal.iro.lecuyer.rng.*;
import umontreal.iro.lecuyer.randvar.*;
import umontreal.iro.lecuyer.stat.Tally;
import java.io.*;
public class RoundRobinQueue {
int NumberOfTermainals = 20; // Number of terminals.
double q; // Quantum size.
double overhead = 0.001; // Amount of overhead (h).
double meanThinkingTime = 5.0; // Mean thinking time.
double alpha = 0.5; // Parameters of the Weibull service times.
double lambda = 1.0; // ''
double delta = 0.0; // ''
int N = 1100; // Total number of tasks to simulate.
int N0 = 100; // Number of tasks for warmup.
int NumberOfTasks; // Number of tasks ended so far.
RandomStream streamThink = new MRG32k3a();
RandomVariateGen genThink = new ExponentialGen(streamThink, 1.0 / meanThinkingTime);
RandomStream streamServ = new MRG32k3a("Gen. for service requirements");
RandomVariateGen genServ = new WeibullGen(streamServ, alpha, lambda, delta);
Resource server = new Resource(1, "The server");
Tally meanInRep = new Tally("Average for current run");
Tally statDiff = new Tally("Diff. on mean response times");
class Terminal extends SimProcess {
public void actions() {
double arrivTime; // Arrival time of current request.
double timeNeeded; // Server's time still needed for it.
while (NumberOfTasks < N) {
delay(genThink.nextDouble());
arrivTime = Sim.time();
timeNeeded = genServ.nextDouble();
while (timeNeeded > q) {
server.request(1);
delay(q + overhead);
timeNeeded -= q;
server.release(1);
}
server.request(1); // Here, timeNeeded <= q.
delay(timeNeeded + overhead);
server.release(1);
NumberOfTasks++;
if (NumberOfTasks > N0) meanInRep.add(Sim.time() - arrivTime);
// Take the observation if warmup is over.
}
Sim.stop(); // N tasks have now completed.
}
}
private void simulOneRun() {
SimProcess.init();
server.init();
meanInRep.init();
NumberOfTasks = 0;
for (int i = 1; i <= NumberOfTermainals; i++)
new Terminal().schedule(0.0);
Sim.start();
}
// Simulate numRuns pairs of runs and prints a confidence interval
// on the difference of perf. for q sizes q1 and q2.
public void simulateConfigs(double numRuns, double q1, double q2) {
double mean1; // To memorize average for first configuration.
for (int rep = 0; rep < numRuns; rep++) {
q = q1;
simulOneRun();
mean1 = meanInRep.average();
streamThink.resetStartSubstream();
streamServ.resetStartSubstream();
q = q2;
simulOneRun();
statDiff.add(mean1 - meanInRep.average());
streamThink.resetNextSubstream();
streamServ.resetNextSubstream();
}
statDiff.setConfidenceIntervalStudent();
System.out.println(statDiff.report(0.9, 3));
}
public static void main(String[] args) {
new RoundRobinQueue().simulateConfigs(10, 0.1, 0.2);
}
}
You are missing couple of jar files in your classpath. From the documentation it looks like you would need the following jars.
http://www-labs.iro.umontreal.ca/~simardr/ssj/examples/examples.pdf
colt.jar,Blas.jar,optimization.jar ( this one in particular for your problem),jfreechart-.jar and jcommon-.jar
i fixed the problem , the program needed an optimization library here
to work , i added the jar file and it worked fine

Correcting and Condensing Java Program

I think I've almost figured out my java program. It is designed to read a text file and find the largest integer by using 10 different threads. I'm getting this error though:
Error:(1, 8) java: class Worker is public, should be declared in a file named Worker.java
I feel my code may be more complex than it needs to be so I'm trying to figure out how to shrink it down in size while also fixing the error above. Any assistance in this matter would be greatly appreciated and please let me know if I can clarify anything. Also, does the "worker" class have to be a seperate file? I added it to the same file but getting the error above.
import java.io.BufferedReader;
import java.io.FileReader;
public class datafile {
public static void main(String[] args) {
int[] array = new int[100000];
int count;
int index = 0;
String datafile = "dataset529.txt"; //string which contains datafile
String line; //current line of text file
try (BufferedReader br = new BufferedReader(new FileReader(datafile))) { //reads in the datafile
while ((line = br.readLine()) != null) { //reads through each line
array[index++] = Integer.parseInt(line); //pulls out the number of each line and puts it in numbers[]
}
}
Thread[] threads = new Thread[10];
worker[] workers = new worker[10];
int range = array.length / 10;
for (count = 0; count < 10; count++) {
int startAt = count * range;
int endAt = startAt + range;
workers[count] = new worker(startAt, endAt, array);
}
for (count = 0; count < 10; count++) {
threads[count] = new Thread(workers[count]);
threads[count].start();
}
boolean isProcessing = false;
do {
isProcessing = false;
for (Thread t : threads) {
if (t.isAlive()) {
isProcessing = true;
break;
}
}
} while (isProcessing);
for (worker worker : workers) {
System.out.println("Max = " + worker.getMax());
}
}
}
public class worker implements Runnable {
private int startAt;
private int endAt;
private int randomNumbers[];
int max = Integer.MIN_VALUE;
public worker(int startAt, int endAt, int[] randomNumbers) {
this.startAt = startAt;
this.endAt = endAt;
this.randomNumbers = randomNumbers;
}
#Override
public void run() {
for (int index = startAt; index < endAt; index++) {
if (randomNumbers != null && randomNumbers[index] > max)
max = randomNumbers[index];
}
}
public int getMax() {
return max;
}
}
I've written a few comments but I'm going to gather them all in an answer so anyone in future can see the aggregate info:
At the end of your source for the readtextfile class (which should be ReadTextile per java naming conventions) you have too many closing braces,
} while (isProcessing);
for (Worker worker : workers) {
System.out.println("Max = " + worker.getMax());
}
}
}
}
}
The above should end on the first brace that hits the leftmost column. This is a good rule of thumb when making any Java class, if you have more than one far-left brace or your last brace isn't far-left you've probably made a mistake somewhere and should go through checking your braces.
As for your file issues You should have all your classes named following Java conventions and each class should be stored in a file called ClassName.java (case sensitive). EG:
public class ReadTextFileshould be stored in ReadTextFile.java
You can also have Worker be an inner class. To do this you could pretty much just copy the source code into the ReadTextFile class (make sure it's outside of the main method). See this tutorial on inner classes for a quick overview.
As for the rest of your question Code Review SE is the proper place to ask that, and the smart folks over there probably will provide better answers than I could. However I'd also suggest using 10 threads is probably not the most efficient way in to find the largest int in a text file (both in development and execution times).

Multithread increases calculation time - Java

I was asked to check calculation time depending on number of threads working on the problem. Therefore I had written a program that calculates integral using Monte Carlo method. I am dividing the range for number of threads. After that I stats threads, which calculate their part, and finally sum partial results to get general one.
The problem is that time of calculation increases with number of threads instead of decreasing (i7 processor, Windows 7)
A few people working on it, and we do not know why is that. I hope someone will give me an advice.
I attach code:
import java.io.File;
import java.io.FileWriter;
import java.io.PrintWriter;
import java.util.ArrayList;
import java.util.concurrent.ConcurrentLinkedQueue;
public class Runner {
private static final int MAXT = 10; // maksymalna ilość wątków
static PrintWriter outM;
static PrintWriter outMTime;
public static void main(String[] args){
double xp = 2;
double xk = 3;
filesOp();
// Wypisywanie kolumn tabeli
for(int threadNumber=1; threadNumber<=MAXT; threadNumber++){
outM.print("\t"+ threadNumber);
outMTime.print("\t"+ threadNumber);
}
double time1;
double time2;
//double startTime=System.currentTimeMillis(); // Przed wystartowaniem programu
for(int n=10000; n<=10000000; n=n*10){
System.out.println("Licze dla: " + n + " punktow.");
outM.print("\n"+n);
outMTime.print("\n"+n);
for(int threadNumber=1; threadNumber<=MAXT; threadNumber++){
outM.print("\t");
outMTime.print("\t");
time1=System.nanoTime();
multiThread(xp, xk, n, threadNumber);
time2=System.nanoTime();
outMTime.print((time2-time1)/1000000);
// czas pracy dla danej liczby wątków
}
}
outM.close();
outMTime.close();
}
public static void multiThread(double xp, double xk, int n, int threadNumber){
// Funkcja licząca całkę wielowątkowo.
// Całka do policzenia jest dzielona pomiędzy wątki
ArrayList<Thread> threadList = new ArrayList<Thread>();
ConcurrentLinkedQueue<Double> results = new ConcurrentLinkedQueue<Double>();
for(int i=0; i<threadNumber; i++){
MonteCarlo mc = new MonteCarlo( xp+(i*((xk-xp)/threadNumber)), xp+((i+1)*((xk-xp)/threadNumber)), (int)(n/threadNumber), results);
Thread t = new Thread(mc);
threadList.add(t);
t.start();
}
//for(int j=0; j<threadNumber; j++){ // pętla czeka na zakończenie wątków
for(Thread t : threadList){
try {
//while(t.isAlive()){}
//threadList.get(j).join();
t.join();
} catch (Exception e) {
e.printStackTrace();
}
}
double wynik = 0;
//for(int k=0; k<results.size(); k++){
for(double r: results){
//wynik = wynik + results.remove();
wynik= wynik + r;
}
outM.print(wynik);
}
public static void filesOp(){
File fileTemp;
fileTemp = new File("wyniki.txt");
if (fileTemp.exists()) fileTemp.delete();
fileTemp = new File("pomiary.txt");
if (fileTemp.exists()) fileTemp.delete();
try {
outM = new PrintWriter(new FileWriter("wyniki.txt", true));
outMTime = new PrintWriter(new FileWriter("pomiary.txt", true));
} catch (Exception e) {
e.printStackTrace();
}
}
}
public class MonteCarlo implements Runnable{
double xp;
double xk;
long n;
ConcurrentLinkedQueue<Double> results;
MonteCarlo(double xp, double xk, long n, ConcurrentLinkedQueue<Double> results){
this.xp=xp;
this.xk=xk;
this.n=n;
this.results=results;
}
//funkcja dla ktorej obliczamy calke
private static double func(double x) {
return x*x+3;
}
private static double funcIn(double x, double y) {
if (( y > 0) && (y <= func(x)))
return 1;
else if (( y > 0) && (y <= func(x)))
return -1;
return 0;
}
//random number from a to b
private static double randomPoint(double a, double b) {
return a + Math.random() * (b-a);
}
public void run(){
double yp, yk, calka;
int pointsIn;
yp = 0;
yk = Math.ceil(Math.max(func(xp), func(xk)));
pointsIn = 0;
for (long i=0; i<n; i++) {
pointsIn += funcIn(randomPoint(xp, xk), randomPoint(yp, yk));
}
calka = (pointsIn / (double)n) * ((xk-xp) * (yk-yp));
results.add(calka);
}
}
And the example of results:
1 2 3 4 5 6 7 8 9 10
10000 6.185818 2.821405 3.721287 3.470309 4.068365 3.604195 4.323075 4.192455 6.159694 4.239105
100000 10.994522 15.874134 34.992323 40.851124 36.199631 49.54579 45.122417 61.427132 55.845435 60.7661
1000000 108.653008 274.443662 340.274574 407.054352 437.455361 469.853467 496.849012 584.519687 571.09329 594.152023
10000000 1066.059033 2877.947652 3600.551966 4175.707089 4488.434247 5081.572093 5501.217804 6374.335759 6128.274553 6339.043475
The problem most likely lies in
private static double randomPoint(double a, double b) {
return a + Math.random() * (b-a);
}
Math.random() performs poorly under heavy contention. If you are using java 7 or later, try this instead:
private static double randomPoint(double a, double b) {
return ThreadLocalRandom.current().nextDouble(a, b);
}
Using static funtions frequently is one of the pitfalls in Multithreading.
A more general answer can be found in this post already.

Sentiment analysis using SentiWordNet

I am in desperate need for some help with the following.
For my master thesis I have to conduct a sentiment analysis on some Amazon, Twitter and Facebook data. I have saved these data in a csv document. Now I want to use SentiWordNet to obtain the polarity scores. However I'm unable to run the script provided on their website using python.
First of all I have to say that I am completely new to Java. So please don't blame me for not knowing it all. I have spent a lot of time searching on the internet for some information or tutorials with no luck. There was one topic on this site from a person with a similar problem (How to use SentiWordNet), although I came across a different problem. Whenever I run the script below, I get the following message: ImportError: No module named java.io.BufferedReader. I tried to search on the internet for a solution, but I couldn't find any. Could someone please help me out with how to run this script. For starters, I have already removed the garbage in the sentiwordnet.txt file. The pathway to the SentiWordNet.txt file is \Users\Mo\Documents\etc. This is also the pathway for the csv file. Btw I'm running this script on OSX with python 2.7.5.
Thank you so much in advance for your help!!!
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Set;
import java.util.Vector;
public class SWN3 {
private String pathToSWN = "data"+File.separator+"SentiWordNet_3.0.0.txt";
private HashMap<String, String> _dict;
public SWN3(){
_dict = new HashMap<String, String>();
HashMap<String, Vector<Double>> _temp = new HashMap<String, Vector<Double>>();
try{
BufferedReader csv = new BufferedReader(new FileReader(pathToSWN));
String line = "";
while((line = csv.readLine()) != null)
{
String[] data = line.split("\t");
Double score = Double.parseDouble(data[2])-Double.parseDouble(data[3]);
String[] words = data[4].split(" ");
for(String w:words)
{
String[] w_n = w.split("#");
w_n[0] += "#"+data[0];
int index = Integer.parseInt(w_n[1])-1;
if(_temp.containsKey(w_n[0]))
{
Vector<Double> v = _temp.get(w_n[0]);
if(index>v.size())
for(int i = v.size();i<index; i++)
v.add(0.0);
v.add(index, score);
_temp.put(w_n[0], v);
}
else
{
Vector<Double> v = new Vector<Double>();
for(int i = 0;i<index; i++)
v.add(0.0);
v.add(index, score);
_temp.put(w_n[0], v);
}
}
}
Set<String> temp = _temp.keySet();
for (Iterator<String> iterator = temp.iterator(); iterator.hasNext();) {
String word = (String) iterator.next();
Vector<Double> v = _temp.get(word);
double score = 0.0;
double sum = 0.0;
for(int i = 0; i < v.size(); i++)
score += ((double)1/(double)(i+1))*v.get(i);
for(int i = 1; i<=v.size(); i++)
sum += (double)1/(double)i;
score /= sum;
String sent = "";
if(score>=0.75)
sent = "strong_positive";
else
if(score > 0.25 && score<=0.5)
sent = "positive";
else
if(score > 0 && score>=0.25)
sent = "weak_positive";
else
if(score < 0 && score>=-0.25)
sent = "weak_negative";
else
if(score < -0.25 && score>=-0.5)
sent = "negative";
else
if(score<=-0.75)
sent = "strong_negative";
_dict.put(word, sent);
}
}
catch(Exception e){e.printStackTrace();}
}
public String extract(String word, String pos)
{
return _dict.get(word+"#"+pos);
}
}
Firstly, how are you running the class. Through command line or within an IDE such as Eclipse.
If you are using a command line you must ensure you classpath has been set properly. If you are unfamiliar with such matters I would encourage creating a java project in an IDE and running it from there as the classpath will be configured for you. Creating your first Java project

Categories

Resources