Server code:
import java.net.ServerSocket;
import java.net.Socket;
import java.nio.charset.Charset;
import java.util.Random;
public class Socket_server {
public static void main(String[] args) throws Exception {
ServerSocket sc = new ServerSocket(9990);
while (true) {
Socket socket = sc.accept();
java.io.OutputStream out = socket.getOutputStream();
String message = getRandomIntegerBetweenRange(100, 120) + "";
byte b[] = message.getBytes(Charset.defaultCharset());
out.write(b);
out.close();
socket.close();
}
}
private static double getRandomIntegerBetweenRange(double max, double min) {
double x = (int) (Math.random() * ((max - min) + 1)) + min;
return x;
}
}
Spark code:
import java.util.Collections;
import org.apache.avro.ipc.specific.Person;
import org.apache.spark.SparkConf;
import org.apache.spark.api.java.function.MapFunction;
import org.apache.spark.sql.Dataset;
import org.apache.spark.sql.Encoder;
import org.apache.spark.sql.Encoders;
import org.apache.spark.sql.Row;
import org.apache.spark.sql.SparkSession;
import org.apache.spark.sql.streaming.StreamingQuery;
import org.apache.spark.sql.streaming.Trigger;
import scala.Function1;
public class App1 {
public static void main(String[] args) throws Exception {
SparkConf conf = new SparkConf();
conf.setMaster("local[*]");
conf.setAppName("app");
SparkSession spark = SparkSession.builder().config(conf).getOrCreate();
Dataset<Row> lines = spark.readStream().format("socket").option("host", "localhost").option("port", 9990)
.load();
StreamingQuery query = lines.writeStream().format("console").start();
query.awaitTermination();
}
}
I am running server code which is generating random values and after that i am running spark Structured Streaming code to read it and create DataFrame from it. But as my spark code start it just read the first value from the server after that it does not read any further value.When i am using this same server with spark streaming then that is reading values continuously. So can anyone help in what is wrong with code.
Related
How can I count the number of lines in a file?
Below is the code I have written, but with an exception.
import java.io.File;
import java.net.URI;
import java.util.Scanner;
import java.io.FileNotFoundException;
import java.util.InputMismatchException;
public class Main {
public static void main(String[] args)throws Exception {
String[] MovieList = new String[25];
File MovieFile= new File("TheMovieList.txt");
Scanner ms = new Scanner(MovieFile);
while(true){
int i= ms.nextInt();
System.out.println();
}
}
}
And I had
Exception in thread "main" java.util.InputMismatchException
What's wrong, and how can I fix this?
You can check, that the input can be interpreted as an int value:
...
if (ms.hasNextInt()) {
int i= ms.nextInt();
...
}
To count the lines in the file you can do this.
import java.io.File;
import java.net.URI;
import java.util.Scanner;
import java.io.FileNotFoundException;
import java.util.InputMismatchException;
public class Main {
public static void main(String[] args)throws Exception {
File MovieFile= new File("TheMovieList.txt");
Scanner ms = new Scanner(MovieFile);
int count = 0;
while (ms.hasNextLine()) {
count++;
ms.nextLine();
}
System.out.println("Number of lines in the file is " + count);
}
}
I am new to the forum and I need to solve a problem which I do not see the error because the code is very clear
I have 2 projects: Server & client. In both of them I have the serialized class "ClaseServer"
package ser.des;
import java.io.Serializable;
import java.util.ArrayList;
public class ClaseServidor implements Serializable{
private static final long serialVersionUID = -73813883259606471L;
String ip;
int puerto;
ArrayList<String> coleccion;
public ClaseServidor(){
}
public ClaseServidor(String ip, int puerto, ArrayList<String> coleccion) {
super();
this.ip = ip;
this.puerto = puerto;
this.coleccion = coleccion;
}
public String getIp() {
return ip;
}
public void setIp(String ip) {
this.ip = ip;
}
public int getPuerto() {
return puerto;
}
public void setPuerto(int puerto) {
this.puerto = puerto;
}
public ArrayList<String> getColeccion() {
return coleccion;
}
public void setColeccion(ArrayList<String> coleccion) {
this.coleccion = coleccion;
}
}
The class "hiloservidor" that is responsible for receiving requests and where I read the objects sent by the client.
package servidor;
import java.io.BufferedReader;
import java.io.EOFException;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.Serializable;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.net.SocketAddress;
import java.net.SocketException;
import java.util.Scanner;
import java.util.Timer;
import java.util.TimerTask;
import java.util.logging.Level;
import java.util.logging.Logger;
public class hiloservidor extends Thread{
private Socket ss;
private int counter;
public hiloservidor(Socket i,int c){
this.ss=i;
this.counter=c;
}
#Override
public void run(){
try{
boolean done=false;
System.out.println("hello client "+counter);
while(!done){
ObjectInputStream entrada = new ObjectInputStream(ss.getInputStream());
System.out.println("reading....");
ClaseServidor cla = (ClaseServidor) entrada.readObject();
System.out.println("Done");
}
}
catch(Exception e){
}
}
}
And the client class where I send the objects
package ser.des;
import java.io.File;
import java.io.IOException;
import java.io.ObjectOutputStream;
import java.io.Serializable;
import java.net.InetAddress;
import java.net.Socket;
import java.net.UnknownHostException;
import java.util.ArrayList;
import java.util.Scanner;
public class SerDes{
public static void main(String args[]){
try {
boolean salir=false;
int i = 1;
Socket cliente = new Socket("127.0.0.1",4890);
ArrayList<String>coleccion = new ArrayList<String>();
coleccion.add("Libro1");
coleccion.add("Libro2");
coleccion.add("Libro3");
coleccion.add("Libro4");
coleccion.add("Libro5");
coleccion.add("Libro6");
ClaseServidor miServidor = new ClaseServidor( InetAddress.getLocalHost().getHostAddress(),15253,coleccion);
while(salir==false){
ObjectOutputStream msgToServer = new ObjectOutputStream(cliente.getOutputStream());
System.out.println(InetAddress.getLocalHost().getHostAddress());
msgToServer.writeObject(miServidor);
msgToServer.flush();
System.out.println("datos del servidor enviados");
salir = true;
}
}
catch(Exception e){
}
}
}
The problem is in the "readObject" line of the class "hiloservidor".
It may be a class "ClaseServer" problem but I do not see exactly what it is. If you can help me ... Thanks
So I have a few other classes like this one, I call the method in using an object in the run file. I want to write every output of every class into the same text file. However at the moment only one output is being saved to the text file, as it is overwriting each time, how do I do this using a print writer seen below?
Any guidance is much appreciated!
Class:
package cw;
import java.io.FileWriter;
import java.io.PrintWriter;
import java.util.Scanner;
import javax.swing.JFileChooser;
import java.io.IOException;
public class LineCounter {
public static void TotalLines() throws IOException {
Scanner sc = new Scanner(TextAnalyser.class.getResourceAsStream("test.txt"));
PrintWriter out = new PrintWriter(new FileWriter("C:\\Users\\Sam\\Desktop\\Report.txt"));
int linetotal = 0;
while (sc.hasNextLine()) {
sc.nextLine();
linetotal++;
}
out.println("The total number of lines in the file = " + linetotal);
out.close();
System.out.println("The total number of lines in the file = " + linetotal);
}
}
Run File:
package cw;
import java.io.FileWriter;
import java.io.PrintWriter;
import java.util.Scanner;
import javax.swing.JFileChooser;
import java.io.IOException;
public class TextAnalyser {
public static void main(String[] args) throws IOException {
Scanner sc = new Scanner(TextAnalyser.class.getResourceAsStream("test.txt"));
LineCounter Lineobject = new LineCounter();
WordCounter Wordobject = new WordCounter();
NumberCounter Numberobject = new NumberCounter();
DigitCounter Digitobject = new DigitCounter();
SpaceCounter Spaceobject = new SpaceCounter();
NumberAverage Noavgobject = new NumberAverage();
WordAverage Wordavgobject = new WordAverage();
Palindromes Palindromeobject = new Palindromes();
VowelCounter Vowelobject = new VowelCounter();
ConsonantCounter Consonantobject = new ConsonantCounter();
WordOccurenceTotal RepeatsObject = new WordOccurenceTotal();
Lineobject.TotalLines();
Wordobject.TotalWords();
Numberobject.TotalNumbers();
Digitobject.TotalDigits();
Spaceobject.TotalSpaces();
Noavgobject.NumberAverage();
Wordavgobject.WordAverage();
Vowelobject.TotalVowels();
Consonantobject.TotalConsonant();
Palindromeobject.TotalPalindromes();
//RepeatsObject.TotalRepeats();
}
}
You want to use the second argument of the FileWriter constructor to set the append mode:
new FileWriter("name_of_your_file.txt", true);
instead of:
new FileWriter("name_of_your_file.txt");
I can't find the mistake java.lang.NullPointerException in the line 'e = servidor.listarTrinos();' listaTrinosSistema is empty. Skip ServicioDatosInterface and Trino.
package Basededatos;
import java.rmi.RemoteException;
import java.rmi.registry.LocateRegistry;
import java.rmi.registry.Registry;
import java.rmi.server.UnicastRemoteObject;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import Common.Gui;
import Common.ServicioDatosInterface;
import Common.Trino;
import Common.Utils;
public class Basededatos {
private static ServicioDatosImpl servidor;
public static void main(String[] args) throws Exception {
Utils.setCodeBase(ServicioDatosInterface.class);
ServicioDatosImpl servidor = new ServicioDatosImpl();
ServicioDatosInterface remote = (ServicioDatosInterface)UnicastRemoteObject.exportObject(servidor, 8888);
Registry registry = LocateRegistry.createRegistry(Registry.REGISTRY_PORT);
registry.rebind("Pepito", remote);
menu();
}
public static void menu() throws RemoteException{
int opt = 0;
do {
opt = Gui.menu("Menu Base de Datos", new String[]{ "listarTrinos" });
switch (opt) {
case 0: listarTrinos(); break;
}
}
while (opt != 1);
}
public static void listarTrinos() throws RemoteException{
List<Trino> e;
e = servidor.listarTrinos();
System.out.print("Trinos enviados");
Iterator<Trino> nombreIterator = e.iterator();
while(nombreIterator.hasNext()){
String elemento = nombreIterator.next().toString();
System.out.print(elemento+" / ");
}
}
}
package Basededatos;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.rmi.RemoteException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import Common.ServicioDatosInterface;
import Common.Trino;
public class ServicioDatosImpl implements ServicioDatosInterface{
private List<Trino> listaTrinosSistema = new ArrayList<Trino>();
public List<Trino> listarTrinos() throws RemoteException{
//if (listaTrinosSistema.isEmpty() == true || listaTrinosSistema.size()==0) return null;
//else return listaTrinosSistema;
return listaTrinosSistema;
}
private static ServicioDatosImpl servidor;
This declares a static variable
public static void main(String[] args) throws Exception {
Utils.setCodeBase(ServicioDatosInterface.class);
ServicioDatosImpl servidor = new ServicioDatosImpl();
and this declares and initializes a local variable which has the same name as the static variable. The static variable stays null. It should be
servidor = new ServicioDatosImpl()
If you have a NPE on e = servidor.listarTrinos() then servidor is null.
This is because you have a static attribute servidor and you create a variable servidor in the main method.
Following code is working fine for localhost to remote server. But my application runs on seperate server so i need to pick files from remote1 server and then put it in remote2 server. How to do with J2SSH.
package com.test.sftp;
import java.io.File;
import java.text.DateFormat;
import java.util.Calendar;
import java.util.Date;
import java.util.Properties;
import com.sshtools.daemon.SshServer;
import com.sshtools.j2ssh.SftpClient;
import com.sshtools.j2ssh.SshClient;
import com.sshtools.j2ssh.authentication.AuthenticationProtocolState;
import com.sshtools.j2ssh.authentication.PasswordAuthenticationClient;
import com.sshtools.j2ssh.configuration.SshConnectionProperties;
import com.sshtools.j2ssh.sftp.SftpFile;
import com.sshtools.j2ssh.transport.ConsoleKnownHostsKeyVerification;
import com.sshtools.j2ssh.transport.IgnoreHostKeyVerification;
import com.sshtools.j2ssh.transport.InvalidHostFileException;
import com.sshtools.j2ssh.transport.publickey.SshPublicKey;
import java.io.BufferedInputStream;
import java.io.DataInputStream;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.MalformedURLException;
import java.net.URI;
import java.net.URL;
public class SftpTest {
/**
* #param args
*/
public String userName;
public String uri;
public Date date;
public String sdate;
public DateFormat formatter ;
public String serviceStart;
public String hostname=null;
public String username=null;
public String password=null;
public String hostname1=null;
public String username1=null;
public String password1=null;
public String remotepath=null;
public String localpath=null;
public String remoteFilename=null;
public void con()
{
SshClient ssh=null;
SftpClient sftp=null;
SftpFile sstp = null;
try
{
hostname="173.202.29.29";
username="xxxxx";
password="xxxxxxx";
SshClient client = new SshClient();
SshConnectionProperties connectionProperties = new SshConnectionProperties();
connectionProperties.setTransportProvider(SshConnectionProperties.USE_STANDARD_SOCKET); // or USE_STANDARD_SOCKET or USE_SOCKS4_PROXY or USE_SOCKS5_PROXY
connectionProperties.setProxyHost("inetgate.highmark.com");
connectionProperties.setProxyPort(22);
connectionProperties.setHost(hostname);
connectionProperties.setPort(22);
client.connect(connectionProperties, new IgnoreHostKeyVerification());
PasswordAuthenticationClient authenticationClient = new PasswordAuthenticationClient();
authenticationClient.setUsername(username);
authenticationClient.setPassword(password);
int result = client.authenticate(authenticationClient);
System.out.println("result value ::"+result);
if (result == AuthenticationProtocolState.COMPLETE)
System.out.println("success Authentication");
else
System.out.println("failed Authentication");
System.out.println(client.isConnected());
SftpClient sftpClient = client.openSftpClient();
String localpath = "C:/Documents and Settings/lidkv15/Desktop/images";
sftpClient.lcd(localpath);
//sftpClient.cd("/");
File folder = new File("C:/Documents and Settings/user/Desktop/images");
File[] listOfFiles = folder.listFiles();
for (int i = 0; i < listOfFiles.length; i++) {
if (listOfFiles[i].isFile()) {
String str ="C:/Documents and Settings/lidkv15/Desktop/images/"+listOfFiles[i].getName();
// sftpClient.put(str,"/usr/project/images/");
System.out.println("File " + listOfFiles[i].getName());
} else if (listOfFiles[i].isDirectory()) {
System.out.println("Directory " + listOfFiles[i].getName());
}
}
sftpClient.quit();
client.disconnect();
}
catch(Exception e)
{
System.out.println("Exception while connecting to the remote server" + e);
}
}
public static void main(String args[]) throws Exception
{
SftpTest obj = new SftpTest();
obj.con();// calling the function
}
}
Not sure that I understood your question ...
If you only have SFTP access to both servers, and your code runs locally (on the client), then your only option is to download the file and upload it to another server. If you have SSH access to server1, then you can create a script which will upload/download file to/from server 2.
And if your code is running on server1 and you need to upload the file from server1 to server2, then how is it different (other than the local path is different) from your current situation when you upload the code from client1 to server2?