Develop Reference

i am trying to convert .jsp to .pdf format using pd4ml. when start executing my code i am getting execption below

public static void main(String[] args) {
try {
PdfViewerStarter jt = new PdfViewerStarter();
jt.doConversion("http://pd4ml.com/sample.htm", "D:/pd4ml.pdf");
} catch (Exception e) {
e.printStackTrace();
}
}
public void doConversion(String url, String outputPath)
throws InvalidParameterException, MalformedURLException, IOException {
File output = new File(outputPath);
java.io.FileOutputStream fos = new java.io.FileOutputStream(output);
PD4ML pd4ml = new PD4ML();
pd4ml.setHtmlWidth(userSpaceWidth);
pd4ml.setPageSize(pd4ml.changePageOrientation(PD4Constants.A4));
pd4ml.setPageInsetsMM(new Insets(topValue, leftValue, bottomValue, rightValue));
pd4ml.useTTF("c:/windows/fonts", true);
pd4ml.render(new URL(url), fos);
fos.close();
if (Desktop.isDesktopSupported()) {
Desktop.getDesktop().open(output);
} else {
System.out.println("Awt Desktop is not supported!");
}
System.out.println(outputPath + "\ndone.");
}
Error:
Error. ss_css2.jar is not in the classpath. See README.txt
Exception in thread "main" java.lang.NoClassDefFoundError: org/w3c/css/sac/CSSException
at org.zefer.html.doc.PD4MLHtmlParser.o00000(Unknown Source)
at org.zefer.html.doc.PD4MLHtmlParser.<init>(Unknown Source)
at org.zefer.pd4ml.PD4ML.super(Unknown Source)
at org.zefer.pd4ml.PD4ML.render(Unknown Source)
at TestForPdfPD4ML.doConversion(TestForPdfPD4ML.java:42)
at TestForPdfPD4ML.main(TestForPdfPD4ML.java:24)
Caused by: java.lang.ClassNotFoundException: org.w3c.css.sac.CSSException
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 6 more

It may be asking for the ClassPath of the Jar File where the needed (imported) libraries are contained. If you are running from Command Line try something like this:
java -cp .;"The full path where the Jar File is\jarfile.jar" YourRunningClassFile

Is there a way to load a csv/json file into a jar externally?

I am trying to load a json file into a jar file externally. This is a small spring boot application. This json file keeps on changing, so instead of bundling this inside the jar, I want to keep it outside so that there is no need to build the jar everytime this file changes.
Right now, I have my java jar command as below:java -jar build/libs/my-customer-service.jar --app.build.number=1234 -Dloader.path=./config/PotentialCustomers.json --spring.config.location=./src/main/resources/my-customer-service-application.properties
And I am accessing this file like below:
#PostConstruct
public void init() {
try (InputStream iStream = new ClassPathResource("PotentialCustomers.json").getURL().openStream()) {
ObjectMapper am = new ObjectMapper();
Map<String,String> customerMap = am.readValue(iStream, new TypeReference<Map<String, String>>() {
});
} catch (IOException e) {
e.printStackTrace(); }
}
I am getting this exception:
java.io.FileNotFoundException: class path resource [PotentialCustomers.json] cannot be resolved to URL because it does not exist
at org.springframework.core.io.ClassPathResource.getURL(ClassPathResource.java:195)
at com.jetblue.jbms.api.util.PhoneNumBlackListChecker.init(PhoneNumBlackListChecker.java:24)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:366)

In first place, make sure you have folder structure like this ready.
./ - the root of your program
|__ your_jar_file.jar
|__ some_properties_file.properties
To access some_properties_file.properties from code inside jar, use utility function like one below.
Properties appProperties = new Properties();
FileInputStream file;
String path = "./some_properties_file.properties";
file = new FileInputStream(path);
appProperties .load(file);
String propValue= appProperties .getProperty("Property_Name");

Kerberos Authentication via JAAS context fails when system user != principal user

I need to authenticate against a Kerberos Realm with the user test. I want to do this by creating a secure JAAS context via a corresponding jaas.conf config file:
com.sun.security.jgss.krb5.initiate {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=true
principal="test#MYREALM.DE"
useKeyTab=true
keyTab="/home/ubuntu/kerberos_test/test.keytab"
storeKey=true;
};
The krb5.ini file contains the information about the Realm and the Kerberos servers:
[libdefaults]
default_realm = MYREALM.DE
[realms]
MYREALM.DE = {
admin_server = my.server.de
kdc = my.server.de
}
And here the Java Code of my Application, which authenticates against the Kerberos Realm and prints the directories in / of my HDFS:
public class KerberosAuthentication {
public static void main(String[] args) {
try {
String webHdfsUrl = "webhdfs://my.server:50070";
Configuration conf = new Configuration();
conf.set("fs.defaultFS", webHdfsUrl);
conf.set("hadoop.security.authentication", "kerberos");
FileSystem fs = FileSystem.get(conf);
FileStatus[] fsStatus = fs.listStatus(new Path("/"));
for(FileStatus status : fsStatus) {
System.out.println(status.getPath().toString());
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
When I run the code with the following arguments (in Eclipse or as java -cp ...jar):
-Djava.security.krb5.conf=/home/ubuntu/kerberos_test/krb5.ini
-Djava.security.auth.login.config=/home/ubuntu/kerberos_test/jaas.conf
-Djavax.security.auth.useSubjectCredsOnly=true
I get the following results:
System username == Principal username (test): Everything works fine, I get an output of the root directories of my HDFS
System username (e.g. ubuntu) != Principal username (test), I get the following exception:
Exception output:
java.io.IOException: Usernames not matched: name=ubuntu != expected=test
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.toIOException(WebHdfsFileSystem.java:399)
at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$600(WebHdfsFileSystem.java:98)
at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.shouldRetry(WebHdfsFileSystem.java:686)
at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:652)
at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:472)
at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:502)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:498)
at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.listStatus(WebHdfsFileSystem.java:1330)
at kerberos.KerberosAuthentication.main(KerberosAuthentication.java:31)
Caused by: org.apache.hadoop.ipc.RemoteException(java.io.IOException): Usernames not matched: name=ubuntu != expected=test
at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:374)
at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:98)
at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:623)
... 8 more
How can I authenticate with test, if my system user is another one? Is there a possibility to add some settings to the jaas.conf or the Java code?

Project throwing IOException (File Not Found) when jar is run

I've made a project in java, using Eclipse.
Here is the project structure:
When I'm running the project in Eclipse as a java application, it runs perfectly fine.
Now, I need to export it as a jar. So, I created the jar using the method described in 3rd answer on this link (answered by Fever):
Failing to run jar file from command line: “no main manifest attribute”
Here is the output of jar tf EventLogger.jar:
META-INF/MANIFEST.MF
com/
com/project/
com/project/eventLogger/
com/project/eventLogger/KafkaConsumerGroup.class
com/project/eventLogger/KafkaProducer.class
com/project/eventLogger/ConsumeConfig.class
com/project/eventLogger/ConsumerThread.class
com/project/eventLogger/Formatter.class
com/project/eventLogger/Execute.class
com/project/eventLogger/Config.class
com/project/eventLogger/package-info.class
com/project/eventLogger/ProdConfig.class
com/project/eventLogger/FormatConfig.class
resources/
resources/Config.properties
resources/ConsumerConfig.properties
resources/FormatterConfig.properties
resources/ProducerConfig.properties
resources/log4j.properties
Here is the manifest file:
Manifest-Version: 1.0
Built-By: vishrant
Class-Path: lib/json-simple-1.1.1.jar lib/junit-4.10.jar lib/hamcrest-
core-1.1.jar lib/kafka_2.9.2-0.8.2.2.jar lib/jopt-simple-3.2.jar lib/
kafka-clients-0.8.2.2.jar lib/log4j-1.2.16.jar lib/lz4-1.2.0.jar lib/
metrics-core-2.2.0.jar lib/slf4j-api-1.7.6.jar lib/snappy-java-1.1.1.
7.jar lib/slf4j-log4j12-1.6.1.jar lib/zkclient-0.3.jar lib/zookeeper-
3.4.6.jar lib/jline-0.9.94.jar lib/netty-3.7.0.Final.jar lib/scala-li
brary-2.9.2-RC3.jar
Build-Jdk: 1.8.0_74
Created-By: Maven Integration for Eclipse
Main-Class: com.project.eventLogger.Execute
and, here is the exception:
java.io.FileNotFoundException: ConsumerConfig.properties (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at java.io.FileInputStream.<init>(FileInputStream.java:93)
at com.project.eventLogger.ConsumeConfig.loadPropertiesFile(ConsumeConfig.java:34)
at com.project.eventLogger.ConsumeConfig.<clinit>(ConsumeConfig.java:42)
at com.project.eventLogger.Execute.main(Execute.java:18)
Exception in thread "main" java.lang.ExceptionInInitializerError
at com.project.eventLogger.Execute.main(Execute.java:18)
Caused by: java.lang.NullPointerException
at com.project.eventLogger.ConsumeConfig.<clinit>(ConsumeConfig.java:47)
... 1 more
Seeing the exception, it is clear that it is not able to load ConsumerConfig.properties which is being done in ConsumeConfig.java.
Here is ConsumeConfig.java:
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.HashMap;
import java.util.Properties;
/**
* #author vishrant
*
*/
public class ConsumeConfig {
public static String zookeeper;
public static String balance;
public static String bootstrap_servers;
public static String zk_session_to;
public static String zk_sync;
public static String auto_commit;
public static String[] topics;
private static String kafka_bin;
private static final String PROPERTIES_FILE_PATH = "src/main/resources/ConsumerConfig.properties";
private static Properties loadPropertiesFile() throws IOException {
Properties properties = new Properties();
InputStream in = new FileInputStream(PROPERTIES_FILE_PATH);
properties.load(in);
return properties;
}
static {
Properties property = null;
try {
property = loadPropertiesFile();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
zookeeper = property.getProperty("ZOOKEEPER");
balance = property.getProperty("BALANCE");
bootstrap_servers = property.getProperty("BOOTSTRAP_SERVERS");
zk_session_to = property.getProperty("ZK_SESSION_TO");
zk_sync = property.getProperty("ZK_SYNC_TIME");
auto_commit = property.getProperty("AUTO_COMMIT_INTERVAL");
topics = property.getProperty("CONSUMER_TOPICS").split(",");
kafka_bin = property.getProperty("KAFKA_BIN_PATH");
}
}
Can someone tell me what is the problem and how to resolve this?
This runs perfectly well when run in Eclipse itself.
EDIT1:
Now, the exception is:
Exception in thread "main" java.lang.ExceptionInInitializerError
at com.project.eventLogger.Execute.main(Execute.java:18)
Caused by: java.lang.NullPointerException
at java.util.Properties$LineReader.readLine(Properties.java:434)
at java.util.Properties.load0(Properties.java:353)
at java.util.Properties.load(Properties.java:341)
at com.project.eventLogger.ConsumeConfig.loadPropertiesFile(ConsumeConfig.java:35)
at com.project.eventLogger.ConsumeConfig.<clinit> (ConsumeConfig.java:42)
... 1 more
line no 35:
props.load(resourceStream);
This is the code now:
private static final String PROPERTIES_FILE_PATH = "ConsumerConfig.properties";
private static Properties loadPropertiesFile() throws IOException {
ClassLoader loader = Thread.currentThread().getContextClassLoader();
Properties props = new Properties();
InputStream resourceStream = loader.getResourceAsStream(PROPERTIES_FILE_PATH);
props.load(resourceStream);
return props;
}

I see the following problems to be the cause:
The jar file is not created correctly for a maven project because the directory resources will normally be not copied to the target directory as is, i.e., instead of
resources/Config.properties
it should look like without the resources directory:
Config.properties
directly under the root directory of the jar file.
The second point is you are using the following in your code
private static final String PROPERTIES_FILE_PATH = "src/main/resources/ConsumerConfig.properties";
This path will not be seen outside of your IDE (in this case Eclipse) because src/main/resources should not exist in the jar file as you could see in the list of your jar file.
The last point is, you should use getResourceAsStream() of the class loader as Vikrant Kashyap already pointed.

try this
// Change Your File Path First.
private static final String PROPERTIES_FILE_PATH = "ConsumerConfig.properties";
private static Properties loadPropertiesFile() throws IOException {
Properties properties = new Properties();
// First way to load ResourceAsStream.
// ClassLoader loader = Thread.currentThread().getContextClassLoader();
// InputStream resourceStream = loader.getResourceAsStream(PROPERTIES_FILE_PATH);
// Second way to load ResourceAsStream.
InputStream resourceStream = ConsumeConfig.class.getResourceAsStream(PROPERTIES_FILE_PATH);
properties.load(resourceStream);
return properties;
}

java.lang.NoClassDefFoundError when reading hadoop SequenceFile

I am trying to read a SequenceFile with custom Writeable in it.
Here's the code:
public static void main(String[] args) throws IOException {
//String iFile = null;
String uri = "/tmp/part-r-00000";
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(URI.create(uri), conf);
Path path = new Path(uri);
MyClass value = new MyClass();
SequenceFile.Reader reader = null;
try {
reader = new Reader(fs, path, conf);
while(reader.next(value)){
System.out.println(value.getUrl());
System.out.println(value.getHeader());
System.out.println(value.getImages().size());
break;
}
} catch (Exception e) {// Catch exception if any
System.err.println("Error: " + e.getMessage());
}
finally {
IOUtils.closeStream(reader);
}
}
When I run this, I get following exception:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/configuration/Configuration
at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<init>(DefaultMetricsSystem.java:37)
at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<clinit>(DefaultMetricsSystem.java:34)
at org.apache.hadoop.security.UgiInstrumentation.create(UgiInstrumentation.java:51)
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:196)
at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:159)
at org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:216)
at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:409)
at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:395)
at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:1418)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1319)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:226)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:109)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:210)
at com.iathao.run.site.emr.DecryptMapReduceOutput.main(DecryptMapReduceOutput.java:32)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.configuration.Configuration
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
... 14 more
All libraries are packaged into the jar file and are present. What's wrong, and how do I fix this?

The hadoop-common-*.jar has to be included for the org.apache.commons.configuration.Configuration class. Put the jar as dependencies.