i am trying to read values from properties file and
when i tried to run this program
its giving the output as
null
import java.io.FileInputStream;
import java.util.Properties;
public class JavaApplication1 {
final private static String osName = System.getProperty("os.name");
static final Properties configFile = new Properties() {
{
try {
configFile.load(new FileInputStream("config.properties"));
} catch (Exception e) {
}
}
};
private static String DIR = osName.equals("Linux") ? configFile.getProperty("tempDirForLinux") : configFile.getProperty("tempDirForWindows");
public static void main(String[] args) {
System.out.println(DIR);
}
}
The part that is a bit odd in your example is where you create an anonymous Properties class and then load the properties into that same class in an initialization statement. I'm not sure how that is meant to work (and I'm guessing doesn't)
This is probably what you want rather
public class JavaApplication1 {
final private static String osName = System.getProperty("os.name");
static final Properties configFile = new Properties();
static {
try {
configFile.load(new FileInputStream("config.properties"));
} catch (Exception e) {
e.printStackTrace();
}
};
private static String DIR = osName.equals("Linux") ? configFile.getProperty("tempDirForLinux") : configFile.getProperty("tempDirForWindows");
public static void main(String[] args) throws IOException {
System.out.println(DIR);
}
}
Related
if properties file contains below type values how to read it directly into map
user={'name':'test','age':'23','place':'london'}.
Thanks in advance!
You can inject values into a Map from the properties file using the #Value annotation like this.
#Value("#{${user}}")
private Map<String, String> user;
Entry in your application.properties must be:
user = {"name":"test","age":"23","place":"London"}
test.properties file
name=test
age=23
place=london
code
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.Enumeration;
import java.util.Properties;
public class ReadPropertiesFile {
public static void main(String[] args) {
try {
File file = new File("test.properties");
FileInputStream fileInput = new FileInputStream(file);
Properties properties = new Properties();
properties.load(fileInput);
fileInput.close();
Enumeration enuKeys = properties.keys();
while (enuKeys.hasMoreElements()) {
String key = (String) enuKeys.nextElement();
String value = properties.getProperty(key);
System.out.println(key + ": " + value);
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Hope, this would help.. :)
This will read a property file and put it in Properties object.
import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;
public class PropertiesConfig {
private Properties prop;
private PropertiesConfig() {
super();
init();
}
private static class PropertiesInstance {
private static PropertiesConfig instance = null;
public static PropertiesConfig getInstance() {
if (null == instance) {
instance = new PropertiesConfig();
}
return instance;
}
}
public static PropertiesConfig getInstance() {
return PropertiesInstance.getInstance();
}
private void init() {
prop = new Properties();
try (InputStream input = getClass().getResourceAsStream("/sample_config.properties")) {
// load a properties file
prop.load(input);
} catch (IOException ex) {
ex.printStackTrace();
}
}
public String getProperty(String key) {
return prop.getProperty(key);
}
}
Now you can use any library to convert a value to Map as your value looks like a JSON.
Code example to achieve this through Jackson:
public static Map < String, Object > covertFromJsonToMap(String json) throws JsonTransformException {
try {
return mapper.readValue(json, new TypeReference < HashMap < String, Object >> () {});
} catch (Exception e) {
LOGGER.error("Error " + json, e);
throw new JsonTransformException("Error in json parse", e);
}
}
So something like this will do:
covertFromJsonToMap(PropertiesConfig.getInstance().get("user"));
Looks like you have a JSON representation of your map as a value. Once you read the value as a String in Java you can use Gson to convert it to map
Gson gson = new Gson();
String json = <YOUR_STRING>
Map<String,Object> map = new HashMap<String,Object>();
map = (Map<String,Object>) gson.fromJson(json, map.getClass());
I currently have the following method which allows me to pick properties that have been defined in my resources/config.properties file
private final Properties properties = new Properties();
{
final ClassLoader loader = getClass().getClassLoader();
try(InputStream config = loader.getResourceAsStream("Resources/config.properties")){
properties.load(config);
} catch(IOException e){
throw new IOError(e);
}
}
But I now want to pick my properties from the classpath so I have moved my config.properties from resources and placed it directly under src. But I'm struggling to know what changes I now need to make to my method to enable me to read from classpath.
Check example here
import java.io.IOError;
import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;
public class ReaderProp {
public final Properties properties = new Properties();
public ReaderProp()
{
final ClassLoader loader = getClass().getClassLoader();
try(InputStream config = loader.getResourceAsStream("error.properties")){
properties.load(config);
} catch(IOException e){
throw new IOError(e);
}
}
public static void main(String[] args) {
// TODO Auto-generated method stub
ReaderProp readerProp = new ReaderProp();
System.out.println(readerProp.properties.get("E1000_SE_ERROR-CODE"));// output E1000
}
}
Check error.properties
======================
E1000_SE_ERROR-CODE = E1000
//Find another answer
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;
public class ReaderProp {
private final Properties configProp = new Properties();
/**
* read property file
*
* #param propertyName
* #param path
*
* #throws IOException
*/
public ReaderProp(String propertyName, String path) {
try {
InputStream in;
File file;
if (path.equals("")) {
in = this.getClass().getClassLoader().getResourceAsStream(propertyName);
}
else {
file = new File(path + propertyName);
in = new FileInputStream(file);
}
configProp.load(in);
}
catch (IOException e) {
}
}
public static void main(String[] args) {
ReaderProp readerProp = new ReaderProp("error.properties",new File("").getAbsolutePath()+"\\src\\");
System.out.println(readerProp.configProp.get("E1000_SE_ERROR-CODE"));// output E1000
}
}
I am trying to read dat from kafka topic and writing it to HDFS filesystem, I buid my project using the apex malhar from [ https://github.com/apache/apex-malhar/tree/master/examples/kafka].
Unfortunally, after setting up the kafka properties and hadoop config the data are not created in my hdfs 2.6.0 system.
PS: the console dosen't show any error and everything seems to work fine
here the code I am using for my app
public class TestConsumer {
public static void main(String[] args) {
Consumer consumerThread = new Consumer(KafkaProperties.TOPIC);
consumerThread.start();
ApplicationTest a = new ApplicationTest();
try {
a.testApplication();
} catch (Exception e) {
e.printStackTrace();
}
}
}
Here the ApplicationTest class example from apex malhar
package org.apache.apex.examples.kafka.kafka2hdfs;
import org.apache.log4j.Logger;
import javax.validation.ConstraintViolationException;
import org.junit.Rule;
import org.apache.apex.malhar.kafka.AbstractKafkaInputOperator;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.net.NetUtils;
import com.datatorrent.api.LocalMode;
import info.batey.kafka.unit.KafkaUnitRule;
/**
* Test the DAG declaration in local mode.
*/
public class ApplicationTest
{
private static final Logger LOG = Logger.getLogger(ApplicationTest.class);
private static final String TOPIC = "kafka2hdfs";
private static final int zkPort = NetUtils.getFreeSocketPort();
private static final int brokerPort = NetUtils.getFreeSocketPort();
private static final String BROKER = "localhost:" + brokerPort;
private static final String FILE_NAME = "test";
private static final String FILE_DIR = "./target/tmp/FromKafka";
// broker port must match properties.xml
#Rule
private static KafkaUnitRule kafkaUnitRule = new KafkaUnitRule(zkPort, brokerPort);
public void testApplication() throws Exception
{
try {
// run app asynchronously; terminate after results are checked
LocalMode.Controller lc = asyncRun();
lc.shutdown();
} catch (ConstraintViolationException e) {
LOG.error("constraint violations: " + e.getConstraintViolations());
}
}
private Configuration getConfig()
{
Configuration conf = new Configuration(false);
String pre = "dt.operator.kafkaIn.prop.";
conf.setEnum(pre + "initialOffset", AbstractKafkaInputOperator.InitialOffset.EARLIEST);
conf.setInt(pre + "initialPartitionCount", 1);
conf.set(pre + "topics", TOPIC);
conf.set(pre + "clusters", BROKER);
pre = "dt.operator.fileOut.prop.";
conf.set(pre + "filePath", FILE_DIR);
conf.set(pre + "baseName", FILE_NAME);
conf.setInt(pre + "maxLength", 40);
conf.setInt(pre + "rotationWindows", 3);
return conf;
}
private LocalMode.Controller asyncRun() throws Exception
{
Configuration conf = getConfig();
LocalMode lma = LocalMode.newInstance();
lma.prepareDAG(new KafkaApp(), conf);
LocalMode.Controller lc = lma.getController();
lc.runAsync();
return lc;
}
}
After runAsync and before shutdown, you would need to wait for the expected results (otherwise the DAG will exit immediately). That's actually what happens in the example.
I am trying to make a base file plugin which other threads will inherit. But I am stuck at a point where the file exists and can be read from a normal thread but when I try to read that file from an abstract Base file, it says File not found. Here's my base class :-
package com.evol.fp;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.PrintWriter;
public abstract class BaseFilePlugin extends Thread implements BaseFileReader{
String filename = "";
File file = null;
boolean fileStarted = false;
boolean fileEnded = false;
public BaseFilePlugin() {
file = new File(filename);
}
public void readFile() {
BufferedReader br = null;
System.out.println("Base call: " + filename);
try {
System.out.println("inbside ");
br = new BufferedReader(new InputStreamReader(new FileInputStream(file)));
if(br.readLine().trim().isEmpty()) {
endFile();
return;
} else {
startFile(filename);
String record;
while((record = br.readLine().trim()) != null) {
parseRecord(record);
}
endFile();
}
} catch(Exception ioe) {
ioe.printStackTrace();
} finally {
try {
br.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
public abstract void parseRecord(String record);
public void startFile(String filename) {
this.fileStarted = true;
this.fileEnded = false;
}
public void endFile() {
file.delete();
this.fileEnded = true;
this.fileStarted = false;
}
public void run() {
while(true) {
System.out.println("Inside run, fileName: " + filename);
System.out.println("Filestarted: " + fileStarted + ", file exists: " + file.exists());
if(!fileStarted) {
readFile();
}
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
/**
* #return the filename
*/
public String getFilename() {
return filename;
}
/**
* #param filename the filename to set
*/
public void setFilename(String filename) {
this.filename = filename;
}
}
I am aware of multithreading but never implemented with base class to parse records from a file, if someone tells me what's the problem that will be great. I know that the file exists for sure. Here's my child class: -
package com.evol.fp;
public class FileReaderThread extends BaseFilePlugin {
public FileReaderThread() {
super.setFilename("E:\\soham\\soham.txt");
}
#Override
public void parseRecord(String record) {
System.out.println(record);
}
}
But its not calling the child's parseRecord method, using a simple main method:-
package com.evol.fp;
public class StartProcess {
public static void main(String[] args) {
FileReaderThread thd = new FileReaderThread();
thd.start();
}
}
I think it's because the parent constructor (BaseFilePlugin.class) is called first before you set your filename in super.setFile("E:\\soham\\soham.txt");
If you can remove the parent constructor instead and replace your setFileName into setFile where file is iniatilize .e.g
// public BaseFilePlugin() {
// file = new File(filename);
// }
....
....
/**
* #return the file
*/
public String getFile() {
return file
}
/**
* #param file the file to set
*/
public void setFile(String file) {
file = new File(file);
}
and in your subclass
public FileReaderThread() {
super.setFile("E:\\soham\\soham.txt");
}
BaseFilePlugin's constructor creates its file with an empty string since initially String filename = "";.
The client calls setFilename(...) which updates filename. However, file is still the same instance when the object was first created (which is using an empty string as the file name).
I would suggest to pass the file name as part of the constructor so file is properly initialized:
public BaseFilePlugin(String filename) {
this.filename = filename;
file = new File(filename);
}
Optionally, if it makes sense that an instance can read only 1 file, then make those class attributes final, and remove the setFilename() method.
I have MyLogger class which contains:
public class MyLogger {
private final static Logger logger = Logger.getLogger(MyLogger.class.getName());
private static FileHandler fileHandler;
private static String loggerFileName;
public MyLogger() {
}
public static void createMyLogger(String filename){
loggerFileName = filename;
try {
File loggerFile = new File(filename);
boolean fileExists = loggerFile.exists();
if(fileExists){
loggerFile.delete();
File lockFile = new File(filename+".lck");
if(lockFile.exists())
lockFile.delete();
}
fileHandler = new FileHandler(filename,true);
} catch (IOException e) {
e.printStackTrace();
}
logger.addHandler(fileHandler);
SimpleFormatter simpleFormatter = new SimpleFormatter();
fileHandler.setFormatter(simpleFormatter);
}
public static void log(String msg) {
logger.log(Level.INFO, msg);
}
public static void log(Exception ex) {
logger.log(Level.SEVERE, "Exception",ex);
}
public static String getLoggerFileName() {
return loggerFileName;
}
public static void setLoggerFileName(String loggerFileName) {
MyLogger.loggerFileName = loggerFileName;
}
And I am having my further execution in threads, i.e. When I am starting first process then Logger File is created and logs are recorded, but when I starts another process then again different thread is created and also new logger file is created but because of static methods and reference It mixed up the both process logs in both logger files...
When I start process for every thread following method is called:
public void start(String process) {
try{
String filename = process.replace(".com", "");
MyLogger.createXPathLogger(filename.concat(WordUtils.capitalize(type))+ ".log");
MyLogger.log("got parameters ===>> process : "+process);
}catch (Exception e) {
MyLogger.log("Exception In main() method....");
MyLogger.log("*****"+process+" process failed In main() method.*****");
MyLogger.log(e);
}
}
So what can I do for this, Ho can I do thread safe logging?
thanks in advance..
The idea here won't really work. One logger gets created and you add the each handler to the logger.
What you really need is a concept of a log context. You need to isolate the creation of the logger on a per-thread basis which AFAIK you can't do with J.U.L.
This is a very rough around the edges example of using JBoss Log Manager to do this:
import java.io.File;
import java.io.IOException;
import java.util.logging.FileHandler;
import java.util.logging.Level;
import java.util.logging.Logger;
import java.util.logging.SimpleFormatter;
import org.jboss.logmanager.LogContext;
public class MyLogger {
private static final ThreadLocal<MyLogger> LOCAL_LOGGER = new ThreadLocal<MyLogger>();
private final Logger logger;
private MyLogger(final Logger logger) {
this.logger = logger;
}
public static MyLogger createMyLogger(final String filename) {
MyLogger result = LOCAL_LOGGER.get();
if (result == null) {
final LogContext logContext = LogContext.create();
final Logger logger = logContext.getLogger(MyLogger.class.getName());
final FileHandler fileHandler;
try {
final File loggerFile = new File(filename);
if (loggerFile.exists()) {
loggerFile.delete();
File lockFile = new File(filename + ".lck");
if (lockFile.exists())
lockFile.delete();
}
fileHandler = new FileHandler(filename, true);
} catch (IOException e) {
throw new RuntimeException("Could not create file handler", e);
}
logger.addHandler(fileHandler);
fileHandler.setFormatter(new SimpleFormatter());
final MyLogger myLogger = new MyLogger(logger);
LOCAL_LOGGER.set(myLogger);
result = myLogger;
}
return result;
}
public void log(String msg) {
logger.log(Level.INFO, msg);
}
public void log(Exception ex) {
logger.log(Level.SEVERE, "Exception", ex);
}
}