Why is Jenkins having problems with UTF 8 - java

I have a Jenkins job running. I just want to get all files. In every file name there is a Chinese letter. So the problem is now that Jenkins has problems reading in those files. Jenkins makes just "?" out of the Asian letter. The second problem is. Actually it is more than 100 files. But Jenkins only gives me 20 files. Maybe now a lot of files will look the same because of the question mark "?" .
Does anyone know how I can fix this. The problem only occurs on Jenkins ( running on Linux ) . On my local machine in Eclipse it works though.
File resourcePath = new File("resources/china_data/");
File[] files = resourcePath.listFiles();
for (final File file : files)
{
System.out.console(file.getName);
}

An alternative solution is to use the new java.nio.Path api in place of the java.io.File api
Also try setting the below in your code initially.
System.setProperty("sun.jnu.encoding","utf-8");
System.setProperty("file.encoding","UTF-8");

Assuming you are using System.out.println, this happens when the program runs with an ASCII locale:
$ cat Main.java
import java.util.*;
import java.io.*;
class Main {
public static void main(String[] args) throws Exception {
File resourcePath = new File("resources/china_data/");
File[] files = resourcePath.listFiles();
for (final File file : files)
{
System.out.println(file.getName());
}
}
}
$ javac Main.java
$ LC_CTYPE=C java Main
???????
When the program runs with a UTF-8 capable locale, either from the environment or configured through Java, you get the expected result:
$ LC_CTYPE=en_US.UTF-8 java Main
中华人民共和国
$ LC_CTYPE=C java -Dfile.encoding=UTF-8 Main
中华人民共和国
If you're not sure how to configure your server, you can also do this from within Java:
System.setOut(new PrintStream(System.out, true, "UTF-8"));

Related

Setx Called From Java Not Working Correctly

I'm using the setx command to modify the PATH system environment variable. Here is the command I'm using:
setx PATH "%PATH%;C:\Python34" /M
When this is ran as batch file and as administrator, it works properly and adds the new entry to the PATH.
However if I'm trying to run this from within my Java application it starts to behave weirdly.
First of all, the Java code used to execute the batch file:
import java.io.IOException;
import java.io.InputStream;
import org.apache.commons.io.IOUtils;
public class BatchFile
{
public static void execute(String batchFilePath) throws IOException,
InterruptedException
{
Process process = Runtime.getRuntime().exec("cmd /c " + batchFilePath);
process.waitFor();
printResults(process);
}
private static void printResults(Process process) throws IOException
{
String standardOutput = getString(process.getInputStream());
String standardError = getString(process.getErrorStream());
if (!standardOutput.equals(""))
{
System.out.println(standardOutput);
}
if (!standardError.equals(""))
{
System.out.println(standardError);
}
}
private static String getString(InputStream inputStream) throws IOException
{
return IOUtils.toString(inputStream).trim();
}
}
Invoked like this:
BatchFile.execute("MyBat.bat");
Before running, my PATH looks like this:
c:\devkitPro\msys\bin;C:\devkitPro\devkitPPC\bin;C:\devkitPro\devkitPPC\powerpc-eabi\bin;C:\Program Files\Java\jdk1.8.0_45\bin;C:\ProgramData\Oracle\Java\javapath;C:\Python27\Lib\site-packages\PyQt4;C:\devkitPro\devkitARM\bin;C:\Python27\Lib\site-packages\PyQt4;C:\Program Files (x86)\Wiimm\WIT;C:\Windows\System32;C:\MinGW\bin;C:\MinGW\libexec\gcc\mingw32\4.8.1;C:\Python34;C:\MinGW\msys\1.0\bin;C:\Program Files (x86)\Git\cmd;C:\Windows\SysWOW64;C:\Program Files\TortoiseSVN\bin;C:\Program Files (x86)\MiKTeX 2.9\miktex\bin\;C:\Program Files (x86)\QuickTime\QTSystem\;C:\Program Files (x86)\Skype\Phone\
After running the Java code however, it turns into this:
C:/Program Files/Java/jdk1.8.0_45/bin/../jre/bin/server;C:/Program Files/Java/jdk1.8.0_45/bin/../jre/bin;C:/Program Files/Java/jdk1.8.0_45/bin/../jre/lib/amd64;C:/Program Files/Java/jdk1.8.0_45/bin/../jre/bin/server;C:/Program Files/Java/jdk1.8.0_45/bin/../jre/bin;C:/Program Files/Java/jdk1.8.0_45/bin/../jre/lib/amd64;C:/Program Files/Java/jdk1.8.0_45/bin/../jre/bin/server;C:/Program Files/Java/jdk1.8.0_45/bin/../jre/bin;C:/Program Files/Java/jdk1.8.0_45/bin/../jre/lib/amd64;c:\devkitPro\msys\bin;C:\devkitPro\devkitPPC\bin;C:\devkitPro\devkitPPC\powerpc-eabi\bin;C:\Program Files\Java\jdk1.8.0_45\bin;C:\ProgramData\Oracle\Java\javapath;C:\Python27\Lib\site-packages\PyQt4;C:\devkitPro\devkitARM\bin;C:\Python27\Lib\site-packages\PyQt4;C:\Program Files (x86)\Wiimm\WIT;C:\Windows\System32;C:\MinGW\bin;C:\MinGW\libexec\gcc\mingw32\4.8.1;C:\Python34;C:\MinGW\msys\1.0\bin;C:\Program Files (x86)\Git\cmd;C:\Windows\SysWOW64;C:\Program Files\TortoiseSVN\bin;C:\Program Files (x86)\MiKTeX 2.9\miktex\bin\;C:\Program Files
The message being printed out on the console is as follows:
D:\Programs\Portable\Eclipse\workspace\My Application>setx PATH "C:/Program Files/Java/jdk1.8.0_45/bin/../jre/bin/server;C:/Program Files/Java/jdk1.8.0_45/bin/../jre/bin;C:/Program Files/Java/jdk1.8.0_45/bin/../jre/lib/amd64;C:/Program Files/Java/jdk1.8.0_45/bin/../jre/bin/server;C:/Program Files/Java/jdk1.8.0_45/bin/../jre/bin;C:/Program Files/Java/jdk1.8.0_45/bin/../jre/lib/amd64;C:/Program Files/Java/jdk1.8.0_45/bin/../jre/bin/server;C:/Program Files/Java/jdk1.8.0_45/bin/../jre/bin;C:/Program Files/Java/jdk1.8.0_45/bin/../jre/lib/amd64;c:\devkitPro\msys\bin;C:\devkitPro\devkitPPC\bin;C:\devkitPro\devkitPPC\powerpc-eabi\bin;C:\Program Files\Java\jdk1.8.0_45\bin;C:\ProgramData\Oracle\Java\javapath;C:\Python27\Lib\site-packages\PyQt4;C:\devkitPro\devkitARM\bin;C:\Python27\Lib\site-packages\PyQt4;C:\Program Files (x86)\Wiimm\WIT;C:\Windows\System32;C:\MinGW\bin;C:\MinGW\libexec\gcc\mingw32\4.8.1;C:\Python34;C:\MinGW\msys\1.0\bin;C:\Program Files (x86)\Git\cmd;C:\Windows\SysWOW64;C:\Program Files\TortoiseSVN\bin;C:\Program Files (x86)\MiKTeX 2.9\miktex\bin\;C:\Program Files (x86)\QuickTime\QTSystem\;C:\Program Files (x86)\Skype\Phone\;D:\Programs\Portable\Eclipse;;C:\Python34" /M
SUCCESS: Specified value was saved.
WARNING: The data being saved is truncated to 1024 characters.
As you can see, a lot of rubbish Java directories have been added to the PATH contents at the beginning and it exceed the maximum limit of 1024 characters so it cuts off at the end. At the end there is an unwanted Eclipse directory added and there also are two semicolons before the actual Python path.
How do I get rid of the unwanted behavior when invoking the batch file using Java? I want the correct default behavior like when the setx command is invoked normally on cmd.
A while ago I was stuck at the same problem.
The simple answer is:
Do not use eclipse to start your program.
Eclipse will change your environment variables quite a lot for the execution. (including PATH)
You might have guessed it now but eclipse closes the PATH with a ; so when you call
setx PATH "%PATH%;C:\Python34" /M
you add another ; in front of the python path.
Just check by changing your batch to
echo %PATH%
Then run from eclipse and make another testrun from the command line (java -cp . MyMainClass) to see the difference.
Regarding the "WARNING: The data being saved is truncated to 1024 characters." limitation of setx you might want to have a look there: https://superuser.com/questions/387619/overcoming-the-1024-character-limit-with-setx

Run Program and Config File using Command Line

I have a Java project which comprises of the program named runProgram and configuration file named config.properties inside the package pack. I am trying to run the project as a .jar file from the command prompt in Windows 7. But I don't know how to indicate the specific configuration file that I want to use. I tried "open file" with the file name and it didn't work.
Program:
package pack;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.Properties;
public class runProgram {
public static void main(String[] args){
double value;
Properties configFile = new Properties();
try {
configFile.load(helloworld.class.getClassLoader().getResourceAsStream(
"\\pack\\config.properties"));
} catch (IOException e) {
e.printStackTrace();
}
value = Double.parseDouble(configFile.getProperty("value"));
System.out.println("Value is: "+value);
}
}
Config File:
value = 75
I give it a shot, but I don't know if I'm getting your question correctly.
But I don't know how to indicate the specific configuration file that I want to use.
If this is about running the tool, while specifying another config file every time, I'd go with Apache CLI, to parse and handle command line arguments. Then the user can specify the file every time he/she is running your tool.
(Under 'usage' you can find some examples, but I'm not allowed to post more than two links right now)
But if you want to know more about handling java properties files, here's a good tutorial:
http://www.mkyong.com/java/java-properties-file-examples/
Also, for OS independent path definitions, have a look at System.getProperty("file.separator"), which, as the name says, tries to receive the OS specific path separator (e.g., '\' for Windows, '/' for UNIX systems).
Hope this helps.

execute jar file in java program

I want to create an java program to compress an css file using YUI
I am new learner in java.
My Code is:
import java.io.BufferedInputStream;
import java.io.IOException;
public class Run extends Object
{
public static void main(String args[]) throws IOException, InterruptedException
{
System.out.println("Calling jar");
Process p = Runtime.getRuntime().exec("java -Xmx32m -jar yui.jar in.css");
BufferedInputStream bis = new BufferedInputStream(p.getInputStream());
synchronized(p)
{
p.waitFor();
}
System.out.println(p.exitValue());
int b = 0;
while((b = bis.read()) > 0)
{
System.out.print((char)b);
}
System.out.println("Called jar");
}
}
I took reference from here.
the command:
java -Xmx32m -jar yui.jar in.css
works fine in cmd but I get no output when I run above program
the output I get for above is:
Calling jar
1
Called jar
Please tell Me what I am doing wrong or what is the right way of doing this.
You are getting the "1" because there is nothing to read in the stream.
I'm guessing that you're trying to run the file from Eclipse or some other IDE. If so, you need to place your yui.jar and in.css files to same directory relative to where your Run class is.
If you're using default run configurations, you'll just want to put the files into the root directory of your eclipse project. For example, this works for me:
Test
src
com
test
Run.java
yui.jar
in.css
A better way to handle your situation is to use absolute or relative paths instead of just specifiying yui.jar or in.css. Create two variables for the two relative paths and then create the command string.

Strange getName() result on File in linux

I have some code that reads the name from an UNC path:
File f = new File(//fileshare/folder/file.txt)
System.out.println(f.getName())
On windows this gives:
file.txt
but on linux it gives:
//fileshare/folder/file.txt
why?
On my system (Ubuntu 12.04) it gives file.txt:
$ cat Test.java
import java.io.File;
class Test {
public static void main(String[] args) {
File f = new File("//fileshare/folder/file.txt");
System.out.println(f.getName());
}
}
$ javac Test.java && java Test
file.txt
f.getPath() returns /fileshare/folder/file.txt, which shows that multiple slashes get reduced to one, as is customary (required?) on Unix systems.
Of course, a UNC path is meaningless on anything but Windows.
Try using org.apache.commons.io.FilenameUtils from Apache Commons, the method getName() should ensure consistent parsing of the full path regardless of the platform where you are running.

Get file extension in OS X using Java

I want to get the file extension in my Java code:
File f = new File(....);
String name = f.getName();
int pos = name.lastIndexOf('.');
String ext = name.substring(pos+1);
I tried this code but it didn't work in OS X Lion, in Windows it works.
UPDATE
for example
i have a png file in my desktop with this name Screen Shot 2012-02-23 at 1.26.27 PM
i get the 27 PM extension with this code .
Your code is doing exactly what you've asked it to do. It's finding the filename extension on the file. The problem is that you have file with a stupid name that should have a ".png" extension, but doesn't. If you still want your code to handle this, you might need to look at interpreting the file header to determine the type of the file, which is presumably what OS X is doing to know that it is really a .png file.
In UNIX not all files have extension, probably that's what is happening with you, but until you provide an stacktrace or a more clear definition of didn't work I can't tell.
I took your code and create a sample:
localhost:Desktop oscarryz$ cat Fe.java
import java.io.File;
import static java.lang.System.out;
class Fe {
public static void main( String ... args ) {
File f = new File("Screen Shot 2012-03-06 at 10.03.35 PM.png");
out.println("File exists: " + f.exists() );
out.println("Extension: " +
f.getName().substring( f.getName().lastIndexOf('.')+1)
);
}
}
localhost:Desktop oscarryz$ javac Fe.java
localhost:Desktop oscarryz$ java Fe
File exists: true
Extension: png
You are probably dropping the extension when passing the file name, because the mechanism you show does works on OSX perfectly.
What output you are getting i have OSX SnowLeopard on that this is working fine please keep in mind file separator you need to be select as per operating system. that can be a problem as you haven't shown file name i can suggest this only please check file separator.
Solution is from garsinfotech.com labs
Try to use FilenameUtils#getExtension().

Categories

Resources