Develop Reference

Logging files names that contain Norwegian letters in the file name in Unix OS using a Jar executable

I have a simple java program that when run is supposed to traverse through the whole directory on a Unix server and log all files on the fileserver that contain Norwegian letters "å,ø,æ".
This is how it looks on the fileserver using winSCP:
In the end the logs.log file should look like this:
2022-10-25 14:27:02 INFO Logger:99 - File: 'DN_Oppmålings.pdf'
2022-10-25 14:27:02 INFO Logger:99 - File: 'Salg_av_gærden.pdf'
However, this is how it ends up in the log file, all Norwegian letters are represented with a square.
I can't seem to figure out why it happens. It probably has something to do with the encodings. Because when I run it on windows locally, everything runs as expected and I get the result I need. But when I build the project as an executable jar and run on the server it gets wrong.
Here is the code I am using.
public static void renameFiles3(File[] files) throws IOException {
for (File filename : files) {
if (filename.isDirectory()) {
renameFiles3(filename.listFiles());
} else {
String fileNameString = filename.getName();
if (fileNameString.contains("å") || fileNameString.contains("ø") || fileNameString.contains("æ")){
logger.info("File: '" + filename.getName());
}
}
}
}
public static void main(String[] args) {
File[] files = new File(path).listFiles();
try {
renamer.renameFiles3(files);
} catch catch(IOException ex){
logger.error(ex.toString());
}
}
Someone pointed out that the encoding should be specified, but I am not sure how that is done. If I run "locale" command on the Unix server this is what I get as output.
[e1111111#ilt repository]$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
I use Putty to run the jar file. Here are the configs.
Stacktrace of the error I get when running the jar:
java.nio.file.NoSuchFileException: ./documentRepository/DN_Oppm�lings.pdf
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:430)
at java.base/sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:267)
at java.base/java.nio.file.Files.move(Files.java:1422)
at com.example.fixfilenamesonfileserver.Renamer.renameFiles2(Renamer.java:105)
at com.example.fixfilenamesonfileserver.Renamer.renameFiles2(Renamer.java:89)
at com.example.fixfilenamesonfileserver.Renamer.renameFiles2(Renamer.java:89)
at com.example.fixfilenamesonfileserver.Renamer.renameFiles2(Renamer.java:89)
at com.example.fixfilenamesonfileserver.Renamer.renameFiles2(Renamer.java:89)
at com.example.fixfilenamesonfileserver.Renamer.renameFiles2(Renamer.java:89)
at com.example.fixfilenamesonfileserver.Renamer.renameFiles2(Renamer.java:89)
at com.example.fixfilenamesonfileserver.Renamer.main(Renamer.java:154)
What makes it even more strange, is that I can create for instance a folder with mkdir containing Norwegian letters in the name and it would be displayed correctly and also logged correctly if I create a file with Norwegian letters.

Some time ago I wrote an answer for a very similar problem.
As stated in the aforementioned solution, the problem could be related with the use of different charsets in your local Windows laptop (probably, cp-1252 or some variant) and your server.
As suggested, please, consider review the charset which is in place in the JVM in every environment, and review and adapt if necessary the value of the file.encoding system property on your laptop and the server environment, maybe it will help you solve the problem.
Probably running your jar with a proper value for the file.encoding JVM property may do the application work properly:
java -Dfile.encoding=UTF-8 -jar your_app.jar

I suspect there is no problem with your Java nor your file.
The problem is likely with the app you use to view that text. That app is using a font that lacks a glyph for those characters.
Edit your Question to note the app and OS if you want further assistance.

Assuming you are printing the letters to a terminal, the problem is most likely the terminal you use. If you are printing the characters to a terminal, make sure it is set to chcp65001, and a font that supports displaying norwegian letters fully. I have encountered similar problems while trying to display multilingual text due to the shortage of support for multiple languages in the same font.
So, to summarize, first set the terminal code-page encoding to chcp 65001, and then change the font of the terminal to a font that supports norwegian letter fully, and then run the jar file from the terminal like : java -jar <jarname>.jar

Loading java code at runtime

I got a little project where I have to compute a list. The computation depends on serveal factors.
The point is that these factors change from time to time and the user should be allowed to change this by it's self.
Up to now, the factors are hard-coded and no changes can be done without recompiling the code.
At the moment the code looks like this:
if (someStatement.equals("someString")) {
computedList.remove("something");
}
My idea is to make an editable and human readable textfile, configfile, etc. which is loaded at runtime/ at startup? This file should hold the java code from above.
Any ideas how to do that? Please note: The targeted PCs do not have the JDK installed, only an JRE.

An effective way of going about this is using a static initializer. Static Block in Java A good and concise explanation can be found under this link.

One option here that would allow this would be to use User Input Dialogs from the swing API - then you could store the users answer's in variables and export them to a text file/config file, or just use them right in the program without saving them. You would just have the input dialogs pop up at the very beginning of the program before anything else happens, and then the program would run based off those responses.

You could use Javascript for the configuration file language, instead of java. Java 7 SE and later includes a javascript interpreter that you can call from Java. it's not difficult to use, and you can inject java objects into the javascript environment.
Basically, you'd create a Javascript environment, insert the java objects into it which the config file is expected to configure, and then run the config file as javascript.

Okay, here we go... I found an quite simple solution for my problem.
I am using Janino by Codehaus (Link). This library has an integrated Java compiler and seems to work like the JavaCompiler class in Java 7.
BUT without having the JDK to be installed.
Through Janino you can load and compile *.java files(which are human readable) at runtime, which was exactly what I needed.
I think the examples and code-snippets on their homepage are just painful, so here's my own implementation:
Step one is to implement an interface with the same methods your Java file has which is loaded at runtime:
public interface ZuordnungInterface {
public ArrayList<String> Zuordnung(ArrayList<String> rawList);}
Then you call the Janino classloader when you need the class:
File janinoSourceDir = new File(PATH_TO_JAVAFILE);
File[] srcDir = new File[] { janinoSourceDir };
String encoding = null;
ClassLoader parentClassLoader = this.getClass().getClassLoader();
ClassLoader cl = new JavaSourceClassLoader(parentClassLoader, srcDir,
encoding);
And create an new instance
ZuordnungsInterface myZuordnung = (ZuordnungInterface) cl.loadClass("zuordnung")
.newInstance();
Note: The class which is loaded is named zuordnung.java, so there is no extension needed in the call cl.loadClass("zuordnung").
And finaly the class I want to load and compile at runtime of my program, which can be located wherever you want it to be (PATH_TO_JAVAFILE):
public class zuordnung implements ZuordnungInterface {
public ArrayList<String> Zuordnung(ArrayList<String> rawList){
ArrayList<String> computedList = (ArrayList<String>) rawList.clone();
if (Model.getSomeString().equals("Some other string")) {
computedList.add("Yeah, I loaded an external Java class");
}
return computedList;
}}
That's it. Hope it helps others with similar problems!

Java 1.4.2 File.listFiles not working properly with CIFS mounts - workaround?

I'm using Java 1.4.2 and Debian 6.0.3. There's a shared Windows folder in the network, which is correctly mounted to /mnt/share/ via fstab (e.g. it's fully visible from OS and allows all operations) using CIFS. However, when I try to do this in Java:
System.out.println(new File("/mnt/share/").listFiles().length)
it would always return 0, meaning File[] returned by listFiles is empty. The same problem applies to every subdirectory of /mnt/share/. list returns empty array as well. Amusingly enough, other File functions like "create", "isDirectory" or even "delete" work fine. Directories mounted from USB flash drive (fat32) also work fine.
I tested this on 2 different "shared folders" from different Windows systems; one using domain-based authentication system, another using "simple sharing" - that is, guest access. The situation seems weird, since mounted directories should become a part of a file system, so any program could use it. Or so I thought, at least.
I want to delete a directory in my program, and I currently see no other way of doing it except recursive walking on listFiles, so this bug becomes rather annoying. The only "workaround" I could think of is to somehow run an external bash script, but it seems like a terrible solution.
Edit: It seems this is 1.4.2-specific bug, everything works fine in Java 6. But I can't migrate, so the problem remains.
Could you suggest some workaround? Preferably without switching to third-party libs instead of native ones, I can't say I like the idea of rewriting the whole project for the sake of single code line.

Since Java 1.2 there is method File.getCanonicalFile(). In your case with mounted directory you should use exactly this one in such style:
new File("/mnt/share/").getCanonicalFile().listFiles()

So, two and half years later after giving up I encounter the same problem, again stuck with 1.4.2 because I need to embed the code into obsolete Oracle Forms 10g version.
If someone, by chance, stumbles onto this problem and decides to solve it properly, not hack his way through, it most probably has to do with (highly) unusual inode mapping that CIFS does upon mounting the remote filesystem, causing more obscure bugs some of which can be found on serverfault. One of the side-effects of such mapping is that all directories have zero hard-link count. Another one is that all directories have "size" of exactly 0, instead of usual "sector size or more", which can be checked even with ls.
I can't be sure without examining the (proprietary) source code, but I can guess that Java prior to 1.5 used some shortcut like checking link count internally instead of actually calling readdir() with C, which works equally well for any mounted FS.
Anyway, the second side-effect can be used to create a simple wrapper around File which won't rely on system calls unless it suspects a directory is mounted using CIFS. Other versions of list and listFiles functions in java.io.File, even ones using filters, rely on list() internally, so it's OK to override only it.
I didn't care about listFiles returning File[] not FileEx[] so I didn't bother to override it, but is should be simple enough. Obviously, that code can work only in Unix-like systems having ls command handy.
package FSTest;
import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.ArrayList;
public class FileEx extends File
{
public FileEx(String path)
{
super(path);
}
public FileEx(File f)
{
super(f.getAbsolutePath());
}
public String[] list()
{
if (this.canRead() && this.isDirectory())
{
/*
* Checking the length of dir is not the most reliable way to distinguish CIFS mounts.
* However, zero directory length generally indicates something unusual,
* so calling ls on it wouldn't hurt. Ordinary directories don't suffer any overhead this way.
* If this "zero-size" behavior is ever changed by CIFS but list() still won't work,
* it will be safer to call super.list() first and call this.listUsingExec if returned array has 0 elements.
* Though it might have serious performance implications, of course.
*/
if (this.length() > 0)
return super.list();
else
return this.listUsingExec();
}
else
return null;
}
private String[] listUsingExec()
{
Process p;
String command = "/bin/ls -1a " + this.getAbsolutePath();
ArrayList list = new ArrayList();
try
{
p = Runtime.getRuntime().exec(command);
p.waitFor();
BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()));
for (String line = reader.readLine(); line != null; line = reader.readLine())
{
if (!line.equalsIgnoreCase(".") && !line.equalsIgnoreCase(".."))
list.add(line);
}
String[] ret = new String[list.size()];
list.toArray(ret);
return ret;
}
catch (IOException e)
{
return null;
}
}
}

Syntax error on token "Invalid Character", delete this token

I am not sure why is it giving this error. Braces seem to be right. Another thing is that the same program works in Windows-eclipse but not in eclipse for Mac. What could be the reason?
import java.util.Vector;
public class Debug
{
private int something = 0;
private Vector list = new Vector();
public void firstMethod()
{
thirdMethod(something);
something = something + 1;
}
public void secondMethod()
{
thirdMethod(something);
something = something + 2;
}
public void thirdMethod(int value)
{
something = something + value;
}
public static void main(String[] args)
{
Debug debug = new Debug();
debug.firstMethod();
debug.secondMethod();
}
}

Ah, ok - it's probably a control-Z or other unprintable character at the end of the file that is ignored in Windows but not on the Mac. You copied the source from Windows to the Mac. Delete the last few characters and re-enter them - I think it will go away. I don't do Mac, though - I'm just guessing.

I had the same problem importing my projects from mac to linux Slackware.
Mac OSX creates some temporary files with the same name of the files in folders (._filename) in all folders.
Usually these files are invisible in Mac OSX, but in the other OSs no.
Eclipse can find these files and tries to handle like sources (._filename.java).
I solved deleting these files.

Only way i could resolve this problem was press Ctrl+A to select all text of file then Ctrl+C to copy them then delete file and create new class with intellij idea then Ctrl+P to paste text in new file. this resolve my problem and compiler never show error after do this solution.

It can happen when we copy and paste .It happens when there may be some character which is unrecognized in one platform but recognized in other.
I would suggest don't copy rather try to write the entire code by yourself. It should work

I got the same error when I imported a project I created in a Mac, to Windows. As #Massimo says Mac creates ._filename,java files which eclipse running in windows consider as source files. This is what causes the problem.
They are hidden files, which you can see when you select the option, "Show hidden files and folders" under folder options in Windows machine. Deleting these files solves the problem.

I got this message trying to call a subjob from a tRunJob component. In the tRunJob I had both checked "transmit whole context" AND listed individual parameters in the parameters/values box. Once I removed the additional parameters it worked.

There are probably hidden characters in the line. If you move your cursor through the characters and your cursor doesn't move in one character, that means there is an invalid character in the line. Delete those and it should work. Also try copying and pasting the line to an hex editor and you will see the invalid characters in it.

i face this problem many times in eclipse . What i found is that select all code - cut it using Ctrl + x and then save the file and again paste the code using Ctrl + V . This works for me many times when i copy the code from another editor.

I also faced similar issue while copying the code from one machine to another.
The issue was with Space only you need to identify the red mark in your eclipse code.

On Windows, if you copy the source to Notepad - save the file (as anything), ensuring ASCI encoding is selected - the character will be converted to a question-mark which you can then delete - then copy the code back to Eclipse.

In eclipse right click on the file -> Properties -> resources
In Text file encoding select US-ASCII
This way you will see all the special char, you can then find & replace
And then format the code

Make JAR as a standalone executable

Is there a way to convert JAR lib into JAR standalone?
I need to find a standalone java executable that convert PDF into TIFF and I've found these JARs: http://www.icefaces.org/JForum/posts/list/17504.page
Any ideas?

Easiest might be to create another Jar with a Main() entry point, and then just use the java.exe executable to run it:
e.g.
> java.exe -cp MyJarMain.jar;MyPDFJar.jar com.mydomain.MyMain myPDF.pdf
Where MyMain is a class with a Main static method.
You'll need something with a main entry point to pass in and interpret some command line arguments (myPDF.pdf in my made-up example)

You could do an assembly (are you using maven?) and make sure the Main-Class entry in the manifest.mf points to the main class.

Since there is no main-Method, you have to write one, or write a whole new class to call the class/method TiffConver.convertPDF .
The question is, how you're going to use it. From the command line, you need no executable jar. From the Gui, maybe you want to pass a file to be converted by drag and drop? Then you should take the parameter(s) passed to main as Input-PDF-Names (if they end in .pdf) and pass the names iteratively to TiffConverter, for "a.pdf b.pdf" =>
TiffConver.convertPDF ("a.pdf", "a.tiff");
TiffConver.convertPDF ("b.pdf", "b.tiff");
TiffCoverter will silently overwrite existing tiffs, so check that before or change the code there - this is clearly bad habit, and look out for more such things - I didn't.
/*
* Remove target file if exists
*/
File f = new File(tif);
if (f.exists()) {
f.delete();
}
Maybe you wan't to write a swing-wrapper, which let's you choose Files interactively to be converted. This would be a nice idee, if no filename is given.
If the user passes "a.pdf xy.tiff" you could rename the converted file to xy, as additional feature.
Without a main-class, however, a standalone jar would be magic.
However, building a native executale is almost always a bad idea. You loose portability, you don't profit from security- and performance improvements to the JVM or fixed bugs. For multiple programs you need always an independend bugfix, which you might have to manage yourself, if you don't have a package-management as most linux distros have.
after clearing some questions:
public static void main (String [] args) {
if (args.length == 1 && args[0].endsWith (".pdf")) {
String target = args[0].replaceAll (".pdf$", ".tif");
convertPDF (args[0], target);
}
}
This method you put into TiffConvert. It will allow you to convert a simple pdf-File, and generate a tif-File with the same basename but ending in .tif, silently overwriting an existing one of the same name.
I guess you now need to know how to start it?