While using PrintWriter.write() method, I have noticed that sometimes [ issues is observed randomly] junk characters creep into the end/result file.
Tried an alternate API also as below.
StandardOpenOption[] openOption = {StandardOpenOption.CREATE , StandardOpenOption.APPEND};
writer = Files.newBufferedWriter(outFile, this.charsetEncoding ,openOption);
This issue is being seen in a linux server using openjdk.
java version "1.8.0_202"
Java(TM) SE Runtime Environment (build 1.8.0_202-b32)
Java HotSpot(TM) 64-Bit Server VM (build 25.202-b32, mixed mode)
Code :
osw = new OutputStreamWriter(new FileOutputStream(outFile, true), "ISO-8859-1");
coldelimiter = "\t";
rowdelimiter = "\n";
writer = new PrintWriter(new BufferedWriter(osw));
while (iterator.hasNext()) {
final DataToWrite data = iterator.next();
writer.write(data.getDataName());
writer.write(coldelimiter);
}
writer.write(rowdelimiter);
writer.flush();
Expected :
testSystem testSystem 1 14 1 managedSystem 1 1 1310 2019-04-10 01:00:00 2019-04-10 10 Alpha testSystem +0100 1 0 4 15 29201 15MIN 2019-04-10 00:00:00 2019 13
Actual :
testSystem testSystem 1 14 1 manag)edSystem 1 1 1310 2019-04-10 01:00:00 2019-04-10 10 Alpha testSystem +0100 1 0 4 15 29201 15MIN 2019-04-10 00:00:00 2019 13 `q
I expect new line [ \n ] at the end, but sometimes junk characters like `q,) etc are getting appended at random columns.
Any idea why this is happening and how this can be avoided ?
Related
This is the first time i have encounter such problem with file access by Java on linux. The problem is just like the header says - FileNotFoundException is thrown when file actually exists. Moreover application with same configuration (props.txt file) runs like it should on windows.
Let me provide a little bit of console output
datasu#dedi2392:~/netcrawler/dkpto$ ls -l
total 20
-rwxrw-rw- 1 datasu datasu 114 Aug 7 15:53 autoupdate
drwxr-xr-x 4 datasu datasu 4096 Aug 8 11:57 data
drwxr-xr-x 2 datasu datasu 4096 Aug 8 11:57 log
-rw-rw-rw- 1 datasu datasu 32 Aug 8 12:44 props.txt
-rwxrw-rw- 1 datasu datasu 126 Aug 8 12:55 propsUpdate
datasu#dedi2392:~/netcrawler/dkpto$ ./propsUpdate
Parent: /usr/home/datasu/netcrawler/dkpto
1# -> propsUpdate
2# -> autoupdate
3# -> props.txt
4# -> data
5# -> log
(No such file or directory)ava.io.FileNotFoundException: /usr/home/datasu/netcrawler/dkpto/props.txt
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.(Unknown Source)
at netcrawler.Autoupdater.readProperties(Autoupdater.java:71)
at netcrawler.Autoupdater.start(Autoupdater.java:54)
at netcrawler.Autoupdater.main(Autoupdater.java:47)
datasu#dedi2392:~/netcrawler/dkpto$ java -version
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)
datasu#dedi2392:~/netcrawler/dkpto$
and here is Java code responsible for generating that output (at least after calling ./propsUpdate)
private void readProperties(String args) throws FileNotFoundException, IOException {
System.out.println("Parent: " + new File(args).getAbsoluteFile().getParentFile().getAbsolutePath());
CommonTools.PrintArray(new File(args).getAbsoluteFile().getParentFile().list());
properties.load(new FileInputStream(new File(args).getAbsoluteFile())); // this line throws the exception
stageNumber = Integer.parseInt(properties.getProperty(PROP_STAGE_NUMBER_KEY, "0"));
}
So why the props.txt file is not found when it is actually there ?
The string "args" probably has a nonprinting character at the end, like a space. You could use String.trim() to remove such characters before using that variable.
Is your home folder really this path?
/usr/home/datasu
/home/datasu is where it normally is on linux.
Also, try changing that line to this:
properties.load(new FileInputStream(new File(args));
If you're calling that as ./propsUpdate ./props.txt that will work from the current working directory.
Is there a java profile tool that works without a GUI in Linux, just like top? I don't have the permission to use tools like jprofile and jvisualvm to work in remote model.
Try this: http://java.sun.com/developer/technicalArticles/Programming/HPROF.html You can query heap and cpu details.
You can use HPROF.
Command used: javac -J-agentlib:hprof=cpu=samples Hello.java
CPU SAMPLES BEGIN (total = 126) Fri Oct 22 12:12:14 2004
rank self accum count trace method
1 53.17% 53.17% 67 300027 java.util.zip.ZipFile.getEntry
2 17.46% 70.63% 22 300135 java.util.zip.ZipFile.getNextEntry
3 5.56% 76.19% 7 300111 java.lang.ClassLoader.defineClass2
4 3.97% 80.16% 5 300140 java.io.UnixFileSystem.list
5 2.38% 82.54% 3 300149 java.lang.Shutdown.halt0
6 1.59% 84.13% 2 300136 java.util.zip.ZipEntry.initFields
7 1.59% 85.71% 2 300138 java.lang.String.substring
8 1.59% 87.30% 2 300026 java.util.zip.ZipFile.open
9 0.79% 88.10% 1 300118 com.sun.tools.javac.code.Type$ErrorType.<init>
10 0.79% 88.89% 1 300134 java.util.zip.ZipFile.ensureOpen
I'm trying to install WWW::HTMLUnit on Windows 7. There're step that I run through:
Install Inline::Java 0.53
Install WWW::HTMLUnit 0.15
At step 2, after nmake, I type nmake test to test module but it failed. Here's output:
C:\nmake test
Microsoft (R) Program Maintenance Utility Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.
C:\Perl\bin\perl.exe "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib\lib', 'blib\arch')" t/*.t
t/00_basic...........
t/00_basic...........NOK 1/1# Failed test 'use WWW::HtmlUnit;'
# at t/00_basic.t line 9.
# Tried to use 'WWW::HtmlUnit'.
# Error: Class com.gargoylesoftware.htmlunit.WebClient not found at C:/Perl/site/lib/Inline/Java.pm line 619
# BEGIN failed--compilation aborted at (eval 4) line 2, <GEN7> line 4.
# Looks like you failed 1 test of 1.
t/00_basic...........dubious
Test returned status 1 (wstat 256, 0x100)
DIED. FAILED test 1
Failed 1/1 tests, 0.00% okay
t/01_hello...........Class com.gargoylesoftware.htmlunit.WebClient not found at C:/Perl/site/lib/Inline/Java.pm line 619
BEGIN failed--compilation aborted at t/01_hello.t line 4, <GEN7> line 4.
t/01_hello...........dubious
Test returned status 26 (wstat 6656, 0x1a00)
t/02_hello_sweet.....dubious
Test returned status 19 (wstat 4864, 0x1300)
t/03_clickhandler....Class com.gargoylesoftware.htmlunit.WebClient not found at C:/Perl/site/lib/Inline/Java.pm line 619
BEGIN failed--compilation aborted at t/03_clickhandler.t line 6, <GEN7> line 4.
t/03_clickhandler....dubious
Test returned status 29 (wstat 7424, 0x1d00)
DIED. FAILED tests 1-8
Failed 8/8 tests, 0.00% okay
Failed Test Stat Wstat Total Fail List of Failed
-------------------------------------------------------------------------------
t/00_basic.t 1 256 1 1 1
t/01_hello.t 26 6656 ?? ?? ??
t/02_hello_sweet.t 19 4864 ?? ?? ??
t/03_clickhandler.t 29 7424 8 16 1-8
Failed 4/4 test scripts. 9/9 subtests failed.
Files=4, Tests=9, 3 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU)
Failed 4/4 test programs. 9/9 subtests failed.
NMAKE : fatal error U1077: 'C:\Perl\bin\perl.exe' : return code '0x1d'
Stop.
From above log, I could see that:
class Error: com.gargoylesoftware.htmlunit.WebClient could not be found.
I have no idea that I missed anything.
Any help would be appreciated.
Thanks.
Minh.
I found it.
There's different between path in Unix and Windows system. Unix uses ':' for a delimiter but Windows uses ';'. So what I've done is that open HTMLUnit.pm and change all of ':' to ';'.
With HTMLUnit version 0.15 I made changes at these lines below:
Line 78:
return join ';', map { "$jar_path/$_" } qw( # return join ':', map { "$jar_path/$_" } qw(
Line 127:
$custom_jars = join(';', #{$parameters{'jars'}}); # $custom_jars = join(':', #{$parameters{'jars'}});
Line 148:
CLASSPATH => collect_default_jars() . ";" . $custom_jars, # CLASSPATH => collect_default_jars() . ":" . $custom_jars,
And it works like a magic.
(it wouldn't let me comment on an existing answer)
I see your answer about ':' vs ';'. I'll try to include a fix in the next WWW::HtmlUnit release (I am the author of the perl bindings).
How are these sequence files generated ? I saw a link about sequence file here,
http://wiki.apache.org/hadoop/SequenceFile
Are these written using default Java serializer ? and How do I read a sequence file ?
Sequence files are generated by MapReduce tasks and and can be used as common format to transfer data between MapReduce jobs.
You can read them in the following manner:
Configuration config = new Configuration();
Path path = new Path(PATH_TO_YOUR_FILE);
SequenceFile.Reader reader = new SequenceFile.Reader(FileSystem.get(config), path, config);
WritableComparable key = (WritableComparable) reader.getKeyClass().newInstance();
Writable value = (Writable) reader.getValueClass().newInstance();
while (reader.next(key, value))
// perform some operating
reader.close();
Also you can generate sequence files by yourself using SequenceFile.Writer.
The classes used in the example are the following:
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.SequenceFile;
import org.apache.hadoop.io.Writable;
import org.apache.hadoop.io.WritableComparable;
And are contained within the hadoop-core maven dependency:
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>1.2.1</version>
</dependency>
Thanks to Lev Khomich's answer, my problem has been solved.
However, the solution has been deprecated for a while and the new API offers more features and also easy to use.
Check out the source code of hadoop.io.SequenceFile, click here:
Configuration config = new Configuration();
Path path = new Path("/Users/myuser/sequencefile");
SequenceFile.Reader reader = new Reader(config, Reader.file(path));
WritableComparable key = (WritableComparable) reader.getKeyClass()
.newInstance();
Writable value = (Writable) reader.getValueClass().newInstance();
while (reader.next(key, value)) {
System.out.println(key);
System.out.println(value);
System.out.println("------------------------");
}
reader.close();
Extra info, here is the sample output running against the data file generated by Nutch/injector:
------------------------
https://wiki.openoffice.org/wiki/Ru/FAQ
Version: 7
Status: 1 (db_unfetched)
Fetch time: Sun Apr 13 16:12:59 MDT 2014
Modified time: Wed Dec 31 17:00:00 MST 1969
Retries since fetch: 0
Retry interval: 2592000 seconds (30 days)
Score: 1.0
Signature: null
Metadata:
------------------------
https://www.bankhapoalim.co.il/
Version: 7
Status: 1 (db_unfetched)
Fetch time: Sun Apr 13 16:12:59 MDT 2014
Modified time: Wed Dec 31 17:00:00 MST 1969
Retries since fetch: 0
Retry interval: 2592000 seconds (30 days)
Score: 1.0
Signature: null
Metadata:
Thanks!
I have been developing a program lately that compiles and runs a C++ Program from a Java program, I have gotten everything working basically (or atleast to my knowledge) but then I noticed some things being printed to the Error Stream:
cdog5000#srv3:~$ java -Xmx50m -jar main2.jar
Running Command: sudo g++ --static -o "/home/cdog5000/cody.out" "/home/cdog5000/cody.cpp"
Err: g++: "/home/cdog5000/cody.cpp": No such file or directory
Err: g++: no input files
cdog5000#srv3:~$ ls -l
total 4548
-rwxr-xr-x 1 cdog5000 cdog5000 1297588 Feb 3 23:11 a.out
-rwxr-xr-x 1 cdog5000 cdog5000 7978 Feb 2 04:39 cody
-rw-r--r-- 1 cdog5000 cdog5000 106 Feb 4 02:09 cody.cpp
-rwxr-xr-x 1 cdog5000 cdog5000 1297357 Feb 4 02:09 cody.out
-rw-r--r-- 1 root root 410433 Feb 4 02:48 log.txt
-rwxr-xr-x 1 cdog5000 cdog5000 801088 Feb 1 05:24 main.jar
-rw-r--r-- 1 cdog5000 cdog5000 804802 Feb 4 02:49 main2.jar
drwxr-xr-x 3 cdog5000 cdog5000 4096 Feb 3 23:11 sandbox
cdog5000#srv3:~$ sudo g++ --static -o "/home/cdog5000/cody.out" "/home/cdog5000/cody.cpp"
As you can see it works if I do it via the SSH but not the Java code?
The Java code:
public static Exec exec(String cmd){
Exec exec = new Exec(cmd);
try {
long current = System.currentTimeMillis();
Process proc = Runtime.getRuntime().exec(cmd);
exec.setReturnValue(proc.waitFor());
exec.setRunTime(System.currentTimeMillis() - current);
BufferedInputStream bos = new BufferedInputStream(proc.getInputStream());
byte b[] = new byte[1024];
String content = "";
while(bos.read(b) != -1) {
content += new String(b);
}
exec.setStdIn(content.split("\n"));
content = "";
bos = new BufferedInputStream(proc.getErrorStream());
while(bos.read(b) != -1) {
content += new String(b);
}
exec.setStdErr(content.split("\n"));
} catch (Exception e) {
e.printStackTrace();
}
return exec;
}
Thanks for any help and it is apprectiated!
Err: g++: "/home/cdog5000/cody.cpp": No such file or directory
Is telling you the problem.
You have one level of quotes too many, so you're looking for "/home/cdog5000/cody.cpp" rather than /home/cdog5000/cody.cpp.
The Runtime.exec documentation says:
More precisely, the command string is broken into tokens using a StringTokenizer created by the call new StringTokenizer(command) with no further modification of the character categories. The tokens produced by the tokenizer are then placed in the new string array cmdarray, in the same order.
Meaning it only splits on whitespace, it doesn't handle double quotes like the shell does.
Many languages have two functions, one called exec which runs the command verbatim, and system which passes the string to the shell, where it will split words and expand wildcards.
I can't see a system call in Java, so I think you will have to use exec(String[] cmdarray) rather than exec(String command).
GCC doesn't lie like that - it looks like the file isn't there. Are you sure that you're showing us the output from the correct directories?