I made a wrapper ConfigurationFile class to help handle Gdx.files stuff, and it worked fine for a long time, but now it's not working, and I don't know why.
I have two of the following two methods: internal(...) and local(...). The only difference between the two is handling the load from arguments from (File folder, String name) and (String path).
-Snip Now Unnecessary Information-
UPDATE
After more configuring, I came to find out that they're not behaving the same. I have an assets/files/ folder that Gdx.files.internal(...) will access fine, but ConfigurationFile.internal(...) will access files/, and they're set up the same way. I'll give you the two pieces of code that I used for testing.
Using Gdx.files.internal(...) directly (works as expected):
FileHandle handle = Gdx.files.internal("files/virus_data");
BufferedReader reader = null;
try {
reader = new BufferedReader(handle.reader());
String c = "";
while ((c = reader.readLine()) != null) {
System.out.println(c); // prints out all 5 lines on the file.
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (reader != null) reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
Using ConfigurationFile.internal(...):
// First part, calls ConfigurationFile#internal(String path)
ConfigurationFile config = ConfigurationFile.internal("files/virus_data");
// ConfigurationFile#internal(String path)
public static ConfigurationFile internal(String path) {
ConfigurationFile config = new ConfigurationFile();
// This is literally calling Gdx.files.internal("files/virus_data");
config.handle = Gdx.files.internal(path);
config.file = config.handle.file();
config.folder = config.file.getParentFile();
config.init();
return config;
}
// ConfigurationFile#init()
protected void init() {
// File not found.
// Creates a new folder as a sibling of "assets"
// Creates a new file called "virus_data"
if (!folder.exists()) folder.mkdirs();
if (!file.exists()) {
try {
file.createNewFile();
} catch (IOException e) {
e.printStackTrace();
}
} else loadFile();
}
// ConfigurationFile#loadFile()
protected void loadFile() {
BufferedReader reader = null;
try {
reader = new BufferedReader(handle.reader());
String c = "";
while ((c = reader.readLine()) != null) {
System.out.println(c);
if (!c.contains(":")) continue;
String[] values = c.split(":");
String key = values[0];
String value = values[1];
if (values.length > 2) {
for (int i = 2; i < values.length; i++) {
value += ":" + values[i];
}
}
key = key.trim();
value = value.trim();
mapValues.put(key, value);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (reader != null) reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
What I'm having trouble understanding is what's the difference between these two ways that it is causing my ConfigurationFile to create a new File in a folder that is a sibling of assets. Could someone tell me why this is happening?
My suggestion is not to use
Gdx.files.internal(folder + "/" + name);
If you have to use the File api, do it this way:
Gdx.files.internal(new File(folder, name).toString());
This way you avoid weird things that could be happening with path separators.
If Gdx maybe needs relative paths for some reason (perhaps relative to some Gdx internal home directory), you could use NIO to do something like
final Path gdxHome = Paths.get("path/to/gdx/home");
//...
File combined = new File(folder, name);
String relativePath = gdxHome.relativize(combined.toPath()).toString();
Okay, so after intense testing, I found out the problem, which I found to be ridiculous.
Since the file is Internal, that means a new File(...) reference can't be properly made to it, but instead it's an InputStream (if I'm correct), but anyways, using the method FileHandle#file() on an Internal file causes some kind of conversion for the path, so after removing anything that dealed with FileHandle#file() for an Internal file fixed it.
Related
I want to download few files from a URL. I know the starting of the file name. But the next part would be different. Mostly a date. But it could be different for different files. From Java code, is there any way to download file with matching pattern?
If I hit the below URL in chrome, all the files are listed and I have to download the required files manually.
http://<ip_address>:<port>/MR/build/report/scan/daily/2021-12-13_120/data/
File names can b like below. It will have known file name and date. The date can be different. Either the same as in URL or some older one.
scan_report_2021_12_13_120.txt
build_report_2021_12_10_110.txt
my_reportdata_2021_11_30_110.txt
As of now, my Java code is like below. I have to pass the complete URL with exact file name to download the files. Most of the cases it would be same as the date and number in URL. So in the program I take the date part from URL and add it to my file name nd pass as the URL. But for some files it might change and for those I have to manually download.
private static void downloadFile(String remoteURLPath, String localPath) {
System.out.println("DownloadFileTest.downloadFile() Downloading from " + remoteURLPath + " to = " + localPath);
FileOutputStream fos = null;
try {
URL website = new URL(remoteURLPath);
ReadableByteChannel rbc = Channels.newChannel(website.openStream());
fos = new FileOutputStream(localPath);
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (fos != null) {
try {
fos.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
The argument remoteURLPath is passed like http://<ip_address>:<port>/MR/build/report/scan/daily/2021-12-13_120/data/scan_report_2021_12_13_120.txt
And localPath is passed like C:\\MyDir\\MyData\\scan_report_2021_12_13_120.txt
Similarly other files also with date as 2021_12_13_120. Other files wont get downloaded. But will create empty file in the same directory which I will delete later since size is 0.
Is there any way we can pass pattern here?
Like http://<ip_address>:<port>/MR/build/report/scan/daily/2021-12-13_120/data/scan_report_*.txt
And instead of passing complete local path, is there any way to pass only directory where the file should get downloaded with exact same name as in the remote system?
In Linux I can use wget with pattern matching. But was looking for Java way to download in all platforms.
wget -r -np -nH --cut-dirs=10 -A "scan_report*.txt" "http://<ip_address>:<port>/MR/build/report/scan/daily/2021-12-13_120/data/"
Thanks to comment from #FedericoklezCulloca. I modified my code using this answer
The solution I did is read all html page and get all href values as it had only the file names with extension. From there I had another list which I used to get the matching files and those I downloaded then using my code in the Question.
Method to get all href list from URL. may be optimisation can be done. Also I did not use any extra library.
private static List<String> getAllHREFListFromURL(String downloadURL) {
URL url;
InputStream is = null;
List<String> hrefListFromURL = new ArrayList<>();
try {
url = new URL(downloadURL);
is = url.openStream();
byte[] buffer = new byte[1024];
int bytesRead = -1;
StringBuilder page = new StringBuilder(1024);
while ((bytesRead = is.read(buffer)) != -1) {
String str = new String(buffer, 0, bytesRead);
page.append(str);
}
StringBuilder htmlPage = new StringBuilder(page);
String search_start = "href=\"";
String search_end = "\"";
while (!htmlPage.isEmpty()) {
int indexOf = htmlPage.indexOf(search_start);
if (indexOf != -1) {
String substring = htmlPage.substring(indexOf + search_start.length());
String linkName = substring.substring(0, substring.indexOf(search_end));
hrefListFromURL.add(linkName);
htmlPage = new StringBuilder(substring);
} else {
htmlPage = new StringBuilder();
}
}
} catch (MalformedURLException e1) {
e1.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
} finally {
try {
is.close();
} catch (Exception e) {
}
}
return hrefListFromURL;
}
Method to get list of files that I needed.
private static List<String> getDownloadList(List<String> allHREFListFromURL) {
List<String> filesList = getMyFilesList();
List<String> downloadList = new ArrayList<>();
for (String fileName : filesList) {
Predicate<String> fileFilter = Pattern.compile(fileName + "*").asPredicate();
List<String> collect = allHREFListFromURL.stream().filter(fileFilter).collect(Collectors.toList());
downloadList.addAll(collect);
}
return downloadList;
}
private static List<String> getMyFilesList() {
List<String> filesList = new ArrayList<>();
filesList.add("scan_report");
filesList.add("build_report");
filesList.add("my_reportdata");
return filesList;
}
The downloadList I iterate and uses my original download method to download.
I have a problem on my code; basically I have an array containing some key:
String[] ComputerScience = { "A", "B", "C", "D" };
And so on, containing 40 entries.
My code reads 900 pdf from 40 folder corresponding to each element of ComputerScience, manipulates the extracted text and stores the output in a file named A.txt , B.txt, ecc ...
Each folder "A", "B", ecc contains 900 pdf.
After a lot of documents, an exception "Too many open files" is thrown.
I'm supposing that I am correctly closing files handler.
static boolean writeOccurencesFile(String WORDLIST,String categoria, TreeMap<String,Integer> map) {
File dizionario = new File(WORDLIST);
FileReader fileReader = null;
FileWriter fileWriter = null;
try {
File cat_out = new File("files/" + categoria + ".txt");
fileWriter = new FileWriter(cat_out, true);
} catch (IOException e) {
e.printStackTrace();
}
try {
fileReader = new FileReader(dizionario);
} catch (FileNotFoundException e) { }
try {
BufferedReader bufferedReader = new BufferedReader(fileReader);
if (dizionario.exists()) {
StringBuffer stringBuffer = new StringBuffer();
String parola;
StringBuffer line = new StringBuffer();
int contatore_index_parola = 1;
while ((parola = bufferedReader.readLine()) != null) {
if (map.containsKey(parola) && !parola.isEmpty()) {
line.append(contatore_index_parola + ":" + map.get(parola).intValue() + " ");
map.remove(parola);
}
contatore_index_parola++;
}
if (! line.toString().isEmpty()) {
fileWriter.append(getCategoryID(categoria) + " " + line + "\n"); // print riga completa documento N x1:y x2:a ...
}
} else { System.err.println("Dictionary file not found."); }
bufferedReader.close();
fileReader.close();
fileWriter.close();
} catch (IOException e) { return false;}
catch (NullPointerException ex ) { return false;}
finally {
try {
fileReader.close();
fileWriter.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return true;
}
But the error still comes. ( it is thrown at:)
try {
File cat_out = new File("files/" + categoria + ".txt");
fileWriter = new FileWriter(cat_out, true);
} catch (IOException e) {
e.printStackTrace();
}
Thank you.
EDIT: SOLVED
I found the solution, there was, in the main function in which writeOccurencesFile is called, another function that create a RandomAccessFile and doesn't close it.
The debugger sais that Exception has thrown in writeOccurencesFile but using Java Leak Detector i found out that the pdf were already opened and not close after parsing to pure text.
Thank you!
Try using this utility specifically designed for the purpose.
This Java agent is a utility that keeps track of where/when/who opened files in your JVM. You can have the agent trace these operations to find out about the access pattern or handle leaks, and dump the list of currently open files and where/when/who opened them.
When the exception occurs, this agent will dump the list, allowing you to find out where a large number of file descriptors are in use.
i have tried using try-with resources; but the problem remains.
Also running in system macos built-in console print out a FileNotFound exception at the line of FileWriter fileWriter = ...
static boolean writeOccurencesFile(String WORDLIST,String categoria, TreeMap<String,Integer> map) {
File dizionario = new File(WORDLIST);
try (FileWriter fileWriter = new FileWriter( "files/" + categoria + ".txt" , true)) {
try (FileReader fileReader = new FileReader(dizionario)) {
try (BufferedReader bufferedReader = new BufferedReader(fileReader)) {
if (dizionario.exists()) {
StringBuffer stringBuffer = new StringBuffer();
String parola;
StringBuffer line = new StringBuffer();
int contatore_index_parola = 1;
while ((parola = bufferedReader.readLine()) != null) {
if (map.containsKey(parola) && !parola.isEmpty()) {
line.append(contatore_index_parola + ":" + map.get(parola).intValue() + " ");
map.remove(parola);
}
contatore_index_parola++;
}
if (!line.toString().isEmpty()) {
fileWriter.append(getCategoryID(categoria) + " " + line + "\n"); // print riga completa documento N x1:y x2:a ...
}
} else {
System.err.println("Dictionary file not found.");
}
}
}
} catch (IOException e) {
e.printStackTrace();
}
return true;
}
This is the code that i am using now, although the bad managing of Exception, why the files seem to be not closed?
Now i am making a test with File Leak Detector
Maybe your code raises another exception that you are not handling. Try add catch (Exception e) before finally block
You also can move BufferedReader declaration out the try and close it in finally
I am using some sort of tail -f implementation to tail a file for changes (pretty much like this ). For this I am using a RandomAccessFile, periodically check if the file length has increased and if so, seek and read the new lines (everything happening in a separate thread of the FileTailer).
Now, everything is working as expected on Windows, but I tested my program on Linux and it does not work as expected. Here is the run()-method of the FileTailer class. Specifically where it fails on linux is the part where file.seek(filePointer) gets called and then file.readLine(), of which the latter surprisingly returns NULL (although the filePointer gets incremented correctly if I append content to the file getting tailed at runtime).
public void run() {
// The file pointer keeps track of where we are in the file
long filePointer = 0;
// Determine start point
if(startAtBeginning){
filePointer = 0;
}
else {
filePointer = logfile.length();
}
try {
// Start tailing
tailing = true;
RandomAccessFile file = new RandomAccessFile(logfile, "r");
while(tailing) {
// Compare the length of the file to the file pointer
long fileLength = logfile.length();
System.out.println("filePointer = " + filePointer + " | fileLength = " + fileLength);
if(fileLength < filePointer) {
// Log file must have been rotated or deleted;
// reopen the file and reset the file pointer
file = new RandomAccessFile(logfile, "r");
filePointer = 0;
}
if(fileLength > filePointer) {
// There is data to read
file.seek(filePointer);
String line = file.readLine();
System.out.println("new line = " + line);
while(line != null){
if(!line.isEmpty())
try {
fireNewFileLine(line);
} catch (ParseException e) {
e.printStackTrace();
}
line = file.readLine();
}
filePointer = file.getFilePointer();
}
// Sleep for the specified interval
sleep(sampleInterval);
}
// Close the file that we are tailing
file.close();
}
catch(InterruptedException | IOException e){
e.printStackTrace();
}
}
Like I said, everything is working as it should on Windows, but on Linux the String variable "line" is NULL after it should have been filled with the newly appended line, so fireNewLine gets called on NULL and everything goes to crap.
Does anyone have an idea why this happens on Linux Systems?
You don't need all this, or RandomAccessFile. You are always at the end of the file. All you need is this:
public void run() {
try {
// Start tailing
tailing = true;
BufferedReader reader = new BufferedReader(new FileReader(logfile));
String line;
while (tailing) {
while ((line = reader.readLine() != null) {
System.out.println("new line = " + line);
if(!line.isEmpty()) {
try {
fireNewFileLine(line);
} catch (ParseException e) {
e.printStackTrace();
}
}
}
// Sleep for the specified interval
sleep(sampleInterval);
}
// Close the file that we are tailing
reader.close();
} catch(InterruptedException | IOException e) {
e.printStackTrace();
}
}
with maybe some provision for reopening the file.
E&OE
I am creating a pattern lock based project in android.
I have a file called category.txt
The content of the file is as below
Sports:Race:Arcade:
No what i want is that whenever the user draw a pattern for a specific games category the pattern should get append in front of that category.
eg :
Sports:Race:"string/pattern string to be appended here for race"Arcade:
i have used following code but it is not working.
private void writefile(String getpattern,String category)
{
String str1;
try {
file = new RandomAccessFile(filewrite, "rw");
while((str1 = file.readLine()) != null)
{
String line[] = str1.split(":");
if(line[0].toLowerCase().equals(category.toLowerCase()))
{
String colon=":";
file.write(category.getBytes());
file.write(colon.getBytes());
file.write(getpattern.getBytes());
file.close();
Toast.makeText(getActivity(),"In Writefile",Toast.LENGTH_LONG).show();
}
}
}
catch (FileNotFoundException e)
{
e.printStackTrace();
}
catch(IOException io)
{
io.printStackTrace();
}
}
please help !
Using RandomAccessFile you have to calculate the position. I think it's much easier to just replace the file content with a little help from apache-commons-io FileUtils. This might be not the best idea if you have a very large file but it's quite simple.
String givenCategory = "Sports";
String pattern = "stringToAppend";
final String colon = ":";
try {
List<String> lines = FileUtils.readLines(new File("someFile.txt"));
String modifiedLine = null;
int index = 0;
for (String line : lines) {
String[] categoryFromLine = line.split(colon);
if (givenCategory.equalsIgnoreCase(categoryFromLine[0])) {
modifiedLine = new StringBuilder().append(pattern).append(colon).append(givenCategory).append(colon).toString();
break;
}
index++;
}
if (modifiedLine != null) {
lines.set(index, modifiedLine);
FileUtils.writeLines(new File("someFile.txt"), lines);
}
} catch (IOException e1) {
// do something
}
I want to check the file-type of a file. I thought about magic numbers, but how to use it
with Java.
I want only allow Textfiles and filter files like jpg etc. in my programm.
Some ideas, what can I do.
private String path;
private String fileText;
private String textLine;
public LoadModel(String path) {
this.path = path;
this.fileText = "";
FileReader read = null;
BufferedReader bufRead = null;
if (path != null && new File(path).exists()
&& !(new File(path).isDirectory())) {
try {
read = new FileReader(path);
bufRead = new BufferedReader(read);
do {
try {
this.textLine = bufRead.readLine();
} catch (IOException ex) {
Logger.getLogger(LoadModel.class.getName()).log(Level.SEVERE, null, ex);
}
if (this.textLine != null) {
this.fileText = this.fileText + this.textLine + "\n";
}
} while (this.textLine != null);
} catch (FileNotFoundException ex) {
Logger.getLogger(LoadModel.class.getName()).log(Level.SEVERE, null, ex);
}
} else {
HinweisDialogController.hinweisDialogOK("Die angegebene Datei existiert nicht");
}
}
Here you can find the list of API's available for identify mime type in java with code sample.
Also in java 7 have an option
Files.probeContentType(path)
.
You can try java.nio.file.Files.probeContentType which is designed to determine a file content type. For example this test
System.out.println(Files.probeContentType(Paths.get("1.xml")));
System.out.println(Files.probeContentType(Paths.get("1.txt")));
prints
text/xml
text/plain
see API for more details
If you need your code to work on earlier versions of JDK (not JDK7) you may use Apache Tika's MimeType detector, which has MimeType#detect() method
More information here