Develop Reference

Google Protocol Buffers not reading properly

I am just setting up a new Java project which will (maybe, not so sure now) make use of Google Protocol Buffers. I am new to this API, so I started with a very basic test. A test whose outcome, to be honest, really disappointed me. Why isn't this very straight-forward code working?
var output = new ByteArrayOutputStream();
Message.Echo.newBuilder().setMsg("MSG1?").build().writeTo(output);
System.out.println("output.length " + output.toByteArray().length);
Message.Echo.newBuilder().setMsg("MSG2!!").build().writeTo(output);
System.out.println("output.length " + output.toByteArray().length);
var input = new ByteArrayInputStream(output.toByteArray());
System.out.println("input.available " + input.available());
System.out.print(Message.Echo.parseFrom(input));
System.out.println("input.available " + input.available());
System.out.print(Message.Echo.parseFrom(input));
The above code produces the following output:
output.length 7
output.length 15
input.available 15
msg: "MSG2!!"
input.available 0
It entirely misses the first messages, or rather it seems to "overwrite" it in some way since all the 15 bytes get read. Plus it fails to block on the second call considering there are no further bytes to read.
However, changing the two reading lines into:
System.out.print(Message.Echo.parseFrom(input.readNBytes(7)));
System.out.print(Message.Echo.parseFrom(input.readNBytes(15-7)));
correctly prints the two messages. I am running Kubuntu 18.04 with JDK 11. Am I missing something really important (not mentioned in the official tutorial) or is this a bug?
This is the .proto file:
syntax = "proto3";
package ...;
option java_package = "...";
option java_outer_classname = "Message";
message Echo {
string msg = 1;
}

Ok, it seems that in order to write/read multiple message using the same set of streams requires using writeDelimitedTo and parseDelimitedFrom instead, because parseFrom reads until reaching reaches EOF.
It seems that the preferred behaviour is to use a new Socket for each message. It sounds a bit odd to me, but I am sure there are good reason behind this. It should be better explained in the official tutorial though.

AssertThrows not throwing exception when going through Java Streams

So I'm writing unit tests in which I'm testing capability to blacklist and unblacklist users (which is a feature in my code that is itself working fine).
Here's a sample command that works as expected:
assertThrows(ExecutionException.class, () -> onlineStore.lookup("533"));
If I blacklist user "533", and then run the above command, it works fine, because an ExecutionException is raised (because you're trying to lookup a user who is blacklisted). Similarly, if I had NOT blacklisted user "533" but still ran the above command, the test would fail, which is expected too for similar reason (i.e. no exception is now thrown as you're NOT fetching a blacklisted user).
However if I have a List of user IDs called userIds (which user "533" is now part of) and I blacklist them all (funtionality which I know is working fine), and then run the command below:
userIds.stream().map(id -> assertDoesNotThrow(() -> onlineStore.lookup(id)));
... the test passes, even through it should have FAILED. Why ? Because all users are now blacklisted, so when fetching these users, ExecutionExceptions should have been thrown ..
If I now, replace the streams command above with either of the following, they work as expected:
assertThrows(ExecutionException.class, () -> onlineStore.lookup("533"));
assertDoesNotThrow(() -> onlineStore.lookup("533"));
So this all leads me to believe that for some reason, when going through Java Streams, thrown ExecutionExceptions aren't getting caught.
Any explanation for this behavior ?

You're not calling any terminal operation on the stream, so your assertion is never executed.
You're abusing map(), which is supposed to create a new stream by transforming every element. What you actually want to do is to execute a method which has a side effect on every element. That's what forEach is for (and it's also a terminal operation which actually consumes the stream):
userIds.stream().forEach(id -> assertDoesNotThrow(() -> onlineStore.lookup(id)));

Trying to suppress errors while attaching browser

EDIT: I'm using the LeanFT Java SDK 14.50
EDIT2: for text clarification
I'm writing test scripts for a web application that sometimes opens popup browsers for specific actions. So natually when that happens, I will attach the new browser using BrowserFactory.attach(...). The problem is that leanFT does not seem to have a way to validate that the browser exists before attaching it, and if I try to attach it too early, it will fail. And I don't like to use an arbitrairy wait/sleep time as I can never really know how much time it's going to take for the browser to get be ready. So my solution to this is below
private Browser attachPopUpBrowser(BrowserType bt, RegExpProperty url){
Browser browser = null;
int iteration = 0;
//TimeoutLimit.SHORT = 15000
while (browser == null && iteration < TimeoutLimit.SHORT.getLimit()) {
try {
Reporter.setReportLevel(ReportLevel.Off);
browser = BrowserFactory.attach(
new BrowserDescription.Builder()
.type(bt)
.url(url)
.build()
);
Reporter.setReportLevel(ReportLevel.All);
} catch (GeneralLeanFtException e) {
try {
Thread.sleep(1000);
iteration += 1000;
} catch (InterruptedException e1) {
}
}
}
return browser;
}
Now, this works wonderfully with one exception. It generates errors in the leanft test result. Errors that I want to ignore because I know that it will fail a few times before it will succeed. As you can see, I've tried changing the ReportLevel while doing this in order to suppress the error logging, but it doesn't work. I've tried using
Browser[] browsers = BrowserFactory.getallOpenBrowsers(BrowserDescription);
thinking that it will return an empty Array if it finds nothing, but I still get errors while the browser is not ready. Does anyone have suggestions as to how I could work around this?
TL;DR
I'm looking for a way to either suppress the errors generated within my While..Loop or to validate that the browser is ready before attaching it. All of that, so that I can have a nice and clean Run Result at the end of my test (because these errors will present false negatives in all nearly all of my tests)
Addendum
Also, when the attach fails for the first time, I get a an exception
com.hp.lft.sdk.ReplayObjectNotFoundException: attachApplication
as expected, but all subsequent failures are throwing
com.hp.lft.sdk.GeneralLeanFtException: Cannot read property 'match' of null
I've compared both stack traces and they are identical except for the last 2 lines which happen within the ReplayExceptionFactory.CreateDefault() so I think that there is something that gets corrupted during the exception generation, but that is within the leanft.sdk.internal package so there might not be a lot we can do about it right now.I'm guessing that if I did not get that second "cannot read property" exception, I would correctly get the ReplayObjectNotFoundException until the browser is correctly attached.

I'd rather not force an attach endlessly until it works. Even if we'd solve the false negatives, we'd still have a not so good approach to the problem.
The cleanest solution would be to see if there is anything to attach to in the first place.
And you can do just that by getting all the browser instances that meets your description.
Browser[] browsers = BrowserFactory.getAllOpenBrowsers(new BrowserDescription.Builder().build());
Any element in this collection is an already "attached" browser - you can start using it.
If the list doesn't contain your browser instance, rerun the query.

file.lastModified() is never what was set with file.setLastModified()

I do have a problem with millis set and read on Android 2.3.4 on a Nexus One. This is the code:
File fileFolder = new File(Environment.getExternalStorageDirectory(), appName + "/"
+ URLDecoder.decode(folder.getUrl()));
if (fileFolder != null && !fileFolder.exists()) {
fileFolder.setLastModified(1310198774);
fileFolder.mkdirs();
fileFolder.setLastModified(1310198774);
}
if (fileFolder != null && fileFolder.exists()) {
long l = fileFolder.lastModified();
}
In this small test I write 1310198774 but the result that is returned from lastModified() is 1310199771000.
Even if I cut the trailing "000" there's a difference of several minutes.
I need to sync files between a webservice and the Android device. The lastmodification millis are part of the data sent by this service. I do set the millis to the created/copied files and folders to check if the file/folder needs to be overwritten.
Everything is working BUT the millis that are returned from the filesystem are different from the values that were set.
I'm pretty sure there's something wrong with my code - but I can't find it.
Many thanks in advance.
HJW

On Jelly Bean+, it's different (mostly on Nexus devices yet, and others that use the new fuse layer for /mnt/shell/emulated sdcard emulation):
It's a VFS permission problem, the syscall utimensat() fails with EPERM due to inappropriate permissions (e.g. ownership).
in platform/system/core/sdcard/sdcard.c:
/* all files owned by root.sdcard */
attr->uid = 0;
attr->gid = AID_SDCARD_RW;
From utimensat()'s syscall man page:
2. the caller's effective user ID must match the owner of the file; or
3. the caller must have appropriate privileges.
To make any change other than setting both timestamps to the current time
(i.e., times is not NULL, and both tv_nsec fields are not UTIME_NOW and both
tv_nsec fields are not UTIME_OMIT), either condition 2 or 3 above must apply.
Old FAT offers an override of the iattr->valid flag via a mount option to allow changing timestamps to anyone, FUSE+Android's sdcard-FUSE don't do this at the moment (so the 'inode_change_ok() call fails) and the attempt gets rejected with -EPERM. Here's FAT's ./fs/fat/file.c:
/* Check for setting the inode time. */
ia_valid = attr->ia_valid;
if (ia_valid & TIMES_SET_FLAGS) {
if (fat_allow_set_time(sbi, inode))
attr->ia_valid &= ~TIMES_SET_FLAGS;
}
error = inode_change_ok(inode, attr);
I also added this info to this open bug.

So maybe I'm missing something but I see some problems with your code above. Your specific problem may be due (as #JB mentioned) to Android issues but for posterity, I thought I'd provide an answer.
First off, File.setLastModified() takes the time in milliseconds. Here are the javadocs. You seem to be trying to set it in seconds. So your code should be something like:
fileFolder.setLastModified(1310198774000L);
As mentioned in the javadocs, many filesystems only support seconds granularity for last-modification time. So if you need to see the same modification time in a file then you should do something like the following:
private void changeModificationFile(File file, long time) {
// round the value down to the nearest second
file.setLastModified((time / 1000) * 1000);
}

If this all doesn't work try this (ugly) workaround quoted from https://code.google.com/p/android/issues/detail?id=18624:
//As a workaround, this ugly hack will set the last modified date to now:
RandomAccessFile raf = new RandomAccessFile(file, "rw");
long length = raf.length();
raf.setLength(length + 1);
raf.setLength(length);
raf.close();

Works on some devices but not on others. Do not design a solution that relies on it working. See https://code.google.com/p/android/issues/detail?id=18624#c29
Here is a simple test to see if it works.
public void testSetLastModified() throws IOException {
long time = 1316137362000L;
File file = new File("/mnt/sdcard/foo");
file.createNewFile();
file.setLastModified(time);
assertEquals(time, file.lastModified());
}

If you only want to change the date/time of a directory to the current date/time (i.e., "now"), then you can create some sort of temporary file inside that directory, write something into it, then immediately delete it. This has the effect of changing the 'lastModified()' date/time of the directory to the present date/time. This won't work though, if you want to change the directory date/time to some other random value, and can't be applied to a file, obviously.

can't find file... using eclipse and file/filereader/bufferedreader

http://pastebin.com/m5fa7685e
It seems to fail when getting f3.. Output is:
not ready
File is null
Exception in thread "main" java.lang.NullPointerException
at BuabFile.parseBUAB(BuabFile.java:93)
at AddressBook.createBrowseForm(AddressBook.java:232)
at AddressBook.(AddressBook.java:51)
at Main.main(Main.java:4)"
But not before then - no file not found errors or anything...

My guess would be that the parseBUAB() method receives a "null" argument. Which means that it could be that it is the AddressBook class is responsible for the error.

It looks like you forgot to assign a value to BuabFile.file static field. You may want to add this to the end of your readFile() method:
BuabFile.file = f3;
I am guessing your AddressBook.createBrowseForm method looks something like this:
String filename = ...;
BuabFile buab = new BuabFile(filename);
buab.readFile();
ArrayList<String> buabLines = buab.returnFile(); // Returns null because readFile() never assigned a value to BuabFile.file
ArrayList<Buab> buabList = buab.parseBUAB(buabLines);

From all I can see, you just call parseBUAB(..) with a null value. I can't see the call to that method so you have to check the rest of your code.
For your 'not ready' output, which is created because your BufferedReader f3 is 'not ready', the API says
True if the next read() is guaranteed not to block for input, false otherwise.
Maybe you just call it too fast and the file is not loaded yet. Play with Thread.sleep() before calling ready() on the stream. Maybe a some-milliseconds blocking is just normal for File I/O.
And third - if f3 is the BufferedReader you want to keep, you have to assign it to the member file in the readFile() method. But now that's all I found ;)

I'm confused further but have found an answer sort of - I'm using windows 7 and have tried it on a windows xp computer and the code compiles fine and reads in the file (other errors you lot have noted are to be changed anyway through development - this was just one stick in the way...).
I'm wondering if there is some Windows 7 error with eclipse and opening/reading files...