Develop Reference

Gzipping file on GCS (Google Cloud Storage) via Java API

I have log files being dropped into a GCS bucket regularly (e.g. gs://my-bucket/log.json) I want to setup a java process to process the files, gzip them, and move them to a separate bucket where I archive files (i.e. move it to gs://archived-logs/my-bucket/log.json.gz)
gsutil cp -z seems to be the only option I can find currently. Has anybody implemented it in a feasible manner using their Java API?

Ok, I think I solved it. Standard streams solution in the ending.
GcsOutputChannel gcsOutputChannel = gcsService.createOrReplace(new GcsFilename("my-bucket", "log.json.gz"),
new GcsFileOptions.Builder().build());
GZIPOutputStream outputStream = new GZIPOutputStream(Channels.newOutputStream(gcsOutputChannel));
GcsInputChannel inputChannel = gcsService
.openReadChannel(new GcsFilename("my-bucket", "log.json"), 10000);
InputStream inStream = Channels.newInputStream(inputChannel);
byte[] byteArr = new byte[10000];
while (inStream.read(byteArr) > 0) {
outputStream.write(byteArr);
}

For the latest streaming writer you can follow the below code. Do note that gcp will automatically decompress the object while serving it to web clients. If it doesnt work then add the following header to accept gzip files, "Accept-Encoding": "gzip".
Credentials credentials = GoogleCredentials.fromStream(Files.newInputStream(Paths.get(environmentConfig.getServiceAccount())));
Storage storage = StorageOptions.newBuilder().setProjectId(environmentConfig.getProjectID()).setCredentials(credentials).build().getService();
try (WriteChannel writer = storage.writer(blobInfo)) {
GZIPOutputStream gzipOutputStream = new
GZIPOutputStream(Channels.newOutputStream(writer));
gzipOutputStream.write(ByteBuffer.wrap(objectData, 0, objectData.length).array());
gzipOutputStream.flush();
gzipOutputStream.close();
} catch (IOException ex) {
log.error("Upload Error: {}", ex.getMessage());
}

How to Get image from ComPort

I have some problems in my work..
I have stored TTL serial camera images to MicroSD card successfully using Arduino UNO with the help of Adafruit Tutorial (learn.adafruit.com/ttl-serial-camera/overview) but when i m transferring that images through Zigbee transmitter, At the comport (Zigbee receiver) i m receiving random words. And i think its ASCII.
I want to save images receiving from comport to the folder of my PC.
Is it possible?
I have seen in some forums that use the java or python code, but i can't understand how to use it?
Read image data from COM7 port in Java

I guess this is what you are looking for:
import serial
ser = serial.Serial('/dev/tty.usbserial', 9600)
image = ser.read()
with open('/tmp/image', 'wb') as file:
file.write(image)
Works only in Python 3, in Python 2 you need to use io.open. You may need to install serial-modul first if you don't already have it. I'm not familiar with the Arduino-C-dialect you need to send the image over the com-port...

Arduino IDE's Serial Monitor is using the Serial class to communicate over a serial comm. port.
// Receive & send methods from the SerialMonitor class.
private void send(String s) {
..
serial.write(s);
}
public void message(final String s) {
..
textArea.append(s);
}
My suggestion is to reuse that (Java) code, but since the Serial class is designed for text-only communication you would need to encode the image bytes into e.g. Base64 encoding with this library and decode it on the PC.
If the transfer speed is important, and there is an Arduino binary-based serial communication library, you should use that.
UPDATE
You can read raw bytes from the serial port via the mentioned Serial class like this:
...
Serial port = ...;
byte[] buffer = new byte[1024]; // 1KB buffer
OutputStream imageOutput = new FileOutputStream(...);
// Wait for the image.
while (port.available() > 0) {
int numBytes = port.readBytes(buffer);
if (numBytes > 0) {
imageOutput.write(buffer, numBytes);
}
}
imageOutput.flush();
imageOutput.close();
...

web service in Java using (tomcat + AXIS2), where is the root of the application?

I create a function that downloads a image to my local web server.
When I run this function like a Java Application, it works fine. But when I try to run this method using the Web Service made by AXIS2 ( http://localhost:8080/axis2/services/adoroCinemaService2/downloadPhoto ), the AXIS2 returns Internal server error.
It happens probabily, because I use a "root path" in my code. So, what I need to do to solve this problem? Where is the root of my service? How can I setup this path?
public void downloadPhoto() throws IOException{
URL url = new URL("http://vamosla.mobi/img/bonde.png");
String target = "vamosla.jpg";
HttpURLConnection c = (HttpURLConnection)url.openConnection();
c.setRequestMethod("GET");
c.setDoOutput(true);
c.connect();
FileOutputStream f = new FileOutputStream(new File(target));
InputStream in = c.getInputStream();
byte[] buffer = new byte[1024];
int len1 = 0;
while ( (len1 = in.read(buffer)) > 0 ) {
f.write(buffer,0, len1);
}
f.close();
}

Hm, there are certainly some tricks available to work out where the code currently executes and setup relative paths from there, but i don't think that will work reliably for you.
Therefore, i suggest you configure something like an 'asset.path' either via system properties or via some configuration file you load from your classpath.

getContentLength doesn't work on android for FTP url

Following code prints length -1 for filesize on android, but it works fine on desktop JAVA.
I'm using Android 2.2.
URL url1 = null;
URLConnection uconn = null;
try {
url1 = new URL("ftp://FTPHOST/file.zip");
uconn = url1.openConnection();
uconn.setDoInput(true);
int len= uconn.getContentLength();
int headersize = uconn.getHeaderFields().size();
System.out.println("******************************* "+len);
} catch (Exception e) {
e.printStackTrace();
}
return null;
Let me know if any workaround in android to get filesize..

The Android platform's url connection code uses a different base (Apache HTTP client) under the hood, rather than the Oracle JVM's implementation. Apache HTTP client doesn't natively support FTP download the way the desktop JVM does.
The desktop JVM uses a class that was historically named sun.net.ftp.FtpClient for that FTP functionality. None of the sun classes are available on Android, so that doesn't work. You'll need to get your own FTP client.

Apache FTPClient - incomplete file retrieval on Linux, works on Windows

I have a java application on Websphere that is using Apache Commons FTPClient to retrieve files from a Windows server via FTP. When I deploy the application to Websphere running in a Windows environment, I am able to retrieve all of the files cleanly. However, when I deploy the same application to Webpshere on Linux, there are cases where I am getting an incomplete or corrupt files. These cases are consistent though, such that the same files will fail every time and give back the same number of bytes (usually just a few bytes less than what I should be getting). I would say that I can read approximately 95% of the files successfully on Linux.
Here's the relevant code...
ftpc = new FTPClient();
// set the timeout to 30 seconds
ftpc.enterLocalPassiveMode();
ftpc.setDefaultTimeout(30000);
ftpc.setDataTimeout(30000);
try
{
String ftpServer = CoreApplication.getProperty("ftp.server");
String ftpUserID = CoreApplication.getProperty("ftp.userid");
String ftpPassword = CoreApplication.getProperty("ftp.password");
log.debug("attempting to connect to ftp server = "+ftpServer);
log.debug("credentials = "+ftpUserID+"/"+ftpPassword);
ftpc.connect(ftpServer);
boolean login = ftpc.login(ftpUserID, ftpPassword);
if (login)
{
log.debug("Login success..."); }
else
{
log.error("Login failed - connecting to FTP server = "+ftpServer+", with credentials "+ftpUserID+"/"+ftpPassword);
throw new Exception("Login failed - connecting to FTP server = "+ftpServer+", with credentials "+ftpUserID+"/"+ftpPassword);
}
is = ftpc.retrieveFileStream(fileName);
ByteArrayOutputStream out = null;
try {
out = new ByteArrayOutputStream();
IOUtils.copy(is, out);
} finally {
IOUtils.closeQuietly(is);
IOUtils.closeQuietly(out);
}
byte[] bytes = out.toByteArray();
log.info("got bytes from input stream - byte[] size is "+ bytes.length);
Any assistance with this would be greatly appreciated.
Thanks.

I have a suspicion that the FTP might be using ASCII rather than binary transfer mode, and mapping what it thinks are Window end-of-line sequences in the files to Unix end-of-lines. For files that are really text, this will work. For files that are really binary, the result will be corruption and a slightly shorter file if the file contains certain sequences of bytes.
See FTPClient.setFileType(...).
FOLLOWUP
... so why this would work on Windows and not Linux remains a mystery for another day.
The mystery is easy to explain. You were FTP'ing files from a Windows machine to a Windows machine, so there was no need to change the end-of-line markers.