I'm struggling for few days with this problem and you are my last chance solving it.
The Goal:
To upload a bitmap from android client to google app engine and save it in datastore.
I'm using Serialization to transfer object from client to server and vise versa.
Things I have tried:
sending a Bitmap
but i got java.io.NotSerializableException: android.graphics.Bitmap
then I tried create from the Bitmap Blob at the client like this:
ByteArrayOutputStream bos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.PNG, 100, bos);
Byte[] bArray = bos.toByteArray();
Blob blob = new Blob(bArray);
But at the client side I dont have the google app engine classes (for Blob), So I tried to brute force it by extract specific classess but it created new problems.
So I am asking in which dierction I can go from to solve this problem.
Thanks.
Few notes:
Do not use Java serialization to transfer data between JVMs. Java serialization is not standardized and is not guaranteed to be compatible between JVMs (or even between versions).
To send binary data it's best to use HTTP POST and set Content-Type appropriately (e.g. application/octet-stream).
So, to make this work do this:
Create a servlet which handles POST and gets the binary data. Use servletRequest.getInputStream() to get hold of binary data.
Use Blobstore FileService API to save data to blobstore.
On Android side use a http client to make a POST request and add your bitmap's binary data to it. If you need to add some metadata use Http headers.
This might be useful
How to upload and store an image with google app engine (java)
Alternatively, you can try blobstore api
http://code.google.com/appengine/docs/java/blobstore/overview.html
Here is a production tested way:
Use GAE appengine to upload your bitmap to, and serve for future clients.
On the Android code, follow these steps:
Get an Upload URL from GAE
Upload your bitmap to GAE, and get a blobkey back
Later on, use the blobkey to serve the image to your clients.
GAE Servlet code:
getUploadURL:
BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService();
String url = blobstoreService.createUploadUrl(path_to_your_upload_servlet);
uploadServlet - stores in blobstore, returns the blobkey to the uploader
BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService();
Map<String, List<BlobKey>> uploads = blobstoreService.getUploads(request);
String fileName = uploads.keySet().iterator().next();
final BlobKey blobKey = uploads.get(fileName).get(0);
response.getWriter().println(blobKey.getKeyString());
Android client code:
String uploadUrl = getUrlAsString(..your getUrl servlet path...)
// Upload to GAE (include apache-mime4j.jar and httpmime.jar in your project for this code)
File file = new File(imageFilePath);
HttpPost postRequest = new HttpPost(uploadUrl);
MultipartEntity entity = new MultipartEntity();
entity.addPart("file", new FileBody(file));
postRequest.setEntity(entity);
HttpResponse httpResponse;
HttpClient httpClient = new DefaultHttpClient();
httpClient.getParams().setBooleanParameter("http.protocol.handle-redirects",false);
httpResponse = httpClient.execute(postRequest);
int status = httpResponse.getStatusLine().getStatusCode();
String blobKey = getInputStreamAsString(httpResponse.getEntity().getContent())
Related
I am trying to upload a 260k image file as part of multipart form using Apache HttpAsyncClient library.
I create my form this way:
val multipartEntityBuilder = new MultipartEntityBuilder
multipartEntityBuilder.addBinaryBody("file", file)
val multipartEntity = multipartEntityBuilder.build()
And then I receive a ContentTooLongException when performing request basically because of this line in the library's source code:
https://github.com/apache/httpclient/blob/4.5.3/httpmime/src/main/java/org/apache/http/entity/mime/MultipartFormEntity.java#L102
I searched a lot, but didn't find any explanation why this limitation for contentLength is present in code. Maybe someone could explain it? And my second question: what is the proper way to make an upload request for a file larger than 25 kb?
Thanks!
Found the solution: create inputStream over file and wrap the multipart entity with BufferedHttpEntity and then pass this buffered entity to request:
val multipartEntityBuilder = MultipartEntityBuilder.create()
multipartEntityBuilder.addBinaryBody("file", new FileInputStream(file), ContentType.DEFAULT_BINARY, name)
val multipartEntity = multipartEntityBuilder.build()
val entity = new BufferedHttpEntity(multipartEntity)
I create data in the server (gae) and I want to store it in Blobstore. I saw many answers on how to do this giving a BlobStore URL to the client, but there is no client or HTTP request: it's just an asynchronous task.
Then I guess I should use createUploadUrl(), and instead of giving this URL to a client, from my code HTTP Post my data to it via URL Fetch. This looks weird, isn't there another API for this?
Let's say that the files I want in Blobstore are already stored in my GCS default bucket. Can I just tell Blobstore about them using the gcs location "/gs/bucketname/file"? I tried this by
GcsFilename filename = new GcsFilename(bucketName, fileId);
String gcsKey = "/gs/" + bucketName + "/" + filename.getObjectName();
BlobKey blobKey = blobstoreService.createGsBlobKey(gcsKey);
GcsOutputChannel outputChannel = gcsService.createOrReplace(filename, GcsFileOptions.getDefaultInstance());
ObjectOutputStream oout = new ObjectOutputStream(Channels.newOutputStream(outputChannel));
oout.writeObject(myDataObjectToPersist);
oout.close();
// ...at some other point I have checked the file is correctly stored in
// GCS and I can fetch it using /gs/bucket/fileId
// but it doesn't seem to be in Blobstore, so when
InputStream stream = new BlobstoreInputStream(new BlobKey(blobKey.keyString))
// ... this gives a BlobstoreInputStream$BlobstoreIOException: BlobstoreInputStream received an invalid blob key...
Is this something conceptually wrong - like if I use GcsOutputChannel to save it I will not get it from Blobstore even if I create a BlobKey, or is it something that could work but I just did something wrong?
1K thanks
Why would you want to store the file in blobstore as opposed to writing and reading it directly from GCS?
Yes, you can create a BlobKey for a file stored in GCS, and can use the key in some of the blobstore API (such as fetchData and serve) but unfortunately not in all.
Some of the blobstore API (such as BlobstoreInputStream) depends on BlobInfo and that is not created when using the GCS client.
I have some file upload code that works on appspot.com but not on local Java dev server.
I am uploading to Google Cloud Storage, via:
String bucketName = AppIdentityServiceFactory.getAppIdentityService().getDefaultGcsBucketName();
String uploadUrl = blobstoreService.createUploadUrl(
successUrl,
UploadOptions.Builder.withGoogleStorageBucketName(bucketName)
);
in my upload handler (pointed to by "successUrl"), I try and read the newly uploaded blob from the GCS bucket. This is done like this:
BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService();
Map<String, List<BlobKey>> uploads = blobstoreService.getUploads(req);
Map<String, List<FileInfo>> files = blobstoreService.getFileInfos(req);
...
// use the FileInfo instance to do fileInfo.getGsObjectName()
...
GcsInputChannel readChannel = gcsService.openPrefetchingReadChannel(gcsFilename, 0, 1024 * 1024);
InputStream is = Channels.newInputStream(readChannel));
up on the actual appengine instance, fileInfo.getGsObjectName() returns a working path like this:
/gs/myapp.appspot.com/L2F..LongIDHere...YmJV
which works and can be read, BUT, on the local dev server, I get something like:
/gs/app_default_bucket/fake-encoded_gs_key:YXB...LongIdHere...6de
which throws FileNotFoundException when tried read using the same code, that works on appEngine :-(
I want to upload an image to my server through an android app however I would also like to pass other data along with the image (authentication, intention, etc.).
I have normally been making requests like so:
http://server/script.php?t=authtoken&j_id=12&... etc
However, I assume I cannot simply tack on another query parameter containing the byte array for the image as that would result in a URL with a size on the order of millions of characters.
&image=001101010010110111010001010101010110100101000101010100010... etc
I'm at a loss as to how I should approach this and would appreciate any suggestions. If I am not able to send the data through an http request, how would I handle the incoming data server-side?
Thanks.
Here's how I got it work for those who may find this in the future.
For this example, let's say we want to query the PHP script upload.php on the root of www.server.com with the parameters t=ABC123 and id=12. Along with this request, we also want to upload an image stored in the java.io.File object img. We will also be expecting a response from the server letting us know whether the upload was successful or not.
ANDROID SIDE
On the android side, you will need the following JARs:
apache-mime4j-core-0.7.2.jar
Availabe here and:
httpclient-4.3.1.jar
httpcore-4.3.jar
httpmime-4.3.1.jar
Availabe here.
Here is a snippet on how to make the multipart request and get a response:
public String uploadRequest(String address, File img)
{
HttpParams p = new BasicHttpParams();
p.setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
DefaultHttpClient client = new DefaultHttpClient(p);
HttpPost post = new HttpPost(address);
// No need to add regular params as parts. You can if you want or
// you can just tack them onto the URL as usual.
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
builder.addPart("image", new FileBody(img));
post.setEntity(builder.build());
return client.execute(post, new ImageUploadResponseHandler()).toString();
}
private class ImageUploadResponseHandler implements ResponseHandler<Object>
{
#Override
public Object handleResponse(HttpResponse response) throws ClientProtocolException, IOException
{
HttpEntity responseEntity = response.getEntity();
return EntityUtils.toString(responseEntity);
}
}
An example of using this method in the code (assume the variable img has already been declared containing the File object of the image you wish to upload):
// Notice regular params can be included in the address
String address = "http://www.server.com/upload.php?t=ABC123&id=12";
String resp = uploadRequest(address, img);
// Handle response
PHP SIDE
For the server-side script, the text params can be accessed normally through PHP's $_REQUEST object:
$token = $_REQUEST['token'];
$id = $_REQUEST['id'];
And the image that was uploaded can accessed by using the information stored in the PHP $_FILES object (See the PHP docs for more info):
$img = $_FILES['img'];
i have application that picks the file from a dedicated path on my device and sends it to server.
I m using ksoap2 lib to call .NET webservice to send my file to server. i am using Base 64 encoding.
I can send file with max size of 1MB without encryption and 850Kb with encryption. Encyrption algorithm i am using is 3DES.
If i try to send files larger than above size i get following error: Caused by: java.lang.OutOfMemoryError at org.ksoap2.transport.HttpTransportSE.call(HttpTransportSE.java:121)
My Test environment: Android emulator with API Level 8, Android 2.2 and SDCard memory 512 MB
Is it that i am missing out something? Can using BLOB help me in this scenario
Is there any way to send larger file? i have heard of sending data chunks but have no idea on that . any link or sample code will really help.
to get file data using following code:
here url = where file is stored
public byte[] getFileData( String vURL){
instream = new FileInputStream(vURL);
size = (int) vURL.length();
fileContent = new byte[size];
instream.read(fileContent);
}
Encode the data using following code:
byte[] res = Utilities.getFileData(file);
String mdata = android.util.Base64.encodeToString(res, android.util.Base64.DEFAULT);
calling server side web service and sending data to server
SoapObject request = new SoapObject(nameSpace, methodName);
if (fileData != null && !fileData.equals("")) {
request.addProperty("vBLOBData", fileData);
}
SoapSerializationEnvelope envelope = getEnvelope(request);
HttpTransportSE ht = new HttpTransportSE(url); // ,3000
ht.debug = true;
ht.call(soapAction, envelope);
response = (envelope.getResponse()).toString();
Not able to send filedata more than 1 MB.
Thanks in advance
I don't know what you are trying to achieve, but why you don't divide your file into parts and send each part individually into a loop or into an android background service using a timer that sends a part every x seconds.
Try to set your buffer to 1024 before sending it , its about the limit of your buffer size and your ram
Use GZip Zipping algorithm to zip large file from mobile side; use same unzipping from server.
Also use MultipartEntity can help to upload large file content.
If compression doesn't help - as mentioned in previous post - you will probably have to segment the message yourself.
Segmentation