I have an app in which I have to read a .txt file so that I can store some values and keep them. This is working pretty well, except for the fact that I want to make those values non-readable or "non-understandable" for external users.
My idea was to convert the file content into Hex or Binary and, in the reading process, change it back to Char. The thing is that I don't have access to methods such as String.Format due to my compiler.
Here's how I'm currently reading and keeping the values:
byte[] buffer = new byte[1024];
int len = myFile.read(buffer);
String data = null;
int i=0;
data = new String(buffer,0,len);
Class to open and manipulate the file:
public class File {
private boolean debug = false;
private FileConnection fc = null;
private OutputStream os = null;
private InputStream is = null;
private String fileName = "example.txt";
private String pathName = "logs/";
final String rootName = "file:///a:/";
public File(String fileName, String pathName) {
super();
this.fileName = fileName;
this.pathName = pathName;
if (!pathName.endsWith("/")) {
this.pathName += "/"; // add a slash
}
}
public boolean isDebug() {
return debug;
}
public void setDebug(boolean debug) {
this.debug = debug;
}
public void write(String text) throws IOException {
write(text.getBytes());
}
public void write(byte[] bytes) throws IOException {
if (debug)
System.out.println(new String(bytes));
os.write(bytes);
}
private FileConnection getFileConnection() throws IOException {
// check if subfolder exists
fc = (FileConnection) Connector.open(rootName + pathName);
if (!fc.exists() || !fc.isDirectory()) {
fc.mkdir();
if (debug)
System.out.println("Dir created");
}
// open file
fc = (FileConnection) Connector.open(rootName + pathName + fileName);
if (!fc.exists())
fc.create();
return fc;
}
/**
* release resources
*/
public void close() {
if (is != null)
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
is = null;
if (os != null)
try {
os.close();
} catch (IOException e) {
e.printStackTrace();
}
os = null;
if (fc != null)
try {
fc.close();
} catch (IOException e) {
e.printStackTrace();
}
fc = null;
}
public void open(boolean writeAppend) throws IOException {
fc = getFileConnection();
if (!writeAppend)
fc.truncate(0);
is = fc.openInputStream();
os = fc.openOutputStream(fc.fileSize());
}
public int read(byte[] buffer) throws IOException {
return is.read(buffer);
}
public void delete() throws IOException {
close();
fc = (FileConnection) Connector.open(rootName + pathName + fileName);
if (fc.exists())
fc.delete();
}
}
I would like to know a simple way on how to read this content. Binary or Hex, both would work for me.
So, with some understanding of the question, I believe you're really looking for a form of obfuscation? As mentioned in the comments, the easiest way to do this is likely a form of cipher.
Consider this example implementation of a shift cipher:
Common
int shift = 11;
Writing
// Get the data to be wrote to file.
String data = ...
// cipher the data.
char[] chars = data.toCharArray();
for (int i = 0; i < chars.length; ++i) {
chars[i] = (char)(chars[i] + shift);
}
String cipher = new String(chars);
// Write the data to the cipher file.
...
Reading
// Read the cipher file.
String data = ...
// Decipher the data.
char[] chars = data.toCharArray();
for (int i = 0; i < chars.length; ++i) {
chars[i] = (char)(chars[i] - shift);
}
String decipher = new String(chars);
// Use data as required.
...
Here's an example implementation on Ideone. The output:
Data : I can read this IP 192.168.0.1
Cipher : T+nly+}plo+st~+T[+<D=9<AC9;9<
Decipher: I can read this IP 192.168.0.1
I tried to keep this as low level as possible in order to satisfy the Java 3 requirement.
Note that this is NOT secure by any means. Shift ciphers (like most ciphers in a bubble) are trivial to break by malicious entities. Please do not use this if security is an actual concern.
Your solution is too complex. With java 8, you can try :
String fileName = "configFile.txt";
try (Stream<String> stream = Files.lines(Paths.get(fileName))) {
//TO-DO .Ex
stream.forEach(System.out::println);
} catch (IOException e) {
e.printStackTrace();
}
Related
I have multiple BLOBs that I want to concat to one another, in order to get one joined BLOB. In the process, the contents of the BLOBs may not be stored in memory.
My first idea was to merge the streams like that:
long size = blobs.get(0).length();
InputStream res = blobs.get(0).getBinaryStream();
for (int i = 1; i < blobs.size(); i++){
res = Stream.concat(res, blobs.get(i).getBinaryStream());
size += blobs.get(i).length();
}
Blob blob = Hibernate.getLobCreator(session).createBlob(res, size);
However, this obviously only works with Java 8 streams (not normal BinaryStreams) - and we use Java 7 anways.
My second idea then was to join the BLOBs by directly writing into its stream:
public Blob joinBlobsForHibernate(final Session session, final List<Blob> blobs) throws SQLException {
final LobCreator lc = Hibernate.getLobCreator(session);
final Blob resBlob = lc.createBlob(new byte[0]);
try {
OutputStream stream = resBlob.setBinaryStream(1);
for (final Blob blob : blobs) {
pipeInputStream(blob.getBinaryStream(), stream);
}
return resBlob;
} catch (IOException | SQLException e){
logger.error("Creating the blob threw an exception", e);
return null;
}
}
(pipeInputStream merely pipes the content of one stream into the other:
private void pipeInputStream (final InputStream is, final OutputStream os) throws IOException {
final int buffSize = 128000;
int n;
final byte[] buff = new byte[buffSize];
while ((n = is.read(buff)) >= 0){
os.write(buff, 0, n);
}
)
This however yields in the following exception:
java.lang.UnsupportedOperationException: Blob may not be manipulated from creating session
And besides, I have the suspicion that the BLOB would still temporarily store the whole content in memory.
As a third try I tried using a custom InputStream:
/**
* Combines multiple streams into one
*/
public class JoinedInputStream extends InputStream {
private List<InputStream> parts;
private List<InputStream> marked;
public JoinedInputStream(final List<InputStream> parts) {
this.parts = parts;
}
#Override
public int read() throws IOException {
int res = -1;
while (res == -1 && parts.size() > 0) {
try {
if ((res = parts.get(0).read()) == -1) {
// The stream is done, so we won't try to read from it again
parts.remove(0);
}
} catch (IOException e) {
e.printStackTrace();
throw e;
}
}
return res;
}
#Override
public synchronized void reset() throws IOException {
parts = new ArrayList<>(marked);
if (parts.size() > 0) {
parts.get(0).reset();
}
}
#Override
public synchronized void mark(final int readlimit) {
marked = new ArrayList<>(parts);
if (marked.size() > 0)
marked.get(0).mark(readlimit);
}
#Override
public boolean markSupported() {
return true;
}
#Override
public void close() throws IOException {
super.close();
for (final InputStream part : parts) {
part.close();
}
parts = new ArrayList<>();
marked = new ArrayList<>();
}
}
The BLOB could then be joined like that (don't mind the unnecessary functions, they have other uses):
#Override
public Blob createBlobForHibernate(final Session session, final InputStream stream, final long length) {
final LobCreator lc = Hibernate.getLobCreator(session);
return lc.createBlob(stream, length);
}
#Override
public Blob createBlobForHibernate(final Session session, final List<InputStream> streams, final long length) {
final InputStream joined = new JoinedInputStream(streams);
return createBlobForHibernate(session, joined, length);
}
#Override
public Blob joinBlobsForHibernate(final Session session, final List<Blob> blobs) throws SQLException {
long length = 0;
List<InputStream> streams = new ArrayList<>(blobs.size());
for (final Blob blob : blobs) {
length += blob.length();
streams.add(blob.getBinaryStream());
}
return createBlobForHibernate(session, streams, length);
}
However, this results in the following error (when persisting the newly created entity with the joined BLOB):
Caused by: java.sql.SQLException: LOB-Lese-/Schreibfunktionen aufgerufen, während ein anderer Lese-/Schreibvorgang ausgeführt wird: getBytes()
at oracle.jdbc.driver.T4CConnection.getBytes(T4CConnection.java:3200)
at oracle.sql.BLOB.getBytes(BLOB.java:391)
at oracle.jdbc.driver.OracleBlobInputStream.needBytes(OracleBlobInputStream.java:166)
... 101 more
Or in English:
Lob read/write functions called while another read/write is in progress: getBytes()
I already tried setting hibernate.temp.use_jdbc_metadata_defaults to false (as suggested in this post) - in fact, we already had this property set beforehand and it didn't help.
I have A big file and i want to upload that in Server side. it's very important when occured any problem (like interrupting the internet or power cut ...) if i retry to upload, file uploaded from resume and doesn't need to send file from beginning.
I try this approach with sending file chunks but it seems that's not a good way, because a send chunks(byte arrays) directly in response Entity and this isn't good idea.
whatever if anybody can develop this approach and make this code a better code with better performance i appreciate that. does anybody known Best practice way to doing that??
and if u like my code, vote me
thanks :)
RestController
#RestController
#RequestMapping("/files")
public class Controller {
#Autowired
private MyService service;
#PutMapping("/upload/resume")
public Mono<ResponseEntity> uploadWithResume(#RequestPart("chunk")byte[] chunk,
#RequestPart("fileName")String fileName,
#RequestParam("length")Long length
) throws ParseException {
try {
return service.fileResumeUpload(chunk, fileName, length);
} catch (IOException e) {
e.printStackTrace();
return Mono.just(ResponseEntity.status(HttpStatus.PERMANENT_REDIRECT).build());
}
}
#RequestMapping(value = "/get/uploaded/size", method = RequestMethod.HEAD)
public Mono<ResponseEntity> getUploadedSize(#RequestParam("fileName") String fileName) throws IOException {
if (Files.exists(Paths.get("src/main/resources/" + fileName))) {
String size = String.valueOf(Files.size(Paths.get("src/main/resources/" + fileName)));
return Mono.just(ResponseEntity.ok()
.header("upload-offset", size)
.build());
} else{
return Mono.just(ResponseEntity.notFound()
.header("upload-offset" , "0").build());
}
}
}
Service
public Mono<ResponseEntity> fileResumeUpload(byte[] chunk , String fileName,long length) throws IOException, ParseException {
BufferedOutputStream out = new BufferedOutputStream(new FileOutputStream("src/main/resources/" + fileName, true));
boolean uploaded = true;
try {
out.write(chunk);
} catch (IOException e) {
uploaded = false;
System.err.println("io exception");
} finally {
if (uploaded) {
out.close();
return Mono.just(ResponseEntity.ok()
.header("expiration-date", getExpirationDate())
.build());
} else {
out.close();
return Mono.just(ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).build());
}
}
}
Sending chunks with webTestClient
#Test
public void test1_upload_Expected_200StatusCode(){
try {
String fileName = "film.mkv";
RandomAccessFile raf = new RandomAccessFile(new File("src/test/resources/" + fileName), "rw");
long realSize = raf.length();
List<String> strings = webTestClient.head().uri("/files/get/uploaded/size?fileName=" + fileName)
.exchange().expectBody().returnResult().getResponseHeaders().get("upload-offset");
long uploadedSize = Long.valueOf(strings.get(0));
boolean f = false;
int sizeBuffer = 256 * 1024;
byte[] buffer = new byte[sizeBuffer];
MultiValueMap<String, Object> formData;
WebTestClient.ResponseSpec exchange = null;
System.out.println("first uploaded Size ; " + uploadedSize);
raf.seek(uploadedSize);
while (raf.read(buffer) != -1) {
formData = new LinkedMultiValueMap<>();
formData.add("fileName", fileName);
formData.add("chunk", buffer);
formData.add("length", realSize);
exchange = webTestClient.put().uri("/files/upload/resume")
.contentType(MediaType.MULTIPART_FORM_DATA)
.body(BodyInserters.fromMultipartData(formData))
.exchange();
exchange.expectStatus().isOk();
if (exchange.expectBody().returnResult().getStatus().is5xxServerError()) {
return;
}
if (uploadedSize + 256 * 1024 > realSize) {
sizeBuffer = ((int) (realSize - uploadedSize));
System.out.println(sizeBuffer);
uploadedSize = uploadedSize + sizeBuffer;
System.out.println(uploadedSize);
buffer = new byte[sizeBuffer];
f=true;
} else uploadedSize = uploadedSize + sizeBuffer;
if (f) System.out.println(uploadedSize);
//System.out.println(uploadedSize);
float percent = ((float) uploadedSize / realSize * 100);
System.out.format("%.2f\n", percent);
}
if (exchange!=null)
exchange.expectStatus().isOk();
}
catch (Exception e){
e.printStackTrace();
System.err.println("channel closed!!!");
}
}
Here is the (without most of the functions) definition of a class called note.
public class Note
{
private String text;
String fileName = "";
NoteManager noteManager = null;
List<String> hyperlinks = new ArrayList<String>();
public static final int BUFFER_SIZE = 512;
public Note(NoteManager noteManager) {
this.noteManager = noteManager;
this.text = "";
}
public Note(NoteManager noteManager, String content) {
this(noteManager);
if (content == null)
setText("");
else
setText(content);
}
public Note(NoteManager noteManager, CharSequence content) {
this(noteManager, content.toString());
}
....some functions....
public static Note newFromFile(NoteManager noteManager, Context context,
String filename) throws IOException
{
FileInputStream inputFileStream = context.openFileInput(filename);
StringBuilder stringBuilder = new StringBuilder();
byte[] buffer = new byte[BUFFER_SIZE];
int len;
while ((len = inputFileStream.read(buffer)) > 0)
{
String line = new String(buffer, 0, len);
stringBuilder.append(line);
buffer = new byte[Note.BUFFER_SIZE];
}
Note n = new Note(noteManager, stringBuilder.toString().trim());
n.fileName = filename;
inputFileStream.close();
return n;
}
.... some functions attributed to this class
}
These notes are managed by a class called NoteManager.java, which I have abbreviated below:
public class NoteManager
{
Context context=null;
ArrayList<Note> notes = new ArrayList<Note>();
..... some functions...
public void addNote(Note note)
{
if (note == null || note.noteManager != this || notes.contains(note)) return;
note.noteManager = this;
notes.add(note);
try
{
note.saveToFile(context);
} catch (IOException e)
{
e.printStackTrace();
}
}
....some functions....
public void loadNotes()
{
String[] files = context.fileList();
notes.clear();
for (String fname:files)
{
try
{
notes.add(Note.newFromFile(this, context, fname));
} catch (IOException e)
{
e.printStackTrace();
}
}
}
}
public void addNote(Note note)
{
if (note == null || notes.contains(note)) return;
note.noteManager = this;
notes.add(note);
try
{
note.saveToFile(context);
} catch (IOException e)
{
e.printStackTrace();
}
}
I am trying to work out why this notepad app creates random new notes when the app is fully shutdown and then reopened, however I just cannot see what the problem is. I have cut out all the functions which didnt seem to relate to the problem, so the logical error must be here somewhere.
How does one go about finding what I am guessing to be some kind of circular reference or lack of checks?
Android typically uses UTF-8, with multi-byte characters. Creating a new String on a arbitrary byte sub-array can have issues at begin and end, if you deviate from ASCII.
public static Note newFromFile(NoteManager noteManager, Context context,
String filename) throws IOException
{
Path path = Paths.get(filename);
byte[] bytes = Files.readAllBytes(path);
String content = new String(bytes, "UTF-8");
Note n = new Note(noteManager, content.trim());
n.fileName = filename;
noteManager.add(n); // One registration?
return n;
}
The problem of having multiple instances of a node might need the addition within newFromFile or maybe an extra check:
public void addNote(Note note)
{
if (note == null || note.noteManager != this || notes.contains(note)) {
return;
}
note.noteManager = this;
notes.add(note);
And finally a Note must be well defined.
public class Note extends Comparable<Note> {
private NoteManager noteManager:
private final String content; // Immutable.
public NoteManager(NoteManager noteManager, String content) {
this.noteManager = noteManager;
this.content = content;
}
... compare on the immutable content
... hashCode on content
Not being to be able to change the content, and comparing on the string content, means notes cannot be doubled, change in the set, mixing up the set ordering.
My application on Appengine create a csv file with more 65535 rows
But, I have an error of type OutOfMemoryError when writing :
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2271)
at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
White this code :
public static byte[] joinLines(Collection<String> lines) {
final ByteArrayOutputStream stream = new ByteArrayOutputStream();
boolean firstElement = true;
for (final String part : lines) {
String value = part + LINE_SEPARATOR;
if (firstElement) {
value = addExcelPrefix(value);
firstElement = false;
}
final int currentSize = value.length();
try {
stream.write(value.getBytes(ENCODING), 0, currentSize); // OutOfMemoryError HERE
} catch (UnsupportedEncodingException e) {
LOGGER.info(e.getMessage());
}
}
return stream.toByteArray();
}
So I used FileBackedOutputStream of Guava for solve the problem of OutOfMemoryError :
public static byte[] joinLines(Collection<String> lines) throws IOException {
final FileBackedOutputStream stream = new FileBackedOutputStream(THRESHOLD, true);
boolean firstElement = true;
for (final String part : lines) {
String value = part + LINE_SEPARATOR;
if (firstElement) {
value = addExcelPrefix(value);
firstElement = false;
}
final int currentSize = value.length();
try {
stream.write(value.getBytes(ENCODING), 0, currentSize);
} catch (IOException e) {
LOGGER.error(e.getMessage());
}
}
return stream.asByteSource().read();
}
But, on appengine, I now an error of type SecurityException when creating of temporary file :
java.lang.SecurityException: Unable to create temporary file
at java.io.File.checkAndCreate(File.java:2083)
at java.io.File.createTempFile(File.java:2198)
at java.io.File.createTempFile(File.java:2244)
at com.google.common.io.FileBackedOutputStream.update(FileBackedOutputStream.java:196)
at com.google.common.io.FileBackedOutputStream.write(FileBackedOutputStream.java:178)
How to allow create temporary file on Appengine with FileBackedOutputStream ?
In a bucket, how ?
Thanks
I used GcsService that solves my problem :
protected String uploadBytesForCsv(Map<Integer, Map<Integer, Object>> rows) throws IOException {
LOGGER.info("Get Bytes For Csv");
final Collection<String> lines = cellsToCsv(rows);
LOGGER.info("number line : " + lines.size());
boolean firstElement = true;
final String fileName = getFileName();
final GcsFilename gcsFilename = new GcsFilename(config.getBucketName(), fileName);
final GcsService gcsService = GcsServiceFactory.createGcsService();
final GcsOutputChannel outputChannel = gcsService.createOrReplace(gcsFilename, GcsFileOptions.getDefaultInstance());
for (final String part : lines) {
final ByteArrayOutputStream stream = new ByteArrayOutputStream();
String value = part + LINE_SEPARATOR;
if (firstElement) {
value = addExcelPrefix(value);
firstElement = false;
}
final int currentSize = value.length();
try {
stream.write(value.getBytes(ENCODING), 0, currentSize);
outputChannel.write(ByteBuffer.wrap(stream.toByteArray()));
} catch (UnsupportedEncodingException e) {
LOGGER.info(e.getMessage());
}
stream.flush();
stream.close();
}
outputChannel.close();
return new UrlBuilder(config.getStorageUrlForExport())
.setBucketName(config.getBucketName())
.setFilename(fileName).build();
}
In my servlet I am running a few command line commands in background, I've successfully printed output on console.
My doGet()
public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException
{
String[] command =
{
"zsh"
};
Process p = Runtime.getRuntime().exec(command);
new Thread(new SyncPipe(p.getErrorStream(), response.getOutputStream())).start();
new Thread(new SyncPipe(p.getInputStream(), response.getOutputStream())).start();
PrintWriter stdin = new PrintWriter(p.getOutputStream());
stdin.println("source ./taxenv/bin/activate");
stdin.println("python runner.py");
stdin.close();
int returnCode = 0;
try {
returnCode = p.waitFor();
}
catch (InterruptedException e) {
e.printStackTrace();
} System.out.println("Return code = " + returnCode);
}
class SyncPipe implements Runnable
{
public SyncPipe(InputStream istrm, OutputStream ostrm) {
istrm_ = istrm;
ostrm_ = ostrm;
}
public void run() {
try
{
final byte[] buffer = new byte[1024];
for (#SuppressWarnings("unused")
int length = 0; (length = istrm_.read(buffer)) != -1; )
{
// ostrm_.write(buffer, 0, length);
((PrintStream) ostrm_).println();
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
private final OutputStream ostrm_;
private final InputStream istrm_;
}
Now, I want to save the ostrm_ to a string or list, and use that inside doGet()
How to achieve this?
==============================EDIT============================
Based on answers below, I've edited my code as follows
int length = 0; (length = istrm_.read(buffer)) != -1; )
{
// ostrm_.write(buffer, 0, length);
String str = IOUtils.toString(istrm_, "UTF-8");
//((PrintStream) ostrm_).println();
System.out.println(str);
}
Now, How do I get the str in runnable class into my doGet()?
You can use Apache Commons IO.
Here is the documentation of IOUtils.toString() from their javadocs
Gets the contents of an InputStream as a String using the specified character encoding. This
method buffers the input internally, so there is no need to use a
BufferedInputStream.
Parameters: input - the InputStream to read from encoding - the
encoding to use, null means platform default Returns: the requested
String Throws: NullPointerException - if the input is null IOException
- if an I/O error occurs
Example Usage:
String str = IOUtils.toString(yourInputStream, "UTF-8");
You can call something like the following:
(EDIT: added also the client calls)
public void run() {
try
{
String out = getAsString(istrm_);
((PrintStream) ostrm_).println(out);
} catch (Exception e) {
e.printStackTrace();
}
}
public static String getAsString(InputStream is) throws Exception {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int cur = -1;
while((cur = is.read()) != -1 ){
baos.write(cur);
}
return getAsString(baos.toByteArray());
}
public static String getAsString(byte[] arr) throws Exception {
String res = "";
for(byte b : arr){
res+=(char)b;
}
return res;
}