Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
How would you add system recovery in a multi-threaded environment? For eg: if you have a system where multiple threads pickup files and process them and persist them in database, how would the system recover if there is a database failure you
dont want to process the trades again?
There are many different ways to answer this question depending on your system setup. A little bit more of an explanation would help. I have still provided an example that could possibly work for you though.
I would probably look at a way to mark a file as in process (i.e. database record or moving to a different directory to process the file). Then I would mark the file as finished when it processed (moving it or doing it some other way).
There is a still a possibility of failure after finish processing and marking the file as finished. However, this would limit the amount of files you need to look at for recovery.
If you can keep track of what files were read in the same database you can batch all your databases changes as well as the flag to mark the file as read. You can avoid committing the connection until your have done the changes and flagged the file.
This also has an issue of if the database crashes mid commit, but at the same time you would probably have to restore a backup in this instance and rerun all the files again.
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed last year.
Improve this question
I need to delete files in the background service after a certain time, is it possible to implement this and in what way?
You can try WorkManager. It will delete your file...
Question is pretty vague. Please be specific on which OS you want to this, any specific technology, particular environment... queue of questions on your questions i so endless.
Though to answer at high level:
Solution 1: If windows / Linux, you can write a batch / shell script which you can mark into Windows scheduler / Linux cron job. Batch will have all logic to what to delete from which path, any specific file or starting with etc. scheduler / cron job will decide at what time you want to do this. For this timeline you need to check syntax and for both OS its different. Simple search and you will get everything.
Solution 2: If you want o do it by code, you can write a Spring scheduler code (this can be in any language, but as Java developer, I will prefer this) which will interact with file system and perform file deletion for you after particular interval. For this you may need an application server to deploy.
If application server is available on machine then this will be easy job for you, or else easiest thing to do is Solution 1.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
I have been working on AWS Lambda function with some custom Java codes. It's codes had to get long execution time be required but Lambda has execution timeout as 900 seconds of maximum. So I intended that to be saved memory state of process to S3 as file before the timeout and then load that file to be executed from S3 on next execution time.
How to save all state of process to file and then load to execute that from saved process state?
Your JRE won't support such a risky feature but if you're looking to run extraneous tasks I would not suggest saving process states anyway. If you can add some code and details you'll get a more precise solution to your problem, however some basic pointers...
Make sure the functions you're processing data with can be dynamically paused and started
Write functions to save/load data from a file in any format (json, csv, etc)
Write a function to identify when your dp task is complete
Hard code a limit to load, process, then write in that order
Batch the process in series until you're notified that it's complete
Again this question is really ambiguous so my answer may not be at all what you need. In either case what you want to save is data, not processes. In theory the computer itself is capable of saving the states of all registers, the stack, and program counter in assembly but that's a pretty big no no for a lot of reasons that aren't really part of this discussion
Sorry for my ambiguous question to you!
I knew how a application basically save it's data to file and restore that next runtime.
But my work about the question is wrapping Lambda handler to be included ETL migration job application that has to take time over Lambda maximum timeout.
I was already developed Lambda handler building framework being able to include ETL job and deploy the handler to lambda function but I didn't solve that problem of Lambda timeout.
Meanwhile, I thought about like 'jmap' tool. I just guessed that if 'jmap' can dump heap memory, even it could be restored from dumped file.
As collectively thinking, There isn't a way to solve that problem with my guess.
So I could like better to make application data store architecture in ETL job.
Because of ETL job program is built by other ETL solution, I feel some inconvenience.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I'm trying to write a program in Java for personal accounting. My initial plan was for user to log in, and the program would look him up in a text file and let him in. Then there would be a JTable which would load all his transactions (from a txt) and show them. He would then add new ones or edit/delete ones already there. The program would find the line and change it.
But as I started the implementation, I quickly found out that the manipulation with the text file was very exhausting.
I thought about SQL database, or JSON files, but I don't know, if that's a good idea, and where to start. I'm rather new to java, so even opening a text file was a bit of a hassle for me.
Any thoughts?
Thank you.
Since it is for personal accounting and likely small, you could think of it like any document editing program (Notepad, Word, Excel, ...), meaning:
No login. Each person will have a separate file chosen when you start program.
Load entire file into memory.
Nothing is saved until user clicks "Save" (unless you want some auto-recovery logic in case of program/machine crash).
That means that there are only two operations on the file (Load and Save), and both should be fairly simple.
Advantage: Simple and very fast.
Limitation: Memory constraint if file grows very large, and potential for data loss if auto-recovery/auto-save is not implemented.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I am working on a distributed system project. I am required to create a program that allow multiple users to edit on the same text file concurrently. I have been looking around online for a relatively simple solution but I haven't found one. I've read about BlockingQueue but that doesn't make much sense to me. I have talked to my TA and he suggested that each client will have a copy of the text file, which will they edit. Those sub-files will then be merged to the main copy. However, the problem is that I won't be able to update those sub-files while they are editing the text file.
As I understand it you want an online text editor with which you can modify files concurrently and the updates should happen as real-time as possible.
Here is what I would do:
If a user opens a file he receives a copy of it and the user is added to the list of users which have opened this file.
After a user makes a change, wait X seconds to accumulate further changes and then send them to the server.
The server processes the change requests for a file one after the other (different files can be done in parallel of course and it can also be done more intelligently by splitting files into chunks which can be processed independently in parallel too, at least on the server side [this is only partially true, two changes can be processed in parallel if the intersection of the set of affected chunks in change A and change B is empty])
A change request is either acceppted and all the changes are broadcastest to all user that have the file opened or the change is refused. This can be pretty complicated. The easiest way is to keep track with a version number and refuse all changes that come from older versions. (If you have a version number for each chunk and the size of the chunks is small, you will only run into rejections if two or more people are working at the almost same location in a document at the same time. But it will be quite some work, consider you will have to split/merge/delete/insert chunks if they become too big or small.)
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I am beginning to write a basic "study-buddy" program as a side project. One important feature I want to implement is that the program can access the state of other programs running to prevent you from accessing them / yell at you. For instance, if you had Chrome open to Facebook, or if you launched a video game.
First off, is this even possible/reasonable to accomplish in Java? Second, specifically with Chrome, how can I access the programs state from another program that I am writing? More generally, how can I access ALL programs running on the computer and check to see whether anything violates "study-permissible" programs?
I would put this as a comment, but my reputation point is not enough.
One way is using the commands the operating system provides. You can run a command with
Runtime.getRuntime().exec("<command name>");
This will give you the related process and you can get the output of that process just as manually running the process. Then, you can utilize the output.
Basically if the OS provides you that information manually, you should be able to get the information within Java.