I am developping an application using spring boot as framework. I have 2 methods the first one is deleting data from the database and the other one is deleting folder from the disk, so if i delete from the database and i can't delete from the disc all the operation will fail. So how can i do that with springboot ?
#Override
public ResponseEntity<?> delete(Long id) {
return libelleRepository.findById(id).map(libelle -> {
libelleRepository.delete(libelle);
return ResponseEntity.ok().build();
}).orElseThrow(() -> new GeneralResourceNotFoundException("Libelle not found with id " + id));
}
You can use the Spring's #Transactional for doing this.
Here is the sample code what I have tried. It performs a Database operation followed by a file operation in my example I'm trying to create a file. First am creating the file before performing the database operation and used TransactionSynchronizationAdapter to make sure the transaction is complete before commiting.
Code:
#Autowired
private UserService userService;
#Transactional
public String doFileOperation() {
File testFile = new File("C:\\test.txt");
TxnListener transactionListener = new TxnListener(testFile);
TransactionSynchronizationManager.registerSynchronization(transactionListener);
// DB Operation
userService.addUser();
// File Operation
List<String> lines = Arrays.asList("1st line", "2nd line");
try {
Files.write(Paths.get(testFile.getPath()),
lines,
StandardCharsets.UTF_8,
StandardOpenOption.CREATE,
StandardOpenOption.APPEND);
} catch (IOException e) {
e.printStackTrace();
}
return "";
}
public class TxnListener extends TransactionSynchronizationAdapter {
private File outputFile;
public TxnListener(File outputFile) {
this.outputFile = outputFile;
}
#Override
public void afterCompletion(int status) {
if (STATUS_COMMITTED != status) {
if (outputFile.exists()) {
if (!outputFile.delete()) {
System.out.println("Could not delete File" + outputFile.getPath() + " after failed transaction");
}
}
}
}
}
In case of exception during the Database operation afterCompletion will be called and the file will be deleted.
This way you can maintain the atomicity of the operation.
Related
I have a question about initializing data in Spring-Boot.
I have this :
#PostConstruct
public void run() {
try {
upload(path);
logger.info("Seeding dictionary database...");
} catch (IOException e) {
//.....
}
}
run() method read .json file and fills the database with information from the file when the app starts. But I also have a .sql file that also fills the database when starts. The table created from initialization from the .sql file is related to the table created from the .json file
When app starts i have
INSERT INTO "USER_DICTIONARY"("DICTIONARY_ID", "USER_ID") VALUES (1, 0)
I have this line in my import.sql but this causes errors because DICTIONARY_ID doesn't exist jet because it comes from .json file which is loaded after import.sql
The data retrieved from the .json file is needed to correctly map this table when the sql file is executing.
Is it possible to execute my run() methody before executing .sql file, or can it be solved in some other way ? If so, please help me find the answer.
One option to seed a database is to use the CommandLineRunner in Spring Boot.
Example:
#Component
public class UserDataLoader implements CommandLineRunner {
#Autowired
UserRepository userRepository;
#Override
public void run(String... args) throws Exception {
loadUserData();
}
private void loadUserData() {
if (userRepository.count() == 0) {
User user1 = new User("John", "Doe");
User user2 = new User("John", "Cook");
userRepository.save(user1);
userRepository.save(user2);
}
System.out.println(userRepository.count());
}
}
The CommandRunner interface will execute just after the application starts.
Another way is is to use a #EventListener that listens to the application’s ContextRefreshEvent.
Example:
#Component
public class DBSeed {
#EventListener
public void seed(ContextRefreshedEvent event) {
try {
upload(path);
logger.info("Seeding dictionary database...");
} catch (IOException e) {
//.....
}
}
}
I am working to build a REST API where a large amount of data from the Oracle database can be sent in chunks via streaming to client application (like a file download or direct stream).
I am getting Stream from JpaRepository as given below -
#Query("select u from UsersEntity u")
Stream<UsersEntity> findAllByCustomQueryAndStream();
But now challenge comes to write this stream to StreamingResponseBody Output stream
I tried by many ways but no success -
First Approach -
Stream<UsersEntity> usersResultStream = usersRepository.findAllByCustomQueryAndStream();
StreamingResponseBody stream = outputStream -> {
Iterator<UsersEntity> iterator = usersResultStream.iterator();
try (ObjectOutputStream oos = new ObjectOutputStream(outputStream)) {
while (iterator.hasNext()) {
oos.write(iterator.next().toString().getBytes());
}
}
};
Got Error -
java.sql.SQLException: Closed Resultset: next
at oracle.jdbc.driver.InsensitiveScrollableResultSet.next(InsensitiveScrollableResultSet.java:565) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
Second Approach -
StreamingResponseBody stream = new StreamingResponseBody() {
#Transactional(readOnly = true)
#Override
public void writeTo(OutputStream outputStream) throws IOException {
Stream<UsersEntity> usersResultStream = usersRepository.findAllByCustomQueryAndStream();
try (ObjectOutputStream oos = new ObjectOutputStream(outputStream)) {
usersResultStream.forEach(user->{
try {
oos.write(user.toString().getBytes());
} catch (IOException e) {
e.printStackTrace();
}
});
}
}
};
Got Error -
org.springframework.dao.InvalidDataAccessApiUsageException: You're trying to execute a streaming query method without a surrounding transaction that keeps the connection open so that the Stream can actually be consumed. Make sure the code consuming the stream uses #Transactional or any other way of declaring a (read-only) transaction.
I have uploaded practice code at the below-given link -
Sample POC Link
I don't have any experience with the streaming related task so please help me with this.
If I am in the wrong direction than suggest any other approach to do this within Spring Framework. Please share any reference links if available.
Finally, I resolved the problem by using the service layer. Initially, I was writing the complete logic in Controller Class which was creating the issue.
Controller Class -
#RestController
#RequestMapping("/api")
public class UsersController {
#Autowired
private UserService service;
#GetMapping(value = "/userstream")
public ResponseEntity<StreamingResponseBody> fetchUsersStream() {
StreamingResponseBody stream = this::writeTo;
return new ResponseEntity<>(stream, HttpStatus.OK);
}
private void writeTo(OutputStream outputStream) {
service.writeToOutputStream(outputStream);
}
}
Service Class -
#Service
public class UserService {
#Autowired
private UsersRepository usersRepository;
#Transactional(readOnly = true)
public void writeToOutputStream(final OutputStream outputStream) {
try (Stream<UsersEntity> usersResultStream = usersRepository.findAllByCustomQueryAndStream()) {
try (ObjectOutputStream oos = new ObjectOutputStream(outputStream)) {
usersResultStream.forEach(emp -> {
try {
oos.write(emp.toString().getBytes());
} catch (IOException e) {
e.printStackTrace();
}
});
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
Complete Code is available at github - https://github.com/bagesh2050/HttpResponseStreamingDemo
Still, I am willing for suggestions related to Http Streaming. Please provide if you have better ideas.
No sample shows "such complex" usage of StreamingResponseBody, and I fear it is "not possible" (at least I couldn't manage/fix it, with StreamingResponseBody and Stream query)
...but, what was possible:
Use findAll() (the normal unstreamed List-repo method) within StreamingResponseBody.
(But I understand the "need" of doing the web request asynchronously... and the db request "streamed"...)
Use Callable (async web request) and an #Async CompletableFuture<..> (async db request):
#RestController
#RequestMapping("/api")
public class UsersController {
#Autowired
private UsersRepository usersRepository;
#GetMapping(value = "/async/users")
public Callable<List<UsersEntity>> fetchUsersAsync() {
Callable callable = () -> {
return usersRepository.readAllBy().get();
};
return callable;
}
}
..and a Repository like:
#Repository
public interface UsersRepository extends JpaRepository<UsersEntity, Integer> {
#Async
CompletableFuture<List<UsersEntity>> readAllBy();
}
(see spring-samples)
.. don't forget to #EnableAsync on your application/configuration:
#org.springframework.scheduling.annotation.EnableAsync
#SpringBootApplication
public class Application { ... }
Sorry, it is not even an answer, but my findings - too long for a comment.
The asynchronous web request can be achieved in various ways. (see https://spring.io/blog/2012/05/10/spring-mvc-3-2-preview-making-a-controller-method-asynchronous/, https://niels.nu/blog/2016/spring-async-rest.html, and even not mentioned "reactive" api)
I am trying to delete every 100 records read from a file in 3 tables,using spring jdbc batch delete .If i wrap the logic inside a transactionTemplate, is it going to work as expected, e.g lets say i am creating 3 batch out of 300 records and wrapping the logic inside a transaction ,then is the transaction going to roll back 1st and 2nd batch, if 3rd batch got a problem.My code is included in the question for reference. I am writing the below code to achieve what i have explained above, is my code correct?
TransactionTemplate txnTemplate = new TransactionTemplate(txnManager);
txnTemplate.execute(new TransactionCallbackWithoutResult() {
#Override
public void doInTransactionWithoutResult(final TransactionStatus status) {
try {
deleteFromABCTable(jdbcTemplate, successList);
deleteFromDEFTable(jdbcTemplate, successList);
deleteFromXYZTable(jdbcTemplate, successList);
} catch (Exception e) {
status.setRollbackOnly();
throw e;
}
}
});
My delete methods :-
private void deleteFromABCTable(JdbcTemplate jdbcTemplate, List
successList) {
try {
jdbcTemplate.batchUpdate(
"delete from ABC where document_id in (select document_id
from ABC where item in(?)))",
new BatchPreparedStatementSetter() {
#Override
public void setValues(PreparedStatement ps, int i)
throws SQLException {
ps.setString(0, successList.get(i));
} });
} catch (Exception e) { }
I have the following scheduled piece of code in my Spring Boot Application:
#Scheduled(fixedDelay = DELAY_SECONDS)
private void processJobQueue() {
BlockingQueue<ReportDeliverable> jobQueue = JobQueueFactory.getJobQueueInstance();
while (!jobQueue.isEmpty()) {
//do stuff
if (rCount == 0) {
status = send(reportDeliverable);
if (status == TransferStatus.FAILURE) {
populateQueue(reportDeliverable);
}
if (status == TransferStatus.SUCCESS) { //write the metadata to database
int i = dbMetadataWriter.writeMetadata(reportDeliverable);
}
} else if (rCount == -1) {
populateQueue(reportDeliverable);
} else
logger.info("Record exists in MetaData for {}. Ignoring the File transfer....", reportDeliverable.getFilePath());
}
}
In my DBMetadataWriter component, the writeMetadataWriter() looks something like this:
#Component
public class DBMetadataWriter {
public int writeMetadata(final ReportDeliverable reportDeliverable) {
int nbInserted = 0;
try {
nbInserted = jdbcTemplate.update(PORTAL_METADATA_INSERT, insertDataValues);
} catch (Exception e) {
logger.error("Could not insert metadata for {}, Exception: {} ", reportDeliverable.toString(), e.getMessage());
}
return nbInserted;
}
In some cases, when writing the insert to the database, I get table space issues with the database at which point I think it would be wise for me to shut down the spring boot application until table space problems are resolved.
What would be the correct way to handle these rare cases? What technique can I use to gracefully shutdown the spring boot application and how can I do it in the above code?
My entry point class where I initially validate all my database connections before processing etc has the following...
#Component
public class RegisterReportSchedules implements ApplicationListener<ContextRefreshedEvent> {
#Autowired
private ApplicationContext applicationContext;
#Override
public void onApplicationEvent(ContextRefreshedEvent contextRefreshedEvent) {
}
private void shutdownApplication() {
int exitCode = SpringApplication.exit(applicationContext, (ExitCodeGenerator) () -> 0);
System.exit(exitCode);
}
}
You have exit() method on SpringApplication class, which can be used for exiting Spring boot application gracefully.
It requires 2 paramerter:
ApplicationContext
ExitCodeGenerator
For further reading:
https://docs.spring.io/spring-boot/docs/current/api/org/springframework/boot/SpringApplication.html#exit-org.springframework.context.ApplicationContext-org.springframework.boot.ExitCodeGenerator...-
Code Example:
#Autowired
public void shutDown(ExecutorServiceExitCodeGenerator exitCodeGenerator) {
SpringApplication.exit(applicationContext, exitCodeGenerator);
}
Call this method when you get exception for No Table space
I try make implementation for comparing the files before they are uploaded.
If file whith name is exist in system ask about create new version or just override it.
Here is the problem, how to get file name?
I can't use receiveUpload(), because after this method file is remove from upload component ?
The problem is that once you start an upload using the Upload component, it can only be interrupted by calling the interruptUpload() method, and you cannot resume anytime later.
The interruption is permanent.
This means you cannot pause in the middle of the upload to see if you already have the file in your system. You have to upload the file all the way.
Considering this drawback, you can sill check in your system if you have the file, after the upload finishes. If you have the file, you can show a confirmation dialog in which you decide wether to keep the file or overwrite.
The following is an example in which I check in the "system" (I just keep a String list with the filenames) if the file has already been uploaded:
public class RestrictingUpload extends Upload implements Upload.SucceededListener, Upload.Receiver {
private List<String> uploadedFilenames;
private ByteArrayOutputStream latestUploadedOutputStream;
public RestrictingUpload() {
setCaption("Upload");
setButtonCaption("Upload file");
addSucceededListener(this);
setReceiver(this);
uploadedFilenames = new ArrayList<String>();
}
#Override
public OutputStream receiveUpload(String filename, String mimeType) {
latestUploadedOutputStream = new ByteArrayOutputStream();
return latestUploadedOutputStream;
}
#Override
public void uploadSucceeded(SucceededEvent event) {
if (fileExistsInSystem(event.getFilename())) {
confirmOverwrite(event.getFilename());
} else {
uploadedFilenames.add(event.getFilename());
}
}
private void confirmOverwrite(final String filename) {
ConfirmDialog confirmDialog = new ConfirmDialog();
String message = String.format("The file %s already exists in the system. Overwrite?", filename);
confirmDialog.show(getUI(), "Overwrite?", message, "Overwrite", "Cancel", new ConfirmDialog.Listener() {
#Override
public void onClose(ConfirmDialog dialog) {
if (dialog.isConfirmed()) {
copyFileToSystem(filename);
}
}
});
}
private void copyFileToSystem(String filename) {
try {
IOUtils.write(latestUploadedOutputStream.toByteArray(), new FileOutputStream(filename));
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e2) {
e2.printStackTrace();
}
}
private boolean fileExistsInSystem(String filename) {
return uploadedFilenames.contains(filename);
}
}
Note that I have used 2 external libraries:
Apache Commons IO 2.4 (http://mvnrepository.com/artifact/commons-io/commons-io/2.4) for writing to streams
ConfirmDialog from Vaadin Directory (https://vaadin.com/directory#addon/confirmdialog)
You can get the code snippet for this class from Gist: https://gist.github.com/gabrielruiu/9960772 which you can paste into your UI and test it out.