I have a spring boot based application. I want to download a huge amount of data from a database as a text file through REST API.
Since the amount of data might be really huge, let's say at least 1_000_000 enries, I decided to use Stream provided by Java 8 (fortunately Spring DATA JPA allows streaming queries)
It works fine but I'd like to compress the downloaded result. I know that Tomcat provides such functionality as mentioned here.
But for some reasons the result file is not compressed. Can't find out what I'm doing wrong.
Take a look at the code that I've written so far:
Model
#Entity
#Table(name = "users")
public class User {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
private String name;
private Integer age;
public User() {
}
public User(String name, Integer age) {
this.name = name;
this.age = age;
}
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Integer getAge() {
return age;
}
public void setAge(Integer age) {
this.age = age;
}
#Override
public String toString() {
return "User{" +
"id=" + id +
", name='" + name + '\'' +
", age=" + age +
'}';
}
}
DAO Layer
public interface UserRepository extends JpaRepository<User, Long> {
#Query("SELECT u from User u")
#QueryHints(value = #QueryHint(name = HINT_FETCH_SIZE, value = "" + Integer.MIN_VALUE))
Stream<User> streamAll();
}
And finally Rest Controller
#RestController
public class DownloadController {
#Autowired
private UserRepository userRepository;
#PersistenceContext
EntityManager entityManager;
#RequestMapping(value = "/stream", method = RequestMethod.GET)
#Transactional(readOnly = true)
public void stream(HttpServletResponse response) {
response.addHeader("Content-Type", "text/plain");
response.addHeader("Content-Disposition", "attachment");
response.setCharacterEncoding("UTF-8");
try(Stream<User> users = userRepository.streamAll()) {
PrintWriter out = response.getWriter();
users.forEach(u -> {
out.write(u.toString());
out.write("\n");
entityManager.detach(u);
});
out.flush();
} catch (Exception e) {
throw new RuntimeException("Exception occurred while exporting results", e);
}
}
}
In the application.properties file I added server.compression.enabled=true and since I have set header "Content-Type", "text/plain" I guess it should be compressed by default as mentioned here.
Any ideas would be appreciated.
Thanks in advance.
Related
I developed Spring Boot CRUD application. The database I have connected is PostgreSQL. #GetMapping is working properly and an empty array of objects can be retrieved by the GET request. But in #PostMapping, the POST request gives a 404 error.
đź“ŚSpringRecapApplication.java
package com.example.SpringRecap;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.security.servlet.SecurityAutoConfiguration;
import org.springframework.web.bind.annotation.*;
import java.util.List;
#RestController
#SpringBootApplication(exclude = SecurityAutoConfiguration.class)
//#RequestMapping("api/v1/customers")
//#EnableWebMvc
#RequestMapping(name = "api/v1/customers" ,method = RequestMethod.POST)
public class SpringRecapApplication {
//dependency injection
private static CustomerRepository customerRepository;
public SpringRecapApplication(CustomerRepository customerRepository) {
this.customerRepository = customerRepository;
}
public static void main(String[] args) {
SpringApplication.run(SpringRecapApplication.class, args);
}
#GetMapping
public List<Customer> getCustomer() {
return customerRepository.findAll();
}
record NewCustomerRequest(
String name,
String email,
Integer age
) {
#PostMapping
public void addCustomer(#RequestBody NewCustomerRequest newCustomerRequest) {
Customer customer = new Customer();
customer.setAge(newCustomerRequest.age());
customer.setName(newCustomerRequest.name());
customer.setEmail(newCustomerRequest.email());
customerRepository.save(customer);
}
}
}
customerRepository.save(customer); doesn't allow to make the dependency injection final. ( private static CustomerRepository customerRepository;). IDEA suggests making it static. But it didn't work. When I was using #RequestMapping("api/v1/customers"), a 405 error was received. Then I fixed that issue by doing as below,
#RequestMapping(name = "api/v1/customers" ,method = RequestMethod.POST)
đź“ŚCustomerRepository.java
package com.example.SpringRecap;
import org.springframework.data.jpa.repository.JpaRepository;
public interface CustomerRepository extends JpaRepository<Customer,Integer> {
}
đź“ŚCustomer.java
package com.example.SpringRecap;
import jakarta.persistence.*;
import java.util.Objects;
#Entity
public class Customer {
#Id
#SequenceGenerator(
name = "customer_id_sequence",
sequenceName = "customer_id_sequence",
allocationSize = 1
)
#GeneratedValue(
strategy = GenerationType.SEQUENCE,
generator = "customer_id_sequence"
)
private Integer id;
private String name;
private String email;
private Integer age;
public Customer(Integer id, String name, String email, Integer age) {
this.id = id;
this.name = name;
this.email = email;
this.age = age;
}
public Customer() {
}
public Integer getId() {
return id;
}
public void setId(Integer id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getEmail() {
return email;
}
public void setEmail(String email) {
this.email = email;
}
public Integer getAge() {
return age;
}
public void setAge(Integer age) {
this.age = age;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
Customer customer = (Customer) o;
return Objects.equals(id, customer.id) && Objects.equals(name, customer.name) && Objects.equals(email, customer.email) && Objects.equals(age, customer.age);
}
#Override
public int hashCode() {
return Objects.hash(id, name, email, age);
}
#Override
public String toString() {
return "Customer{" +
"id=" + id +
", name='" + name + '\'' +
", email='" + email + '\'' +
", age=" + age +
'}';
}
}
Postman:
Please put a comment if further information is needed to get the solution.
The problem with your code is that you specified the POST endpoint as part of your DTO and not as part of your controller. As your DTO is not a Spring managed bean, Spring won't map the URL to your endpoint. Anyways, you should move your endpoints into a seperate class. Example:
#RestController
#RequestMapping("api/v1/customers")
public class CustomerController {
private final CustomerRepository customerRepository;
public SpringRecapApplication(CustomerRepository customerRepository) {
this.customerRepository = customerRepository;
}
#GetMapping
public List<Customer> getCustomer() {
return customerRepository.findAll();
}
#PostMapping
public void addCustomer(#RequestBody NewCustomerRequest newCustomerRequest) {
Customer customer = new Customer();
customer.setAge(newCustomerRequest.age());
customer.setName(newCustomerRequest.name());
customer.setEmail(newCustomerRequest.email());
customerRepository.save(customer);
}
// Helper classes
record NewCustomerRequest(String name, String email, Integer age) { }
}
It would be best if you moved your DTO in a seperate class as well. I recommend placing the DTOs in a dto package and your controllers in a controller package.
Two side notes: you shouldn't expose your entities via your API. You should use DTOs for incoming and outgoing data. Check out lombok and mapstruct, they make this pretty easy.
I am a beginner to using SpringBoot, therefore I am following along with this guide: https://www.youtube.com/watch?v=9SGDpanrc8U
I am at the 'Testing Post Request' section of the video. At 1:15:00 of the video, he tests his Post request successfully. I have taken every step that he has taken and my code is identical (to my knowledge) to his, but instead of getting back a 200, I am getting back a 404 and this is what is produced in the logs:
HTTP/1.1 404 Not Found
Server: nginx/1.23.0
Date: Thu, 04 Aug 2022 20:25:49 GMT
Content-Type: text/html
Content-Length: 153
Connection: keep-alive
<html>
<head>
<title>404 Not Found</title>
</head>
<body>
<center>
<h1>404 Not Found</h1>
</center>
<hr>
<center>nginx/1.23.0</center>
</body>
</html>
I'd be happy to share this demo project from my GitHub so the full code I have written can be accessed and reviewed, unless someone is able to understand my problem from the description and can provide a way for me to debug this issue or provide steps I can take to resolve the issue. I'd really like to understand the root cause of this issue if possible.
To reproduce, I use the IntelliJ HTTP Client (https://www.jetbrains.com/help/idea/http-client-in-product-code-editor.html)
...and do the following POST:
###
POST http://localhost:8080/api/v1/student
Content-Type: application/json
{
"name": "Whitney",
"email": "wrob#apple.com",
"dob": "1987-12-09"
}
The above POST will reproduce the 404 error.
I am attempting to POST this student object into my Database.
My StudentController class is set up like this:
#RestController
#RequestMapping(path = "api/v1/student")
public class StudentController {
private final StudentService studentService;
#Autowired
public StudentController(StudentService studentService) {
this.studentService = studentService;
}
#GetMapping
public List<Student> getStudents(){
return studentService.getStudents();
}
#PostMapping
public void registerNewStudent(#RequestBody Student student){
studentService.addNewStudent(student);
}
}
My StudentService class is set up like this:
#Service
public class StudentService {
private final StudentRepository studentRepository;
#Autowired
public StudentService(StudentRepository studentRepository) {
this.studentRepository = studentRepository;
}
public List<Student> getStudents() {
return studentRepository.findAll();
}
public void addNewStudent(Student student) {
Optional<Student> studentOptional = studentRepository.findStudentByEmail(student.getEmail());
if (studentOptional.isPresent()){
throw new IllegalStateException(("email taken"));
}
studentRepository.save(student);
}
}
My StudentConfig class is set up like this:
#Configuration
public class StudentConfig {
#Bean
CommandLineRunner commandLineRunner(StudentRepository repository){
return args -> {
Student david = new Student(
"David",
"drob#apple.com",
LocalDate.of(1985, NOVEMBER, 3)
);
Student alex = new Student(
"Alex",
"alex#apple.com",
LocalDate.of(1995, JANUARY, 1)
);
repository.saveAll(List.of(david, alex));
};
}
}
Then StudentRepository:
#Repository
public interface StudentRepository extends JpaRepository<Student, Long> {
#Query("SELECT s FROM Student s WHERE s.email = ?1")
Optional<Student> findStudentByEmail(String email);
}
Last, the Student class:
#Entity
#Table
public class Student {
#Id
#SequenceGenerator(
name = "student_sequence",
sequenceName = "student_sequence",
allocationSize = 1
)
#GeneratedValue(
strategy = GenerationType.SEQUENCE,
generator = "student_sequence"
)
private Long id;
private String name;
private String email;
private LocalDate dob;
#Transient
private Integer age;
public Student(Long id, String name, String email, LocalDate dob) {
this.id = id;
this.name = name;
this.email = email;
this.dob = dob;
}
public Student(String name, String email, LocalDate dob) {
this.name = name;
this.email = email;
this.dob = dob;
}
public Student() {
}
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getEmail() {
return email;
}
public void setEmail(String email) {
this.email = email;
}
public LocalDate getDob() {
return dob;
}
public void setDob(LocalDate dob) {
this.dob = dob;
}
public Integer getAge() {
return Period.between(this.dob, LocalDate.now()).getYears();
}
public void setAge(Integer age) {
this.age = age;
}
#Override
public String toString() {
return "Student{" +
"id=" + id +
", name='" + name + '\'' +
", email='" + email + '\'' +
", dob=" + dob +
", age=" + age +
'}';
}
}
I set up the project using SpringInitializr
I needed to turn off nginx server that was still on from another project with:
sudo nginx -s stop
Thank you #PeterMmm and #Daniyar Myrzakanov for pointing this out
I have two tables CustomerDetails and Product, I want to fetch customerid from customer table and add it to #joincolumn(order_id) column in same CustomerDetails table.
CustomerDetails.java
#Entity
#Table(name = "CustomerDetails")
public class CustomerDetails {
#Id
#GeneratedValue
#Column(name="CUSTOMER_ID")
private Long custid;
#Column(name="CUSTOMER_NAME")
private String customerName;
#Column(name="EMAIL")
private String email;
#Column(name="ADDRESS")
private String address;
#Column(name="PHONENO")
private String phoneno;
public CustomerDetails() {
}
#Override
public String toString() {
return "CustomerDetails [custid=" + custid + ", customername=" + customerName + ", email=" + email
+ ", address=" + address + ", phoneno=" + phoneno + "]";
}
public CustomerDetails(String customername, String email, String address, String phoneno) {
super();
this.customerName = customername;
this.email = email;
this.address = address;
this.phoneno = phoneno;
}
public Long getCustid() {
return custid;
}
public void setCustid(Long custid) {
this.custid = custid;
}
public String getName() {
return customerName;
}
public void setName(String name) {
this.customerName = name;
}
public String getEmail() {
return email;
}
public void setEmail(String email) {
this.email = email;
}
public String getPassword() {
return address;
}
public void setPassword(String password) {
this.address = password;
}
public String getPhoneno() {
return phoneno;
}
public void setPhoneno(String phoneno) {
this.phoneno = phoneno;
}
}
Product.java
#Entity
#Table(name="Product")
public class Product {
#Id
#GeneratedValue
#Column(name="PRODUCT_ID")
private Long productId;
#Column(name="PRODUCT_NAME")
private String productName;
#Column(name="PRODUCT_BRAND")
private String productBrand;
#Column(name="PRODUCT_PRICE")
private double productPrice;
#OneToOne
private CustomerDetails cd;
public Product(Long productId, String productName, String productBrand, double productPrice, CustomerDetails cd) {
super();
this.productId = productId;
this.productName = productName;
this.productBrand = productBrand;
this.productPrice = productPrice;
this.cd = cd;
}
public Product(String productName, String productType, double productPrice) {
super();
this.productName = productName;
this.productBrand = productType;
this.productPrice = productPrice;
}
public Long getProductId() {
return productId;
}
public void setProductId(Long productId) {
this.productId = productId;
}
public String getProductName() {
return productName;
}
public void setProductName(String productName) {
this.productName = productName;
}
public String getProductBrand() {
return productBrand;
}
public void setProductBrand(String productType) {
this.productBrand = productType;
}
public double getProductPrice() {
return productPrice;
}
public void setProductPrice(double productPrice) {
this.productPrice = productPrice;
}
public CustomerDetails getCd() {
return cd;
}
public void setCd(CustomerDetails cd) {
this.cd = cd;
}
public Product() {
//super();
}
#Override
public String toString() {
return "Product [productId=" + productId + ", productName=" + productName + ", productType=" + productBrand
+ ", productPrice=" + productPrice + "]";
}
}
CustomerDetails repository
#Repository
public interface CdRepo extends JpaRepository<CustomerDetails, Long>
{
}
Product repository
#Repository
public interface ProductRepo extends JpaRepository<Product, Long>
{
}
CustomerService
#Service
#Transactional
public class CustomerService {
private final CdRepo cdRepo;
#Autowired
public CustomerService(CdRepo cdRepo) {
this.cdRepo = cdRepo;
}
public void saveCustomer(CustomerDetails cd)
{
cdRepo.save(cd);
}
}
controller
#RequestMapping(value = {"/addCustomerDetails"}, method = RequestMethod.POST)
public ModelAndView addCustomerDetails(CustomerDetails cd)
{
customerService.saveCustomer(cd);
System.out.println(cd.getCustid());
ModelAndView model = new ModelAndView();
model.setViewName("homepage");
return model;
}
In controller using getCustid() I'm getting current customer's id now I want to insert that id into #joinColumn(order_id)
If I've understood correctly, you want to assign a product to a user (e.g Customer).
For a #OneToOne relation you don't need private CustomerDetails cd; in product class. Although I don't know why are you implementing such thing in that way at all!
Generally If you want to assign two things together, let's say you want to assign a product to a user so that the product would be for that user, you should find the product obj from repository or any where (both product and user must have an id generated by db) and then assign it to user.product.
product service
#Service
public class ProductService {
#Autowired
private ProductRepo productRepository;
public Optional<Product> findProductById(Long id) {
return this.productRepository.findByProductId(id);
}
}
customer service
#Service
#Transactional
public class CustomerService {
private final CdRepo cdRepo;
#Autowired
public CustomerService(CdRepo cdRepo) {
this.cdRepo = cdRepo;
}
public CustomerDetails saveCustomer(CustomerDetails cd, Long productId) {
CustomerDetails dbCustomer = customerService.saveCustomer(cd);
// I'm getting the id from path variable, change it if you have other logics
Optional<Product> dbProduct = this.productService.findProductById(productId);
// I don't know how you handle run time errors so I can't write it, don't
// forget to check the dbProduct in case it didn't exist :)
// In case you did not created getters and setters in CustomerDetails,
// use dbCustomer.product = dbProduct.get();
dbCustomer.setProduct(dbProduct.get());
// update our customer using JPA, after customer update JPA handles everything
return this.cdRepo.save(dbCustomer);
}
}
controller
#RequestMapping(value = {"/addCustomerDetails/{productId}"}, method = RequestMethod.POST)
public ModelAndView addCustomerDetails(CustomerDetails cd, #PathVariable Long productId )
{
CustomerDetails dbCustomer = this.customerService.saveCustomer(cd, productId);
// Use the dbCustomer in your logic ...
ModelAndView model = new ModelAndView();
model.setViewName("homepage");
return model;
}
Write getters and setters in each entity or use Lombok annotation #Data.
Usually when I want to deploy an ecommerce with user and product. I use user, cart, product model.
The problem with the code above is that if you assign that product to a user, it's ONLY for that user. if other users want the same product you have to create all of those products for them. Solution to that would be using product as a series or a model.
Imagine a situation that you want to create a website to sell coffee packages. you only have two type of package to sell. you can create an entity like product for those packages with a quantity for each. Then create a #OneToMany relationship in your cart model to products. It will create a join table to store any possible product id there with cart id. After that, create a #ManyToOne relationship in your cart entity and #OneToMany in your user entity. Now each cart is only for a specific user and not the products.
Some Friendly Advice:
Don't populate your controller with logic. Let service layer handle it.
For each entity, create a package with the name of the entity instead and create 3 classes; the entity itself, response POJO and request POJO.
Use getters and setters for your entites. You can use lombok for that matter. It will handle the situation by generating them.
Use convertor components to create and convert requested entity to the entity itself and also convert entity to response entity.
Avoid interacting with Data base as much as you can. hold the object in a variable like dbCustomer for doing operations.
I am trying to create an application to save data into the Oracle database using CrudRepository. Here is my repositiry:
public interface CustomerRepository extends CrudRepository<Customer, Long> {
List<Customer> findByEmail(String email);
List<Customer> findByDate(Date date);
// custom query example and return a stream
#Query("select c from Customer c where c.email = :email")
Stream<Customer> findByEmailReturnStream(#Param("email") String email);
}
My application.property looks like:
spring.datasource.url=jdbc:oracle:thin:#vgdevst-scan.hhs.local:1521/EONDEV.hhslocal
spring.datasource.username=EON_USER
spring.datasource.password=EON_USERD
spring.datasource.driver-class-oracle.jdbc.driver.OracleDriver
While my customer entity class is :
#Entity
public class Customer {
//http://www.oracle.com/technetwork/middleware/ias/id-generation-083058.html
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "CUST_SEQ")
#SequenceGenerator(sequenceName = "customer_seq", initialValue = 1, allocationSize = 1, name = "CUST_SEQ")
Long id;
String name;
String email;
//#Temporal(TemporalType.DATE)
#Column(name = "CREATED_DATE")
Date date;
public Customer(String name, String email, Date date) {
this.name = name;
this.email = email;
this.date = date;
}
public Customer(Long id, String name, String email, Date date) {
super();
this.id = id;
this.name = name;
this.email = email;
this.date = date;
}
public Customer() {
}
#Override
public String toString() {
return "Customer{" +
"id=" + id +
", name='" + name + '\'' +
", email='" + email + '\'' +
", date=" + date +
'}';
}
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getEmail() {
return email;
}
public void setEmail(String email) {
this.email = email;
}
public Date getDate() {
return date;
}
public void setDate(Date date) {
this.date = date;
}
I am trying to save a new cutomer to database using:
#SpringBootApplication
public class Application implements CommandLineRunner {
private static final SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
#Autowired
DataSource dataSource;
#Autowired
CustomerRepository customerRepository;
public static void main(String[] args) throws Exception {
SpringApplication.run(Application.class, args);
}
#Transactional(readOnly = true)
#Override
public void run(String... args) throws Exception {
System.out.println("DATASOURCE = " + dataSource);
customerRepository.save(new Customer(new Long(4),"Amit","a.r#state.ma.us",new Date()));
System.out.println("\n1.findAll()...");
for (Customer customer : customerRepository.findAll()) {
System.out.println(customer);
}
}
}
I do not see the new customer added either in sops or in database. What am i missing here?
Your problem seems to be that you are executing the save statement in a readOnly transaction. The solution could be as simple as removing that property.
Reading the readOnly flag documentation, it states that:
A boolean flag that can be set to true if the transaction is effectively read-only, allowing for corresponding optimizations at runtime.
Use only #Transactional:
#Transactional
#Override
public void run(String... args) throws Exception {
// the rest of your code ...
}
The code was working just fine.
Its just that in my application code, i had changed the application.property file as per my old code and instead of "spring.datasource.url" i had put "appname.datasource.url", which is why code never interacted with DB.
I am simply trying to create a Spring boot Hibernate CRUD REST API through this code:
EmployeController.java
#RestController
#RequestMapping("/api")
public class EmployeController {
#Autowired
private EmployeService employeService;
#GetMapping("/employe")
public List<Employe> get(){
return employeService.get();
}
}
Employe.java
#Entity
#Table(name="employe")
public class Employe {
#Id
#GeneratedValue(strategy=GenerationType.IDENTITY)
#Column
private int id;
#Column
private String name;
#Column
private String gender;
#Column
private String department;
#Column
private Date dob;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getGender() {
return gender;
}
public void setGender(String gender) {
this.gender = gender;
}
public String getDepartment() {
return department;
}
public void setDepartment(String department) {
this.department = department;
}
public Date getDob() {
return dob;
}
public void setDob(Date dob) {
this.dob = dob;
}
#Override
public String toString() {
return "Employe [id=" + id + ", name=" + name + ", gender=" + gender + ", department=" + department + ", dob="
+ dob + "]";
}
}
EmployeService.java
public interface EmployeService {
List<Employe> get();
Employe get(int id);
void save(Employe employe);
void delete(int id);
}
EmployeServiceImplement.java
#Service
public class EmployeServiceImplement implements EmployeService {
#Autowired
private EmployeDAO employeDAO;
#Transactional
#Override
public List<Employe> get() {
return employeDAO.get();
}
}
EmployeDAO.java
public interface EmployeDAO {
List<Employe> get();
Employe get(int id);
void save(Employe employe);
void delete(int id);
}
EmployeDAOImplement.java
#Repository
public class EmployeDAOImplement implements EmployeDAO {
#Autowired
private EntityManager entityManager;
#Override
public List<Employe> get() {
Session currentSession = entityManager.unwrap(Session.class);
Query<Employe> query = currentSession.createQuery("from Employe", Employe.class);
List<Employe>list = query.getResultList();
return list;
}
}
I have write all the configuration related to MySQl database into the application.properties and when i run this project as Spring Boot App and go to the Postman and tried like this
and i a unable to understan why it always throws 404 error every time , can anyone tell me what i am missing in this code.
Try with this GET request, it may help you:
http://localhost:8080/api
I checked your code.
where is #RestController for your Controller file and where is #RequestMapping For your method in Controller class?
maybe you should write something like this according to your need.
tell me if you need more help.
#RestController
#RequestMapping("/api")
public class EmployeController {
#RequestMapping(value = "/employ")
public void employ() {
}
}
Instead of this -
#Override
public List get()
Use this -
#RequestMapping(value = "/Employe", method = RequestMethod.GET)
public List get()