I have an auto-configured AWS, Spring Boot application, and I'm trying to setup an endpoint that will simply download a particular file from a given bucket in Amazon S3. I uploaded a JPEG file into the bucket from my computer using the AWS console - now I'm trying to download that file using my Spring Boot API.
I'm getting the following error: com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied;
I have created a user and a group (user is in the group) on AWS console; the user/group has full access permissions on S3 as well as administrator access. I downloaded the access-key/secret-key pair and, for testing purposes, literally pasted the keys into my application.properties file as shown below (keys are not shown here, obviously :) ).
I'm confused as to why I'm still getting access denied. I've been searching and working on this for a while; I can't seem to find a solution to this issue that is specific to Spring Boot. Any help would be greatly appreciated.
application.properties:
cloud.aws.credentials.accessKey=myaccesskey
cloud.aws.credentials.secretKey=mysecretkey
cloud.aws.credentials.instanceProfile=false
cloud.aws.stack.auto=false
cloud.aws.region.auto=true
cloud.aws.region.static=myregion
SimpleResourceLoadingBean.java:
#RestController
public class SimpleResourceLoadingBean {
private static Logger log = LoggerFactory.getLogger(HealthMonitorApplication.class);
#Autowired
private ResourceLoader resourceLoader;
#RequestMapping("/getresource")
public String resourceLoadingMethod() throws IOException {
log.info("IN RESOURCE LOADER");
Resource resource = this.resourceLoader.getResource("s3://s3.amazonaws.com/mybucket/myfile.ext");
InputStream inputStream = resource.getInputStream();
return inputStream.toString();
}
}
pom.xml (Just the dependencies that are relevant to the question)
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-aws</artifactId>
<version>1.1.0.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-aws-autoconfigure</artifactId>
<version>1.1.0.RELEASE</version>
</dependency>
Figured out the solution. Besides the application.properties configuration, I had to create a configuration class that would give me access to an AmazonS3Client object when provided the appropriate credentials. I followed this example on GitHub:
https://github.com/brant-hwang/spring-cloud-aws-example/blob/master/src/main/java/com/axisj/spring/cloud/aws/AWSConfiguration.java
AWSConfiguration.java:
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.regions.Region;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3Client;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
#Configuration
public class AWSConfiguration {
#Value("${cloud.aws.credentials.accessKey}")
private String accessKey;
#Value("${cloud.aws.credentials.secretKey}")
private String secretKey;
#Value("${cloud.aws.region}")
private String region;
#Bean
public BasicAWSCredentials basicAWSCredentials() {
return new BasicAWSCredentials(accessKey, secretKey);
}
#Bean
public AmazonS3Client amazonS3Client(AWSCredentials awsCredentials) {
AmazonS3Client amazonS3Client = new AmazonS3Client(awsCredentials);
amazonS3Client.setRegion(Region.getRegion(Regions.fromName(region)));
return amazonS3Client;
}
}
Once this is configured, you can create AmazonS3Client objects (autowired) in your other classes, and use the client to make requests to your S3 cloud. The example uses a wrapper class as a service in order to ease the implementation of additional controller classes.
S3Wrapper.java:
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.model.*;
import org.apache.commons.io.IOUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Service;
import org.springframework.util.StringUtils;
import org.springframework.web.multipart.MultipartFile;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.net.URLEncoder;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
#Service
public class S3Wrapper {
#Autowired
private AmazonS3Client amazonS3Client;
#Value("${cloud.aws.s3.bucket}")
private String bucket;
private PutObjectResult upload(String filePath, String uploadKey) throws FileNotFoundException {
return upload(new FileInputStream(filePath), uploadKey);
}
private PutObjectResult upload(InputStream inputStream, String uploadKey) {
PutObjectRequest putObjectRequest = new PutObjectRequest(bucket, uploadKey, inputStream, new ObjectMetadata());
putObjectRequest.setCannedAcl(CannedAccessControlList.PublicRead);
PutObjectResult putObjectResult = amazonS3Client.putObject(putObjectRequest);
IOUtils.closeQuietly(inputStream);
return putObjectResult;
}
public List<PutObjectResult> upload(MultipartFile[] multipartFiles) {
List<PutObjectResult> putObjectResults = new ArrayList<>();
Arrays.stream(multipartFiles)
.filter(multipartFile -> !StringUtils.isEmpty(multipartFile.getOriginalFilename()))
.forEach(multipartFile -> {
try {
putObjectResults.add(upload(multipartFile.getInputStream(), multipartFile.getOriginalFilename()));
} catch (IOException e) {
e.printStackTrace();
}
});
return putObjectResults;
}
public ResponseEntity<byte[]> download(String key) throws IOException {
GetObjectRequest getObjectRequest = new GetObjectRequest(bucket, key);
S3Object s3Object = amazonS3Client.getObject(getObjectRequest);
S3ObjectInputStream objectInputStream = s3Object.getObjectContent();
byte[] bytes = IOUtils.toByteArray(objectInputStream);
String fileName = URLEncoder.encode(key, "UTF-8").replaceAll("\\+", "%20");
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.setContentType(MediaType.APPLICATION_OCTET_STREAM);
httpHeaders.setContentLength(bytes.length);
httpHeaders.setContentDispositionFormData("attachment", fileName);
return new ResponseEntity<>(bytes, httpHeaders, HttpStatus.OK);
}
public List<S3ObjectSummary> list() {
ObjectListing objectListing = amazonS3Client.listObjects(new ListObjectsRequest().withBucketName(bucket));
List<S3ObjectSummary> s3ObjectSummaries = objectListing.getObjectSummaries();
return s3ObjectSummaries;
}
}
Note: The following dependency will need to be added to pom.xml in order to use the Apache Commons IO library.
pom.xml:
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-io</artifactId>
<version>1.3.2</version>
</dependency>
The accepted answer is using a deprecated APIs. Here's an updated revision.
First, update your maven dependencies:
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.11.274</version>
</dependency>
AWSConfiguration.java
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
#Configuration
public class AWSConfiguration {
#Value("${cloud.aws.credentials.accessKey}")
private String accessKey;
#Value("${cloud.aws.credentials.secretKey}")
private String secretKey;
#Value("${cloud.aws.region}")
private String region;
#Bean
public BasicAWSCredentials basicAWSCredentials() {
return new BasicAWSCredentials(accessKey, secretKey);
}
#Bean
public AmazonS3 amazonS3Client(AWSCredentials awsCredentials) {
AmazonS3ClientBuilder builder = AmazonS3ClientBuilder.standard();
builder.withCredentials(new AWSStaticCredentialsProvider(awsCredentials));
builder.setRegion(region);
AmazonS3 amazonS3 = builder.build();
return amazonS3;
}
}
S3Service.java
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.model.*;
import org.apache.commons.io.IOUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Service;
import org.springframework.util.StringUtils;
import org.springframework.web.multipart.MultipartFile;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.net.URLEncoder;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
#Service
public class S3Service {
#Autowired
private AmazonS3 amazonS3;
#Value("${cloud.aws.s3.bucket}")
private String bucket;
private PutObjectResult upload(String filePath, String uploadKey) throws FileNotFoundException {
return upload(new FileInputStream(filePath), uploadKey);
}
private PutObjectResult upload(InputStream inputStream, String uploadKey) {
PutObjectRequest putObjectRequest = new PutObjectRequest(bucket, uploadKey, inputStream, new ObjectMetadata());
putObjectRequest.setCannedAcl(CannedAccessControlList.PublicRead);
PutObjectResult putObjectResult = amazonS3.putObject(putObjectRequest);
IOUtils.closeQuietly(inputStream);
return putObjectResult;
}
public List<PutObjectResult> upload(MultipartFile[] multipartFiles) {
List<PutObjectResult> putObjectResults = new ArrayList<>();
Arrays.stream(multipartFiles)
.filter(multipartFile -> !StringUtils.isEmpty(multipartFile.getOriginalFilename()))
.forEach(multipartFile -> {
try {
putObjectResults.add(upload(multipartFile.getInputStream(), multipartFile.getOriginalFilename()));
} catch (IOException e) {
e.printStackTrace();
}
});
return putObjectResults;
}
public ResponseEntity<byte[]> download(String key) throws IOException {
GetObjectRequest getObjectRequest = new GetObjectRequest(bucket, key);
S3Object s3Object = amazonS3.getObject(getObjectRequest);
S3ObjectInputStream objectInputStream = s3Object.getObjectContent();
byte[] bytes = IOUtils.toByteArray(objectInputStream);
String fileName = URLEncoder.encode(key, "UTF-8").replaceAll("\\+", "%20");
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.setContentType(MediaType.APPLICATION_OCTET_STREAM);
httpHeaders.setContentLength(bytes.length);
httpHeaders.setContentDispositionFormData("attachment", fileName);
return new ResponseEntity<>(bytes, httpHeaders, HttpStatus.OK);
}
public List<S3ObjectSummary> list() {
ObjectListing objectListing = amazonS3.listObjects(new ListObjectsRequest().withBucketName(bucket));
List<S3ObjectSummary> s3ObjectSummaries = objectListing.getObjectSummaries();
return s3ObjectSummaries;
}
}
Please correct url pattern
Resource resource = this.resourceLoader.getResource("s3://s3.amazonaws.com/mybucket/myfile.ext");
to
Resource resource = this.resourceLoader.getResource("s3://mybucket/myfile.ext");
As per documentation the pattern is s3://<bucket>/<object>.Its working in spring boot 2.0.6.RELEASE and spring cloud Finchley.SR2 (verified).
Reference : Spring Cloud AWS - Downloading files
Related
I have a problem with Spring boot and Angular. I make a POST request which works on POSTMAN and locally but which gives me a 403 in production on tomcat with apache as reverse proxy. But it's working when I am with the embedded tomcat.
I have to try everything soon.
All the solutions I've seen say to disable CSFR but I have no authentication to access my webservice and therefore no spring-security dependency.
I tried anyway but the problem is still there. And in some cases it required me to log in which I don't want to do
import ch.megahertz.swissqrbillsgeneratorapi.properties.FileStorageProperties;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.boot.web.servlet.support.SpringBootServletInitializer;
#SpringBootApplication(scanBasePackages = {"ch.megahertz.swissqrbillsgeneratorapi.*"})
#EnableConfigurationProperties({
FileStorageProperties.class
})
public class SwissQrBillsGeneratorApiApplication extends SpringBootServletInitializer {
static Logger logger = LoggerFactory.getLogger(SwissQrBillsGeneratorApiApplication.class);
public static void main(String[] args) {
logger.info("Run application");
SpringApplication.run(SwissQrBillsGeneratorApiApplication.class, args);
}
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
return application.sources(SwissQrBillsGeneratorApiApplication.class);
}
}
import org.springframework.context.annotation.Configuration;
import org.springframework.web.servlet.config.annotation.CorsRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
#Configuration
public class WebMvcConfig implements WebMvcConfigurer {
#Override
public void addCorsMappings(CorsRegistry registry) {
registry.addMapping("/**")
.allowedMethods("*")
.allowedHeaders("*")
.allowedOrigins("*")
.allowCredentials(false)
.maxAge(-1);
}
}
import ch.megahertz.swissqrbillsgeneratorapi.payload.Invoice;
import ch.megahertz.swissqrbillsgeneratorapi.service.CRMService;
import ch.megahertz.swissqrbillsgeneratorapi.service.FileStorageService;
import lombok.extern.slf4j.Slf4j;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.Resource;
import org.springframework.core.io.UrlResource;
import org.springframework.http.HttpHeaders;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.support.ServletUriComponentsBuilder;
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
//#CrossOrigin(origins = {"https://swissqrbillsgenerator.megahertz.ch/","http://localhost:4200/"})
#Slf4j
#RestController
public class ApiController {
private static final Logger logger = LoggerFactory.getLogger(ApiController.class);
#Autowired
private FileStorageService fileStorageService;
#Autowired
private CRMService crmService;
#GetMapping
public String generateQRBills() {
log.info("Enter in GeT API");
return "Get ok";
}
#PostMapping("/generate")
public ResponseEntity<Resource> uploadFile(#RequestParam("file") MultipartFile file) throws IOException {
System.out.println("Enter in generate API");
logger.info("Enter in generate API");
log.info("Enter in generate POST API");
String fileName = fileStorageService.storeFile(file);
String fileDownloadUri = ServletUriComponentsBuilder.fromCurrentContextPath()
.path("/downloadFile/")
.path(fileName)
.toUriString();
Invoice facture = crmService.getFactureInfo(fileName);
File fileWithQR = fileStorageService.addQrToFile(fileName, facture);
Resource resource = new UrlResource(fileWithQR.toURI());
return ResponseEntity.ok()
.header(HttpHeaders.CONTENT_TYPE,Files.probeContentType(resource.getFile().toPath()))
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + fileName+ "\"")
.body(resource);
}
}
Do you have any idea?
If Postman is returning the query, then the problem is probably in the Angular front end. I believe I was getting a 403 error when I tried to send a String from my backend, it had to be wrapped in an object and unwrapped by Angular to be a string in Angular.
I have to write a spring boot method where i have to download the file which is being returned from another microservice.
Can anyone please tell me how can i do it.
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.InvocationTargetException;
import java.nio.file.StandardCopyOption;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import org.apache.commons.io.FilenameUtils;
import org.apache.commons.io.IOUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
import okhttp3.MediaType;
import okhttp3.MultipartBody;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
public File processFile(MultipartFile uploadedFile) throws IllegalStateException, IOException {
if (!uploadedFile.isEmpty()) {
byte[] bytes = uploadedFile.getBytes();
String fileName = uploadedFile.getOriginalFilename();
File convertedFile = new File(uploadedFile.getOriginalFilename());
uploadedFile.transferTo(convertedFile);
OkHttpClient.Builder builder = new OkHttpClient.Builder();
OkHttpClient client = builder.readTimeout(120, TimeUnit.SECONDS).writeTimeout(120, TimeUnit.SECONDS)
.connectTimeout(120, TimeUnit.SECONDS).build();
MediaType mediaType = MediaType
.parse("multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW");
RequestBody requestBody = new MultipartBody.Builder().setType(MultipartBody.FORM)
.addFormDataPart("file", fileName, RequestBody.create(mediaType, bytes)).build();
Request request = new Request.Builder().url("").post(requestBody)
.addHeader("content-type", "multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW")
.addHeader("authorization", "Basic cGFkc2VsZWN0OjQwOWExZmMzZmExNTYwZjljZTYyOTQxZTU2ZDgyOGI2")
.addHeader("cache-control", "no-cache").build();
Response response = null;
try {
response = client.newCall(request).execute();
} catch (Exception e) {
throw new exception(HttpStatus.INTERNAL_SERVER_ERROR,
resource.getString("application.controller.parser.exception"), e);
}
if (!response.isSuccessful() || null == response) {
throw new exception(HttpStatus.INTERNAL_SERVER_ERROR,
resource.getString("application.controller.parser.exception"));
} else {
}
}
return null;
}
Can anyone please tell me what i need to do in the else part to convert the response i am getting as a file and safe that as a file in my java code.
Change your method return type from "File" to "byte[]" and this will work.
else {
byte[] responseFile = (File)response.body().bytes();
return responseFile;
}
use the byte[] from body() to construct an inputstream. use this inputstream to write into a file. response header would let you know the the data type and define the file type accordingly
I am trying to post an xml request to a third party server and expecting an xml response back.
I am here by attaching the source code for configuring Cnfiguration,gateways & channels, Customized Rest Template class and Test Client to execute. We have various message converters for XML Posting but despite that it is always saying no suitable HTTPMessage Converters
/**** Gateway & Channel Configuration class****/
#Configuration
#MessagingGateway(name = "EntryGateway", defaultRequestChannel = "EntryChannel", defaultRequestTimeout = "2000", defaultReplyChannel = "ExitChannel", defaultReplyTimeout = "2000")
public interface Gateway {
#Gateway(requestChannel = "EntryChannel", requestTimeout = Constants.REQUEST_TIMEOUT_IN_MILLISECONDS, replyChannel = "ExitChannel", replyTimeout = Constants.RESPONSE_TIMEOUT_IN_MILLISECONDS)
ReqResMessage sendRequest(ReqResMessage request);
}
import java.util.ArrayList;
import java.util.List;
import org.aopalliance.aop.Advice;
import org.slf4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Import;
import org.springframework.context.annotation.PropertySource;
import org.springframework.core.env.Environment;
import org.springframework.http.HttpMethod;
import org.springframework.http.MediaType;
import org.springframework.http.converter.FormHttpMessageConverter;
import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.http.converter.StringHttpMessageConverter;
import org.springframework.http.converter.xml.MarshallingHttpMessageConverter;
import org.springframework.integration.annotation.IntegrationComponentScan;
import org.springframework.integration.annotation.ServiceActivator;
import org.springframework.integration.channel.DirectChannel;
import org.springframework.integration.config.EnableIntegration;
import org.springframework.integration.http.outbound.HttpRequestExecutingMessageHandler;
import org.springframework.messaging.MessageChannel;
import org.springframework.messaging.MessageHandler;
import org.springframework.messaging.converter.MessageConverter;
import org.springframework.oxm.Marshaller;
import org.springframework.oxm.Unmarshaller;
import org.springframework.oxm.jaxb.Jaxb2Marshaller;
#Configuration
#ComponentScan("package_name")
#IntegrationComponentScan("package_name")
#EnableIntegration
#Import(value = ExternalSystemsConfiguration.class)
#PropertySource(value = "classpath:properties", ignoreResourceNotFound = true)
public class Configuration {
private static final Logger logger = LogbackConfig.getApplicationLogger(POPConfiguration.class);
#Autowired
Environment env;
#Autowired
ExternalSystemsConfiguration externalSystemsConfiguration;
/**
* Entry channel to the messaging system through the gateway.
*
* #return MessageChannel
*/
#Bean
public MessageChannel EntryChannel() {
return new DirectChannel();
}
#Bean
public MessageChannel RequestChannel() {
return new DirectChannel();
}
#Bean
public MessageChannel ResponseChannel() {
return new DirectChannel();
}
#Bean
public MessageChannel ExitChannel() {
return new DirectChannel();
}
#Bean
#ServiceActivator(inputChannel = "RequestChannel")
public MessageHandler cmmHttpGateway() throws Exception {
logger.debug("Entered Configuration httpGateway() ");
List<Advice> retryAdvices = new ArrayList<>();
retryAdvices.add(externalSystemsConfiguration.requestHandlerRetryAdvice());
HttpRequestExecutingMessageHandler handler = new HttpRequestExecutingMessageHandler(env.getProperty("url")
);
List<HttpMessageConverter<?>> converters = new ArrayList<>();
converters.add(marshallingMessageConverter());
handler.setMessageConverters(converters);
handler.setOutputChannel(popResponseChannel());
handler.setRequiresReply(true);
//handler.setExtractPayload(true);
// handler.
handler.setHttpMethod(HttpMethod.POST);
logger.debug("Exited Configuration httpGateway() ");
return handler;
}
#Bean
public MarshallingHttpMessageConverter marshallingMessageConverter() {
return new MarshallingHttpMessageConverter(
jaxb2Marshaller(),
jaxb2Marshaller()
);
}
#Bean
public Jaxb2Marshaller jaxb2Marshaller() {
System.out.println("jaxb2Marshaller");
Jaxb2Marshaller marshaller = new Jaxb2Marshaller();
marshaller.setClassesToBeBound(new Class[]{
ReqResMessage.class,
ReqResBody.class,
ReqResHeader.class,
Request.class,
Response.class
});
return marshaller;
}
}
/***** Rest Template Comfiguration ******/
import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.InetSocketAddress;
import java.net.Proxy;
import java.security.cert.X509Certificate;
import java.util.ArrayList;
import java.util.Base64;
import java.util.List;
import java.util.Map;
import java.util.Queue;
import javax.net.ssl.SSLException;
import javax.net.ssl.SSLSession;
import javax.net.ssl.SSLSocket;
import org.apache.http.Header;
import org.apache.http.HttpHeaders;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.auth.AuthOption;
import org.apache.http.auth.AuthScheme;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.MalformedChallengeException;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.AuthCache;
import org.apache.http.client.AuthenticationStrategy;
import org.apache.http.client.HttpClient;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpHead;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.params.HttpClientParams;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.conn.params.ConnRoutePNames;
import org.apache.http.conn.ssl.X509HostnameVerifier;
import org.apache.http.impl.auth.BasicScheme;
import org.apache.http.impl.client.BasicAuthCache;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.impl.client.DefaultRedirectStrategy;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.client.LaxRedirectStrategy;
import org.apache.http.impl.client.ProxyAuthenticationStrategy;
import org.apache.http.message.BasicHeader;
import org.apache.http.protocol.HttpContext;
import org.slf4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.PropertySource;
import org.springframework.core.env.Environment;
import org.springframework.http.MediaType;
import org.springframework.http.client.ClientHttpRequestFactory;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.http.client.SimpleClientHttpRequestFactory;
import org.springframework.http.converter.FormHttpMessageConverter;
import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.http.converter.StringHttpMessageConverter;
import org.springframework.http.converter.xml.Jaxb2RootElementHttpMessageConverter;
import org.springframework.http.converter.xml.MarshallingHttpMessageConverter;
import org.springframework.integration.handler.advice.RequestHandlerRetryAdvice;
import org.springframework.oxm.Marshaller;
import org.springframework.oxm.Unmarshaller;
import org.springframework.oxm.jaxb.Jaxb2Marshaller;
import org.springframework.retry.backoff.FixedBackOffPolicy;
import org.springframework.retry.policy.SimpleRetryPolicy;
import org.springframework.retry.support.RetryTemplate;
import org.springframework.web.client.RestTemplate;
#Configuration
#PropertySource(value ="classpath:integration.properties", ignoreResourceNotFound =true)
public class ExternalSystemsConfiguration {
#Autowired
Environment env;
private static final Logger logger = LogbackConfig.getApplicationLogger(ExternalSystemsConfiguration.class);
#Bean
public RequestHandlerRetryAdvice requestHandlerRetryAdvice() {
logger.debug("Entered RetryConfiguration requestHandlerRetryAdvice()");
RequestHandlerRetryAdvice retryAdvice = new RequestHandlerRetryAdvice();
RetryTemplate retryTemplate = new RetryTemplate();
FixedBackOffPolicy fixedBackOffPolicy = new FixedBackOffPolicy();
SimpleRetryPolicy simpleRetryPolicy = new SimpleRetryPolicy();
simpleRetryPolicy.setMaxAttempts(Integer.parseInt(env.getProperty("retryAttempts")));
fixedBackOffPolicy.setBackOffPeriod(Integer.parseInt(env.getProperty("backOffPolicy")));
retryTemplate.setBackOffPolicy(fixedBackOffPolicy);
retryTemplate.setRetryPolicy(simpleRetryPolicy);
retryAdvice.setRetryTemplate(retryTemplate);
return retryAdvice;
}
/**
* This method is used to create rest template with configurable read and connection timeouts.It is used by all http gateways.
* #return RestTemplate
*/
#SuppressWarnings("deprecation")
#Bean
public RestTemplate getRestTemplate(){
RestTemplate restTemplate = new RestTemplate();
BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(new AuthScope("test.com", 443), new UsernamePasswordCredentials("testuser", "testpassword"));
HttpHost target = new HttpHost("test", 443, "https");
System.out.println("BASE64:"+credentialsProvider.toString());
// Create AuthCache instance
AuthCache authCache = new BasicAuthCache();
// Generate BASIC scheme object and add it to the local auth cache
BasicScheme basicAuth = new BasicScheme();
authCache.put(target, basicAuth);
// Add AuthCache to the execution context
HttpClientContext context = HttpClientContext.create();
context.setCredentialsProvider(credentialsProvider);
context.setAuthCache(authCache);
Header header = new BasicHeader(HttpHeaders.CONTENT_TYPE, "application/xml");
List<Header> headers = new ArrayList<>();
headers.add(header);
HttpHost proxy = new HttpHost("", "", "http");
RequestConfig config = RequestConfig.custom()
.setProxy(proxy)
.build();
CloseableHttpClient httpClient = HttpClientBuilder.create().setProxy(proxy).setDefaultHeaders(headers).setProxyAuthenticationStrategy(new ProxyAuthenticationStrategy())
.setDefaultCredentialsProvider(credentialsProvider).setRedirectStrategy(new DefaultRedirectStrategy() {
private String[] REDIRECT_METHODS = new String[] {
HttpGet.METHOD_NAME, HttpHead.METHOD_NAME
};
#Override
protected boolean isRedirectable(String method) {
for (String m : REDIRECT_METHODS) {
if (m.equalsIgnoreCase(method)) {
return true;
}
}
return false;
}
})
.build();
restTemplate.setRequestFactory(new Conn());
/* List<HttpMessageConverter<?>> converters = new ArrayList<>();
converters.add(marshallingMessageConverter());
converters.add(new FormHttpMessageConverter());
converters.add(new StringHttpMessageConverter());
restTemplate.setMessageConverters(converters);*/
return restTemplate;
}
public class Conn extends SimpleClientHttpRequestFactory {
public Conn(){
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("", 8080));
this.setProxy(proxy);
}
#Override
protected void prepareConnection(HttpURLConnection connection, String httpMethod) throws IOException {
// TODO Auto-generated method stub
super.prepareConnection(connection, httpMethod);
connection.setFollowRedirects(true);
String userpassword = "testuser" + ":" + "testpassword";
String encodedAuthorization = Base64.getEncoder().encodeToString(userpassword.getBytes());
System.out.println("basic-----" + encodedAuthorization);
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.addRequestProperty("Authorization", "Basic " + encodedAuthorization);
connection.addRequestProperty("content-type", "application/xml");
}
}
}
/***** Test Client to Post XML Request in the form of Java object as well as String *****/
*/
#ContextConfiguration(classes = Configuration.class)
#RunWith(SpringJUnit4ClassRunner.class)
public class Test {
private static Logger logger = LogbackConfig.getApplicationLogger(Test.class);
#Autowired
private Gateway cmmGateway;
#Autowired
RestTemplate template;
#Test
public void testJsonResponse() {
String xml = "<?xml version=\"1.0\"?>"
+"<Message>"
+ "<Header><NCPDPID>3942100</NCPDPID><SentTime>2016-07-14 06:13:00</SentTime>"
+ "<SenderID>01hw320985</SenderID><MaxRowCount>500</MaxRowCount>"
+ "</Header><Body><Request><Distance>100.00</Distance><LastName>ALLEN</LastName>"
+ "<FirstName></FirstName><Gender></Gender><Phone></Phone><City></City>"
+ "<State></State><LicensedState></LicensedState>" + "<DEA></DEA>"
+ "" + "</Request>"
+ "</Body>" + "</Message>";/*
ReqResMessage user=null;
try{
JAXBContext jaxbContext = JAXBContext.newInstance(ReqResMessage.class);
Unmarshaller jaxbUnmarshaller = jaxbContext.createUnmarshaller();
InputStream is = new ByteArrayInputStream(xml.getBytes());
user= (ReqResMessage) jaxbUnmarshaller.unmarshal(is);
// System.out.println("REquest****"+user.getBody().getRequest());
}catch(Exception e){
e.printStackTrace();
}*/
ReqResMessage re = new ReqResMessage();
ReqResHeader header= new ReqResHeader();
header.setPharmacyNCPDPID("3942100");
header.setRowCount("500");
header.setSenderID("01hw320985");
Request request = new Request();
request.setDistance("100.00");
request.setLastName("ALLEN");
ReqResBody body = new ReqResBody();
body.setRequest(request);
re.setBody(body);
re.setHeader(header);
//System.out.println("Before:"+System.currentTimeMillis());
ReqResMessage response = cmmGateway.sendRequest(re);
}
}
Finally i found solution by setting header enricher out put channel to be passed as request channel to httpoutbound gateway. Find below code snippet,there i set content-type to application/xml that resolves problem.
#Bean
#Transformer(inputChannel = "RequestChannel", outputChannel = "enricherOutputChannel")
public HeaderEnricher makeEnricher() {
Map<String, ? extends HeaderValueMessageProcessor<?>> headersToAdd = Collections
.singletonMap(HttpHeaders.CONTENT_TYPE, new StaticHeaderValueMessageProcessor<>("application/xml"));
HeaderEnricher enricher = new HeaderEnricher(headersToAdd);
enricher.setDefaultOverwrite(true);
return enricher;
}
This took me quite a while to work out so I wanted to share it. Most information came from SO and I wanted to consolidate into this one place.
My requirements are to upload files using a RESTFul POST. Due to possibly large files I wanted to stream the files. I obviously want to be able to read the response.
I planned to use Jersey as the REST Server and Spring's RestTemplate as the client (and for testing).
The problem I faced was streaming POSTs and receiving a response. How can I do that? (Rhetorical question - I answer this!)
It's unnecessary to go through all these hoops with a RequestCallback. Simply use a PathResource.
PathResource pathResource = new PathResource(theTestFilePath);
ResponseEntity<String> response = restTemplate.exchange(url, HttpMethod.POST, new HttpEntity<>(pathResource), String.class);
Spring will use a ResourceHttpMessageConverter to serialize the file identified by the given Path to the request body. Internally, the Spring 4.x implementation uses a buffer size of 4096 bytes (which is also what IOUtils#copy(..) uses).
Obviously, you can provide the response type you want. The example above expects the response body as a String. With a ResponseEntity, you can access all the response headers with
HttpHeaders responseHeaders = response.getHeaders();
I am using SpringBoot 1.2.4.RELEASE with Jersey being pulled in by:
compile("org.springframework.boot:spring-boot-starter-jersey")
I created the project with the brilliant Spring Starter Project (Spring Tool Suite > New or you can do through a website I believe and no doubt IntelliJ has this capability also). And chose 'Jersey (JAX-RS)' option. In the gradle build.gradle I also added the dependency:
compile('commons-io:commons-io:2.4')
I wrote this server side code.
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.URI;
import java.net.URISyntaxException;
import javax.ws.rs.Consumes;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RestController;
import org.me.fileStore.service.FileStoreService;
#RestController
#Path("/filestore")
public class FileStoreRestService {
private static Logger logger = LoggerFactory.getLogger(FileStoreRestService.class);
#Autowired
private FileStoreService fileStoreService;
#POST
#Path("upload")
#Consumes(MediaType.APPLICATION_OCTET_STREAM)
#Produces(MediaType.APPLICATION_JSON)
public Response Upload(InputStream stream) throws IOException, URISyntaxException { //
String location = fileStoreService.upload(stream); // relative path
URI loc = new URI(location);
Response response = Response.created(loc).build();
System.out.println("POST - response: " + response + ", :" + response.getHeaders());
return response;
}
Where i had most troubles was in getting a Response with a location.
Firstly I had to handle streaming large files. I followed https://stackoverflow.com/a/15785322/1019307 as you can see in the test below. I was NOT obtaining a Response no matter what I tried with the HttpMessageConverterExtractor as per that post:
final HttpMessageConverterExtractor<String> responseExtractor =
new HttpMessageConverterExtractor<String>(String.class, restTemplate.getMessageConverters());
After finding https://stackoverflow.com/a/6006147/1019307 I wrote:
private static class ResponseFromHeadersExtractor implements ResponseExtractor<ClientHttpResponse> {
#Override
public ClientHttpResponse extractData(ClientHttpResponse response) {
System.out.println("StringFromHeadersExtractor - response headers: " + response.getHeaders());
return response;
}
}
This gave me this test:
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.io.InputStream;
import java.net.URI;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.HashMap;
import java.util.Map;
import org.apache.commons.io.IOUtils;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.hamcrest.MatcherAssert;
import org.hamcrest.Matchers;
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.test.IntegrationTest;
import org.springframework.boot.test.SpringApplicationConfiguration;
import org.springframework.boot.test.TestRestTemplate;
import org.springframework.http.HttpMethod;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.http.client.ClientHttpRequest;
import org.springframework.http.client.ClientHttpResponse;
import org.springframework.http.client.SimpleClientHttpRequestFactory;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import org.springframework.test.context.web.WebAppConfiguration;
import org.springframework.web.client.RequestCallback;
import org.springframework.web.client.ResponseExtractor;
import org.springframework.web.client.RestTemplate;
#RunWith(SpringJUnit4ClassRunner.class)
#SpringApplicationConfiguration(classes = FileStoreApplication.class)
#WebAppConfiguration
#IntegrationTest("server.port:9000")
public class FileStoreRestServiceTest {
private static Logger logger = LoggerFactory.getLogger(FileStoreRestServiceTest.class);
protected final Log logger2 = LogFactory.getLog(getClass());
String base = "http://localhost:9000/filestore";
private RestTemplate restTemplate = new TestRestTemplate();
#Test
public void testMyMethodExecute() throws IOException {
String content = "This is file contents\nWith another line.\n";
Path theTestFilePath = TestingUtils.getTempPath(content);
InputStream inputStream = Files.newInputStream(theTestFilePath);
String url = base + "/upload";
final RequestCallback requestCallback = new RequestCallback() {
#Override
public void doWithRequest(final ClientHttpRequest request) throws IOException {
request.getHeaders().setContentType(MediaType.APPLICATION_OCTET_STREAM);
IOUtils.copy(inputStream, request.getBody());
}
};
final RestTemplate restTemplate = new RestTemplate();
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setBufferRequestBody(false);
restTemplate.setRequestFactory(requestFactory);
ClientHttpResponse response = restTemplate.execute(url, HttpMethod.POST, requestCallback,
new ResponseFromHeadersExtractor());
URI location = response.getHeaders().getLocation();
System.out.println("Location: " + location);
Assert.assertNotNull(location);
Assert.assertNotEquals(0, location.getPath().length());
}
private static class ResponseFromHeadersExtractor implements ResponseExtractor<ClientHttpResponse> {
#Override
public ClientHttpResponse extractData(ClientHttpResponse response) {
System.out.println("StringFromHeadersExtractor - response headers: " + response.getHeaders());
return response;
}
}
I need to refactor much in that test out into some services.
I have certificate files xx.crt and xx.pfx. I also have password for xx.pfx. How do I configure this spring boot embedded tomcat?
Connector connector = new Connector("org.apache.coyote.http11.Http11NioProtocol");
Http11NioProtocol protocol = (Http11NioProtocol) connector.getProtocolHandler();
connector.setScheme("https");
connector.setSecure(true);
protocol.setSSLEnabled(true);
protocol.setKeystoreFile(??);
protocol.setKeyPass(??);
protocol.setTruststoreFile(??);
return connector;
The easiest way to solve this problem is to convert the PFX file to JKS using the keytool (keytool.exe on Windows):
keytool -importkeystore -srckeystore mypfxfile.pfx -srcstoretype pkcs12 -destkeystore newkeystore.jks -deststoretype JKS
and use protocol.setKeystoreFile() and protocol.setKeyPass() to load it.
The easiest way to use .pfx file with spring boot.
Follow the below steps to make it work.
1) Get the pfx file from service provider.
2) Install the certificate.Refer this link for how to install certificate.
application.properties
server.port= 9091
server.ssl.trust-store= classpath:jks/Test_Certificate.pfx
server.ssl.trust-store-password= XXXXXX
server.ssl.enabled= false
server.ssl.trust-store-type= PKCS12
soap.url=https://localost:8080/test/Calculator.wsdl
TestClientConfiguration.java
import java.io.IOException;
import java.security.KeyStore;
import javax.net.ssl.SSLContext;
import org.apache.http.HttpException;
import org.apache.http.HttpRequest;
import org.apache.http.HttpRequestInterceptor;
import org.apache.http.client.HttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.protocol.HTTP;
import org.apache.http.protocol.HttpContext;
import org.apache.http.ssl.SSLContexts;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.Resource;
import org.springframework.oxm.jaxb.Jaxb2Marshaller;
import org.springframework.ws.transport.http.HttpComponentsMessageSender;
#Configuration
public class TestClientConfiguration {
private final String SOAP_URI = "https://localost:8080/test/Calculator.wsdl";
#Value("${server.ssl.trust-store}")
private Resource trustStore;
#Value("${server.ssl.trust-store-password}")
private String trustStorePassword;
#Value("${server.ssl.trust-store-type}")
private String trustStoreType;
#Bean
public TestServiceClient processMessage(Jaxb2Marshaller marshaller) throws Exception {
TestServiceClient client = new TestServiceClient();
client.setDefaultUri(SOAP_URI);
client.setMarshaller(marshaller);
client.setUnmarshaller(marshaller);
client.setMessageSender(httpComponentsMessageSender());
return client;
}
#Bean
Jaxb2Marshaller jaxb2Marshaller() {
Jaxb2Marshaller jaxb2Marshaller = new Jaxb2Marshaller();
jaxb2Marshaller.setContextPath("com.test.soap.types");
return jaxb2Marshaller;
}
#Bean
public HttpComponentsMessageSender httpComponentsMessageSender() throws Exception {
HttpComponentsMessageSender httpComponentsMessageSender = new HttpComponentsMessageSender();
httpComponentsMessageSender.setHttpClient(httpClient());
return httpComponentsMessageSender;
}
public HttpClient httpClient() throws Exception {
KeyStore keyStore = KeyStore.getInstance(trustStoreType);
keyStore.load(trustStore.getInputStream(), trustStorePassword.toCharArray());
SSLContext sslContext = SSLContexts.custom().loadKeyMaterial(keyStore, trustStorePassword.toCharArray())
.build();
HttpClient httpClient = HttpClients.custom().setSSLContext(sslContext)
.addInterceptorFirst(new ContentLengthHeaderRemover()).build();
return httpClient;
}
}
TestServiceClient.java
import java.io.IOException;
import java.io.StringReader;
import java.io.StringWriter;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBElement;
import javax.xml.bind.Marshaller;
import javax.xml.transform.TransformerException;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.ws.WebServiceMessage;
import org.springframework.ws.client.core.WebServiceMessageCallback;
import org.springframework.ws.client.core.support.WebServiceGatewaySupport;
import org.springframework.ws.soap.SoapHeader;
import org.springframework.ws.soap.SoapMessage;
#Qualifier("testSoapServiceClient")
public class TestServiceClient extends WebServiceGatewaySupport {
#Value("${soap.url}")
private String testsoapurl;
public String processMessage(String mgRequest, Message reqHeader) {
StreamSource source = new StreamSource(new StringReader(mgRequest));
StringWriter stringWriter = new StringWriter();
StreamResult result = new StreamResult(stringWriter);
getWebServiceTemplate().sendSourceAndReceiveToResult(testsoapurl, source, new WebServiceMessageCallback() {
#Override
public void doWithMessage(WebServiceMessage message) throws IOException, TransformerException {
try {
SoapMessage soapMessage = (SoapMessage) message;
//Soap Actio
soapMessage.setSoapAction("urn:test/ProcessMessage");
// get the header from the SOAP message
SoapHeader soapHeader = soapMessage.getSoapHeader();
// create the header element
ObjectFactory factory = new ObjectFactory();
JAXBElement<Message> headerMessage = factory.createMessage(reqHeader);
// create a marshaller
JAXBContext context = JAXBContext.newInstance(Message.class);
Marshaller marshaller = context.createMarshaller();
// marshal the headers into the specified result
marshaller.marshal(headerMessage, soapHeader.getResult());
} catch (Exception e) {
/////
}
}
}, result);
return stringWriter.toString();
}
}
If you look at Http11NioProtocol those methods take string parameters... Anyways here is a related question/answer How can I specify my .keystore file with Spring Boot and Tomcat?