I've found excellent example how to read/update existing file in GCP bucket:
#RestController
public class WebController {
#Value("gs://${gcs-resource-test-bucket}/my-file.txt")
private Resource gcsFile;
#RequestMapping(value = "/", method = RequestMethod.GET)
public String readGcsFile() throws IOException {
return StreamUtils.copyToString(
this.gcsFile.getInputStream(),
Charset.defaultCharset()) + "\n";
}
#RequestMapping(value = "/", method = RequestMethod.POST)
String writeGcs(#RequestBody String data) throws IOException {
try (OutputStream os = ((WritableResource) this.gcsFile).getOutputStream()) {
os.write(data.getBytes());
}
return "file was updated\n";
}
}
But I can't find the way how to upload new file into GCP bucket using spring gloud GCP.
How can I achieve it ?
#Autowired
private Storage gcs;
public void upload(String report) {
gcs.create(BlobInfo
.newBuilder("bucket_name", "fileName.json")
.build(),
report.getBytes());
}
Related
I am writing a RestTemplateErrorHandler.java class to handle RestTemplate exception.
Scenarion: I have a jar file in my class path which has the resttemplate exception handling functionality.
Jar name-
aws-common-config.jar
|- RestTemplateConfig.class
|- RestTemplateErrorHandler.class
RestTemplateConfig.class
#EnableConfigurationProperties({ HttpProperties.class, MksProperties.class })
#RequiredArgsConstructor
public class RestTemplateConfig {
private static final Logger LOG = LogManager
.getLogger(RestTemplateConfig.class);
private final HttpProperties httpProperties;
private final MksProperties mksProperties;
private final RestTemplateErrorHandler restTemplateHandler;
#Bean("NoSSLRestTemplate")
#ConditionalOnProperty(name = "aws.common.resttemplate.enabled", havingValue = "true", matchIfMissing = false)
#Primary
public RestTemplate configRestTemplate()
throws CertificateException, IOException, NoSuchAlgorithmException,
KeyStoreException, KeyManagementException {
RestTemplate restTemplate;
restTemplate = new RestTemplate(new OkHttp3ClientHttpRequestFactory(
getRestTemplateFromOkhttp3Template()));
restTemplate.setErrorHandler(restTemplateHandler);
return restTemplate;
}
private OkHttpClient getRestTemplateFromOkhttp3Template() {
//some code
}
#Bean("AWSRestTemplate")
#ConditionalOnProperty(name = "aws.common.resttemplate.awsenabled", havingValue = "true", matchIfMissing = false)
public RestTemplate configureRestTemplateforMTLS() throws Exception {
RestTemplate restTemplate = new RestTemplate();
try {
HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory(
httpClient());
requestFactory.setConnectTimeout(
httpProperties.getConnect().getTimeout());
requestFactory.setReadTimeout(
httpProperties.getConnect().getReadTimeout());
restTemplate.setRequestFactory(requestFactory);
restTemplate.setErrorHandler(restTemplateHandler);
} catch (Exception ex) {
LOG.error("Exception occurred while creating MTLS Rest Template ",
ex);
throw ex;
}
return restTemplate;
}
private CloseableHttpClient httpClient() throws Exception {
//some code
}
}
RestTemplateErrorHandler.java -
#Component
#Slf4j
public class RestTemplateErrorHandler implements ResponseErrorHandler {
private static final int BUFFER_SIZE = 200;
#Override
public boolean hasError(ClientHttpResponse clientHttpResponse)
throws IOException {
return clientHttpResponse.getStatusCode() != HttpStatus.OK;
}
#Override
public void handleError(ClientHttpResponse clientHttpResponse)
throws IOException {
String errMessage = getErrMessage(clientHttpResponse);
HttpStatus status = clientHttpResponse.getStatusCode();
switch (status) {
case BAD_REQUEST:
throw new CustomException(errMessage,
ErrorResponse.INVALID_REQUEST);
case NOT_FOUND:
throw new CustomException(errMessage, ErrorResponse.NOT_FOUND);
case SERVICE_UNAVAILABLE:
throw new CustomException(errMessage, ErrorResponse.TIME_OUT);
case METHOD_NOT_ALLOWED:
case INTERNAL_SERVER_ERROR:
default:
throw new CustomException(errMessage,
ErrorResponse.INTERNAL_SERVER_ERROR);
}
}
private String getErrMessage(ClientHttpResponse clientHttpResponse) {
try {
return StreamUtils.copyToString(clientHttpResponse.getBody(),
StandardCharsets.UTF_8);
} catch (Exception e) {
log.error("[RestTemplate]read body", e);
return "Error reading response body";
}
}
}
application.properties
aws.common:
appId: TTP
resttemplate:
enabled: true
awsenabled: true
As you could see here
#Override
public boolean hasError(ClientHttpResponse clientHttpResponse)
throws IOException {
return clientHttpResponse.getStatusCode() != HttpStatus.OK;
}
HttpStatus 201 Created is not being handled here.
To Handle this I have written both the java classes in my application.
Now I want to exclude the aws-common-config.jar file's RestTemplateConfig.class and want to load the one I have written in my application.
But everytime aws-common-config.jar file's RestTemplateConfig.class file gets loaded during the startup.
here is my configuration file:
#EnableConfigurationProperties({ HttpProperties.class, MksProperties.class })
#Configuration
#RequiredArgsConstructor
#Component
public class RestTemplateConfiguration {
//body code is same as jar file
}
My RestTemplateErrorHandler.class -
#Component
#Slf4j
public class RestTemplateErrorHandler implements ResponseErrorHandler {
//same code as Jar file
//have created my own CustomException.class and ErrorReponse.class and imported them
}
Here I am calling API using rest template:
#Qualifier("AWSRestTemplate")
#Autowired
private RestTemplate restTemplateAWS;
AWSRestTemplate.exchange(url, HttpMethod.POST, entity, Response.class);
Had tried with
#import
and
#ComponentScan(basePackages = {"com.efx.ews.es"}, excludeFilters = {#ComponentScan.Filter(
type = FilterType.ASSIGNABLE_TYPE,
value = {RestTemplateConfig.class}
)
})
but nothing help.
Can Someone please help me here how can I exclude jar file config class and lode my RestTemplateConfiguration.class.
I can't modify the jar file code and can't remove the jar file dependency as some other features are getting used.
I wrote a spring controller with following methods to deal with a callback http request,
#PostMapping ("/test")
public void notifyTranscodeResult(String paramStr){
...
}
#PostMapping ("/test2")
public void notifyTranscodeResult(#RequestBody ParamClass param){
...
}
but I get errors: Resolved exception caused by handler execution: org.springframework.web.HttpMediaTypeNotSupportedException: Content type 'application/octet-stream' not supported
I can't change the callback http request because they are from other third-party services, how can I change my controller to correctly get the request params?
You need to define consumes attribute.
#PostMapping (path = "/test2", consumes = {MediaType.APPLICATION_OCTET_STREAM_VALUE})
Here is the implementation with Unit Test Case.
#PostMapping(value = "/upload",
consumes = MediaType.APPLICATION_OCTET_STREAM_VALUE)
public String demo(HttpServletRequest httpServletRequest) {
ServletInputStream inputStream;
try {
inputStream = httpServletRequest.getInputStream();
} catch (IOException e) {
throw new RuntimeException(e);
}
final List<String> list = new BufferedReader(new InputStreamReader(inputStream))
.lines().toList();
System.out.println(list);
return "Hello World";
}
Test Case
#Autowired
private MockMvc mockMvc;
#Test
public void shouldTestBinaryFileUpload() throws Exception {
mockMvc
.perform(MockMvcRequestBuilders
.post("/api/user/upload")
.content("Hello".getBytes())
.contentType(MediaType.APPLICATION_OCTET_STREAM))
.andExpect(MockMvcResultMatchers
.status()
.isOk())
.andExpect(MockMvcResultMatchers
.content()
.bytes("Hello World".getBytes()));
}
I have created a Spring boot 2 and angular application (Book Store application - the admin can add new books or edit existing books). I want to serve images which are kept in resources/static/images path. When I add a new images through api call, it works perfectly, but when I try to replace the image with different image (when trying to edit the book), the image gets replaced with new image when I check the folder through file-explorer, but when I visit the link http://localhost:8181/images/16.png it shows the old image. In eclipse, if I right click the project and click refresh then http://localhost:8181/images/16.png shows correct image.
For preventing this issue, I have written the below code so it prevents caching of static/images, but it's not working.
--resource handler
#Configuration
public class WebConfig implements WebMvcConfigurer{
#Override public void addResourceHandlers(ResourceHandlerRegistry registry) {
// Register resource handler for images
System.out.println("indisde cache images");
registry.addResourceHandler("/images/**").addResourceLocations("classpath:/static/images/").setCachePeriod(0);
}
}
--spring application
#SpringBootApplication
public class BookStoreApplication implements CommandLineRunner {
#Autowired
UserService userService;
public static void main(String[] args) {
SpringApplication.run(BookStoreApplication.class, args);
}
#Override
public void run(String... args) throws Exception {
User user2=new User();
user2.setUserName("admin");
user2.setPassword(SecurityUtility.passwordEncoder().encode("admin"));
user2.setEmail("admin#admin.com");
user2.setEnabled(true);
user2.setFirstName("adminFirstName");
user2.setLastName("adminLastName");
user2.setPhone("223456789");
Role role2=new Role();
role2.setRoleId((long) 2);
role2.setRoleName("ROLE_ADMIN");
UserRole userRole2=new UserRole(user2,role2);
Set<UserRole> userRoles2=new HashSet<UserRole>();
userRoles2.add(userRole2);
userService.CreateUser(user2, userRoles2);
}
}
--controller
#RequestMapping(value = "/add/image", method = RequestMethod.POST)
public ResponseEntity uploadImage(#RequestParam(name = "id") Long id, HttpServletRequest request,
HttpServletResponse response) throws IOException {
try {
MultipartHttpServletRequest multipartRequest = (MultipartHttpServletRequest) request;
Iterator<String> imageNames = multipartRequest.getFileNames();
System.out.println(imageNames);
MultipartFile imageMutipart = multipartRequest.getFile(imageNames.next());
byte[] imageBytes = imageMutipart.getBytes();
String imageNameNew = id + ".png";
BufferedOutputStream bout = new BufferedOutputStream(
new FileOutputStream(new File("src/main/resources/static/images/" + imageNameNew)));
bout.write(imageBytes);
bout.flush();
bout.close();
return new ResponseEntity("Upload success", HttpStatus.OK);
} catch (Exception e) {
return new ResponseEntity("Upload failed", HttpStatus.BAD_REQUEST);
}
}
--angular service
upload(bookId: number) {
this.makeFileRequest("http://localhost:8181/book/add/image?id="+bookId, [], this.filesToUpload).then((result) => {
console.log(result);
}, (error) => {
console.log(error);
});
}
modifyImage(bookId: number){
if(this.filesToUpload.length>0){
this.makeFileRequest("http://localhost:8181/book/add/image?id="+bookId, [], this.filesToUpload).then((result) => {
console.log(result);
}, (error) => {
console.log(error);
});
}
}
I was able to solve the issue using the below resource handler
#Configuration
public class AdditionalResourceWebConfiguration implements WebMvcConfigurer {
#Override
public void addResourceHandlers(final ResourceHandlerRegistry registry) {
registry.addResourceHandler("/media/**").addResourceLocations("file:///" + System.getProperty("user.dir") + "/src/main/media/");
}
}
I have Feign client in one service with a method
#PostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
MyDto uploadDocument(#RequestPart("file") MultipartFile file,
#RequestPart("myDto") String myDto);
I have a controller in another service
#PostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public ResponseEntity<MyDto> uploadDocument(#RequestParam("file") MultipartFile file,
#RequestPart("myDto") MyDto myDto) {
.... some code here
}
The issue I faced is that Feign sends myDto with Content-type : text/plain and I have HttpMediaTypeNotSupportedException
Is it possible to send #RequestPart("myDto") String myDto with Content-type : application/json ?
expected Raw request:
----------------------------boundary
Content-Disposition: form-data; name="file"; filename="fileName"
<file>
----------------------------boundary
Content-Disposition: form-data; name="myDto"
**Content-Type: application/json**
{"myDto": ""}
Current raw request:
----------------------------boundary
Content-Disposition: form-data; name="file"; filename="fileName"
<file>
----------------------------boundary
Content-Disposition: form-data; name="myDto"
**Content-Type: text/plain**
{"myDto": ""}
Managed to solve this by replacing the feign-form PojoWriter. By default it's serializing each field of an object as a separate part.
#Bean
public Encoder feignEncoder () {
return new MyFormEncoder(objectMapper, new SpringEncoder(messageConverters));
}
public class MyFormEncoder extends SpringFormEncoder {
/**
* Constructor with specified delegate encoder.
*
* #param delegate delegate encoder, if this encoder couldn't encode object.
*/
public MyFormEncoder(ObjectMapper objectMapper, Encoder delegate) {
super(delegate);
val processor = (MultipartFormContentProcessor) getContentProcessor(MULTIPART);
processor.addFirstWriter(new MyPojoWriter(objectMapper));
}
}
#FieldDefaults(level = PRIVATE, makeFinal = true)
public class MyPojoWriter extends AbstractWriter {
private ObjectMapper objectMapper;
public MyPojoWriter(ObjectMapper objectMapper) {
this.objectMapper = objectMapper;
}
#Override
public boolean isApplicable(Object object) {
return isUserPojo(object);
}
#Override
protected void write(Output output, String key, Object value) throws EncodeException {
var data = "";
try {
data = objectMapper.writeValueAsString(value);
} catch (JsonProcessingException e) {
}
val string = new StringBuilder()
.append("Content-Disposition: form-data; name=\"").append(key).append('"').append(CRLF)
.append("Content-Type: application/json; charset=").append(output.getCharset().name()).append(CRLF)
.append(CRLF)
.append(data)
.toString();
output.write(string);
}
private boolean isUserPojo(#NonNull Object object) {
val type = object.getClass();
val typePackage = type.getPackage();
return typePackage != null && typePackage.getName().startsWith("com.my-package.");
}
}
Update to 2021.
//spring-cloud-openfeign-core
import org.springframework.cloud.openfeign.support.JsonFormWriter;
#Import(JsonFormWriter.class)
public class MyConfig {
#Bean
Encoder feignEncoder(JsonFormWriter jsonFormWriter) {
return new SpringFormEncoder() {{
var processor = (MultipartFormContentProcessor) getContentProcessor(ContentType.MULTIPART);
processor.addFirstWriter(jsonFormWriter);
processor.addFirstWriter(new SpringSingleMultipartFileWriter());
processor.addFirstWriter(new SpringManyMultipartFilesWriter());
}};
You need to define bean JsonFormWriter in your feign client's configuration.
Here is an example of the client:
#FeignClient(
name = "my-client",
configuration = MyClientConfiguration.class
)
public interface MyClient {
#PostMapping(value = "/file/upload", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
void uploadFile(#RequestPart("request") MyFileUploadRequest request,
#RequestPart("file") MultipartFile file);
}
public class MyClientConfiguration {
#Bean
JsonFormWriter jsonFormWriter() {
return new JsonFormWriter();
}
}
And an example of the controller:
#RestController
public class FileUploadApi {
#PostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public void uploadFile(
#RequestPart("request") MyFileUploadRequest request,
#RequestPart("file") MultipartFile file) {
}
This feature was added in scope of this PR: https://github.com/spring-cloud/spring-cloud-openfeign/pull/314
Using the #PathVariable annotation and with a registered SpringFormEncoder you need to convert the "myDto" into a MultipartFile.
The client:
#PostMapping(value = "/files/upload", consumes = MULTIPART_FORM_DATA_VALUE)
MyDto uploadDocument(#PathVariable("file") MultipartFile file, #PathVariable("myDto") MultipartFile myDto)
The encoder:
#RequiredArgsConstructor
public class FeignClientConfiguration {
private final ObjectFactory<HttpMessageConverters> messageConverters;
//To support multipart file upload
#Bean
public Encoder feignFormEncoder() {
return new SpringFormEncoder(new SpringEncoder(messageConverters));
}
}
Creating MultipartFile from the DTO:
public MultipartFile createMultipartFile(#NotNull MyDto myDto) throws JsonProcessingException {
return new org.springframework.mock.web.MockMultipartFile(
"fileName",
"originalFileName",
MediaType.APPLICATION_JSON.toString(),
objectMapper.writeValueAsBytes(myDto));
}
Why this solution with #PathVariable works is described here https://github.com/spring-cloud/spring-cloud-netflix/issues/867
Trying to test a spring controller that we have for multiple file upload. Here is the controller:
#RequestMapping("/vocabularys")
#Controller
public class VocabularyController {
...
The action I want to test:
#RequestMapping(value = "/import", method = {RequestMethod.PUT, RequestMethod.POST})
#ResponseBody
#CacheEvict(value="vocabulary", allEntries=true)
public Object importVocabulary(MultipartHttpServletRequest request, HttpServletResponse response) {
...
The resolver I have in the webmvc-config.xml:
<bean class="org.springframework.web.multipart.commons.CommonsMultipartResolver" id="multipartResolver"/>
The code works just fine and all. I'm running into problems when I am trying to unit/integration test this.
Here is my attempt at the test:
public class VocabularyControllerTest extends BaseControllerTest {
static final private String AdminUsername = "administrator";
#Test
public void shouldBeAbleToUploadAFile() throws Exception {
createTestWorkspace();
login(AdminUsername, "*");
MockMultipartFile file = new MockMultipartFile("test_vocab.xml", new FileInputStream("src/test/files/acme_vocabulary.xml"));
MockMultipartHttpServletRequestBuilder mockMultipartHttpServletRequestBuilder = (MockMultipartHttpServletRequestBuilder) fileUpload("/vocabularys/import").accept(MediaType.ALL).session(httpSession);
mockMultipartHttpServletRequestBuilder.file(file);
mockMultipartHttpServletRequestBuilder.content("whatever");
ResultActions resultActions = mockMvc.perform(mockMultipartHttpServletRequestBuilder);
resultActions.andExpect(status().isFound());
}
}
Ignore the createWorkspace() and login() and stuff - those are for passing through some security filters.
The relevant part of the BaseControllerTest:
#RunWith(SpringJUnit4ClassRunner.class)
#WebAppConfiguration
#ContextHierarchy({
#ContextConfiguration(locations = {
"file:src/test/resources/META-INF/spring/applicationContext.xml",
"file:src/test/resources/META-INF/spring/applicationContext-security.xml",
"file:src/main/resources/META-INF/spring/applicationContext-database.xml",
"file:src/main/resources/META-INF/spring/applicationContext-activiti.xml",
"file:src/main/resources/META-INF/spring/applicationContext-cache.xml",
"file:src/main/resources/META-INF/spring/applicationContext-jms.xml",
"file:src/main/resources/META-INF/spring/applicationContext-mail.xml",
"file:src/main/resources/META-INF/spring/applicationContext-mongo.xml"}),
#ContextConfiguration(locations = {
"file:src/main/webapp/WEB-INF/spring/webmvc-config.xml",
"file:src/test/webapp/WEB-INF/spring/applicationContext-filters.xml"})
})
#Transactional
public class BaseControllerTest extends BaseTest {
#Autowired
WebApplicationContext wac;
#Autowired
MockHttpSession httpSession;
#Autowired
MockServletContext servletContext;
#Autowired
OpenEntityManagerInViewFilter openEntityManagerInViewFilter;
#Autowired
HiddenHttpMethodFilter hiddenHttpMethodFilter;
#Autowired
CharacterEncodingFilter characterEncodingFilter;
#Autowired
SessionFilter sessionFilter;
#Autowired
WorkflowAsSessionFilter workflowAsSessionFilter;
#Autowired
FilterChainProxy springSecurityFilterChain;
#Autowired
RequestFilter requestFilter;
MockMvc mockMvc;
protected static final String TestFileDir = "src/test/files/";
#Before
public void setUp() throws Exception {
this.mockMvc = MockMvcBuilders.webAppContextSetup(this.wac)
.addFilter(openEntityManagerInViewFilter, "/*")
.addFilter(hiddenHttpMethodFilter, "/*")
.addFilter(characterEncodingFilter, "/*")
.addFilter(sessionFilter, "/*")
.addFilter(workflowAsSessionFilter, "/*")
.addFilter(springSecurityFilterChain, "/*")
.addFilter(requestFilter, "/*")
.build();
servletContext.setContextPath("/");
Session session = Session.findBySessionId(httpSession.getId());
if (session == null) {
session = new Session();
session.setJsessionid(httpSession.getId());
session.persist();
}
}
...
The issue is that when I try debugging this, the perform action on the mockMvc object never hits my controller method. I thought it was an issue getting past our security filters (which is why I have all the login and stuff) but I tested other actions in the vocabulary controller and I am able to hit them just fine.
Thoughts? Ideas? Suggestions?
Alright, found the issue.
Spring's MockMultipartHttpServletRequestBuilder returns a MockHttpMultipartServletRequest object eventually.
What the browser does however is post a multipart-encoded request which then gets picked up and parsed by the CommonsMultipartResolver bean defined in the XML.
In the test however, since we are already posting a MockHttpMultipartServletRequest, we don't want the resolver parsing this, so all we got to do is have a profile where the resolver doesn't kick in.
What we have chosen to do however is end up constructing a MockHttpServletRequest that has multipart encoding and put it through the Spring filters so that we can also integration test the resolver kicking in.
Unfortunately I don't see any support/helper in the Spring testing lib which allows you to take a MockHttpServletRequest and addPart() to it, or something to that effect => handcoded browser emulation function :(
The simple way how to test multipart upload is use StandardServletMultipartResolver.
and for test use this code:
final MockPart profilePicture = new MockPart("profilePicture", "stview.jpg", "image/gif", "dsdsdsd".getBytes());
final MockPart userData = new MockPart("userData", "userData", "application/json", "{\"name\":\"test aida\"}".getBytes());
this.mockMvc.perform(
fileUpload("/endUsers/" + usr.getId().toString()).with(new RequestPostProcessor() {
#Override
public MockHttpServletRequest postProcessRequest(MockHttpServletRequest request) {
request.addPart(profilePicture);
request.addPart(userData);
return request;
}
})
MockPart class
public class MockPart extends MockMultipartFile implements Part {
private Map<String, String> headers;
public MockPart(String name, byte[] content) {
super(name, content);
init();
}
public MockPart(String name, InputStream contentStream) throws IOException {
super(name, contentStream);
init();
}
public MockPart(String name, String originalFilename, String contentType, byte[] content) {
super(name, originalFilename, contentType, content);
init();
}
public MockPart(String name, String originalFilename, String contentType, InputStream contentStream) throws IOException {
super(name, originalFilename, contentType, contentStream);
init();
}
public void init() {
this.headers = new HashMap<String, String>();
if (getOriginalFilename() != null) {
this.headers.put("Content-Disposition".toLowerCase(), "form-data; name=\"" + getName() + "\"; filename=\"" + getOriginalFilename() + "\"");
} else {
this.headers.put("Content-Disposition".toLowerCase(), "form-data; name=\"" + getName() + "\"");
}
if (getContentType() != null) {
this.headers.put("Content-Type".toLowerCase(), getContentType());
}
}
#Override
public void write(String fileName) throws IOException {
}
#Override
public void delete() throws IOException {
}
#Override
public String getHeader(String name) {
return this.headers.get(name.toLowerCase());
}
#Override
public Collection<String> getHeaders(String name) {
List<String> res = new ArrayList<String>();
if (getHeader(name) != null) {
res.add(getHeader(name));
}
return res;
}
#Override
public Collection<String> getHeaderNames() {
return this.headers.keySet();
}
}