I am using spring.cloud to connect to Azure Service Bus in Java. Here is maven dependency I am using:
<dependency>
<groupId>com.azure.spring</groupId>
<artifactId>spring-cloud-azure-starter-integration-servicebus</artifactId>
<version>4.5.0</version>
</dependency>
I am able to consume the message from the queue as byte array and it converts the message to string. Here is my main code after receiving a message from queue:
#ServiceActivator(inputChannel = INPUT_CHANNEL)
public void messageReceiver(byte[] payload, #Header(AzureHeaders.CHECKPOINTER) Checkpointer checkpointer) {
String message = new String(payload);
LOGGER.info("New message received: '{}'", message);
checkpointer.success()
.doOnSuccess(s -> LOGGER.info("Message '{}' successfully checkpointed", message))
.doOnError(e -> LOGGER.error("Error found", e))
.block();
}
And here is my example data in JSON as short version:
{
"serverId": 123,
"message": "{some message}"
}
What I would like to do is to create a Java object like this:
public class ExampleMessage {
private final Integer serverId;
private final String message;
and when a message from queue is consumed, it will convert the message to my Java object. I am used to using DataTypeProvider in order to use custom Java object for AMQP message consumption which will convert and validate the conversion behind the scene. Does spring.cloud.azure has built-in method/functionality for deserialization? or Do I manually deserialize and do validation for a consumed message?
Here I was able to convert the Json object to java object using Gson class.
I am just reading a message from the service bus and converting it to java object.
my pom.xml (dependencies)
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.6</version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-messaging-servicebus</artifactId>
<version>7.0.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/io.projectreactor/reactor-core -->
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-core</artifactId>
<version>3.3.11.RELEASE</version>
</dependency>
My test class :
public class TestClass {
public String name ;
public int version;
TestClass(String n , int v)
{
this.name = n ;
this.version = v ;
}
}
The main class :
#SpringBootApplication
public class ServicebustestApplication {
public static void main(String[] args) throws InterruptedException {
SpringApplication.run(ServicebustestApplication.class, args);
String conn = " Endpoint=sb://v-mganorkarjsonobject.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=HglLVGlgMsYZGQMOtUfp4g2oka1CpCbVR0YEHgly7jA= ";
CountDownLatch countdownLatch1 = new CountDownLatch(1);
ServiceBusProcessorClient processorClient = new ServiceBusClientBuilder()
.connectionString("<Your COnnection String >")
.processor()
.queueName("test")
.processMessage(ServicebustestApplication::processMessage)
.processError(context -> processError(context,countdownLatch1))
.buildProcessorClient();
processorClient.start();
TimeUnit.SECONDS.sleep(10);
processorClient.close();
}
private static void processMessage(ServiceBusReceivedMessageContext context) {
ServiceBusReceivedMessage message = context.getMessage();
System.out.printf("Processing message. Session: %s, Sequence #: %s. Contents: %s%n", message.getMessageId(),
message.getSequenceNumber(), message.getBody());
Gson gson = new Gson();
TestClass testobject = gson.fromJson(String.valueOf(message.getBody()),TestClass.class);
System.out.println("name: "+testobject.name +" version: "+ testobject.version+"");
}
private static void processError(ServiceBusErrorContext context, CountDownLatch countdownLatch) {
}
}
Here the callback to process message will process the message and then we can use the GSON to convert the json string to java object.
Gson gson = new Gson();
TestClass testobject = gson.fromJson(String.valueOf(message.getBody()),TestClass.class);
output of the code :
Related
i am a new bee to GCP and trying to develop springboot rest api that connects to GCP Bigtable, do we have a quickstart guide that helps in development.
Yup, there is. Check out this link to get started. You need the Java client library.
https://cloud.google.com/bigtable/docs/reference/libraries#client-libraries-install-java
for Maven:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>libraries-bom</artifactId>
<version>26.1.4</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-bigtable</artifactId>
</dependency>
Then check this for usage:
https://cloud.google.com/bigtable/docs/reference/libraries#client-libraries-usage-java
import com.google.api.gax.rpc.NotFoundException;
import com.google.cloud.bigtable.data.v2.BigtableDataClient;
import com.google.cloud.bigtable.data.v2.BigtableDataSettings;
import com.google.cloud.bigtable.data.v2.models.Row;
import com.google.cloud.bigtable.data.v2.models.RowCell;
public class Quickstart {
public static void main(String... args) {
String projectId = args[0]; // my-gcp-project-id
String instanceId = args[1]; // my-bigtable-instance-id
String tableId = args[2]; // my-bigtable-table-id
quickstart(projectId, instanceId, tableId);
}
public static void quickstart(String projectId, String instanceId, String tableId) {
BigtableDataSettings settings =
BigtableDataSettings.newBuilder().setProjectId(projectId).setInstanceId(instanceId).build();
// Initialize client that will be used to send requests. This client only needs to be created
// once, and can be reused for multiple requests. After completing all of your requests, call
// the "close" method on the client to safely clean up any remaining background resources.
try (BigtableDataClient dataClient = BigtableDataClient.create(settings)) {
System.out.println("\nReading a single row by row key");
Row row = dataClient.readRow(tableId, "r1");
System.out.println("Row: " + row.getKey().toStringUtf8());
for (RowCell cell : row.getCells()) {
System.out.printf(
"Family: %s Qualifier: %s Value: %s%n",
cell.getFamily(), cell.getQualifier().toStringUtf8(), cell.getValue().toStringUtf8());
}
} catch (NotFoundException e) {
System.err.println("Failed to read from a non-existent table: " + e.getMessage());
} catch (Exception e) {
System.out.println("Error during quickstart: \n" + e.toString());
}
}
}
I want to access a database via the data-api which is AWS providing since the start of 2020.
This is my Maven code (only aws dependency shown):
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.11.790</version>
</dependency>
<dependency>
<groupId>software.amazon.rdsdata</groupId>
<artifactId>rds-data-api-client-library-java</artifactId>
<version>1.0.4</version>
</dependency>
This is my Java code
public class Opstarten {
public static final String RESOURCE_ARN = "arn:aws:rds:eu-central <number - name >";
public static final String SECRET_ARN = "arn:aws:secretsmanager:eu-central-1:<secret>";
public static final String DATABASE = "dbmulesoft";
public static void main(String[] args) {
// TODO Auto-generated method stub
new Opstarten().testme();
}
public void testme( ) {
var account1 = new Account(1, "John"); //plain POJO conform AWS manual hello world example
var account2 = new Account(2, "Mary");
RdsDataClient client = RdsDataClient.builder().database(DATABASE)
.resourceArn(RESOURCE_ARN)
.secretArn(SECRET_ARN).build();
client.forSql("INSERT INTO accounts(accountId, name) VALUES(:accountId, :name)").
withParameter(account1).withParameter(account2).execute();
}
}
Error I am having:
Exception in thread "main" java.lang.NullPointerException
at com.amazon.rdsdata.client.RdsDataClient.executeStatement(RdsDataClient.java:134)
at com.amazon.rdsdata.client.Executor.executeAsSingle(Executor.java:92)
at com.amazon.rdsdata.client.Executor.execute(Executor.java:77)
at nl.bpittens.aws.rds.worker.Opstarten.testme(Opstarten.java:47)
at nl.bpittens.aws.rds.worker.Opstarten.main(Opstarten.java:29)
When I debug it I see that the client object is nog null but the rdsDataService is null as a method or parameter of the client object.
I have checked AWS side for Java RDS Data API but nothing is mentioned there.
Any idea whats wrong ?
Looks like you aren't passing RDS data service, you need to do as follows:
AWSRDSData awsrdsData = AWSRDSDataClient.builder().build();
RdsDataClient client = RdsDataClient.builder()
.rdsDataService(awsrdsData)
.database(DATABASE)
.resourceArn(RESOURCE_ARN)
.secretArn(SECRET_ARN)
.build();
You can also configure mapping options as follows:
MappingOptions mappingOptions = MappingOptions.builder()
.ignoreMissingSetters(true)
.useLabelForMapping(true)
.build();
AWSRDSData awsrdsData = AWSRDSDataClient.builder().build();
RdsDataClient client = RdsDataClient.builder()
.rdsDataService(awsrdsData)
.database(DATABASE)
.resourceArn(RESOURCE_ARN)
.secretArn(SECRET_ARN)
.mappingOptions(mappingOptions)
.build();
I have this HTTP response body to de-serialize:
String response = "result : {'url': u'https://somedomain/', 'fields': {'policy':
u'eyJjb25kaXRpb25zIjogW1siYfgfhudGVudC1sZjMyWiJ9', 'AWSAccessKeyId':
u'ASIccccccNA', 'x-amz-security-token': 'FQofgF', 'key': u'bbb.file',
'signature': u'rm9gdflkjfs='}}"
I am using the jackson.core (2.9.0) java package and lib (have also tried GSON) and get this error:
Exception in thread "main" com.fasterxml.jackson.core.JsonParseException:
Unrecognized token 'u': was expecting ('true', 'false' or 'null')
Deserialization code:
MyResponse deserializedResponse = new ObjectMapper()
.configure(JsonParser.Feature.ALLOW_SINGLE_QUOTES, true)
.configure(JsonParser.Feature.ALLOW_UNQUOTED_FIELD_NAMES, true)
.configure(JsonGenerator.Feature.ESCAPE_NON_ASCII, true)
.readValue(response, MyResponse.class);
I have considered something like this but it feels like there should be a better / safer way:
String sanitizedResponse = response.replaceAll("u'", "'");
--
Using Java 1.8.
Any help appreciated.
As python caused this problem I think the best solution is to let python fix it ;-). Fortunately with jython you can stick with a pure java implementation.
First you need to add the jython standalone dependency in your pom.xml:
<dependencies>
<dependency>
<groupId>org.python</groupId>
<artifactId>jython-standalone</artifactId>
<version>2.7.1</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.5</version>
</dependency>
</dependencies>
(As you can see I also used apache commons io for my example so I added it as well)
I put your (invalid) json string into the text file "c:/temp/json.txt" which has the following content:
{'url': u'https://somedomain/', 'fields': {'policy':
u'eyJjb25kaXRpb25zIjogW1siYfgfhudGVudC1sZjMyWiJ9', 'AWSAccessKeyId':
u'ASIccccccNA', 'x-amz-security-token': 'FQofgF', 'key': u'bbb.file',
'signature': u'rm9gdflkjfs='}}
Now here is the code to read the json file, set up the Python Interpreter and handover the json to clean it up:
String content = FileUtils.readFileToString(new File("c:/temp/json.txt"), "UTF-8");
PythonInterpreter pi = new PythonInterpreter();
pi.exec("import json");
pi.exec("jsondata = " + content);
pi.exec("jsonCleaned = json.dumps(jsondata)");
PyObject jsonCleaned = (PyObject) pi.get("jsonCleaned");
System.out.println(jsonCleaned.asString());
pi.close();
The output is:
{"url": "https://somedomain/", "fields": {"signature": "rm9gdflkjfs=", "AWSAccessKeyId": "ASIccccccNA", "x-amz-security-token": "FQofgF", "key": "bbb.file", "policy": "eyJjb25kaXRpb25zIjogW1siYfgfhudGVudC1sZjMyWiJ9"}}
When you put that in a json validator (https://jsonlint.com/) you can see that it is a valid json now.
I can't tell if the performance is good enough for your use case so you have to test that out.
Remark:
In Eclipse there seems to be a bug with that jython version. It shows the following error:
console: Failed to install '': java.nio.charset.UnsupportedCharsetException: cp0.
Although it works nevertheless you can get rid of it by adding the following VM-Argument to your Run-Configuration:
-Dpython.console.encoding=UTF-8
Remark2: For the sake of completeness and to fully answer that question - here is how you can deserialize the cleaned JSON:
Add GSON Dependency to your pom.xml:
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.2</version>
</dependency>
Create representing classes:
Info class
public class Info {
private String url;
private Fields fields;
public String getUrl() {
return url;
}
public void setUrl(String url) {
this.url = url;
}
public Fields getFields() {
return fields;
}
public void setFields(Fields fields) {
this.fields = fields;
}
}
Fields class
import com.google.gson.annotations.SerializedName;
public class Fields {
private String signature;
private String AWSAccessKeyId;
#SerializedName("x-amz-security-token")
private String x_amz_security_token;
private String key;
private String policy;
public String getSignature() {
return signature;
}
public void setSignature(String signature) {
this.signature = signature;
}
public String getAWSAccessKeyId() {
return AWSAccessKeyId;
}
public void setAWSAccessKeyId(String aWSAccessKeyId) {
AWSAccessKeyId = aWSAccessKeyId;
}
public String getX_amz_security_token() {
return x_amz_security_token;
}
public void setX_amz_security_token(String x_amz_security_token) {
this.x_amz_security_token = x_amz_security_token;
}
public String getKey() {
return key;
}
public void setKey(String key) {
this.key = key;
}
public String getPolicy() {
return policy;
}
public void setPolicy(String policy) {
this.policy = policy;
}
}
Finally add the following code after you get your cleaned JSON:
Gson gson = new Gson();
Info info = gson.fromJson(jsonCleaned.asString(), Info.class);
You need to use the following regex that replaces u' from beginning of word boundaries to replace with '
String regexPattern = "(\\bu')";
I am trying to insert a json string which contains an array of documents but getting following exception.
MongoDB server version: 3.0.6
Mongo-Java driver version: 3.1.0
I understand that insertOne() method is used to insert just one document but over here it's an array of documents. I am not sure how to use insertMany() method here.
Please guide.
JSON String that I want to insert:
json = [{"freightCompanyId":201,"name":"USPS","price":8.00},{"freightCompanyId":202,"name":"FedEx","price":10.00},{"freightCompanyId":203,"name":"UPS","price":12.00},{"freightCompanyId":204,"name":"Other","price":15.00}]
Exception Log:
Exception in thread "main" org.bson.BsonInvalidOperationException: readStartDocument can only be called when CurrentBSONType is DOCUMENT, not when CurrentBSONType is ARRAY.
at org.bson.AbstractBsonReader.verifyBSONType(AbstractBsonReader.java:655)
at org.bson.AbstractBsonReader.checkPreconditions(AbstractBsonReader.java:687)
at org.bson.AbstractBsonReader.readStartDocument(AbstractBsonReader.java:421)
at org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:138)
at org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:45)
at org.bson.Document.parse(Document.java:105)
at org.bson.Document.parse(Document.java:90)
at com.ebayenterprise.ecp.jobs.Main.insert(Main.java:52)
at com.ebayenterprise.ecp.jobs.Main.main(Main.java:31)
Main.java
public class Main {
private static final Logger LOG = Logger.getLogger(Main.class);
public static void main(String[] args) throws IOException {
String json = getAllFreightCompanies();
insert(json);
}
private static String getAllFreightCompanies() throws IOException {
FreightCompanyDao freightCompanyDao = new FreightCompanyDaoImpl(DataSourceFactory.getDataSource(DatabaseType.POSTGRES.name()));
List<FreightCompany> freightCompanies = freightCompanyDao.getAllFreightCompanies();
return GenericUtils.toJson(freightCompanies);
}
private static void insert(String json) {
MongoClient mongoClient = new MongoClient("GSI-547576", 27017);
MongoDatabase database = mongoClient.getDatabase("test");
MongoCollection<Document> table = database.getCollection("fc");
Document document = Document.parse(json);
table.insertOne(document);
}
}
GenericUtils.java
public final class GenericUtils {
private static final Logger LOG = Logger.getLogger(GenericUtils.class);
private GenericUtils() {
}
public static String toJson(List<FreightCompany> freightCompanies) throws IOException {
String json = new ObjectMapper().writer().writeValueAsString(freightCompanies);
LOG.debug("json = " + json);
return json;
}
}
pom.xml
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongo-java-driver</artifactId>
<version>3.1.0</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.codehaus.jackson</groupId>
<artifactId>jackson-mapper-asl</artifactId>
<version>1.9.13</version>
</dependency>
You should either insert one by one or create a List of documents and use insertMany()
Here's an example:
MongoClient mongoClient = new MongoClient("GSI-547576", 27017);
MongoDatabase database = mongoClient.getDatabase("test");
MongoCollection < Document > table = database.getCollection("fc");
FreightCompanyDao freightCompanyDao = new FreightCompanyDaoImpl(DataSourceFactory.getDataSource(DatabaseType.POSTGRES.name()));
List < FreightCompany > freightCompanies = freightCompanyDao.getAllFreightCompanies();
for (FreightCompany company: freighetCompanies) {
Document doc = Document.parse(GenericUtils.toJson(company))
collection.insertOne(doc)
}
I'm working on a project with GWT and J2SE clients. The GWT part is working great, but now there are problems with the J2SE client;
"The server understands the content type of the request entity and the
syntax of the request entity is correct but was unable to process the
contained instructions"
"The serialized representation must have this media type:
application/x-java-serialized-object or this one:
application/x-java-serialized-object+xml"
This code was working some months/versions ago... Both PUT and POST produce this error while GET is working. Whats wrong here?
Here's a really simple test case
// Shared Interface
public interface J2SeClientServerResourceInt
{
#Post("json")
public J2seStatusDto postJ2seStatus(J2seStatusDto pJ2seStatusDto);
}
// Java Bean
public class J2seStatusDto implements Serializable
{
private static final long serialVersionUID = 6901448809350740172L;
private String mTest;
public J2seStatusDto()
{
}
public J2seStatusDto(String pTest)
{
setTest(pTest);
}
public String getTest()
{
return mTest;
}
public void setTest(String pTest)
{
mTest = pTest;
}
}
// Server
public class J2seServerResource extends ClaireServerResource implements J2SeServerResourceInt
{
#Override
public J2seStatusDto postJ2seStatusDto(J2seStatusDto pJ2seStatusDto)
{
return pJ2seStatusDto;
}
}
// J2SE Client
public class ClaireJsSeTestClient
{
public static void main(String[] args)
{
Reference lReference = new Reference("http://localhost:8888//rest/j2se");
ClientResource lClientResource = new ClientResource(lReference);
lClientResource.accept(MediaType.APPLICATION_JSON);
J2SeServerResourceInt lJ2SeServerResource = lClientResource.wrap(J2SeServerResourceInt.class);
J2seStatusDto lJ2seStatusDto = new J2seStatusDto("TEST");
J2seStatusDto lJ2seResultDto = lJ2SeServerResource.postJ2seStatusDto(lJ2seStatusDto);
}
}
// Maven J2Se Client
<dependencies>
<dependency>
<groupId>org.restlet.jse</groupId>
<artifactId>org.restlet</artifactId>
<version>3.0-M1</version>
</dependency>
<dependency>
<groupId>org.restlet.jse</groupId>
<artifactId>org.restlet.ext.jackson</artifactId>
<version>3.0-M1</version>
</dependency>
</dependencies>
// Maven GAE Server
<dependency>
<groupId>org.restlet.gae</groupId>
<artifactId>org.restlet</artifactId>
<version>3.0-M1</version>
</dependency>
<dependency>
<groupId>org.restlet.gae</groupId>
<artifactId>org.restlet.ext.servlet</artifactId>
<version>3.0-M1</version>
</dependency>
<dependency>
<groupId>org.restlet.gae</groupId>
<artifactId>org.restlet.ext.jackson</artifactId>
<version>3.0-M1</version>
</dependency>
<dependency>
<groupId>org.restlet.gae</groupId>
<artifactId>org.restlet.ext.gwt</artifactId>
<version>3.0-M1</version>
</dependency>
<dependency>
<groupId>org.restlet.gwt</groupId>
<artifactId>org.restlet</artifactId>
<version>3.0-M1</version>
<scope>compile</scope>
</dependency>
Thierry Boileau fixed our problem (mistake);
https://github.com/restlet/restlet-framework-java/issues/1029#issuecomment-76212062
Due to the constraints of the GAE platform (there is no support of chunked encoding) you have to specify that the request entity is buffered first;
cr.setRequestEntityBuffering(true);
Thanks for the great support at restlet.com