Develop Reference

How to mask out some field in grpc response when logging

I want to mask out some field when logging.
is there any good way to mask out?
after message is served to client.
I'm trying to mask out some field before logging
how do you mask out field?
--served to client
message {
id : "user",
phoneNumber : "12341234"
}
--logged response
message {
id : "user",
phoneNumber : "12******"
}

I don't think there is a way to alter the data like you mentioned, however there is a way to hide field that you don't want to print in the log (I assume its for security reasons). The way yo do that is with FieldMask.
So for the sake of the example I took the liberty of adding another field to the schema I assume you have:
message User {
string id = 1;
string name = 2;
string phone_number = 3;
}
So now let's say that I don't want to show the phone number of my users, I can use a FieldMask to keep only id and name. So first we build a FieldMask (you'll need to add this dependency for accessing FieldMaskUtil):
public static final FieldMask WITH_ID_AND_NAME =
FieldMaskUtil.fromFieldNumbers(User.class,
User.ID_FIELD_NUMBER,
User.NAME_FIELD_NUMBER);
then what I can do is merge this FieldMask with an existing object:
private static void log(User user, FieldMask fieldMask) {
User.Builder userWithMaskedFields = User.newBuilder();
FieldMaskUtil.merge(fieldMask, user, userWithMaskedFields);
System.out.println(userWithMaskedFields);
}
public static void main(String[] args) {
User user = User.newBuilder()
.setId("user")
.setName("a name")
.setPhoneNumber("12341234")
.build();
System.out.println("---Without FieldMask---");
System.out.println(user);
System.out.println("---With FieldMask---");
log(user, WITH_ID_AND_NAME);
}
This will give me the following output:
---Without FieldMask---
id: "user"
name: "a name"
phone_number: "12341234"
---With FieldMask---
id: "user"
name: "a name"
I hope that helps, let me know if you need more details, I can update my answer.

why i am getting rows with null values when i am trying to create view from json file in spark with java

i am reading Json file and creating view in spark with java when i am trying to display it was displaying two extra row starting and ending with null values
i have tried with different options line multi line true but it's not working
class Something
{
public void DoSomething() {
SparkSession session = SparkSession.builder().appName("jsonreader")
.master("local[4]").getOrCreate();
Dataset<Row> jsondataset = session.read()
.json("G:\\data\\employee.json");
jsondataset.select("id","name","age").show();
}
}
+----+-------+----+
| id| name| age|
+----+-------+----+
|null| null|null|
|1201| satish| 25|
|1202|krishna| 28|
|null| null|null|
+----+-------+----+
{
{"id" : "1201", "name" : "satish", "age" : "25"}
{"id" : "1202", "name" : "krishna", "age" : "28"}
}
is my json file and i am getting out put rows with null values like above
can any one help me why i am getting like this

The extra curly brackets are causing this. You will have to handle it either before reading JSON or after reading it ie through spark. Also the NULLs are read as strings not exactly NULL. Below is my workaround, the filter condition will uniquely identify these faulty rows due the "null" being string. :
jsondataset = jsondataset.select("age","id","name").filter("age <> 'null'")
jsondataset.show()
// Result
// +---+----+-------+
// |age|id |name |
// +---+----+-------+
// |25 |1201|satish |
// |28 |1202|krishna|
// +---+----+-------+

One to many Spring Result Mapping

This is my first time posting problem hope someone can help me on this mapping the result into:
List<Clients>
So here's the sample result from database:
NameID | Name | Notes | Date
1 | Client1 | Get this on monday. | null
1 | Client1 | Get this on wednesday. | null
2 | Client2 | Meet on saturday. | null
So here's my mode (java class).
Name.java
private int NameId;
private String ClientName;
.. getter and setter
Notes.java
private int NotesId;
private String ClientNote;
.. getter and setter
Clients.java
private Name name;
private List<Notes> notes;
.. getter and setter
ClientResultSet.java
class ClientResultSet implements ResultSetExtractor<List<Clients>>{
#Override
public List<Clients> extractData(ResultSet rs) throws SQLException, DataAccessException {
Map<int, Clients> map = new HashMap<int, Clients>();
while(rs.next()) {
int NameId= rs.getInt("NameId");
Clients clients= map.get(contactId);
if (clients== null) {
// I'm losing my thoughts here. :'(
}
}
return null;
}
}
Result I want to achieve:
[
{
name:{
NameId: 1,
Name: "Client1"
},
notes[
{
NotesId: 1,
ClientNote: "Get this on monday",
Date: null
},
{
NoteId: 2,
ClientNote: "Get this on wednesday",
Date: null
}
]
},
{
name:{},
notes:[{},{}]
}
... and so on
]
I'm reading ResultSetExtractor but I don't know how to implement it. Thanks in advance and have a good day.

This is how your database should look like, you will create 4 tables, client, name, note, client_note. the client_note table should have the client id and the note id for one to many relationship. so if you want to get all notes for a client with an id of 1; your sql will look like this
SELECT note_id FROM client_note WHERE clientId = 1;
then you can use the note id's you get to query the note table for the actual note objects;
client
int id;
int nameId (foreignKey referencing name.id);
name
int id;
String ClientName;
note
int id;
String ClientNote;
client_note
int id;
int clientId (foreignKey referencing client.id);
int noteId (foreignKey referencing note.id);
Hope this helps?

I think everyone misunderstood my question and I don't blame everyone. I love that my question gets notice and I thank everyone who gave their suggestion to me.
I found the answer here if anyone have the same problem as me:
multiple one-to-many relations ResultSetExtractor
Again, thank you everyone and have a nice day.

Empty JSON vs null conversions to a

I'm trying to create an HTTP PUT endpoint in Java that takes in a delta json from the front end and I'm having some trouble figuring out how to implement "nulls".
For example if we have a database model that looks like
id : 1
firstname : Luke
lastname : Xu
age : 24
fav_color : Red
And we send over a PUT request to /person/1 with a json of {age : 25}. Currently, I have a JOOQ pojo to converts to following JSON to a java model but the problem is it is also updating my database values to be null.
There's no difference between
{age : 25}
and
{id : 1,
firstname : null,
lastname : null,
age : 25,
fav_color : null}
Once it hits my Java end point the java model just sets both cases to "null" and there's no difference between a passed in null or the value wasn't being passed in at all.
I also considered processing an input stream (type JSON) but the problem with this is that our JSON names have to be named exactly the same as the database column names which is also kind of unreasonable.
What is the standard for editing the database if we only want to send a "delta json"????

Since you're using jOOQ, I'd suggest you directly pass the JSON values to the jOOQ UpdatableRecord, which can in fact distinguish between:
null meaning not initialised (or default)
null meaning null
It does so by maintaining a changed() flag for each individual column.
For instance:
{age : 25}
... translates to this Java code:
// record.set(USER.ID, 1) I suspect this is still necessary...?
record.set(USER.AGE, 25);
record.update();
... and to this SQL statement:
UPDATE users
SET age = 25
WHERE id = 1
whereas
{id : 1,
firstname : null,
lastname : null,
age : 25,
fav_color : null}
... translates to this Java code
record.set(USER.ID, 1);
record.set(USER.FIRSTNAME, null);
record.set(USER.LASTNAME, null);
record.set(USER.AGE, 25);
record.set(USER.FAV_COLOR, null);
... and to this SQL statement
UPDATE users
SET firstname = null,
lastname = null,
age = 25,
fav_color = null
WHERE id = 1

JPA double relation with the same Entity

I have these Entities:
#Entity
public class Content extends AbstractEntity
{
#NotNull
#OneToOne(optional = false)
#JoinColumn(name = "CURRENT_CONTENT_REVISION_ID")
private ContentRevision current;
#OneToMany(mappedBy = "content", cascade = CascadeType.ALL, orphanRemoval = true)
private List<ContentRevision> revisionList = new ArrayList<>();
}
#Entity
public class ContentRevision extends AbstractEntity
{
#NotNull
#ManyToOne(optional = false)
#JoinColumn(name = "CONTENT_ID")
private Content content;
#Column(name = "TEXT_DATA")
private String textData;
#Temporal(TIMESTAMP)
#Column(name = "REG_DATE")
private Date registrationDate;
}
and this is the db mapping:
CONTENT
+-----------------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------------------+--------------+------+-----+---------+----------------+
| ID | bigint(20) | NO | PRI | NULL | auto_increment |
| CURRENT_CONTENT_REVISION_ID | bigint(20) | NO | MUL | NULL | |
+-----------------------------+--------------+------+-----+---------+----------------+
CONTENT_REVISION
+-----------------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------------------+--------------+------+-----+---------+----------------+
| ID | bigint(20) | NO | PRI | NULL | auto_increment |
| REG_DATE | datetime | YES | | NULL | |
| TEXT_DATA | longtext | YES | | NULL | |
| CONTENT_ID | bigint(20) | NO | MUL | NULL | |
+-----------------------------+--------------+------+-----+---------+----------------+
I have also these requirements:
Content.current is always a member of Content.revisionList (think about Content.current as a "pointer").
Users can add a new ContentRevision to an existing Content
Users can add a new Content with an initial ContentRevision (cascade persist)
Users can change Content.current (move the "pointer")
Users can modify Content.current.textData, but saves Content (cascade merge)
Users can delete ContentRevision
Users can delete Content (cascade remove to ContentRevision)
Now, my questions are:
Is this the best approach? Any best practice?
Is it safe to cascade merge when the same entity is referenced twice? (Content.current is also Content.revisionList[i])
Are Content.current and Content.revisionList[i] the same instance? (Content.current == Content.revisionList[i] ?)
Thanks
#jabu.10245 I'm very grateful for your effort. Thank you, really.
However, there's a problematic (missing) case from your tests: when you run it inside a container using CMT:
#RunWith(Arquillian.class)
public class ArquillianTest
{
#PersistenceContext
private EntityManager em;
#Resource
private UserTransaction utx;
#Deployment
public static WebArchive createDeployment()
{
// Create deploy file
WebArchive war = ShrinkWrap.create(WebArchive.class, "test.war");
war.addPackages(...);
war.addAsResource("persistence-arquillian.xml", "META-INF/persistence.xml");
war.addAsManifestResource(EmptyAsset.INSTANCE, "beans.xml");
// Show the deploy structure
System.out.println(war.toString(true));
return war;
}
#Test
public void testDetached()
{
// find a document
Document doc = em.find(Document.class, 1L);
System.out.println("doc: " + doc); // Document#1342067286
// get first content
Content content = doc.getContentList().stream().findFirst().get();
System.out.println("content: " + content); // Content#511063871
// get current revision
ContentRevision currentRevision = content.getCurrentRevision();
System.out.println("currentRevision: " + currentRevision); // ContentRevision#1777954561
// get last revision
ContentRevision lastRevision = content.getRevisionList().stream().reduce((prev, curr) -> curr).get();
System.out.println("lastRevision: " + lastRevision); // ContentRevision#430639650
// test equality
boolean equals = Objects.equals(currentRevision, lastRevision);
System.out.println("1. equals? " + equals); // true
// test identity
boolean same = currentRevision == lastRevision;
System.out.println("1. same? " + same); // false!!!!!!!!!!
// since they are not the same, the rest makes little sense...
// make it dirty
currentRevision.setTextData("CHANGED " + System.currentTimeMillis());
// perform merge in CMT transaction
utx.begin();
doc = em.merge(doc);
utx.commit(); // --> ERROR!!!
// get first content
content = doc.getContentList().stream().findFirst().get();
// get current revision
currentRevision = content.getCurrentRevision();
System.out.println("currentRevision: " + currentRevision);
// get last revision
lastRevision = content.getRevisionList().stream().reduce((prev, curr) -> curr).get();
System.out.println("lastRevision: " + lastRevision);
// test equality
equals = Objects.equals(currentRevision, lastRevision);
System.out.println("2. equals? " + equals);
// test identity
same = currentRevision == lastRevision;
System.out.println("2. same? " + same);
}
}
since they are not the same:
if I enable cascading on both properties, an Exception is thrown
java.lang.IllegalStateException:
Multiple representations of the same entity [it.shape.edea2.jpa.ContentRevision#1] are being merged.
Detached: [ContentRevision#430639650];
Detached: [ContentRevision#1777954561]
if I disable cascade on current, the change get lost.
the strange thing is that running this test outside the container results in successful execution.
Maybe it's lazy loading (hibernate.enable_lazy_load_no_trans=true), maybe something else, but it's definitely NOT SAFE.
I wonder if there's a way to get the same instance.

Is it safe to cascade merge when the same entity is referenced twice?
Yes. If you manage an instance of Content, then it's Content.revisionList and Content.current are managed as well. Changes in any of those will be persisted when flushing the entity manager. You don't have to call EntityManager.merge(...) manually, unless you're dealing with transient objects that need to be merged.
If you create a new ContentRevision, then call persist(...) instead of merge(...) with that new instance and make sure it has a managed reference to the parent Content, also add it to the content's list.
Are Content.current and Content.revisionList[i] the same instance?
Yes, should be. Test it to be sure.
Content.current is always a member of Content.revisionList (think about Content.current as a "pointer").
You could do that check in in SQL with a check constraint; or in Java, although you'd have to be sure the revisionList is fetched. By default it's lazy fetched, meaning Hibernate will run another query for this list if you access the getRevisionList() method. And for that you need a running transaction, otherwise you'll be getting a LazyInitializationException.
You could instead load the list eagerly, if that's what you want. Or you could define a entity graph to be able to support both strategies in different queries.
Users can modify Content.current.textData, but saves Content (cascade merge)
See my first paragraph above, Hibernate should save changes on any managed entity automatically.
Users can delete ContentRevision
if (content.getRevisionList().remove(revision))
entityManager.remove(revision);
if (revision.equals(content.getCurrentRevision())
content.setCurrentRevision(/* to something else */);
Users can delete Content (cascade remove to ContentRevision)
Here I'd prefer to ensure that in the database schema, for instance
FOREIGN KEY (content_id) REFERENCES content (id) ON DELETE CASCADE;
UPDATE
As requested, I wrote a test. See this gist for the implementations of Content and ContentRevision I used.
I had to make one important change though: Content.current cannot really be #NotNull, especially not the DB field, because if it were, then we couldn't persist a content and revision at the same time, since both have no ID yet. Hence the field must be allowed to be NULL initially.
As a workaround I added the following method to Content:
#Transient // ignored in JPA
#AssertTrue // javax.validation
public boolean isCurrentRevisionInList() {
return current != null && getRevisionList().contains(current);
}
Here the validator ensures that the there is always a non-null current revision and that it is contained in the revision list.
Now here are my tests.
This one proves that the references are the same (Question 3) and that it is enough to persist content where current and revisionList[0] is referencing the same instance (question 2):
#Test #InSequence(0)
public void shouldCreateContentAndRevision() throws Exception {
// create java objects, unmanaged:
Content content = Content.create("My first test");
assertNotNull("content should have current revision", content.getCurrent());
assertSame("content should be same as revision's parent", content, content.getCurrent().getContent());
assertEquals("content should have 1 revision", 1, content.getRevisionList().size());
assertSame("the list should contain same reference", content.getCurrent(), content.getRevisionList().get(0));
// persist the content, along with the revision:
transaction.begin();
entityManager.joinTransaction();
entityManager.persist(content);
transaction.commit();
// verify:
assertEquals("content should have ID 1", Long.valueOf(1), content.getId());
assertEquals("content should have one revision", 1, content.getRevisionList().size());
assertNotNull("content should have current revision", content.getCurrent());
assertEquals("revision should have ID 1", Long.valueOf(1), content.getCurrent().getId());
assertSame("current revision should be same reference", content.getCurrent(), content.getRevisionList().get(0));
}
The next ensures that it's still true after loading the entity:
#Test #InSequence(1)
public void shouldLoadContentAndRevision() throws Exception {
Content content = entityManager.find(Content.class, Long.valueOf(1));
assertNotNull("should have found content #1", content);
// same checks as before:
assertNotNull("content should have current revision", content.getCurrent());
assertSame("content should be same as revision's parent", content, content.getCurrent().getContent());
assertEquals("content should have 1 revision", 1, content.getRevisionList().size());
assertSame("the list should contain same reference", content.getCurrent(), content.getRevisionList().get(0));
}
And even when updating it:
#Test #InSequence(2)
public void shouldAddAnotherRevision() throws Exception {
transaction.begin();
entityManager.joinTransaction();
Content content = entityManager.find(Content.class, Long.valueOf(1));
ContentRevision revision = content.addRevision("My second revision");
entityManager.persist(revision);
content.setCurrent(revision);
transaction.commit();
// re-load and validate:
content = entityManager.find(Content.class, Long.valueOf(1));
// same checks as before:
assertNotNull("content should have current revision", content.getCurrent());
assertSame("content should be same as revision's parent", content, content.getCurrent().getContent());
assertEquals("content should have 2 revisions", 2, content.getRevisionList().size());
assertSame("the list should contain same reference", content.getCurrent(), content.getRevisionList().get(1));
}
SELECT * FROM content;
id | version | current_content_revision_id
----+---------+-----------------------------
1 | 2 | 2
UPDATE 2
It was hard to reproduce that situation on my machine, but I got it to work. Here is what I've done so far:
I changed all #OneToMany relations to use lazy fetching (the default) and rerun the following test case:
#Test #InSequence(3)
public void shouldChangeCurrentRevision() throws Exception {
transaction.begin();
entityManager.joinTransaction();
Document document = entityManager.find(Document.class, Long.valueOf(1));
assertNotNull(document);
assertEquals(1, document.getContentList().size());
Content content = document.getContentList().get(0);
assertNotNull(content);
ContentRevision revision = content.getCurrent();
assertNotNull(revision);
assertEquals(2, content.getRevisionList().size());
assertSame(revision, content.getRevisionList().get(1));
revision.setTextData("CHANGED");
document = entityManager.merge(document);
content = document.getContentList().get(0);
revision = content.getCurrent();
assertSame(revision, content.getRevisionList().get(1));
assertEquals("CHANGED", revision.getTextData());
transaction.commit();
}
The test passed with lazy fetching. Note that lazy fetching requires it to be executed within a transaction.
For some reason the content revision instance you're editing is not the same as the one in the one-to-many list. To reproduce that I've modified my test as follows:
#Test #InSequence(4)
public void shouldChangeCurrentRevision2() throws Exception {
transaction.begin();
Document document = entityManager.find(Document.class, Long.valueOf(1));
assertNotNull(document);
assertEquals(1, document.getContentList().size());
Content content = document.getContentList().get(0);
assertNotNull(content);
ContentRevision revision = content.getCurrent();
assertNotNull(revision);
assertEquals(2, content.getRevisionList().size());
assertSame(revision, content.getRevisionList().get(1));
transaction.commit();
// load another instance, different from the one in the list:
revision = entityManager.find(ContentRevision.class, revision.getId());
revision.setTextData("CHANGED2");
// start another TX, replace the "current revision" but not the one
// in the list:
transaction.begin();
document.getContentList().get(0).setCurrent(revision);
document = entityManager.merge(document); // here's your error!!!
transaction.commit();
content = document.getContentList().get(0);
revision = content.getCurrent();
assertSame(revision, content.getRevisionList().get(1));
assertEquals("CHANGED2", revision.getTextData());
}
And there, I got exactly your error. Then I modified the cascading setting on the #OneToMany mapping:
#OneToMany(mappedBy = "content", cascade = { PERSIST, REFRESH, REMOVE }, orphanRemoval = true)
private List<ContentRevision> revisionList;
And the error disappeared :-) ... because I removed CascadeType.MERGE.