Develop Reference

Improve Performance with Multiple Row Inserts and Fetches with Oracle SQL Stored Procedures and Java Spring

I currently have stored procedures for Oracle SQL, version 18c, for both inserting and fetching multiple rows of data from one parent table and one child table, being called from my Java Spring Boot application. Everything works fine, but it is extremely slow, for only a few rows of data.
When only inserting 70 records between the two, it takes up to 267 seconds into empty tables. Fetching that same data back out takes about 40 seconds.
Any help would be greatly appreciated or if there is any additional information needed from me.
Below is a cut down and renamed version of my stored procedures for my parent and child tables, actual parent table has 32 columns and child has 11.
PROCEDURE processParentData(
i_field_one varchar2,
v_parent_id OUT number) is
v_new PARENT%ROWTYPE;
BEGIN
v_new.id := ROW_SEQUENCE.nextval;
v_new.insert_time := systimestamp;
v_new.field_one := i_field_one;
insert into PARENT values v_new;
v_parent_id := v_new.id;
END;
PROCEDURE readParentData(
i_field_one IN varchar2,
v_parent OUT SYS_REFCURSOR) AS
BEGIN
OPEN v_parent FOR select h.* from PARENT h
where h.field_one = i_field_one;
END;
PROCEDURE processChild(
i_field_one varchar2,
i_parent_id number) is
v_new CHILD%ROWTYPE;
BEGIN
v_new.id := ROW_SEQUENCE.nextval;
v_new.insert_time := systimestamp;
v_new.field_one := i_field_one;
v_new.parent_id := i_parent_id;
insert into CHILD values v_new;
END;
PROCEDURE readChild(
i_parent_id IN number,
v_child OUT SYS_REFCURSOR) AS
BEGIN
OPEN v_child FOR select h.* from CHILD h
where h.parent_id = i_parent_id;
END;
For my Java code I am using Spring JDBC. After I get the parent data, I then fetch each child data by looping through the parent data and calling readChild with the parent ID for each.
var simpleJdbcCall = new SimpleJdbcCall(jdbcTemplate)
.withCatalogName("PARENT_PACKAGE")
.withProcedureName("processParentData");
SqlParameterSource sqlParameterSource = new MapSqlParameterSource()
.addValue("i_field_one", locationId)
.addValue("v_parent_id", null);
Map<String, Object> out = simpleJdbcCall.execute(sqlParameterSource);
var stopId = (BigDecimal) out.get("v_parent_id");
return stopId.longValue();
var simpleJdbcCall = new SimpleJdbcCall(jdbcTemplate)
.withCatalogName("PARENT_PACKAGE")
.withProcedureName("readParentData")
.returningResultSet("v_parent", BeanPropertyRowMapper.newInstance(Parent.class));
SqlParameterSource sqlParameterSource = new MapSqlParameterSource()
.addValue("i_field_one", location.getId());
Map<String, Object> out = simpleJdbcCall.execute(sqlParameterSource);
return (List<Parent>) out.get("v_parent");
UPDATE 1: As I know and have tested, using the same data and tables, if I use pure JDBC or JPA/Hibernate for inserting and fetching to the tables directly and avoid using stored procedures, then the whole process of inserting and fetching only takes a few seconds.
The issue is, at the company I work at, they have set a policy that all applications going forward are not allowed to have direct read/write access to the database and everything must be done through stored procedures, they say for security reasons. Meaning I need to workout how to do the same thing we have been doing for years with direct read/write access, now with only using Oracle stored procedures.
UPDATE 2: Adding my current Java code for fetching the child data.
for (Parent parent : parents) {
parent.setChilds(childRepository.readChildByParentId(parent.getId()));
}
public List<Child> readChildByParentId(long parentId) {
var simpleJdbcCall = new SimpleJdbcCall(jdbcTemplate)
.withCatalogName("CHILD_PACKAGE")
.withProcedureName("readChild")
.returningResultSet("v_child", BeanPropertyRowMapper.newInstance(Child.class));
SqlParameterSource sqlParameterSource = new MapSqlParameterSource()
.addValue("i_parent_id ", parentId);
Map<String, Object> out = simpleJdbcCall.execute(sqlParameterSource);
return (List<Child>) out.get("v_child");
}

The problem is that the insert you are trying to perform using the stored procedure is not optimized, because you are calling the database every time you try to insert a row.
I strongly recommend you to transform the data to XML (for example, you can also use CSV) and pass it to the procedure, then loop over it and perform the inserts that you need.
Here is an example made using Oracle:
CREATE OR REPLACE PROCEDURE MY_SCHEMA.my_procedure(xmlData clob) IS
begin
FOR CONTACT IN (SELECT *
FROM XMLTABLE(
'/CONTACTS/CONTACT' PASSING
XMLTYPE(contactes)
COLUMNS param_id FOR ORDINALITY
,id NUMBER PATH 'ID'
,name VARCHAR2(100) PATH 'NAME'
,surname VARCHAR2(100) PATH 'SURNAME'
))
LOOP
INSERT INTO PARENT_TABLE VALUES CONTACT.id, CONTACT.name, CONTACT.surname;
end loop;
end;
The XML, you can use a String to pass the data to the procedure:
<CONTACTS>
<CONTACT>
<ID>1</ID>
<NAME>Jonh</NAME>
<SURNAME>Smith</SURNAME>
</CONTACT>
<CONTACTS>

For my Java code I am using Spring JDBC. After I get the parent data, I then fetch each child data by looping through the parent data and calling readChild with the parent ID for each.
Instead of fetching child data in loop, you can modify your procedure to accept list of parent id and return all the data in one call.
It will be helpful if you share spring boot for loop code as well.
Update
Instead of fetching single parent details, you should have update your code like this. Also you have to update your procedure as well.
List<Long> parents = new ArrayList<>();
for (Parent parent : parents) {
parents.add(parent.getId());
}
You can use java streams but that is secondary things.
Now you have to modify your procedure and method to accept multiple parent ids.
List<Child> children = childRepository.readreadChildByParentId(parents);
public List<Child> readChildByParentId(long parentId) {
var simpleJdbcCall = new SimpleJdbcCall(jdbcTemplate)
.withCatalogName("CHILD_PACKAGE")
.withProcedureName("readChild")
.returningResultSet("v_child", BeanPropertyRowMapper.newInstance(Child.class));
SqlParameterSource sqlParameterSource = new MapSqlParameterSource()
.addValue("i_parent_id ", parentId);
Map<String, Object> out = simpleJdbcCall.execute(sqlParameterSource);
return (List<Child>) out.get("v_child");
}
After having all the children you can set parent children via java code.
P.S.
Could you please check if you fetch parents with children if parent is coming from the database?

Your performance problems are probably related with the number of operations performed against the database: you are iterating in Java your collections, and interacting with the database in every iteration. You need to minimize the number of operations performed.
One possible solution can be the use of the standard STRUCT and ARRAY Oracle types. Please, consider for instance the following example:
public static void insertData() throws SQLException {
DriverManagerDataSource dataSource = ...
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
jdbcTemplate.setResultsMapCaseInsensitive(true);
SimpleJdbcCall insertDataCall = new SimpleJdbcCall(jdbcTemplate)
.withCatalogName("parent_child_pkg")
.withProcedureName("insert_data")
.withoutProcedureColumnMetaDataAccess()
.useInParameterNames("p_parents")
.declareParameters(
new SqlParameter("p_parents", OracleTypes.ARRAY, "PARENT_ARRAY")
);
OracleConnection connection = null;
try {
connection = insertDataCall
.getJdbcTemplate()
.getDataSource()
.getConnection()
.unwrap(OracleConnection.class)
;
List<Parent> parents = new ArrayList<>(100);
Parent parent = null;
List<Child> chilren = null;
Child child = null;
for (int i = 0; i < 100; i++) {
parent = new Parent();
parents.add(parent);
parent.setId((long) i);
parent.setName("parent-" + i);
chilren = new ArrayList<>(1000);
parent.setChildren(chilren);
for (int j = 0; j < 1000; j++) {
child = new Child();
chilren.add(child);
child.setId((long) j);
child.setName("parent-" + j);
}
}
System.out.println("Inserting data...");
StopWatch stopWatch = new StopWatch();
stopWatch.start("insert-data");
StructDescriptor parentTypeStructDescriptor = StructDescriptor.createDescriptor("PARENT_TYPE", connection);
ArrayDescriptor parentArrayDescriptor = ArrayDescriptor.createDescriptor("PARENT_ARRAY", connection);
StructDescriptor childTypeStructDescriptor = StructDescriptor.createDescriptor("CHILD_TYPE", connection);
ArrayDescriptor childArrayDescriptor = ArrayDescriptor.createDescriptor("CHILD_ARRAY", connection);
Object[] parentArray = new Object[parents.size()];
int pi = 0;
for (Parent p : parents) {
List<Child> children = p.getChildren();
Object[] childArray = new Object[children.size()];
int ci = 0;
for (Child c : children) {
Object[] childrenObj = new Object[2];
childrenObj[0] = c.getId();
childrenObj[1] = c.getName();
STRUCT childStruct = new STRUCT(childTypeStructDescriptor, connection, childrenObj);
childArray[ci++] = childStruct;
}
ARRAY childrenARRAY = new ARRAY(childArrayDescriptor, connection, childArray);
Object[] parentObj = new Object[3];
parentObj[0] = p.getId();
parentObj[1] = p.getName();
parentObj[2] = childrenARRAY;
STRUCT parentStruct = new STRUCT(parentTypeStructDescriptor, connection, parentObj);
parentArray[pi++] = parentStruct;
}
ARRAY parentARRAY = new ARRAY(parentArrayDescriptor, connection, parentArray);
Map in = Collections.singletonMap("p_parents", parentARRAY);
insertDataCall.execute(in);
connection.commit();
stopWatch.stop();
System.out.println(stopWatch.prettyPrint());
} catch (Throwable t) {
t.printStackTrace();
connection.rollback();
} finally {
if (connection != null) {
try {
connection.close();
} catch (Throwable nested) {
nested.printStackTrace();
}
}
}
}
Where:
CREATE OR REPLACE TYPE child_type AS OBJECT (
id NUMBER,
name VARCHAR2(512)
);
CREATE OR REPLACE TYPE child_array
AS TABLE OF child_type;
CREATE OR REPLACE TYPE parent_type AS OBJECT (
id NUMBER,
name VARCHAR2(512),
children child_array
);
CREATE OR REPLACE TYPE parent_array
AS TABLE OF parent_type;
CREATE SEQUENCE PARENT_SEQ INCREMENT BY 1 MINVALUE 1;
CREATE SEQUENCE CHILD_SEQ INCREMENT BY 1 MINVALUE 1;
CREATE TABLE parent_table (
id NUMBER,
name VARCHAR2(512)
);
CREATE TABLE child_table (
id NUMBER,
name VARCHAR2(512),
parent_id NUMBER
);
CREATE OR REPLACE PACKAGE parent_child_pkg AS
PROCEDURE insert_data(p_parents PARENT_ARRAY);
END;
CREATE OR REPLACE PACKAGE BODY parent_child_pkg AS
PROCEDURE insert_data(p_parents PARENT_ARRAY) IS
l_parent_id NUMBER;
l_child_id NUMBER;
BEGIN
FOR i IN 1..p_parents.COUNT LOOP
SELECT parent_seq.nextval INTO l_parent_id FROM dual;
INSERT INTO parent_table(id, name)
VALUES(l_parent_id, p_parents(i).name);
FOR j IN 1..p_parents(i).children.COUNT LOOP
SELECT child_seq.nextval INTO l_child_id FROM dual;
INSERT INTO child_table(id, name, parent_id)
VALUES(l_child_id, p_parents(i).name, l_parent_id);
END LOOP;
END LOOP;
END;
END;
And Parent and Child are simple POJOs:
import java.util.ArrayList;
import java.util.List;
public class Parent {
private Long id;
private String name;
private List<Child> children = new ArrayList<>();
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public List<Child> getChildren() {
return children;
}
public void setChildren(List<Child> children) {
this.children = children;
}
}
public class Child {
private Long id;
private String name;
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
Please, forgive for the code legibility and incorrect error handling, I will improve the answer later including some information about obtaining the data as well.

The times you mention are horrible indeed. A big boost forward in performance will be to work set based. This means reducing the row by row database calls.
Row by row is synonymous for slow, especially when network round trips are involved.
One call to get the parent.
One call to get the set of children and process them. The jdbc fetch size is a nice tunable here. Give it a chance to work for you.

You do not need to use DYNAMIC SQL OPEN v_parent FOR and also it is not clear how the view v_parent is defined.
Try to check exec plan of this query:
FOR select h.* from PARENT h where h.field_one = ?;
Usually returning recordset via SYS_REFCURSOR increases performance when you return more (let's say) than 10K records.

The SimpleJdbcCall object can be reused in your scenario as only the parameters changes. The SimpleJdbcCall object compiles the jdbc statement on the first invocation. It does some meta-data fetching and it interacts with the Database for that. So, having separate objects would mean fetching same metadata that many times which is not needed.
So, I suggest to initialise all the 4 SimpleJdbcCall objects in the very beginning and then work with them.
var insertParentJdbcCall = new SimpleJdbcCall(jdbcTemplate)
.withCatalogName("PARENT_PACKAGE")
.withProcedureName("processParentData");
var readParentJdbcCall = new SimpleJdbcCall(jdbcTemplate)
.withCatalogName("PARENT_PACKAGE")
.withProcedureName("readParentData")
.returningResultSet("v_parent", BeanPropertyRowMapper.newInstance(Parent.class));
var insertChildJdbcCall = new SimpleJdbcCall(jdbcTemplate)
.withCatalogName("CHILD_PACKAGE")
.withProcedureName("processChildData");
var readChildJdbcCall = new SimpleJdbcCall(jdbcTemplate)
.withCatalogName("CHILD_PACKAGE")
.withProcedureName("readChild")
.returningResultSet("v_child", BeanPropertyRowMapper.newInstance(Child.class));

Java Mongo DB Unable to get valid next id

I am trying insert an item in MongoDB using Java MongoDB driver.Before inserting I am trying to get nextId to insert,but not sure why I am always getting nextId as 4 .I am using below given method to get nextId before inserting any item in Mongo.
private Long getNextIdValue(DBCollection dbCollection) {
Long nextSequenceNumber = 1L;
DBObject query = new BasicDBObject();
query.put("id", -1);
DBCursor cursor = dbCollection.find().sort(query).limit(1);
while (cursor.hasNext()) {
DBObject itemDBObj = cursor.next();
nextSequenceNumber = new Long(itemDBObj.get("id").toString()) + 1;
}
return nextSequenceNumber;
}
I have total 13 record in my mongodb collection.What I am doing wrong here?

Please don't do that. You don't need create a bad management id situation as the driver already do this in the best way, just use the right type and annotation for the field:
#Id
#ObjectId
private String id;
Then write a generic method to insert all entites:
public T create(T entity) throws MongoException, IOException {
WriteResult<? extends Object, String> result = jacksonDB.insert(entity);
return (T) result.getSavedObject();
}
This will create a time-based indexed hash for id's which is pretty much more powerful than get the "next id".
https://www.tutorialspoint.com/mongodb/mongodb_objectid.htm

How can you perform Arithmetic operations like +1 to String
nextSequenceNumber = new Long(itemDBObj.get("id").toString()) + 1;
Try to create a Sequence collection like this.
{"id":"MySequence","sequence":1}
Then use Update to increment the id
// Query for sequence collection
Query query = new Query(new Criteria().where("id").is("MySequence"));
//Increment the sequence by 1
Update update = new Update();
update.inc("sequence", 1);
FindAndModifyOptions findAndModifyOptions = new FindAndModifyOptions();
findAndModifyOptions.returnNew(true);
SequenceCollection sequenceCollection = mongoOperations.findAndModify(query, update,findAndModifyOptions, SequenceCollection.class);
return sequenceModel.getSequence();

I found the work around using b.collection.count().I simply find the total count and incremented by 1 to assign id to my object.

How I can get items from DynamoDBIndexHashKey?

I would like to specify. May I receive elements only from DynamoDBIndexHashKey, not use DynamoDBHashKey?
I have a table with fields
#DynamoDBIndexHashKey (attributeName = "count", globalSecondaryIndexName = "count-index")
#DynamoDBHashKey(attributeName="cluster_output_Id)"
#DynamoDBRangeKey(attributeName="last_fetch)"
I have no #DynamoDBIndexRangeKey
It's code:
MyEntity myEntity = new MyEntity();
myEntity.setCount(1); // Integer
DynamoDBQueryExpression<NewsDynamoDb> queryExpression = new DynamoDBQueryExpression<NewsDynamoDb>()
.withHashKeyValues(myEntity)
.withIndexName("count-index");
queryExpression.setConsistentRead(false);
List<MyEntity> myCollection = mapper.query(MyEntity.class, queryExpression);
Error:
AmazonServiceException: Status Code: 400, AWS Service: AmazonDynamoDBv2, AWS Request ID: I97S04LDGO6FSF56OCJ8S3K167VV4KQNSO5AEMVJF66Q9ASUAAJG, AWS Error Code: ValidationException, AWS Error Message: One or more parameter values were invalid: Invalid number of argument(s) for the EQ ComparisonOperator
How I can get items from DynamoDBIndexHashKey?
P.s. Scan - work but not interesting to me, because in a further I want a sorting
Query with DynamoDBHashKey work. I have problems with DynamoDBIndexHashKey
same example

It is the answer to my question
entity:
#DynamoDBHashKey(attributeName="cluster_output_Id")
public Integer getCluster_output_Id() {
return cluster_output_Id;
}
#DynamoDBIndexHashKey(attributeName = "count", globalSecondaryIndexName = "count-index")
public Integer getCount() {
return count;
}
#DynamoDBRangeKey(attributeName="last_fetch")
#DynamoDBIndexRangeKey(attributeName = "last_fetch", globalSecondaryIndexName = "count-index")
public Date getLast_fetch() {
return last_fetch;
}
code:
dynamoDBMapper = new DynamoDBMapper(amazonDynamoDBClient);
MyClass myClass= new MyClass();
DynamoDBQueryExpression<MyClass > queryExpression = new DynamoDBQueryExpression<MyClass >();
myClass.setCount(1);
queryExpression.setHashKeyValues(myClass);
queryExpression.withIndexName("count-index"); // it's not necessarily
Condition rangeKeyCondition = new Condition();
rangeKeyCondition.withComparisonOperator(ComparisonOperator.NE)
.withAttributeValueList(new AttributeValue().withS(""));
queryExpression.setConsistentRead(false);
List entities = dynamoDBMapper.query(MyClass.class, queryExpression);
Thank you!

like explained here
Table table = dynamoDB.getTable("tableName");
Index index = table.getIndex("count-index");
ItemCollection<QueryOutcome> items = null;
QuerySpec querySpec = new QuerySpec();
querySpec.withKeyConditionExpression("count= :v_count > 0 ")
.withValueMap(new ValueMap() .withString(":v_count","1");
items = index.query(querySpec);
while (iterator.hasNext()) {
//.......
}

You cannot use Query to find items based on sort/range key only.
You can read more here.
In a Query operation, you use the KeyConditionExpression parameter to determine the items to be read from the table or index. You must specify the partition key name and value as an equality condition. You can optionally provide a second condition for the sort key (if present).
In this case your options are:
Scan operation with last_fetch as filter.
Redesign your database to have a GSI with last_fetch as partition key

MongoDB update with layered JSON - how to efficiently? Dot notation?

Mongodb has an update function, where it can increment pre-existing fields. However, I found that it could only update flat JSON. Whenever there's a JSONObject inside of a JSONObject, with a value I want to increment, I can't actually seem to do it. It will return this error:
com.mongodb.WriteConcernException: Write failed with error code 14 and error message
'Cannot increment with non-numeric argument: {laneQty: { BOTTOM: 1 }}'
As you can see, I tried update incrementing laneQty.BOTTOM by 1. I don't want to write an algorithm to change every single layered json field into dot notation(like laneQty.BOTTOM), so is there a way to either turn the JSON into dot notation pre-upsert?
For now my general upsert function looks like this:
public boolean incrementJson(BasicDBObject json, String colName, ArrayList<String> queryParams, ArrayList<String> removeParams){
/*make sure the game id AND the main player id can't both be the same.
If either/or, it's fine. We don't want duplicates.
*/
BasicDBObject query = new BasicDBObject();
DBCollection collection = db.getCollection(colName);
for(int i = 0; i < queryParams.size(); i++){
String param = queryParams.get(i);
query.put(param, json.get(param));
}
for(String param : removeParams){
json.remove(param);
}
return collection.update(query, new BasicDBObject("$inc", json), true, false).isUpdateOfExisting();
}
Is there any suggested upgrades to this code that could make it easily update layered json as well? Thank you!
By the way, it'll be very hard for me to hardcode this. There are a ton of layered objects and that would take me forever. Also, I am not in complete control of which fields are populated in the layers, so I can't just say laneQty.BOTTOM every single time because it will not always exist. Prior to upserting, the BasicDBObject json was actually a java bean parsed into BasicDBObject. This is its constructor if it's of any help:
public ChampionBean(int rank, int division, int assists, int deaths, int kills, int qty, int championId,
HashMap<String, Integer> laneQty, HashMap<String, Integer> roleQty,
ParticipantTimelineDataBean assistedLaneDeathsPerMinDeltas,
ParticipantTimelineDataBean assistedLaneKillsPerMinDeltas, ParticipantTimelineDataBean creepsPerMinDeltas,
ParticipantTimelineDataBean csDiffPerMinDeltas, ParticipantTimelineDataBean damageTakenDiffPerMinDeltas,
ParticipantTimelineDataBean damageTakenPerMinDeltas, ParticipantTimelineDataBean goldPerMinDeltas,
ParticipantTimelineDataBean xpDiffPerMinDeltas, ParticipantTimelineDataBean xpPerMinDeltas, int wins,
int weekDate, int yearDate) {
super();
this.rank = rank;
this.division = division;
this.assists = assists;
this.deaths = deaths;
this.kills = kills;
this.qty = qty;
this.championId = championId;
this.laneQty = laneQty;
this.roleQty = roleQty;
this.assistedLaneDeathsPerMinDeltas = assistedLaneDeathsPerMinDeltas;
this.assistedLaneKillsPerMinDeltas = assistedLaneKillsPerMinDeltas;
this.creepsPerMinDeltas = creepsPerMinDeltas;
this.csDiffPerMinDeltas = csDiffPerMinDeltas;
this.damageTakenDiffPerMinDeltas = damageTakenDiffPerMinDeltas;
this.damageTakenPerMinDeltas = damageTakenPerMinDeltas;
this.goldPerMinDeltas = goldPerMinDeltas;
this.xpDiffPerMinDeltas = xpDiffPerMinDeltas;
this.xpPerMinDeltas = xpPerMinDeltas;
this.wins = wins;
this.weekDate = weekDate;
this.yearDate = yearDate;
}
The participantTimelineDataBean is another bean with 4 int fields inside of it. I want to increment those fields (so yes it's only 2 layers deep, so if there's a solution with 2 layers deep availability I'll take that too).

Use the dot-notation:
new BasicDBObject("$inc", new BasicDBObject("laneQty.BOTTOM", 1) )
Alternative quick&dirty solution: Just collection.save the whole document under the same _id.

Use this library:
https://github.com/rhalff/dot-object
For example if you have an object like this:
var jsonObject = {
info : {
firstName : 'aamir',
lastName : 'ryu'
email : 'aamiryu#gmail.com'
},
}
then your node.js code would be like this:
var dot = require('dot-object');
var jsonObject = // as above ;-);
var convertJsonObjectToDot = dot.dot(jsonObject);
console.log(convertJsonObjectToDot);
Output will be as shown below:
{
info.firstName : 'aamir',
info.lastName : 'ryu',
info.email : 'aamiryu#gmail.com
}
Please bear with me, this is my first answer on stackoverflow ever, since i was searching for the same thing and i found one solution to it, hope it helps you out.

Count number of objects in datastore AppEngine java

I am trying to do a simple application that will persist an object every time a get request is made. In the below code, I use a servlet Put to accomplish this.
public class Put extends HttpServlet {
public void doGet(HttpServletRequest req, HttpServletResponse resp)
throws IOException {
resp.setContentType("text/plain");
PersistenceManagerFactory PMF = JDOHelper
.getPersistenceManagerFactory("transactions-optional");
PersistenceManager pm = PMF.getPersistenceManager();
String id = req.getParameter("id");
String name = req.getParameter("name");
String email = req.getParameter("email");
String productId = req.getParameter("productid");
String timeStamp = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date());
String mailSent = req.getParameter("mailsent");
Product product = new Product(id,name,email,productId,timeStamp,mailSent);
/*
* Get number of objects persisted till now
* Increment the count and use that value as key
*/
Key key = KeyFactory.createKey(Product.class.getSimpleName(),
"1001"); // ??
product.setKey(key);
try {
pm.makePersistent(product);
} finally {
pm.close();
}
}
}
to retrieve all objects I use a Get servlet,
public class Get extends HttpServlet {
public void doGet(HttpServletRequest req, HttpServletResponse resp)
throws IOException {
resp.setContentType("text/plain");
PersistenceManagerFactory PMF = JDOHelper
.getPersistenceManagerFactory("transactions-optional");
PersistenceManager pm = PMF.getPersistenceManager();
/*
* Get number of objects stored
* loop from 0 to the count and print all objects
*
*/
Product e = pm.getObjectById(Product.class, req.getParameter("id"));
resp.getWriter().println();
}
}
My problem is how to get number of objects stored in the datastore?

You should be very careful about using counts in the datastore. All datastore operations are designed to scale with the size of the result set, not the size of the stored set of data. This means that there is no efficient way to count all the entities in your datastore. In large distributed systems, it is difficult to maintain a strongly consistent count, you can see what is necessary to implement this for sharded counters.
Additionally, you should not be storing your data using sequential keys. Additionally, you can run into performance problems by storing your data in sequential order. This is why the default id allocation policy in Datastore switched to using scattered (non-sequential) ids.
In order to loop over all of your entities, you should issue a query over your Product kind.
Query q = pm.newQuery(Product.class);
try {
List<Product> results = (List<Product>) q.execute();
if (!results.isEmpty()) {
for (Product p : results) {
// Process result p
}
} else {
// Handle "no results" case
}
} finally {
q.closeAll();
}
Note that as you get more entities, you'll eventually have too many entities to display on a single page. You should plan for this by setting limits and using cursors to implement paging.
If you want your results in date order, you'll have to order by your timestamp:
Query q = pm.newQuery(Product.class);
q.setOrdering("timestamp");
Also be careful that your query will be eventually consistent. This means that you may not see results in your query for some time after you put it. You'll want to make sure that if this is necessary you rethink your data design to structure it for strong consistency.