Stream insert from java: templateSuffix - java

I would like to stream data from my java code to a BigQuery table using templateSuffix, but I can't make it work properly. My code:
return bigquery.tabledata()
.insertAll(
projectId,
datasetId,
tableId,
new TableDataInsertAllRequest()
.setTemplateSuffix(templateSuffix)
.setRows(singletonList(row))
).execute();
When I run it with projectId, datasetId, MyTable20160426 and 20160426, I get the error:
"message" : "404 Not found: Table projectId:datasetId.MyTable20160426"
When I run it with projectId, datasetId, MyTable and 20160426, I get the error:
"message" : "404 Not found: Table projectId:datasetId.MyTable"
The table MyTable already exists and already is templated on date (I used the bulk upload for GCS) (20160426 is the today date)
How cat I make it work ?
Where should I look to understand what's wrong ?
Thanks

First, the base table projectId:datasetId.MyTable should exist and should already have a schema. Thisis how BigQuery knows how to find the schema of the templated table that gets created.
Second, You should pass MyTable and instead of MyTable20160426 as the table ID in your request.
Third, the existence (or non-existence) of a table is cached. So if you get a "not found" error and then create the table, you'll still get a "not found" error for up to a half hour.
It sounds like you might be able to wait and try again. If this doesn't work, please provide the actual project, dataset, and table ids you're using and e-mail the details to tigani#google.com, and I can help look into what is going on.

Related

Push Data to Existing Table in BigQuery

I am using Java and SQL to push data to a Timestamp partitioned table in BigQuery. In my code, I specify the destination table:
.setDestinationTable(TableId.of("MyDataset", "MyTable"))
When I run it, it creates a table perfectly. However, when I attempt to insert new data, it throws a BigQueryException claiming the table already exists:
Exception in thread "main" com.google.cloud.bigquery.BigQueryException:
Already Exists: Table MyProject:MyDataset.MyTable
After some documentation digging, I found a solution that works:
.setWriteDisposition(WriteDisposition.WRITE_APPEND)
Adding the above appends any data (even if it's duplicate). I'm not sure why the default setting for .setDestinationTable() is the equivalent of WRITE_EMPTY, which returns a duplicate error. The google docs for .setDestinationTable() are:
Describes the table where the query results should be stored. If not
present, a new table will be created to store the results.
The docs should probably clarify the default value.

sql parameter on http links

i'm developping an android application that need to browse a MySQL Database.
I've suceeded on getting a infortion when i want all information on a table, the probleme is, when i want to add parameter like SELECT * FROM TABLE WHERE Name=name i dont know how to add he parameter on the HTTP link.
This is the link to get all Countries " http://cookndinner.atwebpages.com/api/countries " this one works well ( you can try it ) but when i want to select a country going by Code = Fr with this sql request SELECT * FROM COUNTRIES WHERE CODE=FR it doesnt work.
I tried this link : http://cookndinner.atwebpages.com/api/countries/
this one : http://cookndinner.atwebpages.com/api/countries/FR
and this one : http://cookndinner.atwebpages.com/api/countries/code=FR
none of it are working, and it just give all information on the table and doesn't add the parameter
I also trid this one for another table : http://cookndinner.atwebpages.com/api/users?email=louisard2#gmail.com&password=bankai123
but it's not working either
COuld someone help me and indicate how to insert SQL Parameter on Http links pls or if you know a tutorial that may be useful
Thank you
PS : the php part is already done
Starting from the beginning, make sure your query in PHP looks like this:
$code = $_GET['code'];
$result = mysqli_query($con,"SELECT * FROM countries WHERE code = '$code'");
Then your URI should look like this:
String url_get_country = "http://yourHostName/nameOfPhPFile.php?code=" + code
You can pass code as a parameter in method. If this doesn't work, you can change the SQL query to ...
... where code like '$code'

Spring named "IN" query producing error when given more than one entry

I'm using Spring's named queries to access my data and came upon an issue.
I have a rather long named query in my spring data repo:
List<LandscapeLocationEntity> findByIdCustomerIdAndIdProductGroupAndIdProductIdAndIdLocationIdInAndActiveFlag(String customerId, String ProductGroup, String productId, List<Integer> locationId, boolean activeFlag);
The query works perfectly fine as long as I provide a List<Integer> with only one entry. As soon as there is another entry it will throw a java.sql.SQLException: Borrow prepareStatement from pool failed.
As you can see in the screenshot, the call is different from the actual query (LOCATIONID(?,?) vs. LOCATIONID(?)).
I have a workaround which just executes the queries separately but that's not the way I want to have it in the long term.
If you need further information please tell me.
UPDATE:
To prove my point I removed all the other attributes and still get the same error:
Query is now: List<LandscapeLocationEntity> findByIdLocationIdIn(List<Integer> locations);

Failed to make bulk upsert using mongo

I'm trying to do upsert using mongodb driver, here is a code:
BulkWriteOperation builder = coll.initializeUnorderedBulkOperation();
DBObject toDBObject;
for (T entity : entities) {
toDBObject = morphia.toDBObject(entity);
builder.find(toDBObject).upsert().replaceOne(toDBObject);
}
BulkWriteResult result = builder.execute();
where "entity" is morphia object. When I'm running the code first time (there are no entities in the DB, so all of the queries should be insert) it works fine and I see the entities in the database with generated _id field. Second run I'm changing some fields and trying to save changed entities and then I receive the folowing error from mongo:
E11000 duplicate key error collection: statistics.counters index: _id_ dup key: { : ObjectId('56adfbf43d801b870e63be29') }
what I forgot to configure in my example?
I don't know the structure of dbObject, but that bulk Upsert needs a valid query in order to work.
Let's say, for example, that you have a unique (_id) property called "id". A valid query would look like:
builder.find({id: toDBObject.id}).upsert().replaceOne(toDBObject);
This way, the engine can (a) find an object to update and then (b) update it (or, insert if the object wasn't found). Of course, you need the Java syntax for find, but same rule applies: make sure your .find will find something, then do an update.
I believe (just a guess) that the way it's written now will find "all" docs and try to update the first one ... but the behavior you are describing suggests it's finding "no doc" and attempting an insert.

How to get last status message of each record using SQL?

Consider the following scenario.
I have a Java application which uses Oracle database to store some status codes and messages.
Ex: I have patient record that process in 3 layers (assume 1. Receiving class 2. Translation class 3. Sending class). We store data into database in each layer. When we run query it will show like this.
Name Status Status_Message
XYZ 11 XML message received
XYZ 21 XML message translated to swift format
XYZ 31 Completed message send to destination
ABC 11 XML message received
ABC 21 XML message translated to swift format
ABC 91 Failed message send to destination
On Java class I am executing the below query to get the last status message.
select STATUS_MESSAGE from INTERFACE_MESSAGE_STATUS
where NAME = ? order by STATUS
I publish this status message on a webpage. But my problem is I am not getting the last status message; it's behaving differently. It is printing sometimes "XML message received", sometimes "XML message translated to swift format", etc.
But I want to publish the last status like "Completed message send to destination" or "Failed message send to destination" depending on the last status. How can I do that? Please suggest.
You can use a query like this:
select
i.STATUS_MESSAGE
from
INTERFACE_MESSAGE_STATUS i,
(select
max(status) as last_status
from
INTERFACE_MESSAGE_STATUS
where
name = ?) s
where
i.name = ? and
i.status = s.last_status
In the above example I am assuming the status with the highest status is the last status.
I would recommend you to create a view out of this select query and then use that in your codebase. The reason is that it is much easier to read and makes it possible to easily select on multiple last_statuses without complicating your queries too much.
You have no explicit ordering specified. As the data is stored in a HEAP in Oracle, there is no specific order given. In other words: many factors influence the element you get. Only explicit ORDER BY guarantees desired order. And/or creating a index on some of the rows.
My suggestion: add a date_created field to your DB and sort based on that.

Categories

Resources