Develop Reference

Is it possible to transfer a folder from GCS to Bigquery using Java API

When I tried to give the source URI of a folder inside my bucket (which has around 400 CSV files) in the Java program, it has not moved any files to BQ table. If I try with a single csv file , it moves.
package com.example.bigquerydatatransfer;
import com.google.api.gax.rpc.ApiException;
import com.google.cloud.bigquery.datatransfer.v1.CreateTransferConfigRequest;
import com.google.cloud.bigquery.datatransfer.v1.DataTransferServiceClient;
import com.google.cloud.bigquery.datatransfer.v1.ProjectName;
import com.google.cloud.bigquery.datatransfer.v1.TransferConfig;
import com.google.protobuf.Struct;
import com.google.protobuf.Value;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
// Sample to create google cloud storage transfer config
public class Cloud_to_BQ {
public static void main(String[] args) throws IOException {
final String projectId = "dfp-bq";
String datasetId = "mytest1";
String tableId = "PROG_DATA";
String sourceUri = "gs://dfp-bq/C:\\PROG_Reports";
String fileFormat = "CSV";
String fieldDelimiter = ",";
String skipLeadingRows = "1";
Map<String, Value> params = new HashMap<>();
params.put(
"destination_table_name_template", Value.newBuilder().setStringValue(tableId).build());
params.put("data_path_template", Value.newBuilder().setStringValue(sourceUri).build());
params.put("write_disposition", Value.newBuilder().setStringValue("APPEND").build());
params.put("file_format", Value.newBuilder().setStringValue(fileFormat).build());
params.put("field_delimiter", Value.newBuilder().setStringValue(fieldDelimiter).build());
params.put("skip_leading_rows", Value.newBuilder().setStringValue(skipLeadingRows).build());
TransferConfig transferConfig =
TransferConfig.newBuilder()
.setDestinationDatasetId(datasetId)
.setDisplayName("Trial_Run_PROG_DataTransfer")
.setDataSourceId("google_cloud_storage")
.setParams(Struct.newBuilder().putAllFields(params).build())
.setSchedule("every 24 hours")
.build();
createCloudStorageTransfer(projectId, transferConfig);
}
public static void createCloudStorageTransfer(String projectId, TransferConfig transferConfig)
throws IOException {
try (DataTransferServiceClient client = DataTransferServiceClient.create()) {
ProjectName parent = ProjectName.of(projectId);
CreateTransferConfigRequest request =
CreateTransferConfigRequest.newBuilder()
.setParent(parent.toString())
.setTransferConfig(transferConfig)
.build();
TransferConfig config = client.createTransferConfig(request);
System.out.println("Cloud storage transfer created successfully :" + config.getName());
} catch (ApiException ex) {
System.out.print("Cloud storage transfer was not created." + ex.toString());
}
}
}
Is there any way I can move all the files to the BQ table at a stretch?
2022-08-04T07:27:50.185847509ZNo files found matching: "gs://dfp-bq/C:\PROG_Reports" - This is the BQ logs for the Run.

Getting Exception In onGuildMessageReceivedEvent - Discord JDA

I am making a Discord JDA bot that can when a user sends the message: Prefix("$") + hastebin + their code, the bot will create a request to hastebin and paste their code, after that he will take the paste URL and print it to the console(I will send it as a message after I solve the problem).
This is my HastebinCommand class:
package events;
import main.Hastebin;
import Info.Info;
import net.dv8tion.jda.core.events.message.guild.GuildMessageReceivedEvent;
import net.dv8tion.jda.core.hooks.ListenerAdapter;
import java.lang.*;
public class HastebinCommand extends ListenerAdapter
{
Info info;
Hastebin hastebin;
#Override
public void onGuildMessageReceived(GuildMessageReceivedEvent event)
{
String[] message = event.getMessage().getContentRaw().split(" ");
if (message[0].equalsIgnoreCase(info.prefix + "hastebin") || message[0].equalsIgnoreCase(info.prefix + "haste"))
{
if (message.length == 1)
{
//Send an error message
}
else
{
String code = "";
for (int i = 1; i < message.length; i++)
{
code = code + "" + message[i];
}
System.out.println(hastebin.paste(code));
}
}
}
}
This is my Hastebin request class:
package main;
import org.json.JSONObject;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
public class Hastebin
{
public static String paste(String content) throws Exception{
final HttpClient client = HttpClient.newHttpClient();
final HttpRequest request = HttpRequest.newBuilder(URI.create("https://hastebin.com/documents"))
.POST(HttpRequest.BodyPublishers.ofString(content)).build();
final HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
final String responseContent = response.body();
final JSONObject responseJson = new JSONObject(responseContent);
final String key = responseJson.getString("key");
return "https://hastebin.com/" + key;
}
}
My error:
C:\Users\user\Documents\Java\Java Projects\DiscordJDA\SpoonfeedingBot\src\events\HastebinCommand.java
Error:(32, 50) java: unreported exception java.lang.Exception; must be caught or declared to be thrown
I would really appreciate getting help after trying to solve my problem for so long.

Some function you're using is throwing an exception that must be dealt with (checked exception). Whenever you have a problem with an exception you should google the exception first. For example, in this case, you could paste "unreported exception java.lang.Exception; must be caught or declared to be thrown" in google search engine and you'd get your explanation.
Here's a tutorial on Exceptions: https://www.tutorialspoint.com/java/java_exceptions.htm
Also, your StackOverflow post should be about the error itself, in other words "unreported exception java.lang.Exception".

AWS Email Template usage using java (bulk email)

Can some one give me a direction how can I implement this aws email template tutorial by a java code? Through java code I want to set this AWS Email Template and through java only I want to set the parameter values to the template and through java only I want to send the email.
I cant find any tutorial or direction from which I can translate above requests in java code.

The "code" in your link is actually just some JSON templates for sending and formatting email, and a few calls to an AWS command line tool. If you need to make AWS send email calls from a Java process then you need to take a look at:
The SES API
The Javadoc for the Java client lib

I am able to code it successfully. Pasting the example code here.
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Calendar;
import java.util.List;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.simpleemail.AmazonSimpleEmailService;
import com.amazonaws.services.simpleemail.AmazonSimpleEmailServiceClientBuilder;
import com.amazonaws.services.simpleemail.model.BulkEmailDestination;
import com.amazonaws.services.simpleemail.model.BulkEmailDestinationStatus;
import com.amazonaws.services.simpleemail.model.Destination;
import com.amazonaws.services.simpleemail.model.SendBulkTemplatedEmailRequest;
import com.amazonaws.services.simpleemail.model.SendBulkTemplatedEmailResult;
public class AmazonSESSample2 {
public static void main(String[] args) throws IOException {
String accessKeyId = "accessKeyId";
String secretKeyId = "secretKeyId";
String region = "us-east-1";
List<BulkEmailDestination> listBulkEmailDestination = null;
SendBulkTemplatedEmailRequest sendBulkTemplatedEmailRequest = null;
try {
AmazonSimpleEmailService client = getAmazonSESClient(accessKeyId, secretKeyId, region);
listBulkEmailDestination = new ArrayList<>();
for(String email : getRecievers()) {
String replacementData="{"
+ "\"FULL_NAME\":\"AAA BBB\","
+ "\"USERNAME\":\""+email+"\","
+ "}";
BulkEmailDestination bulkEmailDestination = new BulkEmailDestination();
bulkEmailDestination.setDestination(new Destination(Arrays.asList(email)));
bulkEmailDestination.setReplacementTemplateData(replacementData);
listBulkEmailDestination.add(bulkEmailDestination);
}
sendBulkTemplatedEmailRequest = new SendBulkTemplatedEmailRequest();
sendBulkTemplatedEmailRequest.setSource("noreply#mydomain.com");
sendBulkTemplatedEmailRequest.setTemplate("welcome-email-en_GB-v1");
sendBulkTemplatedEmailRequest.setDefaultTemplateData("{\"FULL_NAME\":\"friend\", \"USERNAME\":\"unknown\"}");
sendBulkTemplatedEmailRequest.setDestinations(listBulkEmailDestination);
SendBulkTemplatedEmailResult res = client.sendBulkTemplatedEmail(sendBulkTemplatedEmailRequest);
System.out.println("======================================");
System.out.println(res.getSdkResponseMetadata());
System.out.println("======================================");
for(BulkEmailDestinationStatus status : res.getStatus()) {
System.out.println(status.getStatus());
System.out.println(status.getError());
System.out.println(status.getMessageId());
}
} catch (Exception ex) {
System.out.println("The email was not sent. Error message: " + ex.getMessage());
ex.printStackTrace();
}
}
public static List<String> getRecievers() {
ArrayList<String> list = new ArrayList<>();
list.add("aaa+1#gmail.com");
list.add("aaa+2#gmail.com");
list.add("aaa+3#gmail.com");
list.add("aaa+4#gmail.com");
return list;
}
public static AmazonSimpleEmailService getAmazonSESClient(String accessKeyId, String secretKeyId, String region) {
BasicAWSCredentials awsCreds = new BasicAWSCredentials(accessKeyId, secretKeyId);
AmazonSimpleEmailService client = AmazonSimpleEmailServiceClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(awsCreds))
.withRegion(region)
.build();
return client;
}
}

How to get tweet updates in XML format using java

I have following code that outputs my and my users twitter time line messages in java.
I followed this tutorial to get the code below
http://namingexception.wordpress.com/2011/09/12/how-easy-to-make-your-own-twitter-client-using-java/
import java.io.IOException;
import java.util.List;
import twitter4j.Status;
import twitter4j.Twitter;
import twitter4j.TwitterException;
import twitter4j.TwitterFactory;
import twitter4j.auth.AccessToken;
public class SimpleTweet {
List<Status> statuses;
private final static String CONSUMER_KEY = "XXXXXX";
private final static String CONSUMER_KEY_SECRET = "XXXXXXX-123";
public void start() throws TwitterException, IOException {
Twitter twitter = new TwitterFactory().getInstance();
twitter.setOAuthConsumer(CONSUMER_KEY, CONSUMER_KEY_SECRET);
String accessToken = getSavedAccessToken();
String accessTokenSecret = getSavedAccessTokenSecret();
AccessToken oathAccessToken = new AccessToken(accessToken,accessTokenSecret);
twitter.setOAuthAccessToken(oathAccessToken);
twitter.updateStatus("Hello world :).");
statuses = twitter.getHomeTimeline();
for (Status each : statuses) {
System.out.println("Sent by: #" + each.getUser().getScreenName()
+ " - " + each.getUser().getName() + "\n" + each.getText()
+ "\n");
}
}// start method ends here
private String getSavedAccessTokenSecret() {
return "vxcvvxcvxcvx";
}
private String getSavedAccessToken() {
return "eweweqweqweqwe";
}
public static void main(String[] args) throws Exception {
new SimpleTweet().start();
}
}
And I get following output
Sent by: #tweetrr - rr
Hello to all :).
Sent by: #addthis - AddThis
Just in time for #wordcampnyc, we have updated the AddThis WordPress plugin! Check it:
http://t.co/cgOgRwyl
Now I want the output to be in XML format. I would like to know if there are API's that does this work. Thanks in advance

You can use betwixt from apache (Bitwix example), using which you can convert either bean or hashmap to XML format easily. So, you create bean called UserStatusBean with fields like sentBy, status, message etc, populate the bean and output as XML using [BeanWriter][2].

JGroups: send(null, null, Message)vs send(Address, null, Message)

I've written simple test for using JGroups.
There are two simple applications like this
import org.jgroups.*;
import org.jgroups.conf.ConfiguratorFactory;
import org.jgroups.conf.ProtocolConfiguration;
import org.jgroups.conf.ProtocolStackConfigurator;
import java.util.List;
/**
* #author Sergii.Zagriichuk
*/
public class Test {
public static void main(String[] args) throws Exception {
JChannel ch = new JChannel();
ch.setReceiver(new ReceiverAdapter() {
public void receive(Message msg) {
System.out.println("received message " + msg.getObject());
}
});
ch.connect("one");
}
}
and this
package com.datacradle.example;
import org.jgroups.Global;
import org.jgroups.JChannel;
import org.jgroups.Message;
import org.jgroups.conf.ConfiguratorFactory;
import org.jgroups.conf.ProtocolConfiguration;
import org.jgroups.conf.ProtocolStackConfigurator;
import org.jgroups.stack.IpAddress;
import org.jgroups.util.SingletonAddress;
import org.jgroups.util.Util;
import java.util.List;
/**
* #author Sergii.Zagriichuk
*/
public class Test {
void start(String props) throws Exception {
JChannel chanel = new JChannel();
String line = "Test message";
chanel.connect("one");
// Message msg = new Message(null, null, new TestData(line, 1111, line + " Test suffix"));
Message msg = new Message(new IpAddress("fe33:0:0:0:1986:ba23:d939:f226%12",55435) , null, new TestData(line,1111,line+" sdfasdfasdfasdfasdfa"));
chanel.send(msg);
}
public static void main(final String[] args) throws Exception {
new Test().start(null);
}
}
So, If I use this style for creating message
Message msg = new Message(null, null, new TestData(line, 1111, line + " Test suffix"));
I will receive just a one message(this is for all subscribers in current group),
but if I use this style
Message msg = new Message(new IpAddress("fe33:0:0:0:1986:ba23:d939:f226%12",55435) , null, new TestData(line,1111,line+" sdfasdfasdfasdfasdfa"));
I will receive a lot of messages like in a loop (this is for one dist address)
What is the problem or I should added some additional parameters?
P.S, JGroups 3.0.0 RC1
Thanks.

You should not create a member address using the IpAddress class, as this is something that's opaque. I suggest fetch the target address from a view, e.g.
List<Address> members=channel.getView().getMembers();
Address target=members.get(0);
Message msg=new Message(target, null, "hello");
channel.send(msg);