Develop Reference

VTD-XML reading gives no results

I am trying to read a RSS content using VTD-XML. Below is a sample of RSS.
<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:atom="http://www.w3.org/2005/Atom" version="2.0">
<?xml-stylesheet type="text/xsl" href="rss.xsl"?>
<channel>
<title>MyRSS</title>
<atom:link href="http://www.example.com/rss.php" rel="self" type="application/rss+xml" />
<link>http://www.example.com/rss.php</link>
<description>MyRSS</description>
<language>en-us</language>
<pubDate>Tue, 22 May 2018 13:15:15 +0530</pubDate>
<item>
<title>Title 1</title>
<pubDate>Tue, 22 May 2018 13:14:40 +0530</pubDate>
<link>http://www.example.com/news.php?nid=47610</link>
<guid>http://www.example.com/news.php?nid=47610</guid>
<description>bla bla bla</description>
</item>
</channel>
</rss>
Anyway as you know, some RSS feeds can contain more styling info etc. However in every RSS, the <channel> and <item> will be common, at least for the ones I need to use.
I tried VTD XML to read this as quickly as possible. Below is the code.
VTDGen vg = new VTDGen();
if (vg.parseHttpUrl(appDataBean.getUrl(), true)) {
VTDNav vn = vg.getNav();
AutoPilot ap = new AutoPilot(vn);
ap.selectXPath("/channel/item");
int result = -1;
while ((result = ap.evalXPath()) != -1) {
if (vn.matchElement("item")) {
do {
//do something with the contnets in the item node
Log.d("VTD", vn.toString(vn.getText()));
} while (vn.toElement(VTDNav.NEXT_SIBLING));
}
}
}
Unfortunately this did not print anything. What am I doing wrong here? Also non of the RSS feeds are very big, so I need to read them in couple of miliseconds. This code is on Android.

SoapUI: response with CDATA is giving null. searched various article but no luck

Hello I have below SOAP reponse.
`
<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Body>
<GetWeatherResponse xmlns="http://www.webserviceX.NET">
<GetWeatherResult><![CDATA[<?xml version="1.0" encoding="utf-16"?>
<CurrentWeather>
<Location>Cape Town, Cape Town International Airport, South Africa (FACT) 33-59S 018-36E 0M</Location>
<Time>Jun 04, 2016 - 05:00 AM EDT / 2016.06.04 0900 UTC</Time>
<Wind> from the SE (130 degrees) at 21 MPH (18 KT):0</Wind>
<Visibility> greater than 7 mile(s):0</Visibility>
<SkyConditions> mostly clear</SkyConditions>
<Temperature> 60 F (16 C)</Temperature>
<DewPoint> 44 F (7 C)</DewPoint>
<RelativeHumidity> 55%</RelativeHumidity>
<Pressure> 30.39 in. Hg (1029 hPa)</Pressure>
<Status>Success</Status>
</CurrentWeather>]]></GetWeatherResult>
</GetWeatherResponse>
</soap:Body>
</soap:Envelope>"`
for the following request: http://www.webservicex.net/globalweather.asmx.
I want to read XML Data in above response using script. but it is giving null.
I have tried script as below
def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
def holder = groovyUtils.getXmlHolder(messageExchange.responseContent)
holder.namespaces["ns"] = "http://www.webserviceX.NET/" def
weatherinfo= holder.getNodeValue("//ns:GetWeatherResult/text()")
log.info weatherinfo
bit Instead of getting above reponse I am getting NULL. I have read the SOAPUI documentation on CDATA but its not working.

I got answer. It was just one extra slash in namespace that was giving me null.
Both the below scripts are working now.
> def respXmlHolder = new
> com.eviware.soapui.support.XmlHolder(messageExchange.getResponseContentAsXml())
> respXmlHolder.namespaces["ns1"] ="http://www.webserviceX.NET" def
> CDATAXml= respXmlHolder.getNodeValue("//ns1:GetWeatherResult/text()")
> log.info CDATAXml
def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
def holder = groovyUtils.getXmlHolder(messageExchange.responseContent)
holder.namespaces["ns"] = "http://www.webserviceX.NET"
def weatherinfo= holder.getNodeValue("//ns:GetWeatherResult/text()")
log.info weatherinfo

OutputSream.write is too slow

I am encountering with a senerior like this:
My project has a servlet to catch a request from perl. The request is to download a file. The request is a multipartRequest.
#RequestMapping(value = "/*", method = RequestMethod.POST)
public void tdRequest(#RequestHeader("Authorization") String authenticate,
HttpServletResponse response,
HttpServletRequest request) throws Exception
{
if (ServletFileUpload.isMultipartContent(request))
{
ServletFileUpload sfu = new ServletFileUpload();
FileItemIterator items = sfu.getItemIterator(request);
while (items.hasNext())
{
FileItemStream item = items.next();
if (("action").equals(item.getFieldName()))
{
InputStream stream = item.openStream();
String value = Streams.asString(stream);
if (("upload").equals(value))
{
uploadRequest(items, response);
return;
}
else if (("download").equals(value))
{
downloadRequest(items, response);
return;
}
The problem is not here, it appears on the downloadRequest() function.
void downloadRequest(FileItemIterator items,
HttpServletResponse response) throws Exception
{
log.info("Start downloadRequest.......");
OutputStream os = response.getOutputStream();
File file = new File("D:\\clip.mp4");
FileInputStream fileIn = new FileInputStream(file);
//while ((datablock = dataOutputStreamServiceImpl.readBlock()) != null)
byte[] outputByte = new byte[ONE_MEGABYE];
while (fileIn.read(outputByte) != -1)
{
System.out.println("--------" + (i = i + 1) + "--------");
System.out.println(new Date());
//dataContent = datablock.getContent();
System.out.println("Start write " + new Date());
os.write(outputByte, 0,outputByte.length);
System.out.println("End write " + new Date());
//System.out.println("----------------------");
}
os.close();
}
}
I try to read and write blocks of 1MB from the file. However, it takes too long for downloading the whole file. ( my case is 20mins for file of 100MB)
I try to sysout and I saw a result like this:
The first few blocks can read, write data realy fast:
--------1--------
Mon Dec 07 16:24:20 ICT 2015
Start write Mon Dec 07 16:24:20 ICT 2015
End write Mon Dec 07 16:24:21 ICT 2015
--------2--------
Mon Dec 07 16:24:21 ICT 2015
Start write Mon Dec 07 16:24:21 ICT 2015
End write Mon Dec 07 16:24:21 ICT 2015
--------3--------
Mon Dec 07 16:24:21 ICT 2015
Start write Mon Dec 07 16:24:21 ICT 2015
End write Mon Dec 07 16:24:21 ICT 2015
But the next block is slower than the previous
--------72--------
Mon Dec 07 16:29:22 ICT 2015
Start write Mon Dec 07 16:29:22 ICT 2015
End write Mon Dec 07 16:29:29 ICT 2015
--------73--------
Mon Dec 07 16:29:29 ICT 2015
Start write Mon Dec 07 16:29:29 ICT 2015
End write Mon Dec 07 16:29:37 ICT 2015
--------124--------
Mon Dec 07 16:38:22 ICT 2015
Start write Mon Dec 07 16:38:22 ICT 2015
End write Mon Dec 07 16:38:35 ICT 2015
--------125--------
Mon Dec 07 16:38:35 ICT 2015
Start write Mon Dec 07 16:38:35 ICT 2015
End write Mon Dec 07 16:38:48 ICT 2015
The problem is in the os.write()
I realy cannot understand how the outputStream write, why it take such a long time like that? or I made some mistakes?
Sorry for my bad english. I realy need your support. Thank in advance!
This is the perl code from the client side
# ----- get connected to download the file
#
$Response = $ua->request(POST $remoteHost ,
Content_Type => 'form-data',
Authorization => $Authorization,
'Proxy-Authorization' => $Proxy_Authorization ,
Content => [ DOS => 1 ,
action => 'download' ,
first_run => 0 ,
dl_filename => $dl_filename ,
delivery_dir => $delivery_dir ,
verbose => $Verbose ,
debug => $debug ,
version => $VERSION
]
);
unless ($Response->is_success) {
my $Msg = $Response->error_as_HTML;
# Remove HTML tags - we're in a DOS shell!
$Msg =~ s/<[^>]+>//g;
print "ERROR! SERVER RESPONSE:\n$Msg\n";
print "$remoteHost\n\n" if $Options{'v'};
Error "Could not connect to " . $remoteHost ;
}
my $Result2 = $Response->content();
Error "Abnormal termination...\n$Result2" if $Result2 =~ /_APP_ERROR_/;
open(F, ">$dl_filename") or Error "Could not open '$dl_filename'!";
binmode F; # unless $dl_filename =~ /\.txt$|\.htm$/;
print F $Result2;
close F;
print "received.\n";
}

One problem is that fileIn.read(outputByte) can read random number of bytes, not only full outputByte. You read few KB, then you store full 1MB, and very fast you are running out of space on disk. Try this, notice the "readed" parameter.
void downloadRequest(FileItemIterator items,
HttpServletResponse response) throws Exception
{
log.info("Start downloadRequest.......");
OutputStream os = response.getOutputStream();
File file = new File("D:\\clip.mp4");
FileInputStream fileIn = new FileInputStream(file);
//while ((datablock = dataOutputStreamServiceImpl.readBlock()) != null)
byte[] outputByte = new byte[ONE_MEGABYE];
int readed =0;
while ((readed =fileIn.read(outputByte)) != -1)
{
System.out.println("--------" + (i = i + 1) + "--------");
System.out.println(new Date());
//dataContent = datablock.getContent();
System.out.println("Start write " + new Date());
os.write(outputByte, 0,readed );
System.out.println("End write " + new Date());
//System.out.println("----------------------");
}
os.close();
}
}

It looks like your download performance gets slower and slower, the further you are getting into the download. You start out at one or less seconds per block, by block 72 it is 7+ seconds per block and by block 128 it is 13 seconds per block.
There is nothing on the server side to explain this. Rather, it has the "smell" of the client side doing something wrong. My guess is that the client side is reading the data from the socket into an in-memory data structure, and that data structure (maybe just a String or StringBuffer or StringBuilder) is getting larger and larger. Either the time take to expand it is getting larger, or your memory footprint is growing and the GC is taking longer and longer. (Or both.)
If you showed us the client-side code .....
UPDATE
As I suspected, this line of code will be reading the entire content into the Perl equivalent of a string builder before turning it into a string.
my $Result2 = $Response->content();
Depending on how it is implemented under the hood, this will lead to repeated copying of the data as the builder runs out of buffer space and needs to be expanded. Depending on the buffer expansion strategy that Perl employs for this, it could give O(N^2) behavior, where N is the size of the file you are transferring. (The evidence is that you are not getting O(N) behavior ...)
If you want a faster downloads, you need to stream the data on the client side. Read the response content in chunks and write them to the output file. (I'm not a Perl expert, so I can't offer you code.) This will also reduce the memory footprint on the client side ... which could be important if your file sizes increase.

Twitter4j authentication credentials are missing

I would like to make a tweet with Twitter4j in my Android app. Here is my code:
//TWITTER SHARE.
#Click (R.id. img_btn_twitter)
#Background
public void twitterPostWall(){
try {
//Twitter Conf.
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setDebugEnabled(true)
.setOAuthConsumerKey(CONSUMER_KEY)
.setOAuthConsumerSecret(CONSUMER_SECRET)
.setOAuthAccessToken(ACCESS_KEY)
.setOAuthAccessTokenSecret(ACCESS_SECRET);
TwitterFactory tf = new TwitterFactory(cb.build());
Twitter twitter = new TwitterFactory().getInstance();
twitter.setOAuthConsumer(CONSUMER_KEY, CONSUMER_SECRET);
try {
RequestToken requestToken = twitter.getOAuthRequestToken();
Log.e("Request token: ", "" + requestToken.getToken());
Log.e("Request token secret: ", "" + requestToken.getTokenSecret());
AccessToken accessToken = null;
}
catch (IllegalStateException ie) {
if (!twitter.getAuthorization().isEnabled()) {
Log.e("OAuth consumer key/secret is not set.", "");
}
}
Status status = twitter.updateStatus(postLink);
Log.e("Successfully updated the status to [", "" + status.getText() + "].");
}
catch (TwitterException te) {
Log.e("TWEET FAILED", "");
}
}
I always get this error message from Twitter4j: java.lang.IllegalStateException: Authentication credentials are missing. See http://twitter4j.org/en/configuration.html for the detail. But as you can see I'm using builder to set my key. Can someone help me to fix it please? thanks.

Problem is following lines.
TwitterFactory tf = new TwitterFactory(cb.build());
Twitter twitter = new TwitterFactory().getInstance();
You are passing the configuration to one TwitterFactory instance and using another TwitterFactory instance to get the Twitter instance.
Hence, You are getting
java.lang.IllegalStateException: Authentication credentials are missing
I suggest you to modify your code as follows:
//Twitter Conf.
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setDebugEnabled(true)
.setOAuthConsumerKey(CONSUMER_KEY)
.setOAuthConsumerSecret(CONSUMER_SECRET)
.setOAuthAccessToken(ACCESS_KEY)
.setOAuthAccessTokenSecret(ACCESS_SECRET);
TwitterFactory tf = new TwitterFactory(cb.build());
Twitter twitter = tf.getInstance();
And use this twitter instance. It will work.

I was having issues with the configuration on Twitter4j because I was not providing the right configuration. So in order to fix it, I created the following function to establish my configuration to later be used in another function:
public static void main(String args[]) throws Exception {
TwitterServiceImpl impl = new TwitterServiceImpl();
ResponseList<Status> resList = impl.getUserTimeLine("spacex");
for (Status status : resList) {
System.out.println(status.getCreatedAt() + ": " + status.getText());
}
}
public ResponseList<Status> getUserTimeLine(String screenName) throws TwitterException {
TwitterFactory twitterFactory = new TwitterFactory(getConfiguration().build());
Twitter twitter = twitterFactory.getInstance();
twitter.getAuthorization();
Paging paging = new Paging(1, 10);
twitter.getId();
return twitter.getUserTimeline(screenName, paging);
}
public ConfigurationBuilder getConfiguration() {
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setDebugEnabled(true)
.setOAuthConsumerKey("myConsumerKey")
.setOAuthConsumerSecret("myConsumerSecret")
.setOAuthAccessToken("myAccessToken")
.setOAuthAccessTokenSecret("myAccessTokenSecret");
return cb;
}
To get the required info, you must have a Twitter developer account, and to get the auth info of an app previously created go to: Projects and Apps.
In the end, I was able to retrieve the data from a SpaceX account:
Tue Nov 24 20:58:13 CST 2020: Falcon 9 launches Starlink to orbit – the seventh launch and landing of this booster https://twitter.com/SpaceX/status/1331431972430700545
Tue Nov 24 20:29:36 CST 2020: Deployment of 60 Starlink satellites confirmed https://twitter.com/SpaceX/status/1331424769632215040
Tue Nov 24 20:23:17 CST 2020: Falcon 9’s first stage lands on the Of Course I Still Love You droneship! https://twitter.com/SpaceX/status/1331423180431396864
Tue Nov 24 20:14:20 CST 2020: Liftoff! https://twitter.com/SpaceX/status/1331420926450094080
Tue Nov 24 20:02:38 CST 2020: Watch Falcon 9 launch 60 Starlink satellites ? https://www.spacex.com/launches/index.html https://twitter.com/i/broadcasts/1ypKdgVXWgRxW
Tue Nov 24 19:43:14 CST 2020: T-30 minutes until Falcon 9 launches its sixteenth Starlink mission. Webcast goes live ~15 minutes before liftoff https://www.spacex.com/launches/index.html
Tue Nov 24 18:00:59 CST 2020: RT #elonmusk: Good Starship SN8 static fire! Aiming for first 15km / ~50k ft altitude flight next week. Goals are to test 3 engine ascent,…
Mon Nov 23 15:45:38 CST 2020: Now targeting Tuesday, November 24 at 9:13 p.m. EST for Falcon 9’s launch of Starlink, when weather conditions in the recovery area should improve
Sun Nov 22 20:45:13 CST 2020: Standing down from today’s launch of Starlink. Rocket and payload are healthy; teams will use additional time to complete data reviews and are now working toward backup opportunity on Monday, November 23 at 9:34 p.m. but keeping an eye on recovery weather
Sat Nov 21 22:09:12 CST 2020: More Falcon 9 launch and landing photos ? https://www.flickr.com/photos/spacex https://twitter.com/SpaceX/status/1330362669837082624
Where to get Auth Tokens for your app

Xpath functionality

I have one module which consumes a webservice and fetches a response from it. The response which I am getting contains a value inside it on which I need to set it value. I am using XPath for that. Now when I am running this functionality on my local system I am getting the desired output, but when I am using that in my ear, it's not working.
The code in ear:
private static String stResponsePath="//Envelope//Body//AddNoteResponse//Response//";
private static String Fault_Path="//Envelope/Body/Fault/detail/CSIApplicationException/ServiceProviderEntity/";
/**
* Construct AddNoteResponse bean from
*
* #param addNote response SOAP XML
*/
public AddNoteResponse createAddNoteResponseFromSoapString(String soapString)throws EDDSystemException
{
try
{
Document soapDocument =AddNoteDomParserHelper.convertStringToDoc(soapString);
// EDDLog.outLog.info("soapDocument " + soapDocument.getFirstChild());
// EDDLog.outLog.info("soapDocument " + soapDocument.getLastChild());
EDDLog.outLog.info("soapString :" +soapString );
AddNoteResponse addNoteResponse= new AddNoteResponse();
addNoteResponse.setCode(AddNoteDomParserHelper.**getElementValue**(stResponsePath + "code", soapDocument));
addNoteResponse.setDescription(AddNoteDomParserHelper.**getElementValue**(stResponsePath + "description", soapDocument));
} .....
.....
}
public static String **getElementValue**(String elementName, Document doc) throws Exception
{
String value = "";
XPath xPath = XPathFactory.newInstance().newXPath();
if(xPath ==null)
{
EDDLog.outLog.info(" xpath is null");
}
EDDLog.outLog.info("doc " + doc.getFirstChild());
NodeList docNodes = (NodeList) xPath.evaluate(elementName,doc, XPathConstants.NODESET);
// NodeList docNodes = (NodeList) xPath.evaluate("/Envelope/Body/AddNoteResponse/Response/code",doc, XPathConstants.NODESET);
EDDLog.outLog.info("elementName :" +elementName );
//NodeList docNodes = doc.getElementsByTagNameNS("*", elementName);
if (docNodes == null || docNodes.getLength() == 0) {
docNodes = doc.getElementsByTagName(elementName);
EDDLog.outLog.info("docNodes.getLength() :"+docNodes.getLength());
}
Output of this is:
11 Jun 2013 13:47:40,644 INFO soapString :<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Header> <MessageHeader xmlns:cng="http://csi.cingular.com/CSI/Nam
espaces/Types/Public/CingularDataModel.xsd" xmlns="http://csi.cingular.com/CSI/Namespaces/Types/Public/MessageHeader.xsd"><TrackingMessageHeader><cng:version>v68</cng:version><cng:originalVersion>v
68</cng:originalVersion><cng:messageId>46178</cng:messageId><cng:timeToLive>120000</cng:timeToLive><cng:conversationId>EDD~CNG-CSI~5bfa2f5f-103d-4056-b775-cb2028357a5c</cng:conversationId><cng:date
TimeStamp>2013-06-08T12:44:18.418Z</cng:dateTimeStamp><cng:uniqueTransactionId>ServiceGateway19121#q25csg1c1_4f771d65-41de-4c40-b4cc-0be179d3677c</cng:uniqueTransactionId></TrackingMessageHeader><S
ecurityMessageHeader><cng:userName>EDD</cng:userName><cng:userPassword>EDDtest</cng:userPassword></SecurityMessageHeader><SequenceMessageHeader><cng:sequenceNumber>1</cng:sequenceNumber><cng:totalInSequence>1</cng:totalInSequence></SequenceMessageHeader></MessageHeader></SOAP-ENV:Header> <SOAP-ENV:Body> <AddNoteResponse xmlns:cng="http://csi.cingular.com/CSI/Namespaces/Types/Public/CingularDataModel.xsd" xmlns="http://csi.cingular.com/CSI/Namespaces/Container/Public/AddNoteResponse.xsd"><Response><cng:code>0</cng:code><cng:description>Success</cng:description></Response></AddNoteResponse></SOAP-ENV:Body> </SOAP-ENV:Envelope>
11 Jun 2013 13:47:42,028 INFO doc [SOAP-ENV:Envelope: null]
11 Jun 2013 13:47:44,236 INFO elementName ://Envelope//Body//AddNoteResponse//Response//code
11 Jun 2013 13:47:44,236 INFO docNodes.getLength() :0
11 Jun 2013 13:47:44,243 INFO doc [SOAP-ENV:Envelope: null]
11 Jun 2013 13:47:44,254 INFO elementName ://Envelope//Body//AddNoteResponse//Response//description
11 Jun 2013 13:47:44,254 INFO docNodes.getLength() :0
But when I am trying the same code on my local system it is working fine:
Entering .convertStringToDoc()...
Exiting .convertStringToDoc()...
soapDocument [SOAP-ENV:Envelope: null]
9
xpath is not null
elementName :/Envelope/Body/AddNoteResponse/Response/code
docNodes length:1
value :0
docNodes length2:1
addNoteResponse.getCode
xpath is not null
elementName :/Envelope/Body/AddNoteResponse/Response/description
docNodes length:1
value :Success
docNodes length2:1
This is the response on which I am working with:
<SOAP-ENV:Envelope >
<SOAP-ENV:Header>
<MessageHeader >
<TrackingMessageHeader>
<cng:version>v68</cng:version>
<cng:originalVersion>v68</cng:originalVersion>
<cng:messageId>45266</cng:messageId>
<cng:timeToLive>120000</cng:timeToLive>
<cng:conversationId>xxx</cng:conversationId>
<cng:dateTimeStamp>2013-06-05T10:01:41.09Z</cng:dateTimeStamp>
<cng:uniqueTransactionId>xasas</cng:uniqueTransactionId>
</TrackingMessageHeader>
<SecurityMessageHeader>
<cng:userName>ass</cng:userName>
<cng:userPassword>asas</cng:userPassword>
</SecurityMessageHeader>
<SequenceMessageHeader>
<cng:sequenceNumber>1</cng:sequenceNumber>
<cng:totalInSequence>1</cng:totalInSequence>
</SequenceMessageHeader>
</MessageHeader>
</SOAP-ENV:Header>
<SOAP-ENV:Body>
<AddNoteResponse >
<Response>
<cng:code>0</cng:code>
<cng:description>Success</cng:description>
</Response>
</AddNoteResponse>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
Value I am interested in is:
<cng:code>0</cng:code>
<cng:description>Success</cng:description>