Hi guys i just started doing pig, I was wondering if JsonLoader is capable of parsing all value inside json.
for example:
{"food":"Tacos", "person":"Alice", "amount":3}
and i need to get "food" stored as a relation in chararray and "Tacos" which is the value of "food" to another relation.
after reading many tutorial and documentation, i havent found a built in method to do so.
does it mean that the only solution to this is through UDF?
thanks a lot!
I found the answer that is to use external jar from twitter.
register 'hdfs:/udf/elephant-bird-pig-4.10.jar';
register 'hdfs:/udf/elephant-bird-core-4.10.jar';
register 'hdfs:/udf/elephant-bird-hadoop-compat-4.10.jar';
register 'hdfs:/udf/json-simple-1.1.1.jar';
test.json
{"food":"Tacos", "person":"Alice", "amount":3}
script:
A = LOAD 'hdfs:/test.json' USING com.twitter.elephantbird.pig.load.JsonLoader('-nestedLoad') AS (json:map[]);
DUMP A;
and the output that i wanted is:
([amount#3,food#Tacos,person#Alice])
Thanks!
input- pig.json
{"food":"Tacos", "person":"Alice", "amount":3}
script
A = LOAD '/home/kishore/Data/Pig/pig.json' USING JsonLoader('food:chararray,person:chararray,amount:int');
B = foreach A generate food,person,amount;
Dump B;
output
(Tacos,Alice,3)
Related
i am trying to parse the following json file in java with gson:
So , I am trying to retrieve only the issues into a list of Issue. I have the following class to store the issue information:
But i got an error.. I am trying to get with the following but I think that i am so lost with this ( i am learning about this ) and i dont understand the error
any idea of how to do this? thank you so much!
obj.get("issues")
will actually return a List Object, and not an Issue Object. You need to change the second parameter accordingly.
I have used "consistency" parameter in python program to put data to elasticsearch successfully ,but i can not find a usage of "consistency" in java api. can you help me?
You can write:
client.prepareIndex("index", "doc", "doc1")
.setSource("doc")
.setConsistencyLevel(WriteConsistencyLevel.QUORUM)
.execute()
.actionGet()
Read below links :
https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking_50_java_api_changes.html#_writeconsistencylevel_removed_on_write_requests
https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-index_.html#index-wait-for-active-shards
I think you can use setWaitForActiveShards, which can take a numerical value up to the total number of shard copies or ActiveShardCount.ALL for all shard copies.
I have a json file with data for all the tiles in my game that I store in the assets folder. I try to access and parse it using TileList dataList = json.fromJson(TileList.class, Gdx.file.internal("map-elements/tiles/tiles.json")). This works fine for the desktop version but on the html version, after converting with gwt, I get these errors:
GwtApplication: exception: Error reading file: map-elements/tiles/tiles.json
Error reading file: map-elements/tiles/tiles.json
Couldn't find Type for class 'net.vediogames.archipelo.world.tiles.TileList'
TileList is a simple object that contains an array of TileData which can then be converted into Tile objects. I did it this way to make the json parsing easy.
The solution to the json error is simple. Instead of passing the FileHandle into the json parser, pass the string from the file like this:
TileList dataList = json.fromJson(TileList.class, Gdx.file.internal("map-elements/tiles/tiles.json").readString());
In the end, all I needed to do to solve that issue is add .readString(). As for the Couldn't find Type for class 'net.vediogames.archipelo.world.tiles.TileList' error, I also found a solution but it is more complicated.
JavaScript handles class references differently than Java. So I was not able to use TileList.class without first registering it so LibGDX can generate a Reflection. What I needed to do was add this line into my *.gwt.xml files:
<extend-configuration-property name="gdx.reflect.include" value="net.vediogames.archipelo.world.tiles.TileList" />
If you want a full tutorial about how reflection works and how to include packages or exclude, please view the official LibGDX tutorial here.
Your solution was not working for me, I still got the same error. After some hours of testing I got it to work, using your suggestions and by using the ClassReflection instead of referencing the class itself.
Your example:
TileList dataList = json.fromJson(TileList.class, Gdx.file.internal("map-elements/tiles/tiles.json").readString());
looks in my working code like:
TileList dataList = (TileList) json.fromJson(ClassReflection.forName(TileList.class.getName()), Gdx.file.internal("map-elements/tiles/tiles.json").readString());
This is quite a pain in the a.. but I'm glad it is finally working now.
I am newbie for cloudant and trying to learn full text search on cloudant through tutorial video. I am successful in searching on the cloudant.com through http request ,Now I want that code to be in java as i am working on GWT framework with java. Till now , I am just able to create connection with cloudant.com by studying the particular github project GITHUBLINK
There its given for search like this
Search search = db.search("views101/animals");
SearchResult<Animal> rslt = search
.limit(10)
.includeDocs(true)
.counts(new String[] {"class","diet"})
.querySearchResult("l*", Animal.class);
My Questions:
1. what exactly is this Animal.class refer to?
2. If this not the way what are the steps for full text search on cloudant.
I have created the view and the search index manually on cloudant.com under a designdoc of a database.
Animal.class refers to the class that the documents found will be deserialised into, if you do not have a class to deserialise the data into, you should be able to a HashMap or similar class to access the data returned.
You can find the example source code for the Animal class in this location: https://github.com/cloudant/java-cloudant/blob/88202a1bd7b9b04d96c4b7b8498a1b8f7f99c9e5/src/test/java/com/cloudant/tests/Animal.java.
Like the previous answer indicated, you could also have the results returned as a generic JsonObject that provides access to the properties and values.
I'm very new in using web services. Appreciate if anyone can help me on this.
In my PHP codes, I'm trying to use the SOAP web services from another server (JIRA, java). The JIRA SOAP API is shown here.
$jirasoap = new SoapClient($jiraserver['url']);
$token = $jirasoap->login($jiraserver['username'], $jiraserver['password']);
$remoteissue = $jirasoap->getIssue($token, "issuekey");
I found that my codes have no problem to call the functions listed on that page. However, I don't know how to use the objects returned by the API calls.
My question are:
In my PHP codes, how can I use the methods in the Java class objects returned by SOAP API calls?
For example, the function $remoteissue = $jirasoap->getIssue($a, $b) will return a RemoteIssue. Based on this (http://docs.atlassian.com/rpc-jira-plugin/latest/com/atlassian/jira/rpc/soap/beans/RemoteIssue.html), there are methods like getSummary, getKey, etc. How can I use these functions in my codes?
Based on some PHP examples I found from the internet, it seems that everyone is using something like this:
$remoteissue = $jirasoap->getIssue($token, "issuekey");
$key = $remoteissue->key;
They are not using the object's methods.
Refer to this example, it seems that someone is able to do this in other languages. Can it be done in PHP too?
The problem I'm facing is that, I am trying to get the ID of an Attachment. However, it seems that we can't get the Attachment ID using this method: $attachmentid = $remoteattachment->id;. I am trying to use the $remoteattachment->getId() method.
In PHP codes, after we made a SOAP API call and received the returned objects, how do we know what data fields are available in that object?
For example,
$remoteissue = $jirasoap->getIssue($token, "issuekey");
$summary = $remoteissue->summary;
How do we know ->summary is available in $remoteissue?
When i refer to this document (http://docs.atlassian.com/rpc-jira-plugin/latest/com/atlassian/jira/rpc/soap/beans/RemoteIssue.html), I don't see it mention any data fields in RemoteIssue. How do we know we can get key, summary, etc, from this object? How do we know it is ->summary, not ->getsummary? We need to use a web browser to open the WSDL URL?
Thanks.
This question is over one year old, but to share knowledge and provide an answer to people who have this same question and found this page, here are my findings.
The document mentioned in the question is an overview of the JiraSoapService interface. This is a good reference for what functions can be called with which arguments and what they return.
If you use Java for your Jira SoapClient the returned objects are implemented, but if you use PHP, the returned objects aren't of the type stated in this documentation and do not have any of the methods mentioned. The returned objects are instances of the internal PHP class stdClass, which is a placeholder for undefined objects. The best way to know what is returned is to use var_dump() on the objects returned from the SoapCalls.
$jirasoap = new SoapClient($jiraserver['url']);
$token = $jirasoap->login($jiraserver['username'], $jiraserver['password']);
$remoteissue = $jirasoap->getIssue($token, "PROJ-1");
var_dump($remoteissue);
/* -- You will get something like this ---
object(stdClass)#2 (21) {
["id"]=> string(3) "100"
["affectsVersions"]=> array(0) { }
["assignee"]=> string(4) "user"
...
["created"]=> string(24) "2012-12-13T09:27:49.934Z"
...
["description"]=> string(17) "issue description"
....
["key"]=> string(6) "PROJ-1"
["priority"]=> string(1) "3"
["project"]=> string(4) "PROJ"
["reporter"]=> string(4) "user"
["resolution"]=> NULL
["status"]=> string(1) "1"
["summary"]=> string(15) "Project issue 1"
["type"]=> string(1) "3"
["updated"]=> string(24) "2013-01-21T16:11:43.073Z"
["votes"]=> int(0)
}
*/
// You can access data like this:
$jiraKey = $remoteissue->key;
$jiraProject = $remoteissue->project;
The document you referred to in #2 is to a Java implementation and really doesn't give you any help with PHP. If they do not publish a public API for their service (which would be unusual), then using the WSDL as a reference will let you know what objects and methods are accepted by the service and you can plan your method calls accordingly.
The technique you used to call getIssue(...) seems fine, although you should consider using try...catch in case of a SoapException.
I have used Jira SOAP in .NET project and IntelliSense hinted me what fields are available for returned object.
You can use something like VS.Php for Visual Studio or Php for Visual Studio if you are using Visual Studio.
Or you can choose one of the IDEs from here with support of IntelliSense.