I have a json file that contains 500k Objects and this is its format:
"WORKORDER": [
{
"Attributes": {
"SITEID": {
"content": "BEDFORD"
},
"WONUM": {
"content": "1000"
},
"WOPRIORITY": {
"content": 2
},
"WORKTYPE": {
"content": "CM"
}
}
},
{
"Attributes": {
"SITEID": {
"content": "BEDFORD"
},
"WONUM": {
"content": "1000-10"
},
"WORKTYPE": {
"content": "CM"
}
}
}
Im geting the distinct values like this :
for (int i = 0; i < WORKORDER.length(); i++) {
JSONObject obj = WORKORDER.getJSONObject(i);
JSONObject att = obj.getJSONObject("Attributes");
if( att.has(col)){ // getting col from params in the servlet
JSONObject column = att.getJSONObject(col);
Object colval = column.get("content");
if(!(list.contains(colval))) {
out.println( colval);
list.add(colval);
}
But it takes long time for only 5000 objects !
Is there any way to get the distinct values of any column without parsing the whole Json file, otherwise parsing only the column needed.
You are iterating on a JSON with 500k elements. For each element you check if it was previously added in a List. That means your logic will iterate the list 500k times.
Instead, you should use a HashSet, first, the Set prevent duplicated value. So you just need to set.add(value) but the most interesting is the fact that the instance have a constant complexity value. Since it used buckets to organize the value, it doesn't have to iterate the Set fully.
You can read more about that in amit answer's about How can a HashSet offer constant time add operation?
Note that HashSet gives amortized and average time performance of O(1), not worst case. This means, we can suffer an O(n) operation from time to time.
So, when the bins are too packed up, we just create a new, bigger array, and copy the elements to it.
Note that to use any Hash#### implementation, you need to make sure the instance you store implements hashCode and equals correctly. You can find out more about this in the community post about What issues should be considered when overriding equals and hashCode in Java?.
Now for the solution :
Set<Object> sets = new HashSet<>();
for (int i = 0; i < WORKORDER.length(); i++) {
// ...
Object colval = column.get("content");
if(sets.add(colval)){ //`add` return true if it wasn't present already.
out.println( colval);
}
}
I kept the Object type but this should be correctly typed, at least to be sure that those instance are implementing those methods as needed.
colval being an Object, it is possible it doesn't implements correctly the methods needed so I suggest you parse it correctly. You should use column.getString("content) instead or check the instance type.
To validate this, I have used a method to create a fake JSON:
public static JSONObject createDummyJson(int items) {
JSONObject json = new JSONObject();
JSONArray orders = new JSONArray();
json.put("order", orders);
JSONObject attributes;
JSONObject item;
JSONObject order;
Random rand = new Random();
String[] columns = {"columnA", "columnB", "columnC", "columnD"};
for(int i = 0; i < items; ++i) {
order = new JSONObject();
attributes = new JSONObject();
order.put("Attributes", attributes);
orders.put(order);
for(int j = 0; j < rand.nextInt(1000) % columns.length; ++j) {
item= new JSONObject();
long rValue = rand.nextLong();
item.put("content", j%3 == 0 ? ("" + rValue ) : rValue );
attributes.put(columns[j], item);
}
}
return json;
}
Then ran a basic benchmark for both method and had the following results :
public static void main(String[] args) throws Exception {
final int jsonLength = 500_000;
JSONObject json = createDummyJson(jsonLength);
long start = System.currentTimeMillis();
List<Object> list = parseJson(json);
long end = System.currentTimeMillis();
System.out.format("List - Run in %d ms for %d items and output %d lines%n", end-start, jsonLength, list.size());
start = System.currentTimeMillis();
Set<Object> set = parseJsonSet(json);
end = System.currentTimeMillis();
System.out.format("Set - Run in %d ms for %d items and output %d lines%n", end-start, jsonLength, set.size());
}
public static List<Object> parseJson(JSONObject json) {
String col = "columnC";
JSONArray array = json.getJSONArray("order");
List<Object> list = new ArrayList<>();
for (int i = 0; i < array.length(); i++) {
JSONObject obj = array.getJSONObject(i);
JSONObject att = obj.getJSONObject("Attributes");
if (att.has(col)) { // getting col from params in the servlet
JSONObject column = att.getJSONObject(col);
Object colval = column.get("content");
if (!(list.contains(colval))) {
//System.out.println(colval);
list.add(colval);
}
}
}
return list;
}
public static Set<Object> parseJsonSet(JSONObject json) {
String col = "columnC";
JSONArray array = json.getJSONArray("order");
Set<Object> set = new HashSet<>();
for (int i = 0; i < array.length(); i++) {
JSONObject obj = array.getJSONObject(i);
JSONObject att = obj.getJSONObject("Attributes");
if (att.has(col)) { // getting col from params in the servlet
JSONObject column = att.getJSONObject(col);
Object colval = column.get("content");
if (set.add(colval)) {
//System.out.println(colval);
}
}
}
return set;
}
List - Run in 5993 ms for 500000 items and output 46971 lines
Set - Run in 62 ms for 500000 items and output 46971 lines
I even went to a JSON with 5M row (removed the List that would never end)
Set - Run in 6436 ms for 5000000 items and output 468895 lines
Important, remove the line that print in console every new insertion, it will reduce the execution time a bit.
Related
I have a String array with name and id, I need to convert that String array to List of objects.
This is my code:
private List<ObjectAttribute> getDtls(String newVal) {
ObjectAttribute object = new ObjectAttribute();
List<ObjectAttribute> objLst = new ArrayList<ObjectAttribute>();
String[] newImageVal = [step0005.jpg, 172B6846-0073-4E5B-B10A-DDD928994EA6, step0003.jpg, FBC8D143-2CD7-47E6-B323-31A0928A9338]
for (int i = 0; i <= newImageVal.length - 1; i++) {
object.setImageName(newImageVal[i]);
object.setImageId(newImageVal[++i]);
objLst.add(object);
}
return objLst;
}
but there is a problem that it always returns only the last value in objList. Can anyone correct this code.
move
ObjectAttribute object = new ObjectAttribute();
inside the for loop:
for (int i = 0; i <= newImageVal.length - 1; i++) {
ObjectAttribute object = new ObjectAttribute();
object.setImageName(newImageVal[i]);
object.setImageId(newImageVal[++i]);
objLst.add(object);
}
private List<ObjectAttribute> getDtls(String newVal) {
List<ObjectAttribute> objLst = new ArrayList<ObjectAttribute>();
String[] newImageVal = [step0005.jpg, 172B6846-0073-4E5B-B10A-DDD928994EA6, step0003.jpg, FBC8D143-2CD7-47E6-B323-31A0928A9338]
// String delimiter = ", ";
// newImageVal = newVal.split(delimiter);
for (int i = 0; i <= newImageVal.length - 1; i++) {
ObjectAttribute object = new ObjectAttribute();
object.setImageName(newImageVal[i]);
object.setImageId(newImageVal[++i]);
objLst.add(object);
}
return objLst;
}
The root cause of your problem is, as #StefanBeike wrote, that you instantiate the object only once before the for-loop and then you just keep rewriting its attributes. Moving the instantiation (= calling of new) inside the for-loop fixes the functionality.
However, apart from this, it is a very bad practice to increment the for-loop variable inside the for-loop body. Thus you obscure your intention and you will get a code which is less readable, harder maintainable and easier to break by later changes.
And the main condition should be i < newImageVal.length-1 to handle the array size safely. (To be 100% sure you do not get ArrayIndexOutOfBoundsException.)
There are multiple better ways.
Increment by 2 in the for-loop "header":
for (int i = 0; i < newImageVal.length-1; i += 2) {
ObjectAttribute object = new ObjectAttribute();
object.setImageName(newImageVal[i]);
object.setImageId(newImageVal[i+1]);
objLst.add(object);
}
Use while-loop instead of for-loop:
int i = 0;
while (i < newImageVal.length-1) {
ObjectAttribute object = new ObjectAttribute();
object.setImageName(newImageVal[i++]);
object.setImageId(newImageVal[i++]);
objLst.add(object);
}
or you may do something like this using the streams way:
AtomicInteger ai = new AtomicInteger();
List<ObjectAttribute> objLst = Arrays.stream(newImageVal)
.map(img-> {
ObjectAttribute object = new ObjectAttribute();
object.setImageName(img);
object.setImageId(ai.getAndIncrement());
return obj;
}).collect(Collectors.toList())
This JSONAray:
"cows": [
{
"age": 972,
"name": "Betty"
"status": "publish",
"sticky": "pregnant"
},
{
"age": 977,
"name"; "Kate"
"status": "publish",
"sticky": "heat"
},
{
"age": 959,
"name": "Julie"
"site_age": 63178480,
"sticky": "Nursing"
},
...
}
that contains 20 objects. What I wanted is this: get 3 random objects out of the 20. And the ages of any of the three won't be a certain number say 961.
Currently this what I am doing:
private void parseCowsReq(JSONObject array) {
try {
for (int i = 0; i < 3; i++) {
int randumNum = getRandomCow(array);
JSONObject jsonObject = array.getJSONObject(randumNum);
String cowName = jsonObject.getString("name");
String cowStatus = jsonObject.getString("status");
Log.d(TAG, "Cow name is " + cowName + "cow Status is " + cowStatus);
}
} catch (JSONException e) {
e.printStackTrace();
}
}
private int getRandomCow(JSONArray jsonArray) {
int length = jsonArray.length();
int[] array;
array = new int[length-1];
int rnd = new Random().nextInt(array.length);
return array[rnd];
}
There are of issues with this code.
I don't know how to ensure that the object gotten in line
JSONObject jsonObject = array.getJSONObject(randumNum); won't have an
age of 961
The random number gotten is always 0
Please do you have any idea how this can be done?
you can do it with this:
public ArrayList<Integer> getRandomObject(JSONArray jsonArray, int indexesWeeNeed){
Random rn = new Random();
Set<Integer> generated = new LinkedHashSet<>();
while(generated.size() < indexesWeeNeed){
int index = rn.nextInt(10);
JSONObject jsonObject = (JSONObject) jsonArray.get(index);
int age = jsonObject.getInt("age");
if(age<961) {
generated.add(index);
}
}
ArrayList<Integer> arrayList = new ArrayList<>();
arrayList.addAll(generated);
return arrayList;
}
One part that's messed up is when you call
Random().nextInt(array.length);
array.length is the new array you just created. You need to perform the function on the existing array:
Random().nextInt(jsonArray)
to get a random number other than zero.
As for ensuring you don't get a certain age, I'd suggest breaking up the code to not call the getRandomCow(array) function inside of the for loop. When you retrieve a cow, check the name doesn't match, check the age, and if it works, keep it. If not get another cow.
Well firstly, load the objects:
JSONArray array = /* your array */;
Next, we need a method to retrieve 3 unique objects from the JSONArray (which is actually a List). Let's shuffle the indexes of the json array, so that we don't end up having to repeatedly generate duplicates:
public Stream<JSONObject> randomObjects(JSONArray array, int amount) {
if (amount > array.size()) {
//error out, null, return array, whatever you desire
return array;
}
List<Integer> indexes = IntStream.range(0, array.size()).collect(Collectors.toList());
//random, but less time generating them and keeping track of duplicates
Collections.shuffle(indexes);
Set<Integer> back = new HashSet<>();
Iterator<Integer> itr = indexes.iterator();
while (back.size() < amount && itr.hasNext()) {
int val = itr.next();
if (array.get(val).getInt("age") != 961) { //or any other predicates
back.add(val);
}
}
return back.stream().map(array::get);
}
Using this, we can select the three objects from the list and utilize them how we wish:
randomObjects(array, 3).map(o -> o.getInt("age")).forEach(System.out::println);
//972
//977
//952
When I said "or any other predicates, you can pass those as well via the method:
public Stream<JSONObject> randomObjects(..., Predicate<Integer> validObject) {
//...
int val = itr.next();
if (validObject.test(val)) {
back.add(val);
}
//...
}
Which would mean you could change the blacklisting per method call:
Stream<JSONObject> random = randomObjects(array, 3, val -> array.get(val).getInt("age") != 961);
Hello everyone I'm trying to parse a JSON file from this LINK.
The returned JSON is as follows-
I need to loop through all the instances of journeys first, then I have to loop through legs for each journeys to get the detailed instruction. As you can see each legs consists of instruction in parts which returns a string, my ultimate goal is to combine these string for each journeys and display them as a TextView. So for the above JSON the end goal is to display -
Jubilee line towards Stratford, or North Greenwich
Northern line towards Edgware, or High Barnet
Till now I've been trying to navigate through this JSON without any luck.
Here is the code I've been working on-
try {
//object is the JSON file.
JSONArray Journey = object.getJSONArray("journeys");
if (Journey != null) {
//Retrieving number of possible routes.
for (int i=0;i<Journey.length();i++){
Routes.add(Journey.getJSONObject(i));
}
//Retrieving number of possible legs for each route.
if (!Routes.isEmpty()){
for (int j = 0; j< Routes.size(); j++){
Legs.add(j, Routes.get(j).getJSONArray("legs"));
}
//Trying to retrieve the detailed instruction here and failing.
for(int k=0;k<Routes.get(k).getJSONArray("legs").length();k++){
instructionDetail.add(k,Legs.get(k).getJSONObject(k).getJSONObject("instruction"));
}
}
}
} catch (JSONException e) {
e.printStackTrace();
}
I believe my approach is wrong and I didn't get the loop right.. Suggestions to parse and navigate and any other approach will be highly appreciated!
Thanks!
UPDATE:
JSONArray journeys = new JSONObject("");
for(int i = 0 ; i < journeys.length() ; i++) { // Traverse journeys
JSONObject journey = journeys.getJSONObject(i); // get journeys(i) -> journey
if(journey.has("legs")) { // if journey has "legs" key
JSONArray legs = journey.getJSONArray("legs"); // get the legs array from journey object
for(int j = 0 ; j < legs.length() ; j++) { // Traverse legs
JSONObject leg = legs.getJSONObject(j); // get legs(j) -> leg
if(leg.has("instruction")) { // if leg has "instruction" key in it
JSONObject instruction = leg.getJSONObject("instruction"); // get instruction jsonObject
String detailed = instruction.optString("detailed", "Fallback detailed"); // get detailed string in instruction object
}
}
}
}
Update:
private static class Detail {
String journeyType;
String legType;
String instructionType;
String detail;
public Detail(String journeyType, String legType, String instructionType, String detail) {
this.journeyType = journeyType;
this.legType = legType;
this.instructionType = instructionType;
this.detail = detail;
}
}
...
...
List<Detail> detailList = new ArrayList<>();
JSONArray journeys = new JSONObject("");
for(int i = 0 ; i < journeys.length() ; i++) { // Traverse journeys
JSONObject journey = journeys.getJSONObject(i); // get journeys(i) -> journey
if(journey.has("legs")) { // if journey has "legs" key
JSONArray legs = journey.getJSONArray("legs"); // get the legs array from journey object
for(int j = 0 ; j < legs.length() ; j++) { // Traverse legs
JSONObject leg = legs.getJSONObject(j); // get legs(j) -> leg
if(leg.has("instruction")) { // if leg has "instruction" key in it
JSONObject instruction = leg.getJSONObject("instruction"); // get instruction jsonObject
String journeyType = journey.getString("$type");
String legType = leg.getString("$type");
String instructionType = instruction.getString("$type");
String detailed = instruction.getString("detailed"); // get detailed string in instruction object
detailList.add(new Detail(journeyType, legType, instructionType, detailed));
}
}
}
}
for(Detail detail : detailList) {
TextView textView = new TextView([yourContext]);
textView.setText(detail.detail);
yourContentViewGroup.addView(textView);
// or you can use View.inflate(context, layoutRes, yourContentViewGroup) and design a layout to show other detail instance values
}
I am generating a box and whisker chart with one item per category.I also want to generate a report with the mean, median and all the values per item in the BoxPlot. So, after i create the dataset, defaultboxandwhiskercategorydataset based on the categoryType, I call the method convertReportData to fetch each item in the defaultboxandwhiskercategorydataset and save the mean, median etc into another data object later for report generation. But it just prints only one category. Could anyone please help me to figure out what is wrong?
My boxplot
Code:
public static BoxAndWhiskerCategoryDataset createDataset() {
startTime = inputData.getItimeFrom();
endTime = inputData.getItimeTo();
List<String> categorylist = new ArrayList<>();
categorylist.add("Distance 0-20");
categorylist.add("Distance 20-40");
categorylist.add("Distance 40-60");
categorylist.add("Distance 60-80");
categorylist.add("Distance 80-100");
categorylist.add("Distance >100");
Map<String, List<Double>> map = new HashMap<String, List<Double>>();
map = addDistance(values_list);
DefaultBoxAndWhiskerCategoryDataset defaultboxandwhiskercategorydataset = new DefaultBoxAndWhiskerCategoryDataset();
for (String categoryType : categorylist) {
map.remove(null);
for (Map.Entry<String, List<Double>> entry : map.entrySet()) {
if (entry.getKey().equalsIgnoreCase(categoryType)) {
defaultboxandwhiskercategorydataset.add(entry.getValue(),
categoryType, " ");
}
}
}
convertReportData(defaultboxandwhiskercategorydataset, categorylist);
return defaultboxandwhiskercategorydataset;
}
private static void convertReportData(DefaultBoxAndWhiskerCategoryDataset boxandwhiskercategorydataset, List<String> latencyTypelist) {
report = new HashMap<>();
for (int i = 0; i < boxandwhiskercategorydataset.getColumnKeys().size(); i++) {
BoxAndWhiskerItem item = boxandwhiskercategorydataset.getItem(i, 0);
ReportData data = new ReportData();
data.setMean(item.getMean());
data.setMedian(item.getMedian());
data.setQ1(item.getQ1());
data.setQ3(item.getQ3());
data.setMaxOutlier(item.getMaxOutlier());
data.setMaxRegularNumber(item.getMaxRegularValue());
data.setMinOutlier(item.getMinOutlier());
data.setMinRegularNumber(item.getMinRegularValue());
data.setOutliers(item.getOutliers());
report.put(boxandwhiskercategorydataset.getRowKey(i).toString(),
data);
}
}
The problem is with
for (int i = 0; i < boxandwhiskercategorydataset.getColumnKeys().size(); i++) {
you are using getColumnKeys whereas you have only one Column. It should have been,
for (int i = 0; i < boxandwhiskercategorydataset.getRowKeys().size(); i++) {
I made a tool from a block I made no errors in code. When I try to click build it gives me this terminal error:
How can i fix this? Please.
Here is the code for the RecipesTools.addRecipes
package net.minecraft.src;
public class RecipesTools
{
private String recipePatterns[][] =
{
{
"XXX", " # ", " # "
}, {
"X", "#", "#"
}, {
"XX", "X#", " #"
}, {
"XX", " #", " #"
}
};
private Object recipeItems[][];
public RecipesTools()
{
recipeItems = (new Object[][]
{
new Object[] {
Block.planks, Block.cobblestone, Item.ingotIron, Item.diamond, Item.ingotGold, Block.RadiatedStone
}, new Object[] {
Item.pickaxeWood, Item.pickaxeStone, Item.pickaxeSteel, Item.pickaxeDiamond, Item.pickaxeGold, Item.pickaxeRadiated
}, new Object[] {
Item.shovelWood, Item.shovelStone, Item.shovelSteel, Item.shovelDiamond, Item.shovelGold
}, new Object[] {
Item.axeWood, Item.axeStone, Item.axeSteel, Item.axeDiamond, Item.axeGold
}, new Object[] {
Item.hoeWood, Item.hoeStone, Item.hoeSteel, Item.hoeDiamond, Item.hoeGold
}
});
}
public void addRecipes(CraftingManager craftingmanager)
{
for (int i = 0; i < recipeItems[0].length; i++)
{
Object obj = recipeItems[0][i];
for (int j = 0; j < recipeItems.length - 1; j++)
{
Item item = (Item)recipeItems[j + 1][i];
craftingmanager.addRecipe(new ItemStack(item), new Object[]
{
recipePatterns[j], Character.valueOf('#'), Item.stick, Character.valueOf('X'), obj
});
}
}
craftingmanager.addRecipe(new ItemStack(Item.shears), new Object[]
{
" #", "# ", Character.valueOf('#'), Item.ingotIron
});
}
}
EDIT
I also have given Eclipe 1024mb of RAM and deleted my .Minecraft folder.
CONFLICT # 22
27 achievements
Exception in thread "main" java.lang.ExceptionInInitializerError
at net.minecraft.src.StatList.initCraftableStats(StatList.java:74)
at net.minecraft.src.StatList.initBreakableStats(StatList.java:55)
at net.minecraft.src.Block.<clinit>(Block.java:975)
at net.minecraft.src.TextureWaterFX.<init>(TextureWaterFX.java:13)
at net.minecraft.client.Minecraft.<init>(Minecraft.java:205)
at net.minecraft.src.MinecraftImpl.<init>(MinecraftImpl.java:13)
at net.minecraft.client.Minecraft.startMainThread(Minecraft.java:1984)
at net.minecraft.client.Minecraft.startMainThread1(Minecraft.java:1970)
at net.minecraft.client.Minecraft.main(Minecraft.java:2032)
at Start.main(Start.java:25)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 5
at net.minecraft.src.RecipesTools.addRecipes(RecipesTools.java:44)
at net.minecraft.src.CraftingManager.<init>(CraftingManager.java:19)
at net.minecraft.src.CraftingManager.<clinit>(CraftingManager.java:8)
... 10 more
recipeItems[0].length is 6. But recipeItems[2] and the following only have five elements. Your toplevel loop in addRecipes it therefore wrong.
You should probably be using collection types (vector, list, Array, ...) and iterators for this, would make the code safer and more readable (IMO).
Your outer loop in addRecipies is iterating over the [0] array of recipeItems. In the first child array, there are 6 elements meaning that recipeItems[0][5] will be a valid item. But there is an incorrect assumption that this is true for all recipeItems arrays. At least one of the later arrays has fewer than 6 elements.
You should be iterating over the size of the child arrays rather than the size of the first array.
First of all, you should probably check of what type the Objects in the array are, before you cast them, so the part which currently is
Item item = (Item)recipeItems[j + 1][i];
in the loop, should be replaced with something like this:
Object itemObj = recipeItems[j + 1][i];
if(itemObj instanceof Item)
{
// The current element is an Item
Item item = (Item)recipeItems[j + 1][i];
craftingmanager.addRecipe(new ItemStack(item), new Object[]
{
recipePatterns[j], Character.valueOf('#'), Item.stick, Character.valueOf('X'), obj
});
}
else if(itemObj instanceof Block)
{
// The current element is a Block
Block block = (Block)recipeItems[j + 1][i];
craftingmanager.addRecipe(new ItemStack(block), new Object[]
{
recipePatterns[j], Character.valueOf('#'), Item.stick, Character.valueOf('X'), obj
});
}
else
{
// The current element is none of the types above
// TODO Throw an exception, print a message or quit the game
}
because, I'm pretty sure you can't cast an Item to a Block. This will not fix this problem, that was just a tip, because the code you used there before may cause errors in the future.
The solution to your current problem is what Mat and nicholas.hauschild has already answered. The first two elements of the recipeItems array (recipeItems[0] and recipeItems[1]) have 6 elements, but the rest of the elements have only 5 elements. In your loop, you only take the length of the first element and use it to loop through the rest of the elements too, but they are smaller than the first.
What happens if you try to acces element 6 in an array with 5 elements?
You could replace the loop with something like this:
for(int i = 0; i < recipeItems.length - 1; i++)
{
for(int j = 0; j < recipeItems[i].length; j++)
{
Object obj = recipeItems[0][j];
Object itemObj = recipeItems[i + 1][j];
if(itemObj instanceof Item)
{
// The current element is an Item
Item item = (Item)recipeItems[j + 1][i];
craftingmanager.addRecipe(new ItemStack(item), new Object[]
{
recipePatterns[j], Character.valueOf('#'), Item.stick, Character.valueOf('X'), obj
});
}
else if(itemObj instanceof Block)
{
// The current element is a Block
Block block = (Block)recipeItems[j + 1][i];
craftingmanager.addRecipe(new ItemStack(block), new Object[]
{
recipePatterns[j], Character.valueOf('#'), Item.stick, Character.valueOf('X'), obj
});
}
else
{
// The current element is none of the types above
// TODO Throw an exception, print a message or quit the game
}
}
}
Hope this helps!