I am trying to upload JSON file in MongoDB using Documents, but getting the following error. I'm new to Mongo. Can I get any help?
Exception in thread "main" org.bson.BsonInvalidOperationException: readStartDocument can only be called when CurrentBSONType is DOCUMENT, not when CurrentBSONType is ARRAY.
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoDatabase;
import com.mongodb.MongoClient;
import java.io.*;
import com.mongodb.client.model.BulkWriteOptions;
import com.mongodb.client.model.InsertOneModel;
import org.bson.Document;
import java.io.FileReader;
import java.io.BufferedReader;
import java.util.ArrayList;
import java.util.List;
public class Main {
public static void main(String[] args) {
//System.out.println(new File("").getAbsolutePath());
// Creating a Mongo client
MongoClient client = new MongoClient("localhost", 27017);
MongoDatabase database = client.getDatabase("mongotest");
MongoCollection<Document> collection = database.getCollection("jsondata");
int count = 0;
int batch =100;
List<InsertOneModel<Document>> docs = new ArrayList<>();
//System.out.println(docs.getClass().getName());
try (BufferedReader br = new BufferedReader(new FileReader("mongotest.json"))) {
String line;
while ((line = br.readLine()) != null) {
// System.out.println(Document.parse(line));
docs.add(new InsertOneModel<>(Document.parse(line)));
count++;
if (count == batch) {
collection.bulkWrite(docs, new BulkWriteOptions().ordered(false));
docs.clear();
count = 0;
}
}
}
catch(FileNotFoundException e) {
System.out.println("The file doesn't exist");
}
catch(IOException e){
System.out.println("The file could not be read");
}
}
}
Can you share the mongotest.json contents?
Each line must be an object (surrounded in {}), not an array (surrounded in []).
Edit:
To fix your issue, instead of reading using BufferedReader do this:
...
var json = Files.readString("mongotest.json", StandardCharsets.UTF_8);
var jsonarray = new JSONArray(json);
for (int i = 0; i < jsonarray.length(); i++) {
var line = jsonarray.getJSONObject(i);
docs.add(new InsertOneModel<>(Document.parse(line.toString())));
count++;
}
...
Related
I'm trying to fetch page post from the facebook.fb is giving data of likes based on paging concept so able get 1000 likes of a post and other likes data is in the next link.My code is working fine to get the post data but when I wanted to save the all likes of the of post in single string obj only first 1000 likes are stored in the String obj other likes not stored I Tried out with appending also but json formate is changing.I'm not having any idea how to work out for storing the all the likes of posts in one single string object.
package facebook;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
import facebook4j.Facebook;
import facebook4j.FacebookException;
import facebook4j.FacebookFactory;
import facebook4j.RawAPIResponse;
import facebook4j.auth.AccessToken;
import facebook4j.internal.org.json.JSONArray;
import facebook4j.internal.org.json.JSONException;
import facebook4j.internal.org.json.JSONObject;
public class Facebookthe{
public static void main(String[] args) throws FacebookException,JSONException, IOException {
// Generate facebook instance.
Facebook facebook = new FacebookFactory().getInstance();
// Use default values for oauth app id.
facebook.setOAuthAppId("XXXXXXXXXXX", "XXXXXXXXXXXXXX");
AccessToken accessTokenString = facebook.getOAuthAppAccessToken();
String m ="AnushkaShetty/?fields=posts.limit(1).since(2015).until(now){id,message,name,type,picture,link,caption,description,icon,application,shares,updated_time,source,comments.limit(500).summary(true){comment_count,message,can_remove,id,created_time,can_like,like_count,comments{comment_count,comments{comment_count}}},place,object_id,privacy,status_type,created_time,story,parent_id,story_tags,full_picture,likes.limit(9999).summary(true){id,name,username}},id,hometown,website,about,location,birthday,name,tagged{message_tags},category,category_list,talking_about_count,likes";
RawAPIResponse res1 = facebook.callGetAPI(m);
JSONObject jsonObject55= res1.asJSONObject();
System.out.println(jsonObject55); //609425942504811
//z=sb.append(jsonObject55);
JSONObject posts = jsonObject55.getJSONObject("posts");
JSONArray data = posts.getJSONArray("data");
JSONObject number = data.getJSONObject(0);
JSONObject likes = number.getJSONObject("likes");
JSONObject paging = likes.getJSONObject("paging");
String s = paging.getString("next");
int count =1;
while(s != null)
{
count++;
System.out.println("go to this link");
URL oracle = new URL(s);
URLConnection yc = oracle.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(
yc.getInputStream()));
String inputLine;
/* String oldfile="";
String newfile="";*/
JSONObject obj = new JSONObject();
while ((inputLine = in.readLine()) != null)
{
obj = new JSONObject(inputLine);
}
// z+=sb.append(obj);
//System.out.println(obj);
//System.out.println(count);
JSONObject jo = obj.getJSONObject("paging");
try
{
s = jo.getString("next");
}
catch(Exception e)
{
s=null;
System.out.println("there is no next");
}
in.close();
}
System.out.println(count);
}}
Applied looping on every likes_json object and added to the main likes_Json object.
for (int i = 0; i < addlikes.length(); i++) //looped
{
JSONObject addslikobj = addlikes.getJSONObject(i); //every json object data is stored in addlikobj
likesdata.put(addslikobj);//finally kept all the addlikobjs of each loop is kept in likesdata Jsonobj
}
I am trying to read few csv files which are present inside a folder and putting the absolute paths inside a list and then looping to the list and inserting data into excel in different sheets but for some reason only last loop is overwriting everything.
package excel;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileOutputStream;
import java.io.FileReader;
import java.util.ArrayList;
import java.util.List;
import org.apache.poi.hssf.usermodel.HSSFRow;
import org.apache.poi.hssf.usermodel.HSSFSheet;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;
public class FileReading {
public static void csvToXLS(String filename, String sheetno) {
try {
String csvFile = filename; //csv file address
String excelFile = "E:\\test.xls"; //xlsx file address
HSSFWorkbook workBook = new HSSFWorkbook();
HSSFSheet sheet = workBook.createSheet(sheetno);
String currentLine = null;
int RowNum = 0;
BufferedReader br = new BufferedReader(new FileReader(csvFile));
while ((currentLine = br.readLine()) != null) {
String str[] = currentLine.split(",");
HSSFRow currentRow = sheet.createRow(RowNum);
RowNum++;
for (int i = 0; i < str.length; i++) {
currentRow.createCell(i).setCellValue(str[i]);
}
}
FileOutputStream fileOutputStream = new FileOutputStream(excelFile);
workBook.write(fileOutputStream);
fileOutputStream.close();
System.out.println("Done");
} catch (Exception ex) {
System.out.println(ex.getMessage() + "Exception in try");
}
}
public static List listOfFiles() {
List<String> results = new ArrayList<String>();
File[] files = new File("f:\\csv").listFiles();
for (File file : files) {
if (file.isFile()) {
results.add(file.getAbsolutePath());
}
}
return results;
}
public static void main(String[] args) {
List list = listOfFiles();
int size = list.size();
for (int i = 0; i < size; i++) {
csvToXLS(list.get(i).toString(), "sheet" + i + 1);
}
}
}
You're re-creating and saving the Workbook every time you call csvToXLS, so only the last call survives, overwriting all others.
Create the Workbook only once, before you start your for loop, passing it into csvToXLS. then that method will create a new Sheet on an existing Workbook. Then after the for loop ends, you can write out and save the Workbook.
I did this thanks for the idea #rgettman
package excel;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileOutputStream;
import java.io.FileReader;
import java.util.ArrayList;
import java.util.List;
import org.apache.poi.hssf.usermodel.HSSFRow;
import org.apache.poi.hssf.usermodel.HSSFSheet;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;
public class CsvToXlx {
public static void csvToXLSX(HSSFWorkbook workBook,String filename, String sheetno) {
try {
String csvFile = filename; //csv file address
String excelFile = "E:\\test.xls"; //xlsx file address
//HSSFWorkbook workBook = new HSSFWorkbook();
HSSFSheet sheet = workBook.createSheet(sheetno);
String currentLine = null;
int RowNum = 0;
BufferedReader br = new BufferedReader(new FileReader(csvFile));
while ((currentLine = br.readLine()) != null) {
String str[] = currentLine.split(",");
HSSFRow currentRow = sheet.createRow(RowNum);
RowNum++;
for (int i = 0; i < str.length; i++) {
currentRow.createCell(i).setCellValue(str[i]);
}
}
FileOutputStream fileOutputStream = new FileOutputStream(excelFile);
workBook.write(fileOutputStream);
fileOutputStream.close();
System.out.println("Done");
} catch (Exception ex) {
System.out.println(ex.getMessage() + "Exception in try");
}
}
public static List listOfFiles() {
List<String> results = new ArrayList<String>();
File[] files = new File("f:\\csv").listFiles();
for (File file : files) {
if (file.isFile()) {
results.add(file.getAbsolutePath());
}
}
return results;
}
public static void main(String[] args) {
List list = listOfFiles();
int size = list.size();
HSSFWorkbook workBook = new HSSFWorkbook();
for (int i = 0; i < size; i++) {
csvToXLSX(workBook,list.get(i).toString(), "sheet" + i + 1);
}
}
}
This there anything wrong with my code? I'm new to Java and i'm trying to import a file into MongoDB. However there is a error that i have no idea what is it. I am using Eclipse.
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStreamReader;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.DBCursor;
import com.mongodb.DBObject;
import com.mongodb.Mongo;
import com.mongodb.util.JSON;
import com.mongodb.util.JSONParseException;
public class readwrite {
public static void main(String[] args) throws FileNotFoundException,IOException,JSONParseException{
Mongo mongo = new Mongo("localhost", 27017);
DB db = mongo.getDB("actualdata");
DBCollection collection = db.getCollection("metadata");
String line = null;
StringBuilder sb = new StringBuilder();
try {
FileInputStream fstream = null;
try {
fstream = new FileInputStream("/home/Output/json1-100000-all");
} catch (FileNotFoundException e) {
e.printStackTrace();
System.out.println("File does not exist, exiting");
return;
}
BufferedReader bufferedReader =
new BufferedReader(new InputStreamReader(fstream));
while((line = bufferedReader.readLine()) != null) {
System.out.println(line);
DBObject dbObject;
sb.append(dbObject = (DBObject) JSON.parse(bufferedReader.readLine()));
collection.insert(dbObject);
DBCursor cursorDoc = collection.find();
while (cursorDoc.hasNext()) {
System.out.println(cursorDoc.next());
}
}
bufferedReader.close();
}
catch(FileNotFoundException ex) {
System.out.println("Unable to open file");
}
catch(IOException ex) {
System.out.println(
"Error reading file");
}
}
}
This is the error that is displayed
[
Exception in thread "main" com.mongodb.util.JSONParseException:
{
^
at com.mongodb.util.JSONParser.read(JSON.java:272)
at com.mongodb.util.JSONParser.parseObject(JSON.java:230)
at com.mongodb.util.JSONParser.parse(JSON.java:195)
at com.mongodb.util.JSONParser.parse(JSON.java:145)
at com.mongodb.util.JSON.parse(JSON.java:81)
at com.mongodb.util.JSON.parse(JSON.java:66)
at readwrite.main(readwrite.java:45)
It show me this error when i clicked on at com.mongodb.util.JSONParser.read(JSON.java:272) where it says that the Source is not found. The source attachment does not contain the source for the file JSON.class.
I can print the output of BufferedReader if i did not included the conversion of DBObject. Thanks in advance!
1) Didn't you mean to write JSON.parse(line)
instead of JSON.parse(bufferedReader.readLine())) ?
This might cause it to try and parse 'null' at the last iteration
2) If that doesn't help, could you get the exact string value of 'line' on the failed iteration? (this should be easy using debugger or simple printing to system out)
Regards
I am getting arrayoutofbond error while running below given code,
sometime it is running as expected and sometime it is giving error.
Could anyone help me where I am wrong.
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Date;
public class getFileContent{
public void listFiles() throws IOException, InterruptedException{
File directory = new File("C:\\ScriptLogFile\\");
File[] myarray;
myarray=directory.listFiles();
int i=0;
ArrayList<String> arrayList = new ArrayList<String>();
SimpleDateFormat sdf = new SimpleDateFormat("ddMMyyyy_hhmmss");
Date curDate = new Date();
String strDate = sdf.format(curDate);
String fileName = strDate;
File file = new File("C:\\ExcelReport_"+fileName+".csv");
FileWriter fileWritter = new FileWriter(file, true);
BufferedWriter bwr = new BufferedWriter(fileWritter);
String filename = null;
try {
for (int j = 0; j < myarray.length; j++)
{
File path=myarray[j];
FileInputStream fis = new FileInputStream (path);
BufferedReader br = new BufferedReader(new InputStreamReader(fis));
if(path.isFile()){
if(path.getName().endsWith(".csv")){
filename = path.getName();
String line;
bwr.write(filename+",");
while ((line = br.readLine()) != null) {
if(line.contains("-")){
String[] part = line.split("-");
arrayList.add(part[1]);
bwr.write(arrayList.get(i)+",");
i++;
}
else{
}
}
bwr.write("\r\n");
}
}
}
}catch (FileNotFoundException e) {
e.printStackTrace();
}
bwr.close();
}
public static void main(String[] args) throws IOException, InterruptedException {
getFileContent gfc = new getFileContent();
gfc.listFiles();
}
}
We need a stack trace to see where the exception is raised. However you seem to be making assumptions about the length of part[]. Remember arrays are 0-indexed, the first entry would be at index 0 i.e. part[0]. Even then, in general there really needn't be many entries at all: "xyz".split("-") is an array of length 1 whose only element, "xyz", is at index 0.
How can I use Java application to add system files as an attachment to Couchdb database using Couchdb4J library?
I tried modifying the sample code below but there's an unresolved error. Does anybody know where's my mistake and how to fix it? Thanks in advance.
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.HashMap;
import java.util.Map;
import com.fourspaces.couchdb.CouchResponse;
import com.fourspaces.couchdb.Database;
import com.fourspaces.couchdb.Document;
import com.fourspaces.couchdb.Session;
public class FileScanner {
Session priceListDocsSession = new Session("localhost",5984);
Database db = priceListDocsSession.getDatabase("filesdb");
public static void main(String[] args) {
FileScanner fs = new FileScanner();
fs.processDir(new File("C:\\CouchDB"));
}
void processDir(File f) {
if (f.isFile()) {
Map<String, Object> doc = new HashMap<String, Object>();
doc.put("name", f.getName());
doc.put("path", f.getAbsolutePath());
doc.put("size", f.length());
db.saveDocument(doc);
InputStream is = new FileInputStream(f);
String att=db.putAttachment(doc.getId(),doc.getRev(),f,is);
}
else {
File[] fileList = f.listFiles();
if (fileList == null) return;
for (int i = 0; i < fileList.length; i++) {
try {
processDir(fileList[i]);
} catch (Exception e) {
System.out.println(e);
}
}
}
}
}
The errors appears on the db.saveDocument(doc);
and
String att=db.putAttachment(doc.getId(),doc.getRev(),f,is); saying that .getId() and getRev() is undefined for the type Map
I managed to fixed the problem by adding some of the jcouchdb dependencies on the classpath.