Develop Reference

What are the advantages to writing data values to a CSV as an integer type rather than a string? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
I'm collaborating on a project where I currently have built a program writing data to a CSV file in a string format. My partner on the project is saying he thinks the product would be more usable if it was written in an integer format, while I've been arguing that our visualization features could simply run a parseInt when it reads the CSV data.
I wanted to ask this on here to get some information on what can be gained by writing to a file using a primitive data type rather than a string. Java really seems to be built to write to CSV as a string, but he claims it would be more efficient to write the data as an int. Thoughts?
This is really more of a conceptual question, but I'll include the code I'm using to generate the data table in case context matters.
//Snippet only
private void elementLocator() {
//Declare ArrayList to hold values
data = new ArrayList<ArrayList<String>>();
//Build data table
try {
//Unique xpath string
String prefix = "//*[#id=\"main_table_countries_today\"]/tbody[1]/tr[";
int j = 2;
System.out.println("Retrieving data...");
for(int i = 1; i <= 222; i ++) {
try {
//Initialize array to fill in to data row by row
ArrayList<String> parser = new ArrayList<String>();
for(j = 2; j <= 13; j ++) {
parser.add(driver.findElement(By.xpath(prefix + i + "]/td[" + j + "]")).getText());
}
//Use a boolean indicator to skip any row that has a blank 1st column
String w = parser.get(0);
boolean v = w.isEmpty();
if(v) {
continue;
}
else {
data.add(parser);
}
//Catch errors
} catch (Exception e) {
e.printStackTrace();;
continue;
}
}
}
public void makeCSV() throws IOException {
//Create output file only if it does not already exist
EST est = new EST();
//Pull year, month, day for file name
String dt = est.getDate();
f = new File(home + "\\Climate Dev Pegasus\\Data\\Worldometer\\" + dt + ".csv");
if(!f.exists()) {
try {
//Create FileWriter object with file as parameter
CSVWriter writer = new CSVWriter(new FileWriter(f, true));
//Write headers
String[] headers = "Country/Region, Total Cases, New Cases, Total Deaths, New Deaths, Total Recovered, Active Cases, Serious Cases, Tot Cases/1M pop, Deaths/1M pop, Total Tests, Tests/1M pop".split(",");
writer.writeNext(headers);
writer.flush();
writer.close();
//Give full modification permissions to file
SetPermissions sp = new SetPermissions();
sp.ChangePermissions(f);
}catch (Exception ex) {
ex.printStackTrace();
}
}
else {
}
path = Paths.get(home + "\\Climate Dev Pegasus\\Data\\Worldometer\\" + dt + ".csv");
//Write data to file, allowing commas
FileWriter csvWriter = new FileWriter(f,true);
for(ArrayList<String> x : data) {
for(String y : x) {
String z = appendDQ(y);
//int value = Integer.parseInt(z);
csvWriter.append(z);
csvWriter.append(",");
}
csvWriter.append("\n");
}
System.out.println("Data successfully written to file.");
csvWriter.close();
}

Here is an answer, which came to my mind while thinking about this problem:
Well, I think that's a very fundamental question.
First of all, the most important thing. The program should be easy to understand for other developers and at the same time it should be powerful enough that the end user is not disturbed during use, for example by long loading times.
But to find the answer to your question you should go further. Should the program run on a PC or an embedded system?
Since Java already implements the string class by default, it is quite powerful from scratch. Of course, an integer is always more performant, since it is a primitive data type! Now I assume that the given program should run on a PC or server and not on an embedded system, because for this case a language like C would be much more suitable.
I think in this case it actually makes more sense to use strings, because the comfortable code of Java saves time in development and makes the code more readable for other developers. Furthermore, the use of strings could lead to the need for additional methods to convert values into a format that can be read by the later program. However, this would then cancel out any performance advantage.
Last but not least, one could refer to an interesting example at this point. If you export an Excel table in CSV format, files with long strings will be constructed there as well. And even there, any loading times are not disturbing for the end user (in my opinion)...

Gujarati text in Java String

I have Gujarati Bible and trying to insert each verse in MySQL database using parser written in Java. When I assign Gujarati text to Java String variable it shows junks in debug.
E.g. This is my Gujarati text
હે યહોવા તું મારો દેવ છે;
I assign it to Java String variable as shown below
verse._verseText = "હે યહોવા તું મારો દેવ છે;";
What i see in debug window is all junk characters. Any help is appreciated. If need more information let me know and I will provide as and when asked.
UPDATE
Pasting my parser code here
private Boolean Insert(String _text)
{
BibleVerse verse = new BibleVerse();
String[] data = _text.split("\\|");
try
{
if (data[0].equals(bookName) || bookName.equals("All"))
{
verse._Version = "Gujarati";
verse._book = data[0];
verse._chapter = Integer.parseInt(data[1]);
verse._verse = Integer.parseInt(data[2]);
verse._verseText = new String(data[3].getBytes(), "UTF-8");
_bibleDatabase.Insert(verse);
pcs.firePropertyChange("logupdate", null, data[0] + " " + data[1] + "," + data[2] + " - INSERTED.");
}
else
{
pcs.firePropertyChange("logupdate", null, data[0] + " " + data[1] + "," + data[2] + " - SKIPPED.");
}
return true;
}
catch(Exception e)
{
pcs.firePropertyChange("logupdate", null, "ERROR : " + e.getMessage());
return false;
}
}
Here is the sample line from the text file
Isaiah|25|1|હે યહોવા તું મારો દેવ છે; હું તને મોટો માનીશ, હું તારા નામની સ્તુતિ કરીશ; કેમકે તેં અદભુત કાર્યો કર્યાં છે, તેં વિશ્વાસુપણે તથા સત્યતાથી પુરાતન સંકલ્પો પાર પાડ્યા છે.
UPDATE
Here is the code where I open & read file.
try
{
FileReader _file = new FileReader(this._filename);
_bufferedReader = new BufferedReader(_file);
SwingWorker parseWorker = new SwingWorker()
{
#Override
protected Object doInBackground() throws Exception
{
String line;
String[] data;
int lineno=0;
BibleVerse verse = new BibleVerse();
while ((line = _bufferedReader.readLine()) != null)
{
++lineno;
pcs.firePropertyChange("pgbupdate", null, lineno);
Insert(line);
}
_bufferedReader.close();
return null;
}
#Override
protected void done()
{
pcs.firePropertyChange("logupdate", null, "Parsing complete.");
}
};
parseWorker.execute();
}
catch (Exception e)
{
pcs.firePropertyChange("logupdate", null, "ERROR : " + e.getMessage());
}

The problem is this:
FileReader _file = new FileReader(this._filename);
This reads the file using the platform's default charset. If your data file is not encoded in that charset, you will get incorrect characters.
On Windows, the default charset is almost always UTF-16LE. On most other systems, it's UTF-8.
The easiest solution is to find out the actual encoding of your data file, so you can specify it explicitly in the code. The encoding of a file can be determined with the file command on Unix and Linux systems. In Windows, you may need to examine it with a binary editor, or install something like Cygwin, which has a file command of its own.
Once you know what it is, you should pass it explicitly to the construction of your Reader:
// Replace "UTF-8" with the actual encoding of your data file (if it's not UTF-8).
Reader _file = new InputStreamReader(new FileInputStream(this._filename), "UTF-8");
Once you've done that, there is no reason for any other part of your code to concern itself with bytes. You should replace this:
verse._verseText = new String(data[3].getBytes(), "UTF-8");
with this:
verse._verseText = data[3];

how to inject chinese characters using javascript?
not quite the same problem, but I think the same solution may work in this case.
If the script is inline (in the HTML file), then it's using the
encoding of the HTML file and you won't have an issue.
If the script is loaded from another file:
Your text editor must save the file in an appropriate encoding such as
utf-8 (it's probably doing this already if you're able to save it,
close it, and reopen it with the characters still displaying
correctly) Your web server must serve the file with the right http
header specifying that it's utf-8 (or whatever the enocding happens to
be, as determined by your text editor settings). Here's an example for
how to do this with php: Set http header to utf-8 php If you can't
have your webserver do this, try to set the charset attribute on your
script tag (e.g. > I tried to see what the spec said should happen
in the case of mismatching charsets defined by the tag and the http
headers, but couldn't find anything concrete, so just test and see if
it helps. If that doesn't work, place your script inline

It looks like if you want to store Gujarati text in Java string, you need to use unicode characters. See this: http://jrgraphix.net/r/Unicode/0A80-0AFF
So for example the first Gujarati character:
char example = '0A80';
String result = Character.toString((char)example);

Modifying or adding new tags to a jpg (Metadata) with iim4j

I am trying to modify or add new tags with the iim4j library, but its documentation is poor. I've been searching for examples on the internet and didn't found anything so far. I only got the examples the library offers. There's one example about reading the Metadata and I can get the title, description and tags of the image (the ones I care). I know the library manage these info as DataSet and DataSetInfo so I tried to create new instances of these objects for the info I want to add but I have no results so far.
This is the code for reading the IPTC section of a jpeg file:
public static void dump(File file) throws Exception {
System.out.println("IPTC segment for " + file);
IIMFile iimFile = new IIMFile();
IIMReader reader = new IIMReader(new JPEGIIMInputStream(new FileIIMInputStream(file)), new IIMDataSetInfoFactory());
iimFile.readFrom(reader, 20);
for (Iterator i = iimFile.getDataSets().iterator(); i.hasNext();) {
DataSet ds = (DataSet) i.next();
Object value = ds.getValue();
if (value instanceof byte[]) {
value = "<bytes " + ((byte[]) value).length + ">";
}
DataSetInfo info = ds.getInfo();
if (info instanceof DefaultDataSetInfo)
{
System.out.println("-----------");
System.out.println("Number : " + info.getDataSetNumber());
System.out.println("Name :" + info.getName());
Serializer serializador=info.getSerializer();
if(serializador instanceof StringSerializer)
System.out.println("Serializer :" + serializador.toString());
System.out.println("Repeat : " + info.isRepeatable());
System.out.println("----------");
}
System.out.println(info.toString() + " " + info.getName() + ": " + value);
}
reader.close();
}
I can obtain all the info I need and also I know that kind of objects I must use to add new info on another files. So i tried this one to add a new tag:
String tagToAdd="Tag to add";
int size= tagToAdd.length();
DefaultDataSetInfo valueTag=new DefaultDataSetInfo(537,"Keywords", new StringSerializer(size+ ""),true);
DefaultDataSet dataSet=new DefaultDataSet(valueTag, tagToAdd.getBytes());
iimFile.add(dataSet);
and tried this one to write a new file with a tag:
String tagToAdd="Tag to add";
int size= tagToAdd.length();
DefaultDataSetInfo valueTag=new DefaultDataSetInfo(537,"Keywords", new StringSerializer(size+ ""),true);
DefaultDataSet dataSet=new DefaultDataSet(valueTag, tagToAdd.getBytes());
IIMWriter escritor= new IIMWriter(new DefaultIIMOutputStream(file));
escritor.write(dataSet);
escritor.close();
Tags weren't add. I need some help with this library and its use. Thanks

From my own testing, it seems that IIMWriter is only intended to be used to write the IPTC metadata to a file, and nothing else. Needless to say, that does not make for a valid JPEG file. What you want is to take an existing JPEG file, and generate a new file with the same data, except for the IPTC metadata that should be removed (if any) and replaced with the content of your IIMFile.
That's exactly what JPEGUtil.insertIIMIntoJPEG is for. What you need to do is something like this (where newFile is a File insatnce pointing to where you want to save the modified JPEG):
// 1. Read original IIM
IIMReader reader = new IIMReader(new JPEGIIMInputStream(new FileIIMInputStream(file)), new IIMDataSetInfoFactory());
iimFile.readFrom(reader, 20);
// 2. Add dataset
DataSet dataSet = ... // Whatever you want
iimFile.add(dataSet);
// 3. Create new copy of JPEG file new IIM
try (
InpuStream in = new BufferedInputStream(new FileInputStream(file));
OutputStream out = new BufferedOutputStream(new FileOutputStream(newFile))
) {
JPEGUtil.insertIIMIntoJPEG(out, iimFile, in)
}

How Can I Read The Next Row From A CSV Data Set Config In JMeter?

I am in the process of creating a test place in JMeter which visits a random amount of pages (from 2 - 10), whose URLs are to be fetched from a CSV Data Set. I have created the CSV Data Set and the samplers which are working fine, except that only one row is read from the Data Set per thread, which is not as a I need - I want a new row to be read after the sampler has completed (or before, I'm not fussed).
I saw that this question is very similar and the solution was to use the Raw Data Source Pre-Processor, which does work but requires arduous alterations to the file in question (adding chunk sizes before each line), which is a bit of a pain when the file is about 500 lines long.
Is there a way I can set the CSV Data Set to advance to the next row on reading, or use some post or pre processor, such as beanshell, in order to do this? I have seen people state that CSVRead can do this, but that states that access is per-thread, which would be no good for me.
As a side note - ultimately all I want to do is access a random line in the file which gets passed to a HTTP sampler, if there is an easier or better way to do this I'm open to suggestions.

You can possibly use for this beanshell (= java) code executed from BeanShell Sampler / BeanShell PostProcessor / BeanShell PreProcessor.
The following code will read all the lines from your file and then select single random:
import java.text.*;
import java.io.*;
import java.util.*;
String [] params = Parameters.split(",");
String csvTest = params[0];
String csvDir = params[0];
ArrayList strList = new ArrayList();
try {
File file = new File(System.getProperty("user.dir") + File.separator + csvDir + File.separator + csvTest);
if (!file.exists()) {
throw new Exception ("ERROR: file " + csvTest + " not found in " + csvDir + " directory.");
}
BufferedReader bufRdr = new BufferedReader(new FileReader(file));
String line = null;
while((line = bufRdr.readLine()) != null) {
strList.add(line);
}
bufRdr.close();
Random rnd = new java.util.Random();
vars.put("csvUrl",strList.get(rnd.nextInt(strList.size())));
}
catch (Exception ex) {
IsSuccess = false;
log.error(ex.getMessage());
System.err.println(ex.getMessage());
}
catch (Throwable thex) {
System.err.println(thex.getMessage());
}
Then you can access extracted URL via variable (${csvUrl} in this example).
I doubt only that reading full file on each iteration (if you have to execute this in loop) is good solution from performance point of view.

How to Clear variables from JTextFields without closing whole program?

I have created a text file in which to store some variables which are taken from text fields. But in order to submit new variables to this text file, I need to close my program and reopen it. The dispose(); command closes the JFrame taking me to my main menu but upon opening the menu again and submitting different values, the values from the previous time have been resubmitted. Is there a simple way to amend this?
Here is my write to .txt code:
public class writeto {
static String data = AddProperty.inputdata;
BufferedWriter out;
public writeto(){
try{
out = new BufferedWriter(new FileWriter("writeto.txt", true));
out.write(data);
out.newLine();
out.close();
}catch(IOException e){
System.out.println("you have an error" + e);
}
}
}
and where the method is called in my addproperty class
submitproperty.addActionListener(new ActionListener()
{
public void actionPerformed(ActionEvent e)
{
housenumber1 = houseNumber.getText();
streetname1 = streetName.getText();
town1 = town.getText();
postcode1 = postcode.getText();
beds1 = beds.getText();
price1 = price.getText();
type1 = type.getText();
inputdata = housenumber1 + " " + streetname1 + " " + town1 + " " +
postcode1 +" " + beds1 + " " + price1 + " " + type1;
writeto write = new writeto();
dispose();
}
});
}

From your menu you should always create a new JFrame with new widgets (like text fields). A new text field has no content, if you show a text field again, it will still display it's previous content.
Additional remarks:
Please use standard naming conventions - not only when you show code to others. In your case: class names shall start with a capital letter, camel-case notation is preferred (writeto -> WriteTo)
The writeto class abuses the constructor. The code in your constructor does not create an writeto object but dumps some strings to a file. Put this kind of code to a method, not to a constructor.
The BufferedWriter will not be closed if an exception occurs. Look around at stackoverflow, a lot of questions/answers show the correct io-closeing pattern
disposing the jframe is a risk - the code is executed after pressing a button (correct?), inside a method on a button that is displayed on the frame (correct?). In that case the button may be disposed while a method on the button object is still executed.. Try setVisible(false) if you just want to hide the JFrame (like "close the dialog")

You would benefit greatly from using a database as opposed to a text file. Further your question displays a fundamental lack of knowledge of not only Swing, but basic CRUD (Create, Read, Update, Delete) functionality.
To answer your question you can clear your text field with textField1.setText("");
I would read up on using a database for storing data. It will make life much easier for you.