I have multiple text files and an a database table. The database contains a fixed number of entries and the text files have more entries.
For Example:
------------text1.txt-----------
44-CAT-IV-CORE 626518 T19P45
44-CAT-IV-OUTER 626522 LB0N08
44-CAT-IV-EXTER 626956 AG8N15
44-CAT-IV-DOUT 626965 PQ7715
------------text2.txt-----------
44-CAT-IV-CORE 626518 T19P50
44-CAT-IV-OUTER 626522 LB0N08
44-CAT-IV-EXTER 626956 AG8N15
44-CAT-IV-DOUT 626965 PQ2718
Many files like this....
The database looks like:
|unit| |value| |name-part| |version|
|CAT-IV| |626518| |CAT IV CORE| |T19P43|
|CAT-IV| |626522| |CAT IV OUTER| |LB0N08|
|CAT-IV| |626956| |CAT IV EXTER| |AG8N15|
I want to get those part names and values from the text files whose value or version or both do not match on the database(only for those parts where the name exist in the database, like here we need to ignore CAT-IV-DOUT as it is not on the database)
I tried loading the database values to a text file and then comparing against text files, however it seems inefficient. Is there a better way to do this ?
put all lines of the files in an array (file();)
go trough the databasetable and compare each arrayelement with the current data from the database.
Related
In my anylogic model my agents receive their parameters from a database table, which is based on an Excel file. In the Excel file, each cell has its own code stored, so each time I open the file, the cell values change.
I would like to have that with each automatic run of my model the Excel file is read in again as the database table (i.e. the parameter values of the agents change).
In the "Parameter Variation Experiment" I entered this code under "after iteration":
String tempString = excelFile_DatabasisLinks.getCellStringValue(1, rowCounter, 1);
ModelDatabase modelDB = getEngine().getModelDatabase();
Database myNewFile = new Database(this, "rohdaten2", tempString);
modelDB.importFromExternalDB(myNewFile.getConnection(), "Rohdaten", "rohdaten", true, false);
rowCounter += 1;
I have the code form this anylogic help page. Using a variable to be able to change the path of the file (i.e. the file) seems to work (anylogic doesn't throw an error).
In the currently used dummy model, the agents receive their parameters at the source.
At the sink, the parameter values are written via collections into another excel (results) file.
I put obvious pattern into my data files, to see if the data changes, but I always receive the same excel file in my results file.
I read that anylogic copies the excel tables to its temporary files to make simulation runs faster. I hoped the code above would be a workaround, but it is not.
I'm grateful for any suggestions how to make this work!
I could not find out what is wrong with the above code or how to get it work.
However, I found a workaround using the "excelFile "-Block.
In the main agent (the agent where all other agents live) in the agent actions in "on startup":
excelFile.readFile();
ensures that the excel file is updated before each run. The parameters are added via
agent.set_<parametername>(excelFile.getCellBooleanValue( sheet number, row number, column number)
"on exit" in the source.
I hope this helps everyone with a similar problem.
I am uploading files in a list into S3 using uploadFileList()
So this API takes list (records) as parameter like below
MultipleFileUpload xfer = tm.uploadFileList(bucketName, "TEST",new File(fileLocation), records);
The records in a list like this
21564_114762642_ANA_9ECB7C98-C2D7-428A-B6AD-7A6C62E1A7BE_App.xml.gz
21224_114762642_ANA_9ECB7C98-C2D7-428A-B6AD-7A6C62E1A7BE_App.xml.gz
20780_114762642_ANA_9ECB7C98-C2D7-428A-B6AD-7A6C62E1A7BE_App.xml.gz
20407_114762642_ANA_9ECB7C98-C2D7-428A-B6AD-7A6C62E1A7BE_App.xml.gz
This is working fine as of now .
Now i need to add prefix in the API as the first four digit of the file name like 21564 will be the prefix for the first file
So to do this i have to iterate over the list and add file by file but that will slow down the upload into S3 compare to uploading list .
Is there anyway to add prefix while uploading list into S3 and the files in the list are random but pattern is fix ?
See the S3 documentation about object keys. Because S3 buckets are flat (not a filesystem hierarchy), you can tell Amazon the key prefix to use for your uploaded files, thus grouping them all under the same prefix. For example, I could supply "MovieReviews/" as a prefix for a list of files and the resulting object keys in S3 would begin with that. Some tools understand the slashes and allow you to browse your S3 bucket as a directory hierarchy.
In your case, if the files should use the first N characters as a grouping key, then you can upload them in batches by first grouping according to that substring, e.g. fileList.stream().collect(Collectors.groupingBy(s -> s.substring(0, N)))
I have the below class which is using google guava map as you can see below , now the only query is that in it a list is been created as you can see but i want to keep this list in a properties file seprately since later on many new items will be added and deleted in this list so i want developer to only change the properties file
Below is the code
final Table<String, String, List<String>> values = HashBasedTable.create();
values.put("Tbon", "cy", Lists.newArrayList("cat1","cat12","cat13","cat14"));
values.put("Tbon", "ype", Lists.newArrayList("rat1","rat2","rat3"));
for (Cell<String, String, List<String>> cell: values.cellSet()){
System.out.println(cell.getRowKey()+" "+cell.getColumnKey()+" "+cell.getValue());
}
now as you can see both the array list are hardcoded here "cat1","cat12","cat13","cat14" and "rat1","rat2","rat3" but i want to keep thsese array list in a properties file seprately so that it can be map against row key so the properties file content will be like lets say name of the properties file is abc.properties stored in my local computer C: drive
cy cat1
cy cat12
cy cat13
cy cat14
ype rat1
ype rat2
ype rat3
so please advise if i keep this mappings in a properties filethen how they will be get called in the data structure values
There is plenty of information on how to write property files on your disk on the internet, e.g. here: http://www.mkyong.com/java/java-properties-file-examples/
The default format assumes one value per key so you'll need some splitting and joining. Here's a simple answer to that issue: Multiple values in java.util.Properties
I am using below tag to query the item from DB. The item presents in DB but not showing up because A&M became as A&M instead of A&M. How to solve this?
<TEA>2720A 100 STATE A&M RD VRAD</TEA>
A backend java code queries the item from DB like 'select * from aa where tea=2720A 100 STATE A&M RD VRAD' and returns no record but it is present in DB like A&M. This is the exact issue, how to solve this?
Double encoding, your string is encoded twice.
First encoding A&M -> A&*M
Second encoding A&*M -> A&*amp*M
Check your code for this issue
Of course representing & in XML is done with &.
If just text is stored in the DB, for instance extracted fron between two tags <name>A&M</name>, then any XML API should give "A&M" to be stored.
If entire XML is stored in the DB, one should store it as-is: "<name>A&M</name>"
The problem arises only when String manipulations are done. Say
String xml = "<name>A&M</name>";
String name = xml.replaceFirst("^.*<name>(.*)</name>.*$", "$1");
name = StringEscapeUtils.unescapeXML(name);
Here apache StringEscapeUtils is used. Not unescaping makes trouble.
It probably goes wrong, when mixing extracted text (should-be-unescaped text) with XML manipulation (DOM). And again placing it in an XML structure. The XML APIs in general return values with the entities unescaped, and escape the XML characters <>&"' to entities.
Especially be careful when editing in HTML, that uses the same entity; not showing the actual characters. Here StringEscapeUtils.unescapeHtml4 comes into play.
I have 1 text file which contains numbers from 1 to 11644. Beside the numbers are the names of the xml files that i have in another folder. I have a total of 8466 xml files. I need to match the filename of all the xml files with the id in the text file and extract the value of the id out. All of the id are in random position. An example would be my first xml file id is 7025, which means it's id is 7025. I'm new to java so i really hope someone would enlighten me thanks.
The data structure for this is a map.
Read in the input file, and add each line to a java.util.HashMap<String, Integer>. The key should be the filename. The value should be the id. Thus, for each line, myMap.put(filename, id). Now, when you want to check the ID of a file, do myMap.get(filename). It will return the Integer ID of the file.