Index Lucene: How to get PointValues terms for LongPoint field - java

I'm looking for resolution how to get terms for indexed PointValues by field name. For String it is very simple (my sample code):
IndexReader reader = caseIndexer.getIndexReader();
Fields fields = MultiFields.getFields(reader);
Iterator<String> names = fields.iterator();
Map<String, Terms> map = new HashMap<>();
while (names.hasNext()) {
String name = names.next();
// logger.info("->>fieldName: {}", name);
Terms terms = fields.terms(name);
map.put(name, terms);
TermsEnum termsEnum = terms.iterator();
BytesRef text;
while ((text = termsEnum.next()) != null) {
System.out.println("field=" + name + "; text=" + text.utf8ToString());
}
}
I know how to get simple statistics like max and min value (sample code):
List<FieldInfo> allFields = new ArrayList<>();
for (LeafReaderContext ctx : reader.leaves()) {
LeafReader lr = ctx.reader();
Iterator<FieldInfo> infos = lr.getFieldInfos().iterator();
PointValues values = lr.getPointValues();
while (infos.hasNext()) {
FieldInfo info = infos.next();
allFields.add(info);
if (DocValuesType.SORTED_NUMERIC.equals(info.getDocValuesType())) {
final int numDimensions = values.getNumDimensions(info.name);
final int numBytesPerDimension = values.getBytesPerDimension(info.name);
byte[] leafMinValue = values.getMinPackedValue(info.name);
long size = values.size(info.name);
byte[] leafMaxValue = values.getMaxPackedValue(info.name);
long minValueLong = NumericUtils.sortableBytesToLong(leafMinValue, 0);
long maxValueLong = NumericUtils.sortableBytesToLong(leafMaxValue, 0);
double minValueDouble = NumericUtils.sortableLongToDouble(minValueLong);
System.out.println("field=" + info.name + "; minValueLong=" + minValueLong + "; maxValueLong="
+ maxValueLong + "; minValueDouble=" + minValueDouble + "; numDimensions=" + numDimensions
+ "; numBytesPerDimension=" + numBytesPerDimension + "; size=" + size);
}
}
}
but how to get terms for points?

If you need to get the origin value of a LongPoint Field you need to store the value into a dedicated StoredField as described in LongPoint Field JavaDoc.
In general: Point (range/exact use cases) and DocValues (sorting use cases) fields are used for specific internal lucene usage and the values are not usable for search return values.

Related

I'm having trouble setting the value of a variable that will update my test

My problem is in the part where I'm doing the "if/else" conditions, when I call the function that will perform the comparisons and will define if the test passed or not and will send some information, I'm receiving null.
Problems are among the asterisks. If anyone can help me
This is my code :
public static void fxSpot_GBP_JPY(TradeData data, TradeData output) throws Exception {
if (data == null) {
fail("The input data object was not correctly filled");
}
if (output == null) {
fail("The output data object was not correctly filled");
}
//Used to set the comment, the status and update to JIRA
FieldsJSON fields = new FieldsJSON();
String assertionError = "";
List<String> inputs = new ArrayList<String>();
List<String> outputs = new ArrayList<String>();
String newDate = Utils.formatTimeZoneMinute(data.getTradeDate());
String asOfTrade = Utils.formatTimeZoneMinute(data.getAsOfTradeDate());
String executionDate = Utils.formatTimeZoneMinute(output.getExecutionDateTime());
try {
//Add the data in the list
inputs.add(data.getTransactionNumber()); outputs.add(output.getBloombergId());
inputs.add(newDate); outputs.add(output.getTradeDate());
inputs.add(asOfTrade); outputs.add(executionDate);
inputs.add(data.getSettlementDate()); outputs.add(output.getValueDate());
inputs.add(data.getTradeAmount()); outputs.add(output.getAmount2());
inputs.add(data.getCustomerAccountCounterparty()); outputs.add(output.getMiPartyId());
inputs.add(data.getPrincipalLoanAmount()); outputs.add(output.getAmount());
inputs.add(data.getSecurityPrice()); outputs.add(output.getRate());
inputs.add(data.getISOCodeOf1stCurrency()); outputs.add("BRL");//output.getCurrency2()
inputs.add(data.getISOCodeOf2ndCurrency()); outputs.add(output.getCurrency1());
//Compare values
System.out.println("-------------------");
int y = 0;
int x = 0;
for(String input : inputs) {
for(String out : outputs) {
if(y == x) {
if(input.equals(out)) {
WriterCSV.setOk("Ok");
**String comment = input + " = " + out;
fields.setComment(comment);
fields.setStatus("PASS");**
System.out.println("ok - " + input + " = " + out);
}else {
WriterCSV.setOk("not Ok");
**String comment = input + " = " + out;
fields.setComment(comment);
fields.setStatus("FAIL");**
System.out.println("not Ok - " + input + " = " + out);
}
}
x = x+1; // count of the list of output
}
y = y+1; // count of the list of inputs
x = 0; // reset to 0 the count of outputs
}
// evidence with the name and value of fields compared
WriterCSV.reportSpot_CSV(data,output);
}
Here is my test:
#Test
#Tag("compare")
public void CompareSpot() throws Exception {
//Create a list to read the CSVfile
List<DTOTradeData> dto;
//Used to get the TradeData of list dto.
DTOTradeData dtd = new DTOTradeData();
// Read a csvFile and return a list with the values to new xml
dto = CSVReader.readCSV("spot.csv");
//The xpath of xml
FileDriverSpot spot = new FileDriverSpot();
FileDriver output = new FileDriverSpotOutput();
FieldsJSON fields = new FieldsJSON();
//new xml = dataInput and the outputFile = dataOutput
TradeData dataInput = new TradeData();
TradeData dataOutput = new TradeData();
for (int i = 0; i < dto.size(); i++) {
dtd = dto.get(i); // get TradeData
dtd.getTradeData().setDriver(spot); // set the driver
if (fileExist(Setup.xmlPath + dtd.getInputFile() + ".xml")) {
dataInput = Reader.read(spot, Setup.xmlPath + dtd.getInputFile() + ".xml");
dataOutput = Reader.read(output, Setup.spotPath + dtd.getOutputFile());
try {
// make the comparison
**FunctionalTest.fxSpot_GBP_JPY(dataInput, dataOutput);**
}
catch(AssertionError e) {
String comment = e.toString();
fields.setComment(comment);
}
} else {
fail("The file: " + dtd.getTemplateFile()
+ " needs to go through the writing process before being compared.");
}
//Convert the file to base64
String inputData = UpdateTestStatus.convertToBase64(Setup.xmlPath + dtd.getInputFile() + ".xml");
String outputData = UpdateTestStatus.convertToBase64(Setup.spotPath + dtd.getOutputFile());
String evidenceCompared = UpdateTestStatus.convertToBase64(Setup.reportPath+"ReportSpot.csv");
System.out.println(UpdateTestStatus.updateTestRun(**fields.getStatus(),fields.getComment()**,
inputData,dtd.getInputFile()+ ".xml", //data of the XML and the name of the file
outputData,dtd.getOutputFile(),
evidenceCompared,"ReportSpot.csv",
Setup.testExec, dtd.getJiraId()).asString()); // ID testExecution and ID of
}
}
The test and the code under test each create a separate instance of FieldsJSON. Data set in one instance will not be visible in the other (unless the data is declared static, in which case there's no need to create instances).
You can fix this by using a single instance, either passed to the fxSpot_GBP_JPY method from the test, or returned from that method to the test.

Changing my own tags to data from XML

I have a text editor where user can put text like this:
[[PAYMENTS_N]] You have to pay [[amount]]$ before [[date]].[[/PAYMENTS_N]]
[[PAYMENTS_Z]] You didn't pay [[debt]]$ on time: [[ddate]].[[/PAYMENTS_Z]]
And there are XML files that look like this:
<DATA_NZ><USER><ID>12345</ID><COMP_NZ><COMP>
<STATUS>Z</STATUS>
<AMOUNT_NZ>128.01</AMOUNT_NZ>
<DATE_NZ>28.05.2015</DATE_NZ>
<STATUS>N</STATUS>
<AMOUNT_NZ>12.32</AMOUNT_NZ>
<DATE_NZ>21.09.2015</DATE_NZ>
<STATUS>N</STATUS>
<AMOUNT_NZ>12.32</AMOUNT_NZ>
<DATE_NZ>20.10.2015</DATE_NZ>
</COMP></COMP_NZ></USER></DATA_NZ>
Now I want to change the text from editor into something like this:
Remember about your next payments:
You have to pay 12.32$ before 21.09.2015.
You have to pay 12.32$ before 20.10.2015.
You didn't pay 128.01$ on time: 28.05.2015.
And I have this code:
public void parseTags(String content) {
String nOpenTag, nCloseTag;
String zOpenTag, zCloseTag;
String dnTag, knTag, dzTag, kzTag;
nOpenTag = "[[PAYMENTS_N]]";
nCloseTag = "[[/PAYMENTS_N]]";
zOpenTag = "[[PAYMENTS_Z]]";
zCloseTag = "[[/PAYMENTS_Z]]";
dnTag = "[[amount]]";
knTag = "[[date]]";
dzTag = "[[debt]]";
kzTag = "[[ddate]]";
String textToExtract = "";
String textToExtract = "";
String textToReplace = "";
String parsedContent = content;//; = content;
if (content.contains(nOpenTag) || content.contains(zOpenTag)) {
for (int i = 0; i < getXMLContainer("SZUK_XM4.XML").getUsers().size(); i++) {
if (getXMLContainer("SZUK_XM4.XML").getUsers().get(i) != null) { //
contact = consentService.findByContactTypeAndIdNumber(ContactType.EMAIL, getXMLContainer("SZUK_XM4.XML").getUsers().get(i).getIdNumber());
payments = getXMLContainer("SZUK_XM4.XML").getUsers().get(i).getPaymentContainer().getPayments();
if (contact.size() > 0) {
StringBuilder strB = new StringBuilder();
StringBuilder strB2 = new StringBuilder();
String textToReplace;
for (int c = 0; c < contact.size(); c++) {
strB.setLength(0);
for (int p = 0; p < payments.size(); p++) {
if (payments.get(p).getStatus().contains("N")) {
textToExtract = content.substring(content.indexOf(nOpenTag) + nOpenTag.length(), content.indexOf(nCloseTag));
textToReplace = textToExtract.replace(dnTag, "Day " + payments.get(p).getPaymentDate());
textToReplace = textToReplace.replace(knTag, "Amount " + payments.get(p).getPaymentAmount());
strB.append(textToReplace);
parsedContent = content.replace(textToExtract, strB.toString());
}
if (payments.get(p).getStatus().contains("Z")) {
textToExtract = parsedContent.substring(parsedContent.indexOf(zOpenTag) + zOpenTag.length(), parsedContent.indexOf(zCloseTag));
textToReplace = textToExtract.replace(dzTag, "Day " + payments.get(p).getPaymentDate());
textToReplace = textToReplace.replace(kzTag, "Amount " + payments.get(p).getPaymentAmount());
strB2.append(textToReplace);
parsedContent = parsedContent.replace(textToExtract, strB2.toString());
}
}
mailService.sendTemplateMail(contact.get(c).getContact(), parsedContent);
}
}
}
}
}
}
This code is supposed to find in the database user with given ID from XML.
If user exists then it reads all the data between and replaces [[TAGS]] from the text editor. After everything's done, it should send an email with replaced text.
The problem is that the text it sends isn't correctly replaced. It gives
Remember about your next payments: You have to pay 12.32$ before
21.09.2015. You have to pay 12.32$ before 20.10.2015.
You didn't pay [[debt]]$ on time: [[ddate]].
See the difference in the last line?
I'm using JAXB to read XML, if it matters.
Hope you can help me! Thanks.
replace:
parsedContent = content.replace(textToExtract, strB.toString());
with:
parsedContent = parsedContent.replace(textToExtract, strB.toString());

Sorting arrayList from resultset

I'm working on a project that involves 2 separate db to generate the report. The result of one is passed into the other query and a final report is generated. Now, i wan to sort the final report but having issues with it. Java constructor for "java.util.Arrays" with arguments "" not found.
var fist = new java.util.Arrays();
var list = new java.util.ArrayList();
var gist = new java.util.ArrayList();
var arr = '';
var dbConn = DatabaseConnectionFactory.createDatabaseConnection('Postgres connection');
var result3 = dbConn.executeCachedQuery(...)
while (result3.next()) {
var value1 = result3.getString(1);
var value2 = result3.getString(2);
var dbConn3 = DatabaseConnectionFactory.createDatabaseConnection('Oracle connection'));
var result2 = dbConn3.executeCachedQuery("SELECT name FROM producer WHERE send = '" + value1 + "' AND code = '" + value2 + "' ORDER BY name")
while (result2.next()) {
var sending = result2.getString(1);
}
dbConn3.close();
if (sending != undefined) {
arr += gist.add(sending);
arr += gist.add(value1);
arr += gist.add(value2);
arr += gist.add(result3.getString(3));
fist.add(arr);
}
}
Arrays.sort(fist); //i'm thinking this should sort it before displaying it
while (fist.next()) {
xmlMs += "<tr>"
xmlMs += "<td>" + sending + "</td>";
xmlMs += "<td>" + value1 + "</td>";
xmlMs += "<td>" + value2 + "</td>";
xmlMs += "<td align='center'>" + result3.getString(3) + "</td>";
xmlMs += "</tr>";
}
Well yes, your compiler is telling you that var fist = new java.util.Arrays(); is invalid, java.util.Arrays doesn't provide a public no-args constructor. Did you by chance mean ArrayList instead of Arrays?
Arrays doesn't have a constructor defined in the source code, so new Arrays(); causes the compiler to scream at you.
If you want an array of something you use
type[] varname = new type[size];
Note that the [] is what makes it an Array.
Ex:
int[] x = new int[5];
That will hold an array of 5 ints.

How to Manipulate List to String and print in seperate row

I am trying to export the 4 columns with the below code.the last column organization is a List.
String appname = "abc";
String path = "//home/exportfile//";
String filename = path + "ApplicationExport-" + appname + ".txt";
String ret = "false";
QueryOptions ops = new QueryOptions();
Filter[] filters = new Filter[1];
filters[0] = Filter.eq("application.name", appname);
ops.add(filters);
List props = new ArrayList();
props.add("identity.name");
// Do search
Iterator it = context.search(Link.class, ops, props);
// Build file and export header row
BufferedWriter out = new BufferedWriter(new FileWriter(filename));
out.write("IdentityName,UserName,WorkforceID,Organization");
out.newLine();
// Iterate Search Results
if (it != null) {
while (it.hasNext()) {
// Get link and create object
Object[] record = it.next();
String identityName = (String) record[0];
Identity user = (Identity) context.getObject(Identity.class, identityName);
// Get Identity attributes for export
String workforceid = (String) user.getAttribute("workforceID");
// Get application attributes for export
String userid = "";
List links = user.getLinks();
if (links != null) {
Iterator lit = links.iterator();
while (lit.hasNext()) {
Link l = lit.next();
String lname = l.getApplicationName();
if (lname.equalsIgnoreCase(appname)) {
userid = (String) l.getAttribute("User Name");
List organizations = l.getAttribute("Organization");
StringBuilder sb = new StringBuilder();
String listItemsSeparator = ",";
for (Object organization : organizations) {
sb.append(organization.toString());
sb.append(listItemsSeparator);
}
org = sb.toString().trim();
}
}
}
// Output file
out.write(identityName + "," + userid + "," + workforceid + "," + org);
out.newLine();
out.flush();
}
ret = "true";
}
// Close file and return
out.close();
return ret;
the output of the above code will be.for ex:
IdentityName,UserName,WorkforceID,Organization
dthomas,dthomas001,12345,Finance,HR
How do i get the output in below fashion
IdentityName,UserName,WorkforceID,Organization
dthomas,dthomas001,12345,Finance
dthomas,dthomas001,12345,HR
what and where i need to change the code?
You'll have to write one line to the file for each organization. So, basically, do not concatenate all organizations for a user with the string builder and move the output statements into the for loop that iterates through the organizations.
But it's difficult to provide a working example, because you're code you've shown doesn't compile yet...
This should bring you somewhat closer to the solution:
if (links != null) {
Iterator lit = links.iterator();
while (lit.hasNext()) {
Link l = lit.next();
String lname = l.getApplicationName();
if (lname.equalsIgnoreCase(appname)) {
userid = (String) l.getAttribute("User Name");
List organizations = l.getAttribute("Organization");
for (Object organization : organizations) {
// Output file
out.write(identityName + "," + userid + "," + workforceid + "," + organization);
out.newLine();
out.flush();
}
}
}
}
Remove this innermost for block and associated variables:
StringBuilder sb = new StringBuilder();
for (Object organization : organizations)
{
sb.append(organization.toString());
sb.append(listItemsSeparator);
}
org = sb.toString().trim();
Move the declaration of organizations outside the if (it != null) { block:
// Get application attributes for export
String userid = "";
List organizations = null;
List links = user.getLinks();
if (it != null)
{
Iterator lit = links.iterator();
while (lit.hasNext())
{
Link l = lit.next();
String lname = l.getApplicationName();
if (lname.equalsIgnoreCase(appname))
{
userid = (String) l.getAttribute("User Name");
organizations = l.getAttribute("Organization");
And then change this file output code:
// Output file
out.write(identityName + "," + userid + "," + workforceid + "," + org);
out.newLine();
To this:
// Output file
for (Object organization : organizations)
{
out.write(identityName + "," + userid + "," + workforceid + "," + organization.toString());
out.newLine();
}

Regex Issue With Multiple Groups

I'm trying to create a regex pattern to match the lines in the following format:
field[bii] = float4:.4f_degree // Galactic Latitude
field[class] = int2 (index) // Browse Object Classification
field[dec] = float8:.4f_degree (key) // Declination
field[name] = char20 (index) // Object Designation
field[dircos1] = float8 // 1st Directional Cosine
I came up with this pattern, which seemed to work, then suddenly seemed NOT to work:
field\[(.*)\] = (float|int|char)([0-9]|[1-9][0-9]).*(:(\.([0-9])))
Here is the code I'm trying to use (edit: provided full method instead of excerpt):
private static Map<String, String> createColumnMap(String filename) {
// create a linked hashmap mapping field names to their column types. Use LHM because I'm picky and
// would prefer to preserve the order
Map<String, String> columnMap = new LinkedHashMap<String, String>();
// define the regex patterns
Pattern columnNamePattern = Pattern.compile(columnNameRegexPattern);
try {
Scanner scanner = new Scanner(new FileInputStream(filename));
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
if (line.indexOf("field[") != -1) {
// get the field name
Matcher fieldNameMatcher = columnNamePattern.matcher(line);
String fieldName = null;
if (fieldNameMatcher.find()) {
fieldName = fieldNameMatcher.group(1);
}
String columnName = null;
String columnType = null;
String columnPrecision = null;
String columnScale = null;
//Pattern columnTypePattern = Pattern.compile(".*(float|int|char)([0-9]|[1-9][0-9])");
Pattern columnTypePattern = Pattern.compile("field\\[(.*)\\] = (float|int|char).*([0-9]|[1-9][0-9]).*(:(\\.([0-9])))");
Matcher columnTypeMatcher = columnTypePattern.matcher(line);
System.out.println(columnTypeMatcher.lookingAt());
if (columnTypeMatcher.lookingAt()) {
System.out.println(fieldName + ": " + columnTypeMatcher.groupCount());
int count = columnTypeMatcher.groupCount();
if (count > 1) {
columnName = columnTypeMatcher.group(1);
columnType = columnTypeMatcher.group(2);
}
if (count > 2) {
columnScale = columnTypeMatcher.group(3);
}
if (count >= 6) {
columnPrecision = columnTypeMatcher.group(6);
}
}
int precision = Integer.parseInt(columnPrecision);
int scale = Integer.parseInt(columnScale);
if (columnType.equals("int")) {
if (precision <= 4) {
columnMap.put(fieldName, "INTEGER");
} else {
columnMap.put(fieldName, "BIGINT");
}
} else if (columnType.equals("float")) {
if (columnPrecision==null) {
columnMap.put(fieldName,"DECIMAL(8,4)");
} else {
columnMap.put(fieldName,"DECIMAL(" + columnPrecision + "," + columnScale + ")");
}
} else {
columnMap.put(fieldName,"VARCHAR("+columnPrecision+")");
}
}
if (line.indexOf("<DATA>") != -1) {
scanner.close();
break;
}
}
scanner.close();
} catch (FileNotFoundException e) {
}
return columnMap;
}
When I get the groupCount from the Matcher object, it says there are 6 groups. However, they aren't matching the text, so I could definitely use some help... can anyone assist?
It's not entirely clear to me what you're after but I came up with the following pattern and it accepts all of your input examples:
field\\[(.*)\\] = (float|int|char)([1-9][0-9]?)?(:\\.([0-9]))?
using this code:
String columnName = null;
String columnType = null;
String columnPrecision = null;
String columnScale = null;
// Pattern columnTypePattern =
// Pattern.compile(".*(float|int|char)([0-9]|[1-9][0-9])");
// field\[(.*)\] = (float|int|char)([0-9]|[1-9][0-9]).*(:(\.([0-9])))
Pattern columnTypePattern = Pattern
.compile("field\\[(.*)\\] = (float|int|char)([1-9][0-9]?)?(:\\.([0-9]))?");
Matcher columnTypeMatcher = columnTypePattern.matcher(line);
boolean match = columnTypeMatcher.lookingAt();
System.out.println("Match: " + match);
if (match) {
int count = columnTypeMatcher.groupCount();
if (count > 1) {
columnName = columnTypeMatcher.group(1);
columnType = columnTypeMatcher.group(2);
}
if (count > 2) {
columnScale = columnTypeMatcher.group(3);
}
if (count > 4) {
columnPrecision = columnTypeMatcher.group(5);
}
System.out.println("Name=" + columnName + "; Type=" + columnType + "; Scale=" + columnScale + "; Precision=" + columnPrecision);
}
I think the problem with your regex was it needed to make the scale and precision optional.
field\[(.*)\] = (float|int|char)([0-9]|[1-9][0-9]).*(:(\.([0-9])))
The .* is overly broad, and there is a lot of redundancy in ([0-9]|[1-9][0-9]), and I think the parenthetical group that starts with : and preceding .* should be optional.
After removing all the ambiguity, I get
field\[([^\]]*)\] = (float|int|char)(0|[1-9][0-9]+)(?:[^:]*(:(\.([0-9]+))))?

Categories

Resources