Develop Reference

How can you create an map or list of indexes of a substring position-Java?

I am parsing many lines from a text file. The file lines are fixed length width but depending on beginning of the line ex "0301...." the file data structure is split. there are lines example beginning with 11, 34 etc, and based on that the line is split differently.
Example: if start of line contains "03", then the line would be split on
name = line.substring(2, 10);
surname = line.substring(11, 21);
id = line.substring(22, 34);
adress = line.substring (35, 46);
Another Example: if start of line contains "24", then the line would be split on
name = line.substring(5, 15);
salary = line.substring(35, 51);
empid = line.substring(22, 34);
department = line.substring (35, 46);
So I have many substrings are added to many strings, then written to a new file in csv.
My question would be is there any easy method for storing the coordinates (indexes) of a substring and calling them later easier? Example
name = (2,10);
surname = (11,21);
...
etc.
Or probably any alternative of using substrings? thank you!

You could try something like this. I'll leave the bounds checking and optimization to you, but as a first pass...
public static void main( String[] args ) {
Map<String, Map<String,IndexDesignation>> substringMapping = new HashMap<>();
// Put all the designations of how to map here
substringMapping.put( "03", new HashMap<>());
substringMapping.get( "03" ).put( "name", new IndexDesignation(2,10));
substringMapping.get( "03" ).put( "surname", new IndexDesignation(11,21));
// This determines which mapping value to use
Map<String,IndexDesignation> indexDesignationMap = substringMapping.get(args[0].substring(0,2));
// This holds the results
Map<String, String> resultsMap = new HashMap<>();
// Make sure we actually have a map to use
if ( indexDesignationMap != null ) {
// Now take this particular map designation and turn it into the resulting map of name to values
for ( Map.Entry<String,IndexDesignation> mapEntry : indexDesignationMap.entrySet() ) {
resultsMap.put(mapEntry.getKey(), args[0].substring(mapEntry.getValue().startIndex,
mapEntry.getValue().endIndex));
}
}
// Print out the results (and you can assign to another object here as needed)
System.out.println( resultsMap );
}
// Could also just use a list of two elements instead of this
static class IndexDesignation {
int startIndex;
int endIndex;
public IndexDesignation( int startIndex, int endIndex ) {
this.startIndex = startIndex;
this.endIndex = endIndex;
}
}

Create a class called Line and store these objects rather than the string:
class Line {
int[] name;
int[] surname;
int[] id;
int[] address;
String line;
public Line(String line) {
this.line = line;
String startCode = line.substring(0, 3);
switch(startCode) {
case "03":
this.name = new int[]{2, 10};
this.surname = new int[]{11, 21};
this.id = new int[]{22, 34};
this.address = new int[]{35, 46};
break;
case "24":
// same thing with different indices
break;
// add more cases
}
}
public String getName() {
return this.line.substring(this.name[0], this.name[1]);
}
public String getSurname() {
return this.line.substring(this.surname[0], this.surname[1]);
}
public String getId() {
return this.line.substring(this.id[0], this.id[1]);
}
public String getAddress() {
return this.line.substring(this.address[0], this.address[1]);
}
}
Then:
String line = "03 ..."
Line parsed = new Line(line);
parsed.getName();
parsed.getSurname();
...
If you're going to retrieve the name, surname etc. multiple times from the Line object, you can even cache it the first time so that you're not calling substring multiple times

We can also use regex pattern and streams to achieve the results.
Say, we have a text file like this -
03SomeNameSomeSurname
24SomeName10000
The regex pattern has group name for assigning the attribute name to the parsed text. So, the pattern for the first line is -
^03(?<name>.{8})(?<surname>.{11})
The code is -
public static void main(String[] args) {
// Fixed Width File Lines
List<String> fileLines = List.of(
"03SomeNameSomeSurname",
"24SomeName10000"
);
// List all regex patterns for the specific file
List<Pattern> patternList = List.of(
Pattern.compile("^03(?<name>.{8})(?<surname>.{11})"), // Regex for String - 03SomeNameSomeSurname
Pattern.compile("^24(?<name>.{8})(?<salary>.{5})")); // Regex For String - 24SomeName10000
// Pattern for finding Group Names
Pattern groupNamePattern = Pattern.compile("\\?<([a-zA-Z0-9]*)>");
List<List<String>> output = fileLines.stream().map(
line -> patternList.stream() // Stream over the pattern list
.map(pattern -> pattern.matcher(line)) // Create a matcher for the fixed width line and regex pattern
.filter(matcher -> matcher.find()) // Filter matcher which matches correctly
.map( // Transform matcher results into String (Group Name = Matched Value
matcher ->
groupNamePattern.matcher(matcher.pattern().toString()).results() // Find Group Names for the regex pattern
.map(groupNameMatchResult -> groupNameMatchResult.group(1) + "=" + matcher.group(groupNameMatchResult.group(1))) // Transform into String (Group Name = Matched Value)
.collect(Collectors.joining(","))) // Join results delimited with ,
.collect(Collectors.toList())
).collect(Collectors.toList());
System.out.println(output);
}
The output result has parsed the attribute name and attribute value as a List of String.
[[name=SomeName,surname=SomeSurname], [name=SomeName,salary=10000]]

Splitting team names with scores separated by space and commas

I've multiple lines of game scores as input. The input is something like this.
Lions 1, FCAwesome 1
I'm currently Splitting the line based on either comma or space.
Charset charset = Charset.forName("US-ASCII");
String REGEX = ",?\\s+";
Pattern pattern = Pattern.compile(REGEX);
try(BufferedReader reader = Files.newBufferedReader(path, charset)){
int count = 0;
String line = null;
while((line = reader.readLine()) != null){
String[] arr = pattern.split(line);
}
This works fine for the provided input. However if the team name is has more than one word, my code breaks.
Lions 1, FC Awesome 1
How do I modify my REGEX to handle this case. FC Awesome still needs to be one team name.

Try splitting on space which
has comma before it (including that comma) - to separate team score pairs.
has digit after it - to separate team name and score,
So try with split(",\\s|\\s(?=\\d)")
If there is possible that some parts of team name can start with digit, we can specify our condition more. We can require from [space][digit] to either have after it comma or to be placed at the end of text.
split(",\\s|\\s(?=\\d+(?=,|$))")

try to split whole data by comma, then use getTeam method below
class Team {
String name;
int score;
public Team(String name, int score) {
this.name = name;
this.score = score;
}
#Override
public String toString() {
return this.name + ", " + this.score;
}
public static Team getTeam(String data) {
String score = "";
int i = data.length() - 1;
for (; Character.isDigit(data.charAt(i)); i--) {
score += data.charAt(i);
}
String name = data.substring(0, i);
return new Team(name, Integer.parseInt(new StringBuilder(score).reverse().toString()));
}
}
for example input is like this
LION_## 1234 OLD 5555 ,TEAM2345NAME NAME 123NAME 4444
first name is LION_## 1234 OLD and it's score is 5555
second name is TEAM2345NAME NAME 123NAME and 4444 is it's score
note: both contain numbers or special characters in their name and even space in score part.
now all i need is creating an instance of Team class.like below example:
String all_data = "LION_## 1234 OLD 5555 ,TEAM2345NAME NAME 123NAME 4444";
// spliting data by comma
String parts[] = all_data.split(",");
// calling getTeam method
Team t1 = Team.getTeam(parts[0]);
Team t2 = Team.getTeam(parts[1]);
then use it's fields, for example print them.
System.out.println(t1.name);
System.out.println(t2.score);

How to remove matched words from end of String

I want to remove the following words from end of String ‘PTE’, ‘LTD’, ‘PRIVATE’ and ‘LIMITED’
i tried the code but then i stuck. i tried this
String[] str = {"PTE", "LTD", "PRIVATE", "LIMITED"};
String company = "Basit LTD";
for(int i=0;i<str.length;i++) {
if (company.endsWith(str[i])) {
int position = company.lastIndexOf(str[i]);
company = company.substring(0, position);
}
}
System.out.println(company.replaceAll("\\s",""));
It worked. But suppose the company is Basit LIMITED PRIVATE LTD PTE or Basit LIMITED PRIVATE PTE LTD or any combination of four words in the end. Then the above code just remove the last name i.e., PTE or PRIVATE and so on, and the output is BasitLIMITEDPRIVATELTD.
I want output to be just Basit
How can i do it?
Thanks
---------------Edit---
Please note here the company name is just an example, it is not necessary that it is always the same. may be i have name like
String company = "Masood LIMITED LTD PTE PRIVATE"
or any name that can have the above mentioned words at the end.
Thanks

You can do this in single line. no need to loop through. just use String#replaceAll(regex, str).
company = company.replaceAll("PTE$*?|LTD$*?|PRIVATE$*?|LIMITED$*?","");

If you place the unwanted words in the map it will be ommitted in the resultant string
HashMap map = new HashMap();
map.put("PTE", "");
map.put("LTD", "");
map.put("PRIVATE", "");
map.put("LIMITED", "");
String company = "Basit LTD PRIVATE PTE";
String words[] = company.split(" ");
String resultantStr = "";
for(int k = 0; k < words.length; k++){
if(map.get(words[k]) == null) {
resultantStr += words[k] + " ";
}
}
resultantStr = resultantStr.trim();
System.out.println(" Trimmed String: "+ resultantStr);

If you want to remove these suffixes only at the end of the string, then you could introduce a while loop:
String[] str = {"PTE", "LTD", "PRIVATE", "LIMITED"};
boolean foundSuffix = true;
String company = "Basit LTD";
while (foundSuffix) {
foundSuffix = false;
for(int i=0;i<str.length;i++) {
if (company.endsWith(str[i])) {
foundSuffix = true;
int position = company.lastIndexOf(str[i]);
company = company.substring(0, position);
}
}
}
System.out.println(company.replaceAll("\\s",""));
If you don't mind transforming PTE Basit LIMITED INC to Basit (and also remove the first PTE), then replaceAll should work, as explained by others.

I was trying to do exactly same thing for one of my projects. I wrote this code few days earlier. Now I was exactly trying to find a much better way to do it, that's how I found this Question. But after seeing other answers I decided to share my version of the code.
Collection<String> stopWordSet = Arrays.asList("PTE", "LTD", "PRIVATE", "LIMITED");
String company = "Basit LTD"; //Or Anything
String[] tokens = company.split("[\#\]\\\_\^\[\"\#\ \!\&\'\`\$\%\*\+\(\)\.\/\,\-\;\~\:\}\|\{\?\>\=\<]+");
Stack<String> tokenStack = new Stack<>();
tokenStack.addAll(Arrays.asList(tokens));
while (!tokenStack.isEmpty()) {
String token = tokenStack.peek();
if (stopWordSet.contains(token))
tokenStack.pop();
else
break;
}
String formattedCompanyName = StringUtils.join(tokenStack.toArray());

Try this :
public static void main(String a[]) {
String[] str = {"PTE", "LTD", "PRIVATE", "LIMITED"};
String company = "Basit LIMITED PRIVATE LTD PTE";
for(int i=0;i<str.length;i++) {
company = company.replaceAll(str[i], "");
}
System.out.println(company.replaceAll("\\s",""));
}

All you need is to use trim() and call your function recursively, Or each time you remove a sub string from the end, reset your i to 0.

public class StringMatchRemove {
public static void main(String[] args) {
String str="my name is noorus khan";
String search="noorus";
String newString="";
String word=str.replace(search," ");
StringTokenizer st = new StringTokenizer(word," ");
while(st.hasMoreTokens())
{
newString = newString + st.nextToken() + " ";
}
System.out.println(newString);
}
first using the replace method we get word=my name is ..... khan (Note: here(.) represents the space). Now we should have to remove these spaces for that we are creating a new string adding all the token simply.
Output: my name is khan

Regex pattern in Java to find and remove duplicate emails

public String readEmails(String fileData) {
String regex = "[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*#(?:[a-z0-9]"
+ "(?:[a-z0-9-]*[a-z0-9])?\\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?";
String emails = "", emails2 = "";
fileData = fileData.toLowerCase();
Matcher m = Pattern.compile(regex).matcher(fileData);
while (m.find()) {
emails += m.group()+", ";
}
return emails;
}
I am reading rtf file, finding emails and then storing into DB. I found one email is extracted two time I.e. HYPERLINK "mailto: aa#ymail.com" and then aa#ymail.com
How can I match two similar emails and keep one copy by removing all similar emails?

You can change your code as
Set<String> set = new HashSet<String>();
Matcher m = Pattern.compile(regex).matcher(fileData);
while (m.find()) {
String email = m.group();
if (!set.contains(email)) {
emails += email + ", ";
set.add(email);
}
}
return emails;
}

Instead of saving emails as a string with commas:
Lower case them.
Store them in dictionary (HashSet) to deduplicate them
At the end, create output string out of the elements in dictionary.

How to replace a set of tokens in a Java String?

I have the following template String: "Hello [Name] Please find attached [Invoice Number] which is due on [Due Date]".
I also have String variables for name, invoice number and due date - what's the best way to replace the tokens in the template with the variables?
(Note that if a variable happens to contain a token it should NOT be replaced).
EDIT
With thanks to #laginimaineb and #alan-moore, here's my solution:
public static String replaceTokens(String text,
Map<String, String> replacements) {
Pattern pattern = Pattern.compile("\\[(.+?)\\]");
Matcher matcher = pattern.matcher(text);
StringBuffer buffer = new StringBuffer();
while (matcher.find()) {
String replacement = replacements.get(matcher.group(1));
if (replacement != null) {
// matcher.appendReplacement(buffer, replacement);
// see comment
matcher.appendReplacement(buffer, "");
buffer.append(replacement);
}
}
matcher.appendTail(buffer);
return buffer.toString();
}

I really don't think you need to use a templating engine or anything like that for this. You can use the String.format method, like so:
String template = "Hello %s Please find attached %s which is due on %s";
String message = String.format(template, name, invoiceNumber, dueDate);

The most efficient way would be using a matcher to continually find the expressions and replace them, then append the text to a string builder:
Pattern pattern = Pattern.compile("\\[(.+?)\\]");
Matcher matcher = pattern.matcher(text);
HashMap<String,String> replacements = new HashMap<String,String>();
//populate the replacements map ...
StringBuilder builder = new StringBuilder();
int i = 0;
while (matcher.find()) {
String replacement = replacements.get(matcher.group(1));
builder.append(text.substring(i, matcher.start()));
if (replacement == null)
builder.append(matcher.group(0));
else
builder.append(replacement);
i = matcher.end();
}
builder.append(text.substring(i, text.length()));
return builder.toString();

Unfortunately the comfortable method String.format mentioned above is only available starting with Java 1.5 (which should be pretty standard nowadays, but you never know). Instead of that you might also use Java's class MessageFormat for replacing the placeholders.
It supports placeholders in the form '{number}', so your message would look like "Hello {0} Please find attached {1} which is due on {2}". These Strings can easily be externalized using ResourceBundles (e. g. for localization with multiple locales). The replacing would be done using the static'format' method of class MessageFormat:
String msg = "Hello {0} Please find attached {1} which is due on {2}";
String[] values = {
"John Doe", "invoice #123", "2009-06-30"
};
System.out.println(MessageFormat.format(msg, values));

You could try using a templating library like Apache Velocity.
http://velocity.apache.org/
Here is an example:
import org.apache.velocity.VelocityContext;
import org.apache.velocity.app.Velocity;
import java.io.StringWriter;
public class TemplateExample {
public static void main(String args[]) throws Exception {
Velocity.init();
VelocityContext context = new VelocityContext();
context.put("name", "Mark");
context.put("invoiceNumber", "42123");
context.put("dueDate", "June 6, 2009");
String template = "Hello $name. Please find attached invoice" +
" $invoiceNumber which is due on $dueDate.";
StringWriter writer = new StringWriter();
Velocity.evaluate(context, writer, "TemplateName", template);
System.out.println(writer);
}
}
The output would be:
Hello Mark. Please find attached invoice 42123 which is due on June 6, 2009.

You can use template library for complex template replacement.
FreeMarker is a very good choice.
http://freemarker.sourceforge.net/
But for simple task, there is a simple utility class can help you.
org.apache.commons.lang3.text.StrSubstitutor
It is very powerful, customizable, and easy to use.
This class takes a piece of text and substitutes all the variables
within it. The default definition of a variable is ${variableName}.
The prefix and suffix can be changed via constructors and set methods.
Variable values are typically resolved from a map, but could also be
resolved from system properties, or by supplying a custom variable
resolver.
For example, if you want to substitute system environment variable into a template string,
here is the code:
public class SysEnvSubstitutor {
public static final String replace(final String source) {
StrSubstitutor strSubstitutor = new StrSubstitutor(
new StrLookup<Object>() {
#Override
public String lookup(final String key) {
return System.getenv(key);
}
});
return strSubstitutor.replace(source);
}
}

System.out.println(MessageFormat.format("Hello {0}! You have {1} messages", "Join",10L));
Output:
Hello Join! You have 10 messages"

String.format("Hello %s Please find attached %s which is due on %s", name, invoice, date)

It depends of where the actual data that you want to replace is located. You might have a Map like this:
Map<String, String> values = new HashMap<String, String>();
containing all the data that can be replaced. Then you can iterate over the map and change everything in the String as follows:
String s = "Your String with [Fields]";
for (Map.Entry<String, String> e : values.entrySet()) {
s = s.replaceAll("\\[" + e.getKey() + "\\]", e.getValue());
}
You could also iterate over the String and find the elements in the map. But that is a little bit more complicated because you need to parse the String searching for the []. You could do it with a regular expression using Pattern and Matcher.

My solution for replacing ${variable} style tokens (inspired by the answers here and by the Spring UriTemplate):
public static String substituteVariables(String template, Map<String, String> variables) {
Pattern pattern = Pattern.compile("\\$\\{(.+?)\\}");
Matcher matcher = pattern.matcher(template);
// StringBuilder cannot be used here because Matcher expects StringBuffer
StringBuffer buffer = new StringBuffer();
while (matcher.find()) {
if (variables.containsKey(matcher.group(1))) {
String replacement = variables.get(matcher.group(1));
// quote to work properly with $ and {,} signs
matcher.appendReplacement(buffer, replacement != null ? Matcher.quoteReplacement(replacement) : "null");
}
}
matcher.appendTail(buffer);
return buffer.toString();
}

With Apache Commons Library, you can simply use Stringutils.replaceEach:
public static String replaceEach(String text,
String[] searchList,
String[] replacementList)
From the documentation:
Replaces all occurrences of Strings within another String.
A null reference passed to this method is a no-op, or if any "search
string" or "string to replace" is null, that replace will be ignored.
This will not repeat. For repeating replaces, call the overloaded
method.
StringUtils.replaceEach(null, *, *) = null
StringUtils.replaceEach("", *, *) = ""
StringUtils.replaceEach("aba", null, null) = "aba"
StringUtils.replaceEach("aba", new String[0], null) = "aba"
StringUtils.replaceEach("aba", null, new String[0]) = "aba"
StringUtils.replaceEach("aba", new String[]{"a"}, null) = "aba"
StringUtils.replaceEach("aba", new String[]{"a"}, new String[]{""}) = "b"
StringUtils.replaceEach("aba", new String[]{null}, new String[]{"a"}) = "aba"
StringUtils.replaceEach("abcde", new String[]{"ab", "d"}, new String[]{"w", "t"}) = "wcte"
(example of how it does not repeat)
StringUtils.replaceEach("abcde", new String[]{"ab", "d"}, new String[]{"d", "t"}) = "dcte"

You can use Apache Commons StringSubstitutor:
For example:
// Build map
Map<String, String> valuesMap = new HashMap<>();
valuesMap.put("animal", "quick brown fox");
valuesMap.put("target", "lazy dog");
String templateString = "The ${animal} jumped over the ${target}.";
// Build StringSubstitutor
StringSubstitutor sub = new StringSubstitutor(valuesMap);
// Replace
String resolvedString = sub.replace(templateString);
yielding:
"The quick brown fox jumped over the lazy dog."
You can also customize the prefix and suffix delimiters (${ and } respectively in the example above) by using:
setVariablePrefix
setVariableSuffix
You can also specify a default value using syntax like below:
String templateString = "The ${animal:giraffe} jumped over the ${target}.";
which would yield "The giraffe jumped over the lazy dog." when no animal parameter was supplied.

http://github.com/niesfisch/tokenreplacer

FYI
In the new language Kotlin,
you can use "String Templates" in your source code directly,
no 3rd party library or template engine need to do the variable replacement.
It is a feature of the language itself.
See:
https://kotlinlang.org/docs/reference/basic-types.html#string-templates

In the past, I've solved this kind of problem with StringTemplate and Groovy Templates.
Ultimately, the decision of using a templating engine or not should be based on the following factors:
Will you have many of these templates in the application?
Do you need the ability to modify the templates without restarting the application?
Who will be maintaining these templates? A Java programmer or a business analyst involved on the project?
Will you need to the ability to put logic in your templates, like conditional text based on values in the variables?
Will you need the ability to include other templates in a template?
If any of the above applies to your project, I would consider using a templating engine, most of which provide this functionality, and more.

I used
String template = "Hello %s Please find attached %s which is due on %s";
String message = String.format(template, name, invoiceNumber, dueDate);

The following replaces variables of the form <<VAR>>, with values looked up from a Map. You can test it online here
For example, with the following input string
BMI=(<<Weight>>/(<<Height>>*<<Height>>)) * 70
Hi there <<Weight>> was here
and the following variable values
Weight, 42
Height, HEIGHT 51
outputs the following
BMI=(42/(HEIGHT 51*HEIGHT 51)) * 70
Hi there 42 was here
Here's the code
static Pattern pattern = Pattern.compile("<<([a-z][a-z0-9]*)>>", Pattern.CASE_INSENSITIVE);
public static String replaceVarsWithValues(String message, Map<String,String> varValues) {
try {
StringBuffer newStr = new StringBuffer(message);
int lenDiff = 0;
Matcher m = pattern.matcher(message);
while (m.find()) {
String fullText = m.group(0);
String keyName = m.group(1);
String newValue = varValues.get(keyName)+"";
String replacementText = newValue;
newStr = newStr.replace(m.start() - lenDiff, m.end() - lenDiff, replacementText);
lenDiff += fullText.length() - replacementText.length();
}
return newStr.toString();
} catch (Exception e) {
return message;
}
}
public static void main(String args[]) throws Exception {
String testString = "BMI=(<<Weight>>/(<<Height>>*<<Height>>)) * 70\n\nHi there <<Weight>> was here";
HashMap<String,String> values = new HashMap<>();
values.put("Weight", "42");
values.put("Height", "HEIGHT 51");
System.out.println(replaceVarsWithValues(testString, values));
}
and although not requested, you can use a similar approach to replace variables in a string with properties from your application.properties file, though this may already be being done:
private static Pattern patternMatchForProperties =
Pattern.compile("[$][{]([.a-z0-9_]*)[}]", Pattern.CASE_INSENSITIVE);
protected String replaceVarsWithProperties(String message) {
try {
StringBuffer newStr = new StringBuffer(message);
int lenDiff = 0;
Matcher m = patternMatchForProperties.matcher(message);
while (m.find()) {
String fullText = m.group(0);
String keyName = m.group(1);
String newValue = System.getProperty(keyName);
String replacementText = newValue;
newStr = newStr.replace(m.start() - lenDiff, m.end() - lenDiff, replacementText);
lenDiff += fullText.length() - replacementText.length();
}
return newStr.toString();
} catch (Exception e) {
return message;
}
}