To begin with the XML file 2,84GB and none of SAX or DOM parser seems to be working. I've already tried them and every time crashes. So, I choose to read the file and export the data I want with BufferedReader, parsing the XML file like it is txt.
XML File(small part):
<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE dblp SYSTEM "dblp-2019-11-22.dtd">
<dblp>
<phdthesis mdate="2016-05-04" key="phd/dk/Heine2010">
<author>Carmen Heine</author>
<title>Modell zur Produktion von Online-Hilfen.</title>
<year>2010</year>
<school>Aarhus University</school>
<pages>1-315</pages>
<isbn>978-3-86596-263-8</isbn>
<ee>http://d-nb.info/996064095</ee>
</phdthesis><phdthesis mdate="2020-02-12" key="phd/Hoff2002">
<author>Gerd Hoff</author>
<title>Ein Verfahren zur thematisch spezialisierten Suche im Web und seine Realisierung im Prototypen HomePageSearch</title>
<year>2002</year>
From that XML file I want to retrieve the data which is between the tags <year>. I also used Pattern and Matcher with regEx to find out the information I want. My code so far:
public class Publications {
public static void main(String[] args) throws IOException {
File file = new File("dblp-2020-04-01.xml");
FileInputStream fileStream = new FileInputStream(file);
InputStreamReader input = new InputStreamReader(fileStream);
BufferedReader reader = new BufferedReader(input);
String line;
String regex = "\\d+";
// Reading line by line from the
// file until a null is returned
while ((line = reader.readLine()) != null) {
final Pattern pattern = Pattern.compile("<year>(.+?)</year>", Pattern.DOTALL);
final Matcher matcher = pattern.matcher("<year>"+regex+"</year>");
matcher.find();
System.out.println(matcher.group(1)); // Prints String I want to extract
}
}
}
After compiling , the results aren't what I expected to be. Instead of printing me the exact year everytime the parser finds the ... tag the results are the following:
\d+
\d+
\d+
\d+
\d+
\d+
\d+
\d+
\d+
\d+
Any suggestions?
Please don't try parsing XML using regular expressions. We get hundreds of questions on this forum from people trying to generate XML in peculiar formats because that's the only thing the receiving application can handle, and the reason the receiving application has such restrictions is that it's trying to do the XML parsing "by hand". You're storing up trouble for yourself, for the people you want to exchange data with, and for the people on StackOverflow that you will turn to for help when it all goes pear-shaped. XML standards exist for a reason, and work very well when everyone conforms to them.
The right approach in this case is a streaming XML approach, using SAX, StAX, or streaming XSLT 3.0, and you've abandoned those approaches for completely spurious reasons.
Remark
Regexen are the wrong tool to extract information from xml (or similar structured formats). The general approach is not recommended. For the right way to handle it, cf. Michael Kay's answer.
Answer
You provide the wrong argument in constructing the matcher. Instead of the expression in your code you need to provide the current line:
// ...
final Matcher matcher = pattern.matcher(line);
if (matcher.find()) {
System.out.println(matcher.group(1)); // Prints String I want to extract
}
// ...
Note the extra conditional to check whether the current line does match at all.
Also note that the pattern you match against is defined in the Pattern constructor. Thus to match only <year> tags that contain numerical values, the line has to be changed to
final Pattern pattern = Pattern.compile("<year>(" + regex + ")</year>", Pattern.DOTALL);
Related
I have this text from file which I would like to get values using pattern match:
<!--
#author: batman
#description: 100000
-->
I'm using this code to find comments into XML files and get values using Patterns match:
XMLStreamReader xr = XMLInputFactory.newInstance().createXMLStreamReader(new FileInputStream(file));
Pattern pattern = Pattern.compile("#(?<key>([\\w]+)?): (?<value>(.+)?)");
while (xr.hasNext())
{
if (xr.next() == XMLStreamConstants.COMMENT) {
String comment = xr.getText();
Matcher matcher = pattern.matcher(comment);
if(!matcher.matches()){
continue;
}
.....
}
When I run the code I get error:
persistence-configuration.xsd (No such file or directory)
at [row,col {unknown-source}]: [4,69]
But when I remove matcher.matches() I get several iterations if xr because I have several comments into XML file.
The idea is to get only the comment with the proper match and to skip the rest of the comments. Do you know why the code is not working fine?
I tried also with "#(?<key>[\\w]+): (?<value>.*)" but again I get the same issue.
Is you input source actually there? The exception you get hints at a file problem.
Matcher.matches() tests a String for complete confirmity with the entire pattern. All characters of the input sequence need to be consumed. So if your line starts with blanks (tabs, whitespaces...) then your matcher may fail to match, if you use matches.
You are looking for matcher.find(), which can be used like this:
Pattern myPattern = Pattern.compile("somePattern");
Matcher matcher = pattern.matcher("someInput");
while (matcher.find()) {
// this is where you can check what you found...
}
Besides, you may want to check, at what chunks the stream provides elements. Are they really coming linewise?
Hello I want to extract "Hello, World!" "and" and the Paragraph "This is a minimal....." from the given string in JAVA. I am having problems in extracting, so can anyone help me with it?
So I always get different Strings and want to extract the string between 2 square brackets []......[].
String s1="[sh1] Hello, World! [/s11] and [pp]This is a minimal "hello world" HTML document. It demonstrates the basic structure of an HTML file and anchors. [/xy]"
Thanks
Use the Pattern & Matcher to match square brackets:
Pattern pattern = Pattern.compile("\\[[^\\]]*\\]([^\\]]*)\\[[^\\]]*\\]");
Matcher matcher = pattern.matcher(s1);
while (matcher.find()) {
System.out.println( "Found value: " + matcher.group(1).trim() );
}
Demo: https://ideone.com/kNKBgg
Please don't use RegEx-es to do this (it's what Pattern and Matcher do) - see here for reason why you shouldn't. While you could use this for the particular bracket example, if you expect full-blown HTML don't do it.
If you want to extract content from HTML use a parser, for example SAXParser or DOMParser - see Oracle documentation for examples.
I have a string that contains file names like:
"file1.txt file2.jpg tricky file name.txt other tricky filenames containing áéíőéáóó.gif"
How can I get the file names, one by one?
I am looking for the most safe most through method, preferably something java standard. There has got to be some regular expression already out there, I am counting on your experience.
Edit: expected results:
"file1.txt", "file2.jpg", "tricky file name.txt", "other tricky filenames containing áéíőéáóó.gif"
Thanks for the help,
Sziro
Regular expresion that enrico.bacis suggested (\S.?.\S+)* will not work if there are filenames without characters before "." like .project.
Correct pattern would be:
(([^ .]+ +)*\S*\.\S+)
You can try it here.
Java program that could extract filenames will look like:
String patternStr = "([^ .]+ +)*\\S*\\.\\S+";
String input = "file1.txt .project file2.jpg tricky file name.txt other tricky filenames containing áéíoéáóó.gif";
Pattern pattern = Pattern.compile(patternStr, Pattern.MULTILINE);
Matcher matcher = pattern.matcher(input);
while (matcher.find()) {
System.out.println(matcher.group());
}
If you want to use regular expressions you can find all the occurrences of:
(\S.*?\.\S+)
(you can test it here)
If there are spaces in the file names, it makes it trickier.
If you can assume there are no dots (.) in the file names, you can use the dot to find each individual records as has been suggested.
If you can't assume there are no dots in file names, e.g. my file.new something.txt
In this situation, I'd suggest you create a list of acceptable extentions, e.g. .doc, .jpg, .pdf etc.
I know the list may be long, so it's not ideal. Once you have done this you can look for these extensions and assume what's before it is a valid filename.
String txt = "file1.txt file2.jpg tricky file name.txt other tricky filenames containing áéíőéáóó.gif";
Pattern pattern = Pattern.compile("\\S.*?\\.\\S+"); // Get regex from enrico.bacis
Matcher matcher = pattern.matcher(txt);
while (matcher.find()) {
System.out.println(matcher.group().trim());
}
I'm trying to build a Java regex to search a .txt file for a Windows formatted file path, however, due to the file path containing literal backslashes, my regex is failing.
The .txt file contains the line:
C\Windows\SysWOW64\ntdll.dll
However, some of the filenames in the text file are formatted like this:
C\Windows\SysWOW64\ntdll.dll (some developer stuff here...)
So I'm unable to use String.equals
To match this line, I'm using the regex:
filename = "C\\Windows\\SysWOW64\\ntdll.dll"
read = BufferedReader.readLine();
if (Pattern.compile(Pattern.quote(filename), Pattern.CASE_INSENSITIVE).matcher(read).find()) {
I've tried escaping the literal backslashes, using the replace method, i.e:
filename.replace("\\", "\\\\");
However, this is failing to find, I'm guessing this is because I need to further escape the backslashes after the Pattern has been built, I'm thinking I might need to escape upto an additional four backslashes, i.e:
Pattern.replaceAll("\\\\", "\\\\\\\\");
However, each time I try, the pattern doesn't get matched. I'm certain it's a problem with the backslashes, but I'm not sure where to do the replacement, or if there's a better way of building the pattern.
I think the problem is further being compounded as the replaceAll method also uses a regex, with means the pattern will have it's own backslashes in there, to deal with the case insensitivity.
Any input or advice would be appreciated.
Thanks
Seems like you're attempting to to a direct comparison of String against another. For exact matches, you could do (
if (read.equalsIgnoreCase(filename)) {
of simply
if (read.startsWith(filename)) {
Try this :
While reading each line from the file, replace '\' by '\\'.
Then :
String lLine = "C\\Windows\\SysWOW64\\ntdll.dll";
Pattern lPattern = Pattern.compile("C\\\\Windows\\\\SysWOW64\\\\ntdll\\.dll");
Matcher lMatcher = lPattern.matcher(lLine);
if(lMatcher.find()) {
System.out.println(lMatcher.group());
}
lLine = "C\\Windows\\SysWOW64\\ntdll.dll (some developer stuff here...)";
lMatcher = lPattern.matcher(lLine);
if(lMatcher.find()) {
System.out.println(lMatcher.group());
}
The correct usage will be:
String filename = "C\\Windows\\SysWOW64\\ntdll.dll";
String file = filename.replace('\\', ' ');
My question: What's a good way to parse the information below?
I have a java program that gets it's input from XML. I have a feature which will send an error email if there was any problem in the processing. Because parsing the XML could be a problem, I want to have a feature that would be able to regex the emails out of the xml (because if parsing was the problem then I couldn't get the error e-mails out of the xml normally).
Requirements:
I want to be able to parse the to, cc, and bcc attributes seperately
There are other elements which have to, cc, and bcc attributes
Whitespace does not matter, so my example may show the attributes on a newline, but that's not always the case.
The order of the attributes does not matter.
Here's an example of the xml:
<error_options
to="your_email#your_server.com"
cc="cc_error#your_server.com"
bcc="bcc_error#your_server.com"
reply_to="someone_else#their_server.com"
from="bo_error#some_server.org"
subject="Error running System at ##TIMESTAMP##"
force_send="false"
max_email_size="10485760"
oversized_email_action="zip;split_all"
>
I tried this error_options.{0,100}?to="(.*?)", but that matched me down to reply_to. That made me think there are probably some cases I might miss, which is why I'm posting this as a question.
This piece will put all attributes from your String s="<error_options..." into a map:
Pattern p = Pattern.compile("\\s+?(.+?)=\"(.+?)\\s*?\"",Pattern.DOTALL);
Map a = new HashMap() ;
Matcher m = p.matcher(s) ;
while( m.find() ) {
String key = m.group(1).trim() ;
String val = m.group(2).trim() ;
a.put(key, val) ;
}
...then you can extract the values that you're interested in from that map.
This question is similar to RegEx match open tags except XHTML self-contained tags. Never ever parse XML or HTML with regular expressions. There are many XML parser implementation in Java to do this task properly. Read the document and parse the attributes one by one.
Don't mind, if the users XML is not well-formed, the parsers can handle a lot of sloppiness.
/<error_options(?=\s)[^>]*?(?<=\n)\s*to="([^"]*)"/s;
/<error_options(?=\s)[^>]*?(?<=\n)\s*cc="([^"]*)"/s;
/<error_options(?=\s)[^>]*?(?<=\n)\s*bcc="([^"]*)"/s;