Related
I am writing a Java utility that helps me to generate loads of data for performance testing. It would be really cool to be able to specify a regex for Strings so that my generator spits out things that match this.
Is something out there already baked that I can use to do this? Or is there a library that gets me most of the way there?
Edit:
Complete list of suggested libraries on this question:
Xeger* - Java
Generex* - Java
Rgxgen - Java
rxrdg - C#
* - Depends on dk.brics.automaton
Edit:
As mentioned in the comments, there is a library available at Google Code to achieve this:
https://code.google.com/archive/p/xeger/
See also https://github.com/mifmif/Generex as suggested by Mifmif
Original message:
Firstly, with a complex enough regexp, I believe this can be impossible. But you should be able to put something together for simple regexps.
If you take a look at the source code of the class java.util.regex.Pattern, you'll see that it uses an internal representation of Node instances. Each of the different pattern components have their own implementation of a Node subclass. These Nodes are organised into a tree.
By producing a visitor that traverses this tree, you should be able to call an overloaded generator method or some kind of Builder that cobbles something together.
It's too late to help the original poster, but it could help a newcomer. Generex is a useful java library that provides many features for using regexes to generate strings (random generation, generating a string based on its index, generating all strings...).
Example :
Generex generex = new Generex("[0-3]([a-c]|[e-g]{1,2})");
// generate the second String in lexicographical order that matches the given Regex.
String secondString = generex.getMatchedString(2);
System.out.println(secondString);// it print '0b'
// Generate all String that matches the given Regex.
List<String> matchedStrs = generex.getAllMatchedStrings();
// Using Generex iterator
Iterator iterator = generex.iterator();
while (iterator.hasNext()) {
System.out.print(iterator.next() + " ");
}
// it prints 0a 0b 0c 0e 0ee 0e 0e 0f 0fe 0f 0f 0g 0ge 0g 0g 1a 1b 1c 1e
// 1ee 1e 1e 1f 1fe 1f 1f 1g 1ge 1g 1g 2a 2b 2c 2e 2ee 2e 2e 2f 2fe 2f 2f 2g
// 2ge 2g 2g 3a 3b 3c 3e 3ee 3e 3e 3f 3fe 3f 3f 3g 3ge 3g 3g 1ee
// Generate random String
String randomStr = generex.random();
System.out.println(randomStr);// a random value from the previous String list
Disclosure
The project mentioned on this post belongs to the user answering (Mifmif) the question. As per the rules, this need to be brought up.
Xeger (Java) is capable of doing it as well:
String regex = "[ab]{4,6}c";
Xeger generator = new Xeger(regex);
String result = generator.generate();
assert result.matches(regex);
This question is really old, though the problem was actual for me.
I've tried xeger and Generex and they doesn't seem to meet my reguirements.
They actually fail to process some of the regex patterns (like a{60000}) or for others (e.g. (A|B|C|D|E|F)) they just don't produce all possible values. Since I didn't find any another appropriate solution - I've created my own library.
https://github.com/curious-odd-man/RgxGen
This library can be used to generate both matching and non-matching string.
There is also artifact on maven central available.
Usage example:
RgxGen rgxGen = new RgxGen(aRegex); // Create generator
String s = rgxGen.generate(); // Generate new random value
I've gone the root of rolling my own library for that (In c# but should be easy to understand for a Java developer).
Rxrdg started as a solution to a problem of creating test data for a real life project. The basic idea is to leverage the existing (regular expression) validation patterns to create random data that conforms to such patterns. This way valid random data is created.
It is not that difficult to write a parser for simple regex patterns. Using an abstract syntax tree to generate strings should be even easier.
On stackoverflow podcast 11:
Spolsky: Yep. There's a new product also, if you don't want to use the Team System there our friends at Redgate have a product called SQL Data Generator [http://www.red-gate.com/products/sql_data_generator/index.htm]. It's $295, and it just generates some realistic test data. And it does things like actually generate real cities in the city column that actually exist, and then when it generates those it'll get the state right, instead of getting the state wrong, or putting states into German cities and stuff like... you know, it generates pretty realistic looking data. I'm not really sure what all the features are.
This is probably not what you are looking for, but it might be a good starting off point, instead of creating your own.
I can't seem to find anything in google, so I would suggest tackling the problem by parsing a given regular expression into the smallest units of work (\w, [x-x], \d, etc) and writing some basic methods to support those regular expression phrases.
So for \w you would have a method getRandomLetter() which returns any random letter, and you would also have getRandomLetter(char startLetter, char endLetter) which gives you a random letter between the two values.
I am on flight and just saw the question: I have written easiest but inefficient and incomplete solution. I hope it may help you to start writing your own parser:
public static void main(String[] args) {
String line = "[A-Z0-9]{16}";
String[] tokens = line.split(line);
char[] pattern = new char[100];
int i = 0;
int len = tokens.length;
String sep1 = "[{";
StringTokenizer st = new StringTokenizer(line, sep1);
while (st.hasMoreTokens()) {
String token = st.nextToken();
System.out.println(token);
if (token.contains("]")) {
char[] endStr = null;
if (!token.endsWith("]")) {
String[] subTokens = token.split("]");
token = subTokens[0];
if (!subTokens[1].equalsIgnoreCase("*")) {
endStr = subTokens[1].toCharArray();
}
}
if (token.startsWith("^")) {
String subStr = token.substring(1, token.length() - 1);
char[] subChar = subStr.toCharArray();
Set set = new HashSet<Character>();
for (int p = 0; p < subChar.length; p++) {
set.add(subChar[p]);
}
int asci = 1;
while (true) {
char newChar = (char) (subChar[0] + (asci++));
if (!set.contains(newChar)) {
pattern[i++] = newChar;
break;
}
}
if (endStr != null) {
for (int r = 0; r < endStr.length; r++) {
pattern[i++] = endStr[r];
}
}
} else {
pattern[i++] = token.charAt(0);
}
} else if (token.contains("}")) {
char[] endStr = null;
if (!token.endsWith("}")) {
String[] subTokens = token.split("}");
token = subTokens[0];
if (!subTokens[1].equalsIgnoreCase("*")) {
endStr = subTokens[1].toCharArray();
}
}
int length = Integer.parseInt((new StringTokenizer(token, (",}"))).nextToken());
char element = pattern[i - 1];
for (int j = 0; j < length - 1; j++) {
pattern[i++] = element;
}
if (endStr != null) {
for (int r = 0; r < endStr.length; r++) {
pattern[i++] = endStr[r];
}
}
} else {
char[] temp = token.toCharArray();
for (int q = 0; q < temp.length; q++) {
pattern[i++] = temp[q];
}
}
}
String result = "";
for (int j = 0; j < i; j++) {
result += pattern[j];
}
System.out.print(result);
}
You'll have to write your own parser, like the author of String::Random (Perl) did. In fact, he doesn't use regexes anywhere in that module, it's just what perl-coders are used to.
On the other hand, maybe you can have a look at the source, to get some pointers.
EDIT: Damn, blair beat me to the punch by 15 seconds.
I know there's already an accepted answer, but I've been using RedGate's Data Generator (the one mentioned in Craig's answer) and it works REALLY well for everything I've thrown at it. It's quick and that leaves me wanting to use the same regex to generate the real data for things like registration codes that this thing spits out.
It takes a regex like:
[A-Z0-9]{3,3}-[A-Z0-9]{3,3}
and it generates tons of unique codes like:
LLK-32U
Is this some big secret algorithm that RedGate figured out and we're all out of luck or is it something that us mere mortals actually could do?
It's far from supporting a full PCRE regexp, but I wrote the following Ruby method to take a regexp-like string and produce a variation on it. (For language-based CAPTCHA.)
# q = "(How (much|many)|What) is (the (value|result) of)? :num1 :op :num2?"
# values = { :num1=>42, :op=>"plus", :num2=>17 }
# 4.times{ puts q.variation( values ) }
# => What is 42 plus 17?
# => How many is the result of 42 plus 17?
# => What is the result of 42 plus 17?
# => How much is the value of 42 plus 17?
class String
def variation( values={} )
out = self.dup
while out.gsub!( /\(([^())?]+)\)(\?)?/ ){
( $2 && ( rand > 0.5 ) ) ? '' : $1.split( '|' ).random
}; end
out.gsub!( /:(#{values.keys.join('|')})\b/ ){ values[$1.intern] }
out.gsub!( /\s{2,}/, ' ' )
out
end
end
class Array
def random
self[ rand( self.length ) ]
end
end
This question is very old, but I stumbled across it on my own search, so I will include a couple links for others who might be searching for the same functionality in other languages.
There is a Node.js library here: https://github.com/fent/randexp.js
There is a PHP library here: https://github.com/icomefromthenet/ReverseRegex
The PHP faker package includes a "regexify" method that accomplishes this: https://packagist.org/packages/fzaninotto/faker
If you want to generate "critical" strings, you may want to consider:
EGRET http://elarson.pythonanywhere.com/
that generates "evil" strings covering your regular expressions
MUTREX http://cs.unibg.it/mutrex/
that generates fault-detecting strings by regex mutation
Both are academic tools (I am one of the authors of the latter) and work reasonably well.
I'm having an error when splitting data extract from excel. when I'm on a.java it works. But now I'm converting my scripts to Java Xtend and now I am having the following error.
Here is the code
var dashboards = M3ASmokeTest.ReadExcelFile();
var countdsh = 0;
var countdom = 0;
var countrep = 0;
Thread.sleep(20000);
for(String groupedDomain: dashboards)
{
var domain = dashboards.get(countdom).split(";")[0];
var dboards = dashboards.get(countdsh).split(";")[1];
var reports = dashboards.get(countdsh).split(";")[2];'
}
Must admit, I don't know xtend, but from the docs it says :
https://www.eclipse.org/xtend/documentation/2.3.0/Documentation.pdf page 39
When a method call’s last parameter is a lambda it can be passed right
after the parameter list. For instance if you want to sort some
strings by their length, you could write :
Collections::sort(someStrings) [ a, b | a.length - b.length ]
So the square brackets denote lambda expressions, which is what you're getting (so I'm guessing a bug in xtend that it's incorrectly trying to apply that in your case).
Since it appears xtend automatically converts Arrays to Lists (page 15), try replacing the [0] with get(0), so :
var domain = dashboards.get(countdom).split(";").get(0);
In my software, I need to decide the version of a feature based on 2 parameters. Eg.
Render version 1 -> if (param1 && param2) == true;
Render version 2 -> if (!param1 && !param2) == true;
Render version 3 -> if only param1 == true;
Render version 4 -> if only param2 == true;
So, to meet this requirement, I wrote a code which looks like this -
if(param1 && param2) //both are true {
version = 1;
}
else if(!param1 && !param2) //both are false {
version = 2;
}
else if(!param2) //Means param1 is true {
version = 3;
}
else { //Means param2 is true
version = 4;
}
There are definitely multiple ways to code this but I finalised this approach after trying out different approaches because this is the most readable code I could come up with.
But this piece of code is definitely not scalable because -
Let say tomorrow we want to introduce new param called param3. Then
the no. of checks will increase because of multiple possible
combinations.
For this software, I am pretty much sure that we
will have to accommodate new parameters in future.
Can there be any scalable & readable way to code these requirements?
EDIT:
For a scalable solution define the versions for each parameter combination through a Map:
Map<List<Boolean>, Integer> paramsToVersion = Map.of(
List.of(true, true), 1,
List.of(false, false), 2,
List.of(true, false), 3,
List.of(false, true), 4);
Now finding the right version is a simple map lookup:
version = paramsToVersion.get(List.of(param1, param2));
The way I initialized the map works since Java 9. In older Java versions it’s a little more wordy, but probably still worth doing. Even in Java 9 you need to use Map.ofEntries if you have 4 or more parameters (for 16 combinations), which is a little more wordy too.
Original answer:
My taste would be for nested if/else statements and only testing each parameter once:
if (param1) {
if (param2) {
version = 1;
} else {
version = 3;
}
} else {
if (param2) {
version = 4;
} else {
version = 2;
}
}
But it scales poorly to many parameters.
If you have to enumerate all the possible combinations of Booleans, it's often simplest to convert them into a number:
// param1: F T F T
// param2; F F T T
static final int[] VERSIONS = new int[]{2, 3, 4, 1};
...
version = VERSIONS[(param1 ? 1:0) + (param2 ? 2:0)];
I doubt that there is a way that would be more compact, readable and scalable at the same time.
You express the conditions as minimized expressions, which are compact and may have meaning (in particular, the irrelevant variables don't clutter them). But there is no systematism that you could exploit.
A quite systematic alternative could be truth tables, i.e. the explicit expansion of all combinations and the associated truth value (or version number), which can be very efficient in terms of running-time. But these have a size exponential in the number of variables and are not especially readable.
I am afraid there is no free lunch. Your current solution is excellent.
If you are after efficiency (i.e. avoiding the need to evaluate all expressions sequentially), then you can think of the truth table approach, but in the following way:
declare an array of version numbers, with 2^n entries;
use the code just like you wrote to initialize all table entries; to achieve that, enumerate all integers in [0, 2^n) and use their binary representation;
now for a query, form an integer index from the n input booleans and lookup the array.
Using the answer by Olevv, the table would be [2, 4, 3, 1]. A lookup would be like (false, true) => T[01b] = 4.
What matters is that the original set of expressions is still there in the code, for human reading. You can use it in an initialization function that will fill the array at run-time, and you can also use it to hard-code the table (and leave the code in comments; even better, leave the code that generates the hard-coded table).
Your combinations of parameters is nothing more than a binary number (like 01100) where the 0 indicates a false and the 1 a true.
So your version can be easily calculated by using all the combinations of ones and zeroes. Possible combinations with 2 input parameters are:
11 -> both are true
10 -> first is true, second is false
01 -> first is false, second is true
00 -> both are false
So with this knowledge I've come up with a quite scalable solution using a "bit mask" (nothing more than a number) and "bit operations":
public static int getVersion(boolean... params) {
int length = params.length;
int mask = (1 << length) - 1;
for(int i = 0; i < length; i++) {
if(!params[i]) {
mask &= ~(1 << length - i - 1);
}
}
return mask + 1;
}
The most interesting line is probably this:
mask &= ~(1 << length - i - 1);
It does many things at once, I split it up. The part length - i - 1 calculates the position of the "bit" inside the bit mask from the right (0 based, like in arrays).
The next part: 1 << (length - i - 1) shifts the number 1 the amount of positions to the left. So lets say we have a position of 3, then the result of the operation 1 << 2 (2 is the third position) would be a binary number of the value 100.
The ~ sign is a binary inverse, so all the bits are inverted, all 0 are turned to 1 and all 1 are turned to 0. With the previous example the inverse of 100 would be 011.
The last part: mask &= n is the same as mask = mask & n where n is the previously computed value 011. This is nothing more than a binary AND, so all the same bits which are in mask and in n are kept, where as all others are discarded.
All in all, does this single line nothing more than remove the "bit" at a given position of the mask if the input parameter is false.
If the version numbers are not sequential from 1 to 4 then a version lookup table, like this one may help you.
The whole code would need just a single adjustment in the last line:
return VERSIONS[mask];
Where your VERSIONS array consists of all the versions in order, but reversed. (index 0 of VERSIONS is where both parameters are false)
I would have just gone with:
if (param1) {
if (param2) {
} else {
}
} else {
if (param2) {
} else {
}
}
Kind of repetitive, but each condition is evaluated only once, and you can easily find the code that executes for any particular combination. Adding a 3rd parameter will, of course, double the code. But if there are any invalid combinations, you can leave those out which shortens the code. Or, if you want to throw an exception for them, it becomes fairly easy to see which combination you have missed. When the IF's become too long, you can bring the actual code out in methods:
if (param1) {
if (param2) {
method_12();
} else {
method_1();
}
} else {
if (param2) {
method_2();
} else {
method_none();
}
}
Thus your whole switching logic takes up a function of itself and the actual code for any combination is in another method. When you need to work with the code for a particular combination, you just look up the appropriate method. The big IF maze is then rarely looked at, and when it is, it contains only the IFs themselves and nothing else potentially distracting.
I am working on a Displaykeyboard for disabled peaople and i am thinking about to adding a auto word completion function.
I found a example from oracle that works as i need it. Its the Another Example: TextAreaDemo. The problem is i dont really understand the search algorithm and the problem is when i add some word to the arraylist the search algorithm stops working properly.
String prefix = content.substring(w + 1).toLowerCase();
int n = Collections.binarySearch(words, prefix);
if (n < 0 && -n <= words.size()) {
String match = words.get(-n - 1);
if (match.startsWith(prefix)) {
// A completion is found
String completion = match.substring(pos - w);
// We cannot modify Document from within notification,
// so we submit a task that does the change later
SwingUtilities.invokeLater(
new CompletionTask(completion, pos + 1));
}
} else {
// Nothing found
mode = Mode.INSERT;
}
Is there a way to modify the example so it will work with any words?
Make sure you're not just adding the word to the end of the list and then using binarySearch(). It's documentation says the following
The list must be sorted into ascending order according to the natural
ordering of its elements (as by the sort(List) method) prior to making
this call.
Read more about it here: https://docs.oracle.com/javase/7/docs/api/java/util/Collections.html#binarySearch(java.util.List,%20T)
I need to work around a Java bug in JDK 1.5 which was fixed in 1.6. I'm using the following condition:
if (System.getProperty("java.version").startsWith("1.5.")) {
...
} else {
...
}
Will this work for other JVMs? Is there a better way to check this?
java.version is a system property that exists in every JVM. There are two possible formats for it:
Java 8 or lower: 1.6.0_23, 1.7.0, 1.7.0_80, 1.8.0_211
Java 9 or higher: 9.0.1, 11.0.4, 12, 12.0.1
Here is a trick to extract the major version: If it is a 1.x.y_z version string, extract the character at index 2 of the string. If it is a x.y.z version string, cut the string to its first dot character, if one exists.
private static int getVersion() {
String version = System.getProperty("java.version");
if(version.startsWith("1.")) {
version = version.substring(2, 3);
} else {
int dot = version.indexOf(".");
if(dot != -1) { version = version.substring(0, dot); }
} return Integer.parseInt(version);
}
Now you can check the version much more comfortably:
if(getVersion() < 6) {
// ...
}
What about getting the version from the package meta infos:
String version = Runtime.class.getPackage().getImplementationVersion();
Prints out something like:
1.7.0_13
Runtime.version()
Since Java 9, you can use Runtime.version(), which returns a Runtime.Version:
Runtime.Version version = Runtime.version();
These articles seem to suggest that checking for 1.5 or 1.6 prefix should work, as it follows proper version naming convention.
Sun Technical Articles
J2SE SDK/JRE Version String Naming Convention
Version 1.5.0 or 5.0?
"J2SE also keeps the version number 1.5.0 (or 1.5) in some places that are visible only to developers, or where the version number is parsed by programs"
"java.version system property"
Version 1.6.0 Used by Developers
"Java SE keeps the version number 1.6.0 (or 1.6) in some places that are visible only to developers, or where the version number is parsed by programs."
"java.version system property"
The simplest way (java.specification.version):
double version = Double.parseDouble(System.getProperty("java.specification.version"));
if (version == 1.5) {
// 1.5 specific code
} else {
// ...
}
or something like (java.version):
String[] javaVersionElements = System.getProperty("java.version").split("\\.");
int major = Integer.parseInt(javaVersionElements[1]);
if (major == 5) {
// 1.5 specific code
} else {
// ...
}
or if you want to break it all up (java.runtime.version):
String discard, major, minor, update, build;
String[] javaVersionElements = System.getProperty("java.runtime.version").split("\\.|_|-b");
discard = javaVersionElements[0];
major = javaVersionElements[1];
minor = javaVersionElements[2];
update = javaVersionElements[3];
build = javaVersionElements[4];
Example for Apache Commons Lang:
import org.apache.commons.lang.SystemUtils;
Float version = SystemUtils.JAVA_VERSION_FLOAT;
if (version < 1.4f) {
// legacy
} else if (SystemUtils.IS_JAVA_1_5) {
// 1.5 specific code
} else if (SystemUtils.isJavaVersionAtLeast(1.6f)) {
// 1.6 compatible code
} else {
// dodgy clause to catch 1.4 :)
}
Just a note that in Java 9 and above, the naming convention is different. System.getProperty("java.version") returns "9" rather than "1.9".
Does not work, need --pos to evaluate double:
String version = System.getProperty("java.version");
System.out.println("version:" + version);
int pos = 0, count = 0;
for (; pos < version.length() && count < 2; pos++) {
if (version.charAt(pos) == '.') {
count++;
}
}
--pos; //EVALUATE double
double dversion = Double.parseDouble(version.substring(0, pos));
System.out.println("dversion:" + dversion);
return dversion;
}
Here's the implementation in JOSM:
/**
* Returns the Java version as an int value.
* #return the Java version as an int value (8, 9, etc.)
* #since 12130
*/
public static int getJavaVersion() {
String version = System.getProperty("java.version");
if (version.startsWith("1.")) {
version = version.substring(2);
}
// Allow these formats:
// 1.8.0_72-ea
// 9-ea
// 9
// 9.0.1
int dotPos = version.indexOf('.');
int dashPos = version.indexOf('-');
return Integer.parseInt(version.substring(0,
dotPos > -1 ? dotPos : dashPos > -1 ? dashPos : 1));
}
If you can have dependency to apache utils you can use org.apache.commons.lang3.SystemUtils.
System.out.println("Is Java version at least 1.8: " + SystemUtils.isJavaVersionAtLeast(JavaVersion.JAVA_1_8));
Don't know another way of checking this, but this: http://java.sun.com/j2se/1.5.0/docs/api/java/lang/System.html#getProperties()" implies "java.version" is a standard system property so I'd expect it to work with other JVMs.
Here is the answer from #mvanle, converted to Scala:
scala> val Array(javaVerPrefix, javaVerMajor, javaVerMinor, _, _) = System.getProperty("java.runtime.version").split("\\.|_|-b")
javaVerPrefix: String = 1
javaVerMajor: String = 8
javaVerMinor: String = 0
https://docs.oracle.com/javase/9/docs/api/java/lang/Runtime.Version.html#version--
Runtime.version().version()
For 17.0.1 it returns [17, 0, 1]
One can use Runtime.version().version().get(0) to get the major java version.
In kotlin:
/**
* Returns the major JVM version, e.g. 6 for Java 1.6, 8 for Java 8, 11 for Java 11 etc.
*/
public val jvmVersion: Int get() = System.getProperty("java.version").parseJvmVersion()
/**
* Returns the major JVM version, 1 for 1.1, 2 for 1.2, 3 for 1.3, 4 for 1.4, 5
* for 1.5 etc.
*/
fun String.parseJvmVersion(): Int {
val version: String = removePrefix("1.").takeWhile { it.isDigit() }
return version.toInt()
}