Generating millions of random string in Java - java

I would like to generate millions of passwords randomly between 4 millions and 50 millions. The problem is the time it takes the processor to process it.
I would like to know if there is a solution to generate a lot of passwords in only a few seconds (max 1 minute for 50 millions).
I've done that for now but it takes me more than 3 min (with a very good config and I would like to run it on small config).
private final static String policy = "azertyuiopqsdfghjklmwxcvbnAZERTYUIOPQSDFGHJKLMWXCVBN1234567890";
private static List<String> names = new ArrayList<String>();
public static void main(String[] args) {
names.add("de");
init();
}
private static String generator(){
String password="";
int randomWithMathRandom = (int) ((Math.random() * ( - 6)) + 6);
for(var i=0;i<8;i++){
randomWithMathRandom = (int) ((Math.random() * ( - 6)) + 6);
password+= policy.charAt(randomWithMathRandom);
}
return password;
}
public static void init() {
for (int i = 0; i < 40000000; i++) {
names.add(generator());
}
}
btw I can't take a ready-made list. I think the most 'expensive' waste of time is the input into the list.
My current config :
ryzen 7 4800h
rtx 2600
SSD NVME
RAM 3200MHZ
UPDATE :
I tried with 20Millions and it's display an error: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main"

Storing 50 million passwords as Strings in-memory could cause problems since either the stack or the heap may overflow. From this point of view, I think the best we can do is to generate a chunk of passwords, store them in a file, generate the next chunk, append them to the file... until the desired amount of passwords is created. I hacked together a small program that generates random Strings of length 32. As alphabet, I used all ASCII-characters between '!' (ASCII-value 33) and '~' (ASCII-value 126).
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;
import java.text.DecimalFormat;
import java.text.DecimalFormatSymbols;
import java.util.Random;
import java.util.concurrent.TimeUnit;
class Scratch {
private static final int MIN = '!';
private static final int MAX = '~';
private static final Random RANDOM = new Random();
public static void main(final String... args) throws IOException {
final Path passwordFile = Path.of("passwords.txt");
if (!Files.exists(passwordFile)) {
Files.createFile(passwordFile);
}
final DecimalFormat df = new DecimalFormat();
final DecimalFormatSymbols ds = df.getDecimalFormatSymbols();
ds.setGroupingSeparator('_');
df.setDecimalFormatSymbols(ds);
final int numberOfPasswordsToGenerate = 50_000_000;
final int chunkSize = 1_000_000;
final int passwordLength = 32;
int generated = 0;
int chunk = 0;
final long start = System.nanoTime();
while (generated < numberOfPasswordsToGenerate) {
final StringBuilder passwords = new StringBuilder();
for (
int index = chunk * chunkSize;
index < (chunk + 1) * chunkSize && index < numberOfPasswordsToGenerate;
++index) {
final StringBuilder password = new StringBuilder();
for (int character = 0; character < passwordLength; ++character) {
password.append(fetchRandomLetterFromAlphabet());
}
passwords.append(password.toString()).append(System.lineSeparator());
++generated;
if (generated % 500_000 == 0) {
System.out.printf(
"%s / %s%n",
df.format(generated),
df.format(numberOfPasswordsToGenerate));
}
}
++chunk;
Files.writeString(passwordFile, passwords.toString(), StandardOpenOption.APPEND);
}
final long consumed = System.nanoTime() - start;
System.out.printf("Done. Took %d seconds%n", TimeUnit.NANOSECONDS.toSeconds(consumed));
}
private static char fetchRandomLetterFromAlphabet() {
return (char) (RANDOM.nextInt(MAX - MIN + 1) + MIN);
}
}
On my laptop, the program yields good results. It completes in about 33 seconds and all passwords are stored in a single file.
This program is a proof of concept and not production-ready. For example, if a password.txt does already exist, the content will be appended. For me, the file already has 1.7 GB after one run, so be aware of this. Furthermore, the generated passwords are temporarily stored in a StringBuilder, which may present a security risk since a StringBuilder cannot be cleared (i.e. its internal memory structured cannot be zeroed). Performance could further be improved by running the password generation multi-threaded, but I will leave this as an exercise to the reader.
To use the alphabet presented in the question, we can remove static fields MIN and MAX, define one new static field private static final char[] ALPHABET = "azertyuiopqsdfghjklmwxcvbnAZERTYUIOPQSDFGHJKLMWXCVBN1234567890".toCharArray(); and re-implement fetchRandomLetterFromAlphabet as:
private static char fetchRandomLetterFromAlphabet() {
return ALPHABET[RANDOM.nextInt(ALPHABET.length)];
}
We can use the following code-snippet to read-back the n-th (starting at 0) password from the file in constant time:
final int n = ...;
final RandomAccessFile raf = new RandomAccessFile(passwordFile.toString(), "r");
final long start = System.nanoTime();
final byte[] bytes = new byte[passwordLength];
// byte-length of the first n passwords, including line breaks:
final int offset = (passwordLength + System.lineSeparator().toCharArray().length) * n;
raf.seek(offset); // skip the first n passwords
raf.read(bytes);
// reset to the beginning of the file, in case we want to read more passwords later:
raf.seek(0);
System.out.println(new String(bytes));

I can give you some tips to optimize your code and make it faster, you can use them along with other.
If you know the number of passwords you need, you should create a string array and fill it with the variable in your loop.
If you have to use a dynamic size data structure, use linked list.
Linked list is better than array list when adding elements is your main target, and worse if you want to access them more then add them.
Use string builder instead of += operator on strings.
The += operator is very 'expensive' in complexity of time, because it always creates new strings. Using string builder append method can speed up your code.
Instead of using Math.random() and multiple the result to your range number, create a static Random object and use yourRandomInstance.next(int range).
Consider use the ascii table to get random character instead of using str.charAt(int index) method, it may speed up your code too, i offer you to check it.

Related

Java Multithreading Implementation for generating unique codes

My question is how I would implement multithreading to this task correctly.
I have a program that takes quite a long time to finish executing. About an hour and a half. I need to generate about 10,000 random and unique number codes. The code below is how I first implemented it and have it right now.
import java.util.Random;
import java.util.ArrayList;
public class Main
{
public static void main(String[] args) {
Random random = new Random();
// This holds all the codes
ArrayList<String> database = new ArrayList<>();
int counter = 0;
while(counter < 10000){
// Generate a 10 digit long code and append to sb
StringBuilder sb = new StringBuilder();
for(int i = 0; i < 10; i++){
sb.append(random.nextInt(10));
}
String code = String.valueOf(sb);
sb.setLength(0);
// Check if this code already exists in the database
// If not, then add the code and update counter
if(!database.contains(code)){
database.add(code);
counter++;
}
}
System.out.println("Done");
}
}
This of course is incredibly inefficient. So my question is: Is there is a way to implement multithreading that can work on this single piece of code? Best way I can word it is to give two cores/ threads the same code but have them both check the a single ArrayList? Both cores/ threads will generate codes but check to make sure the code it just made doesn't already exist either from the other core/ thread or from itself. I drew a rough diagram below. Any insight, advice, or pointers is greatly appreciated.
Using a more appropriate data structure and a more appropriate representation of the data, this should be a lot faster and easier to read, too:
Set<Long> database = new HashSet<>(10000);
while(database.size() < 10000){
database.add(ThreadLocalRandom.current().nextLong(10_000_000_000L);
}
Start with more obvious optimizations:
Do not use ArrayList, use HashSet. ArrayList contains() time complexity is O(n), while HashSet is O(1). Read this question about Big O summary for java collections framework. Read about Big O notation.
Initialize your collection with appropriate initial capacity. For your case that would be:
new HashSet<>(10000);
Like this underlying arrays won't be copied to increase their capacity. I would suggest to look/debug implementations of java collections to better understand how they work under the hood. Even try to implement them on your own.
Before you delve into complex multithreading optimizations, fix the simple problems - like bad collection choices.
Edit: As per suggestion from #Thomas in comments, you can directly generate a number(long) in the range you need - 0 to 9_999_999_999. You can see in this question how to do it. Stringify the resulting number and if length is less than 10, pad with leading zeroes.
Example:
(use ConcurrentHashMap, use threads, use random.nextLong())
public class Main {
static Map<String,Object> hashMapCache = new ConcurrentHashMap<String,Object>();
public static void main(String[] args) {
Random random = new Random();
// This holds all the codes
ArrayList<String> database = new ArrayList<>();
int counter = 0;
int NumOfThreads = 20;
int total = 10000;
int numberOfCreationsForThread = total/NumOfThreads;
int leftOver = total%NumOfThreads;
List<Thread> threadList = new ArrayList<>();
for(int i=0;i<NumOfThreads;i++){
if(i==0){
threadList.add(new Thread(new OneThread(numberOfCreationsForThread+leftOver,hashMapCache)));
}else {
threadList.add(new Thread(new OneThread(numberOfCreationsForThread,hashMapCache)));
}
}
for(int i=0;i<NumOfThreads;i++){
threadList.get(i).start();;
}
for(int i=0;i<NumOfThreads;i++){
try {
threadList.get(i).join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
for(String key : hashMapCache.keySet()){
database.add(key);
}
System.out.println("Done");
}}
OneThread:
public class OneThread implements Runnable{
int numberOfCreations;
Map<String,Object> hashMapCache;
public OneThread(int numberOfCreations,Map<String,Object> hashMapCache){
this.numberOfCreations = numberOfCreations;
this.hashMapCache = hashMapCache;
}
#Override
public void run() {
int counter = 0;
Random random = new Random();
System.out.println("thread "+ Thread.currentThread().getId() + " Start with " +numberOfCreations);
while(counter < numberOfCreations){
String code = generateRandom(random);
while (code.length()!=10){
code = generateRandom(random);
}
// Check if this code already exists in the database
// If not, then add the code and update counter
if(hashMapCache.get(code)==null){
hashMapCache.put(code,new Object());
counter++;
}
}
System.out.println("thread "+ Thread.currentThread().getId() + " end with " +numberOfCreations);
}
private static String generateRandom(Random random){
return String.valueOf(digits(random.nextLong(),10));
}
/** Returns val represented by the specified number of hex digits. */
private static String digits(long val, int digits) {
val = val > 0 ? val : val*-1;
return Long.toString(val).substring(0,digits);
}
}

Compartmentalizing loops over a large iteration

The Goal of my question is to enhance the performance of my algorithm by splitting the range of my loop iterations over a large array list.
For example: I have an Array list with a size of about 10 billion entries of long values, the goal I am trying to achieve is to start the loop from 0 to 100 million entries, output the result for the 100 million entries of whatever calculations inside the loop; then begin and 100 million to 200 million doing the previous and outputting the result, then 300-400million,400-500million and so on and so forth.
after I get all the 100 billion/100 million results, then I can sum them up outside of the loop collecting the results from the loop outputs parallel.
I have tried to use a range that might be able to achieve something similar by trying to use a dynamic range shift method but I cant seem to have the logic fully implemented like I would like to.
public static void tt4() {
long essir2 = 0;
long essir3 = 0;
List cc = new ArrayList<>();
List<Long> range = new ArrayList<>();
// break point is a method that returns list values, it was converted to
// string because of some concatenations and would be converted back to long here
for (String ari1 : Breakpoint()) {
cc.add(Long.valueOf(ari1));
}
// the size of the List is huge about 1 trillion entries at the minimum
long hy = cc.size() - 1;
for (long k = 0; k < hy; k++) {
long t1 = (long) cc.get((int) k);
long t2 = (long) cc.get((int) (k + 1));
// My main question: I am trying to iterate the entire list in a dynamic way
// which would exclude repeated endpoints on each iteration.
range = LongStream.rangeClosed(t1 + 1, t2)
.boxed()
.collect(Collectors.toList());
for (long i : range) {
// Hard is another method call on the iteration
// complexcalc is a method as well
essir2 = complexcalc((int) i, (int) Hard(i));
essir3 += essir2;
}
}
System.out.println("\n" + essir3);
}
I don't have any errors, I am just looking for a way to enhance performance and time. I can do a million entries in under a second directly, but when I put the size I require it runs forever. The size I'm giving are abstracts to illustrate size magnitudes, I don't want opinions like a 100 billion is not much, if I can do a million under a second, I'm talking massively huge numbers I need to iterate over doing complex tasks and calls, I just need help with the logic I'm trying to achieve if I can.
One thing I would suggest right off the bat would be to store your Breakpoint return value inside a simple array rather then using a List. This should improve your execution time significantly:
List<Long> cc = new ArrayList<>();
for (String ari1 : Breakpoint()) {
cc.add(Long.valueOf(ari1));
}
Long[] ccArray = cc.toArray(new Long[0]);
I believe what you're looking for is to split your tasks across multiple threads. You can do this with ExecutorService "which simplifies the execution of tasks in asynchronous mode".
Note that I am not overly familiar with this whole concept but have experimented with it a bit recently and give you a quick draft of how you could implement this.
I welcome those more experienced with multi-threading to either correct this post or provide additional information in the comments to help improve this answer.
Runnable Task class
public class CompartmentalizationTask implements Runnable {
private final ArrayList<Long> cc;
private final long index;
public CompartmentalizationTask(ArrayList<Long> list, long index) {
this.cc = list;
this.index = index;
}
#Override
public void run() {
Main.compartmentalize(cc, index);
}
}
Main class
private static ExecutorService exeService = Executors.newCachedThreadPool();
private static List<Future> futureTasks = new ArrayList<>();
public static void tt4() throws ExecutionException, InterruptedException
{
long essir2 = 0;
long essir3 = 0;
ArrayList<Long> cc = new ArrayList<>();
List<Long> range = new ArrayList<>();
// break point is a method that returns list values, it was converted to
// string because of some concatenations and would be converted back to long here
for (String ari1 : Breakpoint()) {
cc.add(Long.valueOf(ari1));
}
// the size of the List is huge about 1 trillion entries at the minimum
long hy = cc.size() - 1;
for (long k = 0; k < hy; k++) {
futureTasks.add(Main.exeService.submit(new CompartmentalizationTask(cc, k)));
}
for (int i = 0; i < futureTasks.size(); i++) {
futureTasks.get(i).get();
}
exeService.shutdown();
}
public static void compartmentalize(ArrayList<Long> cc, long index)
{
long t1 = (long) cc.get((int) index);
long t2 = (long) cc.get((int) (index + 1));
// My main question: I am trying to iterate the entire list in a dynamic way
// which would exclude repeated endpoints on each iteration.
range = LongStream.rangeClosed(t1 + 1, t2)
.boxed()
.collect(Collectors.toList());
for (long i : range) {
// Hard is another method call on the iteration
// complexcalc is a method as well
essir2 = complexcalc((int) i, (int) Hard(i));
essir3 += essir2;
}
}

Trying to assign a random ID to an object [duplicate]

I've been looking for a simple Java algorithm to generate a pseudo-random alpha-numeric string. In my situation it would be used as a unique session/key identifier that would "likely" be unique over 500K+ generation (my needs don't really require anything much more sophisticated).
Ideally, I would be able to specify a length depending on my uniqueness needs. For example, a generated string of length 12 might look something like "AEYGF7K0DM1X".
Algorithm
To generate a random string, concatenate characters drawn randomly from the set of acceptable symbols until the string reaches the desired length.
Implementation
Here's some fairly simple and very flexible code for generating random identifiers. Read the information that follows for important application notes.
public class RandomString {
/**
* Generate a random string.
*/
public String nextString() {
for (int idx = 0; idx < buf.length; ++idx)
buf[idx] = symbols[random.nextInt(symbols.length)];
return new String(buf);
}
public static final String upper = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
public static final String lower = upper.toLowerCase(Locale.ROOT);
public static final String digits = "0123456789";
public static final String alphanum = upper + lower + digits;
private final Random random;
private final char[] symbols;
private final char[] buf;
public RandomString(int length, Random random, String symbols) {
if (length < 1) throw new IllegalArgumentException();
if (symbols.length() < 2) throw new IllegalArgumentException();
this.random = Objects.requireNonNull(random);
this.symbols = symbols.toCharArray();
this.buf = new char[length];
}
/**
* Create an alphanumeric string generator.
*/
public RandomString(int length, Random random) {
this(length, random, alphanum);
}
/**
* Create an alphanumeric strings from a secure generator.
*/
public RandomString(int length) {
this(length, new SecureRandom());
}
/**
* Create session identifiers.
*/
public RandomString() {
this(21);
}
}
Usage examples
Create an insecure generator for 8-character identifiers:
RandomString gen = new RandomString(8, ThreadLocalRandom.current());
Create a secure generator for session identifiers:
RandomString session = new RandomString();
Create a generator with easy-to-read codes for printing. The strings are longer than full alphanumeric strings to compensate for using fewer symbols:
String easy = RandomString.digits + "ACEFGHJKLMNPQRUVWXYabcdefhijkprstuvwx";
RandomString tickets = new RandomString(23, new SecureRandom(), easy);
Use as session identifiers
Generating session identifiers that are likely to be unique is not good enough, or you could just use a simple counter. Attackers hijack sessions when predictable identifiers are used.
There is tension between length and security. Shorter identifiers are easier to guess, because there are fewer possibilities. But longer identifiers consume more storage and bandwidth. A larger set of symbols helps, but might cause encoding problems if identifiers are included in URLs or re-entered by hand.
The underlying source of randomness, or entropy, for session identifiers should come from a random number generator designed for cryptography. However, initializing these generators can sometimes be computationally expensive or slow, so effort should be made to re-use them when possible.
Use as object identifiers
Not every application requires security. Random assignment can be an efficient way for multiple entities to generate identifiers in a shared space without any coordination or partitioning. Coordination can be slow, especially in a clustered or distributed environment, and splitting up a space causes problems when entities end up with shares that are too small or too big.
Identifiers generated without taking measures to make them unpredictable should be protected by other means if an attacker might be able to view and manipulate them, as happens in most web applications. There should be a separate authorization system that protects objects whose identifier can be guessed by an attacker without access permission.
Care must be also be taken to use identifiers that are long enough to make collisions unlikely given the anticipated total number of identifiers. This is referred to as "the birthday paradox." The probability of a collision, p, is approximately n2/(2qx), where n is the number of identifiers actually generated, q is the number of distinct symbols in the alphabet, and x is the length of the identifiers. This should be a very small number, like 2‑50 or less.
Working this out shows that the chance of collision among 500k 15-character identifiers is about 2‑52, which is probably less likely than undetected errors from cosmic rays, etc.
Comparison with UUIDs
According to their specification, UUIDs are not designed to be unpredictable, and should not be used as session identifiers.
UUIDs in their standard format take a lot of space: 36 characters for only 122 bits of entropy. (Not all bits of a "random" UUID are selected randomly.) A randomly chosen alphanumeric string packs more entropy in just 21 characters.
UUIDs are not flexible; they have a standardized structure and layout. This is their chief virtue as well as their main weakness. When collaborating with an outside party, the standardization offered by UUIDs may be helpful. For purely internal use, they can be inefficient.
Java supplies a way of doing this directly. If you don't want the dashes, they are easy to strip out. Just use uuid.replace("-", "")
import java.util.UUID;
public class randomStringGenerator {
public static void main(String[] args) {
System.out.println(generateString());
}
public static String generateString() {
String uuid = UUID.randomUUID().toString();
return "uuid = " + uuid;
}
}
Output
uuid = 2d7428a6-b58c-4008-8575-f05549f16316
static final String AB = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
static SecureRandom rnd = new SecureRandom();
String randomString(int len){
StringBuilder sb = new StringBuilder(len);
for(int i = 0; i < len; i++)
sb.append(AB.charAt(rnd.nextInt(AB.length())));
return sb.toString();
}
If you're happy to use Apache classes, you could use org.apache.commons.text.RandomStringGenerator (Apache Commons Text).
Example:
RandomStringGenerator randomStringGenerator =
new RandomStringGenerator.Builder()
.withinRange('0', 'z')
.filteredBy(CharacterPredicates.LETTERS, CharacterPredicates.DIGITS)
.build();
randomStringGenerator.generate(12); // toUpperCase() if you want
Since Apache Commons Lang 3.6, RandomStringUtils is deprecated.
You can use an Apache Commons library for this, RandomStringUtils:
RandomStringUtils.randomAlphanumeric(20).toUpperCase();
In one line:
Long.toHexString(Double.doubleToLongBits(Math.random()));
Source: Java - generating a random string
This is easily achievable without any external libraries.
1. Cryptographic Pseudo Random Data Generation (PRNG)
First you need a cryptographic PRNG. Java has SecureRandom for that and typically uses the best entropy source on the machine (e.g. /dev/random). Read more here.
SecureRandom rnd = new SecureRandom();
byte[] token = new byte[byteLength];
rnd.nextBytes(token);
Note: SecureRandom is the slowest, but most secure way in Java of generating random bytes. I do however recommend not considering performance here since it usually has no real impact on your application unless you have to generate millions of tokens per second.
2. Required Space of Possible Values
Next you have to decide "how unique" your token needs to be. The whole and only point of considering entropy is to make sure that the system can resist brute force attacks: the space of possible values must be so large that any attacker could only try a negligible proportion of the values in non-ludicrous time1.
Unique identifiers such as random UUID have 122 bit of entropy (i.e., 2^122 = 5.3x10^36) - the chance of collision is "*(...) for there to be a one in a billion chance of duplication, 103 trillion version 4 UUIDs must be generated2". We will choose 128 bits since it fits exactly into 16 bytes and is seen as highly sufficient for being unique for basically every, but the most extreme, use cases and you don't have to think about duplicates. Here is a simple comparison table of entropy including simple analysis of the birthday problem.
For simple requirements, 8 or 12 byte length might suffice, but with 16 bytes you are on the "safe side".
And that's basically it. The last thing is to think about encoding so it can be represented as a printable text (read, a String).
3. Binary to Text Encoding
Typical encodings include:
Base64 every character encodes 6 bit, creating a 33% overhead. Fortunately there are standard implementations in Java 8+ and Android. With older Java you can use any of the numerous third-party libraries. If you want your tokens to be URL safe use the URL-safe version of RFC4648 (which usually is supported by most implementations). Example encoding 16 bytes with padding: XfJhfv3C0P6ag7y9VQxSbw==
Base32 every character encodes 5 bit, creating a 40% overhead. This will use A-Z and 2-7, making it reasonably space efficient while being case-insensitive alpha-numeric. There isn't any standard implementation in the JDK. Example encoding 16 bytes without padding: WUPIL5DQTZGMF4D3NX5L7LNFOY
Base16 (hexadecimal) every character encodes four bit, requiring two characters per byte (i.e., 16 bytes create a string of length 32). Therefore hexadecimal is less space efficient than Base32, but it is safe to use in most cases (URL) since it only uses 0-9 and A to F. Example encoding 16 bytes: 4fa3dd0f57cb3bf331441ed285b27735. See a Stack Overflow discussion about converting to hexadecimal here.
Additional encodings like Base85 and the exotic Base122 exist with better/worse space efficiency. You can create your own encoding (which basically most answers in this thread do), but I would advise against it, if you don't have very specific requirements. See more encoding schemes in the Wikipedia article.
4. Summary and Example
Use SecureRandom
Use at least 16 bytes (2^128) of possible values
Encode according to your requirements (usually hex or base32 if you need it to be alpha-numeric)
Don't
... use your home brew encoding: better maintainable and readable for others if they see what standard encoding you use instead of weird for loops creating characters at a time.
... use UUID: it has no guarantees on randomness; you are wasting 6 bits of entropy and have a verbose string representation
Example: Hexadecimal Token Generator
public static String generateRandomHexToken(int byteLength) {
SecureRandom secureRandom = new SecureRandom();
byte[] token = new byte[byteLength];
secureRandom.nextBytes(token);
return new BigInteger(1, token).toString(16); // Hexadecimal encoding
}
//generateRandomHexToken(16) -> 2189df7475e96aa3982dbeab266497cd
Example: Base64 Token Generator (URL Safe)
public static String generateRandomBase64Token(int byteLength) {
SecureRandom secureRandom = new SecureRandom();
byte[] token = new byte[byteLength];
secureRandom.nextBytes(token);
return Base64.getUrlEncoder().withoutPadding().encodeToString(token); //base64 encoding
}
//generateRandomBase64Token(16) -> EEcCCAYuUcQk7IuzdaPzrg
Example: Java CLI Tool
If you want a ready-to-use CLI tool you may use dice:
Example: Related issue - Protect Your Current Ids
If you already have an id you can use (e.g., a synthetic long in your entity), but don't want to publish the internal value, you can use this library to encrypt it and obfuscate it: https://github.com/patrickfav/id-mask
IdMask<Long> idMask = IdMasks.forLongIds(Config.builder(key).build());
String maskedId = idMask.mask(id);
// Example: NPSBolhMyabUBdTyanrbqT8
long originalId = idMask.unmask(maskedId);
Using Dollar should be as simple as:
// "0123456789" + "ABCDE...Z"
String validCharacters = $('0', '9').join() + $('A', 'Z').join();
String randomString(int length) {
return $(validCharacters).shuffle().slice(length).toString();
}
#Test
public void buildFiveRandomStrings() {
for (int i : $(5)) {
System.out.println(randomString(12));
}
}
It outputs something like this:
DKL1SBH9UJWC
JH7P0IT21EA5
5DTI72EO6SFU
HQUMJTEBNF7Y
1HCR6SKYWGT7
Here it is in Java:
import static java.lang.Math.round;
import static java.lang.Math.random;
import static java.lang.Math.pow;
import static java.lang.Math.abs;
import static java.lang.Math.min;
import static org.apache.commons.lang.StringUtils.leftPad
public class RandomAlphaNum {
public static String gen(int length) {
StringBuffer sb = new StringBuffer();
for (int i = length; i > 0; i -= 12) {
int n = min(12, abs(i));
sb.append(leftPad(Long.toString(round(random() * pow(36, n)), 36), n, '0'));
}
return sb.toString();
}
}
Here's a sample run:
scala> RandomAlphaNum.gen(42)
res3: java.lang.String = uja6snx21bswf9t89s00bxssu8g6qlu16ffzqaxxoy
A short and easy solution, but it uses only lowercase and numerics:
Random r = new java.util.Random ();
String s = Long.toString (r.nextLong () & Long.MAX_VALUE, 36);
The size is about 12 digits to base 36 and can't be improved further, that way. Of course you can append multiple instances.
Surprising, no one here has suggested it, but:
import java.util.UUID
UUID.randomUUID().toString();
Easy.
The benefit of this is UUIDs are nice, long, and guaranteed to be almost impossible to collide.
Wikipedia has a good explanation of it:
" ...only after generating 1 billion UUIDs every second for the next 100 years, the probability of creating just one duplicate would be about 50%."
The first four bits are the version type and two for the variant, so you get 122 bits of random. So if you want to, you can truncate from the end to reduce the size of the UUID. It's not recommended, but you still have loads of randomness, enough for your 500k records easy.
An alternative in Java 8 is:
static final Random random = new Random(); // Or SecureRandom
static final int startChar = (int) '!';
static final int endChar = (int) '~';
static String randomString(final int maxLength) {
final int length = random.nextInt(maxLength + 1);
return random.ints(length, startChar, endChar + 1)
.collect(StringBuilder::new, StringBuilder::appendCodePoint, StringBuilder::append)
.toString();
}
public static String generateSessionKey(int length){
String alphabet =
new String("0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"); // 9
int n = alphabet.length(); // 10
String result = new String();
Random r = new Random(); // 11
for (int i=0; i<length; i++) // 12
result = result + alphabet.charAt(r.nextInt(n)); //13
return result;
}
import java.util.Random;
public class passGen{
// Version 1.0
private static final String dCase = "abcdefghijklmnopqrstuvwxyz";
private static final String uCase = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
private static final String sChar = "!##$%^&*";
private static final String intChar = "0123456789";
private static Random r = new Random();
private static StringBuilder pass = new StringBuilder();
public static void main (String[] args) {
System.out.println ("Generating pass...");
while (pass.length () != 16){
int rPick = r.nextInt(4);
if (rPick == 0){
int spot = r.nextInt(26);
pass.append(dCase.charAt(spot));
} else if (rPick == 1) {
int spot = r.nextInt(26);
pass.append(uCase.charAt(spot));
} else if (rPick == 2) {
int spot = r.nextInt(8);
pass.append(sChar.charAt(spot));
} else {
int spot = r.nextInt(10);
pass.append(intChar.charAt(spot));
}
}
System.out.println ("Generated Pass: " + pass.toString());
}
}
This just adds the password into the string and... yeah, it works well. Check it out... It is very simple; I wrote it.
Using UUIDs is insecure, because parts of the UUID aren't random at all. The procedure of erickson is very neat, but it does not create strings of the same length. The following snippet should be sufficient:
/*
* The random generator used by this class to create random keys.
* In a holder class to defer initialization until needed.
*/
private static class RandomHolder {
static final Random random = new SecureRandom();
public static String randomKey(int length) {
return String.format("%"+length+"s", new BigInteger(length*5/*base 32,2^5*/, random)
.toString(32)).replace('\u0020', '0');
}
}
Why choose length*5? Let's assume the simple case of a random string of length 1, so one random character. To get a random character containing all digits 0-9 and characters a-z, we would need a random number between 0 and 35 to get one of each character.
BigInteger provides a constructor to generate a random number, uniformly distributed over the range 0 to (2^numBits - 1). Unfortunately 35 is not a number which can be received by 2^numBits - 1.
So we have two options: Either go with 2^5-1=31 or 2^6-1=63. If we would choose 2^6 we would get a lot of "unnecessary" / "longer" numbers. Therefore 2^5 is the better option, even if we lose four characters (w-z). To now generate a string of a certain length, we can simply use a 2^(length*numBits)-1 number. The last problem, if we want a string with a certain length, random could generate a small number, so the length is not met, so we have to pad the string to its required length prepending zeros.
I found this solution that generates a random hex encoded string. The provided unit test seems to hold up to my primary use case. Although, it is slightly more complex than some of the other answers provided.
/**
* Generate a random hex encoded string token of the specified length
*
* #param length
* #return random hex string
*/
public static synchronized String generateUniqueToken(Integer length){
byte random[] = new byte[length];
Random randomGenerator = new Random();
StringBuffer buffer = new StringBuffer();
randomGenerator.nextBytes(random);
for (int j = 0; j < random.length; j++) {
byte b1 = (byte) ((random[j] & 0xf0) >> 4);
byte b2 = (byte) (random[j] & 0x0f);
if (b1 < 10)
buffer.append((char) ('0' + b1));
else
buffer.append((char) ('A' + (b1 - 10)));
if (b2 < 10)
buffer.append((char) ('0' + b2));
else
buffer.append((char) ('A' + (b2 - 10)));
}
return (buffer.toString());
}
#Test
public void testGenerateUniqueToken(){
Set set = new HashSet();
String token = null;
int size = 16;
/* Seems like we should be able to generate 500K tokens
* without a duplicate
*/
for (int i=0; i<500000; i++){
token = Utility.generateUniqueToken(size);
if (token.length() != size * 2){
fail("Incorrect length");
} else if (set.contains(token)) {
fail("Duplicate token generated");
} else{
set.add(token);
}
}
}
Change String characters as per as your requirements.
String is immutable. Here StringBuilder.append is more efficient than string concatenation.
public static String getRandomString(int length) {
final String characters = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJLMNOPQRSTUVWXYZ1234567890!##$%^&*()_+";
StringBuilder result = new StringBuilder();
while(length > 0) {
Random rand = new Random();
result.append(characters.charAt(rand.nextInt(characters.length())));
length--;
}
return result.toString();
}
import java.util.Date;
import java.util.Random;
public class RandomGenerator {
private static Random random = new Random((new Date()).getTime());
public static String generateRandomString(int length) {
char[] values = {'a','b','c','d','e','f','g','h','i','j',
'k','l','m','n','o','p','q','r','s','t',
'u','v','w','x','y','z','0','1','2','3',
'4','5','6','7','8','9'};
String out = "";
for (int i=0;i<length;i++) {
int idx=random.nextInt(values.length);
out += values[idx];
}
return out;
}
}
I don't really like any of these answers regarding a "simple" solution :S
I would go for a simple ;), pure Java, one liner (entropy is based on random string length and the given character set):
public String randomString(int length, String characterSet) {
return IntStream.range(0, length).map(i -> new SecureRandom().nextInt(characterSet.length())).mapToObj(randomInt -> characterSet.substring(randomInt, randomInt + 1)).collect(Collectors.joining());
}
#Test
public void buildFiveRandomStrings() {
for (int q = 0; q < 5; q++) {
System.out.println(randomString(10, "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789")); // The character set can basically be anything
}
}
Or (a bit more readable old way)
public String randomString(int length, String characterSet) {
StringBuilder sb = new StringBuilder(); // Consider using StringBuffer if needed
for (int i = 0; i < length; i++) {
int randomInt = new SecureRandom().nextInt(characterSet.length());
sb.append(characterSet.substring(randomInt, randomInt + 1));
}
return sb.toString();
}
#Test
public void buildFiveRandomStrings() {
for (int q = 0; q < 5; q++) {
System.out.println(randomString(10, "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789")); // The character set can basically be anything
}
}
But on the other hand you could also go with UUID which has a pretty good entropy:
UUID.randomUUID().toString().replace("-", "")
I'm using a library from Apache Commons to generate an alphanumeric string:
import org.apache.commons.lang3.RandomStringUtils;
String keyLength = 20;
RandomStringUtils.randomAlphanumeric(keylength);
It's fast and simple!
You mention "simple", but just in case anyone else is looking for something that meets more stringent security requirements, you might want to take a look at jpwgen. jpwgen is modeled after pwgen in Unix, and is very configurable.
import java.util.*;
import javax.swing.*;
public class alphanumeric {
public static void main(String args[]) {
String nval, lenval;
int n, len;
nval = JOptionPane.showInputDialog("Enter number of codes you require: ");
n = Integer.parseInt(nval);
lenval = JOptionPane.showInputDialog("Enter code length you require: ");
len = Integer.parseInt(lenval);
find(n, len);
}
public static void find(int n, int length) {
String str1 = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ";
StringBuilder sb = new StringBuilder(length);
Random r = new Random();
System.out.println("\n\t Unique codes are \n\n");
for(int i=0; i<n; i++) {
for(int j=0; j<length; j++) {
sb.append(str1.charAt(r.nextInt(str1.length())));
}
System.out.println(" " + sb.toString());
sb.delete(0, length);
}
}
}
Here is the one-liner by abacus-common:
String.valueOf(CharStream.random('0', 'z').filter(c -> N.isLetterOrDigit(c)).limit(12).toArray())
Random doesn't mean it must be unique. To get unique strings, use:
N.uuid() // E.g.: "e812e749-cf4c-4959-8ee1-57829a69a80f". length is 36.
N.guid() // E.g.: "0678ce04e18945559ba82ddeccaabfcd". length is 32 without '-'
You can use the following code, if your password mandatory contains numbers and alphabetic special characters:
private static final String NUMBERS = "0123456789";
private static final String UPPER_ALPHABETS = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
private static final String LOWER_ALPHABETS = "abcdefghijklmnopqrstuvwxyz";
private static final String SPECIALCHARACTERS = "##$%&*";
private static final int MINLENGTHOFPASSWORD = 8;
public static String getRandomPassword() {
StringBuilder password = new StringBuilder();
int j = 0;
for (int i = 0; i < MINLENGTHOFPASSWORD; i++) {
password.append(getRandomPasswordCharacters(j));
j++;
if (j == 3) {
j = 0;
}
}
return password.toString();
}
private static String getRandomPasswordCharacters(int pos) {
Random randomNum = new Random();
StringBuilder randomChar = new StringBuilder();
switch (pos) {
case 0:
randomChar.append(NUMBERS.charAt(randomNum.nextInt(NUMBERS.length() - 1)));
break;
case 1:
randomChar.append(UPPER_ALPHABETS.charAt(randomNum.nextInt(UPPER_ALPHABETS.length() - 1)));
break;
case 2:
randomChar.append(SPECIALCHARACTERS.charAt(randomNum.nextInt(SPECIALCHARACTERS.length() - 1)));
break;
case 3:
randomChar.append(LOWER_ALPHABETS.charAt(randomNum.nextInt(LOWER_ALPHABETS.length() - 1)));
break;
}
return randomChar.toString();
}
You can use the UUID class with its getLeastSignificantBits() message to get 64 bit of random data, and then convert it to a radix 36 number (i.e. a string consisting of 0-9,A-Z):
Long.toString(Math.abs( UUID.randomUUID().getLeastSignificantBits(), 36));
This yields a string up to 13 characters long. We use Math.abs() to make sure there isn't a minus sign sneaking in.
Here it is a Scala solution:
(for (i <- 0 until rnd.nextInt(64)) yield {
('0' + rnd.nextInt(64)).asInstanceOf[Char]
}) mkString("")
Using an Apache Commons library, it can be done in one line:
import org.apache.commons.lang.RandomStringUtils;
RandomStringUtils.randomAlphanumeric(64);
Documentation
public static String randomSeriesForThreeCharacter() {
Random r = new Random();
String value = "";
char random_Char ;
for(int i=0; i<10; i++)
{
random_Char = (char) (48 + r.nextInt(74));
value = value + random_char;
}
return value;
}
I think this is the smallest solution here, or nearly one of the smallest:
public String generateRandomString(int length) {
String randomString = "";
final char[] chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz01234567890".toCharArray();
final Random random = new Random();
for (int i = 0; i < length; i++) {
randomString = randomString + chars[random.nextInt(chars.length)];
}
return randomString;
}
The code works just fine. If you are using this method, I recommend you to use more than 10 characters. A collision happens at 5 characters / 30362 iterations. This took 9 seconds.
public class Utils {
private final Random RANDOM = new SecureRandom();
private final String ALPHABET = "0123456789QWERTYUIOPASDFGHJKLZXCVBNMqwertyuiopasdfghjklzxcvbnm";
private String generateRandomString(int length) {
StringBuffer buffer = new StringBuffer(length);
for (int i = 0; i < length; i++) {
buffer.append(ALPHABET.charAt(RANDOM.nextInt(ALPHABET.length())));
}
return new String(buffer);
}
}

counting elements in an array imported from a data file

I am writing a program that will import values from a txt file in to an array, I then need to count how many of those elements are greater than or equal to 36. The data imports fine, and the total amount of values it displays is correct, but I can not get it display the amount of times the number 36 is found in the file. Thanks for any help!
public static void main(String[] args) throws Exception {
int[] enrollments = new int [100];
int count;
int FullClass;
double ClassPercentage;
return count (number of data items)
count = CreateArray(enrollments);
System.out.println (count );
FullClass = AddValues (enrollments);
System.out.println (FullClass)
ClassPercentage= FullClass/count;
System.out.print(ClassPercentage +"% of classes are full");
}//end main
/**
*
* #param classSizes
*/
public static int CreateArray(int[] classSizes) throws Exception{
int count = 0;
File enrollments = new File("enrollments.txt");
Scanner infile = new Scanner (enrollments);
while (infile.hasNextInt()){
classSizes[count] = infile.nextInt();
count++}//end while
return count; //number of items in an array
} // end CreateArray
/**************************************************************************/
/**
*
* #throws java.lang.Exception
*/
public static int AddValues (int[] enrollments) throws Exception{
{
int number = 0;
int countOf36s = 0;
while (infile.hasNextInt()) {
number = infile.next();
classSizes[count] = number;
if(number>=36) {
countOf36s++;
}
count++;
}
return countOf36s;
}// end AddValues
}//end main
Try this code to count the numbers that are greater than or equal to 36 while you are reading the file only. Change the code in your createArray method or write the below logic where ever you want to.
I tried executing this program. It works as expected. See below code
import java.util.*;
import java.io.*;
public class Test { //Name this to your actual class name
public static void main(String[] args) throws Exception {
int[] enrollments = new int [100]; //assuming not more than 100 numbers in the text file
int count; //count of all the numbers in text file
int FullClass; //count of numbers whose value is >=36
double ClassPercentage;
count = CreateArray(enrollments);
System.out.println (count);
FullClass = AddValues (enrollments);
System.out.println (FullClass);
ClassPercentage= FullClass/count;
System.out.print(ClassPercentage +"% of classes are full");
}
//Method to read all the numbers from the text file and store them in the array
public static int CreateArray(int[] classSizes) throws Exception {
int count = 0;
File enrollments = new File("enrollments.txt"); //path should be correct or else you get an exception.
Scanner infile = new Scanner (enrollments);
while (infile.hasNextInt()) {
classSizes[count] = infile.nextInt();
count++;
}
return count; //number of items in an array
}
//Method to read numbers from the array and store the count of numbers >=36
public static int AddValues (int[] enrollments) throws Exception{
int number = 0;
int countOf36s = 0;
for(int i=0; i<enrollments.length; i++) {
number = enrollments[i];
if(number>=36) {
countOf36s++;
}
}
return countOf36s;
}
}
Your code indicates that you might have misunderstood a couple of concepts and stylistic things. As you say in your comments you are new at this and would like some guidance as well as the answer to the question - here it is:
Style
Method names and variable names are by convention written starting with a lower case letter and then in camel case. This is in contrast to classes that are named starting with an upper case letter and camel case. Sticking to these conventions make code easier to read and maintain. A full list of conventions is published - this comment particularly refers to naming conventions.
Similarly, by convention, closing braces are put on a separate line when they close loops or if-else blocks.
throws Exception is very general - it's usual to limit as much as possible what Exceptions your code actually throws - in your case throws FileNotFoundException should be sufficient as this is what Scanner or File can throw at runtime. This specificity can be useful to any code that uses any of your code in the future.
Substance
You are creating the array up front with 100 members. You then call CreateArray which reads from a file while that file has more integers in it. Your code does not know how many that is - let's call it N. If N <= 100 (there are 100 integers or less), that's fine and your array will be populated from 0 to N-1. This approach is prone to confusion, though - the length of your array will be 100 no matter how many values it has read from the file - so you have to keep track of the count returned by CreateArray.
If N > 100 you have trouble - the file reading code will keep going, trying to add numbers to the array beyond its maximum index and you will get a runtime error (index out of bounds)
A better approach might be to have CreateArray return an ArrayList, which can have dynamic length and you can check how many there are using ArrayList.size()
Your original version of AddValues called CreateArray a second time, even though you pass in the array which already contains the values read from file. This is inefficient as it does all the file I/O again. Not a problem with this small example, but you should avoid duplication in general.
The main problem. As per prudhvi you are checking the number of integers in the file against 36, not each value. You can rectify this as suggested in that answer.
You do ClassPercentage= FullClass/count; Although ClassPercentage is a double, somewhat counter intuitively - because both the variables on the Right Hand Side (RHS) are int, you will have an int returned from the division which will always round down to zero. To make this work properly - you have to change (cast) one of the variables on the RHS to double before division e.g. ClassPercentage= ((double)FullClass)/count;.
If you do keep using arrays rather than ArrayList, be careful what happens when you pass them into methods. You are passing by reference, which means that if you change an element of an array in your method, it remains changed when you return from that method.
In your new version you do
...
classSizes[count] = number;
if(number>=36) {
...
You almost certainly mean
...
number = classSizes[count];
if(number>=36) {
...
which is to say in programing the order of the assignment equals is important, so a = b is not equivalent to b = a
Code
A cleaned up version of your code - observing all the above (I hope):
import java.io.File;
import java.io.FileNotFoundException;
import java.util.ArrayList;
import java.util.Scanner;
public class ClassCounter
{
public static void main(String[] args) throws FileNotFoundException
{
int count;
int fullClass;
double classPercentage;
ArrayList<Integer> enrollments = createArray();
count = enrollments.size();
System.out.println(count);
fullClass = addValues(enrollments);
System.out.println(fullClass);
classPercentage = fullClass / count;
System.out.print(classPercentage + "% of classes are full");
}
/**
* scans file "enrollments.txt", which must contain a list of integers, and
* returns an ArrayList populated with those integers.
*
* #throws FileNotFoundException
*/
public static ArrayList<Integer> createArray() throws FileNotFoundException
{
ArrayList<Integer> listToReturn = new ArrayList<Integer>();
File enrollments = new File("enrollments.txt");
Scanner infile = new Scanner(enrollments);
while (infile.hasNextInt())
{
listToReturn.add(infile.nextInt());
}
return listToReturn;
}
/**
* returns the number of cases where enrollments >= 36 from the list of
* all enrollments
*
* #param enrollments - the list of enrollments in each class
* #throws FileNotFoundException
*/
public static int addValues(ArrayList<Integer> enrollments)
{
int number = 0;
int countOf36s = 0;
int i = 0;
while (i < enrollments.size())
{
number = enrollments.get(i);
if (number >= 36)
{
countOf36s++;
}
}
return countOf36s;
}
}

Unique random number for a particular timestamp

I am kind of learning concepts of Random number generation & Multithreading in java.
The idea is to not generating a repeated random number of range 1000 in a particular millisecond (Considering, not more than 50 data, in a multithreaded way will be processed in a millisecond). So that list of generated random number at the specific time is unique. Can you give me any idea as i am ending up generating couple of repeated random numbers (also, there is a considerable probability) in a particular milli second.
I have tried the following things where i failed.
Random random = new Random(System.nanoTime());
double randomNum = random.nextInt(999);
//
int min=1; int max=999;
double randomId = (int)Math.abs(math.Random()* (max - min + 1) + min);
//
Random random = new Random(System.nanoTime()); // also tried new Random();
double randomId = (int)Math.abs(random.nextDouble()* (max - min + 1) + min);
As I am appending the timestamp that is being generated, in a multithreaded environment i see the same ids (around 8-10) that is being generated (2-4 times) for 5000+ unique data.
First, you should use new Random(), since it looks like this (details depend on Java version):
public Random() { this(++seedUniquifier + System.nanoTime()); }
private static volatile long seedUniquifier = 8682522807148012L;
I.e. it already makes use of nanoTime() and makes sure different threads with the same nanoTime() result get different seeds, which new Random(System.nanoTime()) doesn't.
(EDIT: Pyranja pointed out this is a bug in Java 6, but it's fixed in Java 7:
public Random() {
this(seedUniquifier() ^ System.nanoTime());
}
private static long seedUniquifier() {
// L'Ecuyer, "Tables of Linear Congruential Generators of
// Different Sizes and Good Lattice Structure", 1999
for (;;) {
long current = seedUniquifier.get();
long next = current * 181783497276652981L;
if (seedUniquifier.compareAndSet(current, next))
return next;
}
}
private static final AtomicLong seedUniquifier
= new AtomicLong(8682522807148012L);
)
Second, if you generate 50 random numbers from 1 to 1000, the probability some numbers will be the same is quite high thanks to the birthday paradox.
Third, if you just want unique ids, you could just use AtomicInteger counter instead of random numbers. Or if you want a random part to start with, append a counter as well to guarantee uniqueness.
This class will allow you to get nonrepeating values from a certain range until the whole range has been used. Once the range is used, it will be reinitialized.
Class comes along with a simple test.
If you want to make the class thread safe, just add synchronized to nextInt() declaration.
Then you can use the singleton pattern or just a static variable to access the generator from multiple threads. That way all your threads will use the same object and the same unique id pool.
public class NotRepeatingRandom {
int size;
int index;
List<Integer> vals;
Random gen = new Random();
public NotRepeatingRandom(int rangeMax) {
size = rangeMax;
index = rangeMax; // to force initial shuffle
vals = new ArrayList<Integer>(size);
fillBaseList();
}
private void fillBaseList() {
for (int a=0; a<size; a++) {
vals.add(a);
}
}
public int nextInt() {
if (index == vals.size()) {
Collections.shuffle(vals);
index = 0;
}
int val = vals.get(index);
index++;
return val;
}
public static void main(String[] args) {
NotRepeatingRandom gen = new NotRepeatingRandom(10);
for (int a=0; a<30; a++) {
System.out.println(gen.nextInt());
}
}
}
If I understand your question correctly, multiple threads are creating their own instances of Random class at the same time and all threads generate the same random number?
Same number is generated, because all random instances where created at the same time, i.e. with the same seed.
To fix this, create only one instance of Random class, which is shared by all threads so that all your threads call nextDouble() on the same instance. Random.nextDouble() class is thread safe and will implicitly update its seed with every call.
//create only one Random instance, seed is based on current time
public static final Random generator= new Random();
Now all threads should use the same instance:
double random=generator.nextDouble()

Categories

Resources