Develop Reference

Assigning a value to a MacAddress variable

In a part of my code, I'm trying to assign a value to a MacAdress variable (dstMac). I also have an int variable called indexMax whose value can be 0 or 1 or 2 (it will be changed in my program).
Now what I'm trying to do is to assign the value to dstMac based on the indexMax. I mean, if indexMax = 0 then dstMac = value0, if indexMax = 1 then dstMac = value1 and ... (the name of the variable is chosen based on the value of indexMax)
MacAddress dstMac;
MacAddress value0 = MacAddress.of("00:00:00:00:00:01");
MacAddress value1 = MacAddress.of("00:00:00:00:00:02");
MacAddress value2 = MacAddress.of("00:00:00:00:00:11");
if (maxvalue > 50){
dstMac = value%d, indexMax // Just for showing what I'm trying to do, I know that this is wrong
}
I tried something like dstMac = String.format("value%d", indexMax);, but I received this error:
Type mismatch: cannot convert from String to MacAddress
So how can I do this?

To achieve this logic you can use some structure to save you mac values and then choose one from it. Just a sample:
List<MacAddress> addresses = Arrays.asList(value0, value1, value2);
MacAddress dstMac = Collections.min(addresses, new Comparator<MacAddress> {
public int compare(MacAddress a, MacAddress b) { // some custom logic to choose min mac address, for example
if (a > b)
return 1;
if (a == b)
return 0;
return -1;
}
});
Or you can simply iterate it and return first acceptable:
List<MacAddress> addresses = Arrays.asList(value0, value1, value2);
MacAddress dstMac = selectOne(addresses);
....
MacAddress selectOne(List<MacAddress> addresses) {
for (MacAddress mac : addresses) {
if (some condition) {
return mac
}
}
return null; // if there is no acceptable mac address
}

Convert String representation to minimal Number Object

Having a String representation of a number(no decimals), what's the best way to convert it to either one of java.lang.Integer or java.lang.Long or java.math.BigInteger? The only condition is that the converted type should be of minimal datatype required to hold the number.
I've this current implementation that works fine, but I would like to know if there's a better code without exception handling.
package com.stackoverflow.programmer;
import java.math.BigInteger;
public class Test {
public static void main(String[] args) {
String number = "-12121111111111111";
Number numberObject = null;
try {
numberObject = Integer.valueOf(number);
} catch (NumberFormatException nfe) {
System.out.println("Number will not fit into Integer type. Trying Long...");
try {
numberObject = Long.valueOf(number);
} catch (NumberFormatException nfeb) {
System.out.println("Number will not fit into Long type. Trying BigInteger...");
numberObject = new BigInteger(number);
}
}
System.out.println(numberObject.getClass() + " : "
+ numberObject.toString());
}
}

From what you said, here is what I would have done:
import java.math.BigInteger;
import java.util.Arrays;
import java.util.List;
public class TestSO09_39463168_StringToMinimalNumber {
public static void main(String[] args) {
List<String> strNumbers = Arrays.asList("0", //int
"123", //int
"-456", //int
"2147483700", // Long
"-2147483700", // Long
"9223372036854775900", //BigInt
"-9223372036854775900" //BigInt
);
for(String strNumber : strNumbers){
Number number = stringToMinimalNumber(strNumber);
System.out.println("The string '"+strNumber+"' is a "+number.getClass());
}
}
public static Number stringToMinimalNumber(String s){
BigInteger tempNumber = new BigInteger(s);
if(tempNumber.compareTo(BigInteger.valueOf(Long.MAX_VALUE)) > 0 || tempNumber.compareTo(BigInteger.valueOf(Long.MIN_VALUE)) < 0){
return tempNumber;
} else if(tempNumber.compareTo(BigInteger.valueOf(Integer.MAX_VALUE)) > 0 || tempNumber.compareTo(BigInteger.valueOf(Integer.MIN_VALUE)) < 0){
return tempNumber.longValue(); //Autobox to Long
} else {
return tempNumber.intValue(); //Autobox to Integer
}
}
}
You must use a temporary BigInteger, or else you'll end up with lazarov's solution, which is correct, but you can't really do something like that for reason mentionned in the comments.
Anyway, every BigInteger (the ones that are not returned) will be garbage collected. As for autoboxing, I don't think it's that of a bad thing. You could also make "BigInteger.valueOf(Long.MAX_VALUE))" as a constant. Maybe the compiler or the JVM will do this on its own.
I'm not really sure of how efficient it is, and using only BigInteger might be a good idea (as Spotted did), because I serioulsy doubt it would really improve the rest of your code to use the right size, and it might even be error prone if you try to use these Numbers with each other ... But again, it all depend on what you need. (and yes, using Exception as flow control is a really bad idea, but you can add a try catch on the BigInteger tempNumber = new BigInteger(s); to throw your own exception if s is not a number at all)
For recreational purpose, I have made the solution without using a BigInteger, and only with String parsing (this is still not what I recommand to do, but it was fun :)
public static final String INT_MAX_VALUE = "2147483647";
public static final String LONG_MAX_VALUE = "9223372036854775807";
public static Number stringToMinimalNumberWithoutBigInteger(String numberStr){
//Removing the minus sign to test the value
String s = (numberStr.startsWith("-") ? numberStr.substring(1,numberStr.length()) : numberStr);
if(compareStringNumber(s, LONG_MAX_VALUE) > 0){
return new BigInteger(numberStr);
} else if(compareStringNumber(s, INT_MAX_VALUE) > 0){
return new Long(numberStr);
} else {
return new Integer(numberStr);
}
}
//return postive if a > b, negative if a < b, 0 if equals;
private static int compareStringNumber(String a, String b){
if(a.length() != b.length()){
return a.length() - b.length();
}
for(int i = 0; i < a.length(); i++){
if( a.codePointAt(i) != b.codePointAt(i) ){ //Or charAt()
return a.codePointAt(i) - b.codePointAt(i);
}
}
return 0;
}

Please don't use exceptions for handling flow control, this is a serious anti-pattern (also here).
As you mentionned in the comments, the real thing you've been asked is to convert a List<String> into a List<Number>.
Also, if I understand correctly, you know that:
You should encounter only numbers without decimals
The biggest value you can encounter is possibly unbound
Based on that, the following method will do the job in a more clever way:
private static List<Number> toNumbers(List<String> strings) {
return strings.stream()
.map(BigInteger::new)
.collect(Collectors.toList());
}
Eidt: if you're not very familiar with the stream concept, here's the equivalent code without streams:
private static List<Number> toNumbers(List<String> strings) {
List<Number> numbers = new ArrayList<>();
for (String s : strings) {
numbers.add(new BigInteger(s));
}
return numbers;
}

Well if you want to do it "by hand" try something like this:
We define the max values as strings :
String intMax = "2147483647";
String longMax = "9223372036854775807";
and our number:
String ourNumber = "1234567890"
Now our logic will be simple :
We will check lenghts of strings firstly
If our numbers length < int max length : IT IS INT
If our numbers length == int max length : Check is it INT or LONG
If our numbers length > int max length :
3.1 If our numbers length < long max length : IT IS LONG
3.2 If our numbers length == long max length : Check is it LONG or BIG INTEGER
3.3 If our numbers length > long max length : IT IS BIG INTEGER
The code should look something like this (I have not tried to compile it may have syntax or other errors) :
if(ourNumber.lenght() < intMax.length ){
System.out.println("It is an Integer");
} else if(ourNumber.lenght() == intMax.length){
// it can be int if the number is between 2000000000 and 2147483647
char[] ourNumberToCharArray = ourNumber.toCharArray();
char[] intMaxToCharArray = intMax.toCharArray();
int diff = 0;
for(int i = 0; i < ourNumberToCharArray.length; i++) {
diff = Character.getNumericValue(intMaxToCharArray[i]) - Character.getNumericValue(ourNumberToCharArray[i]);
if(diff > 0) {
System.out.println("It is a Long");
break;
} else if(diff < 0) {
System.out.println("It is an Integer");
break;
}
}
if(diff == 0){
System.out.println("It is an Integer");
}
} else {
if(ourNumber.lenght() < longMax.length()) {
System.out.println("It is a Long");
} else if(ourNumber.lenght() == longMax.length()){
char[] ourNumberToCharArray = ourNumber.toCharArray();
char[] longMaxToCharArray = longMax.toCharArray();
int diff = 0;
for(int i = 0; i < ourNumberToCharArray.length; i++) {
diff = Character.getNumericValue(longMaxToCharArray[i]) - Character.getNumericValue(ourNumberToCharArray[i]);
if(diff > 0) {
System.out.println("It is a BigInteger");
break;
} else if(diff < 0) {
System.out.println("It is a Long");
break;
}
}
if(diff == 0){
System.out.println("It is a Long");
}
} else {
System.out.println("It is a BigInteger");
}
}
Then logic that checks if the numbers match or not is the same in both cases you can but it in a function for example.

nearest neighbor algorithm copy element (city) to output array java

So I have a program written so far that reads in a csv file of cities and distances in the following format:
Alaska Mileage Chart,Anchorage,Anderson,Cantwell,
Anchorage,0,284,210,
Anderson,284,0,74,
Cantwell,210,74,0,
So the algorithm works and outputs the cities in the order they should be visited following the shortest path using the nearest neighbor algorithm always starting with Anchorage as the city of origin or starting city.
Using this data, the example output for the algorithm is: 1,3,2. I have ran this with a 27 element chart and had good results as well. I am using this small one for writing and debugging purposes.
Ideally the output I am looking for is the Name of the City and a cumulative milage.
Right now I am having working on trying to get the cities into an array that I can print out. Help with both parts would be appreciated or help keeping in mind that is the end goal is appreciated as well.
My thought was that ultimately I may want to create an array of {string, int}
so my output would look something like this..
Anchorage 0
Cantwell 210
Anderson 284
I am able to set the first element of the array to 1, but can not get the 2nd and 3rd element of the new output array to correct
This is the code I am having a problem with:
public class TSPNearestNeighbor {
private int numberOfNodes;
private Stack<Integer> stack;
public TSPNearestNeighbor()
{
stack = new Stack<>();
}
public void tsp(int adjacencyMatrix[][])
{
numberOfNodes = adjacencyMatrix[1].length;
// System.out.print(numberOfNodes);
// System.out.print(Arrays.deepToString(adjacencyMatrix));
int[] visited = new int[numberOfNodes];
// System.out.print(Arrays.toString(visited));
visited[1] = 1;
// System.out.print(Arrays.toString(visited));
stack.push(1);
int element, dst = 0, i;
int min = Integer.MAX_VALUE;
boolean minFlag = false;
System.out.print(1 + "\n");
//System.arraycopy(arr_cities, 0, arr_final, 0, 1); // Copies Anchorage to Pos 1 always
//System.out.print(Arrays.deepToString(arr_final)+ "\n");
while (!stack.isEmpty())
{
element = stack.peek();
i = 1;
min = Integer.MAX_VALUE;
while (i <= numberOfNodes-1)
{
if (adjacencyMatrix[element][i] > 1 && visited[i] == 0)
{
if (min > adjacencyMatrix[element][i])
{
min = adjacencyMatrix[element][i];
dst = i;
minFlag = true;
}
}
i++;
}
if (minFlag)
{
visited[dst] = 1;
stack.push(dst);
System.out.print(dst + "\n");
minFlag = false;
continue;
}
stack.pop();
}
}

Given the existing structure you are using, you can output the cities in the path using:
public void printCities(Stack<Integer> path, int[][] distances, List<String> names) {
int cumulativeDistance = 0;
int previous = -1;
for (int city: path) {
if (previous != -1)
cumulativeDistance += distances[previous][city];
System.out.println(names.get(city) + " " + cumulativeDistance);
previous = city;
}
}

I'd like to answer your question slightly indirectly. You are making life hard for yourself by using arrays of objects. They make the code difficult to read and are hard to access. Things would become easier if you create a City class with appropriate methods to help you with the output.
For example:
class City {
private final String name;
private final Map<City,Integer> connections = new HashMap<>();
public static addConnection(City from, City to, int distance) {
from.connections.put(to, distance);
to.connections.put(from, distance);
}
public int getDistanceTo(City other) {
if (connections.containsKey(other))
return connections.get(other);
else
throw new IllegalArgumentException("Non connection error");
}
}
I've left out constructor, getters, setters for clarity.
Now outputting your path becomes quite a bit simpler:
public void outputPath(List<City> cities) {
int cumulativeDistance = 0;
City previous = null;
for (City current: cities) {
if (previous != null)
cumulativeDistance += previous.getDistanceTo(current);
System.out.println(current.getName + " " + cumulativeDistance);
previous = current;
}
}

Binary search in a sorted (memory-mapped ?) file in Java

I am struggling to port a Perl program to Java, and learning Java as I go. A central component of the original program is a Perl module that does string prefix lookups in a +500 GB sorted text file using binary search
(essentially, "seek" to a byte offset in the middle of the file, backtrack to nearest newline, compare line prefix with the search string, "seek" to half/double that byte offset, repeat until found...)
I have experimented with several database solutions but found that nothing beats this in sheer lookup speed with data sets of this size. Do you know of any existing Java library that implements such functionality? Failing that, could you point me to some idiomatic example code that does random access reads in text files?
Alternatively, I am not familiar with the new (?) Java I/O libraries but would it be an option to memory-map the 500 GB text file (I'm on a 64-bit machine with memory to spare) and do binary search on the memory-mapped byte array? I would be very interested to hear any experiences you have to share about this and similar problems.

I am a big fan of Java's MappedByteBuffers for situations like this. It is blazing fast. Below is a snippet I put together for you that maps a buffer to the file, seeks to the middle, and then searches backwards to a newline character. This should be enough to get you going?
I have similar code (seek, read, repeat until done) in my own application, benchmarked
java.io streams against MappedByteBuffer in a production environment and posted the results on my blog (Geekomatic posts tagged 'java.nio' ) with raw data, graphs and all.
Two second summary? My MappedByteBuffer-based implementation was about 275% faster. YMMV.
To work for files larger than ~2GB, which is a problem because of the cast and .position(int pos), I've crafted paging algorithm backed by an array of MappedByteBuffers. You'll need to be working on a 64-bit system for this to work with files larger than 2-4GB because MBB's use the OS's virtual memory system to work their magic.
public class StusMagicLargeFileReader {
private static final long PAGE_SIZE = Integer.MAX_VALUE;
private List<MappedByteBuffer> buffers = new ArrayList<MappedByteBuffer>();
private final byte raw[] = new byte[1];
public static void main(String[] args) throws IOException {
File file = new File("/Users/stu/test.txt");
FileChannel fc = (new FileInputStream(file)).getChannel();
StusMagicLargeFileReader buffer = new StusMagicLargeFileReader(fc);
long position = file.length() / 2;
String candidate = buffer.getString(position--);
while (position >=0 && !candidate.equals('\n'))
candidate = buffer.getString(position--);
//have newline position or start of file...do other stuff
}
StusMagicLargeFileReader(FileChannel channel) throws IOException {
long start = 0, length = 0;
for (long index = 0; start + length < channel.size(); index++) {
if ((channel.size() / PAGE_SIZE) == index)
length = (channel.size() - index * PAGE_SIZE) ;
else
length = PAGE_SIZE;
start = index * PAGE_SIZE;
buffers.add(index, channel.map(READ_ONLY, start, length));
}
}
public String getString(long bytePosition) {
int page = (int) (bytePosition / PAGE_SIZE);
int index = (int) (bytePosition % PAGE_SIZE);
raw[0] = buffers.get(page).get(index);
return new String(raw);
}
}

I have the same problem. I am trying to find all lines that start with some prefix in a sorted file.
Here is a method I cooked up which is largely a port of Python code found here: http://www.logarithmic.net/pfh/blog/01186620415
I have tested it but not thoroughly just yet. It does not use memory mapping, though.
public static List<String> binarySearch(String filename, String string) {
List<String> result = new ArrayList<String>();
try {
File file = new File(filename);
RandomAccessFile raf = new RandomAccessFile(file, "r");
long low = 0;
long high = file.length();
long p = -1;
while (low < high) {
long mid = (low + high) / 2;
p = mid;
while (p >= 0) {
raf.seek(p);
char c = (char) raf.readByte();
//System.out.println(p + "\t" + c);
if (c == '\n')
break;
p--;
}
if (p < 0)
raf.seek(0);
String line = raf.readLine();
//System.out.println("-- " + mid + " " + line);
if (line.compareTo(string) < 0)
low = mid + 1;
else
high = mid;
}
p = low;
while (p >= 0) {
raf.seek(p);
if (((char) raf.readByte()) == '\n')
break;
p--;
}
if (p < 0)
raf.seek(0);
while (true) {
String line = raf.readLine();
if (line == null || !line.startsWith(string))
break;
result.add(line);
}
raf.close();
} catch (IOException e) {
System.out.println("IOException:");
e.printStackTrace();
}
return result;
}

I am not aware of any library that has that functionality. However, a correct code for a external binary search in Java should be similar to this:
class ExternalBinarySearch {
final RandomAccessFile file;
final Comparator<String> test; // tests the element given as search parameter with the line. Insert a PrefixComparator here
public ExternalBinarySearch(File f, Comparator<String> test) throws FileNotFoundException {
this.file = new RandomAccessFile(f, "r");
this.test = test;
}
public String search(String element) throws IOException {
long l = file.length();
return search(element, -1, l-1);
}
/**
* Searches the given element in the range [low,high]. The low value of -1 is a special case to denote the beginning of a file.
* In contrast to every other line, a line at the beginning of a file doesn't need a \n directly before the line
*/
private String search(String element, long low, long high) throws IOException {
if(high - low < 1024) {
// search directly
long p = low;
while(p < high) {
String line = nextLine(p);
int r = test.compare(line,element);
if(r > 0) {
return null;
} else if (r < 0) {
p += line.length();
} else {
return line;
}
}
return null;
} else {
long m = low + ((high - low) / 2);
String line = nextLine(m);
int r = test.compare(line, element);
if(r > 0) {
return search(element, low, m);
} else if (r < 0) {
return search(element, m, high);
} else {
return line;
}
}
}
private String nextLine(long low) throws IOException {
if(low == -1) { // Beginning of file
file.seek(0);
} else {
file.seek(low);
}
int bufferLength = 65 * 1024;
byte[] buffer = new byte[bufferLength];
int r = file.read(buffer);
int lineBeginIndex = -1;
// search beginning of line
if(low == -1) { //beginning of file
lineBeginIndex = 0;
} else {
//normal mode
for(int i = 0; i < 1024; i++) {
if(buffer[i] == '\n') {
lineBeginIndex = i + 1;
break;
}
}
}
if(lineBeginIndex == -1) {
// no line begins within next 1024 bytes
return null;
}
int start = lineBeginIndex;
for(int i = start; i < r; i++) {
if(buffer[i] == '\n') {
// Found end of line
return new String(buffer, lineBeginIndex, i - lineBeginIndex + 1);
return line.toString();
}
}
throw new IllegalArgumentException("Line to long");
}
}
Please note: I made up this code ad-hoc: Corner cases are not tested nearly good enough, the code assumes that no single line is larger than 64K, etc.
I also think that building an index of the offsets where lines start might be a good idea. For a 500 GB file, that index should be stored in an index file. You should gain a not-so-small constant factor with that index because than there is no need to search for the next line in each step.
I know that was not the question, but building a prefix tree data structure like (Patrica) Tries (on disk/SSD) might be a good idea to do the prefix search.

This is a simple example of what you want to achieve. I would probably first index the file, keeping track of the file position for each string. I'm assuming the strings are separated by newlines (or carriage returns):
RandomAccessFile file = new RandomAccessFile("filename.txt", "r");
List<Long> indexList = new ArrayList();
long pos = 0;
while (file.readLine() != null)
{
Long linePos = new Long(pos);
indexList.add(linePos);
pos = file.getFilePointer();
}
int indexSize = indexList.size();
Long[] indexArray = new Long[indexSize];
indexList.toArray(indexArray);
The last step is to convert to an array for a slight speed improvement when doing lots of lookups. I would probably convert the Long[] to a long[] also, but I did not show that above. Finally the code to read the string from a given indexed position:
int i; // Initialize this appropriately for your algorithm.
file.seek(indexArray[i]);
String line = file.readLine();
// At this point, line contains the string #i.

If you are dealing with a 500GB file, then you might want to use a faster lookup method than binary search - namely a radix sort which is essentially a variant of hashing. The best method for doing this really depends on your data distributions and types of lookup, but if you are looking for string prefixes there should be a good way to do this.
I posted an example of a radix sort solution for integers, but you can use the same idea - basically to cut down the sort time by dividing the data into buckets, then using O(1) lookup to retrieve the bucket of data that is relevant.
Option Strict On
Option Explicit On
Module Module1
Private Const MAX_SIZE As Integer = 100000
Private m_input(MAX_SIZE) As Integer
Private m_table(MAX_SIZE) As List(Of Integer)
Private m_randomGen As New Random()
Private m_operations As Integer = 0
Private Sub generateData()
' fill with random numbers between 0 and MAX_SIZE - 1
For i = 0 To MAX_SIZE - 1
m_input(i) = m_randomGen.Next(0, MAX_SIZE - 1)
Next
End Sub
Private Sub sortData()
For i As Integer = 0 To MAX_SIZE - 1
Dim x = m_input(i)
If m_table(x) Is Nothing Then
m_table(x) = New List(Of Integer)
End If
m_table(x).Add(x)
' clearly this is simply going to be MAX_SIZE -1
m_operations = m_operations + 1
Next
End Sub
Private Sub printData(ByVal start As Integer, ByVal finish As Integer)
If start < 0 Or start > MAX_SIZE - 1 Then
Throw New Exception("printData - start out of range")
End If
If finish < 0 Or finish > MAX_SIZE - 1 Then
Throw New Exception("printData - finish out of range")
End If
For i As Integer = start To finish
If m_table(i) IsNot Nothing Then
For Each x In m_table(i)
Console.WriteLine(x)
Next
End If
Next
End Sub
' run the entire sort, but just print out the first 100 for verification purposes
Private Sub test()
m_operations = 0
generateData()
Console.WriteLine("Time started = " & Now.ToString())
sortData()
Console.WriteLine("Time finished = " & Now.ToString & " Number of operations = " & m_operations.ToString())
' print out a random 100 segment from the sorted array
Dim start As Integer = m_randomGen.Next(0, MAX_SIZE - 101)
printData(start, start + 100)
End Sub
Sub Main()
test()
Console.ReadLine()
End Sub
End Module

I post a gist https://gist.github.com/mikee805/c6c2e6a35032a3ab74f643a1d0f8249c
that is rather complete example based on what I found on stack overflow and some blogs hopefully someone else can use it
import static java.nio.file.Files.isWritable;
import static java.nio.file.StandardOpenOption.READ;
import static org.apache.commons.io.FileUtils.forceMkdir;
import static org.apache.commons.io.IOUtils.closeQuietly;
import static org.apache.commons.lang3.StringUtils.isBlank;
import static org.apache.commons.lang3.StringUtils.trimToNull;
import java.io.File;
import java.io.IOException;
import java.nio.Buffer;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.file.Path;
public class FileUtils {
private FileUtils() {
}
private static boolean found(final String candidate, final String prefix) {
return isBlank(candidate) || candidate.startsWith(prefix);
}
private static boolean before(final String candidate, final String prefix) {
return prefix.compareTo(candidate.substring(0, prefix.length())) < 0;
}
public static MappedByteBuffer getMappedByteBuffer(final Path path) {
FileChannel fileChannel = null;
try {
fileChannel = FileChannel.open(path, READ);
return fileChannel.map(FileChannel.MapMode.READ_ONLY, 0, fileChannel.size()).load();
}
catch (Exception e) {
throw new RuntimeException(e);
}
finally {
closeQuietly(fileChannel);
}
}
public static String binarySearch(final String prefix, final MappedByteBuffer buffer) {
if (buffer == null) {
return null;
}
try {
long low = 0;
long high = buffer.limit();
while (low < high) {
int mid = (int) ((low + high) / 2);
final String candidate = getLine(mid, buffer);
if (found(candidate, prefix)) {
return trimToNull(candidate);
}
else if (before(candidate, prefix)) {
high = mid;
}
else {
low = mid + 1;
}
}
}
catch (Exception e) {
throw new RuntimeException(e);
}
return null;
}
private static String getLine(int position, final MappedByteBuffer buffer) {
// search backwards to the find the proceeding new line
// then search forwards again until the next new line
// return the string in between
final StringBuilder stringBuilder = new StringBuilder();
// walk it back
char candidate = (char)buffer.get(position);
while (position > 0 && candidate != '\n') {
candidate = (char)buffer.get(--position);
}
// we either are at the beginning of the file or a new line
if (position == 0) {
// we are at the beginning at the first char
candidate = (char)buffer.get(position);
stringBuilder.append(candidate);
}
// there is/are char(s) after new line / first char
if (isInBuffer(buffer, position)) {
//first char after new line
candidate = (char)buffer.get(++position);
stringBuilder.append(candidate);
//walk it forward
while (isInBuffer(buffer, position) && candidate != ('\n')) {
candidate = (char)buffer.get(++position);
stringBuilder.append(candidate);
}
}
return stringBuilder.toString();
}
private static boolean isInBuffer(final Buffer buffer, int position) {
return position + 1 < buffer.limit();
}
public static File getOrCreateDirectory(final String dirName) {
final File directory = new File(dirName);
try {
forceMkdir(directory);
isWritable(directory.toPath());
}
catch (IOException e) {
throw new RuntimeException(e);
}
return directory;
}
}

I had similar problem, so I created (Scala) library from solutions provided in this thread:
https://github.com/avast/BigMap
It contains utility for sorting huge file and binary search in this sorted file...

If you truly want to try memory mapping the file, I found a tutorial on how to use memory mapping in Java nio.

IP subnet verification in JSP

I have the following JSP code, that protects my web page and displays it only to know IP's
String ip_h = request.getRemoteAddr();
String host_h = request.getRemoteHost();
String iplist[] = new String[1];
iplist[0] = "127.0.0.1";
iplist[1] = "10.217.106.248";
int count = iplist.length;
boolean flag = false;
int zz = 0;
//return;
System.out.println(host_h);
while ( (flag==false) && ( zz < count) )
{
if (ip_h.equals(iplist[zz]) || host_h.equals(iplist[zz]) )
{
flag = true;
}
zz++;
}
However, I would to rather check for subnet ranges, i.e. all users belonging to 10.217.0.0/16 are allowed.
How do I do this?

IP addresses (at least, IPv4 addresses) are really intended to be represented as 32-bit integers. If you convert the IP address to an integer first, checking subnet ranges becomes a relatively simple matter of checking (in your example) whether the first 16 bits match the first 16 bits of the range.

wouldn't you rather use the application server to lock down the ip range? In apache you can create an alias for a directory, put your code in the directory, then in the alias directive only allow certain ip or ranges:
Alias /mydir "/usr/local/mydir"
order deny,allow
deny from all
Allow from 10.217.106.248
Allow from 127.0.0.1
allow from 10.217.106 #this is a range
this way you don't have to code this sort of "magic number"
I am sure you can do this type of thing in other web servers

Feel free to use this IpRangeFilter class. See class comment for explanation.
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import org.apache.commons.collections15.Predicate;
import org.apache.commons.lang.builder.EqualsBuilder;
import org.apache.commons.lang.builder.HashCodeBuilder;
/**
* I am a filter used to determine if a given IP Address is covered by the IP range specified in
* the constructor. I accept IP ranges in the form of full single IP addresses, e.g. 10.1.0.23
* or network/netmask pairs in CIDR format e.g. 10.1.0.0/16
*/
public class IpRangeFilter implements Predicate<InetAddress> {
private final long network;
private final long netmask;
private final String ipRange;
private static final Pattern PATTERN = Pattern.compile("((?:\\d|\\.)+)(?:/(\\d{1,2}))?");
public IpRangeFilter(String ipRange) throws UnknownHostException {
Matcher matcher = PATTERN.matcher(ipRange);
if (matcher.matches()) {
String networkPart = matcher.group(1);
String cidrPart = matcher.group(2);
long netmask = 0;
int cidr = cidrPart == null ? 32 : Integer.parseInt(cidrPart);
for (int pos = 0; pos < 32; ++pos) {
if (pos >= 32-cidr) {
netmask |= (1L << pos);
}
}
this.network = netmask & toMask(InetAddress.getByName(networkPart));
this.netmask = netmask;
this.ipRange = ipRange;
} else {
throw new IllegalArgumentException("Not a valid IP range: " + ipRange);
}
}
public String getIpRange() {
return ipRange;
}
public boolean evaluate(InetAddress address) {
return isInRange(address);
}
public boolean isInRange(InetAddress address) {
return network == (toMask(address) & netmask);
}
/**
* Convert the bytes in the InetAddress into a bit mask stored as a long.
* We could use int's here, but java represents those in as signed numbers, which can be a pain
* when debugging.
* #see http://www.captain.at/howto-java-convert-binary-data.php
*/
static long toMask(InetAddress address) {
byte[] data = address.getAddress();
long accum = 0;
int idx = 3;
for ( int shiftBy = 0; shiftBy < 32; shiftBy += 8 ) {
accum |= ( (long)( data[idx] & 0xff ) ) << shiftBy;
idx--;
}
return accum;
}
}

Try the Subnet class from this bug report.

You can use following piece of code, of course it assumes the input data are correct so it needs some beautifying (just in case)
public class IPUtil
{
private static int[] split(String ip)
{
int[] result = new int[4];
StringTokenizer st = new StringTokenizer(ip, ".");
for (int i = 0; i < 4; i++)
{
result[i] = Integer.parseInt(st.nextToken());
}
return result;
}
public static boolean matches(String visitorIpString, String ipString, String maskString)
{
int[] vip = split(visitorIpString);
int[] ip = split(ipString);
int[] mask = split(maskString);
for (int i = 0; i < 4; i++)
{
if ((vip[i] & mask[i]) != ip[i])
{
return false;
}
}
return true;
}
public static void main(String[] args)
{
String ip = "192.168.12.0";
String mask = "255.255.255.0";
String visitorIP = "192.168.12.55";
System.out.println(matches(visitorIP, ip, mask));
}
}