I'm having trouble finding the issue with how I'm generating the corresponding info hash for a torrent file. This is the code I have so far:
InputStream input = null;
try {
MessageDigest sha1 = MessageDigest.getInstance("SHA-1");
input = new FileInputStream(file);
StringBuilder builder = new StringBuilder();
while (!builder.toString().endsWith("4:info")) {
builder.append((char) input.read()); // It's ASCII anyway.
}
ByteArrayOutputStream output = new ByteArrayOutputStream();
for (int data; (data = input.read()) > -1; output.write(data));
sha1.update(output.toByteArray(), 0, output.size() - 1);
this.infoHash = sha1.digest();
System.out.println(new String(Hex.encodeHex(infoHash)));
} catch (NoSuchAlgorithmException | IOException e) {
e.printStackTrace();
} finally {
if (input != null) try { input.close(); } catch (IOException ignore) {}
}
Below is my expected and actual hash:
Expected: d4d44272ee5f5bf887a9c85ad09ae957bc55f89d
Actual: 4d753474429d817b80ff9e0c441ca660ec5d2450
The torrent I'm trying to generate an info hash for can be found here (Ubuntu 14.04 Desktop amd64).
Let me know if I can provide any more info, thanks!
Exceptions contain 4 useful bits of info: Type, Message, Trace, and Cause. You've tossing away 3 out of the 4 relevant bits of info. Also, code is part of a process, and when an error occurs, generally that process cannot be finished at all. And yet on exceptions your process continues. Stop doing this; you've written code that only hurts you. Remove the try, and the catch. Add a throws clause on your method signature. If you can't, the go-to default (and update your IDE if that generated this code to do this) is throw new RuntimeException("Unhandled", e);. This is shorter, does not destroy any of the 4 interesting bits of info, and ends a process.
Separately, the notion that the right way to handle an inputstream close method's IOException being: Just ignore it, is also false. It is highly unlikely to throw, but if it does, you should assume you didn't read every byte. As that would be one explanation for a mismatched hash, it's misguided.
Finally, use the proper language constructs: There is a try-with-resources statement that would work far better here.
You're calling update with output.size() - 1; unless you want to intentionally ignore the last byte, this is a mistake; you're lopping off the last byte read.
Reading bytes into a builder, and then per byte converting the builder to a string and then checking the last character is incredibly inefficient; for a file as small as 1MB that'll cause quite a grind.
Reading a single byte at a time from a raw FileInputStream is also that level of inefficient, because every read will cause file access (reading 1 byte is as expensive as reading a whole buffer full, so, it's about 50000 times slower than it needs to be).
Here's how to do this with somewhat newer API, and look how much nicer this code reads. It also acts better under erroneous conditions:
byte[] data = Files.readAllBytes(Paths.get(fileName));
var search = "4:info".getBytes(StandardCharsets.US_ASCII);
int searchIdx = -1;
for (int i = 0; searchIdx == -1 && i < data.length - search.length; i++) {
for (int j = 0; j < search.length; j++) {
if (data[i + j] != search[j]) break;
if (j == search.length - 1) searchIdx = i + j;
}
}
if (searchIdx == -1) throw new IOException("Input torrent file does not contain marker");
var sha1 = MessageDigest.getInstance("SHA-1");
sha1.update(data, searchIdx, data.length - searchIdx);
byte[] hash = sha1.digest();
StringBuilder hex = new StringBuilder();
for (byte h : hash) hex.append(String.format("%02x", h));
System.out.println(hex);
While rzwitserloot's answer covers some general java coding practices there also are correctness issues on the bittorrent level.
You are using string processing for a structured data format, this is pretty much the same mistake as attempting to parse html with regex. In this case you're assuming that the only place that the data can contain the string 4:info is the top-level dictionary key for the info dict and that the info dictionary is the last entry of the top level dictionary.
Instead you should use a proper bencoding decoder-encoder to extract the info dict and then re-encode it for hashing or a tokenizer to find the exact byte-range covering the info value. Note that you need a validating parser for the former while the latter can also handle some out-of-spec edge cases. Unless you want to implement them yourself you may want to find a library that handles this for you.
Additionally you're assuming that the data is ASCII. bencoding is in fact a binary format that just tends to use ascii by convention in some places. You should operate on byte arrays directly. Your input is already binary, the hasher expects binary so it is quite circuitous to go through strings.
I have this following Objective-C code and I want to translate it into Java code (for Android).
I know I have to use indexOf() but I don't know how to adapt range:NSMakeRange( old_position, ([currentWord length] - old_position) into Java
NSRange end = [currentWord rangeOfString:#"]" options:NSCaseInsensitiveSearch range:NSMakeRange( old_position, ([currentWord length] - old_position))];
if ( end.location != NSNotFound ) {
old_position = end.location + 1;
}
The rangeOfString:options:range: method is quite similar to the indexOf(str, fromIndex) method.
Try this:
Integer start = currentWord.indexOf("[", old_position);
Since the parameter is called fromPosition, you should pass the position from which you want to search, instead of the length of the portion that you want to search.
I have a problem in using the apache commons math library.
I just want to create functions like f(x) = 4x^2 + 2x and I want to compute the derivative of this function --> f'(x) = 8x + 2
I read the article about Differentiation (http://commons.apache.org/proper/commons-math/userguide/analysis.html, section 4.7).
There is an example which I don't understand:
int params = 1;
int order = 3;
double xRealValue = 2.5;
DerivativeStructure x = new DerivativeStructure(params, order, 0, xRealValue);
DerivativeStructure y = f(x); //COMPILE ERROR
System.out.println("y = " + y.getValue();
System.out.println("y' = " + y.getPartialDerivative(1);
System.out.println("y'' = " + y.getPartialDerivative(2);
System.out.println("y''' = " + y.getPartialDerivative(3);
In Line 5 a compile error occurs of course. The function f(x) is called and not defined. What I am getting wrong?
Has anyone any experience with the differentiation/derivation with the apache commons math library or does anyone know another library/framework which can help me?
Thanks
In the paragraph below that example, the author describes ways to create DerivativeStructures. It isn't magic. In the example you quoted, someone was supposed to write the function f. Well, that wasn't very clear.
There are several ways a user can create an implementation of the UnivariateDifferentiableFunction interface. The first method is to simply write it directly using the appropriate methods from DerivativeStructure to compute addition, subtraction, sine, cosine... This is often quite straigthforward and there is no need to remember the rules for differentiation: the user code only represent the function itself, the differentials will be computed automatically under the hood. The second method is to write a classical UnivariateFunction and to pass it to an existing implementation of the UnivariateFunctionDifferentiator interface to retrieve a differentiated version of the same function. The first method is more suited to small functions for which user already control all the underlying code. The second method is more suited to either large functions that would be cumbersome to write using the DerivativeStructure API, or functions for which user does not have control to the full underlying code (for example functions that call external libraries).
Use the first idea.
// Function of 1 variable, keep track of 3 derivatives with respect to that variable,
// use 2.5 as the current value. Basically, the identity function.
DerivativeStructure x = new DerivativeStructure(1, 3, 0, 2.5);
// Basically, x --> x^2.
DerivativeStructure x2 = x.pow(2);
//Linear combination: y = 4x^2 + 2x
DerivativeStructure y = new DerivativeStructure(4.0, x2, 2.0, x);
System.out.println("y = " + y.getValue());
System.out.println("y' = " + y.getPartialDerivative(1));
System.out.println("y'' = " + y.getPartialDerivative(2));
System.out.println("y''' = " + y.getPartialDerivative(3));
The following thread from the Apache mailing list seems to illustrate the two possible ways of how the derivative of a UnivariateDifferentiableFunction can be defined. I am adding a new answer as I'm unable to comment on the previous one (insufficient reputation).
The used sample specification of the function is f(x) = x^2.
(1) Using a DerivativeStructure:
public DerivativeStructure value(DerivativeStructure t) {
return t.multiply(t);
}
(2) By writing a classical UnivariateFunction:
public UnivariateRealFunction derivative() {
return new UnivariateRealFunction() {
public double value(double x) {
// example derivative
return 2.*x;
}
}
}
If I understand well, the advantage of the first case is that the derivative does not need to be obtained manually, as in the second case. In case the derivative is known, there should thus be no advantage of defining a DerivativeStructure, right? The application I have in mind is that of a Newton-Raphson solver, for which generally the function value and its derivative need to be known.
The full example is provided on the aforementioned web site (authors are Thomas Neidhart and Franz Simons). Any further comments are most welcome!
Here is a small Java Code Snippet.
ConfigData[] cd = new ConfigData[1];
cd[0] = new ConfigData();
byte[] tmpbyte ={1,(byte)0x01};
cd[0].settmpdata(tmpbyte);
"ConfigData" is my custom Type (int, Byte Array).
In my last thread i found the tip how to Build / work with a "ByteArray" in php.
But this seems to be a question of the structure of those objects / array.
So..
How can i depict that in PHP.
I hope that this can make some guidelines for you, in the terms of what you can do in PHP, and in a relation to your code snippet (the code is tested):
<?php
// define class
class ConfigData
{
var $intVal;
var $tmpData;
}
$cData = new ConfigData(); // new ConfigData instance
$array[0] = $cData; // put it in a single element array
$array[0]->intVal = 5; // assign an integer to intVal
$array[0]->tmpData = array(1, 1, 2); // assign an array of whatever to tmpData
foreach($array[0]->tmpData as $val) // iterate through assigned array
echo $val." "; // print array item (and append " " )
?>
Now, you might also want to check how byte manipulation in PHP is achieved. I suggest you to do a little Google search, and maybe check the official manual. You question was not specific enough, so I can't say more.
Is there any built-in method in Java to find the size of any datatype?
Is there any way to find size?
No. There is no such method in the standard Java SE class library.
The designers' view is that it is not needed in Java, since the language removes the need for an application1 to know about how much space needs to be reserved for a primitive value, an object or an array with a given number of elements.
You might think that a sizeof operator would be useful for people that need to know how much space their data structures take. However you can also get this information and more, simply and reliably using a Java memory profiler, so there is no need for a sizeof method.
Previous commenters made the point that sizeof(someType) would be more readable than 4. If you accept that readability argument, then the remedy is in your hands. Simply define a class like this ...
public class PrimitiveSizes {
public static int sizeof(byte b) { return 1; }
public static int sizeof(short s) { return 2; }
// etcetera
}
... and statically import it ...
import static PrimitiveSizes.*;
Or define some named constants; e.g.
public static final int SIZE_OF_INT = 4;
Or (Java 8 and later) use the Integer.BYTES constant, and so on.
Why haven't the Java designers implemented this in standard libraries? My guess is that:
they don't think there is a need for it,
they don't think there is sufficient demand for it, and
they don't think it is worth the effort.
There is also the issue that the next demand would be for a sizeof(Object o) method, which is fraught with technical difficulties.
The key word in the above is "they"!
1 - A programmer may need to know in order to design space efficient data structures. However, I can't imagine why that information would be needed in application code at runtime via a method call.
From the article in JavaWorld
A superficial answer is that Java does not provide anything like C's sizeof(). However,
let's consider why a Java programmer might occasionally want it.
A C programmer manages most datastructure memory allocations himself,
and sizeof() is indispensable for knowing memory block sizes to
allocate. Additionally, C memory allocators like malloc() do almost
nothing as far as object initialization is concerned: a programmer
must set all object fields that are pointers to further objects. But
when all is said and coded, C/C++ memory allocation is quite
efficient.
By comparison, Java object allocation and construction are tied
together (it is impossible to use an allocated but uninitialized
object instance). If a Java class defines fields that are references
to further objects, it is also common to set them at construction
time. Allocating a Java object therefore frequently allocates numerous
interconnected object instances: an object graph. Coupled with
automatic garbage collection, this is all too convenient and can make
you feel like you never have to worry about Java memory allocation
details.
Of course, this works only for simple Java applications. Compared with
C/C++, equivalent Java datastructures tend to occupy more physical
memory. In enterprise software development, getting close to the
maximum available virtual memory on today's 32-bit JVMs is a common
scalability constraint. Thus, a Java programmer could benefit from
sizeof() or something similar to keep an eye on whether his
datastructures are getting too large or contain memory bottlenecks.
Fortunately, Java reflection allows you to write such a tool quite
easily.
Before proceeding, I will dispense with some frequent but incorrect
answers to this article's question. Fallacy: Sizeof() is not needed
because Java basic types' sizes are fixed
Yes, a Java int is 32 bits in all JVMs and on all platforms, but this
is only a language specification requirement for the
programmer-perceivable width of this data type. Such an int is
essentially an abstract data type and can be backed up by, say, a
64-bit physical memory word on a 64-bit machine. The same goes for
nonprimitive types: the Java language specification says nothing about
how class fields should be aligned in physical memory or that an array
of booleans couldn't be implemented as a compact bitvector inside the
JVM. Fallacy: You can measure an object's size by serializing it into
a byte stream and looking at the resulting stream length
The reason this does not work is because the serialization layout is
only a remote reflection of the true in-memory layout. One easy way to
see it is by looking at how Strings get serialized: in memory every
char is at least 2 bytes, but in serialized form Strings are UTF-8
encoded and so any ASCII content takes half as much space
The Java Native Access library is typically used for calling native shared libraries from Java. Within this library there exist methods for determining the size of Java objects:
The getNativeSize(Class cls) method and its overloads will provide the size for most classes.
Alternatively, if your classes inherit from JNA's Structure class the calculateSize(boolean force) method will be available.
You can do bit manipulations like below to obtain the size of primitives:
public int sizeofInt() {
int i = 1, j = 0;
while (i != 0) {
i = (i<<1); j++;
}
return j;
}
public int sizeofChar() {
char i = 1, j = 0;
while (i != 0) {
i = (char) (i<<1); j++;
}
return j;
}
As mentioned here, there are possibilities to get the size of primitive types through their wrappers.
e.g. for a long this could be Long.SIZE / Byte.SIZE from java 1.5 (as mentioned by zeodtr already) or Long.BYTES as from java 8
There is a contemporary way to do that for primitives. Use BYTES of types.
System.out.println("byte " + Byte.BYTES);
System.out.println("char " + Character.BYTES);
System.out.println("int " + Integer.BYTES);
System.out.println("long " + Long.BYTES);
System.out.println("short " + Short.BYTES);
System.out.println("double " + Double.BYTES);
System.out.println("float " + Float.BYTES);
It results in,
byte 1
char 2
int 4
long 8
short 2
double 8
float 4
You can use Integer.SIZE / 8, Double.SIZE / 8, etc. for primitive types from Java 1.5.
The Instrumentation class has a getObjectSize() method however, you shouldn't need to use it at runtime. The easiest way to examine memory usage is to use a profiler which is designed to help you track memory usage.
EhCache provides a SizeOf class that will try to use the Instrumentation agent and will fall back to a different approach if the agent is not loaded or cannot be loaded (details here).
Also see the agent from Heinz Kabutz.
I decided to create an enum without following the standard Java conventions. Perhaps you like this.
public enum sizeof {
;
public static final int FLOAT = Float.SIZE / 8;
public static final int INTEGER = Integer.SIZE / 8;
public static final int DOUBLE = Double.SIZE / 8;
}
Try java.lang.Instrumentation.getObjectSize(Object). But please be aware that
It returns an implementation-specific approximation of the amount of storage consumed by the specified object. The result may include some or all of the object's overhead, and thus is useful for comparison within an implementation but not between implementations. The estimate may change during a single invocation of the JVM.
There's a class/jar available on SourceForge.net that uses Java instrumentation to calculate the size of any object. Here's a link to the description: java.sizeOf
Just some testing about it:
public class PrimitiveTypesV2 {
public static void main (String[] args) {
Class typesList[] = {
Boolean.class , Byte.class, Character.class, Short.class, Integer.class,
Long.class, Float.class, Double.class, Boolean.TYPE, Byte.TYPE, Character.TYPE,
Short.TYPE, Integer.TYPE, Long.TYPE, Float.TYPE, Double.TYPE
};
try {
for ( Class type : typesList ) {
if (type.isPrimitive()) {
System.out.println("Primitive type:\t" + type);
}
else {
boolean hasSize = false;
java.lang.reflect.Field fields[] = type.getFields();
for (int count=0; count<fields.length; count++) {
if (fields[count].getName().contains("SIZE")) hasSize = true;
}
if (hasSize) {
System.out.println("Bits size of type " + type + " :\t\t\t" + type.getField("SIZE").getInt(type) );
double value = type.getField("MIN_VALUE").getDouble(type);
long longVal = Math.round(value);
if ( (value - longVal) == 0) {
System.out.println("Min value for type " + type + " :\t\t" + longVal );
longVal = Math.round(type.getField("MAX_VALUE").getDouble(type));
System.out.println("Max value for type " + type + " :\t\t" + longVal );
}
else {
System.out.println("Min value for type " + type + " :\t\t" + value );
value = type.getField("MAX_VALUE").getDouble(type);
System.out.println("Max value for type " + type + " :\t\t" + value );
}
}
else {
System.out.println(type + "\t\t\t type without SIZE field.");
}
} // if not primitive
} // for typesList
} catch (Exception e) {e.printStackTrace();}
} // main
} // class PrimitiveTypes
Not sure for older versions, but since version 1.8 java sdk provides the .BYTES properties for boxed Objects of primitive types.
BYTES ( = SIZE / Byte.size )
import java.util.*;
import java.lang.*;
import java.io.*;
// The main method must be in a class named "Main".
class Main {
public static void main(String[] args) {
System.out.println("size of Integer: " + Integer.BYTES);
System.out.println("size of Character: " + Character.BYTES);
System.out.println("size of Short: " + Short.BYTES);
System.out.println("size of Long: " + Long.BYTES);
System.out.println("size of Double: " + Double.BYTES);
System.out.println("size of Float: " + Float.BYTES);
}
}
Here's a fiddle: https://www.mycompiler.io/view/0N19Y6cWL8F
I don't think it is in the java API. but most datatypes which have a number of elements in it, have a size() method. I think you can easily write a function to check for size yourself?
yes..in JAVA
System.out.println(Integer.SIZE/8); //gives you 4.
System.out.println(Integer.SIZE); //gives you 32.
//Similary for Byte,Long,Double....