Develop Reference

JavaObject from Netlogo has no length using py4j?

I am running nl4py (a python module for NetLogo) in Jupyter notebook. I am trying to get import a list from netlogo into python, but the import is in a Java format. However, when I try to convert the JavaObject to a python format using py4j I get an error of: JavaObject has no len(). Is there a better way to convert JavaObject in python? Thanks.
python 3.8, ipython 7.10.0, nl4py 0.5.0, jdk 15.0.2, Netlogo 6.0, MacOS Catalina 10.15.7
#start of code for nl4py
import nl4py
nl4py.startServer("/Applications/NetLogo 6.0/")
n = nl4py.NetLogoApp()
n.openModel('/Users/tracykuper/Desktop/Netlogo models/Mucin project/1_21_20/PA_metabolite_model_1_21.nlogo')
n.command("setup")
#run abm model for n number of times
#change patch variable under a specific turtle
for i in range(1):
n.command("repeat 10 [go]")
#A = np.array([1,2,3,4],[3,2,-1,-6])) #turtle number, metabolite diff.
#run simulation of metabolic network to get biomass and metabolite values
#change patch variable under a specific turtle
names = ["1", "2", "3"] #turtle names
patch_values = ["-0.5", "50", "-0.5"] #metabolite values
for i in range(len(names)):
x = ('ask turtle {} [ask patch-here [set succinate succinate + {}]]'.format(names[i],patch_values[i]))
n.command(x)
#set new bacteria mass values
values = ["5", "30", "5"] #biomass values
y = ('ask turtle {} [set m m + {}]'.format(names[i],values[i]))
n.command(y)
n.command("ask turtle {} [set color red]".format(names[i]))
import py4j
mass = n.report("mass-list")
print(mass)
self = n.report("self-list")
type(mass)
s = py4j.protocol.get_return_value(mass, object)
[[0.69], [0.8], [0.73], [0.71], [0.5], [0.51], [0.54], [0.82], [0.72], [0.88]]
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-31-0b04d0127b47> in <module>
11 #map(mass + mass,mass)
12
---> 13 s = py4j.protocol.get_return_value(mass, object)
~/opt/anaconda3/envs/netlogo4/lib/python3.6/site-packages/py4j/protocol.py in get_return_value(answer, gateway_client, target_id, name)
319 (e.g., *hello* in `object1.hello()`). Optional.
320 """
--> 321 if is_error(answer)[0]:
322 if len(answer) > 1:
323 type = answer[1]
~/opt/anaconda3/envs/netlogo4/lib/python3.6/site-packages/py4j/protocol.py in is_error(answer)
372
373 def is_error(answer):
--> 374 if len(answer) == 0 or answer[0] != SUCCESS:
375 return (True, None)
376 else:
TypeError: object of type 'JavaObject' has no len()

Storing secret key on token USB gives different key (few different bytes) when doing getKey()

I'm trying to store a symmetric key (SecretKey which is Triple-DES key, ECB mode) on a cryptographic USB token. I use the following code to do it:
private void storeSecretKey( SecretKey secretKey, String alias ) throws StoreException {
try {
log.info("Format: " + secretKey.getFormat());
log.info("Alg: " + secretKey.getAlgorithm());
log.info("STORE KEY (bytes): " + Arrays.toString( secretKey.getEncoded()));
log.info("STORE KEY: " + ArrayUtils.convertToHexString( secretKey.getEncoded(), false, false));
myKeyStore.setKeyEntry(alias, secretKey, tokenPIN.toCharArray(), null);
myKeyStore.store(null);
Key key = myKeyStore.getKey(alias, tokenPIN.toCharArray());
log.info("Format: " + key.getFormat());
log.info("Alg: " + key.getAlgorithm());
log.info("FINAL KEY (bytes): " + Arrays.toString( key.getEncoded()));
log.info("FINAL KEY: " + ArrayUtils.convertToHexString( key.getEncoded(), false, false));
}
catch ( KeyStoreException e ) {
throw new StoreException( "Unable to store encryption key", e );
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
} catch (UnrecoverableEntryException e) {
e.printStackTrace();
}
}
And I get the following logging:
INFO: Format: RAW
INFO: Alg: DESede
INFO: STORE KEY (bytes): [87, -81, -89, -62, 5, -116, -46, 111, -85, -52, -28, -85, -26, -57, -26, -58, -66, -52, -16, 30, 89, -45, 61, -86]
INFO: STORE KEY: 57afa7c2058cd26fabcce4abe6c7e6c6beccf01e59d33daa
INFO: Format: RAW
INFO: Alg: DESede
INFO: FINAL KEY (bytes): [87, -82, -89, -62, 4, -116, -45, 110, -85, -51, -27, -85, -26, -57, -26, -57, -65, -51, -15, 31, 88, -45, 61, -85]
INFO: FINAL KEY: 57aea7c2048cd36eabcde5abe6c7e6c7bfcdf11f58d33dab
The key that is stored on the token (STORE KEY) and the key that I get using getKey() (FINAL KEY) are different. How can it be?
Please inform me if you need any other information it might be missing.
Thank you.

Your USB token simply adjusts the parity of the 3DES key. Each lower end bit of each byte is a parity bit for (triple) DES.
It needs to be set if the addition of the 7 highest bits is an even number, and unset if the number is odd. In the end each byte should have odd parity if you add all the bits together. So the first byte with value 0x57 or 0b0101011_1 and has 4 bits in the highest positions so the final bit has to be 1 - but it already is set, so no adjustment is necessary. The second byte, 0xAF or 0b1010111_1 has 5 bits in the highest position, so the last bit needs to be 0. This is not the case, so it is adjusted to 0b1010111_0 or 0xAE.
If you want to have the same value then you can construct your (triple) DES keys using a SecretKeyFactory instead of using a random number generator directly. The SecretKeyFactory will also adjust the parity for you - no need to program it yourself. I'd recommend this as other implementations could reject bytes where the number of bits is even (although generally the bits are adjusted or ignored). As indicated by James, the lowest bits are not used by (triple) DES during encryption / decryption.
This will create correctly coded triple DES keys (168 bit effective, 192 bits encoded). These are also called DES ABC keys as all three DES keys are different (with a very high probability).
public static SecretKey generate192Bit3DESKey() {
KeyGenerator keyGen;
try {
keyGen = KeyGenerator.getInstance("DESede");
} catch (NoSuchAlgorithmException e) {
throw new IllegalStateException("DESede functionality is required for Java, but it is missing");
}
// NOTE: this is the effective key size excluding parity bits
// use 112 for two key (ABA) triple DES keys (not recommended)
keyGen.init(168);
// this does adjust parity
SecretKey desABCKey = keyGen.generateKey();
return desABCKey;
}
If your data needs adjusting afterwards, you can use this (with only a for loop, no additional branching):
public static byte[] adjustDESParity(final byte[] keyData) {
for (int i = 0; i < keyData.length; i++) {
// count the bits, and XOR with 1 if even or 0 if already odd
keyData[i] ^= (Integer.bitCount(keyData[i]) % 2) ^ 1;
}
return keyData;
}

StringUtils.containsIgnoreCase() is returning wrong result

i have two values:
String a = "00tz"; // (Eclipse internal debug value: [, 0, 0, t, z]) and
String b = "tz"; // (Eclipse internal debug value: [, t, z])
I am reading this values from an ArrayList like
for (String a : stringLists) {
...
}
I get "false" when i compare this two values with StringUtils.containsIgnoreCase(a,b). But it should return true because "tz" is existing in "00tz".
Im using apache.commons.lang3.StringUtils. To change the values a & b didn't worked. The length of "a" is 5 and "b" is 3. It also returns false when i use a.contains(b).
These are the results when i output the value with
System.out.println(Arrays.toString(a.getBytes(StandardCharsets.UTF_8)));
a:[-17, -69, -65, 48, 48, 118, 119]
b:[-17, -69, -65, 118, 119]
Im reading this values from a .txt file which contains several values like this. I read it in this way:
File fileA = new File("test/a.txt");
File fileB = new File("test/b.txt");
lista = (ArrayList<String>) FileUtils.readLines(fileA, "utf-8");
listb = (ArrayList<String>) FileUtils.readLines(fileB, "utf-8");
Do you have an idea what the problem is?
Thank you!

Creating 24bit BMP fails (only with a particular resolution)

I'm creating a 24bit bmp which in general works fine (I've been using this functionality for some time). Now I've tried to write a bmp with 970 x 970 pixels and I'm ending up in corrupt files (I've exported bigger images before, I'm having problems with this particular resolution).
This is how I build the header:
private static byte[] createHeader(int width, int height) {
int size = width * height * 3 + 54;
byte[] ret = new byte[54];
set(ret, 0, (byte) 'B', (byte) 'M');
set(ret, 2, intToDWord(size));
set(ret, 6, intToDWord(0));
set(ret, 10, intToDWord(54));
set(ret, 14, intToDWord(40));
set(ret, 18, intToDWord(width));
set(ret, 22, intToDWord(height));
set(ret, 26, intToWord(1));
set(ret, 28, intToWord(24));
set(ret, 30, intToDWord(0));
set(ret, 34, intToDWord(width * height * 3));
set(ret, 38, intToDWord(0));
set(ret, 42, intToDWord(0));
set(ret, 46, intToDWord(0));
set(ret, 50, intToDWord(0));
return ret;
}
Here is the resulting image (this test image should be completely red):
test_corrupt.bmp (2.6mb)
I've analyzed the header, checked the size, I can't find the reason why this image is not a valid BMP.
Does anyone have a clue? I'm not making any progress.

It may be because of BMP files expects row lengths to be a multiple of 4 bytes. This changes the size you specified in header offset 34 and therefore the size in offset 2. Please refer to the following for details:
http://en.wikipedia.org/wiki/BMP_file_format
Related Part:
For file storage purposes, only the size of each row must be a multiple of 4 bytes while the file offset can be arbitrary
You can compare file by creating a 970x970 Red BMP file using MS Paint.

GeoTIFF import in worldwind ::: cannot read raster : unable to decipher image organization

I'm trying to import some (Matlab-generated) GeoTIFF files into WorldWind but seem to have no luck whatsoever. Any useful hints would greatly be appreciated. The GeoTIFF files do display fine in ArcGIS (allowing me to create a .tfw file when I export), but WorldWind gives me the following message:
SEVERE: Cannot read raster: C:\Users\Matthias\Desktop\geotiff\fldextent_02-
Jan-1977(1)renderedno0.tif : gov.nasa.worldwind.formats.tiff.GeotiffImageReader.read(): unable
to decipher image organization
Jul 09, 2013 6:54:33 PM gov.nasa.worldwind.data.CachedDataRaster drawOnTo
SEVERE: C:\Users\Matthias\Desktop\geotiff\fldextent_02-Jan-1977(1)renderedno0.tif : Cannot read
raster: C:\Users\Matthias\Desktop\geotiff\fldextent_02-Jan-1977(1)renderedno0.tif :
gov.nasa.worldwind.formats.tiff.GeotiffImageReader.read(): unable to decipher image organization
gov.nasa.worldwind.exception.WWRuntimeException: Cannot read raster: C:\Users\Matthias\Desktop
\geotiff\fldextent_02-Jan-1977(1)renderedno0.tif :
gov.nasa.worldwind.formats.tiff.GeotiffImageReader.read(): unable to decipher image organization
at gov.nasa.worldwind.data.CachedDataRaster.getDataRasters(CachedDataRaster.java:255)
at gov.nasa.worldwind.data.CachedDataRaster.drawOnTo(CachedDataRaster.java:290)
at gov.nasa.worldwind.data.TiledRasterProducer.drawDataSources(TiledRasterProducer.java:576)
[...]
I have also looked at the attributes of the GeoTIFF file in FWTools which gives me:
C:\Users\Matthias\Desktop\geotiff>gdalinfo fldextent_02-Jan-1977(1)renderedno0.tif
Driver: GTiff/GeoTIFF
Files: fldextent_02-Jan-1977(1)renderedno0.tif
fldextent_02-Jan-1977(1)renderedno0.tfw
Size is 7200, 7200
Coordinate System is:
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433],
AUTHORITY["EPSG","4326"]]
Origin = (99.000000000000000,7.000000000000000)
Pixel Size = (0.000833333333333,-0.000833333333333)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 99.0000000, 7.0000000) ( 99d 0'0.00"E, 7d 0'0.00"N)
Lower Left ( 99.0000000, 1.0000000) ( 99d 0'0.00"E, 1d 0'0.00"N)
Upper Right ( 105.0000000, 7.0000000) (105d 0'0.00"E, 7d 0'0.00"N)
Lower Right ( 105.0000000, 1.0000000) (105d 0'0.00"E, 1d 0'0.00"N)
Center ( 102.0000000, 4.0000000) (102d 0'0.00"E, 4d 0'0.00"N)
Band 1 Block=128x128 Type=Byte, ColorInterp=Gray
NoData Value=0
The .tfw file reads:
0.0008333333
0.0000000000
0.0000000000
-0.0008333333
99.0004166667
6.9995833333

I have found the issue finally:
The important thing is to create a CLEAN GeoTIFF file in Matlab (RGB and alpha layer for transparency). Here some Matlab guidance, the resulting GeoTIFF can directly be imported into WorldWind:
%%% read intensity values Z (2D matrix) - with values of 0 and above
%%% (we want 0 to be completely transparent in the final geotiff) -
%%% together with spatialref.GeoRasterReference ss
[Z, ss] = geotiffread('./flddph_1976-01-01.tif');
info_3 = geotiffinfo('./flddph_1976-01-01.tif');
%%% generate indexed image with 0 to 255 (255 equals max. intensity)
indexedimage = gray2ind(Z);
indexedimage = double(indexedimage);
%%% normalize so that everything between 0 and 1
normalizedimg = (indexedimage) / 255;
%%% scaling data and applying colormap
imgscaled = uint8(256*normalizedimg); % scale data
cmp = makeColorMap([1 1 0],[1 0.75 0],[1 0 0],256);
% 256 element colormap yellow - orange - red
% (download appropriate function MAKECOLORMAP)
imgrgb = ind2rgb(imgscaled,cmp);
%%% check plot
% subplot(2,1,1)
% imagesc(indexedimage)
% title('indexed image')
% subplot(2,1,2)
% image(img)
% title('rgb image')
%%% generating alpha layer for transparency
%%% (255 for non-transparent, 0 for transparent)
alpha3 = Z;
alpha3(alpha3>0)=255;
alpha3 = uint8(alpha3);
out2 = cat(3,imgrgb,alpha3);
geotiffwrite('test_rgbhope_flddph.tif',out2,ss);