Develop Reference

java.nio.ByteBuffer wrap method partly working with sbt run

I have an issue where I read a bytestream from a big file ~ (100MB) and after some integers I get the value 0 (but only with sbt run ). When I hit the play button on IntelliJ I get the value I expected > 0.
My guess was that the environment is somehow different. But I could not spot the difference.
// DemoApp.scala
import java.nio.{ByteBuffer, ByteOrder}
object DemoApp extends App {
val inputStream = getClass.getResourceAsStream("/HandRanks.dat")
val handRanks = new Array[Byte](inputStream.available)
inputStream.read(handRanks)
inputStream.close()
def evalCard(value: Int) = {
val offset = value * 4
println("value: " + value)
println("offset: " + offset)
ByteBuffer.wrap(handRanks, offset, handRanks.length - offset).order(ByteOrder.LITTLE_ENDIAN).getInt
}
val cards: List[Int] = List(51, 45, 14, 2, 12, 28, 46)
def eval(cards: List[Int]): Unit = {
var p = 53
cards.foreach(card => {
println("p = " + evalCard(p))
p = evalCard(p + card)
})
println("result p: " + p);
}
eval(cards)
}
The HandRanks.dat can be found here: (I put it inside a directory called resources)
https://github.com/Robert-Nickel/scala-texas-holdem/blob/master/src/main/resources/HandRanks.dat
build.sbt is:
name := "LoadInts"
version := "0.1"
scalaVersion := "2.13.4"
On my windows machine I use sbt 1.4.6 with Oracle Java 11
You will see that the evalCard call will work 4 times but after the fifth time the return value is 0. It should be higher than 0, which it is when using IntelliJ's play button.

You are not reading a whole content. This
val handRanks = new Array[Byte](inputStream.available)
allocates only as much as InputStream buffer and then you read the amount in buffer with
inputStream.read(handRanks)
Depending of defaults you will process different amount but they will never be 100MB of data. For that you would have to read data into some structure in the loop (bad idea) or process it in chunks (with iterators, stream, etc).
import scala.util.Using
// Using will close the resource whether error happens or not
Using(getClass.getResourceAsStream("/HandRanks.dat")) { inputStream =>
def readChunk(): Option[Array[Byte]] = {
// can be done better, but that's not the point here
val buffer = new Array[Byte](inputStream.available)
val bytesRead = inputStream.read(buffer)
if (bytesRead >= 0) Some(buffer.take(bytesRead))
else None
}
#tailrec def process(): Unit = {
readChunk() match {
case Some(chunk) =>
// do something
process()
case None =>
// nothing to do - EOF reached
}
}
process()
}

Failed to send an ICMP packet through socket on Android API 22 (Lollipop)

I wrote a small app that pings a remote server by sending ICMP packets using sockets (Android's Os package). I tested it on the API 28 emulator and it worked, but it does not work on API 22 (Android 5.1 Lollipop). I tested it on the VD as well as on my smartphone (same old API) with the same negative result
android.system.ErrnoException: sendto failed: EINVAL (Invalid argument)
I debugged the app on different API versions and the only difference I noticed that ByteBuffer.wrap() produces ByteArrayBuffer instance on API 22 and HeapArrayBuffer instance on newer versions. The payload itself seems to be the same.
Here is the code sample (simplified). Also, there is a test application available: https://github.com/alexeysirenko/android-sockets-icmp-ping-test
You could try to launch it on the different emulators (API 22 and >22) and see the difference
private val timeoutMs = 5000
private val delayMs = 500L
private val ECHO_PORT = 80
private val POLLIN = (if (OsConstants.POLLIN == 0) 1 else OsConstants.POLLIN).toShort()
fun ping(host: String): Unit {
val inetAddress: InetAddress = InetAddress.getByName(host)
if (inetAddress is Inet6Address) throw Exception("IPv6 implementation omitted for simplicity")
val proto = OsConstants.IPPROTO_ICMP
val inet = OsConstants.AF_INET
val type = PacketBuilder.TYPE_ICMP_V4
val socketFileDescriptor = Os.socket(inet, OsConstants.SOCK_DGRAM, proto)
if (!socketFileDescriptor.valid()) throw Exception("Socket descriptor is invalid")
var sequenceNumber: Short = 0
for (i in 0..2) {
sequenceNumber++
val echoPacketBuilder =
PacketBuilder(type, "foobarbazquok".toByteArray())
.withSequenceNumber(sequenceNumber)
val buffer = echoPacketBuilder.build()
/**
* This is the command that throws an exception
*/
val bytesSent = Os.sendto(socketFileDescriptor, buffer,0, buffer.size, 0, inetAddress, ECHO_PORT)
// Response processing code omitted
}
}
class PacketBuilder(val type: Byte, val payload: ByteArray, val sequenceNumber: Short = 0, val identifier: Short = 0xDBB) {
private val MAX_PAYLOAD = 65507
private val CODE: Byte = 0
init {
if (payload.size > MAX_PAYLOAD) throw Exception("Payload limited to $MAX_PAYLOAD")
}
fun build(): ByteArray {
val buffer = ByteArray(8 + payload.size)
val byteBuffer = ByteBuffer.wrap(buffer)
byteBuffer.put(type)
byteBuffer.put(CODE)
val checkPos = byteBuffer.position()
byteBuffer.position(checkPos + 2)
byteBuffer.putShort(identifier)
byteBuffer.putShort(sequenceNumber)
byteBuffer.put(payload)
byteBuffer.putShort(checkPos, checksum(buffer))
byteBuffer.flip()
return buffer
}
fun withSequenceNumber(sequenceNumber: Short): PacketBuilder {
return PacketBuilder(type, payload, sequenceNumber, identifier)
}
/**
* RFC 1071 checksum
*/
private fun checksum(data: ByteArray): Short {
var sum = 0
// High bytes (even indices)
for (i in 0 until data.size step 2) {
sum += data[i].and(0xFF.toByte()).toInt() shl 8
sum = (sum and 0xFFFF) + (sum shr 16)
}
// Low bytes (odd indices)
for (i in 1 until data.size step 2) {
sum += data[i] and 0xFF.toByte()
sum = (sum and 0xFFFF) + (sum shr 16)
}
sum = (sum and 0xFFFF) + (sum shr 16)
return (sum xor 0xFFFF).toShort()
}
companion object {
val TYPE_ICMP_V4: Byte = 8
}
}
If my code is incorrect then I'd expect the same error on all platforms, but as I said it does work on all APIs newer than 22 and I do not know exactly what causes this issue

Scala: Parse yaml file in scala (types casting )

I have a yaml file that I want to read its contents in scala , so I parse it using io.circe.yaml to json
var js = yaml.parser.parse(ymlText)
var json=js.valueOr(null)
var jsonstring=json.toString
val json2 = parse(jsonstring)
the yamltext is like this:
ALL:
Category1:
Subcategory11 : 1.5
Subcategory12 : 0
Subcategory13 : 0
Subcategory14 : 0.5
Category2:
Subcategory21 : 1.5
Subcategory22 : 0.3
Subcategory23 : 0
Subcategory24 : 0
what I want is to filter the subcategories that has Zero values, I've used this code:
val elements = (json2 \\"ALL" ).children.map(x=>(x.values))
var subCategories=elements.map{case(a,b)=>(b)}
var cats=elements.map{case(a,b)=>(b.asInstanceOf[Map[String,Double]])}
cats.map(x=>x.filter{case(a,b)=>b>0.0})
But the last line gives me this error:
scala.math.BigInt cannot be cast to java.lang.Double

I'm not sure why you do toString + parse and which parse is used but you probably don't need it. Also you didn't describe your expected result so here are a few guesses of what you might need:
import java.io._
import io.circe._
import io.circe.yaml._
import io.circe.parser._
def test(): Unit = {
// test data instead of a file
val ymlText =
"""
|ALL:
| Category1:
| Subcategory11 : 1.5
| Subcategory12 : 0
| Subcategory13 : 0
| Subcategory14 : 0.5
| Category2:
| Subcategory21 : 1.5
| Subcategory22 : 0.3
| Subcategory23 : 0
| Subcategory24 : 0
""".stripMargin
var js = yaml.parser.parse(new StringReader(ymlText))
var json: Json = js.right.get
val categories = (json \\ "ALL").flatMap(j => j.asObject.get.values.toList)
val subs = categories.flatMap(j => j.asObject.get.toList)
val elements: List[(String, Double)] = subs.map { case (k, v) => (k, v.asNumber.get.toDouble) }
.filter {
case (k, v) => v > 0.0
}
println(s"elements: $elements")
val allCategories = (json \\ "ALL").flatMap(j => j.asObject.get.toList).toMap
val filteredTree: Map[String, Map[String, Double]] = allCategories
.mapValues(catJson => catJson.asObject.get.toList.map { case (subName, subJson) => (subName, subJson.asNumber.get.toDouble) }
.filter { case (subName, subValue) => subValue > 0.0 }
.toMap)
println(s"filteredTree : $filteredTree")
}
And the output for that is:
elements: List((Subcategory11,1.5), (Subcategory14,0.5), (Subcategory21,1.5), (Subcategory22,0.3))
filteredTree : Map(Category1 -> Map(Subcategory11 -> 1.5, Subcategory14 -> 0.5), Category2 -> Map(Subcategory21 -> 1.5, Subcategory22 -> 0.3))
Hope one of those version is what you needed.

Log producer object in Scala

I am new to scala and java altogether and trying to run a sample producer code. All it does is, takes some raw products and referrers stored in csv files and uses rnd to generate some random log. Following is my code:
object LogProducer extends App {
//WebLog config
val wlc = Settings.WebLogGen
val Products = scala.io.Source.fromInputStream(getClass.getResourceAsStream("/products.csv")).getLines().toArray
val Referrers = scala.io.Source.fromInputStream(getClass.getResourceAsStream("/referrers.csv")).getLines().toArray
val Visitors = (0 to wlc.visitors).map("Visitors-" + _)
val Pages = (0 to wlc.pages).map("Pages-" + _)
val rnd = new Random()
val filePath = wlc.filePath
val fw = new FileWriter(filePath, true)
//adding randomness to time increments for demo
val incrementTimeEvery = rnd.nextInt(wlc.records - 1) + 1
var timestamp = System.currentTimeMillis()
var adjustedTimestamp = timestamp
for (iteration <- 1 to wlc.records) {
adjustedTimestamp = adjustedTimestamp + ((System.currentTimeMillis() - timestamp) * wlc.timeMultiplier)
timestamp = System.currentTimeMillis()
val action = iteration % (rnd.nextInt(200) + 1) match {
case 0 => "purchase"
case 1 => "add_to_cart"
case _ => "page_view"
}
val referrer = Referrers(rnd.nextInt(Referrers.length - 1))
val prevPage = referrer match {
case "Internal" => Pages(rnd.nextInt(Pages.length - 1))
case _ => ""
}
val visitor = Visitors(rnd.nextInt(Visitors.length - 1))
val page = Pages(rnd.nextInt(Pages.length - 1))
val product = Products(rnd.nextInt(Products.length - 1))
val line = s"$adjustedTimestamp\t$referrer\t$action\t$prevPage\t$visitor\t$page\t$product\n"
fw.write(line)
if (iteration % incrementTimeEvery == 0) {
//os.flush()
println(s"Sent $iteration messages!")
val sleeping = rnd.nextInt(incrementTimeEvery * 60)
println(s"Sleeping for $sleeping ms")
}
}
}
It is pretty straightforward where it is basically generating some variables and adding it to the line.
However I am getting a big exception error stack which i am not able to understand:
"C:\Program Files\Java\jdk1.8.0_92\bin\java...
Exception in thread "main" java.nio.charset.MalformedInputException: Input length = 1
at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.fill(BufferedReader.java:161)
at java.io.BufferedReader.readLine(BufferedReader.java:324)
at java.io.BufferedReader.readLine(BufferedReader.java:389)
at scala.io.BufferedSource$BufferedLineIterator.hasNext(BufferedSource.scala:70)
at scala.collection.Iterator.foreach(Iterator.scala:929)
at scala.collection.Iterator.foreach$(Iterator.scala:929)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1417)
at scala.collection.generic.Growable.$plus$plus$eq(Growable.scala:59)
at scala.collection.generic.Growable.$plus$plus$eq$(Growable.scala:50)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48)
at scala.collection.TraversableOnce.to(TraversableOnce.scala:310)
at scala.collection.TraversableOnce.to$(TraversableOnce.scala:308)
at scala.collection.AbstractIterator.to(Iterator.scala:1417)
at scala.collection.TraversableOnce.toBuffer(TraversableOnce.scala:302)
at scala.collection.TraversableOnce.toBuffer$(TraversableOnce.scala:302)
at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1417)
at scala.collection.TraversableOnce.toArray(TraversableOnce.scala:289)
at scala.collection.TraversableOnce.toArray$(TraversableOnce.scala:283)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1417)
at clickstream.LogProducer$.delayedEndpoint$clickstream$LogProducer$1(logProducer.scala:16)
at clickstream.LogProducer$delayedInit$body.apply(logProducer.scala:12)
at scala.Function0.apply$mcV$sp(Function0.scala:34)
at scala.Function0.apply$mcV$sp$(Function0.scala:34)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App.$anonfun$main$1$adapted(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:389)
at scala.App.main(App.scala:76)
at scala.App.main$(App.scala:74)
at clickstream.LogProducer$.main(logProducer.scala:12)
at clickstream.LogProducer.main(logProducer.scala)
Process finished with exit code 1
Can someone please help me identify what the exception mean? Thanks all

So it wasnt hard.. it was my amateurish knowledge. It was a simple IO exception where Intellij wasnt able to get the values from my csv file. When i imported it into resources root directory, it gave me a warning message of wrong encoding.
The error was at this point:
val Products = scala.io.Source.fromInputStream(getClass.getResourceAsStream("/products.csv")).getLines().toArray
thanks for efforts though

It was an encoding issue, for Scala a quick fix would be:
replace:
val Products=scala.io.Source.fromInputStream(getClass.getResourceAsStream("/products.csv")).getLines().toArray
val Referrers = scala.io.Source.fromInputStream(getClass.getResourceAsStream("/referrers.csv")).getLines().toArray
using this:
val Products=scala.io.Source.fromInputStream(getClass.getResourceAsStream("/products.csv"))("UTF-8").getLines().toArray
val Referrers = scala.io.Source.fromInputStream(getClass.getResourceAsStream("/referrers.csv"))("UTF-8").getLines().toArray
For java and more details please check out this link: http://biercoff.com/malformedinputexception-input-length-1-exception-solution-for-scala-and-java/

Number format exception in scala

I am trying to write a scala program to generate an output file omega0_Real.txt which contains pre-calculated values of the cosine function for inputs ranging from 0 to pi/2 radians. Each of these calculated values are 72 bit long and are stored in hex format. The code I have written so far is as follows:
import java.io._
import scala.math._
object omega0_Real {
def main (args: Array[String]) {
val arg = (0.0).to(2-pow(2, -10), pow(2, -10))
val cosArg = arg.map (i => cos(i))
val cosBit = cosArg.map (i => List.tabulate(72)(j = (BigDecimal((i*pow(2,j))).toBigInt % 2)))
val cosStr = cosBit.map (i => i mkString)
val cosBig = cosStr.map (i => BigInt(i, 2))
val cosBigStr = cosBig.map (i => i.toString(16))
val cosList = cosBigStr.toList
val file = "omega0_Real.txt"
val writer = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(file)))
for (x <- cosList) {
writer.write(x + "\n")
}
writer.close()
}
which gives the error: java.lang.NumberFormatException: Illegal embedded sign character followed by many others. Please help me debug this code.
PS: I ran this code line-by-line on sbt console but it did not give any error, although the values generated were erroneous.