I am trying to write a scala program to generate an output file omega0_Real.txt which contains pre-calculated values of the cosine function for inputs ranging from 0 to pi/2 radians. Each of these calculated values are 72 bit long and are stored in hex format. The code I have written so far is as follows:
import java.io._
import scala.math._
object omega0_Real {
def main (args: Array[String]) {
val arg = (0.0).to(2-pow(2, -10), pow(2, -10))
val cosArg = arg.map (i => cos(i))
val cosBit = cosArg.map (i => List.tabulate(72)(j = (BigDecimal((i*pow(2,j))).toBigInt % 2)))
val cosStr = cosBit.map (i => i mkString)
val cosBig = cosStr.map (i => BigInt(i, 2))
val cosBigStr = cosBig.map (i => i.toString(16))
val cosList = cosBigStr.toList
val file = "omega0_Real.txt"
val writer = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(file)))
for (x <- cosList) {
writer.write(x + "\n")
}
writer.close()
}
which gives the error: java.lang.NumberFormatException: Illegal embedded sign character followed by many others. Please help me debug this code.
PS: I ran this code line-by-line on sbt console but it did not give any error, although the values generated were erroneous.
Related
I have an issue where I read a bytestream from a big file ~ (100MB) and after some integers I get the value 0 (but only with sbt run ). When I hit the play button on IntelliJ I get the value I expected > 0.
My guess was that the environment is somehow different. But I could not spot the difference.
// DemoApp.scala
import java.nio.{ByteBuffer, ByteOrder}
object DemoApp extends App {
val inputStream = getClass.getResourceAsStream("/HandRanks.dat")
val handRanks = new Array[Byte](inputStream.available)
inputStream.read(handRanks)
inputStream.close()
def evalCard(value: Int) = {
val offset = value * 4
println("value: " + value)
println("offset: " + offset)
ByteBuffer.wrap(handRanks, offset, handRanks.length - offset).order(ByteOrder.LITTLE_ENDIAN).getInt
}
val cards: List[Int] = List(51, 45, 14, 2, 12, 28, 46)
def eval(cards: List[Int]): Unit = {
var p = 53
cards.foreach(card => {
println("p = " + evalCard(p))
p = evalCard(p + card)
})
println("result p: " + p);
}
eval(cards)
}
The HandRanks.dat can be found here: (I put it inside a directory called resources)
https://github.com/Robert-Nickel/scala-texas-holdem/blob/master/src/main/resources/HandRanks.dat
build.sbt is:
name := "LoadInts"
version := "0.1"
scalaVersion := "2.13.4"
On my windows machine I use sbt 1.4.6 with Oracle Java 11
You will see that the evalCard call will work 4 times but after the fifth time the return value is 0. It should be higher than 0, which it is when using IntelliJ's play button.
You are not reading a whole content. This
val handRanks = new Array[Byte](inputStream.available)
allocates only as much as InputStream buffer and then you read the amount in buffer with
inputStream.read(handRanks)
Depending of defaults you will process different amount but they will never be 100MB of data. For that you would have to read data into some structure in the loop (bad idea) or process it in chunks (with iterators, stream, etc).
import scala.util.Using
// Using will close the resource whether error happens or not
Using(getClass.getResourceAsStream("/HandRanks.dat")) { inputStream =>
def readChunk(): Option[Array[Byte]] = {
// can be done better, but that's not the point here
val buffer = new Array[Byte](inputStream.available)
val bytesRead = inputStream.read(buffer)
if (bytesRead >= 0) Some(buffer.take(bytesRead))
else None
}
#tailrec def process(): Unit = {
readChunk() match {
case Some(chunk) =>
// do something
process()
case None =>
// nothing to do - EOF reached
}
}
process()
}
I wrote a small app that pings a remote server by sending ICMP packets using sockets (Android's Os package). I tested it on the API 28 emulator and it worked, but it does not work on API 22 (Android 5.1 Lollipop). I tested it on the VD as well as on my smartphone (same old API) with the same negative result
android.system.ErrnoException: sendto failed: EINVAL (Invalid argument)
I debugged the app on different API versions and the only difference I noticed that ByteBuffer.wrap() produces ByteArrayBuffer instance on API 22 and HeapArrayBuffer instance on newer versions. The payload itself seems to be the same.
Here is the code sample (simplified). Also, there is a test application available: https://github.com/alexeysirenko/android-sockets-icmp-ping-test
You could try to launch it on the different emulators (API 22 and >22) and see the difference
private val timeoutMs = 5000
private val delayMs = 500L
private val ECHO_PORT = 80
private val POLLIN = (if (OsConstants.POLLIN == 0) 1 else OsConstants.POLLIN).toShort()
fun ping(host: String): Unit {
val inetAddress: InetAddress = InetAddress.getByName(host)
if (inetAddress is Inet6Address) throw Exception("IPv6 implementation omitted for simplicity")
val proto = OsConstants.IPPROTO_ICMP
val inet = OsConstants.AF_INET
val type = PacketBuilder.TYPE_ICMP_V4
val socketFileDescriptor = Os.socket(inet, OsConstants.SOCK_DGRAM, proto)
if (!socketFileDescriptor.valid()) throw Exception("Socket descriptor is invalid")
var sequenceNumber: Short = 0
for (i in 0..2) {
sequenceNumber++
val echoPacketBuilder =
PacketBuilder(type, "foobarbazquok".toByteArray())
.withSequenceNumber(sequenceNumber)
val buffer = echoPacketBuilder.build()
/**
* This is the command that throws an exception
*/
val bytesSent = Os.sendto(socketFileDescriptor, buffer,0, buffer.size, 0, inetAddress, ECHO_PORT)
// Response processing code omitted
}
}
class PacketBuilder(val type: Byte, val payload: ByteArray, val sequenceNumber: Short = 0, val identifier: Short = 0xDBB) {
private val MAX_PAYLOAD = 65507
private val CODE: Byte = 0
init {
if (payload.size > MAX_PAYLOAD) throw Exception("Payload limited to $MAX_PAYLOAD")
}
fun build(): ByteArray {
val buffer = ByteArray(8 + payload.size)
val byteBuffer = ByteBuffer.wrap(buffer)
byteBuffer.put(type)
byteBuffer.put(CODE)
val checkPos = byteBuffer.position()
byteBuffer.position(checkPos + 2)
byteBuffer.putShort(identifier)
byteBuffer.putShort(sequenceNumber)
byteBuffer.put(payload)
byteBuffer.putShort(checkPos, checksum(buffer))
byteBuffer.flip()
return buffer
}
fun withSequenceNumber(sequenceNumber: Short): PacketBuilder {
return PacketBuilder(type, payload, sequenceNumber, identifier)
}
/**
* RFC 1071 checksum
*/
private fun checksum(data: ByteArray): Short {
var sum = 0
// High bytes (even indices)
for (i in 0 until data.size step 2) {
sum += data[i].and(0xFF.toByte()).toInt() shl 8
sum = (sum and 0xFFFF) + (sum shr 16)
}
// Low bytes (odd indices)
for (i in 1 until data.size step 2) {
sum += data[i] and 0xFF.toByte()
sum = (sum and 0xFFFF) + (sum shr 16)
}
sum = (sum and 0xFFFF) + (sum shr 16)
return (sum xor 0xFFFF).toShort()
}
companion object {
val TYPE_ICMP_V4: Byte = 8
}
}
If my code is incorrect then I'd expect the same error on all platforms, but as I said it does work on all APIs newer than 22 and I do not know exactly what causes this issue
This is my code that converts the Retrofit HTTP ResponseBody to a raw String:
Method 1:
fun ByteArray.toHexString(): String {
var cnt = ""
var cnter = 0
return this.joinToString(cnt) {
if (cnter % 2 == 0)
cnt = " "
else
cnt = ""
cnter++
String.format("%02x", it)
}
}
fun convert() {
val result = response.byteStream().readBytes(response.contentLength().toInt())
val rawHtml = result.toHexString()
}
Method 1 result (snippet). It should have a whitespace after every 4th Byte:
255044462d312e340d0a25aaabacad0d0a312030206f626a0d0a3c3c0d0a2f4e616d65732032203020520d0a2f4f7574707574496e74656e7473205b3c3c0d0a2f446573744f757470757450726f66696c652033203020520d0a2f53202f4754535f50444641310d0a2f496e666f202863850eea75051264315790c769f97999de290d0a2f52656769737472794e616d652028290d0a2f4f7574707574436f6e646974696f6e2028290d0a2f54797065202f4f7574707574496e74656e740d0a2f4f7574707574436f6e646974696f6e4964656e746966696572202853a23adc3a21290d0a3e3e0d0a5d0d0a2f5669657765725072...
Method 2:
private fun getRawHTML(responseBody: ResponseBody): String {
val bodyString = responseBody.byteStream()
val reader = BufferedReader(InputStreamReader(bodyString, "iso-8859-1"), 16)
val sb = StringBuilder()
var line: String?
line = reader.readLine()
while (line != null) {
sb.append(line + "\n")
line = reader.readLine()
}
bodyString.close()
return sb.toString()
}
Method 2 result (snippet):
%PDF-1.4
1 0 obj
<<
/Title (þÿ��M��i���n��p��e��n��s��o��v��e��r��z��i��c�.��n��l)
/Creator (þÿ��w�m��p��d��f�� ��0��1��2��.��1��.��2)
/Producer (þÿ�t�� ��4����6)
/CreationDate (D:20181122184902+01'00')
>>
endobj
3 0 obj
<<
/Type /ExtGState
/SA true
/SM 0.02
/ca 1.0
/CA 1.0
/AIS false
/SMask /None>>
/Filter /FlateDecode
>>
stream
xí]MGr½Ï¯èó*å÷` )Ñ ðÁðÁàZ^,FË{ðß÷{YÕ]
When scrolling down in this PDF it shows that the encoding is /Identity-H:
/Name /FBUKTZ+Verdana
/Type /Font
/Subtype /Type0
/BaseFont /FBUKTZ+Verdana
/Encoding /Identity-H
/ToUnicode 28 0 R
/DescendantFonts [29 0 R]
>>
Which charset corresponds to this?
I want to convert this to a PDF file that can be opened by Adobe acrobat reader and shows the original PDF. When I open a correct PDF file with sublime editor, I see this:
2550 4446 2d31 2e37 0a25 e2e3 cfd3 0a31
2030 206f 626a 0a3c 3c2f 416c 7465 726e
6174 652f 4465 7669 6365 5247 422f 4e20
332f 4c65 6e67 7468 2032 3631 352f 4669
Maybe I could rephrase the question to how can I convert the small snippet to this format? I'm using Kotlin and Java.
Here's a Kotlin program that downloads a PDF file from a server and saves it in a way that allows it to be opened in a PDF viewer:
import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.ResponseBody
import java.io.FileOutputStream
fun savePDF(response: ResponseBody) {
val fileOutputStream = FileOutputStream("my.pdf")
val data = response.byteStream().readBytes()
fileOutputStream.write(data)
}
fun main(args: Array<String>) {
val request = Request.Builder()
.url("http://www.oracle.com/events/global/en/java-outreach/resources/java-a-beginners-guide-1720064.pdf")
.build()
val client = OkHttpClient()
val response = client.newCall(request).execute()
val responseBody = response.body()
if (responseBody != null) {
savePDF(responseBody)
}
}
I currently have a ipynb file (ipython notebook) that contains a linear regression code / model(im not entirely sure if it's a model) that I've created earlier.
How do i implement this model in an android application such that if I were to input a value of 'x' in a text box, it'll output in a textview the predicted value of 'y'. Function: Y = mx + b.
I've tried looking at different tutorials, but they were mostly not "step-by-step" guides, which made it really hard to understand, I'm a beginner at coding.
Here's my code for the model:
import tensorflow as tf
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
rng = np.random
from numpy import genfromtxt
from sklearn.datasets import load_boston
# Parameters
learning_rate = 0.0001
training_epochs = 1000
display_step = 50
n_samples = 222
X = tf.placeholder("float") # create symbolic variables
Y = tf.placeholder("float")
filename_queue = tf.train.string_input_producer(["C://Users//Shiina//battdata.csv"],shuffle=False)
reader = tf.TextLineReader() # skip_header_lines=1 if csv has headers
key, value = reader.read(filename_queue)
# Default values, in case of empty columns. Also specifies the type of the
# decoded result.
record_defaults = [[1.], [1.]]
height, soc= tf.decode_csv(
value, record_defaults=record_defaults)
features = tf.stack([height])
# Set model weights
W = tf.Variable(rng.randn(), name="weight")
b = tf.Variable(rng.randn(), name="bias")
# Construct a linear model
pred_soc = tf.add(tf.multiply(height, W), b) # XW + b <- y = mx + b where W is gradient, b is intercept
# Mean squared error
cost = tf.reduce_sum(tf.pow(pred_soc-soc, 2))/(2*n_samples)
# Gradient descent
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)
# Initializing the variables
init = tf.global_variables_initializer()
with tf.Session() as sess:
# Start populating the filename queue.
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(coord=coord)
sess.run(init)
# Fit all training data
for epoch in range(training_epochs):
_, cost_value = sess.run([optimizer,cost])
#Display logs per epoch step
if (epoch+1) % display_step == 0:
c = sess.run(cost)
print( "Epoch:", '%04d' % (epoch+1), "cost=", "{:.9f}".format(c), \
"W=", sess.run(W), "b=", sess.run(b))
print("Optimization Finished!")
training_cost = sess.run(cost)
print ("Training cost=", training_cost, "W=", sess.run(W), "b=", sess.run(b), '\n')
#Plot data after completing training
train_X = []
train_Y = []
for i in range(n_samples): #Your input data size to loop through once
X, Y = sess.run([height, pred_soc]) # Call pred, to get the prediction with the updated weights
train_X.append(X)
train_Y.append(Y)
#Graphic display
plt.plot(train_X, train_Y, 'ro', label='Original data')
plt.ylabel("SoC")
plt.xlabel("Height")
plt.axis([0, 1, 0, 100])
plt.plot(train_X, train_Y, linewidth=2.0)
plt.legend()
plt.show()
coord.request_stop()
coord.join(threads)
I am new to scala and java altogether and trying to run a sample producer code. All it does is, takes some raw products and referrers stored in csv files and uses rnd to generate some random log. Following is my code:
object LogProducer extends App {
//WebLog config
val wlc = Settings.WebLogGen
val Products = scala.io.Source.fromInputStream(getClass.getResourceAsStream("/products.csv")).getLines().toArray
val Referrers = scala.io.Source.fromInputStream(getClass.getResourceAsStream("/referrers.csv")).getLines().toArray
val Visitors = (0 to wlc.visitors).map("Visitors-" + _)
val Pages = (0 to wlc.pages).map("Pages-" + _)
val rnd = new Random()
val filePath = wlc.filePath
val fw = new FileWriter(filePath, true)
//adding randomness to time increments for demo
val incrementTimeEvery = rnd.nextInt(wlc.records - 1) + 1
var timestamp = System.currentTimeMillis()
var adjustedTimestamp = timestamp
for (iteration <- 1 to wlc.records) {
adjustedTimestamp = adjustedTimestamp + ((System.currentTimeMillis() - timestamp) * wlc.timeMultiplier)
timestamp = System.currentTimeMillis()
val action = iteration % (rnd.nextInt(200) + 1) match {
case 0 => "purchase"
case 1 => "add_to_cart"
case _ => "page_view"
}
val referrer = Referrers(rnd.nextInt(Referrers.length - 1))
val prevPage = referrer match {
case "Internal" => Pages(rnd.nextInt(Pages.length - 1))
case _ => ""
}
val visitor = Visitors(rnd.nextInt(Visitors.length - 1))
val page = Pages(rnd.nextInt(Pages.length - 1))
val product = Products(rnd.nextInt(Products.length - 1))
val line = s"$adjustedTimestamp\t$referrer\t$action\t$prevPage\t$visitor\t$page\t$product\n"
fw.write(line)
if (iteration % incrementTimeEvery == 0) {
//os.flush()
println(s"Sent $iteration messages!")
val sleeping = rnd.nextInt(incrementTimeEvery * 60)
println(s"Sleeping for $sleeping ms")
}
}
}
It is pretty straightforward where it is basically generating some variables and adding it to the line.
However I am getting a big exception error stack which i am not able to understand:
"C:\Program Files\Java\jdk1.8.0_92\bin\java...
Exception in thread "main" java.nio.charset.MalformedInputException: Input length = 1
at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.fill(BufferedReader.java:161)
at java.io.BufferedReader.readLine(BufferedReader.java:324)
at java.io.BufferedReader.readLine(BufferedReader.java:389)
at scala.io.BufferedSource$BufferedLineIterator.hasNext(BufferedSource.scala:70)
at scala.collection.Iterator.foreach(Iterator.scala:929)
at scala.collection.Iterator.foreach$(Iterator.scala:929)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1417)
at scala.collection.generic.Growable.$plus$plus$eq(Growable.scala:59)
at scala.collection.generic.Growable.$plus$plus$eq$(Growable.scala:50)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48)
at scala.collection.TraversableOnce.to(TraversableOnce.scala:310)
at scala.collection.TraversableOnce.to$(TraversableOnce.scala:308)
at scala.collection.AbstractIterator.to(Iterator.scala:1417)
at scala.collection.TraversableOnce.toBuffer(TraversableOnce.scala:302)
at scala.collection.TraversableOnce.toBuffer$(TraversableOnce.scala:302)
at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1417)
at scala.collection.TraversableOnce.toArray(TraversableOnce.scala:289)
at scala.collection.TraversableOnce.toArray$(TraversableOnce.scala:283)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1417)
at clickstream.LogProducer$.delayedEndpoint$clickstream$LogProducer$1(logProducer.scala:16)
at clickstream.LogProducer$delayedInit$body.apply(logProducer.scala:12)
at scala.Function0.apply$mcV$sp(Function0.scala:34)
at scala.Function0.apply$mcV$sp$(Function0.scala:34)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App.$anonfun$main$1$adapted(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:389)
at scala.App.main(App.scala:76)
at scala.App.main$(App.scala:74)
at clickstream.LogProducer$.main(logProducer.scala:12)
at clickstream.LogProducer.main(logProducer.scala)
Process finished with exit code 1
Can someone please help me identify what the exception mean? Thanks all
So it wasnt hard.. it was my amateurish knowledge. It was a simple IO exception where Intellij wasnt able to get the values from my csv file. When i imported it into resources root directory, it gave me a warning message of wrong encoding.
The error was at this point:
val Products = scala.io.Source.fromInputStream(getClass.getResourceAsStream("/products.csv")).getLines().toArray
thanks for efforts though
It was an encoding issue, for Scala a quick fix would be:
replace:
val Products=scala.io.Source.fromInputStream(getClass.getResourceAsStream("/products.csv")).getLines().toArray
val Referrers = scala.io.Source.fromInputStream(getClass.getResourceAsStream("/referrers.csv")).getLines().toArray
using this:
val Products=scala.io.Source.fromInputStream(getClass.getResourceAsStream("/products.csv"))("UTF-8").getLines().toArray
val Referrers = scala.io.Source.fromInputStream(getClass.getResourceAsStream("/referrers.csv"))("UTF-8").getLines().toArray
For java and more details please check out this link: http://biercoff.com/malformedinputexception-input-length-1-exception-solution-for-scala-and-java/