Develop Reference

Parquet Binary storing value in encoded format

I'm creating a parquet file from Java using org.apache.parquet.*. No issues with other data types, but when a binary value is written and I cat the parquet file using parquet-tools, it is showing the value in encoded format. Because of that, the parquet is not processed in my system further.
Code block:
case BINARY:
recordConsumer.addBinary(stringToBinary(val));
break;
AND
private Binary stringToBinary(Object value) {
return Binary.fromString(value.toString());
}
Schema used is:
message m {
required INT64 id;
required binary username;
required boolean active;
}
When I cat:
parquet-tools cat <parquetFileName>
I see something like this:
id = 1
username = TmFtZTE=
active = true
id = 2
username = TmFtZTI=
active = false
I want to see the actual Username passed and not the encoded strings.

Try this in your schema
required binary username (UTF8)

Protobuf JsonFormater.printer convert long to String in Json

Currently I encounter a behavior of JsonFormater.printer printing the long(fixed64) value as String in JSON.
Is there any way/option to set the JsonFormater.printer not to do this conversion (Long(fixed64) -> String in Json)?
The Json is consumed by Java app, representing fixed64 as integer in JSON should not be a problem for Java.
Here is the code:
In data.proto
syntax = "proto2";
message data{
required fixed64 user_id = 1;
required int32 member_id = 2
}
Here is the java code, the file format is *.pb.gz
import com.google.protobuf.util.JsonFormat;
.......
//print data in JSON format
final InputStream input = new GZIPInputStream(new FileInputStream(pathToFile));
Message m;
m = defaultMsg.getParserForType().parseDelimitedFrom(input));
String jsonString = JsonFormat.printer().preservingProtoFieldNames().omittingInsignificantWhitespace().print(m);
Generated Java class: Data.java (Generated with protoc 2.6.1)
...
private long userId_;
...
private int memberId_;
...
expected result:
{"user_id":6546585813946021349,member_id":7521}
actual result:
{"user_id":"6546585813946021349",member_id":7521}
The user_id is String in json, but I want it as integer
Thanks
David

It seems this is by design, according to the source code. UINT64 and FIXED64 types are always printed out with surrounding double quotes, no questions asked:
https://github.com/protocolbuffers/protobuf/blob/f9d8138376765d229a32635c9209061e4e4aed8c/java/util/src/main/java/com/google/protobuf/util/JsonFormat.java#L1081-L1084
case INT64:
case SINT64:
case SFIXED64:
generator.print("\"" + ((Long) value).toString() + "\"");
In that same file, a few lines above, you can see that INT32 types are only double quoted if they're keys in a map (which your proto obviously doesn't have).
So, I'd ask for more information on the protobuf mailing list, or maybe report it as a bug/feature-request.

How can I read user data (memory) from EPC RFID tag through LLRP?

I encode two EPC tags through "NiceLabel Pro" with data:
First tag: EPC: 555555555, UserData: 9876543210123456789
Second tag: EPC: 444444444, UserData: 123456789123456789
Now I'm trying to get that data through LLRP (in my Java application):
My LLRPClient (one function):
public void PrepareInventoryRequest() {
AccessCommand accessCommand = new AccessCommand();
// A list to hold the op specs for this access command.
accessCommand.setAccessCommandOpSpecList(GenerateOpSpecList());
// Create a new tag spec.
C1G2TagSpec tagSpec = new C1G2TagSpec();
C1G2TargetTag targetTag = new C1G2TargetTag();
targetTag.setMatch(new Bit(1));
// We want to check memory bank 1 (the EPC memory bank).
TwoBitField memBank = new TwoBitField("2");
targetTag.setMB(memBank);
// The EPC data starts at offset 0x20.
// Start reading or writing from there.
targetTag.setPointer(new UnsignedShort(0));
// This is the mask we'll use to compare the EPC.
// We want to match all bits of the EPC, so all mask bits are set.
BitArray_HEX tagMask = new BitArray_HEX("00");
targetTag.setTagMask(tagMask);
// We only only to operate on tags with this EPC.
BitArray_HEX tagData = new BitArray_HEX("00");
targetTag.setTagData(tagData);
// Add a list of target tags to the tag spec.
List <C1G2TargetTag> targetTagList =
new ArrayList<>();
targetTagList.add(targetTag);
tagSpec.setC1G2TargetTagList(targetTagList);
// Add the tag spec to the access command.
accessCommand.setAirProtocolTagSpec(tagSpec);
accessSpec.setAccessCommand(accessCommand);
...
private List<AccessCommandOpSpec> GenerateOpSpecList() {
// A list to hold the op specs for this access command.
List <AccessCommandOpSpec> opSpecList =
new ArrayList<>();
// Set default opspec which for eventcycle of accessspec 3.
C1G2Read opSpec1 = new C1G2Read();
// Set the OpSpecID to a unique number.
opSpec1.setOpSpecID(new UnsignedShort(1));
opSpec1.setAccessPassword(new UnsignedInteger(0));
// We'll read from user memory (bank 3).
TwoBitField opMemBank = new TwoBitField("3");
opSpec1.setMB(opMemBank);
// We'll read from the base of this memory bank (0x00).
opSpec1.setWordPointer(new UnsignedShort(0));
// Read two words.
opSpec1.setWordCount(new UnsignedShort(0));
opSpecList.add(opSpec1);
return opSpecList;
}
My tag handler function:
private void updateTable(TagReportData tag) {
if (tag != null) {
EPCParameter epcParam = tag.getEPCParameter();
String EPCStr;
List<AccessCommandOpSpecResult> accessResultList = tag.getAccessCommandOpSpecResultList();
for (AccessCommandOpSpecResult accessResult : accessResultList) {
if (accessResult instanceof C1G2ReadOpSpecResult) {
C1G2ReadOpSpecResult op = (C1G2ReadOpSpecResult) accessResult;
if ((op.getResult().intValue() == C1G2ReadResultType.Success) &&
(op.getOpSpecID().intValue() < 1000)) {
UnsignedShortArray_HEX userMemoryHex = op.getReadData();
System.out.println("User Memory read from the tag is = " + userMemoryHex.toString());
}
}
}
...
For the first tag, "userMemoryHex.toString()" = "3938 3736"
For the second tag, "userMemoryHex.toString()" = "3132 3334"
Why? How do I get all user data?
This is my rfid tag.

The values that you get seem to be the first 4 characters of the number (interpreted as an ASCII string):
39383736 = "9876" (when interpreting those 4 bytes as ASCII characters)
31323334 = "1234" (when interpreting those 4 bytes as ASCII characters)
Since the specification of your tag says
Memory: EPC 128 bits, User 32 bits
your tag can only contain 32 bits (= 4 bytes) of user data. Hence, your tag simply can't contain the full value (i.e. 9876543210123456789 or 123456789123456789) that you tried to write as UserData (regardless of whether this was interpreted as a decimal number or a string).
Instead, your writer application seems to have taken the first 4 characters of those values, encoded them in ASCII, and wrote them to the tag.

JAVA GRPC libprotoc generated files and Grpc Ssl Context based subscription issue

I compiled GNMI proto file using proto compiler version 3.4.0 my proto file is shown below:
https://github.com/openconfig/gnmi/blob/master/proto/gnmi/gnmi.proto
syntax = "proto3";
import "google/protobuf/any.proto";
import "google/protobuf/descriptor.proto";
// Package gNMI defines a service specification for the gRPC Network Management
// Interface. This interface is defined to be a standard interface via which
// a network management system ("client") can subscribe to state values,
// retrieve snapshots of state information, and manipulate the state of a data
// tree supported by a device ("target").
//
// This document references the gNMI Specification which can be found at
// http://github.com/openconfig/reference/blob/master/rpc/gnmi
package gnmi;
// Define a protobuf FileOption that defines the gNMI service version.
extend google.protobuf.FileOptions {
// The gNMI service semantic version.
string gnmi_service = 1001;
}
// gNMI_service is the current version of the gNMI service, returned through
// the Capabilities RPC.
option (gnmi_service) = "0.5.0";
service gNMI {
// Capabilities allows the client to retrieve the set of capabilities that
// is supported by the target. This allows the target to validate the
// service version that is implemented and retrieve the set of models that
// the target supports. The models can then be specified in subsequent RPCs
// to restrict the set of data that is utilized.
// Reference: gNMI Specification Section 3.2
rpc Capabilities(CapabilityRequest) returns (CapabilityResponse);
// Retrieve a snapshot of data from the target. A Get RPC requests that the
// target snapshots a subset of the data tree as specified by the paths
// included in the message and serializes this to be returned to the
// client using the specified encoding.
// Reference: gNMI Specification Section 3.3
rpc Get(GetRequest) returns (GetResponse);
// Set allows the client to modify the state of data on the target. The
// paths to modified along with the new values that the client wishes
// to set the value to.
// Reference: gNMI Specification Section 3.4
rpc Set(SetRequest) returns (SetResponse);
// Subscribe allows a client to request the target to send it values
// of particular paths within the data tree. These values may be streamed
// at a particular cadence (STREAM), sent one off on a long-lived channel
// (POLL), or sent as a one-off retrieval (ONCE).
// Reference: gNMI Specification Section 3.5
rpc Subscribe(stream SubscribeRequest) returns (stream SubscribeResponse);
}
// Notification is a re-usable message that is used to encode data from the
// target to the client. A Notification carries two types of changes to the data
// tree:
// - Deleted values (delete) - a set of paths that have been removed from the
// data tree.
// - Updated values (update) - a set of path-value pairs indicating the path
// whose value has changed in the data tree.
// Reference: gNMI Specification Section 2.1
message Notification {
int64 timestamp = 1; // Timestamp in nanoseconds since Epoch.
Path prefix = 2; // Prefix used for paths in the message.
// An alias for the path specified in the prefix field.
// Reference: gNMI Specification Section 2.4.2
string alias = 3;
repeated Update update = 4; // Data elements that have changed values.
repeated Path delete = 5; // Data elements that have been deleted.
}
// Update is a re-usable message that is used to store a particular Path,
// Value pair.
// Reference: gNMI Specification Section 2.1
message Update {
Path path = 1; // The path (key) for the update.
Value value = 2 [deprecated=true]; // The value (value) for the update.
TypedValue val = 3; // The explicitly typed update value.
uint32 duplicates = 4; // Number of coalesced duplicates.
}
// TypedValue is used to encode a value being sent between the client and
// target (originated by either entity).
message TypedValue {
// One of the fields within the val oneof is populated with the value
// of the update. The type of the value being included in the Update
// determines which field should be populated. In the case that the
// encoding is a particular form of the base protobuf type, a specific
// field is used to store the value (e.g., json_val).
oneof value {
string string_val = 1; // String value.
int64 int_val = 2; // Integer value.
uint64 uint_val = 3; // Unsigned integer value.
bool bool_val = 4; // Bool value.
bytes bytes_val = 5; // Arbitrary byte sequence value.
float float_val = 6; // Floating point value.
Decimal64 decimal_val = 7; // Decimal64 encoded value.
ScalarArray leaflist_val = 8; // Mixed type scalar array value.
google.protobuf.Any any_val = 9; // protobuf.Any encoded bytes.
bytes json_val = 10; // JSON-encoded text.
bytes json_ietf_val = 11; // JSON-encoded text per RFC7951.
string ascii_val = 12; // Arbitrary ASCII text.
}
}
// Path encodes a data tree path as a series of repeated strings, with
// each element of the path representing a data tree node name and the
// associated attributes.
// Reference: gNMI Specification Section 2.2.2.
message Path {
// Elements of the path are no longer encoded as a string, but rather within
// the elem field as a PathElem message.
repeated string element = 1 [deprecated=true];
string origin = 2; // Label to disambiguate path.
repeated PathElem elem = 3; // Elements of the path.
string target = 4; // The name of the target
// (Sec. 2.2.2.1)
}
// PathElem encodes an element of a gNMI path, along ith any attributes (keys)
// that may be associated with it.
// Reference: gNMI Specification Section 2.2.2.
message PathElem {
string name = 1; // The name of the element in the path.
map<string, string> key = 2; // Map of key (attribute) name to value.
}
// Value encodes a data tree node's value - along with the way in which
// the value is encoded. This message is deprecated by gNMI 0.3.0.
// Reference: gNMI Specification Section 2.2.3.
message Value {
option deprecated = true;
bytes value = 1; // Value of the variable being transmitted.
Encoding type = 2; // Encoding used for the value field.
}
// Encoding defines the value encoding formats that are supported by the gNMI
// protocol. These encodings are used by both the client (when sending Set
// messages to modify the state of the target) and the target when serializing
// data to be returned to the client (in both Subscribe and Get RPCs).
// Reference: gNMI Specification Section 2.3
enum Encoding {
JSON = 0; // JSON encoded text.
BYTES = 1; // Arbitrarily encoded bytes.
PROTO = 2; // Encoded according to out-of-band agreed Protobuf.
ASCII = 3; // ASCII text of an out-of-band agreed format.
JSON_IETF = 4; // JSON encoded text as per RFC7951.
}
// Error message previously utilised to return errors to the client. Deprecated
// in favour of using the google.golang.org/genproto/googleapis/rpc/status
// message in the RPC response.
// Reference: gNMI Specification Section 2.5
message Error {
option deprecated = true;
uint32 code = 1; // Canonical gRPC error code.
string message = 2; // Human readable error.
google.protobuf.Any data = 3; // Optional additional information.
}
// Decimal64 is used to encode a fixed precision decimal number. The value
// is expressed as a set of digits with the precision specifying the
// number of digits following the decimal point in the digit set.
message Decimal64 {
int64 digits = 1; // Set of digits.
uint32 precision = 2; // Number of digits following the decimal point.
}
// ScalarArray is used to encode a mixed-type array of values.
message ScalarArray {
// The set of elements within the array. Each TypedValue message should
// specify only elements that have a field identifier of 1-7 (i.e., the
// values are scalar values).
repeated TypedValue element = 1;
}
// SubscribeRequest is the message sent by the client to the target when
// initiating a subscription to a set of paths within the data tree. The
// request field must be populated and the initial message must specify a
// SubscriptionList to initiate a subscription. The message is subsequently
// used to define aliases or trigger polled data to be sent by the target.
// Reference: gNMI Specification Section 3.5.1.1
message SubscribeRequest {
oneof request {
SubscriptionList subscribe = 1; // Specify the paths within a subscription.
Poll poll = 3; // Trigger a polled update.
AliasList aliases = 4; // Aliases to be created.
}
}
// Poll is sent within a SubscribeRequest to trigger the device to
// send telemetry updates for the paths that are associated with the
// subscription.
// Reference: gNMI Specification Section Section 3.5.1.4
message Poll {
}
// SubscribeResponse is the message used by the target within a Subscribe RPC.
// The target includes a Notification message which is used to transmit values
// of the path(s) that are associated with the subscription. The same message
// is to indicate that the target has sent all data values once (is
// synchronized).
// Reference: gNMI Specification Section 3.5.1.4
message SubscribeResponse {
oneof response {
Notification update = 1; // Changed or sampled value for a path.
// Indicate target has sent all values associated with the subscription
// at least once.
bool sync_response = 3;
// Deprecated in favour of google.golang.org/genproto/googleapis/rpc/status
Error error = 4 [deprecated=true];
}
}
// SubscriptionList is used within a Subscribe message to specify the list of
// paths that the client wishes to subscribe to. The message consists of a
// list of (possibly prefixed) paths, and options that relate to the
// subscription.
// Reference: gNMI Specification Section 3.5.1.2
message SubscriptionList {
Path prefix = 1; // Prefix used for paths.
repeated Subscription subscription = 2; // Set of subscriptions to create.
// Whether target defined aliases are allowed within the subscription.
bool use_aliases = 3;
QOSMarking qos = 4; // DSCP marking to be used.
// Mode of the subscription.
enum Mode {
STREAM = 0; // Values streamed by the target (Sec. 3.5.1.5.2).
ONCE = 1; // Values sent once-off by the target (Sec. 3.5.1.5.1).
POLL = 2; // Values sent in response to a poll request (Sec. 3.5.1.5.3).
}
Mode mode = 5;
// Whether elements of the schema that are marked as eligible for aggregation
// should be aggregated or not.
bool allow_aggregation = 6;
// The set of schemas that define the elements of the data tree that should
// be sent by the target.
repeated ModelData use_models = 7;
// The encoding that the target should use within the Notifications generated
// corresponding to the SubscriptionList.
Encoding encoding = 8;
// An optional field to specify that only updates to current state should be
// sent to a client. If set, the initial state is not sent to the client but
// rather only the sync message followed by any subsequent updates to the
// current state. For ONCE and POLL modes, this causes the server to send only
// the sync message (Sec. 3.5.2.3).
bool updates_only = 9;
}
// Subscription is a single request within a SubscriptionList. The path
// specified is interpreted (along with the prefix) as the elements of the data
// tree that the client is subscribing to. The mode determines how the target
// should trigger updates to be sent.
// Reference: gNMI Specification Section 3.5.1.3
message Subscription {
Path path = 1; // The data tree path.
SubscriptionMode mode = 2; // Subscription mode to be used.
uint64 sample_interval = 3; // ns between samples in SAMPLE mode.
// Indicates whether values that not changed should be sent in a SAMPLE
// subscription.
bool suppress_redundant = 4;
// Specifies the maximum allowable silent period in nanoseconds when
// suppress_redundant is in use. The target should send a value at least once
// in the period specified.
uint64 heartbeat_interval = 5;
}
// SubscriptionMode is the mode of the subscription, specifying how the
// target must return values in a subscription.
// Reference: gNMI Specification Section 3.5.1.3
enum SubscriptionMode {
TARGET_DEFINED = 0; // The target selects the relevant mode for each element.
ON_CHANGE = 1; // The target sends an update on element value change.
SAMPLE = 2; // The target samples values according to the interval.
}
// QOSMarking specifies the DSCP value to be set on transmitted telemetry
// updates from the target.
// Reference: gNMI Specification Section 3.5.1.2
message QOSMarking {
uint32 marking = 1;
}
// Alias specifies a data tree path, and an associated string which defines an
// alias which is to be used for this path in the context of the RPC. The alias
// is specified as a string which is prefixed with "#" to disambiguate it from
// data tree element paths.
// Reference: gNMI Specification Section 2.4.2
message Alias {
Path path = 1; // The path to be aliased.
string alias = 2; // The alias value, a string prefixed by "#".
}
// AliasList specifies a list of aliases. It is used in a SubscribeRequest for
// a client to create a set of aliases that the target is to utilize.
// Reference: gNMI Specification Section 3.5.1.6
message AliasList {
repeated Alias alias = 1; // The set of aliases to be created.
}
// SetRequest is sent from a client to the target to update values in the data
// tree. Paths are either deleted by the client, or modified by means of being
// updated, or replaced. Where a replace is used, unspecified values are
// considered to be replaced, whereas when update is used the changes are
// considered to be incremental. The set of changes that are specified within
// a single SetRequest are considered to be a transaction.
// Reference: gNMI Specification Section 3.4.1
message SetRequest {
Path prefix = 1; // Prefix used for paths in the message.
repeated Path delete = 2; // Paths to be deleted from the data tree.
repeated Update replace = 3; // Updates specifying elements to be replaced.
repeated Update update = 4; // Updates specifying elements to updated.
}
// SetResponse is the response to a SetRequest, sent from the target to the
// client. It reports the result of the modifications to the data tree that were
// specified by the client. Errors for this RPC should be reported using the
// https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto
// message in the RPC return. The gnmi.Error message can be used to add additional
// details where required.
// Reference: gNMI Specification Section 3.4.2
message SetResponse {
Path prefix = 1; // Prefix used for paths.
// A set of responses specifying the result of the operations specified in
// the SetRequest.
repeated UpdateResult response = 2;
Error message = 3 [deprecated=true]; // The overall status of the transaction.
int64 timestamp = 4; // Timestamp of transaction (ns since epoch).
}
// UpdateResult is used within the SetResponse message to communicate the
// result of an operation specified within a SetRequest message.
// Reference: gNMI Specification Section 3.4.2
message UpdateResult {
// The operation that was associated with the Path specified.
enum Operation {
INVALID = 0;
DELETE = 1; // The result relates to a delete of Path.
REPLACE = 2; // The result relates to a replace of Path.
UPDATE = 3; // The result relates to an update of Path.
}
// Deprecated timestamp for the UpdateResult, this field has been
// replaced by the timestamp within the SetResponse message, since
// all mutations effected by a set should be applied as a single
// transaction.
int64 timestamp = 1 [deprecated=true];
Path path = 2; // Path associated with the update.
Error message = 3 [deprecated=true]; // Status of the update operation.
Operation op = 4; // Update operation type.
}
// GetRequest is sent when a client initiates a Get RPC. It is used to specify
// the set of data elements for which the target should return a snapshot of
// data. The use_models field specifies the set of schema modules that are to
// be used by the target - where use_models is not specified then the target
// must use all schema models that it has.
// Reference: gNMI Specification Section 3.3.1
message GetRequest {
Path prefix = 1; // Prefix used for paths.
repeated Path path = 2; // Paths requested by the client.
// Type of elements within the data tree.
enum DataType {
ALL = 0; // All data elements.
CONFIG = 1; // Config (rw) only elements.
STATE = 2; // State (ro) only elements.
// Data elements marked in the schema as operational. This refers to data
// elements whose value relates to the state of processes or interactions
// running on the device.
OPERATIONAL = 3;
}
DataType type = 3; // The type of data being requested.
Encoding encoding = 5; // Encoding to be used.
repeated ModelData use_models = 6; // The schema models to be used.
}
// GetResponse is used by the target to respond to a GetRequest from a client.
// The set of Notifications corresponds to the data values that are requested
// by the client in the GetRequest.
// Reference: gNMI Specification Section 3.3.2
message GetResponse {
repeated Notification notification = 1; // Data values.
Error error = 2 [deprecated=true]; // Errors that occurred in the Get.
}
// CapabilityRequest is sent by the client in the Capabilities RPC to request
// that the target reports its capabilities.
// Reference: gNMI Specification Section 3.2.1
message CapabilityRequest {
}
// CapabilityResponse is used by the target to report its capabilities to the
// client within the Capabilities RPC.
// Reference: gNMI Specification Section 3.2.2
message CapabilityResponse {
repeated ModelData supported_models = 1; // Supported schema models.
repeated Encoding supported_encodings = 2; // Supported encodings.
string gNMI_version = 3; // Supported gNMI version.
}
// ModelData is used to describe a set of schema modules. It can be used in a
// CapabilityResponse where a target reports the set of modules that it
// supports, and within the SubscribeRequest and GetRequest messages to specify
// the set of models from which data tree elements should be reported.
// Reference: gNMI Specification Section 3.2.3
message ModelData {
string name = 1; // Name of the model.
string organization = 2; // Organization publishing the model.
string version = 3; // Semantic version of the model.
}
I generated GRPC stub using
java plugin protoc-gen-grpc-java-1.8.0-linux-x86_64.exe ( Version 1.8.0)
and did used the recommended pom file from here as shown below:
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>1.9.5</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.truth</groupId>
<artifactId>truth</artifactId>
<version>0.28</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-protobuf</artifactId>
<version>1.8.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-stub</artifactId>
<version>1.8.0</version>
<scope>test</scope>
</dependency>
Now i am trying to subscribe to the service as shown below:
import gnmi.gNMIGrpc; //Resolved
import gnmi.gNMIGrpc.gNMIStub;//Resolved
import io.grpc.ManagedChannel;//Resolved
import io.grpc.netty.GrpcSslContexts; //Not Resolved
import io.grpc.netty.NettyChannelBuilder; //Not Resolved
import io.netty.handler.ssl.SslContext;//Not Resolved
ManagedChannel originChannel;
SslContext sslcontext = null;
try {
sslcontext = GrpcSslContexts.forClient().trustManager(caFile).keyManager(clientCertFile, keyFile)
.build();
String host = "hostname";
int port = 50051;
ManagedChannel channel = NettyChannelBuilder.forAddress(host, port).
usePlaintext(true).sslContext(sslcontext).build();
gNMIStub gnmiStub2 = gNMIGrpc.newStub(channel);
} catch (SSLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Wondering what i am missing here. I am using Client and Certificate to connect to service but unable to resolve dependencies as shown in the above import statements, which are io.grpc.netty.GrpcSslContexts, io.grpc.netty.NettyChannelBuilder, io.netty.handler.ssl.SslContext
I can use either version of libprotoc 3.4.0 , 3.1.0 or 3.0.0. and pretty flexible with version of java plugin for GRPC Stubs i.e. protoc-gen-grpc-java-1.8.0-linux-x86_64.exe
Any idea about recommended combination of Jar files i need to define in my POM file, right version of libprotoc compiler and plugin that will work with JDK 7 in this case?

io.grpc.netty.* is provided by grpc-netty. You need to add it as a dependency.
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-netty</artifactId>
<version>1.8.0</version>
<scope>test</scope>
</dependency>

How can I set pageToken to get item lists from Google Cloud Storage via Java SDK?

I want to set pageToken to get items stored at Google Cloud Storage. I'm using Google API Client Library for Java v1.19.x.
I have no idea to generate pageToken from file path(or file name).
2 files stored in bucket.
my-bucket
/test.csv
/test2.csv
When I tried Google APIs Explorer with following parameters, I could get nextPageToken Cgh0ZXN0LmNzdg==.
And I found out that I can get test.csv string by decoding nextPageToken with base64.
bucket: my-bucket
pageToken:
prefix: test
maxResults: 1
{"kind": "storage#objects", "nextPageToken": "Cgh0ZXN0LmNzdg==", ...}
But How can I get Cgh0ZXN0LmNzdg== from test.csv?
Although I tried Base64 encoding, result didn't match.
import com.google.api.client.repackaged.org.apache.commons.codec.binary.Base64;
String lastFile = "test.csv"
String token = Base64.encodeBase64String(lastFile.getBytes());
String bucket = "my-bucket"
String prefix = "test"
Storage.Objects.List listObjects = client.objects().list(bucket);
listObjects.setPrefix(prefix);
listObjects.setPageToken(token);
long maxResults = 1;
listObjects.setMaxResults(maxResults);
do {
Objects objects = listObjects.execute();
List<StorageObject> items = objects.getItems();
token = objects.getNextPageToken();
listObjects.setPageToken(token);
} while (token != null);

I could get next token from file path string using following codes by myself.
How to get nextToken from path string
String nextToken = base64encode(0x0a + asciiCode + pathString)
asciiCode can be taken between 0x01(SOH) and 0x7f(DEL). It seems to depend on path length.
my-bucket/
a/a(3byte) 0x03
a/ab(4byte) 0x04
test.txt(8byte) 0x08
Notice
If path length is longer than 1024 byte, another rule seems to apply. But I couldn't found out rules.
See also Object Name Requirements
import com.google.common.io.BaseEncoding;
String lastFile = "test.csv"
String token = base64Encode(lastFile);
String bucket = "my-bucket"
String prefix = "test"
Storage.Objects.List listObjects = client.objects().list(bucket);
listObjects.setPrefix(prefix);
listObjects.setPageToken(token);
long maxResults = 1;
listObjects.setMaxResults(maxResults);
do {
Objects objects = listObjects.execute();
List<StorageObject> items = objects.getItems();
token = objects.getNextPageToken();
listObjects.setPageToken(token);
} while (token != null);
private String base64Encode(String path) {
byte[] encoding;
byte[] utf8 = path.getBytes(Charsets.UTF_8);
encoding = new byte[utf8.length + 2];
encoding[0] = 0x0a;
encoding[1] = new Byte(String.valueOf(path.length()));
String s = BaseEncoding.base64().encode(encoding);
return s;
}

I know this question is already answered and is applied to Java, I'd like to mention that this question applies to PHP as well.
With the help of the approved post from sakama above I figured out a PHP version of his solution.
The PHP equivalent for generating the token is as follow:
base64_encode(pack('c', 0x0a) . pack('c', $path_string_length) . pack('a*', $path_string));
The byte pattern seems indeed (as sakama already mentioned) to be:
<line feed><line data length><line data>