Convert Spark Java to Spark scala - java

I am trying to convert my Java code to scala in Spark, but found it very complicated. Is it possible to convert the following Java code to scala? Thanks!
JavaPairRDD<String,Tuple2<String,String>> newDataPair = newRecords.mapToPair(new PairFunction<String, String, Tuple2<String, String>>() {
private static final long serialVersionUID = 1L;
#Override
public Tuple2<String, Tuple2<String, String>> call(String t) throws Exception {
MyPerson p = (new Gson()).fromJson(t, MyPerson.class);
String nameAgeKey = p.getName() + "_" + p.getAge() ;
Tuple2<String, String> value = new Tuple2<String, String>(p.getNationality(), t);
Tuple2<String, Tuple2<String, String>> kvp =
new Tuple2<String, Tuple2<String, String>>(nameAgeKey.toLowerCase(), value);
return kvp;
}
});
I tried the following, but I am sure I have missed many things. And actually it is not clear to me how to do the override function in scala ... Please suggest or share some examples. Thank you!
val newDataPair = newRecords.mapToPair(new PairFunction<String, String, Tuple2<String, String>>() {
#Override
public val call(String t) throws Exception {
val p = (new Gson()).fromJson(t, MyPerson.class);
val nameAgeKey = p.getName() + "_" + p.getAge() ;
val value = new Tuple2<String, String>(p.getNationality(), t);
val kvp =
new Tuple2<String, Tuple2<String, String>>(nameAgeKey.toLowerCase(), value);
return kvp;
}
});

Literal translations from Spark-Java to Spark-Scala typically don't work because Spark-Java introduces many artifacts to cope with the limited type system in Java. Examples in this case: mapToPair in Java is just map in Scala. Tuple2 has a more terse syntax (a,b)
Applying that (and some more) to the snippet:
val newDataPair = newRecords.map{t =>
val p = (new Gson()).fromJson(t, classOf[MyPerson])
val nameAgeKey = p.getName + "_" + p.getAge
val value = (p.getNationality(), t)
(nameAgeKey.toLowerCase(), value)
}
It could be made a bit more concise but I wanted to keep the same structure as the Java counterpart to facilitate the understanding of it.

Related

Java guava Splitter failing when there are duplicate keys

How we can handle duplicate keys while using java guava Splitter function. Here is the sample code which is encountering the following issue. Is there a better way to handle this issue.
String fieldSplit = " ";
String valueSplit = "=";
String message = "ip=1.2.9.0 error=NA ip=1.2.9.0";
Map<String, String> parserMap = Splitter.on(fieldSplit).omitEmptyStrings().withKeyValueSeparator(valueSplit).split(message);
Exception in thread "kv-plugin-ac801a38-66f1-4ffe-86ca-f9eb6c823842-StreamThread-1" org.apache.kafka.streams.errors.StreamsException: Exception caught in process. taskId=0_0, processor=KSTREAM-SOURCE-0000000000, topic=kv-input, partition=0, offset=22, stacktrace=java.lang.IllegalArgumentException: Duplicate key [ip] found.
Im getting the above error. Can somebody suggest a better way to handle this. Since im new to java.
Depends on what you want to do with the duplicate keys.
Map<String, String> is a key value storage that can have only unique keys and only one value.
If you want to store all those values you would need something like Map<String, List<String> or Guava Multimap.
In this case you cannot do this with the Splitter as it cannot handle duplicate keys. You would need to write the logic by yourself.
String fieldSplit = " ";
String valueSplit = "=";
String message = "ip=1.2.9.0 error=NA ip=1.2.9.0";
Map<String, List<String>> parserMap = new HashMap<>();
for (String part : message.split(" ")) {
String[] subparts = part.split("=", 2);
if (!parserMap.contains(subparts[0])) {
parserMap.put(subparts[0], new ArrayList<>());
}
parserMap.get(subparts[0]).add(subparts[1]);
}
If you want to omit those duplicate entries you can still use the Map<String, String> with something like this.
Map<String, String> parserMap = new HashMap<>();
for (String part : message.split(" ")) {
String[] subparts = part.split("=", 2);
if (!parserMap.contains(subparts[0])) {
parserMap.put(subparts[0], subparts[1]);
}
}
Throwing on a duplicate key is a documented behavior of MapSplitter#split, so depending on what you want, you have to write your own "key-value" spliter consisting of two splitters. Please look at examples below, you can collect results to map with desired behavior (overwrite or discard) or even try out collecting to ListMultimap, but it makes result's values being stored in lists, even if there's only one value.
public class SO66139006 {
private static final Splitter PAIRS_SPLITTER = Splitter.on(' '); // .trimResults().omitEmptyStrings() if necessary
private static final Splitter KEY_VALUE_SPLITTER = Splitter.on('=').limit(2);
#Test
public void shouldOverwriteValuesOnDuplicateKey() {
//given
String message = "ip=42.42.42.0 error=NA ip=1.2.9.0";
//when
Map<String, String> result = parseOverwritingValues(PAIRS_SPLITTER, KEY_VALUE_SPLITTER, message);
//then
assertThat(result) // {ip=1.2.9.0, error=NA}
.containsExactly(entry("ip", "1.2.9.0"), entry("error", "NA"));
}
private Map<String, String> parseOverwritingValues(Splitter pairsSplitter, Splitter keyValueSplitter, String message) {
return Streams.stream(pairsSplitter.split(message))
.map(keyValueSplitter::splitToList)
.collect(toImmutableMap(
list -> list.get(0),
list -> list.get(1),
(oldValue, newValue) -> newValue
));
}
#Test
public void shouldDiscardValuesOnDuplicateKey() {
//given
String message = "ip=42.42.42.0 error=NA ip=1.2.9.0";
//when
Map<String, String> result = parseDiscardingValues(PAIRS_SPLITTER, KEY_VALUE_SPLITTER, message);
//then
assertThat(result) // {ip=42.42.42.0, error=NA}
.containsExactly(entry("ip", "42.42.42.0"), entry("error", "NA"));
}
private Map<String, String> parseDiscardingValues(Splitter pairsSplitter, Splitter keyValueSplitter, String message) {
return Streams.stream(pairsSplitter.split(message))
.map(keyValueSplitter::splitToList)
.collect(toImmutableMap(
list -> list.get(0),
list -> list.get(1),
(oldValue, newValue) -> oldValue
));
}
#Test
public void shouldAppendValuesOnDuplicateKey() {
//given
String message = "ip=42.42.42.0 error=NA ip=1.2.9.0";
//when
ListMultimap<String, String> result = parseMultipleValues(PAIRS_SPLITTER, KEY_VALUE_SPLITTER, message);
//then
assertThat(result.asMap()) // {ip=[42.42.42.0, 1.2.9.0], error=[NA]}
.containsExactly(entry("ip", ImmutableList.of("42.42.42.0", "1.2.9.0")), entry("error", ImmutableList.of("NA")));
}
private ListMultimap<String, String> parseMultipleValues(Splitter pairsSplitter, Splitter keyValueSplitter, String message) {
return Streams.stream(pairsSplitter.split(message))
.map(keyValueSplitter::splitToList)
.collect(toImmutableListMultimap(
list -> list.get(0),
list -> list.get(1)
));
}
#Test
public void shouldThrowByDefault() {
//given
String fieldSplit = " ";
String valueSplit = "=";
String message = "ip=1.2.9.0 error=NA ip=1.2.9.0";
//when
final Throwable throwable = catchThrowable(() -> Splitter.on(fieldSplit).omitEmptyStrings().withKeyValueSeparator(valueSplit).split(message));
//then
assertThat(throwable)
.isInstanceOf(IllegalArgumentException.class)
.hasMessage("Duplicate key [ip] found.");
}
}

Convert nested for loop into java8 stream

I am trying to convert the below code into Java 8 Stream for nested for loop.
I have tried to take stream for outer loop but not sure how to write condition and assign a variable there.
final Map<String, String> events = new HashMap<>();
for (final Event s : result.getEvents()) {
String eventDetail = "";
for (final Data d : s.getData()) {
if (StringUtils.isNotEmpty(d.getValue()) && StringUtils.isNotEmpty(eventDetail)) {
eventDetail = eventDetail + "-" + d.getValue();
} else {
eventDetail = eventDetail + d.getValue();
}
}
events.put(s.getReferenceID(), eventDetail);
}
Result should be map value.
It looks like your goal is to concatenate the value members of the Data instances of each
Event into a "-" separated String, and map this String to the Event's reference ID.
This can be done with Collectors.joining():
Map<String, String>
events = result.getEvents()
.stream()
.map(s -> new SimpleEntry<>(s.getReferenceID(),s.getData().stream().map(Data::getValue).collect(Collectors.joining("-"))))
.collect(Collectors.toMap(Map.Entry::getKey,Map.Entry::getValue));
or, if you wish to eliminate empty values:
Map<String, String>
events = result.getEvents()
.stream()
.map(s -> new SimpleEntry<>(s.getReferenceID(),s.getData().stream().map(Data::getValue).filter(StringUtils::isNotEmpty).collect(Collectors.joining("-"))))
.collect(Collectors.toMap(Map.Entry::getKey,Map.Entry::getValue));
Is this what you want?
Function<Event, String> mapper = event -> event.getData().stream()
.map(Data::getValue)
.filter(StringUtils::isNotEmpty)
.reduce("", (value1, value2) -> value1 + "-" + value2);
final Map<String, String> events = result.getEvents().stream()
.collect(Collectors.toUnmodifiableMap(Event::getReferenceID, mapper));

How to convert a nested complex scala Map to Java

I'm trying to use a Scala library in my Java program and I have some difficulties to convert a complex Scala Map to Java.
My used Scala object method has the following return type: scala.collection.mutable.Map<String, Map<Object, Seq<Object>>>
How do I convert that to a Java equivalent of Map<String, Map<Object, List<Object>>> ?
I already played around with the JavaConversions and JavaConvertors packages but no luck :(
public void getPartitionAssignmentForTopics(final List<String> topics) {
final Seq<String> seqTopics = scala.collection.JavaConversions.asScalaBuffer(topics).toList();
scala.collection.mutable.Map<String, Map<Object, Seq<Object>>> map2 = zkUtils
.getPartitionAssignmentForTopics(seqTopics);
val map:scala.collection.mutable.Map[String, Map[Object, Seq[Object]]] = scala.collection.mutable.Map()
map:
collection.mutable.Map[String, Map[Object, Seq[Object]]] =Map()
map.mapValues(_.mapValues(_.asJava).asJava).asJava
res2:
java.util.Map[String, java.util.Map[Object, java.util.List[Object]]] ={}
}
This does not compile :)
With playing around I meant that I use the following code to convert from Scala Seq to Java List:
scala.collection.JavaConversions.seqAsJavaList(zkUtils.getAllTopics());
I ended up with the following code. Not really nice :D
public java.util.Map<String, java.util.Map<Integer, java.util.List<Integer>>> getPartitionAssignmentForTopics(final List<String> topics) {
final scala.collection.Seq<String> seqTopics = scala.collection.JavaConversions.asScalaBuffer(topics).toList();
scala.collection.mutable.Map<String, scala.collection.Map<Object, scala.collection.Seq<Object>>> tmpMap1 =
zkUtils.getPartitionAssignmentForTopics(seqTopics);
final java.util.Map<String, java.util.Map<Integer, java.util.List<Integer>>> result = new HashMap<>();
java.util.Map<String, Map<Object, Seq<Object>>> tmpMap2 = JavaConversions.mapAsJavaMap(tmpMap1);
tmpMap2.forEach((k1, v1) -> {
String topic = (String)k1;
java.util.Map<Object, Seq<Object>> objectSeqMap = JavaConversions.mapAsJavaMap(v1);
java.util.Map<Integer, List<Integer>> tmpResultMap = new HashMap<>();
objectSeqMap.forEach((k2, v2) -> {
Integer tmpInt = (Integer)k2;
List<Integer> tmpList = (List<Integer>)(Object)JavaConversions.seqAsJavaList(v2);
tmpResultMap.put(tmpInt, tmpList);
});
result.put(topic, tmpResultMap);
});
return result;
}

how to format output of Join RDD using java

JavaPairRDD<String, Tuple2<Tuple2<String, Integer>, Double>> accountNew =
accountRecPair.join(accountCnt).join(accountSum);
( Key, (value))
------------------------------
(12,(ID1,12,1062.0,2),68605.0))
i would like myoutput without "(" and ")"
ID1,12,1062.0,2,68605.0
Since tuples are not collections (they are more like case classes), there is no easy way to flatten the structure. You have to explicitly map your result after each join to extract the data the nested tuple structure and put them in a flat tuple structure.
JavaRDD<String> outputFile = accountNew.map(
new Function< Tuple2<String, Tuple2<Tuple2<String, Integer>, Double>>, String>() {
public String call(
Tuple2<String, Tuple2<Tuple2<String, Integer>, Double>> rec)
{
String orderRec ;
// orderRec = rec._1 ;
Tuple2<Tuple2<String, Integer>, Double> rec1 = rec._2() ;
Tuple2<String, Integer> rec2 = rec1._1() ;
orderRec = rec2._1 + "," + rec2._2().toString() + "," + rec1._2().toString() ;
return orderRec;
}
}
) ;
Here is what I did to format the output.
Blockquote

How to convert map to url query string?

Do you know of any utility class/library, that can convert Map into URL-friendly query string?
Example:
I have a map:
"param1"=12,
"param2"="cat"
I want to get:
param1=12&param2=cat
final output
relativeUrl+param1=12&param2=cat
The most robust one I saw off-the-shelf is the URLEncodedUtils class from Apache Http Compoments (HttpClient 4.0).
The method URLEncodedUtils.format() is what you need.
It doesn't use map so you can have duplicate parameter names, like,
a=1&a=2&b=3
Not that I recommend this kind of use of parameter names.
Here's something that I quickly wrote; I'm sure it can be improved upon.
import java.util.*;
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
public class MapQuery {
static String urlEncodeUTF8(String s) {
try {
return URLEncoder.encode(s, "UTF-8");
} catch (UnsupportedEncodingException e) {
throw new UnsupportedOperationException(e);
}
}
static String urlEncodeUTF8(Map<?,?> map) {
StringBuilder sb = new StringBuilder();
for (Map.Entry<?,?> entry : map.entrySet()) {
if (sb.length() > 0) {
sb.append("&");
}
sb.append(String.format("%s=%s",
urlEncodeUTF8(entry.getKey().toString()),
urlEncodeUTF8(entry.getValue().toString())
));
}
return sb.toString();
}
public static void main(String[] args) {
Map<String,Object> map = new HashMap<String,Object>();
map.put("p1", 12);
map.put("p2", "cat");
map.put("p3", "a & b");
System.out.println(urlEncodeUTF8(map));
// prints "p3=a+%26+b&p2=cat&p1=12"
}
}
I found a smooth solution using java 8 and polygenelubricants' solution.
parameters.entrySet().stream()
.map(p -> urlEncodeUTF8(p.getKey()) + "=" + urlEncodeUTF8(p.getValue()))
.reduce((p1, p2) -> p1 + "&" + p2)
.orElse("");
In Spring Util, there is a better way..,
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.util.MultiValueMap;
import org.springframework.util.concurrent.ListenableFuture;
import org.springframework.web.util.UriComponents;
import org.springframework.web.util.UriComponentsBuilder;
MultiValueMap<String, String> params = new LinkedMultiValueMap<String, String>();
params.add("key", key);
params.add("storeId", storeId);
params.add("orderId", orderId);
UriComponents uriComponents = UriComponentsBuilder.fromHttpUrl("http://spsenthil.com/order").queryParams(params).build();
ListenableFuture<ResponseEntity<String>> responseFuture = restTemplate.getForEntity(uriComponents.toUriString(), String.class);
Update June 2016
Felt compelled to add an answer having seen far too many SOF answers with dated or inadequate answers to very common problem - a good library and some solid example usage for both parse and format operations.
Use org.apache.httpcomponents.httpclient library. The library contains this org.apache.http.client.utils.URLEncodedUtils class utility.
For example, it is easy to download this dependency from Maven:
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5</version>
</dependency>
For my purposes I only needed to parse (read from query string to name-value pairs) and format (read from name-value pairs to query string) query strings. However, there are equivalents for doing the same with a URI (see commented out line below).
// Required imports
import org.apache.http.NameValuePair;
import org.apache.http.client.utils.URLEncodedUtils;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.nio.charset.StandardCharsets;
// code snippet
public static void parseAndFormatExample() throws UnsupportedEncodingException {
final String queryString = "nonce=12345&redirectCallbackUrl=http://www.bbc.co.uk";
System.out.println(queryString);
// => nonce=12345&redirectCallbackUrl=http://www.bbc.co.uk
final List<NameValuePair> params =
URLEncodedUtils.parse(queryString, StandardCharsets.UTF_8);
// List<NameValuePair> params = URLEncodedUtils.parse(new URI(url), "UTF-8");
for (final NameValuePair param : params) {
System.out.println(param.getName() + " : " + param.getValue());
// => nonce : 12345
// => redirectCallbackUrl : http://www.bbc.co.uk
}
final String newQueryStringEncoded =
URLEncodedUtils.format(params, StandardCharsets.UTF_8);
// decode when printing to screen
final String newQueryStringDecoded =
URLDecoder.decode(newQueryStringEncoded, StandardCharsets.UTF_8.toString());
System.out.println(newQueryStringDecoded);
// => nonce=12345&redirectCallbackUrl=http://www.bbc.co.uk
}
This library did exactly what I needed and was able to replace some hacked custom code.
If you actually want to build a complete URI, try URIBuilder from Apache Http Compoments (HttpClient 4).
This does not actually answer the question, but it answered the one I had when I found this question.
I wanted to build on #eclipse's answer using java 8 mapping and reducing.
protected String formatQueryParams(Map<String, String> params) {
return params.entrySet().stream()
.map(p -> p.getKey() + "=" + p.getValue())
.reduce((p1, p2) -> p1 + "&" + p2)
.map(s -> "?" + s)
.orElse("");
}
The extra map operation takes the reduced string and puts a ? in front only if the string exists.
Another 'one class'/no dependency way of doing it, handling single/multiple:
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
public class UrlQueryString {
private static final String DEFAULT_ENCODING = "UTF-8";
public static String buildQueryString(final LinkedHashMap<String, Object> map) {
try {
final Iterator<Map.Entry<String, Object>> it = map.entrySet().iterator();
final StringBuilder sb = new StringBuilder(map.size() * 8);
while (it.hasNext()) {
final Map.Entry<String, Object> entry = it.next();
final String key = entry.getKey();
if (key != null) {
sb.append(URLEncoder.encode(key, DEFAULT_ENCODING));
sb.append('=');
final Object value = entry.getValue();
final String valueAsString = value != null ? URLEncoder.encode(value.toString(), DEFAULT_ENCODING) : "";
sb.append(valueAsString);
if (it.hasNext()) {
sb.append('&');
}
} else {
// Do what you want...for example:
assert false : String.format("Null key in query map: %s", map.entrySet());
}
}
return sb.toString();
} catch (final UnsupportedEncodingException e) {
throw new UnsupportedOperationException(e);
}
}
public static String buildQueryStringMulti(final LinkedHashMap<String, List<Object>> map) {
try {
final StringBuilder sb = new StringBuilder(map.size() * 8);
for (final Iterator<Entry<String, List<Object>>> mapIterator = map.entrySet().iterator(); mapIterator.hasNext();) {
final Entry<String, List<Object>> entry = mapIterator.next();
final String key = entry.getKey();
if (key != null) {
final String keyEncoded = URLEncoder.encode(key, DEFAULT_ENCODING);
final List<Object> values = entry.getValue();
sb.append(keyEncoded);
sb.append('=');
if (values != null) {
for (final Iterator<Object> listIt = values.iterator(); listIt.hasNext();) {
final Object valueObject = listIt.next();
sb.append(valueObject != null ? URLEncoder.encode(valueObject.toString(), DEFAULT_ENCODING) : "");
if (listIt.hasNext()) {
sb.append('&');
sb.append(keyEncoded);
sb.append('=');
}
}
}
if (mapIterator.hasNext()) {
sb.append('&');
}
} else {
// Do what you want...for example:
assert false : String.format("Null key in query map: %s", map.entrySet());
}
}
return sb.toString();
} catch (final UnsupportedEncodingException e) {
throw new UnsupportedOperationException(e);
}
}
public static void main(final String[] args) {
// Examples: could be turned into unit tests ...
{
final LinkedHashMap<String, Object> queryItems = new LinkedHashMap<String, Object>();
queryItems.put("brand", "C&A");
queryItems.put("count", null);
queryItems.put("misc", 42);
final String buildQueryString = buildQueryString(queryItems);
System.out.println(buildQueryString);
}
{
final LinkedHashMap<String, List<Object>> queryItems = new LinkedHashMap<String, List<Object>>();
queryItems.put("usernames", new ArrayList<Object>(Arrays.asList(new String[] { "bob", "john" })));
queryItems.put("nullValue", null);
queryItems.put("misc", new ArrayList<Object>(Arrays.asList(new Integer[] { 1, 2, 3 })));
final String buildQueryString = buildQueryStringMulti(queryItems);
System.out.println(buildQueryString);
}
}
}
You may use either simple (easier to write in most cases) or multiple when required. Note that both can be combined by adding an ampersand...
If you find any problems let me know in the comments.
This is the solution I implemented, using Java 8 and org.apache.http.client.URLEncodedUtils. It maps the entries of the map into a list of BasicNameValuePair and then uses Apache's URLEncodedUtils to turn that into a query string.
List<BasicNameValuePair> nameValuePairs = params.entrySet().stream()
.map(entry -> new BasicNameValuePair(entry.getKey(), entry.getValue()))
.collect(Collectors.toList());
URLEncodedUtils.format(nameValuePairs, Charset.forName("UTF-8"));
There's nothing built into Java to do this. But, hey, Java is a programming language, so. Let's program it!
map
.entrySet()
.stream()
.map(e -> e.getKey() + "=" + e.getValue())
.collect(Collectors.joining("&"))
This gives you param1=12&param2=cat. Now we need to join the URL and this bit together. You'd think you can just do: URL + "?" + theAbove but if the URL already contains a question mark, you have to join it all together with "&" instead. One way to check is to see if there's a question mark in the URL someplace already.
Also, I don't quite know what is in your map. If it's raw stuff, you probably have to safeguard the call to e.getKey() and e.getValue() with URLEncoder.encode or similar.
Yet another way to go is that you take a wider view. Are you trying to append a map's content to a URL, or... Are you trying to make an HTTP (S) request from a Java process with the stuff in the map as (additional) HTTP params? In the latter case, you can look into an HTTP library like OkHttp which has some nice APIs to do this job, then you can forego any need to mess about with that URL in the first place.
Using EntrySet and Streams:
map
.entrySet()
.stream()
.map(e -> e.getKey() + "=" + e.getValue())
.collect(Collectors.joining("&"));
You can use a Stream for this, but instead of appending query parameters myself I'd use a Uri.Builder. For example:
final Map<String, String> map = new HashMap<>();
map.put("param1", "cat");
map.put("param2", "12");
final Uri uri =
map.entrySet().stream().collect(
() -> Uri.parse("relativeUrl").buildUpon(),
(builder, e) -> builder.appendQueryParameter(e.getKey(), e.getValue()),
(b1, b2) -> { throw new UnsupportedOperationException(); }
).build();
//Or, if you consider it more readable...
final Uri.Builder builder = Uri.parse("relativeUrl").buildUpon();
map.entrySet().forEach(e -> builder.appendQueryParameter(e.getKey(), e.getValue())
final Uri uri = builder.build();
//...
assertEquals(Uri.parse("relativeUrl?param1=cat&param2=12"), uri);
Here's a simple kotlin solution:
fun Map<String, String>.toUrlParams(): String =
entries.joinToString("&") {
it.key.toUrlEncoded() + "=" + it.value.toUrlEncoded()
}
fun String.toUrlEncoded(): String = URLEncoder.encode(
this, StandardCharsets.UTF_8
)
To improve a little bit upon #eclipse's answer: In Javaland a request parameter map is usually represented as a Map<String, String[]>, a Map<String, List<String>> or possibly some kind of MultiValueMap<String, String> which is sort of the same thing. In any case: a parameter can usually have multiple values. A Java 8 solution would therefore be something along these lines:
public String getQueryString(HttpServletRequest request, String encoding) {
Map<String, String[]> parameters = request.getParameterMap();
return parameters.entrySet().stream()
.flatMap(entry -> encodeMultiParameter(entry.getKey(), entry.getValue(), encoding))
.reduce((param1, param2) -> param1 + "&" + param2)
.orElse("");
}
private Stream<String> encodeMultiParameter(String key, String[] values, String encoding) {
return Stream.of(values).map(value -> encodeSingleParameter(key, value, encoding));
}
private String encodeSingleParameter(String key, String value, String encoding) {
return urlEncode(key, encoding) + "=" + urlEncode(value, encoding);
}
private String urlEncode(String value, String encoding) {
try {
return URLEncoder.encode(value, encoding);
} catch (UnsupportedEncodingException e) {
throw new IllegalArgumentException("Cannot url encode " + value, e);
}
}
If you need just the query string (not the whole URL) and you are using Spring Framework, you can do this:
import org.springframework.web.util.UriComponentsBuilder;
...
final String queryString = UriComponentsBuilder.newInstance()
.queryParam("many", "7", "14", "21")
.queryParam("single", "XYZ")
.build()
.toUri()
.getQuery();
System.out.println(queryString);
the result is:
many=7&many=14&many=21&single=XYZ
I make these functions than also send just the property name when the value is null.
public static String urlEncode(Map<?, ?> map) {
return map.entrySet().stream().map(
entry -> entry.getValue() == null
? urlEncode(entry.getKey())
: urlEncode(entry.getKey()) + "=" + urlEncode(entry.getValue())
).collect(Collectors.joining("&"));
}
public static String urlEncode(Object obj) {
return URLEncoder.encode(obj.toString(), StandardCharsets.UTF_8);
}
For multivalue map you can do like below (using java 8 stream api's)
Url encoding has been taken cared in this.
MultiValueMap<String, String> params = new LinkedMultiValueMap<>();
String urlQueryString = params.entrySet()
.stream()
.flatMap(stringListEntry -> stringListEntry.getValue()
.stream()
.map(s -> UriUtils.encode(stringListEntry.getKey(), StandardCharsets.UTF_8.toString()) + "=" +
UriUtils.encode(s, StandardCharsets.UTF_8.toString())))
.collect(Collectors.joining("&"));
Personally, I'd go for a solution like this, it's incredibly similar to the solution provided by #rzwitserloot, only subtle differences.
This solution is small, simple & clean, it requires very little in terms of dependencies, all of which are a part of the Java Util package.
Map<String, String> map = new HashMap<>();
map.put("param1", "12");
map.put("param2", "cat");
String output = "someUrl?";
output += map.entrySet()
.stream()
.map(x -> x.getKey() + "=" + x.getValue() + "&")
.collect(Collectors.joining("&"));
System.out.println(output.substring(0, output.length() -1));
Kotlin
mapOf(
"param1" to 12,
"param2" to "cat"
).map { "${it.key}=${it.value}" }
.joinToString("&")
a very lightweight answer it works for me
public static String queryStr(Map<String, String> data) throws UnsupportedEncodingException {
StringBuilder query = new StringBuilder();
for (Entry<String, String> entry : data.entrySet()) {
if (query.length() > 0) {
query.append('&');
}
query.append(entry.getKey()).append('=');
query.append(URLEncoder.encode(entry.getValue(), "UTF-8"));
}
return query.toString();
}

Categories

Resources