Lets say I have a very long string:
trillions of chunks
|
v
/asdf/........./bar/baz/foo
^
|
what I try to find is closer to the right:
the data after 9999999th '/'
I need all the chunks of data up to this slash, but not any slashes. I see this as a stream and want to do the following:
I start to read symbols from the back and count slashes.
Anything but slash I put into Last-In-First-Out data structure.
In order not to wait for the whole operation to finish, I start reading data from the lifo datastructure as it becomes available.
I terminate after the 9999999th '/'
Can something like this be accomplished with reactive streams and how?
I think the following code will achve what you want
#Test
public void reactiveParser() throws InterruptedException {
ConnectableFlux<String> letters = Flux.create((Consumer<? super FluxSink<String>>) t -> {
char[] chars = "sfdsfsdf/sdf/sdfs/dfsdfsd/fsd/fsd/fs/df/sdf".toCharArray();
for (char c : chars) {
t.next(String.valueOf(c));
}
}).publish();
letters
.window(
letters.filter( t -> t.equals("/"))
)
.flatMap( t -> t.collectList())
.map( t -> t.stream().collect(Collectors.joining()))
.subscribe(t -> {
System.out.println(t);
});
letters.connect();
}
The example above utilizes the project reactor. Which is pretty cool way of doing the reactive stuff inside of java.
There is plenty of optimization that can be done in the following code. Not using Strings to represent a single letter would be one of them.
But the basic idea is there. You create flux/observable that emits a letters as they come in and make that observable shareable (you have to window over emitting values) and then just collect them in to a single sting. The code bellow should give the following output:
sfdsfsdf
/sdf
/sdfs
/dfsdfsd
/fsd
/fsd
/fs
/df
Of course you have to utilize non-blocking connection so the bytes could be read asynchronously.
Related
I have couple of xmls which needs to be compared with different set of similar xml and while comparing i need to ignore tags based on a condition, for example
personal.xml - ignore fullname
address.xml - igone zipcode
contact.xml - ignore homephone
here is the code
Diff documentDiff=DiffBuilder
.compare(actualxmlfile)
.withTest(expectedxmlfile)
.withNodeFilter(node -> !node.getNodeName().equals("FullName"))
.ignoreWhitespace()
.build();
How can i add conditions at " .withNodeFilter(node -> !node.getNodeName().equals("FullName")) " or is there a smarter way to do this
You can join multiple conditions together using "and" (&&):
private static void doDemo1(File actual, File expected) {
Diff docDiff = DiffBuilder
.compare(actual)
.withTest(expected)
.withNodeFilter(
node -> !node.getNodeName().equals("FullName")
&& !node.getNodeName().equals("ZipCode")
&& !node.getNodeName().equals("HomePhone")
)
.ignoreWhitespace()
.build();
System.out.println(docDiff.toString());
}
If you want to keep your builder tidy, you can move the node filter to a separate method:
private static void doDemo2(File actual, File expected) {
Diff docDiff = DiffBuilder
.compare(actual)
.withTest(expected)
.withNodeFilter(node -> testNode(node))
.ignoreWhitespace()
.build();
System.out.println(docDiff.toString());
}
private static boolean testNode(Node node) {
return !node.getNodeName().equals("FullName")
&& !node.getNodeName().equals("ZipCode")
&& !node.getNodeName().equals("HomePhone");
}
The risk with this is you may have element names which appear in more than one type of file - where that node needs to be filtered from one type of file, but not any others.
In this case, you would also need to take into account the type of file you are handling. For example, you can use the file names (if they follow a suitable naming convention) or use the root elements (assuming they are different) - such as <Personal>, <Address>, <Contact> - or whatever they are, in your case.
However, if you need to distinguish between XML file types, for this reason, you may be better off using that information to have separate DiffBuilder objects, with different filters. That may result in clearer code.
I had provided the separate method in the below link for !node.getNodeName().equals("FullName")(which you are using in your code), I think by using that separate method you can just pass the array of nodes which you want to ignore and see the results. And incase you wish to add any other conditions based on your requirement, you can try and play in this method.
https://stackoverflow.com/a/68099435/13451711
Having spat the Google Guava kool-aid back out of our mouths, and diving head-first into our new infatuation with VAVR and its sparkling ideals, say we are map()ping a Stream, performing foldLeft() on a Traversable, or similar, and one of the inner functions throws a checked exception. Now we are staring down at compiler errors, unhandled exceptions.
How aught such exceptions be handled ideally, with idiomatic VAVR. What does the resulting code look like, what are the patterns. We have Options and Trys... How does it all come together.
Oh, and tricks like sneaky exceptions disinterest us.
You can use CheckedFunction[0-8].liftTry() to convert a function that throws checked exceptions to a total function that returns the result of the original function wrapped in a Try. If the original function returns a value without throwing, it will be wrapped in a Success, if it throws, the exception will be wrapped in a Failure.
You will then need to take a decision on how to handle errors in the context of multiple values. Here are some examples on what you could do with a bunch of Try values.
Array<String> input = Array.of(
"123", "456", "789", "not a number", "1111", "another non-number"
);
// try and parse all the strings
Array<Try<Integer>> trys = input.map(CheckedFunction1.liftTry(Integer::parseInt));
// you can take just the successful values
Array<Integer> values = trys.flatMap(Try::iterator);
// you can look just for the failures
Array<Throwable> failures = trys.filter(Try::isFailure).map(Try::getCause);
// you can partition by the outcome and extract values/errors
Tuple2<Traversable<Integer>, Traversable<Throwable>> partition =
trys.partition(Try::isSuccess)
.map(
seq -> seq.map(Try::get),
seq -> seq.map(Try::getCause)
);
// you can do a short-circuiting parse of the original sequence
// this will stop at the first error and return it as a failure
// or take all success values and wrap them in a Seq wrapped in a Try
Try<Seq<Integer>> shortCircuit = Try.sequence(
input.iterator() //iterator is lazy, so it's not fully evaluated if not needed
.map(CheckedFunction1.liftTry(Integer::parseInt))
);
// Failure(java.lang.NumberFormatException: For input string: "not a number")
Of course, you can use any other vavr collection in place of Array.
I have some concatenated observable using flatMap such as
api.call1()
.flatMap{a-> api.call2(a)}
.flatMap{b-> api.call3(a,b) //<- no access to a}
the issue is that In third concatenated call I need what I got from previous one but also the result from the first one
this is the scheme:
A->(a)->B->(b)-> C( needs b but also a)->...
I try to solve it out like that :
1 approach ( nested 3rd observable inside the 2nd one)
api.call1()
.flatMap{a-> api.call2(a).flatMap{b-> api.call3(a,b)}}
it works but I know that this is not a good approach (because of passing parms from outside of the pipe)
2 approach (better but a lot of boilerplate)
api.call1().flatMap{ a->
Observable.combineLatest(Observable.just(a),api.call2(a),
BiFunction{ a,b -> Pair(a,b)})
.flatMap(pair:Pair<A,B> -> api.call3(pair.first,pair.second))
}}
anybody know better approach (some fashion operator)
Thanks everybody sory for my poor ingles.
There is a second overload of flatMap which takes both the input and the output. You can then combine these into a Pair to send to the next flatMap
api.call1()
.flatMap(
{ a -> api.call2(a) },
{ a, b -> new Pair(a, b) }
)
.flatMap { pair-> api.call3(pair.first, pair.second) }
I have some data stored in a JPA Repository that I am trying to process. I would like to be able to use Java 8 Streams to do so, but can not figure out how to get the required information. This particular 'Entity' is actually only for recovery, so it holds items that would need to be processed after something like a power-fail/restart.
Using pre-Java 8 for-loops the code would look like:
List<MyEntity> deletes = myEntityJpaRepository.findByDeletes();
for (MyEntity item : deletes) {
String itemJson = item.getData();
// use a Jackson 'objectMapper' already setup to de-serialize
MyEventClass deleteEvent = objectMapper.readValue(itemJson, MyEventClass.class);
processDelete(deleteEvent, item.getId());
}
The problem arises from the two parameter method called at the very end. Using Streams, I believe I would do:
// deletes.stream()
// .map(i -> i.getData())
// .map(event -> objectMapper.readValue(event, MyEventClass.class))
// .forEach(??? can't get 'id' here to invoke 2 parameter method);
I have a solution (without Streams) that I can live with. However I would think this problem comes up a lot, thus my question is: IN GENERAL, is there a way using Streams to accomplish what I am trying to do?
Why not a Pair return on your map operation:
.map(i -> new Pair<>(i.getData(), i.getId()))
.map(pair -> new Pair<>(objectMapper.readValue(pair.getLeft(), MyEventClass.class), pair.getRight())
.forEach(p -> processDelete(pair.getLeft(), pair.getRight()))
I did not compile this, so there might be minor things to fix. But in general, you would need a Holder to pass your objects to the next stage in such a case. Either a Pair or some type or even a array.
Why not doing it simply this way?
deletes.forEach(item ->
processDelete(objectMapper.readValue(item.getData(), MyEventClass.class),
item.getId()));
This is a start at least, I guess it is dependent on why you want to use stream and how much you want to make it more functional
List<MyEntity> deletes = myEntityJpaRepository.findByDeletes();
deletes.stream().foreach(item -> {
String itemJson = item.getData();
// use a Jackson 'objectMapper' already setup to de-serialize
MyEventClass deleteEvent = objectMapper.readValue(itemJson, MyEventClass.class);
processDelete(deleteEvent, item.getId());
});
How to filter a list/collection of streams based on url parameters, for example:
?filter=(type=="audio"&&systemBitrate<100000)||(type=="video"&&systemBitrate<1024000)
I know this can be done using statically:
List<StreamItem> results = streamList.stream().filter(s -> s.type == "audio" && s.systemBitrate < 100000).collect(Collectors.toList());
Simple object:
public class StreamItem {
String name;
String type;
int systemBitrate;
}
The idea is to dynamically filter playback manifest in a similar way to the one below and play only selected tracks:
curl -v 'http://demo.unified-streaming.com/video/tears-of-steel/tears-of-steel.ism/Manifest?filter=(type=="audio"%26%26systemBitrate<100000)||(type=="video"%26%26systemBitrate<1024000)'
One way to do it is to use one of the "Expression Language" libraries,
to compile your filter expression and then apply it to the elements of your stream.
Below is a short example using MVEL:
pom.xml
<dependency>
<groupId>org.mvel</groupId>
<artifactId>mvel2</artifactId>
<version>2.3.1.Final</version>
</dependency>
java code
Serializable expr = MVEL.compileExpression(
"(type==\"audio\"&&systemBitrate<100000)||(type==\"video\"&&systemBitrate<1024000)"
);
Arrays.asList(
new StreamItem("audio",10000),
new StreamItem("audio",200000),
new StreamItem("video",200000),
new StreamItem("video",2000000)
)
.stream()
.filter(e->MVEL.executeExpression(expr,e,boolean.class))
.forEach(System.out::println);
output
Element{type='audio', systemBitrate=10000}
Element{type='video', systemBitrate=200000}
Please note that your StreamItem class must have getters defined for both type and systemBitrate properties, for the MVEL to be able to resolve them.
Don't expect this to be blazing fast, yet it still should be fast enough for most practical tasks, taking that expression is compiled, before use.
Filtering a list of 1000000 (one million) StreamItems, using the expression above, takes ~150ms on my laptop, on average.