Streaming Key Values

This post describes a implementation for streaming a String of space separated Key Values with Java 8. In order to implement a streaming keyvalue reader two simple classes are implemented. One to handle navigation in the string and one to make a Iterable from the String. The implementation I provide is basic but has enough extension points to meet more extensive requirements.

The problem.

We all have faced the same problem when dealing with data from other sources. Question is always do I take a existing library of a DIY version. All implementations I know of are based on the pre Java 8 era. For the purpose of my current project (Java 8) I wanted to stream a String of Key Values so I could easily perform Streaming operations on it.

A naive approach

A quick start for implementing the required functionality is doing a multiple split on the input string i.e. split on the entry separator and next on the value separator. This might work but you always run into issues with escaped characters and such. So this would not work :-(

The Streaming approach

Functional programming is sort of available in Java 8. We can do streams and lambda’s. Processing a stream of KeyValue in a more functional manner has advantages.

Below are the steps I performed to make a String of keyvalues available asa a Stream

Spliterator to the rescue

In order to implement your own Stream on an arbitrary input is writing a Spliterator. Spliterators are not that easy to implement, all kinds of concurrency issues come up with that one. It is much simpler to implement an Iterable<Map.Entry<String, String>>. In that case you only need to implement @Override a single method to start working with the data as a Stream.

This only leaves the actual processing of the String data.

Parsing the String data.

When parsing the data you need to keep track of the previous characters and mark the start of either a Key or Value. For this purpose I created a KeyValueReader. This reader reads the string in a forward motion. For example to read a single KeyValue you skipUntil you find the = sign. When this is found you have the key. Next you skip until the " character. This is the start of your value. skip again until the " and you have the value.

class KeyValueReader {
private final String line;
private int idx;
private int mark;
public KeyValueReader(String l) {
this.line = l;
}
public void mark() {
this.mark = this.idx;
}
public String getMarkedSegment() {
correctState(this.mark <= this.idx, "mark is greater than this.idx");
return this.line.substring(this.mark, this.idx);
}
public int getc() {
correctState(this.idx + 1 < this.line.length(), "Attempt to read past end");
return this.line.charAt(this.idx++);
}
public boolean available() {
return idx < (line.length() - 1);
}
public boolean is(char c) {
return idx < line.length() && this.line.charAt(this.idx) == c;
}
private boolean was(char c) {
correctState(this.idx >= 1, "Reading before beginning of data");
return this.line.charAt(this.idx - 1) == c;
}
}
view raw KeyValueReader hosted with ❤ by GitHub

Making the KeyValueReaderIterable

The KeyValueIterable is an implementation of the Iterable<T> interface. The only method that you need to provide is the Iterator<T> iterator(). Which in his turn is the implementation of the Iterator. Implementing a Iterator is straight forward. As mentioned The KeyValueReader can navigate forwards in a String. With that it is easy to achieve the next and hasNext functionality required to implement a Iterator.

class KeyValueReaderIterable extends KeyValueReader implements Iterable<Map.Entry<String, String>> {
/**
* Default constructor
* @param l line
*/
public KeyValueReaderIterable(String l) {
super(l);
}
@Override
public Iterator<Map.Entry<String, String>> iterator() {
return new Iterator<Map.Entry<String, String>>() {
@Override
public boolean hasNext() {
return available();
}
@Override
public Map.Entry<String, String> next() {
return nextEntry();
}
};
}
private String getKey() {
if (is('\"')) {
getc();
}
mark();
skipUntil('=');
return getMarkedSegment();
}
private String getValue() {
skipTo('"');
mark();
skipUntil('"');
return getMarkedSegment();
}
private Map.Entry<String, String> nextEntry() {
return new AbstractMap.SimpleEntry<>(getKey().trim(), getValue());
}
/**
* Stream a KVString
* @param data kv data string
* @return {@link Stream} of {@link Map.Entry}
*/
public static Stream<Map.Entry<String, String>> stream(final String data) {
return StreamSupport.stream(new KeyValueReaderIterable(data).spliterator(), true);
}
}

Proof on the pudding

Below is a test case that shows how you can use this Iterable to parse a line of key values.

(https://github.com/elucidator/keyvalue-parser)

private static final String INPUT = "key=\"value\" withEqualsSign=\"Base64==\" isEmpty=\"\" withSpaces=\" s p a c e s \" withEscapeChar=\"aaa\\\\bbb\" withEscapeChar2=\"aaa\\\"bbb\" withEscapeChar3=\"aaa\\]bbb\"";
@Test
public void variousVariations() throws Exception {
final List<Map.Entry<String, String>> collect = KeyValueReaderIterable.stream(INPUT).collect(Collectors.toList());
assertThat(collect, hasItem(new AbstractMap.SimpleEntry<>("key", "value")));
assertThat(collect, hasItem(new AbstractMap.SimpleEntry<>("withEqualsSign", "Base64==")));
assertThat(collect, hasItem(new AbstractMap.SimpleEntry<>("isEmpty", "")));
assertThat(collect, hasItem(new AbstractMap.SimpleEntry<>("withSpaces", " s p a c e s ")));
assertThat(collect, hasItem(new AbstractMap.SimpleEntry<>("withEscapeChar", "aaa\\\\bbb")));
assertThat(collect, hasItem(new AbstractMap.SimpleEntry<>("withEscapeChar2", "aaa\\\"bbb")));
assertThat(collect, hasItem(new AbstractMap.SimpleEntry<>("withEscapeChar3", "aaa\\]bbb")));
}

The complete implementation can be found at [GitHub]