Stream.sorted() - For unordered streams, no stability guarantees are made.

So I was writing code, and I had to sort a stream, because originally I had a Set, and everyone knows Sets are unordered by default.

But my eye fell on the javadoc on .sorted(). To be more specific:

For ordered streams, the sort is stable. For unordered streams, no stability guarantees are made.

And the sentence kind of made me nervous.

Example

An example of an unstable sort is visible in the unit test below. The unit test will sometimes pass and sometimes fail.

The reason for this is that Ken Thompson and Simon Thompson are equivalent in the sorting and, as it is an unsorted stream to begin with, its order can change between program runs.

Something to keep in mind.

References

Oracle Javadoc - Stream (Java SE 21 & JDK 21)

https://docs.oracle.com/en%2Fjava%2Fjavase%2F21%2Fdocs%2Fapi%2F%2F/java.base/java/util/stream/Stream.html#sorted()

Streams and Filters

A simple little thing.

I like to use streams and filters, and I was wondering what's the best way to go about some things.

For example: I wish to search for a person in a list of Persons.

  @Test
  public void testSimple() {
    Person personToFind = new Person(null, "Mr.", "Bear", "Netherlands");
    Person otherPersonToFind = new Person(null, "Linda", "Lovelace", "England");
    assertThat(Persons.get().stream()
        .filter(person -> Objects.equals(personToFind.name(), person.name()) &&
            Objects.equals(personToFind.surname(), person.surname()) &&
            Objects.equals(personToFind.country(), person.country()))
        .findFirst()).isPresent();

    assertThat(Persons.get().stream()
        .filter(person -> Objects.equals(otherPersonToFind.name(), person.name()) &&
            Objects.equals(otherPersonToFind.surname(), person.surname()) &&
            Objects.equals(otherPersonToFind.country(), person.country()))
        .findFirst()).isEmpty();
  }

This works, but the single filter with all the && in it seems a bit unreadable.

Of course, I could replace the x&&y&&z by three filters, as it boils down tot the same thing.

  @Test
  public void testSimple2() {
    Person personToFind = new Person(null, "Mr.", "Bear", "Netherlands");
    Person otherPersonToFind = new Person(null, "Linda", "Lovelace", "England");
    assertThat(Persons.get().stream()
        .filter(person -> Objects.equals(personToFind.name(), person.name()))
        .filter(person -> Objects.equals(personToFind.surname(), person.surname()))
        .filter(person -> Objects.equals(personToFind.country(), person.country()))
        .findFirst()).isPresent();

    assertThat(Persons.get().stream()
        .filter(person -> Objects.equals(otherPersonToFind.name(), person.name()))
        .filter(person -> Objects.equals(otherPersonToFind.surname(), person.surname()))
        .filter(person -> Objects.equals(otherPersonToFind.country(), person.country()))
        .findFirst()).isEmpty();
  }

But my colleague always likes to use specific methods for lambdas, even if they're only a little complex. It just reads easier.

private boolean compare(Person person, Person otherPerson) {
    return Objects.equals(otherPerson.name(), person.name()) &&
        Objects.equals(otherPerson.surname(), person.surname()) &&
        Objects.equals(otherPerson.country(), person.country());
  }

  @Test
  public void testSimple3() {
    Person personToFind = new Person(null, "Mr.", "Bear", "Netherlands");
    Person otherPersonToFind = new Person(null, "Linda", "Lovelace", "England");
    assertThat(Persons.get().stream()
        .filter(person -> compare(person, personToFind))
        .findFirst()).isPresent();

    assertThat(Persons.get().stream()
        .filter(person -> compare(person, otherPersonToFind))
        .findFirst()).isEmpty();
  }

My personal preference is the last one.

Is Stream.findFirst() Short-circuited?

So, the assumption is: if I use findFirst on a stream, none of the items in the stream after the first match are evaluated.

I assumed that it was, and it is, but it's always nice to see this verified in a simple test.

  private List<String> list = new ArrayList<>();

  private boolean add(String message, boolean returnValue) {
    list.add(message);
    return returnValue;
  }

  public boolean check() {
    List<Supplier<Boolean>> checks = new ArrayList<>();
    checks.add(super::onLeave);
    checks.add(() -> {
      list.add("First expression");
      return true;
    });
    checks.add(() -> {
      list.add("Second expression");
      return true;
    });
    checks.add(() -> {
      list.add("Third expression");
      return false;
    });
    checks.add(() -> {
      list.add("Fourth expression");
      return true;
    });
    checks.add(() -> {
      list.add("Fifth expression");
      return true;
    });
    checks.add(() -> {
      list.add("Sixth expression");
      return false;
    });
    return checks.stream()
        .filter(t -> t.get().equals(Boolean.FALSE))
        .findFirst()
        .isEmpty();
  }

  @Test
  public void testShortCircuit() {
    assertThat(check()).isFalse();
    assertThat(list)
        .containsExactly("First expression", "Second expression", "Third expression");
  }

As this test passes, it seems that way.

In the very beginning it took some time for me to wrap my head around it, but the operations you define on a stream (.map, .filter, etc.) are not all processed on every item in the stream.

All operations are processed on the first item of the stream, then on the second item of the stream. From this it follows, that a .findFirst() operation will immediately terminate operations if it finds one and the rest of the stream will be ignored.

Refactoring

So, I saw some code that I didn't like, and I decided to refactor it.

The code was thusly.

public boolean onLeave() {
    boolean valid = super.onLeave();
    Account account = database.getAccount();
    ShoppingList shoppingList = database.getShoppinglist(account);
    if (valid) {
      valid = !shoppingList.isEmpty();
    }
    if (valid) {
      valid = !blockedAccounts.contains(account);
    }
    if (valid) {
      String country = account.getPerson().country();
      valid = webshop.alsoShipsTo(country);
    }
    if (valid) {
      valid = account.hasCreditcardAttached() ||
          account.hasPrepaidcardAttached() ||
          account.hasCash();
    }
    return valid;
  }

And I thought to myself, it's really just a list of expressions, where each one is evaluated until you find one that evaluates to false, and then you stop.

So, that's basically a Stream where you filter out all the "false" values, and get the first one.

So, I decided to refactor it just like that.

public boolean onLeave() {
    Account account = database.getAccount();
    ShoppingList shoppingList = database.getShoppinglist(account);

    List<Supplier<Boolean>> checks = new ArrayList<>();
    checks.add(super::onLeave);
    checks.add(() -> !shoppingList.isEmpty());
    checks.add(() -> !blockedAccounts.contains(account));
    checks.add(
        () -> {
          String country = account.getPerson().country();
          return webshop.alsoShipsTo(country);
        });
    checks.add(() -> account.hasCreditcardAttached() ||
        account.hasPrepaidcardAttached() ||
        account.hasCash());

    return checks.stream()
        .filter(t -> t.get().equals(Boolean.FALSE))
        .findFirst()
        .isEmpty();
  }

I proudly showed this to my colleagues, and they immediately shook their heads in disgust.

Apparently, in my eagerness to use Lambdas and Streams and all that cool stuff, what I really had done is recreated the short-circuit version of an If statement in Streams.

I find myself turning to Lambdas and Streams when in reality these are not needed, and my eventual solution works fine without them.

So, rewriting this as a short-circuited IF statement looks like this:

public boolean onLeave() {
    Account account = database.getAccount();
    ShoppingList shoppingList = database.getShoppinglist(account);

    return super.onLeave() &&
        !shoppingList.isEmpty() &&
        !blockedAccounts.contains(account) &&
        webshop.alsoShipsTo(account.getPerson().country()) &&
        (account.hasCreditcardAttached() ||
            account.hasPrepaidcardAttached() ||
            account.hasCash());
  }

Granted, a few more and my IntelliJ will start to complain about the number of expressions in the if statement, and it's possible to clean it up a little by creating separate methods for some of the expressions. But I feel it looks fine.

So, in conclusion, just because you know something cool and shiny, it's no reason to try and use it everywhere!

It is a corrolary that in order to properly use something, you must have some knowledge or experience of when and where it's best to be used.

Combining two collections using streams

So, I have two collections, and I wish to combine both lists somehow.

Requirements are thusly:

• I have a collection newPersons
• I have a collection oldPersons
• I want a collection containing all the oldPersons but replaced (*some of) the oldPersons with the newPersons (based on id).

public record Person(Long id, String name, String surname) {
}

Using the record class above.

Solution 1.

Create a new list based on oldPersons, replace items in this new list with equivalent items in newPersons.

This solution leaves much to be desired. It's confusing and error prone.

I was looking for something better.

  public List<Person> mergeMetadata(List<Person> newPersons,
      List<Person> oldPersons) {
    var result = new ArrayList<>(oldPersons);
    newPersons.forEach(newPerson -> {
      result.stream()
          .filter(person -> person.id().equals(newPerson.id()))
          .findFirst()
          .ifPresent(result::remove);
      result.add(newPerson);
    });
    return result;
  }

Solution 2.

Sometimes getting back to our roots using for-loops can help readability.

We could try it the other way around, see if that helps.

This time we create a new list based on newPersons and add an oldPerson if it's not already in the list.

This seems a little more clear.

  public List<Person> mergeMetadata(List<Person> newPersons,
      List<Person> oldPersons) {
    var result = new ArrayList<>(newPersons);
    for (Person oldPerson : oldPersons) {
      if (result.stream().noneMatch(x -> x.id().equals(oldPerson.id()))) {
        result.add(oldPerson);
      }
    }
    return result;
  }

Solution 3.

Merge two collections into one list, by using a map.

  public List<Person> mergeMetadata(List<Person> newPersons,
      List<Person> oldPersons) {
    Map<Long, Person> result = Stream
        .concat(newPersons.stream(), oldPersons.stream())
        .collect(Collectors.toMap(Person::id, Function.identity(), (l, r) -> l));
    return new ArrayList<>(result.values());
  }

Although this solution seems to be the shortest (in code), using a Map function can be a bit daunting (it was for me) because of inherent complexity in the method call for creating it.

Still, perhaps it's just me and my inexperience with combining maps and streams.

I don't know if there's an even better way. Will keep an eye out.

Using FindFirst in an Odd Way?

So I have a list, and I'm pretty sure it's either empty or contains one element.

So I was thinking too hard, and fabricated something like this:

public Optional<Item> getItem() {
  return items.isEmpty() ? Optional.empty() : Optional.of(items.get(0)));
}

And then I thought: "What am I thinking!?" and used the stream API instead:

public Optional<Item> getItem() {
  return items.stream().findFirst();
}

There.

Seems easy enough.

NullPointerException in Stream

Just a small blurb. I recently helped a colleague who got a NullPointerException whilst dealing with streams.

It looked very innocuous at first.

  @Test(expectedExceptions = NullPointerException.class)
  public void testStream() {
    Optional<String> first = Stream.of(new Person(null, "mrBear"), new Person("George", "Boole"),
            new Person("Ada", "Lovelace"),
            new Person("Tim", "Berners-Lee"), new Person("James", "Gosling"), new Person("Linus", "Torvalds"))
        .map(Person::firstName).findFirst();
  }

Well, it turns out that, if you have a list with null-values, and you try a findFirst() on the stream, and the first value is null, Java will attempt to wrap the null-value into an Optional, which is not allowed.

You get a fancy stacktrace, that shows just this:

java.lang.NullPointerException
	at java.base/java.util.Objects.requireNonNull(Objects.java:208)
	at java.base/java.util.Optional.of(Optional.java:113)
	at java.base/java.util.stream.FindOps$FindSink$OfRef.get(FindOps.java:194)
	at java.base/java.util.stream.FindOps$FindSink$OfRef.get(FindOps.java:191)
	at java.base/java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:150)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:647)
	at com.mrbear.streams.NullPointerExceptionTest.testStream(NullPointerExceptionTest.java:18)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:571)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:707)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:979)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
	at org.testng.TestRunner.privateRun(TestRunner.java:648)
	at org.testng.TestRunner.run(TestRunner.java:505)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415)
	at org.testng.SuiteRunner.run(SuiteRunner.java:364)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1187)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1116)
	at org.testng.TestNG.runSuites(TestNG.java:1028)
	at org.testng.TestNG.run(TestNG.java:996)
	at com.intellij.rt.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:66)
	at com.intellij.rt.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:105)

Just something to keep in mind.

ConcurrentModificationException and Streams - what not to do

I was wondering what shenanigans I can get up to with streams and mutating the underlying collection, and what a terrible idea this is.

As we all know, it's a really bad idea to mutate a list on which you are iterating, and Java makes certain this fails with a ConcurrentModificationException.

It all starts with the original problem, a for-loop that also mutates the list on which is iterates:

  @Test(expectedExceptions = ConcurrentModificationException.class)
  public void simpleTest() {
    List<Long> longs = new ArrayList<>(List.of(12L, 13L, 16L, 20L, 55L, -5L, 12L, 5L, 100L, 1000L));
    for (Long aLong : longs) {
      if (aLong > 50L || aLong < 0L) {
        longs.remove(aLong);
      }
    }
    assertThat(longs).isEqualTo(List.of(12L, 13L, 16L, 20L, 12L, 5L));
  }

Nowadays, we can use the forEach method on a collection, turning it into something like this with exactly the same problem:

  @Test(expectedExceptions = ConcurrentModificationException.class)
  public void forEachTest() {
    List<Long> longs = new ArrayList<>(List.of(12L, 13L, 16L, 20L, 55L, -5L, 12L, 5L, 100L, 1000L));
    longs.forEach(t -> {
      if (t > 50L || t < 0L) {
        longs.remove(t);
      }
    });
    assertThat(longs).isEqualTo(List.of(12L, 13L, 16L, 20L, 12L, 5L));
  }

If you are using a stream, the same problem occurs, as the stream refers to the underlying collection:

  @Test(expectedExceptions = ConcurrentModificationException.class)
  public void streamTest() {
    List<Long> longs = new ArrayList<>(List.of(12L, 13L, 16L, 20L, 55L, -5L, 12L, 5L, 100L, 1000L));
    longs.stream()
        .filter(Objects::nonNull)
        .filter(t -> t > 50L || t < 0L)
        .forEach(longs::remove);
    assertThat(longs).isEqualTo(List.of(12L, 13L, 16L, 20L, 12L, 5L));
  }

One way, though admittedly not a great way, to fix this, is to transfer it to a list first. Like so:

  @Test
  public void streamWithToListTest() {
    List<Long> longs = new ArrayList<>(List.of(12L, 13L, 16L, 20L, 55L, -5L, 12L, 5L, 100L, 1000L));
    longs.stream()
        .filter(t -> t > 50L || t < 0L)
        .collect(Collectors.toList())
        .forEach(longs::remove);
    assertThat(longs).isEqualTo(List.of(12L, 13L, 16L, 20L, 12L, 5L));
  }

But my colleague at work mentioned the new removeIf function, which seems to work particularly well:

  @Test
  public void removeIfTest() {
    List<Long> longs = new ArrayList<>(List.of(12L, 13L, 16L, 20L, 55L, -5L, 12L, 5L, 100L, 1000L));
    longs.removeIf(t -> t > 50L || t < 0L);
    assertThat(longs).isEqualTo(List.of(12L, 13L, 16L, 20L, 12L, 5L));
  }

Of course, the whole thing is patently ridiculous, because you might as well just filter unwanted elements out of the stream and create a new list from it.

But in some cases, where the "remove" is not as simple as chucking an element out of a collection (for example something as involved as, oh, I don't know, ImportantRestService.removeItem(Item item)), one has to find other ways to deal with it, which is where the second-to-last solution might come in handy.

I don't know. I'm open to suggestions.

Functional Programming - Why Bother?

So I have been discussing functional programming with a colleague at work, specifically concerning streams and collections, and he mentions that most of the times he finds a for-loop easier to read than the whole stream().filter().etc.

Of course, this opinion is very much a subjective point of view and depends on your experience with software design, I think.

So I started thinking where the functional programming style with collections and streams offer advantages.

Inversion of Control

Let us say you need to do something with (possibly) all the items in a list.

The iterative style would be to tell the list to give us (one by one) all its items (in a forloop), that we may do something with them.

The declarative style is to tell the list to do something with its items and give us the result.

This is the inversion of control, the execution of the task switches to another party (the list).

If you think about it, it's quite silly to ask a list to provide us with all its items. The list contains all the items already, it is better if we let the list do the task, to keep execution at the place where it makes the most sense, where the items actually are.

After all, the list is the one who probably knows the most efficient way to wrangle with its contents.

Example

The following example shows both a foreach-loop and a stream implementation.

First a little setup code:

The foreach loop would look as follows:

The stream version would look as follows:

Example

An interesting example can be seen in reference [1], where a stream() actually talks to a database. This way the List or Stream is responsible for proper talking to the database, and we do not get bothered with SQL statements.

If you think about it, then a stream().filter().filter().filter().sort(x).findFirst() can actually be mapped to an SQL statement (select * from x where filter1 and filter2 and filter3 order by x limit 1)

Conclusions

So one of the interesting conclusions we can draw is that instead of imperative programming (telling the computer how to do it), we can switch to declarative programming (telling the computer what to do, and letting the computer figure it out).

And it kind of struck a spark with me.

Things work similarly in real life. You can ask your husband to do the grocery shopping, and you provide him a shoppinglist.

But most of the time, we also add expected "behaviour" (functions). Like, buy the eggs at the greengrocer and not in the supermarket, also get some gas, and go to the superstore first, as it closes the soonest. That sort of thing.

And I see this happening more and more.

If I were to draw this to its inevitable conclusion, I guess we'll end up via Machine Learning all the way to advanced AI, where we tell computers what to do, instead of having to specify HOW to do it.

Naturally, this is going to blow up in our face a couple of times. For example, when the AI decides to do something in an illogical way, but that just means there's an unfortunate gap in the training model.

And it makes me wonder, that, in the future my programming skills will be used, no longer to tell the computer how to do things, no longer to tell the computer what to do, but telling the computer when it does something, what I expect the outcome to be (i.e. writing/changing training models.)

Which is not a problem for me. The way I see it, my profession will not disappear, it will simply change like it has in the past and will in the future.

It's up to me to change with it.

References

[1] Express JPA Queries as Java streams

https://piotrminkowski.com/2021/07/13/express-jpa-queries-as-java-streams/