pilo

CQRS with Spring

Tue, 16 Oct 2018 00:00:00 +0000

Hi folks. The New Stack just published my article about CQRS. It is a full story about how, when and why to apply this pattern with various Spring tools. You can find it here. And I did the drawings myself!

How To Test Your Distributed Message-Driven Application With Spring?

Sat, 05 May 2018 00:00:00 +0000

Messaging is cool. It allows to deal with a broad set of various problems that developers face on a daily basis. It is not a silver bullet though. One must assess if their architectural drivers fit this pattern. A list of tips that might help you with this decision:

Communication between some components needs to be asynchronous - producer fires and forgets.
Consumer should process the messages at its own pace - independent from producer’s pace. This is so-called throttling.
Communication should be reliable - message broker can store and forward messages. Thus, producer does not need to care about retrying.
We deal with occasionally connected devices. Those are component that tend to be offline or down but need to synchronize as soon as the reconnect. Disconnection must not affect the producer. The need for two components to be alive at the same time is called temporal coupling.
There is a need for multicast/broadcast.
One of our components uses event sourcing. You can learn about that concept listening to this talk.
There is already a production setup of a message broker in your infrastructure.

Note that those are mosty tips at distributed level. One can think about benefits of having messaging in monoliths architectures too. Like Inversion of Control - a way to continue a business process in a different part of same application. Or concurrency control implemented in actor model. An actor has its internal queue of messages processed one at a time. In this article we will read about testing message-driven systems in distributed syastems. Systems which use messaging as transport mechanism (first five tips) and event sourcing.

Let’s imagine an enterprise that issues credit cards under one condition - a potential client’s age. There is a REST API to do so:

@RestController("/applications")
class CreditCardApplicationController {

    private final ApplyForCardService applyForCardService;

    CreditCardApplicationController(ApplyForCardService applyForCardService) {
        this.applyForCardService = applyForCardService;
    }

    @PostMapping
    ResponseEntity applyForCard(@RequestBody CardApplication cardApplication) {
        return applyForCardService
            .apply(cardApplication.getPesel())
            .map(card -> ok().build())
            .orElse(status(FORBIDDEN).build());
    }
}

And an application service responsible for the decision:

@Service
class ApplyForCardService {

    private final CreditCardRepository creditCardRepository;

    ApplyForCardService(CreditCardRepository creditCardRepository) {
        this.creditCardRepository = creditCardRepository;
    }

    @Transactional
    public Optional<CreditCard> apply(int age) {
        if (bornBeforeSeventies(age)) {
            return Optional.empty();
        }
        CreditCard card = CreditCard.withDefaultLimit(age);
        creditCardRepository.save(card);
        return of(card);
    }
}

Imagine a new business need. System must react to a certain decision. If a card is granted, system must deal with that fact in a particular manner. If a card application is rejected, it must do something else. Whatever it is. It might be an email, an update in a reporting tool or anything else. But this process continues in a different application in the distributed environment. Plus this application can be temporary offline. Of course, it should not affect the granting/rejecting process. Did I mention that there is a huge probability that more applications would like to know final result of the process?

By now, it should be clear that messaging might be a good fit for this problem. We will add one line of coded escribing what has happened. A message saying that either the card was granted or the application was rejected. Nothing more. A special kind of a message that represent a fact in a past is called an event. We need a component that will be responsible for publishing those events. Its responsibility is clear, but let’s postpone the implementation. Additionally, there might be several ones, so let’s add an interface for the time being:

interface DomainEventPublisher {
    void publish(DomainEvent event);
}

interface DomainEvent {
    String getType();
}

DomainEvent interface contains only an information about the event’s type. This might be handy while distributing or subscribing to specific messages.

After you, dear test

Any production code that is not proved by a test is just a rumour, so here goes an unit test:

def 'should emit CardGranted when client born in 70s or later'() {
    when:
        applyForCardService.apply("89121514667")
    then:
        1 * domainEventsPublisher.publish( { it instanceof CardGranted } )
}

def 'should emit CardApplicationRejected when client born before 70s'() {
    when:
        applyForCardService.apply("66121514667")
    then:
        1 * domainEventsPublisher.publish( { it instanceof CardApplicationRejected } )
}

And in order for the test to pass:

@Transactional
public Optional<CreditCard> apply(String pesel) {
    if (bornBeforeSeventies(pesel)) {
        domainEventsPublisher.publish(new CardApplicationRejected(pesel));
        return Optional.empty();
    }
    CreditCard card = CreditCard.withDefaultLimit(pesel);
    creditCardRepository.save(card);
    domainEventsPublisher.publish(new CardGranted(card.getCardNo(), card.getCardLimit(), pesel));
    return of(card);
}

Voilà. Let’s go to production. All tests are green and that means we are good to go. But ... the application does not start on my local machine. There is no spring bean that implements the DomainEventPublisher interface. The unit test might be useful to drive a good design. Especially if you are a mockist TDD practitioner. But it is clearly not enough. Communicating with an external infrastructure from a business code is rather an integration task, isn’t it? So how about an integration test? But do we want to have a dependency on a message broker in our tests? Does it need to run to build the application? Or should each of the developers have their own local instance of the broker? The answer to every of those questions is no. Tests should run in isolation. Let’s finally implement the interface. And let's test the whole process with an integration test.

We can use a wonderful tool - Spring Cloud Stream - to implement the messaging part. In short, this is a framework that abstracts messaging paths with so-called channels. Channels are bound to a specific broker’s destinations by classpath scanning. The tools looks for binders to Kafka or RabbitMQ. Let’s choose RabbitMQ. Our credit card application is a producer to one channel. That means it is a source for messages. Simillary, the consumer is a sink. We need to remove the dependency to a real broker in tests. To do so, we are going to replace the production implementation with @MockBean.

The integration test (in jUnit, because it is easier to use @MockBean in jUnit):

@MockBean DomainEventsPublisher domainEventsPublisher;

@Autowired CreditCardApplicationController cardApplicationController;

@Test
public void shouldEmitCardGrantedEvent() {
    // when
    cardApplicationController.applyForCard(new CardApplication("70345678"));

    // then
    verify(domainEventsPublisher).publish(isA(CardGranted.class));
}

@Test
public void shouldEmitCardApplicationRejectedEvent() {
    // when
    cardApplicationController.applyForCard(new CardApplication("60345678"));

    // then
    verify(domainEventsPublisher).publish(isA(CardApplicationRejected.class));
}

Implementation:

class RabbitMqDomainEventPublisher implements DomainEventsPublisher {

    private final Source source;

    RabbitMqDomainEventPublisher(Source source) {
        this.source = source;
    }

    @Override
        public void publish(DomainEvent domainEvent) {
        Map<String, Object> headers = new HashMap<>();
        headers.put("type", domainEvent.getType());
        source.output().send(new GenericMessage<>(domainEvent, headers));
    }
}

And listener at consumer’s side:

@Component
class Listener {
    private static final Logger log = LoggerFactory.getLogger(Listener.class);

    @StreamListener(target = Sink.INPUT, condition = "headers['type'] == 'card-granted'")
        public void receiveGranted(@Payload CardGranted granted) {
        log.info("\n\nGRANTED [" + granted.getClientPesel() + "] !!!! :) :) :)\n\n");
    }

    @StreamListener(target = Sink.INPUT, condition = "headers['type'] == 'card-application-rejected'")
        public void receiveRejected(@Payload CardApplicationRejected rejected) {
        log.info("\n\nREJECTED [" + rejected.getClientPesel() + "] !!!! :( :( :(\n\n");
    }
}

Note the use of headers - this allows us to distribute messages to specific methods at the consumer’s side.

This time we are definitely good to deploy to production. We have both unit and integration tests. Let’s run the application on our local machine for the last time. ...and it failed for the same reason. No bean that implements the DomainEventPublisher interface. We forgot to register the implementation as a spring bean. But wait a minute, how come the integration test passes? Take a closer look at the @MockBean docs:

“Mocks can be registered by type or by bean name. Any existing single bean of the same type defined in the context will be replaced by the mock. If no existing bean is defined a new one will be added. “

Unit test vs Integration test

So the tool added the bean to the application context. Even though its real implementation was not there. And this is by design. We should have been more careful. Let's notice some facts about our new integration test:

It setups the whole context.
It is significantly longer than the unit test.
It even uses an embedded H2 database by default.
… but it still tests our messaging in the mockist’s way!
… but it still can be green without the actual implementation (just like the unit test)
… but still the communication with broker is not tested.

The architectural diagram is worth more than a thousand words so let’s take a look at two of those:

The left one shows what was tested by the unit test. Everything else is blurred. And the second one highlights what was tested by the integration test.

In both cases, the actual messaging (DomainEventPublisher) was not tested. Moreover, the integration test examines more not that essential components. Not essential, taking into account that DomainEventPublisher is still left untested. There must be a better way. Let’s take a deeper look at Spring Cloud Stream’s documentation. In a paragraph about testing we can find an interesting note:

“... we are using the MessageCollector provided by Spring Cloud Stream’s test support to capture the message has been sent to the output channel as a result. Once we have received the message, we can validate that the component functions correctly.”

Spring Cloud Stream Testing Support

So here it is! Testing support and in particular MessageCollector can help us test our code. Without a dependency to a real broker. Instead, the messages can be held in an internal in-memory blocking queue. Here goes the test:

@SpringBootTest
class ApplyForCardWithEventMessageCollectorTest extends Specification {

    @Autowired CreditCardApplicationController cardApplicationController
    @Autowired Source source
    @Autowired MessageCollector messageCollector

    BlockingQueue<GenericMessage<String>> events

    def setup() {
        events = messageCollector.forChannel(source.output())
    }

    def 'should be able to get card when born in 70s or later'() {
    when:
        cardApplicationController.applyForCard(new CardApplication(20))
    then:
        events.poll().getHeaders().containsValue("card-granted")
    }

    def 'should  not be able to get card when born before 70s'() {
    when:
        cardApplicationController.applyForCard(new CardApplication(90))
    then:
        events.poll().getHeaders().containsValue("card-application-rejected")
    }
}

That means we can finally use the same bean as in production code. We finally test the whole producer's side:

Our application is starting on our local machine! We can finally deploy to production. Just before doing so, let’s also start a consumer application and a docker image of RabbitMQ. Let’s trigger some messages and see them received at the consumer’s side:

It worked! Or did it? Wait a minute. The listener is able to get the message, but its content is empty! Why there is null value? What could have gone wrong? The complete test was created. Plus, it examines exactly the same path as the production code. But there is one thing that we need to know about it. MessageCollector captures the message in an internal queue before actual serialization. This serialization happens in the production code. It happens while sending a message to a channel! That leads us to a point that there must be something wrong with our serialization. The channel is configured to talk in JSON, so probably we need to help Jackson a bit. Let’s have a quick look at the classes that represent events:

public class CardApplicationRejected implements DomainEvent {

    private final String clientPesel;
    private final Instant timestamp = Instant.now();

    public CardApplicationRejected(String clientPesel) {
        this.clientPesel = clientPesel;
    }

    @Override public String getType() {
        return "card-application-rejected";
    }
}

A careful reader will quickly spot the problem. The getters are not there and Jackson by default will not serialize any field without a getter. Let’s fix that and rerun:

Success. It finally worked. The producer and consumer can communicate without issues. But does it mean there is no way of testing it without doing the manual check? Does it mean that each time we develop a message-driven system, we must set it up on a local machine? And manually prove its correctness? Of course not.

Spring Cloud Contract

To automatically test the whole process we would have to bring up the consumer side as a dependency. Plus a message broker. We have already said that this is not the best idea since we want to run tests in isolation. They should not be dependent on a presence of another component. Fortunately, there is a tool called Spring Cloud Contract. With this framework we can check if microservices will communicate on production environment. And all that without having to create a test that brings all of them up. Plus, it works with both messaging and REST APIs. Sounds like a perfect fit.

The producer declares a contract. It says that if there is a rejection of a card application, some message should be sent to a specific channel. The contract defines the body of that message:

label: card_rejected
input:
  triggeredBy: sendRejected()
outputMessage:
  sentTo: channel
  headers:
    type: card-application-rejected
    contentType: application/json
  body:
    clientPesel: 86010156812
    timestamp: 2018-01-01T12:12:12.000Z
  matchers:
    body:
    - path: $.timestamp
      type: by_regex
      predefined: iso_8601_with_offset

The contract is packaged as a maven artifact. Based on that contract, a test is automatically generated. Its purpose is to check if the producer adheres to what is in the contract. It looks like this:

@Test
public void validate_shouldSendACardRejectedEvent() throws Exception {
    // when:
    sendRejected();

    // then:
    ContractVerifierMessage response = contractVerifierMessaging.receive("channel");
    assertThat(response).isNotNull();
    assertThat(response.getHeader("type")).isNotNull();
    assertThat(response.getHeader("type").toString()).isEqualTo("card-application-rejected");
    assertThat(response.getHeader("contentType")).isNotNull();
    assertThat(response.getHeader("contentType").toString()).isEqualTo("application/json");
    // and:
    DocumentContext parsedJson = JsonPath.parse(contractVerifierObjectMapper.writeValueAsString(response.getPayload()));
    assertThatJson(parsedJson).field("['clientPesel']").isEqualTo(86010156812L);
    // and:
    assertThat(parsedJson.read("$.timestamp", String.class)).matches("([0-9]{4})-(1[0-2]|0[1-9])-(3[01]|0[1-9]|[12][0-9])T(2[0-3]|[01][0-9]):([0-5][0-9]):([0-5][0-9])(\\.\\d{3})?(Z|[+-][01]\\d:[0-5]\\d)");
}

Note that the test examines the message body. So in our previous example, it would have spotted the problem with the lack of any getter. Plus, this test is fired each time we would build the producer application. Each version of a producer comes with a corresponding version of stubs defined in a contract. How cool is that?

Transaction semantics

Have you noticed any problem with the code that grants/rejects credit cards? Something about events, maybe? Let’s have a second look:

@Transactional
public Optional<CreditCard> apply(String pesel) {
    // (...)
    creditCardRepository.save(card);
    domainEventsPublisher
        .publish(new CardGranted(card.getCardNo(), card.getCardLimit(), pesel));
    return of(card);
}

How to atomically save a newly created card to our database and send a message to our message broker? Remember that actual commit to the database takes place after returning from the body of this method. The proxy does that. Two-phase-commit is not an option because we don’t want to downgrade the performance. Let’s examine following scenarios:

Sending to the broker failed. Database transaction rollbacks and everything is fine. Neither credit card nor message was saved.
Sending to the broker was successful. Spring proxy tries to commit and database is down. The message was sent while the card was not saved.

One can come up with an idea of reordering the steps. First, let’s make sure that the card was saved. Only then we are entitled to announce that via event. This scenario can be implemented using a callback registered with TransactionSynchronizationManager. But what if the broker is not online? Or our application was killed between successful commit and callback execution? . The system will try to send that only once, it is crucial to understand that this is at-most-once guaranty.

What if we must send this message eventually? What if the requirement says that it must appear if a card was successfully saved? We can take advantage of the fact that most probably there is an ACID database in our toolbox. Let’s store both in one storage, using one transaction! A separate thread can take a look at all events saved in that storage and fetch ones not yet sent to the broker. Then send them and mark as sent. All in a couple of lines in new implementation of DomainEventPublisher:

@Component
public class FromDBDomainEventPublisher implements DomainEventsPublisher {

    private final DomainEventStorage domainEventStorage;
    private final ObjectMapper objectMapper;
    private final Source source;

    public FromDBDomainEventPublisher(DomainEventStorage domainEventStorage,
                                        ObjectMapper objectMapper,
                                        Source source) {
        this.domainEventStorage = domainEventStorage;
        this.objectMapper = objectMapper;
        this.source = source;
    }

    @Override
    public void publish(DomainEvent domainEvent) {
        try {
            domainEventStorage
            .save(new StoredDomainEvent(objectMapper.writeValueAsString(domainEvent)));
        } catch (JsonProcessingException e) {
            throw new RuntimeException(e);
        }
    }

    @Scheduled(fixedRate = 2000)
    @Transactional
    public void publishExternally() {
        domainEventStorage
        .findAllBySentOrderByTimestampDesc(false)
        .forEach(event -> {
                    source.output().send(new GenericMessage<>(event.getContent()));
                    event.sent();
                    });
    }
}

Notice that the method which periodically publishes new events suffers from the same problem of lack of an atomic transaction. It tries to connect to the broker and to save state. But this time it is after successful card creation. And it has retries. If the broker is down, we will retry. After successful communication with the broker, the database can be offline. Hence, we will not mark the event as processed. We will resend it, implementing what is called at-least-once guaranty. Whether you should go with at-least-once or at-most-once depends on the business problem. For marketing purposes probably at-most-once is fine. But if a message carries a significant information (like how to activate the card), then probably the clients want that message. And will not mind getting it twice. At least they will be less angry than when not receiving it at all. And if your consumer is idempotent, then there is no problem at all.

Store only one thing

How we can deal with cooridnation of two different components? With database and with message broker? Can we not send to one of them? Is state or message redundant? Can one of them be derived from another? If we think of messages as events or as changes that affect state, then it becomes clear that state can be derived from events. Thus, only the communication with the broker would be needed. The state can be queried from a log of changes represented by events. This is what event sourcing is about. You can read more about reliable events delivery here.

The code used in this example is here.

Event Storming with Spring and a Splash of DDD

Wed, 11 Apr 2018 00:00:00 +0000

I wrote a post on the spring blog. It is about how to tackle complex enterprises with Event Storming, Spring and DDD. You can find it here

On Eventual Consistency and REST

Mon, 16 Jan 2017 00:00:00 +0000

Typically in event-sourced systems (with Command Query Responsibility Segregation) that need to display data to a client, we have three components that must co-operate

write model that accepts commands and writes events to its event store
read model that accepts events and returns DTOs to a client
client (e.g. web browsers) that writes commands to write model and queries read model for DTOs

This actually leads us to a situation where we have 3 well defined boundaries with well defined contracts. That means we can easily distribute work among 3 teams. Write model team need to have deep knowledge in the domain, but do not need to touch UI and database (!). Read model team need to perform simple database updates and provide an API for front-end. Front-end in turn must do as usual - request (to write model) and queries (to read model).

Events from the write model can be delivered to subscribers (here, read models) synchronously. That means that we ACK a command if and only if both write model and read model were successfully updated. If one of them fails, client must retry the request. This actually favors Consistency vs. Availability and tangles read model and write model to be backed by the same data source. This solution does not allow us to scale our read models independently from our write model. Another option is to use distributed transactions, which reduce performance. To cut the long story short, synchronous mode reduces flexibility, scalability and autonomy - things that ES/CQRS gave us in the first place.

Having said that, we often prefer asynchronous mode and acknowledging a command as soon as events were produced and saved in the write model. This is also in-line with our natural notion of imperative tense and past tense:

1. "DoSomething! (command)"
2. "OK, SomethingDone (ACK with event)" or "Nope, piss off!" (N-ACK)

In this mode the information that something was done (event) is delivered to our read model slightly later by the means of read model subscription. The subscription can be fulfilled by the means of a message broker or by pulling the write model by read model for new changes. The way of delivery does not really matter. What matters is that we now favor Availability with the cost of losing Strong Consistency. Our read model will be actually Eventually Consistent - if no further updates are done to the write model, it will return the last updated data. Otherwise, it can return data that is a little stale.

Is that a problem?

Let's consider a read model that is being used by a reporting team to feed their Excel sheets and send e-mails. By the time they update their excel sheet or return from a coffee to check e-mail or go to a printer to fetch the printed version, received data can be already stale. Even though a read model is strongly consistent with the write model, it does not prevent end users from USING this data in eventually consistent way.

Imagine how companies used to work back then without computers. It took time to deliver information on a piece of paper from one desk to another. Everything was eventually consistent. Than we (developers) came and introduced strong consistency and ACID. That is why business people now think of "eventually consistent" data as "incorrect" data. Even when they use their strongly consistent data to print reports and analyze them at home few hours later. Banks worked perfectly fine without computers back then. Moreover, now bank systems are eventually consistent. Even in critical parts of the data, like your account balance. Having said that, eventual consistency is almost always fine. All you need to do is slightly educate users of your system.

But…Read your own Writes

There are some guaranties you would like to ensure. Imagine situation when user issues a command "ChangeHomeAddress" and refreshes the browser. If he is unlucky, new address is not there and he feels like the system is broken. He will probably retry his request. It is HIS update, so he cares about it. He does not care if updates that are consequences of other users actions arrive 1 sec later or earlier. But he would like to see EFFECTS of HIS actions. That guaranty is called "Read your own writes".

How to ensure "Read your own writes" when the read model is asynchronously updated? First let's talk about how events are stored in the write model. Consider we have a ShopItem with UUID. ShopItem produces events in response to a command (e.g. Pay command and ItemPaid event). Produced events are stored in event store (write model). Each event has UUID of a ShopItem, occurred_at date and sequence number. Events related to ShopItem with particular UUID are grouped into so-called Event Steams. Event Stream is responsible for tracking events from one instance of an aggregate. An aggregate in Domain Driven Design terms is a cluster of objects that create a transactional boundary. Aggregate is stored as a series of events in events stream, that means that event stream is our transactional boundary - only one person can write to event-stream at a time (because only one person can modify an aggregate at a time). That means that we can have a version counter for optimistic locking for each of the streams. Each write to an event-stream bumps up version counter or raises optimistic locking exception. That means that when we have 3 events in event stream (with sequence numbers 1, 2 and 3) the version counter of corresponding event stream is 3. In other words, version counter is denormalized value from max sequence number in given event stream.

Let's now go back to our three components: client, write model and read model. We previously said that in asynchronous mode, a command is accepted when events are persisted in write model. That means that when write model returns and ACK to a client, it knows what event it had just written. In particular, he knows what is a version counter (sequence number of last event) of domain item modified by the command. He can return this information to a client, so that client will know what version of domain item was created due to his change. Now the client can include this information when querying read model. He can specify that he EXPECTS to get data at a version equal or greater to the one returned with ACK. Why equal or greater? Because someone could have written a new value in between requests to write and read model. The server can then do two things. It can either return the data or fail because it has not received expected event yet. The important is that the server knows that the value will be delivered just soon. That is a powerful information, because the server can return Retry-After X seconds HTTP header. Client's browser will (or should) retry the request after X seconds, so that the initial failure is transparent to the client. To cut it short:

Issue a command to write model
Read current version from ACK
Put expected version to request to read model
Expect data or expect browser to retry soon transparently (because of Retry-After header set by read model server)

Let's see how read model server can implement that logic. First we have to create an endpoint for querying a particular shop item by UUID. We will use Spring Boot.

    @RequestMapping(method = RequestMethod.GET, value = "/{uuid}")
    public ResponseEntity<ShopItem> readById(
			@PathVariable String uuid, 
	                @RequestHeader(value="Expect") Integer expectedVersion) {
       //..
    }

But how do we pass expected version in the request to the read model? One option is to put it as request parameter and query under URL /uuid?expectedVersion=2, when we expect version number equal of greater than 2. The other option is to use HTTP header. Which one? There actually is one called EXPECT. According to <a href=https://tools.ietf.org/html/rfc7231#section-5.1.1">RFC7231</a> it is used to:

The "Expect" header field in a request indicates a certain set of behaviors (expectations) that need to be supported by the server in order to properly handle this request.

To align more with HTTP, let's put expected version under EXPECT parameter instead of request parameter. We do test first. When expected version is greater than current version we expect that server says Retry-After 2 seconds.

 def 'should ask to retry after when state is not yet present'() {
        given:
            UUID itemUUID = UUID.randomUUID()
            itemIsOrderedAtVersion(itemUUID, ANY_TIME, currentVersion)
        when:
            ResultActions result = mockMvc
					.perform(get("/${itemUUID}")
					.header(EXPECT, expectedVersion))
        then:
            result.andExpect(header().longValue("Retry-After", 2))
            result.andExpect(status().isServiceUnavailable())
        where:
            currentVersion || expectedVersion
            1              || 2
            2              || 3
            4              || 10
            100            || 201
    }

The second scenario ensures that when expected version is smaller or equal to current version, we return the data.

 def 'should return the data when expected version is equal or smaller than current version'() {
        given:
            UUID itemUUID = UUID.randomUUID()
            itemIsOrderedAtVersion(itemUUID, ANY_TIME, currentVersion)
        when:
            ResultActions result = mockMvc
					.perform(get("/${itemUUID}")
					.header(EXPECT, expectedVersion))
        then:
            result.andExpect(status().isOk())
        where:
            currentVersion || expectedVersion
            2              || 2
            3              || 3
            40             || 10
            1000           || 201
    }

Now let's take a look at the endpoint implementation. The client can leave expected version as empty. Let's assume that when it is not defined then the data is as good as it gets - the recent one.

    @RequestMapping(method = RequestMethod.GET, value = "/{uuid}")
    public ResponseEntity<ShopItem> readById(
			@PathVariable String uuid, 
			@RequestHeader(value=HttpHeaders.EXPECT) Integer expectedVersion) {
        final ShopItem item = jdbcReadModel.getItemBy(UUID.fromString(uuid));
        if (dataAtExpectedState(item, expectedVersion)) {
            return ResponseEntity.ok(item);
        }
        return retrySoon();

    }

    private boolean dataAtExpectedState(ShopItem item, Integer expectedVersion) {
        return expectedVersion == null || expectedVersion <= item.getVersion_value();
    }

    private ResponseEntity<ShopItem> retrySoon() {
        final HttpHeaders headers = new HttpHeaders();
        headers.set(HttpHeaders.RETRY_AFTER, Integer.toString(2));
        return new ResponseEntity<>(headers, HttpStatus.SERVICE_UNAVAILABLE);
    }

The last scenario ensures that after retrying client gets expected data. Of course, if read model was updated in between two requests.

 def 'should return the data after retrying'() {
        given:
            UUID itemUUID = UUID.randomUUID()
            itemIsOrderedAtVersion(itemUUID, ANY_TIME, 1)
        when:
            ResultActions result = mockMvc
					.perform(get("/${itemUUID}")
					.header("Expect", 2))
        then:
            result.andExpect(header().longValue("Retry-After", 2))
            result.andExpect(status().isServiceUnavailable())
        when:
            itemIsPaidAtVersion(itemUUID, ANY_TIME, 2)
        and:
            ResultActions secondResult = mockMvc
					.perform(get("/${itemUUID}")
					.header("Expect", 2))
        then:
            secondResult.andExpect(status().isOk())
    }

This was an example of using Http Headers (Expect and Retry-After) to implement transparent retries and ensure "read your own writes" in eventually consistent system.

The other option is to inform the user that we are only able to return data from a concrete point of time (our last known update). The read model can keep track of when a concrete event stream was recently updated. On the screen, next to our data there can be a time-stamp which says how stale it is. This is actually pretty easy to introduce by the means of <a href="https://tools.ietf.org/html/rfc7232#section-2.2"Last-Modified</a> HTTP Header. Take a look at the test:

  def 'should return last modified date in response'() {
        given:
            UUID itemUUID = UUID.randomUUID()
            itemIsOrderedAtVersion(itemUUID, ANY_TIME, 1)
        when:
            ResultActions result = mockMvc
					.perform(get("/${itemUUID}")
					.header("Expect", 1))
        then:
            result.andExpect(
			header()
				.string(
					LAST_MODIFIED, 
					HTTP_DATE_FORMAT.format(from(ANY_TIME))))
            result.andExpect(status().isOk())

    }

And we had to modify previous implementation, so that it returns new header:

   @RequestMapping(method = RequestMethod.GET, value = "/{uuid}")
    public ResponseEntity<ShopItem> readById(
			@PathVariable String uuid, 
			@RequestHeader(value= EXPECT) Integer expectedVersion) {
        final ShopItem item = jdbcReadModel.getItemBy(UUID.fromString(uuid));
        if (dataAtExpectedState(item, expectedVersion)) {
            return okWithLastModifiedDate(item);
        }
        return retrySoon();

    }

    private ResponseEntity<ShopItem> okWithLastModifiedDate(ShopItem item) {
        final HttpHeaders headers = new HttpHeaders();
        headers.set(
		HttpHeaders.LAST_MODIFIED, 
		HTTP_DATE_FORMAT.format(item.getLastModifiedDate()));
        return new ResponseEntity<>(item, headers, HttpStatus.OK);
    }

With this behavior, we can notify our client that the data he is seeing might be a little stale, so he might refresh the browser. Be careful here. A common mistake in a read model is to save last updated date as a process date (arrival date). Our real model can be down for 2 days and when it gets up again it will process events that happened 2 days ago. That would in turns display wrong information to a client - the date is actually 2 days older. We always have to take into account the date of the creation of an event, which is passed to the read model.

The last option is tricking the user directly in the client UI. As soon as we got the ACK, we know the data will be eventually in the read model. So we might directly modify our temporal view in web browser and hope that user will not refresh the page until expected version arrived in read model. This is rather rare way of dealing with eventual consistency.

Couple of notes

How we return current version in ACK from write model? Most often custom return type is introduced - it contains status code and the current version. Another way is to use the E-tag HTTP header, which <a href=https://tools.ietf.org/html/rfc7232#section-2.3.1> according to specification</a> can be used for:

"The "E-Tag" header field in a response provides the current entity-tag for the selected representation, as determined at the conclusion of handling the request. An entity-tag is an opaque validator for differentiating between multiple representations of the same resource, regardless of whether those multiple representations are due to resource state changes over time, content negotiation resulting in multiple representations being valid at the same time, or both."

As it is described, the value should be opaque, so we should encrypt our version parameter. We don't want to reveal how big our event-stream is and how it is build due to the security reasons. The same should be actually done when we query read model and specify expected version.

RFC also says that we can specify concrete time in Retry-After header. We should avoid that, because this can result in very high traffic at this specific time. Imagine we inform many of our clients to retry at a specific time. Although the clocks of the clients are not synchronized, the traffic around that time can be significant.

Sadly although both Mozilla and Chrome ensure supporting Retry-After header, it does not work in the newest versions. The browsers do not care that much, because the servers rarely send that header. It creates an endless circle. But remember, the client is not always a browser. So it is good to have Retry-After header.

Code can be found here

My talk about Spring Stream, Event Sourcing and CQRS at spring.io

Sat, 12 Nov 2016 00:00:00 +0000

Last month I did a few talks about leveraging Spring Cloud Stream to create fault tolerant and loosely coupled services that use Event Sourcing and Command Query Responsibility Segregation. Unfortuneatly, none of those was recorded due to technical reasons. But, I also had a great pleasure to had a Google Hangout about those subjects with guys from the Spring team. They liked it up to the point to putting the video online, at their blog page. So if you are interested and have fifty minutes, please leave a comment.

In the next month (hopefully), I will be creating a series of blog posts about techniques I've mentioned in that video. We will also see an example of Process Manager implemented as Akka actor. Stay tuned!

Event sourcing with CQRS sample revisited

Sun, 14 Aug 2016 00:00:00 +0000

Previously I posted a short note about my sample event sourced application that follows Command Query Responsibility Segregation principle. Today I would like to go a little bit further and describe its particular components and their responsibilities.

Overall structure

There are 4 sub packages:

domain - which is the core of the business. In so called onion architecture it stays in the middle, it does not depend on any other package. It does not even depend on spring, since no container is needed there. Hence, there are only unit tests provided. It exposes one interface to get ShopItem, this interface is called ShopItemRepository. Its presence let the application to inverse the dependencies - every implementation of ShopItemRepository points towards domain
boundary - entry point to the application. It contains only one service, called ShopItems. It receives commands exposed by domain and invokes them on proper ShopItem instance.. This service could be represented as a REST controller (although it does not handle REST, but a few more annotations could change that)
eventstore - it exposes one public class - EventSourcedShopItemRepository, which implements ShopItemRepository exposed by domain. Hence, it can be treated as plug-in shipped with application. Currently, it uses relational database to store events, but probably document database would be more suitable choice, since events are serialized to JSON by EventSerializer. Every single stored event is also published to any other interested party with the help of ApplicationEventPublisher
readmodel - listens to events published by mentioned ApplicationEventPublisher and mutates denormalized shop item model represented by ShopItemDto.

Package: domain

Application simulates a shop with various items. Since shopping domain is much more complicated, it probably would not be modeled like this in a real software. Anyway, a shop item can be in 4 states: INITIALIZED, BOUGHT, PAID, PAYMENT_MISSING. Changing a state means emitting an event. Let's take a look at state diagram.

The most important mind shift is how the shop item aggregate is modeled. It is not a traditional entity, but a simple POJO. Hence, no external dependencies are needed. Note that ShopItem is immutable, which favors functional programming and gives a lot of benefits. Every business method returns new instance of ShopItem.

Secondly, it is worth seeing how easy it is to test such an aggregate. We only care about what is being emitted as event, we don't care about internal state representations. Thus, test for paying for shop item looks as follows:

 def 'should emit item paid event when paying for bought item'() {
        when:
            ShopItem tx = bought(uuid).pay(now())
        then:
            tx.getUncommittedChanges().size() == 1
            tx.getUncommittedChanges().head().type() == ItemPaid.TYPE
    }

getUncommitedChanges() returns events created during business method invocation. We also should take care of illegal state transitions, for example marking payment as missing when when someone already has paid should be impossible:

def 'cannot mark payment missing when item already paid'() {
        when:
            paid(uuid).markTimeout(now())
        then:
            thrown(IllegalStateException)
    }

Last, but not least - we want those operations to be idempotent. Idempotency in this context means not emitting the same event twice.

   def 'marking payment timeout should be idempotent'() {
        when:
            ShopItem tx = withTimeout(uuid).markTimeout(now())
        then:
            tx.getUncommittedChanges().isEmpty()
    }

Package: eventstore

Mentioned abovegetUncommitedChanges()method is crucial point for implementing storage with events. Every time we try to save a new change, we call "save" in EventSourceShopItemRepository, which looks as follows:

 public ShopItem save(ShopItem aggregate) {
        final List<DomainEvent> pendingEvents = aggregate.getUncommittedChanges();
        eventStore.saveEvents(
                aggregate.getUuid(),
                pendingEvents
                        .stream()
                        .map(eventSerializer::serialize)
                        .collect(toList()));
        return aggregate.markChangesAsCommitted();
    }

So basically we get all changes emitted by an aggregate, serialize them to JSON, store in database and flush pending events in aggregate. Let's look how event store handles saving an event.

interface EventStore extends JpaRepository<EventStream, Long> {

    Optional<EventStream> findByAggregateUUID(UUID uuid);

    default void saveEvents(UUID aggregateId, List<EventDescriptor> events) {
        final EventStream eventStream = findByAggregateUUID(aggregateId)
                .orElseGet(() -> new EventStream(aggregateId));
        eventStream.addEvents(events);
        save(eventStream);
    }

    default List<EventDescriptor> getEventsForAggregate(UUID aggregateId) {
        return findByAggregateUUID(aggregateId)
                        .map(EventStream::getEvents)
                        .orElse(emptyList());

    }
}

It just looks for EventStream connected with this aggregate UUID and adds new serialized events in a form of EventDescriptor with event serialized to JSON. If there is no stream yet (means we try to store the brand new item, new stream is created). Everything is implemented with help of spring data jpa repository. Concurrent changes done in EventStream (adding new EventDescriptors to it) can be done with optimistic locking.

Having EventStore implemented helped us to create EventSourcedShopItemRepository. It delegates to event store and looks for events connected with given aggregate. Next, it applies them sequentially, creating an aggregate instance. State can be reconstructed to represent aggregate from any given time. This is visible in the following test case.

	 def 'should reconstruct item at given moment'() {
		given:
		    ShopItem stored = initialized()
		            .buy(uuid, TOMORROW, PAYMENT_DEADLINE_IN_HOURS)
		            .pay(DAY_AFTER_TOMORROW)
		when:
		    shopItemRepository.save(stored)
		and:
		    ShopItem bought = shopItemRepository.getByUUIDat(uuid, TOMORROW)
		    ShopItem paid = shopItemRepository.getByUUIDat(uuid, DAY_AFTER_TOMORROW)

		then:
		    bought.state == BOUGHT
		    paid.state == PAID
	    }

Package: readmodel

Take a look how read model is updated. Basically it uses jdbc and direct sql calls to update denormalized database schema. No fany ORM needed here. Read model update is triggered with help of TransactionalEventListener.

	 //...

	 @TransactionalEventListener
	    public void handle(DomainEvent event) {
		if (event instanceof ItemBought) {
		    final ItemBought itemBought = (ItemBought) event;
		    jdbcReadModelUpdater.updateOrCreateItemAsBlocked(event.uuid(), event.when(), itemBought.getPaymentTimeoutDate());
		} else if (event instanceof ItemPaid) {
		    jdbcReadModelUpdater.updateItemAsPaid(event.uuid(), event.when());
		} else if (event instanceof ItemPaymentTimeout) {
		    jdbcReadModelUpdater.updateItemAsPaymentMissing(event.uuid(), event.when());
		} else {
		    throw new IllegalArgumentException("Cannot handle event " + event.getClass());
		}
	    }

Note that we update read model in the same transaction, hence usage of TransactionalEventListener. Thus, the read model is consistent with write model, because they share the same data source. Normally we want our write model to be decoupled from read model, preferably in different data source. Read model would be eventually consistent and would read all the events stored by write model in event store. There is one more interesting thing in readmodel package. Since it knows about all the data and has it in denormalized form, it is able to perform actions. Here, PaymentTimeoutChecker periodically checks whether payment is missing. If that happens, it throws a command to write model. That causes aggregate state change.

Conclusion

With usage of CQRS we have simple and performant read model, that we can query without complex joins. Our read model is decoupled from write model and does not pollute domain model. That means that it is just a projection of domain events and has no impact of how domain is implemented. Having all the events stored in event store gives us huge debugging possibilities. We can reconstruct aggregate state from any given moment. Storing only events results in having simple shema for write model (only 2 tables here), throwing away all the work that db admins need to do when storing complex domain in relational database.

Sample event sourced application

Mon, 27 Jun 2016 00:00:00 +0000

In my last post I wrote about domain events publishing. Events were published at the end of aggregate public methods as a natural consequence of internal state modification. Let's look again at this code sample:

    class User {

        //...

        @Transactional
        public void register(LocalDateTime at, RegisteredBy by) {
            markRegistered();
            markSuspiciousIfAtNight(at);
            DomainEventsPublisher.publish(new UserRegistered(this.id(), at));
        }

    }

The problem that we might face is different source of truth between created UserRegistered and internal changes done by markRegistered and markSuspiciousIfAtNight(at) method invocations. The path to mistake is fairly simple, we could throw this domain event with Instant.now() instead of at parameter. Thus, misleading listeners of domain events, because internal state was modified differently.

If we were storing our aggregate state with the help of event sourcing, so by serializing domain events, this mistake cannot be done. I've created a sample application that might show how to deal with such an approach. Those serialized events can be published to external listeners. The same events from which we rebuild our domain object</a>

Take a look how ease it is to test such an aggregate. By invocking public method, we expect that proper domain events were raised. Simple as that. It is also tested in integration test, together with storage. Please note, that with this approach it is simple to create Shop Item as truly immutable aggregate and favour functional programming.

***Event Store***

Event store was implemented very simply - as database table EventStream that represents all events comming from one aggregate instance, which holds collection of all events (EventDescriptors).

***CQRS***

Read model in CQRS applcations is build when changes to write model are performed. That is why event sourcing can naturally lead to CQRS usage - is there any better way for listening for write model changes than subscribing to events raised from write model? The same events, from which write model is built? Take a look how simply it is to update read model in event sourced application. Read model there is implemented by directly using JDBC queries with denormalized table , beacuse why not?

Reliable Domain Events

Sat, 23 Apr 2016 00:00:00 +0000

We already know that domain events are a great tool to integrate different parts of your system (so called bounded contexts) and decouple them. Instead of directly calling outside component, we can notify our infrastructure, which in turn informs interested parties about what has just occurred. This another form of Inversion of Control helps us to achieve our goal. Consider following piece of code:

    class User {

        //...

        @Transactional
        public void register(LocalDateTime at, RegisteredBy by) {
            markRegistered();
            markSuspiciousIfAtNight(at);
            DomainEventsPublisher.publish(new UserRegistered(this.id(), at));
        }

    }

    public class UserRegistered implements DomainEvent {

        private final UserRegistered userId;
        private final LocalDateTime at;

        //getters, constructor

    }

    class DomainEventsPublisher {

        //...

        private final Register handlersRegistry;

        public static void publish(DomainEvent event) {
            handlersRegistry
                .getHandlers(event.getClass())
                .forEach(handler -> handler.handle(event));
        }

    }

    class SendCommunicationOnUserRegistered implements DomainEventHandler<UserRegistered> {

    @Override
    public void handle(UserRegistered event) {
        Communication c = sendRegistrationEmail(event.getUserId());
        storeCommunication(c);
    }

There is a big problem with this simple Observer pattern in above example. Namely, everything happens in the same database transaction, which has started at the time of calling register method. This has several implications. First of all, we did not decouple user registration from e-mail sending and we just have mentioned this is what domain events are for. If our mail server is down, registration fails. Why would our user care about an email? He correctly filled registration form and does not even know there is an email coming.

From the use case point of view, sending an email should not imply successful invocation. Secondly, even though it looks like the code deals only with users (it registers and marks as suspicious when needed), it modifies another aggregate - communications (communication would be probably modeled as a Generic Subdomain, but to simplify things consider it as another bounded context). The rule of thumb says that we should not modify several aggregates in one transaction, because these concepts should not be so directly related. Sending an email (or any other action done as a consequence of registering user) may take a lot of time, do I/O calls, etc.

But things get worse. Consider situation when everything was fine with our mail server, we want to save user to database and something fails and transaction rollbacks. Now we have a big problem, because confused user got error page as a response, but seconds later successful email with registration greetings.

This clearly shows that those concepts should be unrelated. One may come with an idea to fire all handlers asynchronously (so outside current transaction) but that solves only one issue: we could register users when we cannot send emails. We need something better. Fortunately, we can fire our emails just after transaction commit. Thus, we are sure everything was fine during registration and without doubts we can send the welcoming message. Spring gives us possibility to do that in a few lines of code with TransactionSynchronizationManager:

    class DomainEventsPublisher {

        //...

        public static void publish(DomainEvent event) {
            handlersRegistry
                .getHandlers(event.getClass())
                .forEach(handler -> handleAsynchronously(event, handlers));
        }

        private static void handleAsynchronously(DomainEvent event, DomainEventHandler handler) {
            if (TransactionSynchronizationManager.isActualTransactionActive()) {
                processAfterCommit(event, handler);
            } else {
                processNow(event, handler);
            }
        }

        private static void processNow(final DomainEvent event, DomainEventHandler handler) {
            handler.handle(event);
        }

        private static void processAfterCommit(final DomainEvent event, DomainEventHandler handler) {
            TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() {
                @Override
                public void afterCommit() {
                    handler.handle(event);
                }
            });
        }
   }

This example could be simplified with Spring 4.2 and @TransactionalEventListener. Thanks to this code, we are fine with cause and effect relationship. Taking advantage of java memory model nomenclature: we know that successful registration happens before sending an email. One may argue that there is a slight window of time when we are not consistent. As stated at the beginning, we don't need to be consistent when working with several bounded context. The important is that they will be eventually consistent. Anyway, we still did not solve the problem with unresponsive mail server. When it fails, the information is lost. We can implement something which will retry this operation with a reasonable back-off, but when our application crashes, we still don't have any mean to recover to former state. We need to deal with that issue.

It gets clearer that we need to store somewhere the intent of sending the email. The intent was of course UserRegistered event. By exposing this intent to for example JMS infrastructure, we can implement the retry mechanism. We can go further and store any event coming from a given bounded context. Repository of all domain events is in fact called Event Store. Below example contains code that serializes events to JSON and publishes them to JMS. We could publish them anywhere else, for example to a file on a local disk or to an Akka actor.

class ExternalEventStore {
 
    //...

    private final ProducerTemplate producerTemplate;
    private final ObjectMapper objectMapper = new ObjectMapper();
    
    void publish(DomainEvent event) throws JsonProcessingException {
        final String body = objectMapper.writeValueAsString(event);
        producerTemplate.sendBody("queue.url", InOnly, body);
    }
}

We could invoke ExternalEventStore from our handleNow method from previous example and make some other component consume those messages and send emails. But that would be just delegating the problem from mail server to another part of infrastructure - our queue. Putting a message to a queue might fail and the message would be lost. Moreover, our application can fail somewhere in between commit and invoking this asynchronous transaction listener. We also cannot move this code back to the transaction, because that would make us stuck at the beginning of our problem and queue failures would result in our users not being able to register.

We have to develop a consistent solution in which an occurrence of an event reflects that it really happened in our system. Also when it really happens, it should be followed by an event. In other words, those two statements should be in bi-conditional logical connective.

The problem is that our JMS component is not backed by the same data source as our domain model. One solution is to use global transaction and two-phase commits. The problem is that it might decrease performance significantly. Plus, not every part of our infrastructure must support that mechanism. More clever idea is to share the same data source for our messaging infrastructure and domain model (if those two parts support this kind of data soure). The side effect here is that we need to share the same schema, which might not be the nicest solution.

In my opinion the best option is to translate our events to our domain model storage and save them in the same transaction. Later on, another pool of threads should process them and publish somewhere else. Now our event store would look as follows:

public class InternalEventStore {

    //...

    private final ObjectMapper eventSerializer;
    private final SessionFactory sessionFactory;

    void store(DomainEvent event) throws JsonProcessingException  {
        session().save(serialized(event));
    }

    private PersistentEvent serialized(DomainEvent event) throws JsonProcessingException   {
        return new PersistentEvent(eventSerializer.writeValueAsString(event));
    }

    List<PersistentEvent> listPending(int limit) {
       //...
    }
}

public class PersistentEvent {

    public enum Status {
        PENDING, SENT
    }

    private Long id;

    private String uuid = UUID.randomUUID().toString();

    private LocalDateTime occuredAt = LocalDateTime.now();

    private LocalDateTime sentAt;

    private String body;

    private Status status = PENDING;

    public void sent(LocalDateTime at) {
        this.sentAt = at;
        this.status = SENT;
    }

    //...

}

and it would be invoked from DomainEventProcessor in the same transaction (so we came back to synchronous observator pattern):

 class DomainEventsPublisher {

    //...

    private final Register handlersRegistry;
    private final EventStore eventStore;

    public void publish(DomainEvent event) {
        handlersRegistry
            .getHandlers(event.getClass())
            .forEach(handler -> handler.handle(event));
        eventStore.store(event);
    }
}

Note that we left the register with handlers working under the same transaction. It is fine, because we may need to listen to this event somewhere else in the same bounded context. That way we won't modify another aggregates.

But we still want our events to appear in our JMS infrastructure. We can run a periodic job which scans list of our events and sends them to a queue or a topic:

class PublishPendingEventsScheduler {
    
    private final ExternalEventStore eventStore;
    private final ExternalEventStore publisher;

    @Scheduled(initialDelay = 3000, fixedDelayString = "${events.publisher.freq:3000}")
    public void sendEvents() {
        eventStore.listPending(100).forEach(this::publish);
    }
    
    private void publish(PersistentEvent event) {
        publisher.publish(event); // publisher needs to have publish(PersistentEvent event) method
        event.sent(now());
    }
}

We mark every event as sent, so that it won't be picked up in further invocation. If our message infrastructure fails, we try again soon. We might argue that this code suffers from the same problem as the whole example. Our database might fails at the time we want to save this event as sent. That is a fair concern, because we already have sent it to JMS. That means it can arrive at consumer side several times. It is important that consumer handles those events in an idempotent way - by for example storing PersistentEvent's uuid and doing de-duplication. Resending at producer side and idempotency at consumer side gives us at most once delivery.

In this post I tried to describe reliable domain events mechanism by implementing simple event store. Actually, events stores have much more benefits: we can examine every historical result of commands invoked in our system, run some forecasting algorithms, do event sourcing (reconstructing an aggregate by composing events it has produced) in different bounded contexts.

Stub and Verify?

Fri, 13 Nov 2015 00:00:00 +0000

Several times lately I came across Spock tests, which conceptually looked as follows:

    private static final BigDecimal ANY_AMOUNT = new BigDecimal("100.00")
    
    Payment payment = aSamplePayment()
    Client client = aSampleClient()
    
    PaymentCreator paymentCreator = Mock(PaymentCreator)
    PaymentRepository repository = Mock(PaymentRepository)

    @Subject
    Payments payments = new Payments(paymentCreator, repository)

    def "should create outgoing payment for loan"() {
        when:
            Payment actualPayment = payments.saveOutgoingFor(client, ANY_AMOUNT)
        then:
            1 * paymentCreator.createOutgoing(client, ANY_AMOUNT) >> payment
            1 * repository.save(payment)
            actualPayment == payment
    }

Spock allows you to stub and verify collaborator at once. See section 'Combining Mocking and Stubbing'. It is followed by sections which describe couple of discouraged solutions to testing, like partial stubs or stubbing static methods. In my opinion above section should also have this sidenote:

(Think twice before using this feature. It might be better to change the design of the code under specification.)

Design smell!

Consider following implementation under test:

 Payment saveOutgoingFor(Client client, BigDecimal amount) {
        Payment payment = paymentCreator.createOutgoing(client, amount);
        paymentRepository.save(payment);
        return payment;
    }

What it does is creating new payment and saving it in some kind of storage. Actually, the paymentRepository.save() method could return stored payment, so that we could get rid of last line. Such design can be explained like the following "do something with the payment, return it, do something else, and return it". Two actions are being done by two collaborating services. Actually we can cut it to "do something andThen do something else". Rings a bell? It is just function composition and you can read why dependency injection in most cases is just function composition here. What we care about is the result of function composition, which is being done inside one method. We don't care about partial steps and about their implementation. We are interested in the result, to be more precise we check expected state.

I bolded state not without a reason. Classical TDD user would tend to use stubs and check state, mockist TDD user likes mocks and behaviours. More about that you can read in famous Fowler's article. So what is actually wrong in checking partial steps in such scenarios? Well, it leaks implementation details. Now, we force the implementation to actually use PaymentCreator in a fixed way, contrary to stub, which can be desribed as "if you use one of my methods, I'll return this fixture". Both classical and mockists TDD enthusiasts have their rights, but in my opinion those two concepts should not be combined.

I express this opinion to one of my friends and he replied: "But what if your first step does something very important that you want to check. Plus it returns value that is used in next lines. Let's say it sends an email". That is a fair question and for simplicity reasons let's assume that mail sender sends information about payment being created to the client and actually really returns something that we care of later (for example a result indicating mail failure/success). The approach showed at the beginning of this article solves that issue, but...

Test isolation

You want your tests to fail for one reason. That is one of the most ground rules when writing tests. That means that we don't want to check how the payment looks like and if client was informed in the same test. To me, the tests should look like:

def "should create outgoing payment for loan"() {
        given:
            paymentCreator.createOutgoing(client, ANY_AMOUNT) >> payment
        expect:
            payments.saveOutgoingFor(client, ANY_AMOUNT) == payment
    }

    def "should store the payment"() {
        given:
            paymentCreator.createOutgoing(client, ANY_AMOUNT) >> payment
        when:
            payments.saveOutgoingFor(client, ANY_AMOUNT)
        then:
            1 * repository.save(payment)
    }

    def "should inform the client"() {
        when:
            payments.saveOutgoingFor(client, ANY_AMOUNT)
        then:
            1 * mailSender.informClient()
    }

That way every test fail only for one reason. Not more. It is easier to refractor code, it stops rigidity. Rigidity means that the software is difficult to change. A small change causes a cascade of subsequent changes. In that case, small change breaks unrelated tests.

My personal rule of thumb

I (try to) practise TDD all the time. I faced descirbed problems several times, and I came up with my rule when to use mocks and when go for stubs. I am a big fan of CQRS idea and I truly love it how it fits into above considerations. So when my unit test needs to check whether some action was perfomed (command was sent) I choose mocks and verify behaviours. When i rely on my collaborators to return something (queries) I always go for stubs. I don't think that "veryfing what you stubbed" is a good idea.

spring.io.2015

Thu, 14 May 2015 00:00:00 +0000

Last month I had chance to attend an event hosted by spring.io guys, namely conference Spring IO 2015. It was held in Barcelona and gathered 41 speakers, whose talks were around spring framework, groovy, grails and clouds. As a developer, I can hardly live without spring, so this was a perfect chance to enhance my spring-related experience. Although I was not amazed by all of the presented subjects, some talks brought my attention. I would like to quickly summarize one of those.

<a href=http://www.springio.net/inside-an-spring-event-sourced-cqrs-application-or-why-microservices-can-actually-work/”>Inside an Spring Event Sourced CQRS application</a>

You are probably familiar with concept of Command and Query Responsibility Segregation. If not I recommend this great reference. To cut the long story short, it deals with having two different models: one for writes and one for reads. The read model is user-friendly, meaning it is often denormalized for a convinient projections. Thus, it is faster. Especially comparing to traditional CRUD-based system which makes a dozens of join operations for a reader.

Eugen Paraschiv has shown his experience with using this technique together with event sourcing. Event sourcing is a concept which ensures that every change in application is related in an event. Those events are stored in a chronological order and then replayed. It was shown how CQRS-like nature of the system naturally leads to event sourcing. Having multiple reads models is easy when there is a single point of truth - an event - that we can listen to and update ourself as we want

The buttom line here is another thing. As those projections are independent of each other and rely only on domain events, we can run them in separated spring contexts. We can go further and run them in different processes and JVMs. Do you see how far it goes? Yep, we can run them on separated hardware gaining the buzzworld of 2015: microservice-oriented architecture. Starting with CQRS, event sourcing and different projections for different domain areas naturally leads there.

Event sourcing with CQRS brings another values to us. Events are immutable, simple standalone objects. Storing those is better for performance than persisting complex relational model. It is also simplification of the model - it really tells you what exactly have just happened. Events help you integrating with different systems, because they can just start listening to our events. Moreover, they can replay all the events from the beginning - meaning they can pretend being integrated ever since the first event was emitted.

This is not over yet. Event sourcing is great debugging tool. Imagine that you need to troubleshoot your application on production and check what happen say 10 days ago. With event sourcing it is super easy - you just need to replay all the events before that timestamp. Voila, you have just replicated the requested situation. One may say, that replayng millions of events is not efficient. Sure, that is a valid point, but for this reason snapshots are introduced. You can store snapshots for example once for 100 events, reducing the maximum number of events to be replayed to 100.

Talk by Eugen was one of the bests ones in the whole event. Check out summary of others that may interest you: <a href=http://www.springio.net/everything-you-need-to-know-about-java-classloaders//">Everything you need to know about Java Classloaders</a>, Designing for Distributed Systems with Reactor and Reactive Streams and Real-time with Spring: SSE and WebSockets