-
Notifications
You must be signed in to change notification settings - Fork 8
Expand file tree
/
Copy pathtesting.rss.xml
More file actions
838 lines (593 loc) · 122 KB
/
Copy pathtesting.rss.xml
File metadata and controls
838 lines (593 loc) · 122 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0">
<channel>
<title>Alexis King's Blog: Posts tagged 'testing'</title>
<description>Alexis King's Blog: Posts tagged 'testing'</description>
<link>http://lexi-lambda.github.io/tags/testing.html</link>
<lastBuildDate>Thu, 29 Jun 2017 12:08:49 UT</lastBuildDate>
<pubDate>Thu, 29 Jun 2017 12:08:49 UT</pubDate>
<ttl>1800</ttl>
<item>
<title>Unit testing effectful Haskell with monad-mock</title>
<link>http://lexi-lambda.github.io/blog/2017/06/29/unit-testing-effectful-haskell-with-monad-mock/?utm_source=testing&utm_medium=RSS</link>
<guid isPermaLink="false">urn:http-lexi-lambda-github-io:-blog-2017-06-29-unit-testing-effectful-haskell-with-monad-mock</guid>
<pubDate>Thu, 29 Jun 2017 12:08:49 UT</pubDate>
<author>Alexis King</author>
<description>
<p>Nearly eight months ago, <a href="/blog/2016/10/03/using-types-to-unit-test-in-haskell/">I wrote a blog post about unit testing effectful Haskell code</a> using a library called test-fixture. That library has served us well, but it wasn’t as easy to use as I would have liked, and it worked better with certain patterns than others. Since then, I’ve learned more about Haskell and more about testing, and I’m pleased to announce that I am releasing an entirely new testing library, <a href="https://hackage.haskell.org/package/monad-mock">monad-mock</a>.</p>
<!-- more-->
<h1 id="a-first-glance-at-monad-mock">A first glance at monad-mock</h1>
<p>The monad-mock library is, first and foremost, designed to be <em>easy</em>. It doesn’t ask much from you, and it requires almost zero boilerplate.</p>
<p>The first step is to write an mtl-style interface that encodes an effect you want to mock. For example, you might want to test some code that interacts with the filesystem:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">class</span> <span class="kt">Monad</span> <span class="n">m</span> <span class="ow">=&gt;</span> <span class="kt">MonadFileSystem</span> <span class="n">m</span> <span class="kr">where</span>
<span class="n">readFile</span> <span class="ow">::</span> <span class="kt">FilePath</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="kt">String</span>
<span class="n">writeFile</span> <span class="ow">::</span> <span class="kt">FilePath</span> <span class="ow">-&gt;</span> <span class="kt">String</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="nb">()</span>
</pre></div>
</div>
<p>Now you just have to write your code as normal. For demonstration purposes, here’s a function that defines copying a file in terms of <code>readFile</code> and <code>writeFile</code>:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">copyFile</span> <span class="ow">::</span> <span class="kt">MonadFileSystem</span> <span class="n">m</span> <span class="ow">=&gt;</span> <span class="kt">FilePath</span> <span class="ow">-&gt;</span> <span class="kt">FilePath</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="nb">()</span>
<span class="nf">copyFile</span> <span class="n">a</span> <span class="n">b</span> <span class="ow">=</span> <span class="kr">do</span>
<span class="n">contents</span> <span class="ow">&lt;-</span> <span class="n">readFile</span> <span class="n">a</span>
<span class="n">writeFile</span> <span class="n">b</span> <span class="n">contents</span>
</pre></div>
</div>
<p>Making this function work on the real filesystem is trivial, since we just need to define an instance of <code>MonadFileSystem</code> for <code>IO</code>:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">instance</span> <span class="kt">MonadFileSystem</span> <span class="kt">IO</span> <span class="kr">where</span>
<span class="n">readFile</span> <span class="ow">=</span> <span class="kt">Prelude</span><span class="o">.</span><span class="n">readFile</span>
<span class="n">writeFile</span> <span class="ow">=</span> <span class="kt">Prelude</span><span class="o">.</span><span class="n">writeFile</span>
</pre></div>
</div>
<p>But how do we test this? Well, we <em>could</em> run some real code in <code>IO</code>, which might not be so bad for such a simple function, but this seems like a bad idea. For one thing, a bad implementation of <code>copyFile</code> could do some pretty horrible things if it misbehaved and decided to overwrite important files, and if you’re constantly running a test suite whenever a file changes, it’s easy to imagine causing a lot of damage. Running tests against the real filesystem also makes tests slower and harder to parallelize, and it only gets much worse once you are doing more complex effects than interacting with the filesystem.</p>
<p>Using monad-mock, we can test this function in just a couple of lines of code:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">import</span> <span class="nn">Control.Exception</span> <span class="p">(</span><span class="nf">evaluate</span><span class="p">)</span>
<span class="kr">import</span> <span class="nn">Control.Monad.Mock</span>
<span class="kr">import</span> <span class="nn">Control.Monad.Mock.TH</span>
<span class="kr">import</span> <span class="nn">Data.Function</span> <span class="p">((</span><span class="o">&amp;</span><span class="p">))</span>
<span class="kr">import</span> <span class="nn">Test.Hspec</span>
<span class="nf">makeMock</span> <span class="s">"FileSystemAction"</span> <span class="p">[</span><span class="n">ts</span><span class="o">|</span> <span class="kt">MonadFileSystem</span> <span class="o">|</span><span class="p">]</span>
<span class="nf">spec</span> <span class="ow">=</span> <span class="n">describe</span> <span class="s">"copyFile"</span> <span class="o">$</span>
<span class="n">it</span> <span class="s">"reads a file and writes its contents to another file"</span> <span class="o">$</span>
<span class="n">evaluate</span> <span class="o">$</span> <span class="n">copyFile</span> <span class="s">"foo.txt"</span> <span class="s">"bar.txt"</span>
<span class="o">&amp;</span> <span class="n">runMock</span> <span class="p">[</span> <span class="kt">ReadFile</span> <span class="s">"foo.txt"</span> <span class="kt">:-&gt;</span> <span class="s">"contents"</span>
<span class="p">,</span> <span class="kt">WriteFile</span> <span class="s">"bar.txt"</span> <span class="s">"contents"</span> <span class="kt">:-&gt;</span> <span class="nb">()</span> <span class="p">]</span>
</pre></div>
</div>
<p>That’s it!</p>
<p>The last two lines of the above snippet are the real interesting bits, which specify the actions that are expected to be executed, and it couples them with their results. You will find that if you tweak the list in any way, such as reordering the actions, eliminating one or both of them, or adding an additional action to the end, the test will fail. We could even turn this into a property-based test that generated arbitrary file paths and file contents.</p>
<p>Admittedly, in this trivial example, the mock is a little silly, since converting this into a property-based test would demonstrate how much we’ve basically just reimplemented the function in our test. However, once our function starts to do somewhat more complicated things, then our tests become more meaningful. Here’s a similar function that only copies a file if it is nonempty:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">copyNonemptyFile</span> <span class="ow">::</span> <span class="kt">MonadFileSystem</span> <span class="n">m</span> <span class="ow">=&gt;</span> <span class="kt">FilePath</span> <span class="ow">-&gt;</span> <span class="kt">FilePath</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="nb">()</span>
<span class="nf">copyNonemptyFile</span> <span class="n">a</span> <span class="n">b</span> <span class="ow">=</span> <span class="kr">do</span>
<span class="n">contents</span> <span class="ow">&lt;-</span> <span class="n">readFile</span> <span class="n">a</span>
<span class="n">unless</span> <span class="p">(</span><span class="n">null</span> <span class="n">contents</span><span class="p">)</span> <span class="o">$</span>
<span class="n">writeFile</span> <span class="n">b</span> <span class="n">contents</span>
</pre></div>
</div>
<p>This function has some logic which is very clearly <em>not</em> expressed in its type, and it would be difficult to encode that information into the type in a safe way. Fortunately, we can guarantee that it works by writing some tests:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">describe</span> <span class="s">"copyNonemptyFile"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="n">it</span> <span class="s">"copies a file with contents"</span> <span class="o">$</span>
<span class="n">evaluate</span> <span class="o">$</span> <span class="n">copyNonemptyFile</span> <span class="s">"foo.txt"</span> <span class="s">"bar.txt"</span>
<span class="o">&amp;</span> <span class="n">runMock</span> <span class="p">[</span> <span class="kt">ReadFile</span> <span class="s">"foo.txt"</span> <span class="kt">:-&gt;</span> <span class="s">"contents"</span>
<span class="p">,</span> <span class="kt">WriteFile</span> <span class="s">"bar.txt"</span> <span class="s">"contents"</span> <span class="kt">:-&gt;</span> <span class="nb">()</span> <span class="p">]</span>
<span class="n">it</span> <span class="s">"does nothing with an empty file"</span> <span class="o">$</span>
<span class="n">evaluate</span> <span class="o">$</span> <span class="n">copyNonemptyFile</span> <span class="s">"foo.txt"</span> <span class="s">"bar.txt"</span>
<span class="o">&amp;</span> <span class="n">runMock</span> <span class="p">[</span> <span class="kt">ReadFile</span> <span class="s">"foo.txt"</span> <span class="kt">:-&gt;</span> <span class="s">""</span> <span class="p">]</span>
</pre></div>
</div>
<p>These tests are much more useful, and they have some actual value to them. Imagine we had accidentally written <code>when</code> instead of <code>unless</code>, an easy typo to make. Our tests would fail with some useful error messages:</p>
<pre><code>1) copyNonemptyFile copies a file with contents
uncaught exception: runMockT: expected the following unexecuted actions to be run:
WriteFile "bar.txt" "contents"
2) copyNonemptyFile does nothing with an empty file
uncaught exception: runMockT: expected end of program, called writeFile
given action: WriteFile "bar.txt" ""</code></pre>
<p>You now know enough to write tests with monad-mock.</p>
<h1 id="why-unit-test">Why unit test?</h1>
<p>When the issue of testing is brought up in Haskell, it is often treated with a certain distaste by a portion of the community. There are some points I’ve seen a number of times, and though they take different forms, they boil down to two ideas:</p>
<ol>
<li>
<p>“Haskell code does not need tests because the type system can prove correctness.”</p></li>
<li>
<p>“Testing in Haskell is trivial because it is a pure language, and testing pure functions is easy.”</p></li></ol>
<p>I’ve been writing Haskell professionally for over a year now, and I can happily say that there <em>is</em> some truth to both of those things! When my Haskell code typechecks, I feel a confidence in it that I would not feel were I using a language with a less powerful type system. Furthermore, Haskell encourages a “pure core, impure shell” approach to system design that makes testing many things pleasant and straightforward, and it completely eliminates the worry of subtle nondeterminism leaking into tests.</p>
<p>That said, Haskell is not a proof assistant, and its type system cannot guarantee everything, especially for code that operates on the boundaries of what Haskell can control. For much the same reason, I find that my pure code is the code I am <em>least</em> likely to need to test, since it is also the code with the strongest type safety guarantees, operating on types in my application’s domain. In contrast, the effectful code is often what I find the most value in extensively testing, since it often contains the most subtle complexity, and it is frequently difficult or even impossible to encode into types.</p>
<p>Haskell has the power to provide remarkably strong correctness guarantees with a surprisingly small amount of effort by using a combination of tests and types, using each to accommodate for the other’s weaknesses and playing to each technique’s strengths. Some code is test-driven, other code is type-driven. Most code ends up being a mix of both. Testing is just a tool like any other, and it’s nice to feel confident in one’s ability to effectively structure code in a decoupled, testable manner.</p>
<h1 id="why-mock">Why mock?</h1>
<p>Even if you accept that testing is good, the question of whether or not to <em>mock</em> is a subtler issue. To some people, “unit testing” is synonymous with mocks. This is emphatically not true, and in fact, overly aggressive mocking is one of the best ways to make your test suite completely worthless. The monad-mock approach to mocking is a bit more principled than mocking in many dynamic, object-oriented languages, but it comes with many of the same drawbacks: mocks couple your tests to your implementation in ways that make them less valuable and less meaningful.</p>
<p>For the <code>MonadFileSystem</code> example above, I would actually probably <em>not</em> use a mock. Instead, I would use a <strong>fake</strong>, in-memory filesystem implementation:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">newtype</span> <span class="kt">FakeFileSystemT</span> <span class="n">m</span> <span class="n">a</span> <span class="ow">=</span> <span class="kt">FakeFileSystemT</span> <span class="p">(</span><span class="kt">StateT</span> <span class="p">[(</span><span class="kt">FilePath</span><span class="p">,</span> <span class="kt">String</span><span class="p">)]</span> <span class="n">m</span> <span class="n">a</span><span class="p">)</span>
<span class="kr">deriving</span> <span class="p">(</span><span class="kt">Functor</span><span class="p">,</span> <span class="kt">Applicative</span><span class="p">,</span> <span class="kt">Monad</span><span class="p">)</span>
<span class="nf">fakeFileSystemT</span> <span class="ow">::</span> <span class="kt">Monad</span> <span class="n">m</span> <span class="ow">=&gt;</span> <span class="p">[(</span><span class="kt">FilePath</span><span class="p">,</span> <span class="kt">String</span><span class="p">)]</span>
<span class="ow">-&gt;</span> <span class="kt">FakeFileSystemT</span> <span class="n">m</span> <span class="n">a</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="p">[(</span><span class="kt">FilePath</span><span class="p">,</span> <span class="kt">String</span><span class="p">)])</span>
<span class="nf">fakeFileSystemT</span> <span class="n">fs</span> <span class="p">(</span><span class="kt">FakeFileSystemT</span> <span class="n">x</span><span class="p">)</span> <span class="ow">=</span> <span class="n">second</span> <span class="n">sort</span> <span class="o">&lt;$&gt;</span> <span class="n">runStateT</span> <span class="n">x</span> <span class="n">fs</span>
<span class="kr">instance</span> <span class="kt">Monad</span> <span class="n">m</span> <span class="ow">=&gt;</span> <span class="kt">MonadFileSystem</span> <span class="p">(</span><span class="kt">FakeFileSystemT</span> <span class="n">m</span><span class="p">)</span> <span class="kr">where</span>
<span class="n">readFile</span> <span class="n">path</span> <span class="ow">=</span> <span class="kt">FakeFileSystemT</span> <span class="o">$</span> <span class="n">get</span> <span class="o">&gt;&gt;=</span> <span class="nf">\</span><span class="n">fs</span> <span class="ow">-&gt;</span> <span class="n">lookup</span> <span class="n">path</span> <span class="n">fs</span> <span class="o">&amp;</span>
<span class="n">maybe</span> <span class="p">(</span><span class="n">fail</span> <span class="o">$</span> <span class="s">"readFile: no such file ‘"</span> <span class="o">++</span> <span class="n">path</span> <span class="o">++</span> <span class="s">"’"</span><span class="p">)</span> <span class="n">return</span>
<span class="n">writeFile</span> <span class="n">path</span> <span class="n">contents</span> <span class="ow">=</span> <span class="kt">FakeFileSystemT</span> <span class="o">.</span> <span class="n">modify</span> <span class="o">$</span> <span class="nf">\</span><span class="n">fs</span> <span class="ow">-&gt;</span>
<span class="p">(</span><span class="n">path</span><span class="p">,</span> <span class="n">contents</span><span class="p">)</span> <span class="kt">:</span> <span class="n">filter</span> <span class="p">((</span><span class="o">/=</span> <span class="n">path</span><span class="p">)</span> <span class="o">.</span> <span class="n">fst</span><span class="p">)</span> <span class="n">fs</span>
</pre></div>
</div>
<p>The above snippet demonstrates how easy it is to define a <code>MonadFileSystem</code> implementation in terms of <code>StateT</code>, and while this may seem like a lot of boilerplate, it really isn’t. You have to write a fake <em>once</em> per interface, and the above block is a minuscule twelve lines of code. With this technique, you are still able to write tests that depend on the state of the filesystem before and after running the implementation, but you decouple yourself from the precise process of getting there:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">describe</span> <span class="s">"copyNonemptyFile"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="n">it</span> <span class="s">"copies a file with contents"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="kr">let</span> <span class="p">(</span><span class="nb">()</span><span class="p">,</span> <span class="n">fs</span><span class="p">)</span> <span class="ow">=</span> <span class="n">runIdentity</span> <span class="o">$</span> <span class="n">copyNonemptyFile</span> <span class="s">"foo.txt"</span> <span class="s">"bar.txt"</span>
<span class="o">&amp;</span> <span class="n">fakeFileSystemT</span> <span class="p">[</span> <span class="p">(</span><span class="s">"foo.txt"</span><span class="p">,</span> <span class="s">"contents"</span><span class="p">)</span> <span class="p">]</span>
<span class="n">fs</span> <span class="p">`</span><span class="n">shouldBe</span><span class="p">`</span> <span class="p">[</span> <span class="p">(</span><span class="s">"bar.txt"</span><span class="p">,</span> <span class="s">"contents"</span><span class="p">),</span> <span class="p">(</span><span class="s">"foo.txt"</span><span class="p">,</span> <span class="s">"contents"</span><span class="p">)</span> <span class="p">]</span>
<span class="n">it</span> <span class="s">"does nothing with an empty file"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="kr">let</span> <span class="p">(</span><span class="nb">()</span><span class="p">,</span> <span class="n">fs</span><span class="p">)</span> <span class="ow">=</span> <span class="n">runIdentity</span> <span class="o">$</span> <span class="n">copyNonemptyFile</span> <span class="s">"foo.txt"</span> <span class="s">"bar.txt"</span>
<span class="o">&amp;</span> <span class="n">fakeFileSystemT</span> <span class="p">[</span> <span class="p">(</span><span class="s">"foo.txt"</span><span class="p">,</span> <span class="s">""</span><span class="p">)</span> <span class="p">]</span>
<span class="n">fs</span> <span class="p">`</span><span class="n">shouldBe</span><span class="p">`</span> <span class="p">[</span> <span class="p">(</span><span class="s">"foo.txt"</span><span class="p">,</span> <span class="s">""</span><span class="p">)</span> <span class="p">]</span>
</pre></div>
</div>
<p>This is better than using a mock, and I would highly recommend doing it if you can! However, a lot of real applications have to interact with services of much greater complexity than an idealized filesystem, and creating that sort of in-memory fake is not always practical. One such situation might be interacting with AWS CloudFormation, for example:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">class</span> <span class="kt">Monad</span> <span class="n">m</span> <span class="ow">=&gt;</span> <span class="kt">MonadAWS</span> <span class="n">m</span> <span class="kr">where</span>
<span class="n">createStack</span> <span class="ow">::</span> <span class="kt">StackName</span> <span class="ow">-&gt;</span> <span class="kt">StackTemplate</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="p">(</span><span class="kt">Either</span> <span class="kt">AWSError</span> <span class="kt">StackId</span><span class="p">)</span>
<span class="n">listStacks</span> <span class="ow">::</span> <span class="n">m</span> <span class="p">(</span><span class="kt">Either</span> <span class="kt">AWSError</span> <span class="p">[</span><span class="kt">StackSummaries</span><span class="p">])</span>
<span class="n">describeStack</span> <span class="ow">::</span> <span class="kt">StackId</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="p">(</span><span class="kt">Either</span> <span class="kt">AWSError</span> <span class="kt">StackInfo</span><span class="p">)</span>
<span class="c1">-- and so on...</span>
</pre></div>
</div>
<p>AWS is a very complex system, and it can do dozens of different things (and fail in dozens of different ways) based on an equally complex set of inputs. For example, in the above API, <code>createStack</code> needs to parse its template, which can be YAML or JSON, in order to determine which of many possible errors and behaviors can be produced, both on the initial call and on subsequent ones.</p>
<p>Creating a fake implementation of <em>AWS</em> is hardly feasible, and this is where a mock can be useful. By simply writing <code>makeMock "AWSAction" [ts| MonadAWS |]</code>, we can test functions that interact with AWS in a pure way without necessarily needing to replicate all of its complexity.</p>
<h2 id="isolating-mocks">Isolating mocks</h2>
<p>Of course, tests that use mocks provide less value than tests that use “smarter” fakes, since they are far more tightly coupled to the implementation, and it’s dramatically more likely that you will need to change the tests when you change the logic. To avoid this, it can be helpful to create multiple interfaces to the same thing: a high-level interface and a low-level one. If our above <code>MonadAWS</code> is a low-level interface, we could create a high-level counterpart that does precisely what our application needs:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">class</span> <span class="kt">Monad</span> <span class="n">m</span> <span class="ow">=&gt;</span> <span class="kt">MonadDeploy</span> <span class="n">m</span> <span class="kr">where</span>
<span class="n">executeDeployment</span> <span class="ow">::</span> <span class="kt">Deployment</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="p">(</span><span class="kt">Either</span> <span class="kt">DeployError</span> <span class="nb">()</span><span class="p">)</span>
</pre></div>
</div>
<p>When running our application “for real”, we would use <code>MonadAWS</code> to implement <code>MonadDeploy</code>:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">executeDeploymentImpl</span> <span class="ow">::</span> <span class="kt">MonadAWS</span> <span class="n">m</span> <span class="ow">=&gt;</span> <span class="kt">Deployment</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="p">(</span><span class="kt">Either</span> <span class="kt">DeployError</span> <span class="nb">()</span><span class="p">)</span>
<span class="nf">executeDeploymentImpl</span> <span class="ow">=</span> <span class="o">...</span>
</pre></div>
</div>
<p>The nice thing about this is we can actually test <code>executeDeploymentImpl</code> using a <code>MonadAWS</code> mock, so we can still have unit test coverage of the code on the boundaries of our system! Additionally, by containing the mock to a single place, we can test the rest of our code using a smarter fake implementation of <code>MonadDeploy</code>, helping to decouple our code from AWS’s complex API and improve the reliability and usefulness of our test suite.</p>
<p>They key point here is that mocking is just a small piece of the larger testing puzzle in <em>any</em> language, and that is just as true in Haskell. An overemphasis on mocking is an easy way to end up with a test suite that feels useless, probably because it is. Use mocks as a technique to insulate your application from the complexity in others’ APIs, then use more domain-specific testing techniques and type-level assertions to ensure the correctness of your logic.</p>
<h1 id="how-monad-mock-works">How monad-mock works</h1>
<p>If you’ve read this far and are convinced that monad-mock is useful, you may safely stop reading now. However, if you are interested in the details of what it actually does and what makes it tick, the rest of this blog post is going to focus on how the implementation works and how it compares to other techniques.</p>
<p>The centerpiece of monad-mock’s API is its monad transformer, <code>MockT</code>, which is a type constructor that accepts three types:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">newtype</span> <span class="kt">MockT</span> <span class="p">(</span><span class="n">f</span> <span class="ow">::</span> <span class="o">*</span> <span class="ow">-&gt;</span> <span class="o">*</span><span class="p">)</span> <span class="p">(</span><span class="n">m</span> <span class="ow">::</span> <span class="o">*</span> <span class="ow">-&gt;</span> <span class="o">*</span><span class="p">)</span> <span class="p">(</span><span class="n">a</span> <span class="ow">::</span> <span class="o">*</span><span class="p">)</span>
</pre></div>
</div>
<p>The <code>m</code> and <code>a</code> type variables obviously correspond to the usual monad transformer arguments, which represent the underlying monad and the result of the monadic computation, respectively. The <code>f</code> variable is more interesting, since it’s what makes <code>MockT</code> work at all, and it isn’t even a type: it’s a type constructor with kind <code>* -&gt; *</code>. What does it mean?</p>
<p>Looking at the type signature of <code>runMockT</code> gives us a little bit more information about what that <code>f</code> actually represents:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">runMockT</span> <span class="ow">::</span> <span class="p">(</span><span class="kt">Action</span> <span class="n">f</span><span class="p">,</span> <span class="kt">Monad</span> <span class="n">m</span><span class="p">)</span> <span class="ow">=&gt;</span> <span class="p">[</span><span class="kt">WithResult</span> <span class="n">f</span><span class="p">]</span> <span class="ow">-&gt;</span> <span class="kt">MockT</span> <span class="n">f</span> <span class="n">m</span> <span class="n">a</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="n">a</span>
</pre></div>
</div>
<p>This type signature provides two pieces of key information:</p>
<ol>
<li>
<p>The <code>f</code> parameter is constrained by the <code>Action f</code> constraint.</p></li>
<li>
<p>Running a mocked computation requires supplying a list of <code>WithResult f</code> values. This list corresponds to the list of expectations provided to <code>runMock</code> in earlier examples.</p></li></ol>
<p>To understand both of these things, it helps to examine the definition of an actual datatype that can have an <code>Action</code> instance. For the filesystem example, the action datatype looks like this:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">data</span> <span class="kt">FileSystemAction</span> <span class="n">r</span> <span class="kr">where</span>
<span class="kt">ReadFile</span> <span class="ow">::</span> <span class="kt">FilePath</span> <span class="ow">-&gt;</span> <span class="kt">FileSystemAction</span> <span class="kt">String</span>
<span class="kt">WriteFile</span> <span class="ow">::</span> <span class="kt">FilePath</span> <span class="ow">-&gt;</span> <span class="kt">String</span> <span class="ow">-&gt;</span> <span class="kt">FileSystemAction</span> <span class="nb">()</span>
</pre></div>
</div>
<p>Notice how each constructor clearly corresponds to one of the methods of <code>MonadFileSystem</code>, with a type to match. Now the purpose of the type provided to the <code>FileSystemAction</code> constructor (in this case <code>r</code>) should hopefully become clear: it represents the type of the value <em>produced</em> by each method. Also note that the type is completely phantom—it does not appear in negative position in any of the constructors.</p>
<p>With this in mind, we can take a look at the definition of <code>WithResult</code>:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">data</span> <span class="kt">WithResult</span> <span class="n">f</span> <span class="kr">where</span>
<span class="p">(</span><span class="kt">:-&gt;</span><span class="p">)</span> <span class="ow">::</span> <span class="n">f</span> <span class="n">r</span> <span class="ow">-&gt;</span> <span class="n">r</span> <span class="ow">-&gt;</span> <span class="kt">WithResult</span> <span class="n">f</span>
</pre></div>
</div>
<p>This is what defines the <code>(:-&gt;)</code> constructor from earlier in the blog post, and you can see that it effectively just represents a tuple of an action and a value of its associated result. It’s completely type-safe, since it ensures the result matches the type argument to the action.</p>
<p>Finally, this brings us to the <code>Action</code> class, which is not complex, but is unfortunately necessary:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">class</span> <span class="kt">Action</span> <span class="n">f</span> <span class="kr">where</span>
<span class="n">eqAction</span> <span class="ow">::</span> <span class="n">f</span> <span class="n">a</span> <span class="ow">-&gt;</span> <span class="n">f</span> <span class="n">b</span> <span class="ow">-&gt;</span> <span class="kt">Maybe</span> <span class="p">(</span><span class="n">a</span> <span class="kt">:~:</span> <span class="n">b</span><span class="p">)</span>
<span class="n">showAction</span> <span class="ow">::</span> <span class="n">f</span> <span class="n">a</span> <span class="ow">-&gt;</span> <span class="kt">String</span>
</pre></div>
</div>
<p>Notice that these methods are effectively just <code>(==)</code> and <code>show</code>, lifted to type constructors of kind <code>* -&gt; *</code>. One significant difference is that <code>eqAction</code> produces <code>Maybe (a :~: b)</code> instead of <code>Bool</code>, where <code>(:~:)</code> is from <code>Data.Type.Equality</code>. This is a type equality witness, which means a successful equality between two values allows the compiler to be sure that the two <em>types</em> are equal. This is necessary for the implementation of <code>runMockT</code> due to the phantom type in actions—in order to convince GHC that we can properly return the result of a mocked action, we need to assure it that the value we’re going to return is actually of the proper type.</p>
<p>Implementing this typeclass is not particularly burdensome, but it’s entirely boilerplate, so even if you want to define your own action type (that is, you don’t want to use <code>makeMock</code>), you can use the <code>deriveAction</code> function from <code>Control.Monad.Mock.TH</code> to derive an <code>Action</code> instance on an existing datatype.</p>
<h2 id="connecting-the-mock-to-its-class">Connecting the mock to its class</h2>
<p>Now that we have an action with which to mock a class, we need to actually define an instance of that class for <code>MockT</code>. For this process, monad-mock provides a <code>mockAction</code> function with the following type:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">mockAction</span> <span class="ow">::</span> <span class="p">(</span><span class="kt">Action</span> <span class="n">f</span><span class="p">,</span> <span class="kt">Monad</span> <span class="n">m</span><span class="p">)</span> <span class="ow">=&gt;</span> <span class="kt">String</span> <span class="ow">-&gt;</span> <span class="n">f</span> <span class="n">r</span> <span class="ow">-&gt;</span> <span class="kt">MockT</span> <span class="n">f</span> <span class="n">m</span> <span class="n">r</span>
</pre></div>
</div>
<p>This function accepts two arguments: the name of the method being mocked and the action that represents the current call. This is easier to illustrate with an actual instance of <code>MonadFileSystem</code> using <code>MockT</code> and our <code>FileSystemAction</code> type:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">instance</span> <span class="kt">Monad</span> <span class="n">m</span> <span class="ow">=&gt;</span> <span class="kt">MonadFileSystem</span> <span class="p">(</span><span class="kt">MockT</span> <span class="kt">FileSystemAction</span> <span class="n">m</span><span class="p">)</span> <span class="kr">where</span>
<span class="n">readFile</span> <span class="n">a</span> <span class="ow">=</span> <span class="n">mockAction</span> <span class="s">"readFile"</span> <span class="p">(</span><span class="kt">ReadFile</span> <span class="n">a</span><span class="p">)</span>
<span class="n">writeFile</span> <span class="n">a</span> <span class="n">b</span> <span class="ow">=</span> <span class="n">mockAction</span> <span class="s">"writeFile"</span> <span class="p">(</span><span class="kt">WriteFile</span> <span class="n">a</span> <span class="n">b</span><span class="p">)</span>
</pre></div>
</div>
<p>This allows <code>readFile</code> and <code>writeFile</code> to defer to the mock, and providing the names of the functions as strings helps monad-mock to produce useful error messages upon failure. Internally, <code>MockT</code> is a <code>StateT</code> that keeps track of a list of <code>WithResult f</code> values as its state. Each call to the mock checks the action against the internal list of calls, and if they match, it returns the associated result. Otherwise, it throws an exception.</p>
<p>This scheme is simple, but it seems to work remarkably well. There are some obvious enhancements that will probably be eventually necessary, like allowing action results that run in the underlying monad <code>m</code> in order to support things like <code>throwError</code> from <code>MonadError</code>, but so far, it hasn’t been necessary for what we’ve been using it for. Certain tricky signatures defy this simple technique, such as signatures where a monadic action appears in a negative position (that is, the signatures you need things like <a href="https://hackage.haskell.org/package/monad-control">monad-control</a> or <a href="https://hackage.haskell.org/package/monad-unlift">monad-unlift</a> for), but we’ve found that most of our effects don’t have any reason to include such signatures.</p>
<h1 id="a-brief-comparison-with-freer-monads">A brief comparison with free(r) monads</h1>
<p>At this point, astute readers will likely be thinking about free monads, which parts of this technique greatly resemble. The representation of actions as GADTs is especially similar to <a href="https://hackage.haskell.org/package/freer">freer</a>, which does something extremely similar. Indeed, you can think of this technique as something that combines a freer-style representation with mtl-style classes. Given that freer already does this, you might ask yourself what the point is.</p>
<p>If you are already sold on free monads, monad-mock may very well be uninteresting to you. From the perspective of theoretical novelty, monad-mock is not anything new or different. However, there are a variety of practical reasons to prefer mtl over free, and it’s nice to see how easy it is to enjoy the testing benefits of free without too much extra effort.</p>
<p>An in-depth comparison between mtl and free is well outside the scope of this blog post. However, the key point is that this technique <em>only</em> affects test code, so the real runtime implementation will not be affected in any way. This means you can take advantage of the performance benefits and ecosystem support of mtl without sacrificing simple, expressive testing.</p>
<h1 id="conclusion">Conclusion</h1>
<p>To cap things off, I want to emphasize monad-mock’s role as a single part of a larger initiative we’ve been making for the better part of the past eighteen months. Haskell is a language with ever-evolving techniques and style, and it’s sometimes dizzying to figure out how to use all the pieces together to develop robust, maintainable applications. While monad-mock might not be anything drastically different from existing testing techniques, my hope is that it can provide an opinionated mechanism to make testing easy and accessible, even for complex interactions with other services and systems.</p>
<p>I’ve made an effort to make it abundantly clear in this blog post that monad-mock is <em>not</em> a silver bullet to testing, and in fact, I would prefer other techniques for ensuring correctness whenever possible. Even so, mocking is a nice tool to have in your toolbox, and it’s a good fallback to get even the worst APIs under test coverage.</p>
<p>If you want to try out monad-mock for yourself, <a href="https://hackage.haskell.org/package/monad-mock">take a look at the documentation on Hackage</a> and start playing around! It’s still early software, so it’s not the most proven or featureful, but we’ve managed to get mileage out of it already, all the same. If you find any problems, have a use case it does not support, or just find something about it unclear, please do not hesitate to <a href="https://github.com/cjdev/monad-mock">open an issue on the GitHub repository</a>—we obviously can’t fix issues we don’t know about.</p>
<p>Thanks as always to the many people who have contributed ideas that have shaped my philosophy and approach to testing and have helped provide the tools that make this library work. Happy testing!</p></description></item>
<item>
<title>Using types to unit-test in Haskell</title>
<link>http://lexi-lambda.github.io/blog/2016/10/03/using-types-to-unit-test-in-haskell/?utm_source=testing&utm_medium=RSS</link>
<guid isPermaLink="false">urn:http-lexi-lambda-github-io:-blog-2016-10-03-using-types-to-unit-test-in-haskell</guid>
<pubDate>Mon, 03 Oct 2016 01:20:43 UT</pubDate>
<author>Alexis King</author>
<description>
<p>Object-oriented programming languages make unit testing easy by providing obvious boundaries between units of code in the form of classes and interfaces. These boundaries make it easy to stub out parts of a system to test functionality in isolation, which makes it possible to write fast, deterministic test suites that are robust in the face of change. When writing Haskell, it can be unclear how to accomplish the same goals: even inside pure code, it can become difficult to test a particular code path without also testing all its collaborators.</p>
<p>Fortunately, by taking advantage of Haskell’s expressive type system, it’s possible to not only achieve parity with object-oriented testing techniques, but also to provide stronger static guarantees as well. Furthermore, it’s all possible without resorting to extra-linguistic hacks that static object-oriented languages sometimes use for mocking, such as dynamic bytecode generation.</p>
<!-- more-->
<h1 id="first-an-aside-on-testing-philosophy">First, an aside on testing philosophy</h1>
<p>Testing methodology is a controversial topic within the larger programming community, and there are a multitude of different approaches. This blog post is about <em>unit testing</em>, an already nebulous term with a number of different definitions. For the purposes of this post, I will define a unit test as a test that stubs out collaborators of the code under test in some way. Accomplishing that in Haskell is what this is primarily about.</p>
<p>I want to be clear that I do not think that unit tests are the only way to write tests, nor the best way, nor even always an applicable way. Depending on your domain, rigorous unit testing might not even make sense, and other forms of tests (end-to-end, integration, benchmarks, etc.) might fulfill your needs.</p>
<p>In practice, though, implementing those other kinds of tests seems to be well-documented in Haskell compared to pure, object-oriented style unit testing. As my Haskell applications have grown, I have found myself wanting a more fine-grained testing tool that allows me to both test a piece of my codebase in isolation and also use my domain-specific types. This blog post is about that.</p>
<p>With that disclaimer out of the way, let’s talk about testing in Haskell.</p>
<h1 id="drawing-seams-using-types">Drawing seams using types</h1>
<p>One of the primary attributes of unit tests in object-oriented languages, especially statically-typed ones, is the concept of “seams” within a codebase. These are internal boundaries between components of a system. Some boundaries are obvious—interactions with a database, manipulation of the file system, and performing I/O over the network, to name a few examples—but others are more subtle. Especially in larger codebases, it can be helpful to isolate two related but distinct pieces of functionality as much as possible, which makes them easier to reason about, even if they’re actually part of the same codebase.</p>
<p>In OO languages, these seams are often marked using interfaces, whether explicitly (in the case of static languages) or implicitly (in the case of dynamic ones). By programming to an interface, it’s possible to create “fake” implementations of that interface for use in unit tests, effectively making it possible to stub out code that isn’t directly relevant to the code being tested.</p>
<p>In Haskell, representing these seams is a lot less obvious. Consider a fairly trivial function that reverses a file’s contents on the file system:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">reverseFile</span> <span class="ow">::</span> <span class="kt">FilePath</span> <span class="ow">-&gt;</span> <span class="kt">IO</span> <span class="nb">()</span>
<span class="nf">reverseFile</span> <span class="n">path</span> <span class="ow">=</span> <span class="kr">do</span>
<span class="n">contents</span> <span class="ow">&lt;-</span> <span class="n">readFile</span> <span class="n">path</span>
<span class="n">writeFile</span> <span class="n">path</span> <span class="p">(</span><span class="n">reverse</span> <span class="n">contents</span><span class="p">)</span>
</pre></div>
</div>
<p>This function is impossible to test without testing against a real file system. It simply performs I/O directly, and there’s no way to “mock out” the file system for testing purposes. Now, admittedly, this function is so trivial that a unit test might not seem worth the cost, but consider a slightly more complicated function that interacts with a database:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">renderUserProfile</span> <span class="ow">::</span> <span class="kt">Id</span> <span class="kt">User</span> <span class="ow">-&gt;</span> <span class="kt">IO</span> <span class="kt">HTML</span>
<span class="nf">renderUserProfile</span> <span class="n">userId</span> <span class="ow">=</span> <span class="kr">do</span>
<span class="n">user</span> <span class="ow">&lt;-</span> <span class="n">fetchUser</span> <span class="n">userId</span>
<span class="n">posts</span> <span class="ow">&lt;-</span> <span class="n">fetchRecentPosts</span> <span class="n">userId</span>
<span class="n">return</span> <span class="o">$</span> <span class="n">div</span>
<span class="p">[</span> <span class="n">h1</span> <span class="p">(</span><span class="n">userName</span> <span class="n">user</span> <span class="o">&lt;&gt;</span> <span class="s">"’s Profile"</span><span class="p">)</span>
<span class="p">,</span> <span class="n">h2</span> <span class="s">"Recent Posts"</span>
<span class="p">,</span> <span class="n">ul</span> <span class="p">(</span><span class="n">map</span> <span class="p">(</span><span class="n">li</span> <span class="o">.</span> <span class="n">postTitle</span><span class="p">)</span> <span class="n">posts</span><span class="p">)</span>
<span class="p">]</span>
</pre></div>
</div>
<p>It might now be a bit more clear that it could be useful to test the above function without running a real database and doing all the necessary context setup before each test case. Indeed, it would be nice if a test could just provide stubbed implementations for <code>fetchUser</code> and <code>fetchRecentPosts</code>, then make assertions about the output.</p>
<p>One way to solve this problem is to pass the results of those two functions to <code>renderUserProfile</code> as arguments, turning it into a pure function that could be easily tested. This becomes obnoxious for functions of even just slightly more complexity, though (it is not unreasonable to imagine needing a handful of different queries to render a user’s profile page), and it requires significantly restructuring code simply because the tests need it.</p>
<p>The above code is not only difficult to test, however—it has another problem, too. Specifically, both functions return <code>IO</code> values, which means they can effectively do <em>anything</em>. Haskell has a very strong type system for typing terms, but it doesn’t provide any guarantees about effects beyond a simple yes/no answer about function purity. Even though the <code>renderUserProfile</code> function should really only need to interact with the database, it could theoretically delete files, send emails, make HTTP requests, or do any number of other things.</p>
<p>Fortunately, it’s possible to solve <em>both</em> problems—a lack of testability and a lack of type safety—using the same general technique. This approach is reminiscent of the interface-based seams of object-oriented languages, but unlike most object-oriented approaches, it provides additional type safety guarantees without the need to explicitly modify the code to support some kind of dependency injection.</p>
<h2 id="making-implicit-interfaces-explicit">Making implicit interfaces explicit</h2>
<p>Statically typed, object-oriented languages provide interfaces as a language construct to encode certain kinds of contracts into the type system, and Haskell has something similar. Typeclasses are, in many ways, an analog to OO interfaces, and they can be used in a similar way. In the above case, let’s write down interfaces that the <code>reverseFile</code> and <code>renderUserProfile</code> functions can use:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">class</span> <span class="kt">Monad</span> <span class="n">m</span> <span class="ow">=&gt;</span> <span class="kt">MonadFS</span> <span class="n">m</span> <span class="kr">where</span>
<span class="n">readFile</span> <span class="ow">::</span> <span class="kt">FilePath</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="kt">String</span>
<span class="n">writeFile</span> <span class="ow">::</span> <span class="kt">FilePath</span> <span class="ow">-&gt;</span> <span class="kt">String</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="nb">()</span>
<span class="kr">class</span> <span class="kt">Monad</span> <span class="n">m</span> <span class="ow">=&gt;</span> <span class="kt">MonadDB</span> <span class="n">m</span> <span class="kr">where</span>
<span class="n">fetchUser</span> <span class="ow">::</span> <span class="kt">Id</span> <span class="kt">User</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="kt">User</span>
<span class="n">fetchRecentPosts</span> <span class="ow">::</span> <span class="kt">Id</span> <span class="kt">User</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="p">[</span><span class="kt">Post</span><span class="p">]</span>
</pre></div>
</div>
<p>The really nice thing about these interfaces is that our function implementations don’t have to change <em>at all</em> to take advantage of them. In fact, all we have to change is their types:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">reverseFile</span> <span class="ow">::</span> <span class="kt">MonadFS</span> <span class="n">m</span> <span class="ow">=&gt;</span> <span class="kt">FilePath</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="nb">()</span>
<span class="nf">reverseFile</span> <span class="n">path</span> <span class="ow">=</span> <span class="kr">do</span>
<span class="n">contents</span> <span class="ow">&lt;-</span> <span class="n">readFile</span> <span class="n">path</span>
<span class="n">writeFile</span> <span class="n">path</span> <span class="p">(</span><span class="n">reverse</span> <span class="n">contents</span><span class="p">)</span>
<span class="nf">renderUserProfile</span> <span class="ow">::</span> <span class="kt">MonadDB</span> <span class="n">m</span> <span class="ow">=&gt;</span> <span class="kt">Id</span> <span class="kt">User</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="kt">HTML</span>
<span class="nf">renderUserProfile</span> <span class="n">userId</span> <span class="ow">=</span> <span class="kr">do</span>
<span class="n">user</span> <span class="ow">&lt;-</span> <span class="n">fetchUser</span> <span class="n">userId</span>
<span class="n">posts</span> <span class="ow">&lt;-</span> <span class="n">fetchRecentPosts</span> <span class="n">userId</span>
<span class="n">return</span> <span class="o">$</span> <span class="n">div</span>
<span class="p">[</span> <span class="n">h1</span> <span class="p">(</span><span class="n">userName</span> <span class="n">user</span> <span class="o">&lt;&gt;</span> <span class="s">"’s Profile"</span><span class="p">)</span>
<span class="p">,</span> <span class="n">h2</span> <span class="s">"Recent Posts"</span>
<span class="p">,</span> <span class="n">ul</span> <span class="p">(</span><span class="n">map</span> <span class="p">(</span><span class="n">li</span> <span class="o">.</span> <span class="n">postTitle</span><span class="p">)</span> <span class="n">posts</span><span class="p">)</span>
<span class="p">]</span>
</pre></div>
</div>
<p>This is pretty neat, since we haven’t had to alter our code at all, but we’ve managed to completely decouple ourselves from <code>IO</code>. This has the direct effect of both making our code more abstract (we no longer rely on the “real” file system or a “real” database, which makes our code easier to test) and restricting what our functions can do (just from looking at the type signatures, we know what side-effects they can perform).</p>
<p>Of course, since we’re now coding against an interface, our code doesn’t actually do much of anything. If we want to actually use the functions we’ve written, we’ll have to define instances of <code>MonadFS</code> and <code>MonadDB</code>. When actually running our code, we’ll probably still use <code>IO</code> (or some monad transformer stack with <code>IO</code> at the bottom), so we can define trivial instances for that existing use case:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">instance</span> <span class="kt">MonadFS</span> <span class="kt">IO</span> <span class="kr">where</span>
<span class="n">readFile</span> <span class="ow">=</span> <span class="kt">Prelude</span><span class="o">.</span><span class="n">readFile</span>
<span class="n">writeFile</span> <span class="ow">=</span> <span class="kt">Prelude</span><span class="o">.</span><span class="n">writeFile</span>
<span class="kr">instance</span> <span class="kt">MonadDB</span> <span class="kt">IO</span> <span class="kr">where</span>
<span class="n">fetchUser</span> <span class="ow">=</span> <span class="kt">SQL</span><span class="o">.</span><span class="n">fetchUser</span>
<span class="n">fetchRecentPosts</span> <span class="ow">=</span> <span class="kt">SQL</span><span class="o">.</span><span class="n">fetchRecentPosts</span>
</pre></div>
</div>
<p>Even if we go no further, <strong>this is already incredibly useful</strong>. By restricting the sorts of effects our functions can perform at the type level, it becomes a lot easier to see which code is interacting with what. This can be invaluable when working in a part of a moderately large codebase that you are unfamiliar with. Even if the only instance of these typeclasses is <code>IO</code>, the benefits are immediately apparent.</p>
<p>Of course, this blog post is about testing, so we’re going to go further and take advantage of these seams we’ve now drawn. The question is: how?</p>
<h1 id="testing-with-typeclasses-an-initial-attempt">Testing with typeclasses: an initial attempt</h1>
<p>Given that we now have functions depending on an interface instead of <code>IO</code>, we can create separate instances of our typeclasses for use in tests. Let’s start with the <code>renderUserProfile</code> function. We’ll create a simple wrapper around the <code>Identity</code> type, since we don’t actually care much about the “effects” of our <code>MonadDB</code> methods:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">import</span> <span class="nn">Data.Functor.Identity</span>
<span class="kr">newtype</span> <span class="kt">TestM</span> <span class="n">a</span> <span class="ow">=</span> <span class="kt">TestM</span> <span class="p">(</span><span class="kt">Identity</span> <span class="n">a</span><span class="p">)</span>
<span class="kr">deriving</span> <span class="p">(</span><span class="kt">Functor</span><span class="p">,</span> <span class="kt">Applicative</span><span class="p">,</span> <span class="kt">Monad</span><span class="p">)</span>
<span class="nf">unTestM</span> <span class="ow">::</span> <span class="kt">TestM</span> <span class="n">a</span> <span class="ow">-&gt;</span> <span class="n">a</span>
<span class="nf">unTestM</span> <span class="p">(</span><span class="kt">TestM</span> <span class="p">(</span><span class="kt">Identity</span> <span class="n">x</span><span class="p">))</span> <span class="ow">=</span> <span class="n">x</span>
</pre></div>
</div>
<p>Now, we’ll create a trivial instance of <code>MonadDB</code> for <code>TestM</code>:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">instance</span> <span class="kt">MonadDB</span> <span class="kt">TestM</span> <span class="kr">where</span>
<span class="n">fetchUser</span> <span class="kr">_</span> <span class="ow">=</span> <span class="n">return</span> <span class="kt">User</span> <span class="p">{</span> <span class="n">userName</span> <span class="ow">=</span> <span class="s">"Alyssa"</span> <span class="p">}</span>
<span class="n">fetchRecentPosts</span> <span class="kr">_</span> <span class="ow">=</span> <span class="n">return</span>
<span class="p">[</span> <span class="kt">Post</span> <span class="p">{</span> <span class="n">postTitle</span> <span class="ow">=</span> <span class="s">"Metacircular Evaluator"</span> <span class="p">}</span> <span class="p">]</span>
</pre></div>
</div>
<p>With this instance, it’s now possible to write a simple unit test of the <code>renderUserProfile</code> function that doesn’t need a real database running at all:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">spec</span> <span class="ow">=</span> <span class="n">describe</span> <span class="s">"renderUserProfile"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="n">it</span> <span class="s">"shows the user’s name"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="kr">let</span> <span class="n">result</span> <span class="ow">=</span> <span class="n">unTestM</span> <span class="p">(</span><span class="n">renderUserProfile</span> <span class="p">(</span><span class="n">intToId</span> <span class="mi">1234</span><span class="p">))</span>
<span class="n">result</span> <span class="p">`</span><span class="n">shouldContainElement</span><span class="p">`</span> <span class="n">h1</span> <span class="s">"Alyssa’s Profile"</span>
<span class="n">it</span> <span class="s">"shows a list of the user’s posts"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="kr">let</span> <span class="n">result</span> <span class="ow">=</span> <span class="n">unTestM</span> <span class="p">(</span><span class="n">renderUserProfile</span> <span class="p">(</span><span class="n">intToId</span> <span class="mi">1234</span><span class="p">))</span>
<span class="n">result</span> <span class="p">`</span><span class="n">shouldContainElement</span><span class="p">`</span> <span class="n">ul</span> <span class="p">[</span> <span class="n">li</span> <span class="s">"Metacircular Evaluator"</span> <span class="p">]</span>
</pre></div>
</div>
<p>This is pretty nice, and running the above tests reveals a nice property of these kinds of isolated test cases: the test suite runs <em>really, really fast</em>. Communicating with a database, even in extremely simple ways, takes a measurable amount of time, especially with dozens of tests. In contrast, even with hundreds of tests, our unit test suite runs in less than a tenth of a second.</p>
<p>This all seems to be successful, so let’s try and apply the same testing technique to <code>reverseFile</code>.</p>
<h2 id="testing-side-effectful-code">Testing side-effectful code</h2>
<p>Looking at the type signature for <code>reverseFile</code>, we have a small problem:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">reverseFile</span> <span class="ow">::</span> <span class="kt">MonadFS</span> <span class="n">m</span> <span class="ow">=&gt;</span> <span class="kt">FilePath</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="nb">()</span>
</pre></div>
</div>
<p>Specifically, the return type is <code>()</code>. Making any assertions against the result of this function would be completely worthless, given that it’s guaranteed to be the same exact thing each time. Instead, <code>reverseFile</code> is inherently side-effectful, so we want to be able to test that it properly interacts with the file system in the correct way.</p>
<p>In order to do this, a simple wrapper around <code>Identity</code> won’t be enough, but we can replace it with something more powerful: <code>Writer</code>. Specifically, we can use a writer monad to “log” what gets called in order to test side-effects. We’ll start by creating a new <code>TestM</code> type, just like last time:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">newtype</span> <span class="kt">TestM</span> <span class="n">a</span> <span class="ow">=</span> <span class="kt">TestM</span> <span class="p">(</span><span class="kt">Writer</span> <span class="p">[</span><span class="kt">String</span><span class="p">]</span> <span class="n">a</span><span class="p">)</span>
<span class="kr">deriving</span> <span class="p">(</span><span class="kt">Functor</span><span class="p">,</span> <span class="kt">Applicative</span><span class="p">,</span> <span class="kt">Monad</span><span class="p">,</span> <span class="kt">MonadWriter</span> <span class="p">[</span><span class="kt">String</span><span class="p">])</span>
<span class="nf">logTestM</span> <span class="ow">::</span> <span class="kt">TestM</span> <span class="n">a</span> <span class="ow">-&gt;</span> <span class="p">[</span><span class="kt">String</span><span class="p">]</span>
<span class="nf">logTestM</span> <span class="p">(</span><span class="kt">TestM</span> <span class="n">w</span><span class="p">)</span> <span class="ow">=</span> <span class="n">execWriter</span> <span class="n">w</span>
</pre></div>
</div>
<p>Using this slightly more powerful type, we can write a useful instance of <code>MonadFS</code> that will track the argument given to <code>writeFile</code>:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">instance</span> <span class="kt">MonadFS</span> <span class="kt">TestM</span> <span class="kr">where</span>
<span class="n">readFile</span> <span class="kr">_</span> <span class="ow">=</span> <span class="n">return</span> <span class="s">"hello"</span>
<span class="n">writeFile</span> <span class="kr">_</span> <span class="n">contents</span> <span class="ow">=</span> <span class="n">tell</span> <span class="p">[</span><span class="n">contents</span><span class="p">]</span>
</pre></div>
</div>
<p>Again, the instance is quite simple, but it now enables us to write a straightforward unit test for <code>reverseFile</code>:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">spec</span> <span class="ow">=</span> <span class="n">describe</span> <span class="s">"reverseFile"</span> <span class="o">$</span>
<span class="n">it</span> <span class="s">"reverses a file’s contents on the filesystem"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="kr">let</span> <span class="n">calls</span> <span class="ow">=</span> <span class="n">logTestM</span> <span class="p">(</span><span class="n">reverseFile</span> <span class="s">"foo.txt"</span><span class="p">)</span>
<span class="n">calls</span> <span class="p">`</span><span class="n">shouldBe</span><span class="p">`</span> <span class="p">[</span><span class="s">"olleh"</span><span class="p">]</span>
</pre></div>
</div>
<p>Again, quite simple to both implement and use, and the test itself is blindingly fast. There’s another problem, though, which is that we have technically left part of <code>reverseFile</code> untested: we’ve completely ignored the <code>path</code> argument.</p>
<p>In this contrived example, it may seem silly to test something so trivial, but in real code, it’s quite possible that one would care very much about testing multiple different aspects about a single function. When testing <code>renderUserProfile</code>, this was not hard, since we could reuse the same <code>TestM</code> type and <code>MonadDB</code> instance for both test cases, but in the <code>reverseFile</code> example, we’ve ignored the path entirely.</p>
<p>We <em>could</em> adjust our <code>MonadFS</code> instance to also track the path provided to each method, but this has a few problems. First, it means every test case would depend on all the various properties we are testing, which would mean updating every test case when we add a new one. It would also be simply impossible if we needed to track multiple types—in this particular case, it turns out that <code>String</code> and <code>FilePath</code> are actually the same type, but in practice, there may be a handful of disparate, incompatible types.</p>
<p>Both of the above issues could be fixed by creating a sum type and manually filtering out the relevant elements in each test case, but a much more intuitive approach would be to simply have a separate instance for each case. Unfortunately, in Haskell, creating a new instance means creating an entirely new type. To illustrate how much duplication that would entail, we could create the following type and instance for testing proper propagation of the <code>path</code> argument:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">newtype</span> <span class="kt">TestM&#39;</span> <span class="n">a</span> <span class="ow">=</span> <span class="kt">TestM&#39;</span> <span class="p">(</span><span class="kt">Writer</span> <span class="p">[</span><span class="kt">FilePath</span><span class="p">]</span> <span class="n">a</span><span class="p">)</span>
<span class="kr">deriving</span> <span class="p">(</span><span class="kt">Functor</span><span class="p">,</span> <span class="kt">Applicative</span><span class="p">,</span> <span class="kt">Monad</span><span class="p">,</span> <span class="kt">MonadWriter</span> <span class="p">[</span><span class="kt">FilePath</span><span class="p">])</span>
<span class="nf">logTestM&#39;</span> <span class="ow">::</span> <span class="kt">TestM&#39;</span> <span class="n">a</span> <span class="ow">-&gt;</span> <span class="p">[</span><span class="kt">FilePath</span><span class="p">]</span>
<span class="nf">logTestM&#39;</span> <span class="p">(</span><span class="kt">TestM&#39;</span> <span class="n">w</span><span class="p">)</span> <span class="ow">=</span> <span class="n">execWriter</span> <span class="n">w</span>
<span class="kr">instance</span> <span class="kt">MonadFS</span> <span class="kt">TestM&#39;</span> <span class="kr">where</span>
<span class="n">readFile</span> <span class="n">path</span> <span class="ow">=</span> <span class="n">tell</span> <span class="p">[</span><span class="n">path</span><span class="p">]</span> <span class="o">&gt;&gt;</span> <span class="n">return</span> <span class="s">""</span>
<span class="n">writeFile</span> <span class="n">path</span> <span class="kr">_</span> <span class="ow">=</span> <span class="n">tell</span> <span class="p">[</span><span class="n">path</span><span class="p">]</span>
</pre></div>
</div>
<p>Now it’s possible to add an extra test case that asserts that the proper path is provided to the two filesystem functions:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">spec</span> <span class="ow">=</span> <span class="n">describe</span> <span class="s">"reverseFile"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="n">it</span> <span class="s">"reverses a file’s contents on the filesystem"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="kr">let</span> <span class="n">calls</span> <span class="ow">=</span> <span class="n">logTestM</span> <span class="p">(</span><span class="n">reverseFile</span> <span class="s">"foo.txt"</span><span class="p">)</span>
<span class="n">calls</span> <span class="p">`</span><span class="n">shouldBe</span><span class="p">`</span> <span class="p">[</span><span class="s">"olleh"</span><span class="p">]</span>
<span class="n">it</span> <span class="s">"operates on the file at the provided path"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="kr">let</span> <span class="n">paths</span> <span class="ow">=</span> <span class="n">logTestM&#39;</span> <span class="p">(</span><span class="n">reverseFile</span> <span class="s">"foo.txt"</span><span class="p">)</span>
<span class="n">paths</span> <span class="p">`</span><span class="n">shouldBe</span><span class="p">`</span> <span class="p">[</span><span class="s">"foo.txt"</span><span class="p">,</span> <span class="s">"foo.txt"</span><span class="p">]</span>
</pre></div>
</div>
<p>This works, but it’s ultimately unacceptably complicated. Our test harness code is now significantly larger than the actual tests themselves, and the amount of boilerplate is frustrating. Verbose test suites are especially bad, since forcing programmers to jump through hoops just to implement a single test reduces the likelihood that people will actually write good tests, if they write tests at all. In contrast, if writing tests is easy, then people will naturally write more of them.</p>
<p>The above strategy to writing tests is not good enough, but it does reveal a particular problem: in Haskell, typeclass instances are not first-class values that can be manipulated and abstracted over, they are static constructs that can only be managed by the compiler, and users do not have a direct way to modify them. With some cleverness, however, we can actually create an approximation of first-class typeclass dictionaries, which will allow us to dramatically simplify the above testing mechanism.</p>
<h1 id="creating-first-class-typeclass-instances">Creating first-class typeclass instances</h1>
<p>In order to provide an easy way to construct instances, we need a way to represent instances as ordinary Haskell values. This is not terribly difficult, given that instances are conceptually just records containing a collection of functions. For example, we could create a datatype that represents an instance of the <code>MonadFS</code> typeclass:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">data</span> <span class="kt">MonadFSInst</span> <span class="n">m</span> <span class="ow">=</span> <span class="kt">MonadFSInst</span>
<span class="p">{</span> <span class="n">_readFile</span> <span class="ow">::</span> <span class="kt">FilePath</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="kt">String</span>
<span class="p">,</span> <span class="n">_writeFile</span> <span class="ow">::</span> <span class="kt">FilePath</span> <span class="ow">-&gt;</span> <span class="kt">String</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="nb">()</span>
<span class="p">}</span>
</pre></div>
</div>
<p>To avoid namespace clashes with the actual method identifiers, the record fields are prefixed with an underscore, but otherwise, the translation is remarkably straightforward. Using this record type, we can easily create values that represent the two instances we defined above:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">contentInst</span> <span class="ow">::</span> <span class="kt">MonadWriter</span> <span class="p">[</span><span class="kt">String</span><span class="p">]</span> <span class="n">m</span> <span class="ow">=&gt;</span> <span class="kt">MonadFSInst</span> <span class="n">m</span>
<span class="nf">contentInst</span> <span class="ow">=</span> <span class="kt">MonadFSInst</span>
<span class="p">{</span> <span class="n">_readFile</span> <span class="ow">=</span> <span class="nf">\</span><span class="kr">_</span> <span class="ow">-&gt;</span> <span class="n">return</span> <span class="s">"hello"</span>
<span class="p">,</span> <span class="n">_writeFile</span> <span class="ow">=</span> <span class="nf">\</span><span class="kr">_</span> <span class="n">contents</span> <span class="ow">-&gt;</span> <span class="n">tell</span> <span class="p">[</span><span class="n">contents</span><span class="p">]</span>
<span class="p">}</span>
<span class="nf">pathInst</span> <span class="ow">::</span> <span class="kt">MonadWriter</span> <span class="p">[</span><span class="kt">FilePath</span><span class="p">]</span> <span class="n">m</span> <span class="ow">=&gt;</span> <span class="kt">MonadFSInst</span> <span class="n">m</span>
<span class="nf">pathInst</span> <span class="ow">=</span> <span class="kt">MonadFSInst</span>
<span class="p">{</span> <span class="n">_readFile</span> <span class="ow">=</span> <span class="nf">\</span><span class="n">path</span> <span class="ow">-&gt;</span> <span class="n">tell</span> <span class="p">[</span><span class="n">path</span><span class="p">]</span> <span class="o">&gt;&gt;</span> <span class="n">return</span> <span class="s">""</span>
<span class="p">,</span> <span class="n">_writeFile</span> <span class="ow">=</span> <span class="nf">\</span><span class="n">path</span> <span class="kr">_</span> <span class="ow">-&gt;</span> <span class="n">tell</span> <span class="p">[</span><span class="n">path</span><span class="p">]</span>
<span class="p">}</span>
</pre></div>
</div>
<p>These two values represent two different implementations of <code>MonadFS</code>, but since they’re ordinary Haskell values, they can be manipulated and even <em>extended</em> like any other records. This can be extremely useful, since it makes it possible to create a sort of “base” instance, then have individual test cases override individual pieces of functionality piecemeal.</p>
<p>Of course, although we’ve written these two instances, we have no way to actually use them. After all, Haskell does not provide a way to explicitly provide typeclass dictionaries. Fortunately, we can create a sort of “proxy” type that will use a reader to thread the dictionary around explicitly, and the instance can defer to the dictionary’s implementation.</p>
<h2 id="creating-an-instance-proxy">Creating an instance proxy</h2>
<p>To represent our proxy type, we’ll use a combination of a <code>Writer</code> and a <code>ReaderT</code>; the former to implement the logging used by instances, and the latter to actually thread around the dictionary. Our type will look like this:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">newtype</span> <span class="kt">TestM</span> <span class="n">log</span> <span class="n">a</span> <span class="ow">=</span>
<span class="kt">TestM</span> <span class="p">(</span><span class="kt">ReaderT</span> <span class="p">(</span><span class="kt">MonadFSInst</span> <span class="p">(</span><span class="kt">TestM</span> <span class="n">log</span><span class="p">))</span> <span class="p">(</span><span class="kt">Writer</span> <span class="n">log</span><span class="p">)</span> <span class="n">a</span><span class="p">)</span>
<span class="kr">deriving</span> <span class="p">(</span> <span class="kt">Functor</span><span class="p">,</span> <span class="kt">Applicative</span><span class="p">,</span> <span class="kt">Monad</span>
<span class="p">,</span> <span class="kt">MonadReader</span> <span class="p">(</span><span class="kt">MonadFSInst</span> <span class="p">(</span><span class="kt">TestM</span> <span class="n">log</span><span class="p">))</span>
<span class="p">,</span> <span class="kt">MonadWriter</span> <span class="n">log</span>
<span class="p">)</span>
<span class="nf">logTestM</span> <span class="ow">::</span> <span class="kt">MonadFSInst</span> <span class="p">(</span><span class="kt">TestM</span> <span class="n">log</span><span class="p">)</span> <span class="ow">-&gt;</span> <span class="kt">TestM</span> <span class="n">log</span> <span class="n">a</span> <span class="ow">-&gt;</span> <span class="n">log</span>
<span class="nf">logTestM</span> <span class="n">inst</span> <span class="p">(</span><span class="kt">TestM</span> <span class="n">m</span><span class="p">)</span> <span class="ow">=</span> <span class="n">execWriter</span> <span class="p">(</span><span class="n">runReaderT</span> <span class="n">m</span> <span class="n">inst</span><span class="p">)</span>
</pre></div>
</div>
<p>This might look rather complicated, and it is, but let’s break down exactly what it’s doing.</p>
<ol>
<li>
<p>The <code>TestM</code> type includes two type parameters. The first is the type of value that will be logged (hence the name <code>log</code>), which corresponds to the argument to <code>Writer</code> from previous incarnations of <code>TestM</code>. Unlike those types, though, we want this version to work with any <code>Monoid</code>, so we’ll make it a type parameter. The second parameter is simply the type of the current monadic value, as before.</p></li>
<li>
<p>The type itself is defined as a wrapper around a small monad transformer stack, the first of which is <code>ReaderT</code>. The state threaded around by the reader is, in this case, the instance dictionary, which is <code>MonadFSInst</code>.</p>
<p> However, recall that <code>MonadFSInst</code> accepts a type variable—the type of a monad itself—so we must provide <code>TestM log</code> as an argument to <code>MonadFSInst</code>. This slight bit of indirection allows us to tie the knot between the mutually dependent instances and proxy type.</p></li>
<li>
<p>The base monad in the transformer stack is <code>Writer</code>, which is used to actually implement the logging functionality, just like in prior cases. The only difference now is that the <code>log</code> type parameter now determines what the writer actually produces.</p></li>
<li>
<p>Finally, as before, we use <code>GeneralizedNewtypeDeriving</code> to derive all the relevant <code>mtl</code> classes, adding the somewhat wordy <code>MonadReader</code> constraint to the list.</p></li></ol>
<p>Using this single type, we can now implement a <code>MonadFS</code> instance that defers to the dictionary carried around within <code>TestM</code>’s reader state:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">instance</span> <span class="kt">Monoid</span> <span class="n">log</span> <span class="ow">=&gt;</span> <span class="kt">MonadFS</span> <span class="p">(</span><span class="kt">TestM</span> <span class="n">log</span><span class="p">)</span> <span class="kr">where</span>
<span class="n">readFile</span> <span class="n">path</span> <span class="ow">=</span> <span class="kr">do</span>
<span class="n">f</span> <span class="ow">&lt;-</span> <span class="n">asks</span> <span class="n">_readFile</span>
<span class="n">f</span> <span class="n">path</span>
<span class="n">writeFile</span> <span class="n">path</span> <span class="n">contents</span> <span class="ow">=</span> <span class="kr">do</span>
<span class="n">f</span> <span class="ow">&lt;-</span> <span class="n">asks</span> <span class="n">_writeFile</span>
<span class="n">f</span> <span class="n">path</span> <span class="n">contents</span>
</pre></div>
</div>
<p>This may seem somewhat boilerplate-y, and it is to some extent, but the important consideration is that this boilerplate only needs to be written <em>once</em>. With this in place, it’s now possible to write an arbitrary number of first-class instances that use the above mechanism without extending the mechanism at all.</p>
<p>To see what actually using this code would look like, let’s update the <code>reverseFile</code> tests to use the new <code>TestM</code> implementation, as well as the <code>contentInst</code> and <code>pathInst</code> dictionaries from earlier:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">spec</span> <span class="ow">=</span> <span class="n">describe</span> <span class="s">"reverseFile"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="n">it</span> <span class="s">"reverses a file’s contents on the filesystem"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="kr">let</span> <span class="n">calls</span> <span class="ow">=</span> <span class="n">logTestM</span> <span class="n">contentInst</span> <span class="p">(</span><span class="n">reverseFile</span> <span class="s">"foo.txt"</span><span class="p">)</span>
<span class="n">calls</span> <span class="p">`</span><span class="n">shouldBe</span><span class="p">`</span> <span class="p">[</span><span class="s">"olleh"</span><span class="p">]</span>
<span class="n">it</span> <span class="s">"operates on the file at the provided path"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="kr">let</span> <span class="n">paths</span> <span class="ow">=</span> <span class="n">logTestM</span> <span class="n">pathInst</span> <span class="p">(</span><span class="n">reverseFile</span> <span class="s">"foo.txt"</span><span class="p">)</span>
<span class="n">paths</span> <span class="p">`</span><span class="n">shouldBe</span><span class="p">`</span> <span class="p">[</span><span class="s">"foo.txt"</span><span class="p">,</span> <span class="s">"foo.txt"</span><span class="p">]</span>
</pre></div>
</div>
<p>We can do a little bit better, though. Really, the definitions of <code>contentInst</code> and <code>pathInst</code> are specific to each test case. With ordinary typeclass instances, we cannot scope them to any particular block, but since <code>MonadFSInst</code> is just an ordinary Haskell datatype, we can manipulate them just like any other Haskell values. Therefore, we can just inline those instances’ definitions into the test cases themselves to keep them closer to the actual tests.</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">spec</span> <span class="ow">=</span> <span class="n">describe</span> <span class="s">"reverseFile"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="n">it</span> <span class="s">"reverses a file’s contents on the filesystem"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="kr">let</span> <span class="n">contentInst</span> <span class="ow">=</span> <span class="kt">MonadFSInst</span>
<span class="p">{</span> <span class="n">_readFile</span> <span class="ow">=</span> <span class="nf">\</span><span class="kr">_</span> <span class="ow">-&gt;</span> <span class="n">return</span> <span class="s">"hello"</span>
<span class="p">,</span> <span class="n">_writeFile</span> <span class="ow">=</span> <span class="nf">\</span><span class="kr">_</span> <span class="n">contents</span> <span class="ow">-&gt;</span> <span class="n">tell</span> <span class="p">[</span><span class="n">contents</span><span class="p">]</span>
<span class="p">}</span>
<span class="kr">let</span> <span class="n">calls</span> <span class="ow">=</span> <span class="n">logTestM</span> <span class="n">contentInst</span> <span class="p">(</span><span class="n">reverseFile</span> <span class="s">"foo.txt"</span><span class="p">)</span>
<span class="n">calls</span> <span class="p">`</span><span class="n">shouldBe</span><span class="p">`</span> <span class="p">[</span><span class="s">"olleh"</span><span class="p">]</span>
<span class="n">it</span> <span class="s">"operates on the file at the provided path"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="kr">let</span> <span class="n">pathInst</span> <span class="ow">=</span> <span class="kt">MonadFSInst</span>
<span class="p">{</span> <span class="n">_readFile</span> <span class="ow">=</span> <span class="nf">\</span><span class="n">path</span> <span class="ow">-&gt;</span> <span class="n">tell</span> <span class="p">[</span><span class="n">path</span><span class="p">]</span> <span class="o">&gt;&gt;</span> <span class="n">return</span> <span class="s">""</span>
<span class="p">,</span> <span class="n">_writeFile</span> <span class="ow">=</span> <span class="nf">\</span><span class="n">path</span> <span class="kr">_</span> <span class="ow">-&gt;</span> <span class="n">tell</span> <span class="p">[</span><span class="n">path</span><span class="p">]</span>
<span class="p">}</span>
<span class="kr">let</span> <span class="n">paths</span> <span class="ow">=</span> <span class="n">logTestM</span> <span class="n">pathInst</span> <span class="p">(</span><span class="n">reverseFile</span> <span class="s">"foo.txt"</span><span class="p">)</span>
<span class="n">paths</span> <span class="p">`</span><span class="n">shouldBe</span><span class="p">`</span> <span class="p">[</span><span class="s">"foo.txt"</span><span class="p">,</span> <span class="s">"foo.txt"</span><span class="p">]</span>
</pre></div>
</div>
<p>This is pretty good. We’re now able to create inline instances of our <code>MonadFS</code> typeclass, which allows us to write extremely concise unit tests using ordinary Haskell typeclasses as system seams. We’ve managed to cut down on the boilerplate considerably, though we still have a couple problems. For one, this example only uses a single typeclass containing only two methods. A real <code>MonadFS</code> typeclass would likely have at least a dozen methods for performing various filesystem operations, and writing out the instance dictionaries for every single method, even the ones that aren’t used within the code under test, would be pretty frustratingly verbose.</p>
<p>This problem is solvable, though. Since instances are just ordinary Haskell records, we can create a “base” instance that just throws an exception whenever the method is called:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="nf">baseInst</span> <span class="ow">::</span> <span class="kt">MonadFSInst</span> <span class="n">m</span>
<span class="nf">baseInst</span> <span class="ow">=</span> <span class="kt">MonadFSInst</span>
<span class="p">{</span> <span class="n">_readFile</span> <span class="ow">=</span> <span class="ne">error</span> <span class="s">"unimplemented instance method ‘_readFile’"</span>
<span class="p">,</span> <span class="n">_writeFile</span> <span class="ow">=</span> <span class="ne">error</span> <span class="s">"unimplemented instance method ‘_writeFile’"</span>
<span class="p">}</span>
</pre></div>
</div>
<p>Then code that only uses <code>readFile</code> could only override that particular method, for example:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">let</span> <span class="n">myInst</span> <span class="ow">=</span> <span class="n">baseInst</span> <span class="p">{</span> <span class="n">_readFile</span> <span class="ow">=</span> <span class="o">...</span> <span class="p">}</span>
</pre></div>
</div>
<p>Normally, of course, this would be a terrible idea. However, since this is all just test code, it can be extremely useful in quickly figuring out what methods need to be stubbed out for a particular test case. Since all the code actually gets run at test time, attempts to use unimplemented instance methods will immediately raise an error, informing the programmer which methods need to be implemented to make the test pass. This can also help to significantly cut down on the amount of effort it takes to implement each test.</p>
<p>Another problem is that our approach is specialized exclusively to <code>MonadFS</code>. What about functions that use both <code>MonadFS</code> <em>and</em> <code>MonadDB</code>, for example? Fortunately, that is not hard to solve, either. We can adapt the <code>MonadFSInst</code> type to include fields for all of the typeclasses relevant to our system, turning it into a generic test fixture of sorts:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">data</span> <span class="kt">FixtureInst</span> <span class="n">m</span> <span class="ow">=</span> <span class="kt">FixtureInst</span>
<span class="p">{</span> <span class="c1">-- MonadFS</span>
<span class="n">_readFile</span> <span class="ow">::</span> <span class="kt">FilePath</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="kt">String</span>
<span class="p">,</span> <span class="n">_writeFile</span> <span class="ow">::</span> <span class="kt">FilePath</span> <span class="ow">-&gt;</span> <span class="kt">String</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="nb">()</span>
<span class="c1">-- MonadDB</span>
<span class="p">,</span> <span class="n">_fetchUser</span> <span class="ow">::</span> <span class="kt">Id</span> <span class="kt">User</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="kt">User</span>
<span class="p">,</span> <span class="n">_fetchRecentPosts</span> <span class="ow">::</span> <span class="kt">Id</span> <span class="kt">User</span> <span class="ow">-&gt;</span> <span class="n">m</span> <span class="p">[</span><span class="kt">Post</span><span class="p">]</span>
<span class="p">}</span>
</pre></div>
</div>
<p>Updating <code>TestM</code> to use <code>FixtureInst</code> instead of <code>MonadFSInst</code> is trivial, and all the rest of the infrastructure still works. However, this means that every time a new typeclass is added, three things need to be updated:</p>
<ol>
<li>Its methods need to be added to the <code>FixtureInst</code> record.</li>
<li>Those methods need to be given error-raising defaults in the <code>baseInst</code> value.</li>
<li>An actual instance of the typeclass needs to be written for <code>TestM</code> that defers to the <code>FixtureInst</code> value.</li></ol>
<p>Furthermore, most of this manual manipulation of methods is required every time a particular typeclass changes, whether that means adding a method, removing a method, renaming a method, or changing a method’s type. This is especially frustrating given that all this code is really just mechanical boilerplate that could all be derived by the set of typeclasses being tested.</p>
<p>That last point is especially important: aside from the instances themselves, every piece of boilerplate above is obviously possible to generate from existing types alone. With that piece of information in mind, we can do even better: we can use Template Haskell.</p>
<h1 id="removing-the-boilerplate-using-test-fixture">Removing the boilerplate using <code>test-fixture</code></h1>
<p>The above code was not only rather boilerplate-heavy, it was pretty complicated. Fortunately, you don’t actually have to write it. Enter the library <a href="http://hackage.haskell.org/package/test-fixture"><code>test-fixture</code></a>:</p>
<div class="brush: haskell">
<div class="source">
<pre><span></span><span class="kr">import</span> <span class="nn">Control.Monad.TestFixture</span>
<span class="kr">import</span> <span class="nn">Control.Monad.TestFixture.TH</span>
<span class="nf">mkFixture</span> <span class="s">"FixtureInst"</span> <span class="p">[</span><span class="kt">&#39;&#39;MonadFS</span><span class="p">,</span> <span class="kt">&#39;&#39;MonadDB</span><span class="p">]</span>
<span class="nf">spec</span> <span class="ow">=</span> <span class="n">describe</span> <span class="s">"reverseFile"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="n">it</span> <span class="s">"reverses a file’s contents on the filesystem"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="kr">let</span> <span class="n">contentInst</span> <span class="ow">=</span> <span class="n">def</span>
<span class="p">{</span> <span class="n">_readFile</span> <span class="ow">=</span> <span class="nf">\</span><span class="kr">_</span> <span class="ow">-&gt;</span> <span class="n">return</span> <span class="s">"hello"</span>
<span class="p">,</span> <span class="n">_writeFile</span> <span class="ow">=</span> <span class="nf">\</span><span class="kr">_</span> <span class="n">contents</span> <span class="ow">-&gt;</span> <span class="n">log</span> <span class="n">contents</span>
<span class="p">}</span>
<span class="kr">let</span> <span class="n">calls</span> <span class="ow">=</span> <span class="n">logTestFixture</span> <span class="p">(</span><span class="n">reverseFile</span> <span class="s">"foo.txt"</span><span class="p">)</span> <span class="n">contentInst</span>
<span class="n">calls</span> <span class="p">`</span><span class="n">shouldBe</span><span class="p">`</span> <span class="p">[</span><span class="s">"olleh"</span><span class="p">]</span>
<span class="n">it</span> <span class="s">"operates on the file at the provided path"</span> <span class="o">$</span> <span class="kr">do</span>
<span class="kr">let</span> <span class="n">pathInst</span> <span class="ow">=</span> <span class="n">def</span>
<span class="p">{</span> <span class="n">_readFile</span> <span class="ow">=</span> <span class="nf">\</span><span class="n">path</span> <span class="ow">-&gt;</span> <span class="n">log</span> <span class="n">path</span> <span class="o">&gt;&gt;</span> <span class="n">return</span> <span class="s">""</span>
<span class="p">,</span> <span class="n">_writeFile</span> <span class="ow">=</span> <span class="nf">\</span><span class="n">path</span> <span class="kr">_</span> <span class="ow">-&gt;</span> <span class="n">log</span> <span class="n">path</span>
<span class="p">}</span>
<span class="kr">let</span> <span class="n">paths</span> <span class="ow">=</span> <span class="n">logTestFixture</span> <span class="p">(</span><span class="n">reverseFile</span> <span class="s">"foo.txt"</span><span class="p">)</span> <span class="n">pathInst</span>
<span class="n">paths</span> <span class="p">`</span><span class="n">shouldBe</span><span class="p">`</span> <span class="p">[</span><span class="s">"foo.txt"</span><span class="p">,</span> <span class="s">"foo.txt"</span><span class="p">]</span>
</pre></div>
</div>
<p><strong>That’s it.</strong> The above code automatically generates everything you need to write fast, simple, deterministic unit tests in Haskell. The <code>mkFixture</code> function is a Template Haskell macro that expands into a definition quite similar to the <code>FixtureInst</code> type we wrote by hand, but since it’s automatically generated from the typeclass definitions, it never needs to be updated.</p>
<p>The <code>logTestFixture</code> function replaces the <code>logTestM</code> function we wrote by hand, but it works exactly the same. The <code>Control.Monad.TestFixture</code> library also exports a <code>log</code> function that is a synonym for <code>tell . singleton</code>, but using <code>tell</code> directly still works if you prefer.</p>
<p>The <code>mkFixture</code> function also generates a <code>Default</code> instance, which replaces the <code>baseInst</code> value defined earlier. It functions the same way, though, producing useful error messages that refer to the names of unimplemented typeclass methods that have not been stubbed out.</p>
<p>This blog post is not a <code>test-fixture</code> tutorial—indeed, it is much more complicated than a <code>test-fixture</code> tutorial would be, since it covers what the library is really doing under the hood—but if you’re interested, I would highly recommend you take a look at the <a href="http://hackage.haskell.org/package/test-fixture"><code>test-fixture</code> documentation on Hackage</a>.</p>
<h1 id="conclusion-credits-and-similar-techniques">Conclusion, credits, and similar techniques</h1>
<p>This blog post came about as the result of a need my coworkers and I found when writing Haskell code; we wanted a way to write unit tests quickly and easily, but we didn’t find much advice from the rest of the Haskell ecosystem. The <code>test-fixture</code> library is the result of that exploratory work, and we currently use it to test a significant portion of our Haskell code.</p>
<p>It would be extremely unfair to suggest that I was the inventor of this technique or the inventor of the library. Two of my coworkers, <a href="https://github.com/jxv">Joe Vargas</a> and <a href="https://github.com/aztecrex">Greg Wiley</a>, came up with the general approach and wrote <code>Control.Monad.TestFixture</code>, and I simply wrote the Template Haskell macro to eliminate the boilerplate. With that in mind, I think I can say with some fairness that I think this technique is a joy to use when unit testing is a desirable goal, and I would definitely recommend it if you are interested in doing isolated testing in Haskell.</p>
<p>The general technique of using typeclasses to emulate effects was in part inspired by the well-known <code>mtl</code> library. An alternate approach to writing unit-testable Haskell code is using free monads, but overall, I prefer this approach over free monads because the typeclass constraints add type safety in ways that free monads do not (at least not without additional boilerplate), and this approach also lends itself well to static analysis-based boilerplate reduction techniques. It has its own tradeoffs, though, so if you’ve had success with free monads, then I certainly make no claim this is a superior approach, just one that I’ve personally found pleasant.</p>
<p>As a final note, if you <em>do</em> check out <code>test-fixture</code>, feel free to leave feedback by opening issues on <a href="https://github.com/cjdev/test-fixture/issues">the GitHub issue tracker</a>—even things like confusing documentation are worth a bug report.</p></description></item></channel></rss>