Crunchy Development

YAML Superpowers, part 2: Multiline Strings

Thu, 19 Aug 2021 00:00:00 +0000

This is part 2 of a series of post about lesser-known features of YAML – especially the ones useful in contexts like CI or tools’ config files. In this post we will discover the many ways YAML can represent strings, including multiline strings, keeping or stripping indentation, and more!

Missed part 1? head here!

Simple strings (“Flow” scalars)

Let’s start easy with single-line strings. It’s already worth noting that, in YAML:

When using double quotes "" around your strings, it allows you to also escape some special characters using the \ sigil inside that string (see below).
When you use single quotes '', the string won’t be escaped. This means that every character will be interpreted verbatim – including \. The only exception is if you need to include a literal ' inside a single-quoted string, in which case you can repeat it (i.e. use '' inside the single-quoted string)
Quotes can be omitted around strings, unless it’s ambiguous to do so. Omiting quotes is the same as using single quotes (i.e. characters won’t be escaped)

When you use double-quotes, you can then use escape codes for special characters, like the following ones:

"\x12", "\u1234" or "\U00102030" for 8, 16 and 32 bits unicode codepoints respectively
"\\" (same as unescaped '\') and "\"" (same as '"')
"\r" and "\n" for CR and LF, respectively, and "\t" for tab
"\_" for a non-breaking space (NBSP)
You can also put a \ at the end of a line (before your newline character) to escape that newline and thus ignore it. This is a nice trick for manually wrapping long strings, though the block scalar syntax below is usually way nicer to use.
And some more (see the spec or refcard)

This means that the strings in the following array of strings are all valid:

 - Hello
 - 'Hello'
 - "Hello"
 - '"Hello", they said.'
 - '# this is not a comment but a string'
 - "Smile \u263A"
 - "column1\tcolumn2\tcolumn3\n"
 - "\x0d\x0a is the same as \r\n, namely CRLF"
 - 'Single quotes means \ is not an escape character ¯\_(ツ)_/¯' 
 - 'Except there''s still a way to include the single quote character into those.'

Multi-line blocks and long strings

YAML also supports writing strings on multiple lines in your YAML, with two different styles: literal and folded. Those are represented by a | (for literal style) or > (for folded style) header line (see examples below), and we will go into their differences soon.

But first, note that any leading indentation in those multi-line blocks is stripped by YAML, so you don’t have to worry about it; in fact, let’s talk a bit about indentation first before going further.

Indentation in multi-line blocks

In a multi-line block, YAML first determines the indentation level of the whole block:

Either by guessing it from the number of leading spaces of the first non-empty line, if you don’t specify anything (most common case)
Or by using the explicit value you provide after the | or >, to indicate the number of additional spaces compared to the parent node.

The leading spaces corresponding to (implicit or explicit) indentation are then ignored in the final interpreted string.

multi-line-yaml-blocks:
  examples:
    string1: >
      This is a long string, where the indentation
      was guessed to be 6 leading spaces (aka 2 more
      leading spaces than the parent node). Also note that
      this long string will be folded into a single-line string with no newline.
    string2: >2
         This is another long string, where the indentation was explicitly
      indicated to be 2 more spaces than the parent node.
         Which means that the 1st and 3rd lines, which happens to start with
      5 additional leading spaces relative to the parent node's existing indentation,
      will start with 3 literal space characters (as the first 2 are the indentation).

In most cases you’ll use implicit indentation (I’ve rarely seen the explicit indentation value being used in concrete use cases to be honest), but it’s good to know you can make it explicit if needed, especially if your string is supposed to start with spaces that are not to be interpreted as indentation.

Literal style: `|` – keep the newlines

When using the block syntax, the literal style, denoted by | header, is the simplest.

As the name suggests, it keeps the content as is, including rendering newlines as actual newlines in the end content. The only thing that is stripped is the leading indentation.

This is most useful when you want to embed strings with multiple lines in your YAML; a typical example being the content of a bash script with multiple commands defined in a step of your CI config file.

steps:
  - label: "Build the app"
    key: "build"
    command: |
      echo "--- Install gems"
      bundle install
      echo "--- Build the app"
      bundle exec fastlane build

Folded style: `>` – fold the newlines into spaces

On the other hand, the folded style, denoted by a > header, replaces newlines by a space character – unless it ends an empty or more-indented line.

This is very useful when you want to break long lines of text into smaller ones (i.e. manual hard-wrapping) for readability. Note that empty (or all-spaces) lines, as well as more-indented lines, are not affected by this folding.

notify:
  - slack:
      channels: ["#build-notifs"]
      message: >
        Your build have failed. You might want to check your
        CI logs for more details about the failure, or ping
        your friendly neighbourhood Infrastructure Engineer
        on call to ask for help.
    if: build.state == "failed"

Chomp mode

In addition to the style (literal | or folded >), and optional explicit indentation number, you can also specify a chomp mode in the block header. The chomp mode defines how YAML will interpret trailing newlines and spaces at the end of your block, and can be one of the following:

“Clip” mode: this is the default behavior, the one used when you don’t specify any specific chomp indicator in your header. In this mode, the final newline is preserved, but any additional trailing empty line are ignored.
“Strip” mode: indicated by a - in the block header, this will strip not only trailing empty lines like in clip mode, but also the final newline at the end of the string.
“Keep” mode: indicated by a + in the block header, this will keep both the final newline and any potential trailing empty lines too.

examples:
  clip: >
    This content will end with a LF character
    but not include the final empty lines.
    
    
  strip: >-
    This content will neither contain a trailing LF character
    nor the trailing empty line.
    
  keep: >+
    This content will keep both the LF
    and the trailing empty lines.
    
    
equivalent-output:
  clip: "This content will end with a LF character but not include the final empty lines.\n"
  strip: "This content will neither contain a trailing LF character nor the trailing empty line."
  keep: "This content will keep both the LF and the trailing empty lines.\n\n\n"

Optional block Comment

So every block syntax starts with a header (with | or >, optional indentation number, and optional chomp indicator). The only other thing that is also allowed on that header line is a single-line # comment; the rest (the content of the block literal) has to start on the next line.

Adding a single-line comment can be nice to provide some context:

notify:
  - slack:
      channels: ["#build-notifs"]
      message: > # This message will be folded, i.e. newlines will be replaced by spaces.
        Your build have failed. You might want to check your
        CI logs for more details about the failure, or ping
        your friendly neighbourhood Infrastructure Engineer
        on call to ask for help.
    if: build.state == "failed"

Conclusion

And here you thought strings would be a pretty simple topic in YAML… 😅

In this post we saw how YAML allows us to easily represent long strings, as well as strings containing special characters and newlines, without having to escape all the newlines manually, and making those long, potentially-multiline strings to be way more readable.

I hope you learned something and you will be able to put this to good use in your YAML config files 😊

Next up will be an even more powerful and lesser-known concept of YAML: anchors and aliases. Meet me in part 3 to discover how you can avoid repeating yourself, and how to mutualise common, repeated values in your YAML files!

YAML Superpowers, part 1: JSON is YAML

Tue, 17 Aug 2021 00:00:00 +0000

This post is part 1 of a series of posts where I plan to focus on little-known features of YAML like multiline string processing, aliases and anchors, base64 support, tags and more.

Why all those posts? YAML is not that complicated!

Amongst other things, YAML is widely used as a configuration format for different tools, including configuration files for tools like SwiftLint or SwiftGen, but also notoriously to configure most of the Continuous Integration providers, like GitHub Actions, CircleCI, BuildKite, and many more.

But most people only know the basics of YAML, while many less-known features of this format are powerful and could prove very useful to improve your config files readability and ease of maintenance.

In this series, I’ll focus on examples related to configuring a CI like BuildKite (because that’s what we are currently migrating to where I work at Automattic) but those features apply to other CI providers and use cases of YAML.

Back to Basics

I’m sure most of you already know the basics of the YAML syntax; they are not that complicated after all. But even about the basics, it’s worth making sure we’re all on the same page first:

YAML is a format to represent structured data. It can especially represent dictionaries (what YAML calls “maps”), lists/arrays (what YAML calls “sequences”), and literals (like strings, numbers, booleans, …)
One thing that most people might not know is that YAML is a superset of JSON. This means that any JSON is a valid YAML file! YAML just extends the JSON syntax to provide more features (like comments etc) and alternatives to represent the same data structures.

For example, in YAML, you often see a list (aka “sequence” in YAML parlance) represented like this:

 - "item1"
 - "item2"
 - "item3"
 - "etc"

But did you know you could also represent a list using the JSON “square brackets” syntax too?

["item1", "item2", "item3", "etc"]

That’s right, that syntax is the same as what you use to represent arrays in JSON. And it’s no coincidence, because JSON is a subset of YAML, so that’s also valid YAML, and both syntaxes are interchangeable in YAML!

Likewise, you often see dictionaries (aka “maps” in YAML parlance) represented like this in most YAML files:

"key1": "value1"
"key2": "value2"
"key3": "value3"

But another, totally valid way to represent a map in YAML is to use this alternative syntax, which is the syntax you’re already used to in JSON:

{ "key1": "value1", "key2": "value2", "key3": "value3" }

Both those syntaxes represent the same thing and are interchangeable. You can even write your file with purely JSON-compatible syntax and add it a .yml extension, and that would be accepted by any YAML parser. Just try it: take any JSON file that you might have around, and paste its content to a YAML linter and it will gladly accept it!

What makes YAML usually more attractive as a config file format over JSON is that those alternative syntaxes are meant to make the structure more human-readable than JSON (by representing lists a bit like bullet points, etc), while JSON is meant to be more machine-oriented.

The fact that it also allows adding comments, unlike JSON¹, and that quotes around strings are optional², also helps make your YAML config files easier to read and write for a human.

A concrete example: a CI config

At that point I feel it can also be useful to take a concrete example, especially because when used in the context of CI config files, YAML structures can become a bit more complex and specific.

For example, it’s common in most CIs to have YAML nodes that appear as “lists of single-key dictionaries”, which might be confusing at first – and not always straighforward to realize what those really are at first glance. Those look like regular dictionaries, but are not. This pattern of “sequence of single-key dictionaries” is in fact the way YAML represents an ordered map (while a regular dictionary/map is unordered by definition). Here’s an example:

steps:
  - label: "Build the app"
    key: build
    plugins:
      - automattic/bash-cache#v1.5.0:
          bucket: "a8c-cache"
      - automattic/git-s3-cache#v1.1.0:
          bucket: "a8c-repo-mirrors"
          repo: "wordpress-mobile/wordpress-ios/"
    env:
      IMAGE_ID: xcode-12.5.1
    command: "bundle exec fastlane build_for_testing"
  - label: "Run Tests"
    key: test
    plugins:
      - automattic/bash-cache#v1.5.0
      - automattic/git-s3-cache#v1.1.0:
          bucket: "a8c-repo-mirrors"
          repo: "wordpress-mobile/wordpress-ios/"
    env:
      IMAGE_ID: xcode-12.5.1
    command: "bundle exec fastlane tests"

This extract of a typical BuildKite config file³ defines:

A top-level dictionary with the key steps. The value associated with that key is an array of 2 elements – as you can see by the two - that are at indentation level 2
Each of these 2 items is a dictionary, which both happen to contain the same 5 keys: label, key, plugins, env and command.
The value for the label, key and command keys are strings in both cases.
The value of the env key is itself another dictionary (map) to define a list of environment variables for that step; in our case we only define a single IMAGE_ID key/env var, be we could have added more there.
The value of the plugins key is what might look the most unusual, as it is such a so-called ordered map, aka an array of single-key dictionaries.
- In fact, for the first “step” described in this YAML, the value for plugins is an array of 2 items, each of them being a dictionary with only a single key – so deep down it’s in fact 2 single-key dictionaries and not a single dictionary with 2 keys as one might think… even if in practice for all intents and purposes you will probably read it as an ordered dictionary with 2 keys for interpretation of the config file.
- The first single-key dictionary has the key automattic/bash-cache#v1.5.0, and its value is yet another dictionary (intended to define the “options” of that “plugin”) which happens to only have the bucket: key
- The second single-key dictionary has the key automattic/git-s3-cache#v1.1.0 and its value is yet another dictionary, this time with 2 keys.
- For the second step though (the one to configure the test step), the value of the plugins key is in fact an array of mixed values, the first one being a single string, while the second one being a single-key dictionary like in the first step.

💡 More comfortable with JSON? Open to see the same structure using only JSON-compatible syntax.

{
  "steps": [
    {
      "label": "Build the app",
      "key": "build",
      "plugins": [
        {
          "automattic/bash-cache#v1.5.0": {
            "bucket": "a8c-cache"
          }
        },
        {
          "automattic/git-s3-cache#v1.1.0": {
            "bucket": "a8c-repo-mirrors",
            "repo": "wordpress-mobile/wordpress-ios/"
          }
        }
      ],
      "env": {
        "IMAGE_ID": "xcode-12.5.1"
      },
      "command": "bundle exec fastlane build_for_testing"
    },
    {
      "label": "Run Tests",
      "key": "test",
      "plugins": [
        "automattic/bash-cache#v1.5.0",
        {
          "automattic/git-s3-cache#v1.1.0": {
            "bucket": "a8c-repo-mirrors",
            "repo": "wordpress-mobile/wordpress-ios/"
          }
        }
      ],
      "env": {
        "IMAGE_ID": "xcode-12.5.1"
      },
      "command": "bundle exec fastlane tests"
    }
  ]
}

These 2 ways of listing the various plugins in a CI step is actually common in most CIs (this is an example from a BuildKite config file, but e.g. CircleCI has similar use cases of arrays of mixed types, with Strings and single-key dictionaries too).

This is a common way in YAML to describe an ordered list of items (here BuildKite plugins) while allowing some of them to define “options” (by making the plugin “name” be the key of a single-key dictionary, and providing the options as the value for that key), while others might not need any option (and most CI config syntaxes allow you to use simple strings mentioning the plugin “name” for those cases instead of single-key dict with no value⁴).

Already having your head spinning a bit with that not-so-basic example 😅? Yeah, that’s why I wanted to make sure we were all on the same page with “basic” syntax – because even that can lead to twisted YAML structures as you saw – even before going further to introduce lesser known YAML features 😉.

What’s Next: Advanced features of YAML

Ready to go further? That’s the whole goal of this article series after all! 😁

In addition to all that above, YAML also provides quite powerful features that will be the focus of the next parts of this article series 🥳. Here’s our program:

Multiline strings (part 2), including various ways to process indentation
Anchors & aliases (part 3), to avoid repeating yourself
Merging dictionaries (part 4), which is especially useful when used with anchors
Hex/Binary/Octal numbers, Booleans and the Null value (part 5)
Tags (part 6), to add type information and help disambiguate values
Base64 representation of binary data (part 7)

To be honest, parts 2,3 and 4 are probably the ones that will be the most useful for CI configs, and the rest is mostly some bonus 😜. There’s even more to YAML⁵, but this will hopefully already give you an extensive-enough tour to what will likely be the features you might find the most useful in the context of using YAML as config files.

See you in part 2!

Technically JSON5, which is a successor of JSON, does support comments; but the more common JSON that you’ve seen around for ages and that we still find in most places doesn’t. ↩
This is why in most cases people omit quotes in both single string literals, but also in arrays of strings and dictionary keys. Quotes are still required/useful if disambiguation is needed, e.g. using "42" to represent the string “42” as opposed to the number 42. ↩
Borrowed from our Wordpress-iOS source code here ↩
In most CIs, including BuildKite, it would also be valid to use a single-key dictionary with a null value for items which don’t need options to be provided as values, instead of using a single String. So we could have used - automattic/bash-cache#v1.5.0: null as well here. But let’s keep the talk about null values for part 5 of this article series ☺️. ↩
I’ll probably not go into preprocessor directives, or defining your own tags, or putting multiple YAML documents into a single YAML file, etc… because they are way less useful to the context of using YAML as CI or tools’ config files, but feel free to search for those feature if you’re curious about YAML in general! ↩

StringInterpolation in Swift 5 — AttributedStrings

Sun, 16 Dec 2018 00:00:00 +0000

In the previous post, we introduced the new StringInterpolation design coming to Swift 5. In this second part, I’ll focus on one application of that new ExpressibleByStringInterpolation, to make NSAttributedString prettier.

The goal

One of the first application I thought about when seeing that new StringInterpolation design in Swift 5 was to make it easy to build an NSAttributedString.

My goal was to be able to create an attributed string using a syntax like this¹:

let username = "AliGator"
let str: AttrString = """
  Hello \(username, .color(.red)), isn't this \("cool", .color(.blue), .oblique, .underline(.purple, .single))?

  \(wrap: """
    \(" Merry Xmas! ", .font(.systemFont(ofSize: 36)), .color(.red), .bgColor(.yellow))
    \(image: #imageLiteral(resourceName: "santa.jpg"), scale: 0.2)
    """, .alignment(.center))

  Go there to \("learn more about String Interpolation", .link("https://github.com/apple/swift-evolution/blob/master/proposals/0228-fix-expressiblebystringinterpolation.md"), .underline(.blue, .single))!
  """

That big String uses the multi-line string literals syntax (new in Swift 4.0, in case you missed it) — and even goes as far as wrapping another multi-line String literal inside another (see the \(wrap: …) segment)! — and contains interpolations to add some styling to parts of that big String… so a lot of new features of Swift coming together!

The result NSAttributedString, once rendered in a UILabel or NSTextView, should then look like this:

☝️ Yes, that above with the text and image… is really just an NSAttributedString (and not a complex view with layout or anything)! 🤯

First implementation

So how are we going to implement this? Well, similar to how we implemented GitHubComment in part 1 of course!

So, let’s start with the declaration of the dedicated type first, before even tackling string interpolation:

struct AttrString {
  let attributedString: NSAttributedString
}

extension AttrString: ExpressibleByStringLiteral {
  init(stringLiteral: String) {
    self.attributedString = NSAttributedString(string: stringLiteral)
  }
}

extension AttrString: CustomStringConvertible {
  var description: String {
    return String(describing: self.attributedString)
  }
}

Simple enough, right? That’s just a wrapper around NSAttributedString. Now, let’s add support for ExpressibleByStringInterpolation that would allow both literals but also strings annotated with NSAttributedString attributes:

extension AttrString: ExpressibleByStringInterpolation {
  init(stringInterpolation: StringInterpolation) {
    self.attributedString = NSAttributedString(attributedString: stringInterpolation.attributedString)
  }

  struct StringInterpolation: StringInterpolationProtocol {
    var attributedString: NSMutableAttributedString

    init(literalCapacity: Int, interpolationCount: Int) {
      self.attributedString = NSMutableAttributedString()
    }

    func appendLiteral(_ literal: String) {
      let astr = NSAttributedString(string: literal)
      self.attributedString.append(astr)
    }

    func appendInterpolation(_ string: String, attributes: [NSAttributedString.Key: Any]) {
      let astr = NSAttributedString(string: string, attributes: attributes)
      self.attributedString.append(astr)
    }
  }
}

At that stage, we’re already able to use it that way to easily build an NSAttributedString:

let user = "AliSoftware"
let str: AttrString = """
  Hello \(user, attributes: [.foregroundColor: NSColor.blue])!
  """

That’s already nice as it is, right?

Adding styling convenience

But dealing with attributes as a dictionary [NAttributedString.Key: Any] isn’t really nice. Especially since that Any isn’t typed, and forces us to know the expected type of the value for each key…

So let’s make that nicer by creating a dedicated Style type² to help us building attributes dictionaries:

extension AttrString {
  struct Style {
    let attributes: [NSAttributedString.Key: Any]
    static func font(_ font: NSFont) -> Style {
      return Style(attributes: [.font: font])
    }
    static func color(_ color: NSColor) -> Style {
      return Style(attributes: [.foregroundColor: color])
    }
    static func bgColor(_ color: NSColor) -> Style {
      return Style(attributes: [.backgroundColor: color])
    }
    static func link(_ link: String) -> Style {
      return .link(URL(string: link)!)
    }
    static func link(_ link: URL) -> Style {
      return Style(attributes: [.link: link])
    }
    static let oblique = Style(attributes: [.obliqueness: 0.1])
    static func underline(_ color: NSColor, _ style: NSUnderlineStyle) -> Style {
      return Style(attributes: [
        .underlineColor: color,
        .underlineStyle: style.rawValue
      ])
    }
    static func alignment(_ alignment: NSTextAlignment) -> Style {
      let ps = NSMutableParagraphStyle()
      ps.alignment = alignment
      return Style(attributes: [.paragraphStyle: ps])
    }
  }
}

This allows us to use Style.color(.blue) to create a Style wrapping [.foregroundColor: NSColor.blue] easily.

But let’s not stop there then, and make our StringInterpolation handle such Style attributes now!

So the idea is to be able to write this:

let str: AttrString = """
  Hello \(user, .color(.blue)), how do you like this?
  """

Wouldn’t it be nice? Well, let’s just implement the right appendInterpolation for that!

extension AttrString.StringInterpolation {
  func appendInterpolation(_ string: String, _ style: AttrString.Style) {
    let astr = NSAttributedString(string: string, attributes: style.attributes)
    self.attributedString.append(astr)
  }

And here you have it! But… this only supports one Style at a time then. Why not allow passing multiple Style as parameters there? And to do so, instead of allowing a [Style] parameter, forcing us to wrap the list of styles in brackets at call site… why not use variadic parameters here?

So instead of the previous implementation, let’s instead implement it that way:

extension AttrString.StringInterpolation {
  func appendInterpolation(_ string: String, _ style: AttrString.Style...) {
    var attrs: [NSAttributedString.Key: Any] = [:]
    style.forEach { attrs.merge($0.attributes, uniquingKeysWith: {$1}) }
    let astr = NSAttributedString(string: string, attributes: attrs)
    self.attributedString.append(astr)
  }
}

And now we can mix multiple styles!

let str: AttrString = """
  Hello \(user, .color(.blue), .underline(.red, .single)), how do you like this?
  """

Supporting Images

Another capability of NSAttributedString is to add images as part of the string, by using NSAttributedString(attachment: NSTextAttachment). To do that, it’s just a matter of implementing appendInterpolation(image: NSImage) to use it.

For that feature, I wanted to add the ability to rescale the image as well. And because I tried all that in a macOS playground, which has its graphic context flipped, I also had to draw the image flipped. (Note that this detail might be different for the iOS implementation supporting UIImage). So here’s what I came up with:

extension AttrString.StringInterpolation {
  func appendInterpolation(image: NSImage, scale: CGFloat = 1.0) {
    let attachment = NSTextAttachment()
    let size = NSSize(
      width: image.size.width * scale,
      height: image.size.height * scale
    )
    attachment.image = NSImage(size: size, flipped: false, drawingHandler: { (rect: NSRect) -> Bool in
      NSGraphicsContext.current?.cgContext.translateBy(x: 0, y: size.height)
      NSGraphicsContext.current?.cgContext.scaleBy(x: 1, y: -1)
      image.draw(in: rect)
      return true
    })
    self.attributedString.append(NSAttributedString(attachment: attachment))
  }
}

Wrapping styles in one another

Finally, sometimes, you want to apply a style to a large section of text, which itself might contain styles inside subsections of that text. Like "<b>Hello <i>world</i></b>" in HTML where the whole section is bold but contains sub-parts in oblique.

Our API doesn’t support that yet, so let’s add it. The idea there is to be able to apply a list of Style... to something that’s not just a String but already an AttrString with already existing attributes.

The implementation will be similar to appendInterpolation(_ string: String, _ style: Style...), but will mutate the AttrString.attributedString to add attributes to it, intead of creating a brand new NSAttributedString from a plain String.

extension AttrString.StringInterpolation {
 func appendInterpolation(wrap string: AttrString, _ style: AttrString.Style...) {
    var attrs: [NSAttributedString.Key: Any] = [:]
    style.forEach { attrs.merge($0.attributes, uniquingKeysWith: {$1}) }
    let mas = NSMutableAttributedString(attributedString: string.attributedString)
    let fullRange = NSRange(mas.string.startIndex..<mas.string.endIndex, in: mas.string)
    mas.addAttributes(attrs, range: fullRange)
    self.attributedString.append(mas)
  }
}

And with all that, we have achieved our goal and are finally able to create an AttributedString using this single string with interpolations:

let username = "AliGator"
let str: AttrString = """
  Hello \(username, .color(.red)), isn't this \("cool", .color(.blue), .oblique, .underline(.purple, .single))?

  \(wrap: """
    \(" Merry Xmas! ", .font(.systemFont(ofSize: 36)), .color(.red), .bgColor(.yellow))
    \(image: #imageLiteral(resourceName: "santa.jpg"), scale: 0.2)
    """, .alignment(.center))

  Go there to \("learn more about String Interpolation", .link("https://github.com/apple/swift-evolution/blob/master/proposals/0228-fix-expressiblebystringinterpolation.md"), .underline(.blue, .single))!
  """

Conclusion

I hope that you enjoyed this series on StringInterpolation and that it gave you a glimpse at the power brought by that new design.

You can download my Playground here¹ with the full implementation for GitHubComment (see part 1), AttrString, and even some fun I tried with an simplistic implementation for RegEx.

There are plenty more nice ideas around here to make use of that new ExpressibleByStringInterpolation API coming in Swift 5 — including some from Erica Sadun’s blog here, here and here — so don’t hesitate to read more about it… and even have fun with it yourself!

For the code in that post & playground, you’ll need to use Swift 5, which is now shipped with Xcode 10.2. ↩ ↩²
Of course I’ve only implemented a limited list of styles there, for demo purposes. The idea would be to extend that Style type to support way more styles in the future, and ideally cover all possible NSAttributedString.Key that exists. ↩

StringInterpolation in Swift 5 — Introduction

Sat, 15 Dec 2018 00:00:00 +0000

In Swift 4, the StringInterpolation protocol got deprecated, because its original design was inefficient and inflexible, with the goal of redisigning it entirely after Swift 4. Since then, SE-0228 introduced a new design for StringInterpolation, which is going to be part of Swift 5, and opens a whole lot of powerful possibilities.

This feature is part of Swift 5, so you will need Xcode 10.2 and Swift 5 to use it.

Note: This article is an introduction to the new StringInterpolation. A followup article with a more advanced example is available in part 2 of this article.

The new StringInterpolation design

I really encourage you to read the SE-0228 proposal to get an idea of the design and motivations behind the new API.

Basically, to make a type conform to ExpressibleByStringInterpolation, you have to:

Make this type have a subtype StringInterpolation, that must itself conform to StringInterpolationProtocol and will be responsible to handle the interpretation of the interpolation
That subtype just have to implement appendLiteral(_ literal: String) and one or more appendInterpolation(…) method, with the signature of your choosing depending on what you want to support
This StringInterpolation subtype will serve as a “Builder Pattern” for your main type, and the compiler will call those append… methods to build the object step by step
Then your main type needs to implement init(stringInterpolation: StringInterpolation) to instantiate itself with the result of those incremental steps.

The fact that you can implement whatever appendInterpolation(…) method you like means that you can choose what interpolation to support. This is a super powerful feature that opens a large range of possibilities!

For example, if you implement func appendInterpolation(_ string: String, pad: Int) that means that you’ll be able to build your type using an interpolation like: "Hello \(name, pad: 10), how are you?". The interpolation just has to match one of the appendInterpolation signatures that your StringInterpolation subtype support.

Simple example

Let’s start with a simple type to demonstrate how it goes. Let’s build a GitHubComment type that would allow you to reference issue numbers and users.

The goal of that example is to be then able to write something like this:

let comment: GitHubComment = """
  See \(issue: 123) where \(user: "alisoftware") explains the steps to reproduce.
  """

So, how do we implement this?

First let’s declare the basic struct GitHubComment and make it ExpressibleByStringLiteral (because ExpressibleByStringInterpolation inherits that protocol so let’s get that implementation out of the way) and CustomStringConvertible (for nice debugging when printing in the console)

struct GitHubComment {
  let markdown: String
}

extension GitHubComment: ExpressibleByStringLiteral {
  init(stringLiteral value: String) {
    self.markdown = value
  }
}

extension GitHubComment: CustomStringConvertible {
  var description: String {
    return self.markdown
  }
}

Then, we’ll make GitHubComment conform to ExpressibleByStringInterpolation, which means having a StringInterpolation subtype that will handle what to do when:

initializing itself: init(literalCapacity: Int, interpolationCount: Int) lets you the possibility to reserve some capacity to the buffer you’ll use while building the type step by step. In our case, we could simply have used a String and append the segments to it while building the instance, but I instead chose to use a parts: [String], that we’ll assemble together later
implement appendLiteral(_ string: String) to append the literal text to the parts
implement appendInterpolation(user: String) to append the markdown representation of a link to that user’s profile when encountering \(user: xxx)
implement appendInterpolation(issue: Int) to append the markdown representation of a link to that issue
then implement init(stringInterpolation: StringInterpolation) on GitHubComment to build a comment from those parts

extension GitHubComment: ExpressibleByStringInterpolation {
  struct StringInterpolation: StringInterpolationProtocol {
    var parts: [String]

    init(literalCapacity: Int, interpolationCount: Int) {
      self.parts = []
      // - literalCapacity is the number of characters in literal segments (L)
      // - interpolationCount is the number of interpolation segments (I)
      // We estimate that we generally have a structure like "LILILIL"
      // — e.g. "Hello \(world, .color(.blue))!" — hence the 2n+1
      self.parts.reserveCapacity(2*interpolationCount+1)
    }

    mutating func appendLiteral(_ literal: String) {
      self.parts.append(literal)
    }

    mutating func appendInterpolation(user name: String) {
      self.parts.append("[\(name)](https://github.com/\(name))")
    }

    mutating func appendInterpolation(issue number: Int) {
      self.parts.append("[#\(number)](issues/\(number))")
    }
  }

  init(stringInterpolation: StringInterpolation) {
    self.markdown = stringInterpolation.parts.joined()
  }
}

And that’s it! we’re done!

Notice how, because of the signatures of the appendInterpolation methods we’ve implemented, we’re allowed to use Hello \(user: "alisoftware") but not Hello \(user: 123), as appendInterpolation(user:) expects a String as parameter. Similarly, \(issue: 123) in your string will only allow an Int because appendInterpolation(issue:) takes an Int as parameter.

In fact, if you try to use interpolations that are not supported by your StringInterpolation subtype, the compiler will give you nice errors:

let comment: GitHubComment = """
  See \(issue: "bob") where \(username: "alisoftware") explains the steps to reproduce.
  """
//             ^~~~~         ^~~~~~~~~
// error: cannot convert value of type 'String' to expected argument type 'Int'
// error: incorrect argument label in call (have 'username:', expected 'user:')

It’s only the beginning

This new design opens a large range of possibilities for making your own types ExpressibleByStringInterpolation. Some ideas include:

Making a HTML type conform so that you can write HTML using interpolations
Making a SQLStatement type conform so you can write SQL statements easily
Using string interpolation to support more custom formatting, like formatting Double or Date values inside your interpolated string
Making a RegEx type conform so that you can write regular expressions with a fancy syntax
Making a AttributedString type conform to build an NSAttributedString using string interpolation

Brent Royal-Gordon, who came up with that awesome design for the new String Interpolation alongside Michael Ilseman, provided some more example in this gist

Personally I gave a shot at building NSAttributedStrings using StringInterpolation, and found the result really beautiful 🤩 So I am really excited to share it with you in part 2 of this post 🙂

See you there! 👋

Private properties in protocols

Sun, 02 Sep 2018 00:00:00 +0000

In Swift, protocols can’t specify access control to the properties they declare. If a property is listed in a protocol, you have to make conforming types declare those properties explicitly.
But sometimes, even if you need those properties in order to provide your implementations, you don’t want those properties to be used outside the type. Let’s see how to workaround that problem.

A simplified example

Let’s see that you want to create a dedicated object to manage your ViewControllers navigation, like a Coordinator.

Every coordinator is going to have a root UINavigationController, and share some common capabilities, like pushing and poping other ViewControllers on it. So at first it might look like this¹:

// Coordinator.swift

protocol Coordinator {
  var navigationController: UINavigationController { get }
  var childCoordinator: Coordinator? { get set }

  func push(viewController: UIViewController, animated: Bool)
  func present(childViewController: UIViewController, animated: Bool)
  func pop(animated: Bool)
}

extension Coordinator {
  func push(viewController: UIViewController, animated: Bool = true) {
    self.navigationController.pushViewController(viewController, animated: animated)
  }
  func present(childCoordinator: Coordinator, animated: Bool) {
    self.navigationController.present(childCoordinator.navigationController, animated: animated) { [weak self] in
      self?.childCoordinator = childCoordinator
    }
  }
  func pop(animated: Bool = true) {
    if let childCoordinator = self.childCoordinator {
      self.dismissViewController(animated: animated) { [weak self] in
        self?.childCoordinator = nil
      }
    } else {
      self.navigationController.popViewController(animated: animated)
    }
  }
}

And then when we want to declare a new Coordinator object, we’d do something like this:

// MainCoordinator.swift

class MainCoordinator: Coordinator {
  let navigationController: UINavigationController = UINavigationController()
  var childCoordinator: Coordinator?

  func showTutorialPage1() {
    let vc = makeTutorialPage(1, coordinator: self)
    self.push(viewController: vc)
  }
  func showTutorialPage2() {
    let vc = makeTutorialPage(2, coordinator: self)
    self.push(viewController: vc)
  }

  private func makeTutorialPage(_ num: Int, coordinator: Coordinator) -> UIViewController { … }
}

The problem: leaking implementation details

There are two problems with this solution regarding protocol visibility:

When we want to declare a new Coordinator object, we have to explicitly declare a let navigationController: UINavigationController property AND a var childCoordinator: Coordinator? every time. Even if we don’t use them explicitly in implementations of our conforming types — they are just there because we need them for the default implementations provided by the protocol Coordinator to work.
Those two properties we have to declare have to be of the same visibility (the implicit internal access control level in our case) as our MainCoordinator, because that’s a requirement of our protocol Coordinator. That makes them also visible to the outside, i.e. to code using MainCoordinator

So the problem is that we have to declare some properties every time while it’s only some implementation details, but also that these implementation details are leaked to the outside interface, allowing consumers of that class to do things they shouldn’t be allowed to do, like:

let mainCoord = MainCoordinator()
// Consumers shouldn't be allowed to access the navigationController directly but they can
mainCoord.navigationController.dismissViewController(animated: true)
// and neither should they be allowed to do stuff like this
mainCoord.childCoordinator = mainCoord

You might think that we could just not declare those two properties in the protocol in the first place, as we don’t want them to be visible. But if we do that, we wouldn’t be able to provide default implementations via our extension Coordinator, as those default implementations need those properties to exist in order for their code to compile.

One could also hope that Swift would allow declaring those properties fileprivate in the protocol, but as of Swift 4, you can’t specify any access control attributes inside protocols.

So how could we solve that, to both provide those default implementations which require those properties, and not letting them leak to the outside interface?

A solution

A trick to achieve that is to hide those properties inside an intermediate object, and make the properties of that object fileprivate.

That way, even if we’ll still have conforming types to declare that property in their public interface, consumers of that interface won’t be able to access internal properties of that object. While our default implementation of the protocol will be able to access them — as long as it’s declared in the same file (as they’ll be fileprivate).

This would look like this:

// Coordinator.swift

class CoordinatorComponents {
  fileprivate let navigationController: UINavigationController = UINavigationController()
  fileprivate var childCoordinator: Coordinator? = nil
}

protocol Coordinator: AnyObject {
  var coordinatorComponents: CoordinatorComponents { get }

  func push(viewController: UIViewController, animated: Bool)
  func present(childCoordinator: Coordinator, animated: Bool)
  func pop(animated: Bool)
}

extension Coordinator {
  func push(viewController: UIViewController, animated: Bool = true) {
    self.coordinatorComponents.navigationController.pushViewController(viewController, animated: animated)
  }
  func present(childCoordinator: Coordinator, animated: Bool = true) {
    let childVC = childCoordinator.coordinatorComponents.navigationController
    self.coordinatorComponents.navigationController.present(childVC, animated: animated) { [weak self] in
      self?.coordinatorComponents.childCoordinator = childCoordinator // retain the child strongly
    }
  }
  func pop(animated: Bool = true) {
    let privateAPI = self.coordinatorComponents
    if privateAPI.childCoordinator != nil {
      privateAPI.navigationController.dismiss(animated: animated) { [weak privateAPI] in
        privateAPI?.childCoordinator = nil
      }
    } else {
      privateAPI.navigationController.popViewController(animated: animated)
    }
  }
}

And the conforming type MainCoordinator would now:

Only have to declare a single let coordinatorComponents = CoordinatorComponents() property, not having to know what’s inside that CoordinatorComponents type (hiding the implementation details)
Wouldn’t be able to access any of the coordinatorComponents properties from its MainCoordinator.swift file, as they are declared fileprivate

public class MainCoordinator: Coordinator {
  let coordinatorComponents = CoordinatorComponents()

  func showTutorialPage1() {
    let vc = makeTutorialPage(1, coordinator: self)
    self.push(viewController: vc)
  }
  func showTutorialPage2() {
    let vc = makeTutorialPage(2, coordinator: self)
    self.push(viewController: vc)
  }

  private func makeTutorialPage(_ num: Int, coordinator: Coordinator) -> UIViewController { … }
}

Sure, you still need to declare let coordinatorComponents in the conforming type to provide the storage, and this declaration has to be visible (can’t be made private) as it’s part of the requirement for conforming to protocol Coordinator. But:

that’s only one property to declare instead of 2 (or more in more complex cases)
and more importantly, even if it’s accessible from the conforming type’s implementation, and also from the outside interface, you can’t do anything with it.

Sure you can still access myMainCoordinator.coordinatorComponents but you won’t be able to do anything with it as all its properties are fileprivate!

Conclusion

Swift might not provide all the features you want right outside of the box. One might hope that some day protocols would be able to allow declaring access control attributes to the properties and function requirements they declare, or some way to make those hidden to the public API.

But in the meantime, having those kind of tricks and workaround up your sleeve can make your public API nicer and more safe, avoiding to leak implementation details or to give access to properties that shouldn’t be modified outside of the implementation while still using the Mixin pattern and providing default implementations.

That’s some simplified example ; don’t focus on the implementation of the Coordinator pattern here — that’s not really the point of that example, which focuses more about the need to have publicly accessible properties declared in the protocol. ↩

Exploring @dynamicMemberLookup

Thu, 14 Jun 2018 00:00:00 +0000

With Xcode 10 and Swift 4.2, the new @dynamicMemberLookup proposal is now available in Swift. Let’s have some fun with it.

Welcome Xcode 10

With the WWDC’18 just finished, the first beta of Xcode 10 is already available and really pleasant with a lots of very welcome improvements, including:

way better autocompletion: more contextual completion, more reactive, more accurate, also works in auxiliary files in Playgrounds, …
better Playgrounds: errors and warnings properly shown inline in red/yellow ribbons, new REPL mode is really cool, way more stable, … they are actually usable again!

But Xcode 10 also comes with Swift 4.2 included. Which brings some interesting new features recently implemented through the Swift-Evolution process and included in this release of Swift

Welcome Swift 4.2… and @dynamicMemberLookup

One of the new features included in Swift 4.2 is dynamicMemberLookup. This allows to call object properties that will be dynamically resolved at runtime.

This proposal had some controversy, and one thing I didn’t personally like on this new feature is that it meant that typed annotated with @dynamicMemberLookup would not, by design, show any potential compilation error. Which is understandable, as the whole need for that proposal was to be able to call properties which we didn’t know at compile-time.

But this also meant that once the type was annotated with @dynamicMemberLookup, you wouldn’t be able to choose at call-site if you wanted an expression to allow dynamic member lookup or wanted the expression on that type to be type-checked

Example of a problematic dynamicMemberLookup

Let’s take the example given in the SE-0195 proposal:

@dynamicMemberLookup
enum JSON {
  case intValue(Int)
  case stringValue(String)
  case arrayValue(Array<JSON>)
  case dictionaryValue(Dictionary<String, JSON>)
  subscript(dynamicMember member: String) -> JSON? {
    if case .dictionaryValue(let dict) = self {
      return dict[member]
    }
    return nil
  }
  var count: Int {
    switch self {
    case .intValue, .stringValue: return 1
    case .arrayValue(let a): return a.count
    case .dictionaryValue(let d): return d.count
    }
  }
}

The problem I have with this example is that we can call whatever property we want on a JSON instance, because it’s @dynamicMemberLookup this property lookup will always compile, but sometimes returns the real property and sometimes doing a dynamic lookup, without any distinction at call site:

let j = JSON.dictionaryValue([
  "comment": .stringValue("Not being able to tell the difference at call site is confusing"),
  "count": .intValue(42),
  "count2": .intValue(1337)
  ])

// Show the issue: hard to know when this is solved dynamically vs at compile time
j.count  // This one will return 3 — from `var count: Int` in `enum JSON`. And not 42 from the dictionary
j.count2 // While this one will return the 1337 value from the dictionary
j.cuont  // And this will compile despite the typo (and return nil because no such key exists in the JSON)

Improving the situation: make it explicit at call site

What I want to be able to do is to distinguish explicitly between calls that are checked at compile time and those that are dynamically looked up (and are not compile-time checked)

An example of the call site I’d find more clear would be like this:

j.count // compile-time checked. Compiles, calls the property on Dictionary, returns 3
j.cuont // compile-time error
j^.count2 // explicit that this is dynamically looked-up and not compile-time checked. Dynamically lookup inside the dictionary at runtime, and returns the value of key "count2" (which is 1337)

Let’s build it!

The solution is actually quite short to implement. We’ll create a ^ postfix operator that will wrap the instance it’s applied on into some proxy object that is the one being @dynamicMemberLookup. That proxy object will just wrap the dictionary we applied ^ on, and on dynamic lookup, will search the key in the dictionary to return the corresponding value (if it exists).

@dynamicMemberLookup
public struct DynamicLookupProxy {
  let dict: [String: Any]
  public subscript(dynamicMember member: String) -> Any? {
    return dict[member]
  }
}

postfix operator ^
public postfix func ^ (lhs: [String: Any]) -> DynamicLookupProxy {
  return DynamicLookupProxy(lhs)
}

And now if we test this on our example:

let j: [String: Any] = [
  "comment": "Being able to tell the difference at call site is explicitly is nicer",
  "count": 42,
  "count2": 1337
]

j.count // checked at compile time. Returns 3
j^.count // dynamic lookup. Returns 42


j.cuont // compile-time error
j^.cuont // dynamic lookup, but key not found. Returns nil.

Success!

Chain all the things!

The only problem with this is that our dynamic lookup returns Any. This means we have to cast it to the desired object, and we can’t chain those calls:

let j2: [String: Any] = [
  "name": "Olivier",
  "address": [
    "street": "Swift Street",
    "number": 1337,
    "city": "PlaygroundVille"
  ]
]
j2^.name
j2^.address^.street // 🛑 Cannot convert value of type 'Any?' to expected argument type '[String : Any]'. Fix: Insert ' as! ([String : Any])'

How do we solve that? Surely not by the force-cast suggested by that Fix-It!

Well, if instead of returning Any? we make the subscript generic (yes, that’s also a recent addition, in Swift 4!) to make it be able to return some typed value, we could make it infer it being [String: Any] in some contexts?

So let’s add that generic subscript to our DynamicLookupProxy struct:

@dynamicMemberLookup
public struct DynamicLookupProxy {
  private let dict: [String: Any]
  public init(_ dict: [String: Any]) {
    self.dict = dict
  }

  public subscript<T>(dynamicMember member: String) -> T? {
    return dict[member] as? T
  }
  public subscript(dynamicMember member: String) -> Any? {
    return dict[member]
  }
}

Note: I still kept the -> Any? implementation so that if the return type is not specified/inferable, it’ll falls back to Any? instead of forcing you to specify the type explicitly

And now look at that!

j2^.name // Returns "Olivier"
j2^.address?^.street // Returns "Swift Street"
j2^.address?^.zipcode // Returns nil, as there's no zipcode
j2.count // Calls the compile-time property count on Dictionary, returns 2

The line j2^.address is actually equivalent to j2["address"]… so why bother having all that? It’s not that different after all, and maybe it’s better to keep strings explicit? Maybe… but now we can also chain those properties to access subkeys without having to do explicit casts, as opposed to using string subscripts!

// Nice call site with our new solution, but still explicit that it's dynamic lookup:
j2^.address?^.street
// What we'd have to write without `@dynamicMemberLookup` to have the same:
(j2["address"] as? [String: Any])?["street"]

And even better, since we’re now using generics, we can even let the compiler infer the return type of the dynamic member lookup by hinting the type of the variable we store the result to:

let addr: [String: Any]? = j2^.address
addr?^.street // Returns "Swift Street"
addr?["street"] // same thing, really, but arguably less nice

addr?.count // Still no ambiguity: no '^' here, this one is compile-time checked (returns 3)

Avoid the question mark?

The last thing we could consider doing is to get rid of the ? and consider that dynamic member lookup on a nil value should directly return nil — without having to rely on optional chaining to do that.

Indeed, having to use ? in front of every ^ in the chain (besides the first one) can feel repetitive.

The solution actually consists of a simple change: just allow our ^ operator to be applied on optional dictionaries. If our dictionary isn’t optional, the compiler will lift it to optional for us, and if it’s optional, we can imagine just return our DynamicLookupProxy but with an empty dictionary — so that any following dynamic lookup will always return a nil value anyway.

So just change our func ^ signature and implementation to this:

public postfix func ^ (lhs: [String: Any]?) -> DynamicLookupProxy {
  return DynamicLookupProxy(lhs ?? [:])
}

And we’re good to go! Now we can update our call sites to look like this:

// No need for the extra `?` before the second `^` anymore.
let street: String? = j2^.address^.street // Still returns "Swift Street"

Was that a good idea? Wasn’t it better to keep explicit the fact that the return type of any dynamic lookup was optional by requiring the optional chaining syntax and that ? before continuing chaining with ^.street? That’s another debate, and I’ll let you decide. But at least you know that’s possible 😉

More exploration: creating a context/scope

If you’ve made it so far, maybe you’re even ready to take it a step further? (if not, you can skip this last part, I won’t blame you 😄)

In the playground attached below, I’ve explored that idea a bit more, by allowing another syntax. This one looks more like creating a context or scope in which everything we call is dynamically looked up (instead of looking like chaining ^ calls):

let street: String? = j2.dynamicLookup { $0.address?.street }

To implement this, I’ve created a separate proxy type called DynamicLookupContext similar to our first DynamicLookupProxy, but this time, our subscript on DynamicLookupContext does not returns the value found in the dictionary, but instead wraps that value in a new DynamicLookupContext and return that. Which means we can chain them (without having to re-lift them at every step like we had to do by repeating ^ at each step in our previous solution).

Then the func dynamicLookup<T> I’ve added on [String: Any] takes care of wrapping that initial dictionary we want to query into a DynamicLookupContext, let you execute your chain of dynamic lookups on this (via a closure you provide), and extract the output from the result returned by the end of that chain:

@dynamicMemberLookup
public struct DynamicLookupContext {
  let value: Any

  public subscript(dynamicMember member: String) -> DynamicLookupContext? {
    let dict = value as? [String: Any]
    guard let value = dict?[member] else { return nil }
    return DynamicLookupContext(value: value)
  }
  public subscript(index: Int) -> DynamicLookupContext? {
    guard let array = value as? [Any] else { return nil }
    return DynamicLookupContext(value: array[index])
  }
}

public extension Dictionary where Key == String, Value: Any {
  func dynamicLookup<T>(execute: (DynamicLookupContext) -> DynamicLookupContext?) -> T? {
    let wrapped = DynamicLookupContext(value: self)
    let result = execute(wrapped)
    return result?.value as? T
  }
}

As a result, you can now choose between either syntax, whichever you like best:

// With our DynamicLookupProxy and ^ operator we saw before:
let street1: String? = j2^.address^.street
// With our new DynamicLookupContext and dynamicLookup method above:
let street2: String? = j2.dynamicLookup { $0.address?.street }
// Without any usage of those @dynamicMemberLookup tricks, we'd have to do this instead:
let addr = j2["address"] as? [String: Any]
let street3 = addr["street"] as? String

I’ve attached an Xcode 10 playground with my experimentation around this and the code presented above.

Conclusion

What do you think of this solution?

To be honest, I am torn as to consider this really useful in production code or just a toy idea.

I like the fact that it’s now clear at call site when we intend to do a dynamic lookup and when we intend to do a real property access that is checked at compile time. We can choose when we want one or the other.
On the other hand, the fact that the code still looks like real Swift instructions — without explicit String-typed keys clearly appearing — can make this confusing; people have to know what this ^ operator we created does and that it allows part of the expression to not be checked by the compiler.

As always when you create a custom operator — and even more so here where it kind of “disable the compile-time checks on some parts of the expression” — it can be hard for newcomers on your code to understand how that works, so even if this is fun and powerful, we also have to keep that in mind.

What do you all think? Do you like the idea? Would you use that in production to allow you to quickly parse parts of an arbitrary JSON or Plist or dictionary? Or should it just be kept as a fun experiment in a playground?

Using Dedicated Objects as Delegates & Datasources

Sun, 20 May 2018 00:00:00 +0000

Today for my come-back post (hopefully), I’ll be talking about a tip for your UITableViewDataSource and UITableViewDelegate. The idea is in fact applicable to a lot of other things following the same delegate pattern, but the use case of UITableView will hopefully be the most concrete example for most people.

The idea

A lot of developers, me included, have taken the habit of making your UIViewController the delegate and dataSource of their UITableView. That’s all good and fine, but in effort to minimize the code size of your UIViewController (you know, that Massive-View-Controller thing all we want to avoid) and limit the responsibility of your UIViewController (separation of concern), there’s a better solution.

The general idea is in fact very simple: extract your UITableViewDataSource and UITableViewDelegate into a separate object. And make e.g. your UIViewController retain that object and set it as the tableView’s delegate and dataSource.

In practice, you can move all the responsibility to a single object (making it both the delegate and dataSource, and making it the responsibility to provide the cells), or you can make that separate objects (one for delegate, one for dataSource), or you can make that separate object just be a proxy, that will be reusable for multiple ViewControllers, and delegate more simple queries to each UIViewController using it to customize just the right things.

Concrete example

Let’s start with a very simple example: a list of products.

struct Product {
  let name: String
  let category: String
  let price: Double
}

Splitting your DataSource into a dedicated type would look like that:

class ListDataSource: NSObject {
  let products: [Product]

  init(products: [Product]) {
    self.products = products
  }

  func product(at indexPath: IndexPath) -> Product {
    return self.products[indexPath.row]
  }
}

extension ListDataSource: UITableViewDataSource {
  func numberOfSections(in tableView: UITableView) -> Int {
    return 1
  }
  func tableView(_ tableView: UITableView, numberOfRowsInSection section: Int) -> Int {
    return self.products.count
  }
  func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
    return cell(for: product(at: indexPath))
  }

  func cell(for product: Product) -> UITableViewCell {
    // Of course in a real project we'd dequeue a custom cell from tableView, etc.
    let cell = UITableViewCell()
    cell.textLabel?.text = product.name
    return cell
  }
}

And then you’ll use it this way in your UIViewController:

class DemoViewController : UIViewController {
  var tableView: UITableView!

  var currentDataSource: UITableViewDataSource? {
    didSet {
      self.tableView.dataSource = currentDataSource
      self.tableView.reloadData()
    }
  }

  override func viewDidAppear(_ animated: Bool) {
    super.viewDidAppear(animated)
    let products = loadProducts()
    self.currentDataSource = ListDataSource(products: products)
  }
}

In that very simplistic example, the benefits might not seem very visible at first, especially because for now our DemoViewController doesn’t do much yet. But in a real world example where that DemoViewController probably does a lot more — like some logic specific to your app, or handling any @IBAction you have on the rest of your screen, etc — I hope you can see how at least extracting all the UITableView display logic would help here.

It’s also generally easier to reason about, as all the logic about the display of that DataSource is now extracted in its dedicated file & class, instead of it being mixed with the rest of the code of your UIViewController, and that separation will probably help you when tweaking or editing the way the UITableView should display its data, by focusing on one specific and isolated file, instead of scrolling amongst other unrelated code.

Retain your DataSources!

One tricky thing however when you put your dataSource in a dedicated, separate object, is to not forget to retain it.

The dataSource property on UITableView is weak (as all dataSources and delegates should be), so if you just affect tableView.dataSource to a newly created ListDataSource(…) without retaining it, that ListDataSource instance will then be released from memory and your UITableView will go back to being empty.

So when you use that trick, just don’t forget to retain your object serving as DataSource. In the example above, I made the DemoViewController retain it by using a var currentDataSource property, and used that occasion to use a didSet on it to propagate it to the tableView and reload it afterwards.

Improving the DataSource

Now, imagine that without changing your app’s logic itself, you just wanted to adjust the way the list of products were displayed, e.g. group your products in sections according to their category.

All you’d have to do is to modify the implementation of your DataSource class, without changing anything in your DemoViewController!

But let’s take that one step further: instead of modifying the existing DataSource, let’s keep it but also duplicate it to create a new one (SectionDataSource below). By keeping both of them, we’ll then be able to switch between the two at runtime later!

So let’s create our new SectionDataSource, very similar to the first one, but which now handles sections:

class SectionsDataSource: NSObject {
  struct Section {
    let title: String
    let items: [Product]
  }
  let sections: [Section]

  init(products: [Product]) {
    // Dispatch products into a dictionary according to their category
    let groups = Dictionary(grouping: products, by: { product in product.category })
    // Convert that dictionary into an array of Sections, then sort it by section title
    self.sections = groups.map(Section.init).sorted(by: { $0.title < $1.title })
  }

  func product(at indexPath: IndexPath) -> Product {
    return self.sections[indexPath.section].items[indexPath.row]
  }
}

extension SectionsDataSource: UITableViewDataSource {
  func numberOfSections(in tableView: UITableView) -> Int {
    return self.sections.count
  }
  func tableView(_ tableView: UITableView, titleForHeaderInSection section: Int) -> String? {
    return self.sections[section].title
  }
  func tableView(_ tableView: UITableView, numberOfRowsInSection section: Int) -> Int {
    return self.sections[section].items.count
  }
  func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
    return cell(for: product(at: indexPath))
  }

  func cell(for product: Product) -> UITableViewCell {
    let cell = UITableViewCell()
    cell.textLabel?.text = product.name
    return cell
  }
}

And now that we have our new SectionsDataSource class, all we have to do in our DemoViewController is… replace the self.currentDataSource = ListDataSource(products: products) line by self.currentDataSource = SectionsDataSource(products: products), and that’s all!

See how everything continues to work, while the majority of your changes was located in a dedicated file and your ViewController was barely affected. That also helps to focus your editing in one specific class in a single file, instead of finding in a Massive ViewController which lines to modify.

Switching DataSources

Now that we have both ListDataSource and SectionsDataSource, we can easily imagine switching them at runtime. You could just add a UISegmentedControl or similar to your UI, and depending on which segment gets selected, you’d set self.currentDataSource to either ListDataSource(products: products) or SectionsDataSource(products: products), and that’s all!

Imagine if you had to do all that logic in your DemoViewController directly instead of having separate objects… That would very likely mean:

A lot of if tests inside numberOfRowsInSection and cellForRowAtIndexPath methods to return different things depending on the mode (flat list or sections by categories)
Be sure to make those if conditions consistent (have the same conditions and branches in numberOfRowsInSection and cellForRowAtIndexPath so that the count returned in one matches the object index used in the other)
Either two properties to hold the data (an array of Products for the flat list, and an array of Sections for the sectionned list mode), or just use an array of Sections and trick the logic by using an array with only one section in the case of the flat list… not very consistent and readable months later anyway…

A lot could go wrong in this setup, from mismatching the conditions in different methods you have to implement to having to focus on understanding the logic as a whole and understand the big picture at once to see how everything works together, with risks of inconsistencies along the way.

But instead, because we separated the concerns of the UITableViewDataSource in their own classes, it’s all easier to read and reason about!

Going further

From there we can go way further and improve this a lot:

Of course we can imagine adding more and more classes for displaying your UITableView differently
But we could also imagine make those DataSources generic over a type T (instead of using them only for the Product type) to reuse them to display other types elsewhere in our application
We could then accept a closure in the init of SectionsDataSource<T> to tell how to extract the section name (String) for each T (where before we hardcoded that it was product.category)
We could extend the idea to UITableViewDelegate
We could allow to customize the cells

UITableViewDelegate

For extending the idea to UITableViewDelegate, you could go multiple ways:

You could either imagine making your existing classes extend UITableViewDelegate, so that you’d use the same dedicated object for both being the dataSource and the delegate of your tableView: extension SectionsDataSource: UITableViewDelegate, etc.
Or you could decide to use an entirely separate object again, so you’d have one dedicated object for your tableView.dataSource, and a different one to use as your tableView.delegate

Both options are valid, it depends of your needs and what your dedicated implementations are doing. The fact that for UITableView, the delegate and dataSource are often closely related might make you go towards the first solution though ¹.

Also, imagining you go with the first route (use the same object for dataSource and delegate), you could give closures to that object on init to tell it what to do especially when a Product is selected, then forward delegate method to call that closure:

// let onProductSelected: (Product) -> Void
extension ListDataSource: UITableViewDelegate {
  func tableView(_ tableView: UITableView, didSelectRowAt indexPath: IndexPath) {
    self.onProductSelected(product(at: indexPath))
  }
}

Once again, the benefit of that is, if you do the same on the SectionsDataSource, then just switching your currentDataSource from ListDataSource to SectionsDataSource wouldn’t require to change your ViewController, which would still work as expected, even if in one case the list is flat and in the other it’s structured in sections!

That’s because now your DataSource object, whichever one is used, is responsible for translating your IndexPath into Product instances, so that you don’t have to care about how those products are organized in the DataSource, all you have to implement in your ViewController is what to do when a given Product is selected (instead of when a given IndexPath is selected), abstracting away all that IndexPath->Product translation logic out of your ViewController.

Allowing custom cells

For allowing custom cells to be provided by the outside (i.e. being able to use the same ListDataSource for different tableViews which are using different cells, without having to duplicate ListDataSource for each cell type), once again you have multiple solutions, some of them being:

Provide a cellFactory closure to your DataSource classes at init, to tell how to build and return an UITableViewCell from a Product
Provide the cell class to use to instantiate each cell (alongside its cellIdentifier), but make it sure that cell class conforms to some protocol declaring a method like func fill(with product: Product) that your cell class would have to implement, and that your ListDataSource and SectionsDataSource would call after instantiating the cell to populate it
Make your DataSource classes themselves have a delegate property of some custom protocol type (e.g. protocol CellProvider), and conform to that protocol to implement how to build and fill a cell with a Product

There are probably other ways to let your UIViewController tell how to build each cell for cases where you want that flexibility, all being quite similar in concept, but I hope that gives you some interesting ideas.

Explore & Playground

It’s time for you to play with all those concepts yourself! Especially the ones I didn’t detail much in that last part (Generics, etc) could be interesting to explore on your own. But if you’re curious about my take on one possible solution, I’ve attached a playground for you to browse the code we discussed — and even a solution using Generics so those DataSources are reusable for types other that Products in other places in the same app.

Also, keep in mind that this idea we just applied is indeed well suited for UITableViewData & UITableViewDelegate, but can also be used for other delegate patterns as well 😉

Happy Swifting!

For example, the titleForHeaderInSection function is in the UITableViewDataSource protocol, but the viewForHeaderInSection function is in the UITableViewDelegate protocol… go figure; also, they both expect the cells to be indexed the same way by your IndexPaths, so using a delegate which assumes one way of organizing your data (e.g. in a flat list) and a dataSource which assumes another (e.g. in sections) probably wouldn’t fit well together ↩

SwiftGen 4.2 and other news

Sun, 19 Feb 2017 00:00:00 +0000

SwiftGen — my tool to generate Swift code so you can use your images, localized strings, fonts, storyboards and other assets in a type-safe way — has just been released in version 4.2 after a big internal refactoring. I’ve also been working on other OSS projects lately.

This article intends to give you some news on all those various OSS projects as well as what has been going on lately and what’s next to come.

Table of Contents: SwiftGen • Reusable • fastlane • AppDevCon • NSBudapest & CraftConf • Blog Articles

SwiftGen

The thing that took most of my time lately is probably the new SwiftGen release, which just went through a big internal refactoring, consisting of moving SwiftGen to its own GitHub organization and splitting it into multiple repositories.

We even have a brand new logo now!

Why splitting SwiftGen into multiple repositories?

Since the Sourcery project came to life — and because this new tool also uses Stencil the same as we always did at SwiftGen — Krzysztof asked us to share the code we use to drive Stencil, to use it with his new tool too. This way SwiftGen and Sourcery could use the same tags, filters, and extensions to write their templates similarly, an users would have consistent understanding of the template language between both tools. That meant splitting that common code in its own framework, to make it independent of SwiftGen so that Sourcery could depend on it too.
Having separate repositories also helps separate the code of SwiftGen into clear modules with clear responsibilities.
- It should also help potential contributors to understand the code better, as each part of the SwiftGen pipeline will be clearly separated and documented.
- This also allows those modules to be used independently of SwiftGen’s CLI, if other tools want to build something around them.
It allows those modules, as well as templates, to evolve with their own versions lifecycles, for example making it possible to release a new module or tag new templates without having to release a new version of SwiftGen as a whole.

Why move SwiftGen into its own GitHub organization?

Since David Jennes has become a core contributor (thanks again to him as he did most of the work on the refactoring PRs!), I felt like SwiftGen has more and more become a community than just my own project
This use of a GitHub organization will hopefully help making the SwiftGen community grow, by attaching SwiftGen to an organization rather than an individual (me)
Since we needed to split SwiftGen in multiple repositories anyway, having a GitHub organizaton to host them all was the logical choice

What changed for users?

Actually, not that much. That big refactoring was necessary internally, but that shouldn’t change anything for the end users. That’s why this release is a minor one (4.1 -> 4.2) and not a major version bump.

This release also fixes some bugs, mainly the warning on import YourAppModule that was present for storyboards, and adds some new features, mainly support for --param X=Y to pass arbitrary parameters to your templates. See the CHANGELOGs for more details.

What’s next?

We already have the next milestores for SwiftGen planned:

Version 4.2.1 will focus on improving documentation:
- Update the global documentation explaining the new organization in the repo (what’s the responsibility of each module)
- A Markdown per template to document them each in detail and help you choose the right template
- Providing a CONTRIBUTING.md and a CODE_OF_CONDUCT.md and documentation to help contributors feel even more welcome and help them contribute
Version 5.0.0 will clean some stuff up:
- Removing old legacy code and template variables (that were only present because Stencil wasn’t as advanced back then as it is now)
- Re-organize and rename the templates bundled with SwiftGen (especially so that SwiftGen won’t default to using Swift 2 templates, providing more alternatives, and organizing them in directories to overcome their growing number)

If you have created your own custom templates, as some templates variables will be removed in SwiftGen 5.0, I highly suggest that you start adapting them as soon as possible so they’ll be ready and still working for SwiftGen 5.0 when it’s released.
This simply consists of using the new variables instead of the old ones. Both the deprecated variables and the new ones are present in SwiftGen 4.2, so this transition should go smoothly.

To help you migrate your templates to be ready for the upcoming SwiftGen 5.0, you can follow the dedicated Migration Guide.

Reusable

During my absence from this blog, we also worked on my Reusable pod, which is a “Mixin” to help you work with reusable views (like UITableViewCell and UICollectionViewCell subclasses), views designed in a XIB, and VCs loaded via a Storyboard, all in a type-safe way.

This pod is a great complement to SwiftGen to make your code even safer and continue getting rid of all String-based APIs.

If you want to know more about how it works, you can read my blog post about Mixins as well as the dedicated blog post about using Generics to dequeue cells safely.

We recently released Reusable 4 which improves the API and makes it even safer, so take a look and update your projects!

fastlane

I also have the great pleasure to now be part of the fastlane Core Contributor team since recently.

As a Core Contributor, I plan to contribute to fastlane more, but also help other people wanting to contribute to fastlane to feel welcome, help them do their first ruby PRs and make the fastlane community grow even more!

AppDevCon (Amsterdam, March 16-17)

Next month, I’ll be talking at the AppDevCon conference (formerly known as “mDevCon”) in Amsterdam.

This is a wonderful and popular conference in Europe, and I encourage you to go if you can. And Amsterdam is such a beautiful city, that would make such a good excuse to visit anyway!

Among other talks at AppDevCon are:

A tutorial session about Sourcery and SwiftGen by Paweł Brągoszewski: “Adding magic to Swift with SwiftGen and Sourcery”
My talk about Mixins over Inheritance
A talk by Andrea Falcone, member of the fastlane team, about “Supercharging your mobile app release with fastlane”

NSBudapest & CraftConf (Budapest, April 25-28)

At the end of April I’ll also be talking at the NSBudapest meetup that will take place during the Craft Conference in Budapest, Hungary.

I’ll be talking about SwiftGen here, as well as code generation in general, but there are also plenty of other very interesting talks there, including about fastlane and Sourcery too (and I hear Budapest is a wonderful city to visit as well!), so I’m definitely looking forward to it… and maybe meet some of you too!

Blog Articles

I know that I’ve let you down since a little while here, as I haven’t been writing since a long time.

If you read all those news above, you probably understand how busy I have been these past few month (and that’s just about my free OSS work, but my full-time paid work has been pretty full too) and why I didn’t have much time for blogging!

Now that all those big tasks are mostly behind (even if I’ll still continue to be busy with fastlane and my talks, I expect to have a little more free time), it’s about time I start writing again — starting with that part 2 of my capture semantics article that is long overdue!

In the meantime, I hope you’ll enjoy all my hard work on my OSS tools and talks, and will see you soon for some technical articles again!

Still Alive

Sun, 27 Nov 2016 00:00:00 +0000

This is a quick post to reassure you that this blog is still alive 😜 And announcing the subjects that I plan to write about in the upcoming articles.

I’ve been quite swamped lately, not having much free time to do OpenSource work on my various OSS projects and others’, let alone writing in this blog.

Having to do the transition of all my pods and tools (OHHTTPStubs, Reusable, SwiftGen, …) to Swift 2.3 then Swift 3.0 was very time-consuming and didn’t help having much time for anything else. All the time I had to work on my side projects was dedicated to Swift 3 conversion, triaging issues and reviewing Pull Requests.

On this note, I’d like to thank Ben Chatelain and Ceyhun Ozugur for helping a lot porting Reusable to Swift 3, and a huge thanks to David Jennes and Adam Tierney for helping a lot with SwiftGen lately.

But fret not, I haven’t forgotten about the part two of my “Closures Capture Semantics” article and already have plenty of subjects waiting in my drafts about Phantom Types, Type Safety, Networking Architectures, ViewController LifeCycle Behaviors, Functional ViewControllers, Coordinators and much more!

PS: For those of you who like video games and music, I strongly encourage you to go see the VideoGames Live show. It’s a concert of game music played by an orchestra, really stunning. You can see the videos I’ve filmed from it here on my Twitter.

Talking at conferences

Tue, 06 Sep 2016 00:00:00 +0000

This month, I’m going to talk at (NSSpain and FrenchKit) about Swift 🎉 I’m looking forward to speak for the first time at such great conferences, and I really hope to see you there!

Talking at NSSpain & FrenchKit

This post is just a quickie to do a small announcement, in case you didn’t see me announce it on Twitter a few month ago.

This month, I’m going to give my first big conference talks. People who know me already know I don’t have any problem talking a lot in public 😇 and I’ve already talked in meetings like CocoaHeads, but that will be the first time I’ll speak at big conferences.

And for my first time, I already chose quite a challenge:

My first talk will be at NSSpain, on Sept 15-16, 2016
My second talk will be at FrenchKit, on the week right after (Sept 23-24)

The first talk will be quite a challenge for me, because it will be my first big conference, speaking in English (not my language), in a foreign country, but more importantly doing not just slides but mostly live-coding — with Swift 3… so in Xcode 8 beta 😱 yeah I too like to live dangerously! — and about a subject that I haven’t talked about in my blog or elsewhere yet 😓. Yeah, that’s a lot of potential obstacles for my first talk, but hey, I’m up for the challenge!

The second one will probably be easier, first because I’ll already have th experience from the first one, but also because it will be in my country 🇫🇷, with slides and not live-coding, and on a subject I already talked about. But hey, you never know how events could turn once on stage, and especially given how talkative I can be, I hope I’ll fit into the 30 minutes window that I’ve been allocated!

I’m not sure I’m entirely ready for those 😰 but nevertheless I really can’t wait to meet a lot of people there and be part of that adventure 🎉! I really enjoyed going to conferences last year as an attendee, so this time going as a speaker I really think this is going to be even more amazing!

This also explains why I haven’t been very present the last few weeks, quite busy preparing my talks… but worry not, I haven’t forgotten about the part 2 of my last article about Closure Capture Semantics and will hopefully resume my blogging around october after that!

I can’t wait to see a lot of awesome people at NSSpain and FrenchKit, and if you come to one of these conferences, don’t hesitate to come and say hello! 👋

Crunchy Development

YAML Superpowers, part 2: Multiline Strings

Simple strings (“Flow” scalars)

Multi-line blocks and long strings

Indentation in multi-line blocks

Literal style: | – keep the newlines

Folded style: > – fold the newlines into spaces

Chomp mode

Optional block Comment

Conclusion

YAML Superpowers, part 1: JSON is YAML

Why all those posts? YAML is not that complicated!

Back to Basics

A concrete example: a CI config

What’s Next: Advanced features of YAML

StringInterpolation in Swift 5 — AttributedStrings

The goal

First implementation

Adding styling convenience

Supporting Images

Wrapping styles in one another

Conclusion

StringInterpolation in Swift 5 — Introduction

The new StringInterpolation design

Simple example

It’s only the beginning

Private properties in protocols

A simplified example

The problem: leaking implementation details

A solution

Conclusion

Exploring @dynamicMemberLookup

Welcome Xcode 10

Welcome Swift 4.2… and @dynamicMemberLookup

Example of a problematic dynamicMemberLookup

Improving the situation: make it explicit at call site

Let’s build it!

Chain all the things!

Avoid the question mark?

More exploration: creating a context/scope

Conclusion

Using Dedicated Objects as Delegates & Datasources

The idea

Concrete example

Retain your DataSources!

Improving the DataSource

Switching DataSources

Going further

UITableViewDelegate

Allowing custom cells

Explore & Playground

SwiftGen 4.2 and other news

SwiftGen

Why splitting SwiftGen into multiple repositories?

Why move SwiftGen into its own GitHub organization?

What changed for users?

What’s next?

Reusable

fastlane

AppDevCon (Amsterdam, March 16-17)

NSBudapest & CraftConf (Budapest, April 25-28)

Blog Articles

Still Alive

Talking at conferences

Talking at NSSpain & FrenchKit

Literal style: `|` – keep the newlines

Folded style: `>` – fold the newlines into spaces