June 12, 2019

Decodable

Simple things should be simple, complex things should be possible. — Alan Kay

As programmers, we like to organize stuff into data structures like classes, structs, and enums. And (especially as Swift programmers), we like our data to be strongly typed. However, apps often have to talk to the rest of the world, usually file systems and the cloud”. In those contexts, data is serialized”, or transmogrified into an array of bytes. So we want tidy, reliable ways of turning data structures into streams, and streams into data structures. There are a number of frameworks that do this work. Two that Apple provides are Codable and NSCoding. This post is about Codable, and perhaps I will write about NSCoding in a future episode.

Vocabulary

Converting data structures into streams is called serializing” or encoding”. If you want to transform your structure into Data, suitable for streaming, your class, struct, or enum must conform to Encodable.

Similarly, converting data back into a class, struct, or enum is called deserializing” or decoding”. Your data structure needs to conform to Decodable so that it can construct itself from Data.

An Accessible Example

Here is a concrete example. Perhaps a designer has sent you a mockup that looks a little like this. It has some really prominent dark text for the title, and the subtitle is only a little smaller and slightly lighter gray. Then there’s more text that’s even smaller and even lighter.

Mockup of text in decreasing size and contrastMockup of text in decreasing size and contrast

When I got a mockup like this, I felt like I had to strain a bit to read it. So I worried that it might not be accessible to others. I’m not a designer, and I’m certainly not an accessibility expert, so I struggled with how to decide what was enough” contrast, and how to effectively share these concerns with the designer.

It soothes my brain immensely to learn that there is a definition of enough” contrast, backed by field research, the biology of how humans perceive light, and math! There’s a single number to define the contrast between any two colors, and a grading scale for what numbers are good enough. Not only that, but there are websites for looking up these numbers, such as webaim.org, so we don’t even have to do the math.

Here’s the contrast for black and this particularly delightful shade of purple. This combination gets a triple-A rating when used for large text, but only a double-A rating for small text. These colors are fine together for titles, but it would be better to find a slightly higher contrast for paragraphs of text, if possible. Screenshot of WebAIM.org’s color contrast checker with black and purple values

So, that’s a nice accessibility tip, but this blog post is about serializing data. This website is in the cloud, and it has an API. If you add &api to the URL, instead of a webpage, the site serves up some JSON data containing the same information.

// The WebAIM service provides contrast analysis in JSON format.
// https://webaim.org/resources/contrastchecker/?fcolor=A157E8&bcolor=000000&api
{
  "ratio": 5.04,
  "AA": "pass",
  "AALarge": "pass",
  "AAA": "fail",
  "AAALarge": "pass"
}

And if you squint at this JSON just a little, it starts to look like a Swift struct.

// This Swift struct has the same data types and names as the WebAIM response.
struct WebColorContrastResponse {
  let ratio: CGFloat
  let AA: String
  let AALarge: String
  let AAA: String
  let AAALarge: String
}

Conforming to Decodable

Now you have some data and you want to convert it into your data structure. That means you need to add conformance to Decodable on your data structure. What does it look like to conform to Decodable? There’s only a single required method in the protocol: init(from: Decoder).

Here is an implementation of this initializer, with a CodingKey enum to organize the data’s keys.

// WebColorContrastResponse conforms to Decodable with an explicit implemention of the required initializer.
struct WebColorContrastResponse: Decodable {
  init(from decoder: Decoder) throws {
    let values = try decoder.container(keyedBy: CodingKeys.self)
    ratio = try values.decode(CGFloat.self, forKey: .ratio)
    AA = try values.decode(String.self, forKey: .AA)
    AALarge = try values.decode(String.self, forKey: .AALarge)
    AAA = try values.decode(String.self, forKey: .AAA)
    AAALarge = try values.decode(String.self, forKey: .AAALarge)
  } 
  
  enum CodingKeys: String, CodingKey {
    case ratio = "ratio"
    case AA = "AA"
    case AALarge = "AALarge"
    case AAA = "AAALarge"
    case AAALarge = "AAALarge"
  }
  
  let ratio: CGFloat
  let AA: String
  let AALarge: String
  let AAA: String
  let AAALarge: String
}

This is certainly possible, but a little tedious. The initializer sets each property explicitly, and the CodingKey enum provides a string mapping for each property, so this relatively short code listing mentions each property three times. If you need to add another property later, you’ll have to add information about it in all three places. I don’t like that, you probably don’t like that. Fortunately, the developers of Codable didn’t like it either. They came up with a shortcut so you don’t have to write all this.

In many cases, the Swift compiler can generate all this code automatically. If every property in your struct is Decodable, just declare your struct Decodable and you’re done. Or if your enum is backed by a Decodable type, you can get free conformance on the enum. Just say it conforms and you’re done.

// WebColorContrastResponse declares conformance to Decodable, but does not implement the required initializer.
// Since String and CGFloat are Decodable, the compiler will synthesize the initializer.
struct WebColorContrastResponse: Decodable {
  let ratio: CGFloat
  let AA: String
  let AALarge: String
  let AAA: String
  let AAALarge: String
}

Dealing with the Real World

This is all fine if you have total control of the data format. But life isn’t always so tidy. Maybe your data comes from a server that uses different naming conventions. Now your keys don’t look like Swift property names, or might be confusing in the context of your code. If you want to customize the keys, bring back just that CodingKeys enum. List all your properties, and specify the string values for the ones you want to customize.

// WebColorContrastResponse has custom property names.
// The compiler will still synthesize the Decodable initializer.
struct WebColorContrastResponse: Decodable {
  enum CodingKeys: String, CodingKey {
    case ratio
    case smallDoubleA = "AA"
    case largeDoubleA = "AALarge"
    case smallTripleA = "AAA"
    case largeTripleA = "AAALarge"
  }

  let ratio: CGFloat
  let smallDoubleA: String
  let largeDoubleA: String
  let smallTripleA: String
  let largeTripleA: String
}

Sometimes that’s not enough. Sometimes you need to do more custom handling. Maybe the types don’t match. When I first looked at the JSON from webaim.org, the data structure looked like the original example above, and I wrote a Playground around fetching and processing this data. Then I came back to run my code a few months later, and it didn’t work at all. After way too much frustration, I realized the web service had changed the type of the ratio from a decimal number to a string! (Here is an updated version of that Playground.)

If you encounter a mismatch like this and need to cast types or do other custom handling, you should write an explicit implementation of init(from: Decoder). Like any other initializer, this method must initialize each property. If anything might go wrong, you can throw the same sort of DecodingError that would have been thrown by the compiler-generated initializer, or you can handle the failure case with a fallback value or something else that makes more sense for your situation.

// WebColorContrastResponse has an explicit definition of the initializer to convert the ratio from a String to a CGFloat. 
struct WebColorContrastResponse: Decodable {
  init(from decoder: Decoder) throws {
    let values = try decoder.container(keyedBy: CodingKeys.self)
    let str = try values.decode(String.self, forKey: .ratio)
    guard let ratioAsDouble = Double(str) else {
      throw DecodingError.typeMismatch(…)
    }
    ratio = CGFloat(ratioAsDouble)
    tripleA = try values.decode(String.self, forKey: .tripleA)
    // etc.
  }   

  enum CodingKeys: String, CodingKey {
    case ratio
    case tripleA = "AAA"
    // etc.
  }

  let ratio: CGFloat
  let tripleA: String
  // etc.
}

More Special Cases

For JSON, there are even more ways to adapt JSONDecoder to the realities of your situation. If you’ve got dates to parse, you can configure the JSONDecoder to use one of the standardized strategies, or even define your own if you’ve got some really quirky dates to wrangle. If your data keys don’t need full renaming, just converting from one naming convention to another, you may be able to specify a keyDecodingStrategy to avoid having to create a full CodingKeys enum.

Opinions on Style

Now, if I may, I’d like to offer a couple opinions on stylish implementations of Decodable.

Consider the full struct, with custom key names and an explicit initializer to deal with the type mismatch. Even in this small example, with only five properties, the code starts feeling pretty cumbersome. The first thing you can do to tidy this up is move the protocol conformance to an extension. The separation makes it really clear what parts of the code are the basic definition of the data structure, and which ones are particularly for Decodable. Also, the custom init(from: Decoder) is no longer in the struct definition. When a struct has no explicit initializers in its definition, the compiler synthesizes a basic initializer that has a parameter for each property, in the order they’re defined. With the Decodable initializer in the struct, that basic initializer wasn’t being generated. By moving the Decodable initializer to an extension, the basic one will be synthesized again.

// Moving Decodable conformance to an extension separates it from other logic in the data structure. 

struct WebColorContrastResponse {
  let ratio: CGFloat
  let tripleA: String
  // etc.
}


extension WebColorContrastResponse: Decodable {
  init(from decoder: Decoder) throws {
    let values = try decoder.container(keyedBy: CodingKeys.self)
    let str = try values.decode(String.self, forKey: .ratio)
    guard let ratioAsDouble = Double(str) else {
      throw DecodingError.typeMismatch(…)
    }
    ratio = CGFloat(ratioAsDouble)
    tripleA = try values.decode(String.self, forKey: .tripleA)
    // etc.
  }   

  enum CodingKeys: String, CodingKey {
    case ratio
    case tripleA = "AAA"
    // etc.
  }
}

// The Decodable initializer still works.
let serverResponse = WebColorContrastResponse(from: someDecoder)
// The basic initializer is also available. 
let locallyAssembledResponse = WebColorContrastResponse(ratio: 2.6, tripleA: false, doubleA: false, tripleALarge: false, doubleALarge: false)

It might be worth going even one step further. Reduce the primary web response struct back down to its most basic form, with auto-synthesized conformance to Decodable. Then create a separate data type which can be initialized from this web response struct. The web response provides a concise record of your code’s contract with the service, and the new type can encapsulate all the bridging from that web response, providing a more elegant interface to the rest of your code.

// Creating two separate data structures may be worthwhile. 

// WebContrastResponse defines the service contract.
struct WebContrastResponse: Decodable {
  let ratio: String
  let AA: String
  let AALarge: String
  let AAA: String
  let AAALarge: String
}

// ColorContrast is crafted for convenience and readability in the app's own code.
struct ColorContrast {
  let ratio: CGFloat
  let smallTextRating: Rating
  let largeTextRating: Rating
}

// This extension converts the service response type into the app's preferred data type.
extension ColorContrast {
  init?(webResponse: WebContrastResponse) {
    guard let ratioAsDouble = Double(webResponse.ratio) else { return nil }
    ratio = CGFloat(ratioAsDouble)
    smallTextRating = Rating(webResponse.AA, webResponse.AAA)
    largeTextRating = Rating(webResponse.AALarge, webResponse.AAALarge)
  }
} 

enum Rating {
  case doubleAPass
  case tripleAPass
  case fail

  init(_ doubleA: String, _ tripleA: String) {
    switch(doubleA, tripleA) {
      case ("pass", "pass"): return .tripleAPass
      case ("pass", "fail"): return .doubleAPass
      default: return .fail
    }
  }
}

Encodable—briefly

Encodable is quite similar, and probably doesn’t need an in-depth explanation here. However, there is one really useful configuration option on JSONEncoder. With .outputFormatting you can specify .sortedKeys. Most JSON decoders—including Swift’s JSONDecoder—won’t care what order the keys are in. But it’s very handy in automated tests to compare the JSON as a String or Data with some expected value. If the keys are not in a predictable order, such a test will fail. Even worse, it won’t always fail, because sometimes the dictionary will be in the exact order your test specified.

Author’s Note

You may be thinking this post ended a little abruptly. It is a reconstruction of the first half of a talk I gave at Swift by Midwest in Spring 2019. The second half of the talk covered ways to combine Codable and NSCoding to (1) use Codable Swift structs in State Restoration and (2) wrap NSCoding-conformant objects inside Codable types.


Previous post
They didn’t. “The [software developer tool] team clearly doesn’t use [that tool] themselves.” “The [popular app] team clearly had no testers.” As professional
Next post
Hello again, World! Look, it’s a blog. Maybe more stuff will show up here soon? Like all the posts trapped behind Medium’s nag wall right now?