Ep 024: You Are Here, but Why?

Christoph needs to test his logic, but he must pry it from the clutches of side effects.

• Last week we ended up with "imperative mud": lots of nested I/O and logic.
• (01:45) Difficult to test when all the I/O and logic are mixed together.
  • Object-oriented programming advocates for "mocking" objects with side-effects
  • Mocking reinforces and expands the complexity
• Complexity stems from all the different branches of execution: mainline, exceptions, corner cases, etc.
• With the imperative approach, no way to just "jump" to step 2 and test that in isolation. Have to setup all the mocks to tilt the flow of control in the desired direction.
• The Go language handles the "error flow" problem by making error returns explicit (no exceptions), but that further compounds the branching problem.
• (05:25) A different approach to deal with nesting: "happy path" route
  • Mainline of execution is the most common case.
  • Any error is thrown which has to be handled out of band.
• "Exceptions are much better at mocking you than you are at mocking them."
• Even more problems with mocking: suppose you want to test exponential backoff
  • API handle mock has to fail a couple of times and then succeed.
  • Requires a "stateful" mock!
• "You have whole object hierarchies that are only in use for your tests!"
• (07:35) Key observation: imperative programming uses the location of execution as a way of encoding information.
  • Code in the first branch of the "if" statement "knows" something different than code in the second branch.
  • Position in the flow of control is implicit information
• Change implicit information to explicit information and you can flatten out the code.
• Implicit: "You are here because you did X."
• Explicit: "Separating where I am from what to do next."
• "In an imperative flow, the fact you made it to line 3 means something."
• (09:55) To get out of the problem of nesting, we need to have positive, represented information instead of implicit, contextual information.
• Pattern is: logic, I/O, logic, I/O, logic, I/O, etc.
• Capture each one as a multimethod: a "worker" method and a "decider" method.
  • The "worker" method dispatches on a "command", just does one thing, and returns the result.
  • "My job is not to question why. My job is but to do or die."
  • "I don't know if we need to get into the class struggle."
  • The "decider" method takes a command and a result, looks at them, decides what to do next, returns a command.
• (18:50) You can work out all the scenarios, and the logic for each scenario is pure!
  • Testing becomes a matter of setting the data
  • Steps can be tested independently
• (19:45) A "result" isn't simply spewing back the raw data from the I/O
  • Worker picks out the relevant parts and return those
  • Use spec or schema to document responses
  • Can use meta or a nested key (like :raw) to attach raw data for debugging or the "attempt" log (see Ep 022)
• Logic should operate on well-known fields, but recording raw responses is useful for telling a story later on.
• (22:45) Main flow of execution becomes a simple loop + recur
  1. Dispatch command to worker and bind response
  2. Dispatch command + response to decider and bind new-command
  3. (when new-command (recur new-command))
• (24:30) A downside of the approach: it can be harder to see cases that are not covered, like an unsupported command or unhandled response
• Once again, unit tests can help ensure coverage for the "deciders".
• (27:30) How to test the worker?
  • They don't have any logic except doing the side effect.
  • Even "picking" the right values from the data could be factored as a pure function
  • Can test with the REPL. Try out the side effect and then leave it alone.
• (29:50) Application testing creates another problem
• What about testing the UI? We want to see changes in the data, but we don't want to actually post for real.
• Great topic for next time.

Message Queue discussion:

• (32:05) Do we post the source code for episodes?
• We post some code in the show notes, but less so for a high-level series.
• Send us links to your code and we'll put them on the website.

Related episodes:

• 021: Mutate the Internet
• 022: Evidence of Attempted Posting
• 023: Poster Child

Clojure in this episode:

• loop
• recur
• meta
• when