Episode 010: From Mud to Bricks

Christoph can't stand the spaghetti mess in main. Time to refactor.

• Main function does too much. We want cohesive pieces that each have one job.
• Two part problem:
  • Too much in the main loop.
  • Main starts and then never stops. Have to exit the repl to restart.
• We need 1) separate parts that can be 2) started and stopped.
• Main should just focus on the code needed for command line invocation.
• Let's get the parts component-ized!
• "It's one thing that I've become more sure of in my career is that no application is actually done." "Useful apps only get more complex."
• Internal dependencies are different than external dependencies ("libraries").
• Many internal dependencies create high coupling throughout the code.
• "Once everything starts touching everything you have to understand everything to understand anything."
• Like functions use parameters to limit scope, a component is the next level up and uses resource dependences to limit coupling.
• Each component implements a clear behavior (interface) and can be a resource to other components.
• Can understand component's behavior via its interface (and docs) without reading all the code--just like understanding a function through it's signature and docs.
• We use the "Component" library to make components in Clojure--has REPL integration too.
• Components to make:
  1. web server
  2. polling and fetching loop
  3. the core.async channel used between them
• The Lifecycle interface allows a component to be started and stopped.
  • start must return a reference to the "started" state
  • stop must return a reference to the "stopped" state
  • Gotcha: don't return a nil!
• To use Lifecycle, you'll need to make your component a "record".
• Two main goals of Component:
  1. allow stateful components
  2. define dependencies between components
• Surprise! Any reference can be a "component" as a non-lifecycle dependency.
• Write a function new-system which returns the component "system map"
• Mind your names. Make the system map key match a component's dependent field.
• "Component is a convenient way of being able to specify all those dependencies in a concise map, so you know this is the intersection of all of my application together."
• A component should be able to be understood alone.
• "Component is like giving you application parts as function parameters. Just like when making a function, you don't worry about how something gets passed in as a parameter."
• You still need to understand each of the parts, but you don't have to worry about where the part came from.
• At the top level, you can see all the parts together and how they are connected.
• Immutability gives you bulkheads between each of your components so you can safely reason about them separately.
• Use component.repl to start and stop the whole system without restarting the REPL!
• Need some tooling to keep the main thread from exit. Can use promise, deref and a shutdown handler (see below).
• "We can keep each ball of mud it its own little basket so all the mud doesn't ooze together."

Clojure in this episode:

• defrecord
• promise, deliver
• deref, @
• component/
  • Lifecycle
  • system-map
  • using
  • start, stop
• component.repl/
  • go
  • stop
  • reset

Related projects:

• Component
• component.repl

Code sample from this episode:

(ns app.main
  (:require
    [clojure.core.async :as async]
    [com.stuartsierra.component :as component]
    [app.component
     [fetcher :as fetcher]
     [web :as web]])
  (:gen-class))

(defn new-system
  []
  (component/system-map
    :new-search-chan (async/chan)
    :web (component/using (web/new-component) [:new-search-chan])
    :fetcher (component/using (fetcher/new-component) [:new-search-chan])

(defn -main
  [& args]
  (let [system (component/start (new-system))
        lock (promise)
        stop (fn []
               (component/stop system)
               (deliver lock :release))]
    (.addShutdownHook (Runtime/getRuntime) (Thread. stop))
    @lock
    (System/exit 0)))