Better Feedback Loops (Part One) - Validation and Cycle Time

---

Summary

In this episode of Developer Tea, host Jonathan Cutrell introduces a two-part series on feedback loops as the fundamental building blocks of learning and self-improvement. He explains that feedback loops consist of three stages: measurement (intake of information), evaluation (processing and interpretation), and reaction (taking action based on evaluation), which then affects the next measurement.

Cutrell emphasizes that feedback loops are powerful but not inherently good or bad—they can be harmful if poorly constructed, using examples like panic attacks (negative feedback) versus the body’s temperature regulation (positive feedback). He stresses the importance of intentionally designing feedback loops to avoid self-feeding negative cycles.

The discussion focuses on two key guidelines for building effective feedback loops: validating inputs and optimizing cycle time. Validating inputs means ensuring measurements are accurate and not corrupted by the measurement process itself—like using a faulty stopwatch for sprint times or developer estimates skewed by performance incentives. Optimizing cycle time involves finding the right balance between changing too frequently (causing churn and invalid measurements) and changing too slowly (missing improvement opportunities).

Cutrell connects these concepts, noting that an overly short cycle time can invalidate inputs by measuring only the effects of churn rather than true performance. He promises to explore how to decide what to measure and how to evaluate in the next episode, framing feedback loops as essential mental models for professional development, especially in software engineering.

Throughout the episode, Cutrell maintains that while feedback loops are simple in concept, their explicit construction requires careful thought to harness their power for continuous improvement rather than falling into detrimental patterns.

---

Recommendations

Organizations

• GiveWell — A charity research organization that evaluates charities based on effectiveness, focusing on preventing child deaths from preventable diseases and helping people in extreme poverty. They recommend top charities and match first-time donations up to $1,000.

---

Topic Timeline

• 00:00:00 — Introduction to self-improvement and feedback loops — Jonathan Cutrell opens by reflecting on the decade transition and the universal desire for self-improvement. He introduces feedback loops as the ‘operating system’ for getting better, framing them as fundamental to learning. The episode will focus on constructing effective feedback loops to avoid harmful cycles.
• 00:03:30 — Defining feedback loops: measurement, evaluation, reaction — Cutrell breaks down feedback loops into three stages: measurement (intake of information), evaluation (processing/interpretation), and reaction (taking action). He explains it’s a continuous cycle where each reaction affects the next measurement. Examples include the body’s temperature regulation as a natural feedback loop.
• 00:07:07 — Feedback loops can be harmful: panic attack example — The host cautions that feedback loops aren’t always beneficial—they can become self-feeding negative cycles. He illustrates with a panic attack: shortness of breath triggers anxiety, which causes faster breathing/hyperventilation, reinforcing the sensation of breathlessness. Escaping requires intentionally breaking the loop from outside.
• 00:11:08 — Two guidelines for better feedback loops — Cutrell introduces two key principles for constructing explicit feedback loops. First, validate inputs—ensure measurements are accurate and not corrupted by the measurement process itself. Second, optimize cycle time—the time between measurement and subsequent measurement after a change. Both are crucial for meaningful feedback.
• 00:11:39 — Validating inputs: faulty measurements corrupt feedback — Using examples of a faulty stopwatch for sprint times and developer estimates skewed by performance incentives, Cutrell explains that invalid inputs render feedback loops useless. He emphasizes validating both technical measurements and human-level inputs to avoid measuring distorted signals rather than the intended target.
• 00:13:56 — Cycle time: balancing change frequency and stability — The host discusses cycle time—the interval between measurement and re-measurement after a change. Changing too frequently leads to churn and invalid measurements (like constant team process changes causing endless onboarding). Changing too slowly misses improvement opportunities. Optimal cycle time allows genuine assessment of changes.
• 00:15:32 — Interconnection of validation and cycle time — Cutrell ties the two guidelines together: an overly short cycle time can invalidate inputs by measuring only the effects of churn rather than true performance. He concludes that validating inputs and optimizing cycle time work hand-in-hand to create effective feedback loops, setting up the next episode on deciding what to measure.

---

Episode Info

• Podcast: Developer Tea
• Author: Jonathan Cutrell
• Category: Technology Business Careers Society & Culture
• Published: 2019-12-09T10:00:00Z
• Duration: 00:16:18

References

• URL PocketCasts: https://pocketcasts.com/podcast/developer-tea/cbe9b6c0-7da4-0132-e6ef-5f4c86fd3263/better-feedback-loops-part-one-validation-and-cycle-time/1f7c831f-e470-442c-90fe-42aba6cf0bd7
• Episode UUID: 1f7c831f-e470-442c-90fe-42aba6cf0bd7

---

Podcast Info

• Name: Developer Tea
• Type: episodic
• Site: http://www.developertea.com
• UUID: cbe9b6c0-7da4-0132-e6ef-5f4c86fd3263

---

Transcript

[00:00:00] we’re in the last month of the last year of this decade and when the clock rolls over the time

[00:00:13] won’t be any different although our perceptions may convince us otherwise but when the clock

[00:00:18] rolls over midnight and it’s 2020 then we start a new decade and this change this moment in time

[00:00:27] has many people looking back at the last 10 years and thinking about what they’ve done

[00:00:34] what they’ve become what they’ve contributed to and also thinking forward to the next 10 years

[00:00:43] imagining what those 10 years will hold almost unanimously if you were to ask the people who

[00:00:50] are listening to this podcast right now would you like to improve in the next 10 years

[00:00:56] you

[00:00:57] almost unanimously everyone would say yes similarly if you were to ask people if they

[00:01:03] feel like they have improved of the last 10 years once again most people maybe not as

[00:01:10] unanimous as the first question but most people would say yes to some degree i have improved

[00:01:16] and there’s a lot of other interesting questions that we could ask about looking back in time

[00:01:21] versus looking forward in time but in today’s episode i want to focus in on that self-improvement

[00:01:27] aspect what is our kind of operating system for self-improvement my name is jonathan cutrelli

[00:01:34] listening to developer t my goal on this show is to help driven developers like you find clarity

[00:01:39] perspective and purpose in your careers and this is a common topic on this show how do we get

[00:01:45] better and this is such a big topic we’ll never be done talking about it and there’s not you know

[00:01:51] three points that we could summarize in a single episode that would tell you how to get better

[00:01:56] better and better and better and better and better and better and better and better and better and

[00:01:56] There are a lot of principles in self-improvement, and there’s a lot of luck involved in becoming

[00:02:02] better as well.

[00:02:04] But I want to talk about one fundamental aspect of improvement, and this is centered on the

[00:02:11] process of learning, and more specifically, on the process of feedback loops.

[00:02:17] We’ve talked about feedback loops on the show before, but we’re going to dive in a little

[00:02:21] bit more on this concept of feedback loops and constructing the right feedback loop for

[00:02:27] whatever you are improving on.

[00:02:29] And in fact, we’re going to take this discussion across at least two episodes because there’s

[00:02:35] so much to discuss about feedback loops.

[00:02:39] They seem simple when you first think about this.

[00:02:42] When you first think about the idea of a feedback loop, you think, okay, well, I want to take

[00:02:46] some kind of information after I do a thing and use it to change my feedback loop.

[00:02:51] I want to change in some way whenever I do the thing again.

[00:02:54] And this basic understanding of feedback loops is better than no understanding at all, but

[00:02:59] there are really important things to keep in mind when you build your feedback loops

[00:03:03] because you can end up building a feedback loop that is more harmful than it is helpful.

[00:03:10] So we’re going to talk about ways to explicitly structure your feedback loops so that they

[00:03:16] are more effective.

[00:03:18] And we have at least eight things that I want to talk about.

[00:03:21] In these two episodes, there may be more as we go along.

[00:03:25] But first, I want to kind of step back and talk about what exactly we mean when we say a

[00:03:29] feedback loop.

[00:03:30] So because it’s a loop, we can start at any point in that loop, but we’ll start with the

[00:03:36] measurement. This is often considered kind of the beginning of a feedback loop.

[00:03:40] But again, there is a continuous loop happening.

[00:03:43] So there’s not really a beginning, but we start with the measurement, then we move to the

[00:03:48] evaluation stage.

[00:03:50] We take.

[00:03:51] Whatever we measured and we make sense of that measurement in some way, we we interpret it and

[00:03:57] then we decide what to do about what we interpreted.

[00:04:02] And the thing that we do affects the next measurement.

[00:04:07] So in other words, as we measure and evaluate and then react to some stimulus or some something

[00:04:16] that we can measure, that reaction will affect.

[00:04:20] The next measurement, a very simple example of this is actually found in the human body.

[00:04:27] As it turns out, not all feedback loops that we experience are ones that we set up explicitly.

[00:04:32] In fact, most of the feedback loops we experience are not explicit.

[00:04:38] But as you sit in a room, as you are sitting on, let’s say you’re sitting at your desk and the

[00:04:47] temperature in the room begins to get colder and colder.

[00:04:50] Your body will respond.

[00:04:54] It’s not a measurement in the same sense that you measure the temperature by checking the

[00:05:00] thermostat. You’re not involuntarily checking a thermostat, but the signals that your body is

[00:05:05] sending to your brain tell it, hey, it’s getting colder.

[00:05:10] And in order for me to maintain my body temperature, my normal body temperature, then I need

[00:05:16] to respond in some way.

[00:05:18] Specifically, if it’s cold enough.

[00:05:20] Our bodies kind of automatically redistribute and where the blood is going in our bodies.

[00:05:25] Instead of going to our skin, it will go to our torso.

[00:05:29] And in the simplest version of this feedback loop, we would continue making adjustments like that one

[00:05:36] until our core body temperature is stable at what it’s supposed to be at in the same is true when

[00:05:43] cooling down, if the temperature is rising and it’s higher than our body temperature, if we are in an

[00:05:50] environment where the temperature around us is going above 110 or 120 degrees, our body will

[00:05:57] respond in a different way, once again, until we get back to that normal temperature. And not all

[00:06:04] feedback loops are as simple, and not all of them necessarily terminate like your maintaining of the

[00:06:12] body temperature would, but all learning starts at this fundamental concept of some kind of

[00:06:21] measurement or intake of information. It doesn’t necessarily have to be a direct measurement,

[00:06:26] although when you set up your feedback loops, that’s most likely what you will look for,

[00:06:30] the thing that you can measure that is meaningful. And then evaluation, some kind of processing of

[00:06:37] that information. And the reason that the evaluation is important, that we separate that

[00:06:41] as a state, is because it’s a very important part of the process. And it’s a very important

[00:06:42] we become clear when we talk a little bit more about this. And then the reaction, what do we do

[00:06:48] about what we’ve evaluated? And then the cycle starts all over again. And the reason that we

[00:06:56] talk about this in terms of a cycle is so we can understand that sometimes, once again,

[00:07:01] feedback loops are not to our advantage. They can be harmful to us. A good example of a feedback

[00:07:07] loop that is well-intentioned, but ultimately,

[00:07:10] kind of self-feeding in a negative way, would be a panic attack. Imagine that you’re walking up a

[00:07:18] flight of stairs and you lose your breath. And losing your breath kind of triggers some fight

[00:07:24] or flight mode. You feel some level of anxiety. And because of that anxiety, you start to breathe

[00:07:31] a little bit faster, causing you to hyperventilate and feeling the sense of losing that breath again.

[00:07:38] And so the cycle continues.

[00:07:40] And it’s hard to escape that cycle without intentionally interrupting it from the outside,

[00:07:47] breaking that feedback loop and sending a different signal. So it’s important to understand

[00:07:54] that feedback loops are incredibly powerful. They’re not always good. And they’re certainly

[00:08:00] not always bad, but they are kind of the fundamental building block of learning.

[00:08:08] We’re going to take a quick sponsor break, and then we’re going to come back

[00:08:10] and talk about a few guidelines for building good feedback loops.

[00:08:16] Support for today’s episode comes from GiveWell.

[00:08:21] Giving is hard. When you donate, how do you know what a charity can actually accomplish

[00:08:26] with your money? For example, imagine that you want to help children. And you found two

[00:08:32] trustworthy organizations, but they run entirely different programs. One can save a child’s

[00:08:39] life for every $300,000donated,whiletheothercansaveachil{d}^{\prime }slifeforevery$3,000 donated.

[00:08:48] Now, if you could tell the difference up front, you probably put all of your money on the one

[00:08:52] that was a hundred times better at saving children’s lives. And this is exactly what

[00:08:57] GiveWell helps you do. It’s very difficult to know what charities will actually accomplish

[00:09:03] with your donation. GiveWell spends 20,000 hours each year researching which charities can do the

[00:09:11] most with your money. They recommend a short list of the best charities they’ve found,

[00:09:16] and they share them with donors like you. GiveWell focuses on preventing children from

[00:09:23] dying of cheaply preventable diseases and helping people in dire poverty, and that’s what your

[00:09:28] donation can go and impact. You can learn how much good your

[00:09:33] donation can do.

[00:09:33] by visiting GiveWell.org slash DeveloperT. And here’s the important part.

[00:09:38] Their recommendations are free for anyone to use. And number two, GiveWell doesn’t take any cut of

[00:09:44] your donation. Finally, the last point here. First-time donors will have their donation

[00:09:49] matched up to $1,000 if they donate through GiveWell.org slash DeveloperT. That’s GiveWell.org

[00:09:57] slash DeveloperT. Once again, your first donation is going to be matched up to $1,000

[00:10:02] and GiveWell doesn’t take a single cent of what you give. Thanks again to GiveWell for sponsoring

[00:10:08] today’s episode of DeveloperT. This topic on feedback loops, this is an incredibly important

[00:10:15] topic. It’s kind of fundamental to being a good software developer. And you can go back

[00:10:20] in previous episodes where we’ve talked about this in cross-reference. There’s a ton of

[00:10:25] information on what it means to have good feedback loops. Feedback loops are a topic of conversation

[00:10:32] in professional development. They’re not just a topic of conversation. They’re a topic of conversation

[00:10:32] in professional development. They’re a topic of conversation in professional development. They’re a

[00:10:32] every professional circle. And it’s a meaningful conversation because there’s such an important

[00:10:40] mental model to carry with you. This idea that you have some kind of stimulus, some kind of

[00:10:46] evaluation to make meaning from that stimulus, response, some kind of action that you take as

[00:10:53] a result of the meaning that you made, and then things start back over. That action that you took

[00:10:59] somehow affects the next measurement, the next stimulus,

[00:11:02] that you receive. Very quickly, for the end of today’s episode, I want to cover two guidelines

[00:11:08] for making your explicit feedback loops a little bit better. And as expected, this topic is big

[00:11:16] enough to span multiple episodes. We said at least two. We’ll probably end up doing three

[00:11:21] because there’s so much to talk about with feedback loops. But I want to give you this

[00:11:24] first guideline to get started with this discussion, and that is to validate the inputs

[00:11:30] of your feedback loop.

[00:11:32] Validate the inputs of your feedback loop. What does this mean? Well, let’s imagine that you are

[00:11:39] trying to get better at a sprint. You are a track runner, and you’re trying to improve your sprint

[00:11:47] times. And let’s imagine that you have someone who is taking your times on a stopwatch. But the

[00:11:55] problem is that the stopwatch is faulty. From time to time, when you press the button, there’s a bit

[00:12:01] of a delay.

[00:12:02] Also, from time to time, the stopwatch will randomly stop in advance. So it won’t actually

[00:12:09] count your time accurately. Well, if you knew that, then you would know that basing your actions

[00:12:17] or your reactions off of that stopwatch is probably a bad idea. The inputs are invalid.

[00:12:25] Another place where inputs are often invalid is when developers make estimates about the work

[00:12:30] that they’re going to do. In particular, if you’re going to do a sprint, you’re going to have to

[00:12:32] have a system where making an estimate, the estimate itself, is a measure of your job

[00:12:39] performance. Now the inputs are starting to get a little bit muddy because developers are going to

[00:12:48] be incentivized to make different estimates if it affects their job performance. And so instead of

[00:12:55] having accurate estimates, you start to have, quote, performance estimates. These are very, very

[00:13:02] important. And they’re not really measuring much other than the developer’s willingness to break

[00:13:10] from the norm. And so the thing that we’re trying to measure, which is the amount of effort

[00:13:15] necessary to complete a given type of work, well, it’s no longer available to us. We’ve

[00:13:21] changed the system so that the measurement, the thing that is being measured, is shifted,

[00:13:27] even though it’s called the same thing. It’s important to validate our inputs,

[00:13:32] not just at the conceptual level, like, for example, making sure that some kind of hard

[00:13:38] measurement is actually measured correctly, but also from the human level, making sure that the

[00:13:45] things that we are measuring, we’re not affecting them by measuring them. And this plays right into

[00:13:50] the next guideline that is incredibly important when you create your feedback cycles or your

[00:13:56] feedback loops, and that is the cycle time, the amount of time between a given measurement

[00:14:02] and a change that you make and try to measure again. If we try to change too often, then we can’t

[00:14:11] really accommodate to the change to accurately measure how well it’s going to work on into the

[00:14:17] future. And so we end up having this high churn of change. Let’s imagine that you’re trying to

[00:14:23] change your process as a team, and you’re changing so often, or you’re even changing the people on

[00:14:28] the team so often that nobody ever gets accustomed.

[00:14:32] It’s like this constant, endless onboarding, a treadmill of onboarding, if you will.

[00:14:38] But if you don’t change quickly enough, the feedback loop becomes less valuable. You don’t

[00:14:43] really have the ability to change in the ways that you need to change. And while your process

[00:14:48] may be stable and the outputs may be predictable, you may be losing a lot of opportunity for

[00:14:55] improvement. So these two things, these two ideas of validating your inputs,

[00:15:02] and then secondly, ensuring that your timeline, your cycle time is enough time to measure in a way

[00:15:09] that represents how well something is working, rather than measuring only the effects of the

[00:15:15] churn. These two things go hand in hand, because if your cycle time is too low, well, that

[00:15:21] invalidates your inputs, right?

[00:15:24] Your measurement ends up being invalid because your cycle time, your churn, your process churn ends

[00:15:30] up affecting the measurements.

[00:15:32] These two guidelines can help you make better feedback loops. But how did we decide on these

[00:15:39] feedback loops to begin with? How do we know what to measure and how to evaluate and then what to do

[00:15:44] about it? That’s what we’re going to talk about in the next episode of Developer Tea. Thank you so

[00:15:50] much for listening to today’s episode. Thank you again to GiveWell for sponsoring today’s episode.

[00:15:54] Head over to GiveWell.org slash Developer Tea and you’ll have your first donation matched up to $1,000.

[00:16:02] Today’s episode wouldn’t be possible without Spec.fm and our wonderful producer, Sarah Jackson.

[00:16:07] My name is Jonathan Cottrell, and until next time, enjoy your tea.