Vox

Leitura de 18 min

Practical Application of Information Theory

Summary

This episode connects Claude Shannon’s information theory to practical decision-making in software development and career management. The host explains that information’s core value lies in its ability to reduce uncertainty, drawing parallels to previous discussions on cost of delay and communication theory.

The discussion breaks down information transfer into three categories: information that reduces uncertainty, messages that contain no information (resulting in zero change in uncertainty), and misinformation that may increase uncertainty in some areas while decreasing it in others. The host emphasizes that misinformation isn’t necessarily malicious—it can include well-intentioned but incorrect hunches shared during debugging sessions.

The episode applies these concepts to real-world scenarios like team communication, codebase understanding, and career development. The host suggests that much of what developers do involves managing uncertainty—whether tracing bugs, onboarding new team members, or seeking feedback on resumes. The show’s mission of providing clarity, perspective, and purpose aligns directly with information theory, since clarity emerges from uncertainty reduction.

A practical exercise is offered: identify three areas of uncertainty worth investing in, choose one, brainstorm three ways to reduce that uncertainty, and implement one simple action. The host argues that when uncertainty is high and certainty is valuable, even suboptimal information-gathering steps typically provide positive returns. This makes uncertainty reduction a high-yield activity for career growth and decision-making.

The episode concludes by connecting back to the sponsor, Unblocked, which helps teams reduce uncertainty about their codebases by aggregating information from various sources. The host encourages listeners to share the episode with others who could benefit from understanding how information theory applies to their work and lives.

Recommendations

People

  • Claude E. Shannon — The mathematician who wrote the foundational paper on information theory in the mid-1900s. His work provides the theoretical basis for understanding how information reduces uncertainty, which the episode applies to practical decision-making.

Tools

  • Unblocked — A tool that provides development teams with helpful and accurate answers to questions about their codebase by complementing source code with relevant discussions from GitHub, Slack, Jira, Notion, Confluence, and other sources. It helps teams ship faster by reducing time spent digging for information.

Topic Timeline

  • 00:00:00 — Introduction to information theory and unrealized value — The episode begins by connecting to previous discussions about cost of delay and measuring unrealized value. The host introduces Claude Shannon’s information theory as a framework for understanding how we make decisions about uncertain benefits. Information theory is presented as analogous to cost of delay theory, focusing on how information reduces uncertainty for receivers.
  • 00:03:47 — Core value of information: reducing uncertainty — After the sponsor segment, the host reiterates that the fundamental value of information lies in reducing uncertainty. Common scenarios where developers lack information are described: digging through Slack channels, PRs, code, Jira tickets, and stale wikis. The challenge of maintaining shared understanding as engineering teams grow is highlighted as a key problem that information can solve.
  • 00:05:50 — Mechanics of information transfer and three categories — The host explains the mechanics of information theory with senders, receivers, and uncertainty levels. Three categories of messages are defined: those containing information (reducing uncertainty), those containing no information (zero change in uncertainty), and misinformation. Misinformation is clarified as not necessarily malicious but potentially misleading, using the example of a developer sharing an incorrect hunch about a bug’s source.
  • 00:10:49 — Why information theory matters for developers — The host explains why information theory is relevant to Developer Tea listeners beyond computer science foundations. Since the show focuses on decision-making, clarity, perspective, and purpose, information theory provides the fundamental material for making better decisions. The most valuable use of time involves identifying important areas of uncertainty and finding ways to become more certain about them.
  • 00:13:11 — Practical exercise: identifying and reducing uncertainty — A five-minute exercise is presented: identify three areas of uncertainty worth investing in, pick one, write three top-of-mind ideas for reducing that uncertainty, and choose one simple action to implement. The host explains that when uncertainty is high and certainty is valuable, even suboptimal information-gathering steps typically provide positive returns. Examples include seeking resume feedback or addressing neglected career growth areas.

Episode Info

  • Podcast: Developer Tea
  • Author: Jonathan Cutrell
  • Category: Technology Business Careers Society & Culture
  • Published: 2024-03-27T07:00:00Z
  • Duration: 00:17:28

References

Podcast Info

Transcript

[00:00:00] We’ve recently discussed the concept of cost of delay, and hopefully those

[00:00:13] conversations were useful for you.

[00:00:16] And we’re going to keep on discussing this, uh, this idea of different kinds

[00:00:22] of value represented by what you don’t yet have different kinds of

[00:00:29] unrealized value and measuring unrealized value.

[00:00:37] This is in many ways, one of the most important things that we can do in our

[00:00:44] startups, in our lives, we are making decisions about how to capture value

[00:00:51] that we care about and not just value that we care about, but value that is,

[00:00:56] uh, uncertain.

[00:00:59] We’re trying to predict some benefit of an action.

[00:01:06] And today we’re going to talk about this in terms of information theory.

[00:01:11] In the mid 1900s, Claude E.

[00:01:14] Shannon, uh, wrote a paper on this subject, and we’re not going to go

[00:01:20] through the paper, uh, directly.

[00:01:23] But we can basically frame information and information theory in a way that

[00:01:29] is analogous, uh, or at least it’s similar, uh, to the cost of delay theory.

[00:01:39] And a reminder cost of delay theory is what is some unrealized, uh, benefit or

[00:01:46] some unrealized cost as of present moment that would be realized either

[00:01:52] through action by prevention of that cost or delay by realization of that

[00:02:00] cost, then remembering that costs of delay does not differentiate between

[00:02:04] profit and loss, by the way.

[00:02:07] So this idea of information theory maps very similarly, and this is related in

[00:02:15] some ways to our discussion on communication, of course, communication

[00:02:19] and information are tightly related conceptually, but the more formalized

[00:02:24] version of information theory looks at things from a slight, slightly different

[00:02:29] angle, more of like an economic angle and more specifically the theory of

[00:02:36] information as laid out by Shannon kind of looks like this, you have some

[00:02:41] holder of information, some emitter, if you want to call it that you have

[00:02:47] a receiver of, of that information once it is, once it is emitted.

[00:02:53] And then the receiver has some level of uncertainty.

[00:02:59] Now, depending on what kind of information you’re talking about, is it

[00:03:01] a closed system where there is some, uh, kind of bound for the information

[00:03:07] that can be known, uh, then the uncertainty level can approach zero

[00:03:12] in that situation, we’ll discuss a little bit more about the specifics

[00:03:16] and how this kind of mechanically works, uh, after we talk about today’s sponsor,

[00:03:20] but the information that the emitter provides and that the receiver receives

[00:03:27] can reduce, uh, the uncertainty.

[00:03:32] It can reduce the uncertainty.

[00:03:34] We’re going to talk a little bit about this theory in more depth right

[00:03:37] after we talk about today’s sponsor.

[00:03:47] In this episode, we’re talking about the value of information, specifically

[00:03:51] reducing uncertainty.

[00:03:52] That is the core, uh, kind of way to describe the value of, of, uh, information.

[00:04:00] And it’s likely that you’ve experienced, uh, a lack of information recently.

[00:04:05] And in order to get it, you’ve probably spent hours digging through

[00:04:10] Slack channels, through your PRs, uh, maybe reading the code directly or

[00:04:14] Jira tickets, uh, maybe your wikis are, you’re trying to answer the questions there.

[00:04:18] Uh, we’ll talk in a little bit about misinformation.

[00:04:21] What about your wikis that have gone stale?

[00:04:23] How about the last time you bounced out of the ID to answer a colleague’s question?

[00:04:28] They’re looking for the same kinds of information, uh, that you might have,

[00:04:32] or that they might have locked in their heads, maintaining a shared understanding

[00:04:36] of the code base gets harder and harder as an engineering team grows, more

[00:04:40] services, uh, more, more repos to look across longer history of changes.

[00:04:46] And why did we make that change in the first place?

[00:04:49] It becomes time-consuming, uh, when you’re onboarding new team members or

[00:04:53] working on refactoring existing projects and unblocked solves these time sinks.

[00:04:58] It provides helpful and accurate answers to questions about your code base in seconds.

[00:05:03] The answers are specific to your team and application because it complements

[00:05:07] source code with relevant discussions from GitHub, Slack, Jira, Notion,

[00:05:10] Confluence, and more, like an extended team member who never sleeps unblocked.

[00:05:16] It’s aware of every decision, every discussion for every part of your code

[00:05:20] base and with unblocked teams will ship faster by spending less time digging for.

[00:05:25] Here’s the key word information and dealing with interruptions.

[00:05:29] Check out unblocked and get unblocked.com.

[00:05:37] So the basic theory of information, we’ve already discussed the, uh, the

[00:05:50] holder of the information that’s going to emit it in some way to a receiver.

[00:05:55] Uh, by the way, the theory of communication still lays over nicely on, on this.

[00:06:01] Um, the, the sender and the receiver of a message in this case, we’ve kind

[00:06:06] of, uh, looked at this from a slightly different angle because a message may

[00:06:11] or may not contain information, right?

[00:06:15] This is kind of the differentiation, uh, between this, these two theories.

[00:06:19] One of the critical differentiations is that the, the, uh, the message may

[00:06:24] or may not contain information, meaning that if you were to look at the receiver

[00:06:29] end and measure the amount of uncertainty before and after the absorption of a given

[00:06:36] message, it could be zero, right?

[00:06:40] It could be a Delta of zero rather.

[00:06:42] But if you have information to convey and the message, uh, contains that information,

[00:06:49] then the sender of the message, assuming, uh, the sender gets their message across,

[00:06:54] the receiver receives that message and they, uh, they actually receive the

[00:06:59] information in the message, then the uncertainty will go down.

[00:07:05] The uncertainty will reduce by some amount.

[00:07:08] Now the amount by which it reduces is very important to understand, uh, in

[00:07:14] bounded systems, the amount, uh, that a given message can reduce the uncertainty,

[00:07:22] uh, is kind of a utility of that particular message.

[00:07:25] It is the bandwidth of a given message, assuming that, uh, all of the content

[00:07:31] of that message was actually information.

[00:07:34] All right.

[00:07:35] So let’s, uh, kind of back out of the, uh, combination of these two theories

[00:07:40] or models, uh, explaining different kinds of information transfer.

[00:07:44] And instead we want to talk about kind of the mechanics of this and some

[00:07:48] important caveats associated.

[00:07:51] So we’ve talked a little bit about, uh, information that reduces uncertainty.

[00:07:55] We’ve talked about kind of no information.

[00:07:59] Uh, these are those attempts to, to convey some kind of information and the

[00:08:05] result being no reduction in uncertainty.

[00:08:09] But there’s a third category and hopefully you can see where this is

[00:08:12] going, the idea of misinformation.

[00:08:16] Now I don’t want this to be a loaded term because, uh, misinformation

[00:08:21] as I’m using it right now is not necessarily intentional or malicious.

[00:08:26] It is instead misleading, but not necessarily on purpose.

[00:08:32] Misinformation may come in the form of someone trying to

[00:08:35] explain what they think is wrong.

[00:08:38] Um, uh, let’s say that you are trying to trace down a bug in your software.

[00:08:43] Maybe another engineer has looked into it.

[00:08:45] They have a hunch and they share that hunch with you.

[00:08:48] And they may even share it, uh, in terms of what they believe, rather

[00:08:52] than calling it a hunch, they may say, I think it’s coming from, you know,

[00:08:56] this particular part of the code base.

[00:08:58] Now, while this, while there is some information possibly here, for example,

[00:09:03] you now know what this person has been looking into, right?

[00:09:06] There’s some uncertainty reduction there.

[00:09:09] There may also be some uncertainty addition that occurs.

[00:09:13] In other words, it’s not, uh, cleanly one direction or the other, right?

[00:09:18] You may have increased uncertainty in one area, uh, decreased uncertainty

[00:09:23] in another area and no change in uncertainty in a third area.

[00:09:27] So if you were to look at any piece of information that you receive and try to

[00:09:32] identify the areas of uncertainty that this information speaks to, right?

[00:09:39] Um, interestingly, there are many areas that almost every interaction, uh, we

[00:09:44] encounter, uh, that information is providing us some kind of change

[00:09:50] in our levels of uncertainty.

[00:09:52] Let’s say you have a, uh, a meeting with a report and the report, uh, does

[00:09:58] not want to share what they’re thinking or feeling with you, right?

[00:10:02] As a manager, you might believe that this only goes one way, right?

[00:10:08] That that kind of messaging that they are providing you, uh, is only increasing

[00:10:12] your uncertainty about what’s going on.

[00:10:15] But you’ve also received a decrease in another kind of uncertainty.

[00:10:21] For example, before the meeting, you could pretty unequivocally say

[00:10:24] that you knew less, uh, about this person’s state of mind than you do after.

[00:10:31] This doesn’t necessarily mean that you have an exact understanding of what’s

[00:10:34] going on, but the signals that this person gave had a positive amount of

[00:10:42] information available for you to learn from.

[00:10:46] So why is this all useful to talk about on this show?

[00:10:49] Why are we discussing this on developer team?

[00:10:52] Uh, certainly if you have a CS degree, you may have had some kind of, uh,

[00:10:56] information theory class, um, and it is, it is definitely a foundational

[00:11:02] concept in computer science, but that’s not necessarily why we’re talking

[00:11:05] about it on this show, uh, other than to kind of take advantage of the fact

[00:11:09] that many of you probably already know something about this theory to begin with.

[00:11:14] Instead, because so much of what we do on the show is about decision-making

[00:11:19] and that goes back to the idea of helping provide clarity, perspective,

[00:11:23] and purpose, all three of those things, uh, will help you enable decision-making

[00:11:29] in your careers, uh, information theory is kind of the fundamental material

[00:11:36] that you will use to make decisions.

[00:11:39] And so as a broad and general rule, using this theory as a basis of

[00:11:45] understanding, some of the most valuable ways you can spend your time is to

[00:11:51] decide what areas of uncertainty are important for you to focus on and how

[00:11:59] you may become more certain about those areas.

[00:12:04] This is very simple.

[00:12:06] Your, uh, gathering of information, measuring information, uh, going

[00:12:12] and figuring out who else may have this, this information for you, whatever

[00:12:16] it is that you’re using to reduce that uncertainty, deciding what areas of

[00:12:21] uncertainty you care about in your career, in your personal life, and

[00:12:26] then determining what ways you want to reduce that uncertainty.

[00:12:30] This is the, once again, the fundamental material that you’ll

[00:12:34] use, uh, to enable decision-making.

[00:12:38] If our goal on this show is to help provide clarity, perspective, and

[00:12:42] purpose, clarity, especially is in many ways, the opposite of uncertainty.

[00:12:48] Clarity is based on the ability to find information or reduce uncertainty.

[00:12:56] Not necessarily this theory is broad in its application, but it can be used

[00:13:03] in very narrow circumstances as well.

[00:13:05] So I’d encourage you as a quick kind of five minute, uh, investment, a five

[00:13:11] minute investment in yourself at the end of this episode, you know, we’ve

[00:13:14] known we’ve been doing some longer episodes.

[00:13:16] I’m going to cut this one short to give you your five minutes.

[00:13:19] If you’ve gotten used to the, uh, to the 20 minute episodes, we’re going

[00:13:22] to go a little bit shorter today.

[00:13:23] Take the five minutes.

[00:13:25] Okay.

[00:13:26] Take the five minutes and identify three areas of uncertainty that

[00:13:34] you think are worth investing in.

[00:13:37] Three areas of uncertainty that you think are worth investing in.

[00:13:40] Pick one of them and then write the top three quick ideas, top of mind ideas.

[00:13:46] Uh, these don’t have to be absolutely correct.

[00:13:49] Right.

[00:13:49] And I’ll tell you why in a second.

[00:13:51] But right, the top three ideas that you have for reducing uncertainty in that

[00:13:55] very important area of your life.

[00:13:58] All right.

[00:13:58] And here’s why they don’t have to be perfectly correct.

[00:14:01] Uh, there’s two reasons.

[00:14:03] The first reason, if there is sufficient uncertainty, in other words, if there’s

[00:14:07] a lot of uncertainty, especially, then anything that you choose, uh, to reduce

[00:14:16] that uncertainty is going to have some value.

[00:14:22] Think about it like this.

[00:14:24] If there is a lot of uncertainty and that is paired with a high value on becoming

[00:14:31] more certain, then any step you take is probably going to be more valuable than

[00:14:37] the cost of the step itself.

[00:14:41] So even if it’s a suboptimal version of information gathering, it’s probably going

[00:14:46] to leave you in a better off place.

[00:14:48] So pick three things that you think are going to reduce uncertainty the most.

[00:14:54] And finally choose one thing, one simple thing that you can do today or this week

[00:15:00] that will take you in the right direction.

[00:15:04] This seems simple and it is, it’s oversimplified in this podcast because

[00:15:08] we only have, you know, 15, 20 minutes to talk about this stuff.

[00:15:11] But if you can chase this one idea down of reducing uncertainty in important

[00:15:16] areas of your life, you can chase this idea down.

[00:15:19] You’re very likely going to have a positive result from this.

[00:15:22] All right.

[00:15:23] I want to hear more about what happens as a result of this.

[00:15:26] You may realize, Hey, you know what?

[00:15:28] I’ve been totally neglecting, uh, this part of my career growth, right?

[00:15:32] I’ve been totally neglecting, uh, seeking feedback on my resume.

[00:15:37] That’s a, that’s a very common one.

[00:15:39] You have a lot of uncertainty.

[00:15:40] How good is my resume?

[00:15:41] Right.

[00:15:41] That that’s probably an area that if you’re listening to this podcast right

[00:15:44] now, you don’t have a clear line of sight on your resume quality.

[00:15:49] That’s an area of uncertainty that would have a high yield

[00:15:52] of reducing the uncertainty.

[00:15:55] So what are some steps, right?

[00:15:57] So think about this, uh, in this manner and make it very practical for your

[00:16:02] situation, gathering information will reduce uncertainty, reducing uncertainty

[00:16:07] can have some tangible benefit probably right away.

[00:16:11] Immediately in your career.

[00:16:13] Thank you so much for listening to this episode.

[00:16:15] And thank you again to today’s sponsor unblocked unblocked provides development

[00:16:19] teams, helpful and accurate answers to questions about their code base.

[00:16:23] It tailors answers by complimenting source code with relevant discussions

[00:16:27] from GitHub, Slack, JIRA, and more.

[00:16:29] And with unblocked teams can ship faster by spending less time digging for

[00:16:33] information and dealing with interruptions.

[00:16:36] Check it out and get unblocked.com.

[00:16:39] That’s get unblocked.com.

[00:16:42] If you find out that this episode has been valuable to you, if you go and do

[00:16:46] these exercises that I’ve mentioned, all I’m asking that you do, uh, try to

[00:16:51] ask for something back, uh, because we’ve done over 1200 episodes of the

[00:16:56] show and it’s all available, totally free, uh, to you.

[00:17:00] The one thing that I ask is if this provides any kind of valuable value

[00:17:04] back to you is that you share, uh, share this episode, specifically this

[00:17:08] one with someone you think is going to benefit from it.

[00:17:11] Someone else who you think could benefit from reducing uncertainty, um, and,

[00:17:16] and understanding this information theory.

[00:17:19] Thanks so much for listening and until next time, enjoy your team.

Visão de gráfico