Solve Specific Problems by Composing General Solutions


Summary

The episode introduces a core principle for tackling complexity in software engineering and other fields: complex, specific problems are solved by composing general solutions. The host argues that in well-architected software, every individual piece is understandable, even if the overall mechanism is complex. By breaking a problem down into its most fundamental, understandable operations, one can identify the general solutions that apply.

The role of the software engineer is framed as that of a composer rather than a pure computer scientist. The engineer’s key skill is recognizing which general solutions are applicable to a given scenario and, just as importantly, which ones must be ruled out. As engineers progress in their careers, they develop intuition for not only what tools are available but which are most appropriate and optimized for the use case, including which provide the best future flexibility.

The process is presented as a two-sided coin: work must be broken down into its smallest, most understandable pieces precisely so that general solutions can be identified and composed. If a work breakdown does not yield pieces that can be solved with a general solution, it likely needs to be broken down further. This principle underlies major tooling platforms like AWS and Azure, which provide general building blocks for composing complex systems.


Recommendations

Communities

  • Developer Tea Discord — The host invites listeners to join the Developer Tea Discord community to discuss topics like this episode’s principle and career questions. It’s described as a free, supportive community for engineers looking to grow.

Topic Timeline

  • 00:00:00Reflecting on a complicated problem you’ve solved — The host asks listeners to reflect on a complex problem they have solved, noting that people often don’t think about the total complexity but rather the value generated or specific parts. This leads into the episode’s core principle about how complex problems are actually tackled.
  • 00:01:04The core principle: composing general solutions — The host states the central thesis: complex problems are solved by composing general solutions, especially in software engineering. He emphasizes that in maintainable software, every individual piece is understandable, and complex solutions are built by composing these understandable, generalized operations or tools.
  • 00:02:25Examples and the engineer’s role as a composer — The principle is linked to tooling platforms like AWS and Azure. The host reframes the software engineer’s job: it’s rarely to make general solutions, but to think about what general solutions apply and how to compose them. The engineer is more often a composer than a computer scientist.
  • 00:03:06The hidden skill of recognizing and ruling out solutions — A key skill for a good engineer is recognizing which general solutions are applicable in a scenario and which must be ruled out. Career progression builds intuition for which tools are not only possible but most appropriate, optimized, and offer the best future flexibility.
  • 00:04:03Practical application: breaking problems down — For those new to the concept, the practical method is to break any problem down into its most understandable chunks. This work breakdown is the necessary precursor to applying the composition principle. If the breakdown doesn’t yield something solvable by a general solution, it needs to be broken down further.

Episode Info

  • Podcast: Developer Tea
  • Author: Jonathan Cutrell
  • Category: Technology Business Careers Society & Culture
  • Published: 2022-08-19T17:30:00Z
  • Duration: 00:05:37

References


Podcast Info


Transcript

[00:00:00] I want you to reflect on a complicated problem that you’ve solved.

[00:00:16] Recently or sometime in the distant past, it doesn’t really matter.

[00:00:20] The principles in this episode will remain the same.

[00:00:24] We’re talking about solving complex problems in today’s episode of Developer Tea.

[00:00:31] And if you think back to a complex problem, it might get overwhelming to think about how

[00:00:37] complex that thing is.

[00:00:39] But if you’re like most people, you aren’t thinking about the total complexity of the

[00:00:45] problem that you solved.

[00:00:48] Maybe you’re thinking about the value that it generated once it was solved, or maybe

[00:00:53] you’re thinking about a specific part of that problem.

[00:00:58] And that speaks to the underlying principle that we want to talk about here, which is

[00:01:04] the simple fact that especially in software engineering and in most other fields as well,

[00:01:11] complex problems are solved by composing general solutions.

[00:01:18] I’m going to say this again.

[00:01:20] Complex problems are solved by composing general solutions.

[00:01:26] In well-architected and maintainable software, every individual piece is understandable.

[00:01:35] Now there may be some underlying mechanism that you don’t understand.

[00:01:39] For example, if there’s some complicated math that relies on, let’s say, regression analysis,

[00:01:44] you may not know exactly what those formulas are by heart.

[00:01:49] But if you were to break down each of the individual operations in that formula, you

[00:01:54] probably do know what those things are.

[00:01:56] You probably know what it means to divide one number by another.

[00:02:01] And by composing these individual operations, whether they’re mathematical or otherwise,

[00:02:08] you can create complex solutions.

[00:02:12] Once again, the composition of these generalized tools, these generalized solutions, creates

[00:02:18] the opportunity to solve a complex, specific problem.

[00:02:25] This principle is what tooling like AWS or Azure are built on.

[00:02:31] So as you reflect on that complex problem, point out in your mind what kind of generalized

[00:02:38] simple solutions, what kind of general solutions did you compose and how did they compose?

[00:02:47] The job of the software engineer is rarely to make general solutions, but instead to

[00:02:54] think about what general solutions apply and how they’re composed together.

[00:02:58] You are more often a composer than you are a computer scientist for the average software

[00:03:05] engineer.

[00:03:06] One of the kind of hidden skills or difficult to quantify skills of a good engineer is recognizing

[00:03:13] what kinds of general solutions are applicable in a given scenario and perhaps just as important,

[00:03:20] which ones must be ruled out.

[00:03:23] And as you progress in your career, you’ll begin to gain intuition for which ones are

[00:03:28] not only available or possible to use, but also which ones are most appropriate.

[00:03:35] Which ones are kind of optimized for any given use case and perhaps the most often

[00:03:41] forgotten skill in this particular kind of subset of skills is which of the tools, which

[00:03:49] of the general solutions provides you the best future flexibility, the ability to change

[00:03:55] more most easily in the future.

[00:03:57] But for now, if this is your first time encountering this subject, I encourage you to look at any

[00:04:03] problem you face and the way you kind of understand or kind of mechanically work through

[00:04:09] this is to try to break the problem down into its most understandable chunks.

[00:04:15] This kind of work breakdown is the opposite side of this principle.

[00:04:20] We know that we’re supposed to break work down into its smallest version, smallest pieces,

[00:04:25] but the reason we’re supposed to do this is so we can apply and understand how to compose

[00:04:31] these general solutions.

[00:04:34] If your work breakdown is not providing you something that can be solved with a general

[00:04:38] solution, it’s likely that it either wasn’t broken down properly or it needs to be broken

[00:04:44] down further.

[00:04:45] Thanks so much for listening to today’s episode of Developer Tea.

[00:04:49] If you enjoyed this quick discussion on this principle of solving problems through decomposition

[00:04:55] and then recomposing general solutions to solve specific problems.

[00:05:00] If you enjoyed this discussion, please join the Developer Tea Discord community.

[00:05:05] This is the kind of thing we want to talk about in that community.

[00:05:08] This and of course, all of your career questions are welcome there as well.

[00:05:13] Head over to developertea.com slash discord.

[00:05:15] That’s totally free.

[00:05:16] There’s no entry fee.

[00:05:18] We’re not trying to sell you anything there.

[00:05:20] That is just a community that’s intended to be a support, a place of support for engineers

[00:05:25] who are looking to grow in their careers.

[00:05:27] Thanks so much for listening and until next time.

[00:05:30] Enjoy your tea.