Vox

Leitura de 17 min

Proof of Complexity and Seeking Simplicity

Summary

The episode begins by acknowledging the inherent complexity in software development systems. Developers work with multiple interacting parts including clients, users, dependency management, inherited code, and domain concepts. These elements don’t exist in isolation but interact in ways that create exponential complexity, meaning developers often solve only small facets of larger, more complex problems.

The host illustrates how complexity grows exponentially using combinatorial mathematics. With 100 potential factors, considering single factors gives 100 possibilities, but considering pairs jumps to 4,950 combinations, and triples skyrockets to 161,700 combinations. This demonstrates how adding even one extra dependency or consideration creates future scenarios requiring evaluation of exponentially more possible pathways, though in practice developers can eliminate many possibilities through reasoning and tools.

Despite this reality, developers often have a bias against simplicity, gravitating toward complex solutions. This manifests in code reviews where simple solutions get comments about making code more maintainable through abstraction and refactoring. While these improvements often create better products, the tendency can lead to over-optimization, premature abstraction, and unnecessary complexity before establishing what’s actually needed.

The episode concludes with practical advice: write quality code but only enough to solve current problems. Look for red flags like solving future problems prematurely, over-optimizing, or abstracting before reuse is needed. Implement simple solutions first, then address only the insufficiencies rather than refactoring entire codebases. This approach maintains original simplicity while solving actual problems as they arise.

Recommendations

Tools

  • Manifold — A platform that provides a single workflow to organize cloud services, connect integrations, and share them with teams. It includes a marketplace for discovering services and manages authentication, messaging, monitoring, and content management tools.
  • Calculator Soup — A website mentioned as a resource for combination calculations, used to demonstrate how complexity grows exponentially with interacting factors in software systems.

Topic Timeline

  • 00:00:00 — Introduction to complexity in development systems — The host introduces the inherent complexity of software development systems, mentioning multiple interacting parts including clients, users, dependency management, and domain concepts. He notes that these elements interact in complex ways, creating situations where developers solve only small facets of larger problems. The JavaScript community’s dependency management issues serve as a concrete example of unexpected complexity.
  • 00:02:33 — Transition to discussing simplicity in complex systems — The host transitions from discussing complexity to introducing the episode’s main focus: how to chase simplicity when working inside complex systems. He introduces himself as Jonathan Cottrell and states the podcast’s mission to help developers connect to their career purpose. A sponsorship message from Manifold follows, explaining how managed cloud services can reduce complexity.
  • 00:05:07 — Mathematical demonstration of exponential complexity growth — The host provides a mathematical illustration of how complexity grows exponentially as interacting factors increase. Using combinatorial calculations, he shows that with 100 potential factors, single-factor consideration yields 100 possibilities, pairs yield 4,950 combinations, and triples yield 161,700 combinations. This demonstrates how adding even one extra consideration creates exponentially more evaluation pathways.
  • 00:11:19 — Developer bias against simplicity and toward complexity — The host discusses how developers often have a bias against simplicity, gravitating toward complex solutions. He describes scenarios where simple code solutions get comments in pull requests about making them more maintainable through abstraction and refactoring. While these improvements often create better products, the tendency can lead to over-optimization and unnecessary complexity.
  • 00:14:07 — Practical advice for seeking simplicity in development work — The host offers practical advice for seeking simplicity: remember that complexity grows exponentially, write quality code but only enough to solve current problems, and watch for red flags like premature abstraction. He recommends implementing simple solutions first, then addressing only insufficiencies rather than refactoring entire codebases. This approach maintains simplicity while solving actual problems.

Episode Info

  • Podcast: Developer Tea
  • Author: Jonathan Cutrell
  • Category: Technology Business Careers Society & Culture
  • Published: 2018-12-10T10:00:00Z
  • Duration: 00:17:37

References

Podcast Info

Transcript

[00:00:00] The systems we work on as developers are often complex.

[00:00:08] They have multiple parts, for example, clients, users, some kind of machine that is going

[00:00:16] to consume whatever code you are writing.

[00:00:19] That client might be code that uses your code.

[00:00:23] It might be an application that actually consumes your code in some way, or uses the output

[00:00:30] of your code.

[00:00:32] We have all kinds of dependency management problems.

[00:00:37] In the JavaScript community, for example, we’ve recently seen that dependency management

[00:00:44] can be not only difficult, but it can also have some problems that are unexpected, things

[00:00:51] like security issues.

[00:00:54] Now, we aren’t going to sit here and talk about how to fix all of those problems.

[00:00:59] Instead, just taking a moment to affirm the reality that the things that we do are complex.

[00:01:08] And that’s before we really even get to the actual problem that you’re solving, and the

[00:01:14] code that you’re inheriting from other developers or from yourself in the past, the concepts

[00:01:22] in whatever domain that you’re working in that you have to wrap your mind around and

[00:01:26] then create these virtualized representations of those concepts in your code.

[00:01:32] And the complexity continues to mount because all of these things, they can interact with

[00:01:37] each other.

[00:01:39] All of your code dependencies may actually have implications based on what domain you’re

[00:01:46] working in.

[00:01:47] Code you inherit absolutely has a relationship to dependency management.

[00:01:54] All of the problems that you face, and we’ve only gone through a short list, and that list

[00:01:59] is very long.

[00:02:00] We can’t cover every single thing that creates complexity in development, but all of those

[00:02:05] items can interact.

[00:02:07] And so when you face a problem as a developer, very often you’re only solving small facets

[00:02:15] of a larger, more complex problem.

[00:02:19] And every problem has different context, and those different contexts mean that your solution

[00:02:25] may look completely unique from the next person because of all of those interplaying factors.

[00:02:33] But in today’s episode, I want to talk about the other end of the spectrum, the simple

[00:02:39] end of the spectrum.

[00:02:42] And I want to discuss the idea of how you can chase simplicity when working inside of

[00:02:49] a complex system.

[00:02:51] My name is Jonathan Cottrell, and you’re listening to Developer Tea, and my goal on this show

[00:02:54] is to help driven developers connect to their career purpose and do better work so they

[00:02:58] can have a positive influence on the people around them.

[00:03:02] Today’s episode is sponsored by Manifold.

[00:03:05] You’re probably using managed cloud services.

[00:03:08] If you’re working on anything of any level of complexity, as we’ve already discussed,

[00:03:14] then these managed cloud services actually help you reduce that complexity by eliminating

[00:03:20] the time and effort that you would need to put in to build your own stuff.

[00:03:24] You wouldn’t go and build your own logging platform or try to create your own physical

[00:03:28] servers for really basic server needs, right?

[00:03:32] You wouldn’t go and create your own authentication protocol and service.

[00:03:36] You can use a managed tool or an API that can solve those problems for you, but how

[00:03:42] do you know which tool to integrate?

[00:03:45] How can you learn to stitch all of those tools together, and how do you end up managing the

[00:03:50] access that you and your teammates have to those services?

[00:03:55] Managing all these details and finding these tools that integrate well together can, on

[00:04:00] its own, be a full-time job, but Manifold exists to make your life easier by providing

[00:04:06] a single workflow to organize your services, connect to your integrations, and ultimately

[00:04:12] share all of that with your team.

[00:04:14] You can discover the best services for your project in the Manifold Marketplace, or you

[00:04:18] can bring your own custom integrations and manage them all in a single dashboard.

[00:04:23] With services covering authentication, messaging, monitoring, content management, and plenty

[00:04:29] more, Manifold will keep you on the cutting edge so you can focus on building your project

[00:04:33] rather than focusing on problems that have already been solved.

[00:04:36] And by the way, Manifold is 100% free.

[00:04:41] You pay for the services that you use, but Manifold, the integrator that is working to

[00:04:46] help you solve these problems in a more streamlined way, that part is free.

[00:04:51] On top of that, Manifold is providing you with a $10 credit.

[00:04:55] If you head over to Manifold.co.au slash DevT, you can get started with that $10 credit today.

[00:05:00] Thank you again to Manifold for sponsoring today’s episode.

[00:05:03] So we’re going to talk about simplicity in just a moment, but before we do that, I want

[00:05:07] to kind of drive home this idea of complexity, specifically with relation to how things change

[00:05:16] when they’re interacting with each other.

[00:05:17] We mentioned the multiple dimensions that we have to be thinking on as developers, but

[00:05:23] let’s talk for a moment about how complexity changes as you add these kind of interplaying

[00:05:30] factors.

[00:05:31] So imagine that you have 100 factors that could cause an issue, a problem, a bug in

[00:05:38] your code.

[00:05:39] And we know that we usually have more than 100 factors, but let’s just say that you have

[00:05:44] 100 lines of code and maybe you’re trying to decide which line of code is really the

[00:05:51] source of your problem, right?

[00:05:53] This is kind of a bad example because stack traces help us with these things and not every

[00:05:58] line is equal to every other line, but stick with me here, 100 factors that could go wrong.

[00:06:05] Now if there was only one thing that caused the problem, if there was only one dimension

[00:06:11] to your problem, then you have 100 possibilities.

[00:06:16] Okay, this isn’t so bad, right?

[00:06:18] Because we can pretty quickly eliminate half of those just by kind of eyeballing the problem,

[00:06:25] maybe reading a stack trace and doing a quick Google search, we can probably eliminate maybe

[00:06:30] even 75% or 90% of those potential issues, which leads us to 10 remaining problems.

[00:06:39] This is something we can manage because we can typically kind of look through what’s

[00:06:45] happening in the problem and try different solutions for those 10 issues.

[00:06:49] When we start changing the number of interacting factors, that number goes up and it goes up quickly.

[00:06:58] Let’s say for example that any two unique factors in that 100 could be causing your problem.

[00:07:07] Well it doesn’t go to 200, instead it actually goes up to 4,950.

[00:07:14] This is a combination calculation, and please correct me if I’m wrong if you know something

[00:07:20] more about combination calculations, but complexity grows at an astounding rate.

[00:07:26] Now warning, I’m going to try to explain what this little formula is so you can do it on

[00:07:32] your own if you’d like.

[00:07:33] We have n number of objects, in this case 100, and then r number of samples, and you

[00:07:39] can find this if you just Google combinations calculator, you’ll find this the same calculator

[00:07:43] I’m using to explain this on Calculator Soup is the website.

[00:07:48] But the formula for this combination, this complexity calculation, is that n number,

[00:07:55] 100 factorial, divided by, and this is all one number, the sample factorial, in this

[00:08:03] case 2 factorial, times the object number that n minus r factorial.

[00:08:11] So in this case it’s 100 factorial divided by 2 factorial times 98 factorial.

[00:08:19] If the numbers were 20 and 5, then the calculation would be 20 factorial over 5 factorial times

[00:08:27] 15 factorial.

[00:08:29] And for those of you who don’t remember what a factorial is or you never heard, the simple

[00:08:33] definition of a factorial is the product of a number, a whole integer, and all of its

[00:08:39] preceding whole integers.

[00:08:41] So 5 factorial would be 5 times 4 times 3 times 2 times 1.

[00:08:46] Okay, so why does all of this matter?

[00:08:49] And am I telling you to go and do math?

[00:08:50] Of course not.

[00:08:51] But I really want you to grasp just how complex things can get, right?

[00:08:57] If we decide to go with 3 rather than 2, remember we have 100 possibilities and any grouping

[00:09:04] of 3 unique items, that number jumps up to 161,700 different combinations.

[00:09:13] Now that’s order independent, by the way.

[00:09:16] That means that these combinations are not including combinations of the same items reordered.

[00:09:23] Those are unique combinations that you can make out of 3 different items with 100 sample

[00:09:29] size, with 100 possible items.

[00:09:31] And you can imagine that any given problem that you face in any code base of some reasonable

[00:09:39] size with any kind of domain information that is being added, any kind of packages that

[00:09:47] you’re depending on externally, all of these things are added complexity.

[00:09:53] All of these things could be part, a factor to be considered.

[00:09:59] So the complexity of what we do is enormous, sometimes.

[00:10:03] And sometimes it’s so enormous that it takes us days to figure what we feel like should

[00:10:09] be a very simple thing to figure that thing out.

[00:10:13] And for those of you wondering, when you go to 4, that number jumps to 3,921,225 different

[00:10:21] combinations of 4 items when you have a sample size of 100.

[00:10:26] So this is a very important thing to grasp because as developers, when we add even one

[00:10:33] simple thing, one extra dependency, one more thing to think about, when we don’t keep things

[00:10:43] as simple as we possibly can, we create future scenarios where we have to evaluate exponentially

[00:10:52] more possible pathways.

[00:10:55] Now I will admit that this may be an exaggerated example because you’re not going to walk every

[00:11:02] single one of those pathways.

[00:11:04] You’re probably going to be able to eliminate a large portion of the possible interactions

[00:11:10] between various code pieces.

[00:11:12] But there will be times when very unexpected things will happen in your code.

[00:11:19] So we have a bias towards complexity as developers.

[00:11:25] Or maybe a better way to put it, we have a bias against simplicity.

[00:11:30] Perhaps because we have worked in code bases that end up being complex, and the things

[00:11:37] that we appreciate about software development often are complex.

[00:11:43] And when we see simple things, we at least see them composed into complex applications.

[00:11:49] We see simple code that is, for example, abstracted and has multiple levels of indirection.

[00:11:56] And so our natural instinct, maybe as developers, is to gravitate towards complex solutions.

[00:12:04] And we may even push others that we work with to gravitate towards those complex solutions.

[00:12:11] And we can feel this happening all the time.

[00:12:15] So for example, imagine that you found the simplest solution to a problem that you were

[00:12:22] trying to solve.

[00:12:24] Now that problem or the solution to the problem doesn’t have any kind of abstraction.

[00:12:29] You’re not pulling classes out.

[00:12:32] You’re not refactoring that code.

[00:12:34] You’re not adding new dependencies.

[00:12:37] You’re solving the problem all in the same place and in a very straightforward way.

[00:12:44] Now very often, if you were to write that kind of solution and submit it in a PR based

[00:12:52] on some kind of internal culture or perhaps based on what we’re used to as developers,

[00:12:59] we often will comment on how to make that code more maintainable.

[00:13:04] How to improve that code.

[00:13:07] And it goes from simple to less simple.

[00:13:11] Perhaps not egregiously complicated.

[00:13:14] And most likely those comments are reasonable.

[00:13:18] And the tradeoffs are often worth it.

[00:13:21] And you end up with a better product in the end.

[00:13:25] But in the end, we often avoid submitting that original simple solution and we instead

[00:13:35] try to jump ahead.

[00:13:38] We move away from that simple solution and we gravitate towards complexity.

[00:13:44] And sometimes that gravitation takes us too far.

[00:13:49] Sometimes we over optimize.

[00:13:51] We abstract classes out before we really need to.

[00:13:56] We refactor and then we refactor again.

[00:13:59] We try out multiple external dependencies before we settle on one.

[00:14:04] So my challenge to you is this.

[00:14:07] As you go throughout your work this week, when you encounter a problem, remember that

[00:14:14] complexity grows and it doesn’t grow linearly.

[00:14:19] It grows very fast.

[00:14:21] It grows exponentially.

[00:14:23] So that just enough is not only the most efficient way to solve a problem in the beginning, but

[00:14:31] it also happens to be the most sustainable in the long run.

[00:14:36] If you’re writing quality code, and I want to make this distinction, the code can’t be

[00:14:45] poor quality.

[00:14:47] You can’t choose, for example, nondescriptive variable names or difficult to understand

[00:14:53] method procedures because you don’t want to write a little bit of extra code or you’re

[00:14:58] trying to save byte space.

[00:15:00] That’s not what we’re talking about here.

[00:15:03] So I encourage you to write quality code, but write only enough to solve the problems

[00:15:09] that you are trying to solve.

[00:15:11] If you’re trying to solve future problems or if you’re trying to over optimize your

[00:15:15] code or abstract it too thoroughly and make it reusable before you ever really even need

[00:15:21] to reuse it, those are the signs, those are kind of the red flags that you’re creating

[00:15:28] a more complex solution to a simple problem.

[00:15:32] So focus and seek simplicity.

[00:15:34] And once you actually implement something that is simple enough, then you can decide

[00:15:41] what about it is insufficient.

[00:15:44] And do this amongst your peers.

[00:15:45] Do this in pull requests, in code reviews, do it in a pairing session.

[00:15:51] Try the simple thing first.

[00:15:53] And if it’s not sufficient, then solve the thing that is insufficient.

[00:16:01] Don’t try to refactor all of the code.

[00:16:03] Don’t try to abstract all of the code.

[00:16:06] Only solve the insufficiencies.

[00:16:09] This will maintain the original simplicity and instead of actually trying to go through

[00:16:15] and clean up all of the code, this is something that we do as developers, we try to clean

[00:16:20] up a PR or refactor the entire change set, instead, focus on refactoring those small

[00:16:28] pieces that are insufficient for solving the problem in the way that maybe your company

[00:16:34] standards require, for example.

[00:16:38] Thank you so much for listening to today’s episode of Developer Tea.

[00:16:42] Thank you again to Manifold for sponsoring today’s episode.

[00:16:44] You can get $10 worth of credit to use on any service on Manifold’s Marketplace.

[00:16:49] Head over to manifold.co slash devtea, that’s manifold.co slash d-e-v-t-e-a.

[00:16:55] Thank you again for listening.

[00:16:56] If you’ve enjoyed today’s episode, I encourage you to subscribe and whatever podcasting app

[00:17:01] you use.

[00:17:02] This episode is coming out on a Monday, but we release on Monday, Wednesday and Friday

[00:17:06] virtually every week.

[00:17:08] We have a break coming up for Christmas.

[00:17:10] We’re going to be doing some re-airing, but usually we’re releasing brand new content

[00:17:15] on a weekly basis more than once a week, usually three times a week.

[00:17:20] So if you don’t want to miss out on future episodes, I encourage you to subscribe and

[00:17:23] whatever podcasting app you’re using to listen to Developer Tea today.

[00:17:26] Thank you so much for listening and until next time, enjoy your tea.

Visão de gráfico