#182 - Building a Quality-Driven Culture: Enhancing Quality Practices Using QPAM - Janet Gregory & Selena Delesie
Summary
In this episode, Henry Suryawirawan hosts Janet Gregory and Selena Delesie to discuss their book on assessing quality practices using the QPAM (Quality Practices Assessment Model). They explain that quality culture extends far beyond testing—it encompasses social aspects like feedback loops, learning, and psychological safety, which are foundational to technical excellence.
The conversation delves into the 10 quality aspects of QPAM, organized into social, socio-technical, and technical categories. The hosts emphasize that social practices like feedback loops should be prioritized over technical ones because they enable better collaboration and prevent defects upstream. They challenge the common misconception that defect management is the primary quality indicator, arguing it’s a lagging measure that reveals problems but not quality.
Gregory and Delesie share practical examples, including a story about using mind mapping early in development to uncover critical questions that prevented wasted work. They discuss how QPAM differs from maturity models by offering options rather than prescribing a single “best” path, allowing teams to consciously choose their improvement goals based on their unique context.
The episode concludes with advice for teams wanting to implement QPAM, including using process retrospectives to visualize workflows and identify improvement opportunities. The guests stress that understanding current reality is essential before planning improvements, and that asking uncomfortable questions often leads to breakthrough insights about quality practices.
Recommendations
Books
- Assessing Agile Quality Practices Using QPAM — The main book discussed in the episode, presenting the Quality Practices Assessment Model with 10 quality aspects and four dimensions for teams to evaluate and improve their quality culture.
- A Guide for Facilitating Quality Assessments — The companion book providing detailed guidance on conducting quality assessments, whether as external consultants, internal facilitators, or team members conducting self-assessments.
People
- Lisa Crispin — Mentioned as a co-author and thought leader in the testing space who has also worked with the guests on quality assessment models.
Tools
- QPAM Assessment Spreadsheet — A free downloadable spreadsheet tool mentioned for purchasers of the book to help conduct assessments and analyze results systematically.
Topic Timeline
- 00:02:20 — Selena’s career journey from tester to quality coach — Selena shares her background starting as a software tester and bridging the gap between programmers and testers before formal Agile practices existed. She discusses moving into coaching and consulting with a focus on creating sustainable, healthy environments where people can do amazing work. Her technical background allows her to connect with technical teams while emphasizing social and cultural aspects of quality.
- 00:05:08 — Defining quality culture beyond testing — The hosts explain that quality culture involves mindfulness, reflection, and thoughtful interactions—not just testing practices. They emphasize that testing is only a small part of quality culture, which includes communication norms, psychological safety, and how organizations operate. Quality culture means building quality into products through quality interactions and thoughtful work at every stage.
- 00:08:56 — Origin and purpose of the QPAM book — Janet and Selena explain they created QPAM after being asked to formalize quality assessments for clients. They developed the model to share with the industry so others wouldn’t need external consultants as bottlenecks. The book helps teams understand their current state and make conscious decisions about improvement rather than blindly following prescribed practices.
- 00:11:21 — The 10 quality aspects of QPAM explained — The hosts outline the 10 quality aspects: feedback loops, culture, and learning/improvement (social); development approach, quality and test ownership (socio-technical); and testing breadth, technical debt, automation/tools, deployment pipelines, and defect management (technical). They emphasize prioritizing social aspects first because they enable better technical practices and reduce defect management needs.
- 00:14:16 — Why defect management is lowest priority — Janet explains that defect management sits at the bottom because if other quality aspects are done well, defects become minimal. She notes they debated including it at all, but it serves as an indicator when other practices aren’t working. The key insight: measuring defects only shows how bad quality is, not how good it is—the goal should be defect prevention through better upstream practices.
- 00:16:44 — QPAM as an options model vs. maturity model — The hosts clarify that QPAM is not a maturity model because maturity models imply teams must reach the highest level. Instead, QPAM presents options so teams can consciously choose what’s best for their unique context, product, and people. This approach respects team autonomy and prevents gaming metrics while promoting psychological safety for honest self-assessment.
- 00:20:05 — The four dimensions: Beginning, Unifying, Practicing, Innovating — Janet explains the four dimensions: Beginning (starting from chaos or waterfall), Unifying (adapting to collaborative practices), Practicing (regular delivery with few defects), and Innovating (continuously pushing boundaries). These represent progressive states teams might choose based on their context, not prescribed maturity levels that all teams must achieve.
- 00:23:40 — Feedback loops as the most critical quality aspect — The hosts discuss why feedback loops are QPAM’s top priority, covering three areas: within delivery teams, between customers/stakeholders and teams, and between leadership and teams. They explain that feedback loops are often cut under time pressure but are essential for catching misunderstandings early. Good feedback loops shorten cycle times, increase value, and reduce costs by preventing wrong paths.
- 00:30:39 — Development approach beyond coding practices — Selena explains that development approach encompasses how teams work together—team composition, remote vs. co-located, quality ownership, prioritization methods, and definition of done. It’s not just about coding but includes requirements gathering, release processes, and team collaboration mechanisms. The approach should be conscious rather than ad-hoc adoption of frameworks.
- 00:37:39 — Testing ideas early through mind mapping — Janet shares a story about using mind mapping before feature development to test ideas early. In just 30 minutes, the team uncovered so many questions and risks that they postponed the feature, preventing weeks of wasted work. This illustrates how testing isn’t just about code—it’s about testing understanding, assumptions, and risks at the earliest possible stage.
- 00:40:31 — How to conduct quality assessments — The hosts outline the assessment process: gathering information, conducting assessments through interviews and process retrospectives, analyzing results, and reporting findings. They mention a downloadable spreadsheet tool for purchasers and emphasize that anyone can facilitate assessments using their guidance. Process retrospectives help visualize workflows and reveal misunderstandings.
- 00:45:03 — Revealing insights from quality assessments — Common revelations include programmers realizing the extensive work testers do (previously seen as ‘magic’) and teams understanding the complexity behind seemingly simple changes. Visualizing the entire process helps everyone see dependencies and communication gaps. These insights help teams make conscious choices about improvements rather than guessing what might work.
- 00:49:26 — Technical leadership wisdom: visualize, assess reality, ask questions — Janet advises visualizing processes to enable discussion and improvement. Selena emphasizes understanding current reality before planning next steps—like knowing where you are after being dropped blindfolded. Both encourage asking uncomfortable questions that challenge assumptions, as these often reveal breakthrough insights teams hadn’t considered.
Episode Info
- Podcast: Tech Lead Journal
- Author: Henry Suryawirawan
- Category: Technology
- Published: 2024-07-08T12:00:00Z
- Duration: 00:54:05
References
- URL PocketCasts: https://pocketcasts.com/podcast/tech-lead-journal/952099f0-a7bc-0138-e686-0acc26574db2/182-building-a-quality-driven-culture-enhancing-quality-practices-using-qpam-janet-gregory-selena-delesie/ad9b0a58-bf61-4f8c-8d53-a5bebe783bee
- Episode UUID: ad9b0a58-bf61-4f8c-8d53-a5bebe783bee
Podcast Info
- Name: Tech Lead Journal
- Type: episodic
- Site: https://techleadjournal.dev
- UUID: 952099f0-a7bc-0138-e686-0acc26574db2
Transcript
[00:00:00] in our book, we talk about 10 different aspects of quality. Culture is one, and it’s one of the
[00:00:06] top ones. It’s a social aspect. But testing is really such a small part of the overall
[00:00:13] thinking about equality culture. You have to have the testing practice, but it’s just a really
[00:00:19] tiny part of it. I often see that the feedback loops is the first one to get cut when they’re
[00:00:25] feeling pressured for time, where they’re too busy. So trying to focus on getting that in
[00:00:30] healthy shape, I don’t think that’s a compromise that’s worth making. If you don’t have a quality
[00:00:36] culture, if you don’t have good feedback loops, if you don’t have a good way to learn and improve,
[00:00:43] without those three social aspects, it doesn’t matter how good you are at your technical aspects,
[00:00:48] your quality is going to suffer. This is a question I get really often,
[00:00:54] is we put a lot of pressure on people. We put a lot of pressure on people. We put a lot of
[00:00:54] pressure on people. We put a lot of pressure on people. We put a lot of pressure on people. We put
[00:00:55] defect management at the very bottom. It really is the lowest priority, because if you get all the
[00:01:01] other things right, all of a sudden you don’t have to worry about defect management. It becomes a
[00:01:06] non-issue. We debated about putting it in at all. At one point, I was able to track in the
[00:01:11] organization I was in of several thousand people, 40 to 50% of people’s time is spent on these
[00:01:18] activities. Just focusing on the defects and managing them and tracking them isn’t going to
[00:01:24] If you measure the defects, it tells you how bad the quality is. It does not tell you how good it is.
[00:01:46] Hello, everyone. Welcome back to another new episode of the Tech Lead Journal podcast. Today,
[00:01:50] I have one repeat guest and one new guest, Janet Gregory and
[00:01:54] Selina Delisi. They just recently published a book, which I find very interesting for us to
[00:02:00] discuss. Hopefully, we can share some of the learnings from the new books so that you can
[00:02:04] improve your quality practices. Janet, Selina, welcome to the show.
[00:02:08] Thank you very much.
[00:02:09] Glad to be here.
[00:02:10] All right. Since Janet has appeared in the episode before, maybe this time I would like
[00:02:15] to ask Selina to share any kind of highlights or turning points from your career journey so
[00:02:20] that we can learn from you. Well, I started my career as a software tester.
[00:02:24] Interestingly, back in the day when I started, I did not know that was a real job despite going
[00:02:30] to school for computer programming. Yeah, so I quickly moved into managing and leading projects
[00:02:35] and teams. And one of the things that I enjoyed most was kind of bucking the trend of testers
[00:02:42] sat in their corner and developers sat in their corner and narrated me across. And I was at least
[00:02:49] in the company I was in, I was one of the early folks to try to bridge that chasm.
[00:02:55] And bring the programmers and testers together to work towards finding solutions together.
[00:03:01] And that worked really, really well. And for the projects that I worked on with
[00:03:05] the testers and the programmers, we were doing Agile ways of working without any
[00:03:09] formal Agile awareness. Like we didn’t know it was a thing at the time because this was
[00:03:14] in the early noughts. And then into the next organization I worked in, I helped to bring
[00:03:19] Agile informally into the organization. And said, oh my gosh, this was a lot of what we
[00:03:24] were doing.
[00:03:24] Excellent.
[00:03:24] And then finally, some of my favorite stories about how I was there, so the first one that I
[00:03:24] focusing on collaboration and communication and testing ideas before I wrote the code.
[00:03:29] And we just found that it was really great for everyone who was involved in the project.
[00:03:34] And we were more successful with the projects that we were working on as well.
[00:03:38] From there, I shifted into doing coaching and consulting around quality practices,
[00:03:44] as well as agile.
[00:03:46] And I’ve shifted into doing also executive coaching.
[00:03:49] So there’s a myriad of things.
[00:03:51] But the premise of all of the work that I do is really about focusing on
[00:03:54] the quality of the environment and the quality of the people.
[00:03:57] I do have the technical background to back up what I’m talking about and be able to
[00:04:03] speak with the technical people I work with,
[00:04:06] even though I don’t focus on that exclusively anymore.
[00:04:09] But I found that when we focus on creating environments that are sustainable and healthy
[00:04:14] for people, they’re able to do amazing work.
[00:04:17] And all of the other benefits and gains that the organization
[00:04:21] is looking for just follow suit.
[00:04:23] And that said, having some of the technical experience allows me to speak with really
[00:04:29] technical people who think maybe I’m too fluffy for them.
[00:04:32] So, you know, there’s a bridge of being able to have all of the social, cultural, as well
[00:04:38] as the technical, you know, experience to kind of bring it into a package that really
[00:04:42] goes a long way for everybody.
[00:04:44] Henry Suryawirawan, Thanks for sharing your story.
[00:04:46] I think hearing just what you said, it seems like quite reflected in the book, but I’m
[00:04:51] going to go back to the book, right?
[00:04:52] The gist of the book.
[00:04:53] So talking about quality practices and how the social part and also the technical part
[00:04:57] coming in together into the team.
[00:04:59] So maybe let’s move to the book itself, right?
[00:05:01] The book is titled Assessing Quality Assessments Using QPAM.
[00:05:05] But the first thing that I want to talk about is about the quality culture.
[00:05:08] So in your book, you basically mentioned all these assessments is basically to assess our
[00:05:14] team’s quality culture.
[00:05:15] But in the first place, what is actually quality culture?
[00:05:19] Like how can we define it?
[00:05:20] Henry Suryawirawan, Yeah.
[00:05:20] Yeah.
[00:05:21] So that’s probably the first question.
[00:05:22] That’s funny because I’ve done a short presentation on culture and keeping an agile
[00:05:28] culture as such.
[00:05:30] But when you start to think about what it means, it means the norms.
[00:05:33] How do people act?
[00:05:35] Do they do what they say, you know, walk the talk kind of thing.
[00:05:39] So culture is about how an organization works, that it’s their ecosystem as such.
[00:05:46] When you’re assessing a team, does that team fit into that organization?
[00:05:51] Or does it have its own norms?
[00:05:53] Does it have its own rituals?
[00:05:56] Selina, I’m sure you can add to that.
[00:05:58] Selina Yip, Yeah, I think just in terms of that quality aspect of culture to enhance
[00:06:04] what you’re saying, it’s how do we bring mindfulness and insight and reflection into the work that
[00:06:11] we’re doing, whether it’s writing code or writing a test or understanding our requirements
[00:06:17] or just in how we’re communicating with the people around us.
[00:06:20] And having that idea of, I want this to be a quality interaction, or I want this to be
[00:06:26] a quality test, and being thoughtful about that really makes a big difference.
[00:06:30] It saves the organization, the teams, customers, a lot of problems down the road, right?
[00:06:37] We’re building in a quality product because we have a quality culture.
[00:06:41] Henry Suryawirawan, I like the phrase that you use, right?
[00:06:44] The mindfulness and the reflection of how you conduct stuff, right?
[00:06:48] I think many people associate quality.
[00:06:50] culture with testing culture.
[00:06:52] Is this the right thing to think about, or do you have any different perspective here?
[00:06:57] Selina Yip, The way I look at it is testing is a subset of the activities you would do
[00:07:04] trying to build in that culture.
[00:07:07] I know in our book, we talk about 10 different aspects of quality.
[00:07:12] Culture is one, and it’s one of the top ones.
[00:07:14] It’s a social aspect.
[00:07:16] But how an organization, say, communicates, that’s part of the culture.
[00:07:20] It’s a top-down or bottom-up, does it do some of both, how does it communicate?
[00:07:27] So testing is really such a small part of the overall culture, or the overall thinking
[00:07:34] about a quality culture.
[00:07:36] You have to have the testing practice, but it’s just a really tiny part of it.
[00:07:41] Well, maybe not tiny.
[00:07:42] Yeah.
[00:07:43] Yeah.
[00:07:44] And with testing, it’s not just like you’re actually testing the software, testing the
[00:07:47] code, right?
[00:07:48] It’s we’re testing ideas.
[00:07:49] That’s basically what it comes down to all the way through, right?
[00:07:52] And when we can bring those questions early on, we’re helping to build that quality in
[00:07:58] as well.
[00:07:59] There’s so much more that goes into that, and you kind of touched on the mindfulness
[00:08:04] phrase that I used.
[00:08:05] I don’t believe that a lot of people would look at it that way.
[00:08:09] Probably not.
[00:08:10] But I mean, that is kind of like the underlying thing that I’m watching for.
[00:08:15] If there’s thoughtfulness that’s going into the work that people are doing, regardless
[00:08:18] of the particular situation.
[00:08:19] Yeah.
[00:08:20] Yeah.
[00:08:21] Probably it’s the unconsciousness, right?
[00:08:24] Like people don’t have this mindfulness thinking about the quality practices that they have.
[00:08:29] And they always associate like with testing, testing, automated testing, right?
[00:08:33] How much coverage do we have?
[00:08:34] How many testers do we have?
[00:08:35] How many test cases?
[00:08:36] So I think the point in your book that you try to bring is actually testing is just one
[00:08:41] part of the quality thing, right?
[00:08:42] And there are many other aspects.
[00:08:44] And I think maybe in the first place, what made you wanted to write this book?
[00:08:49] So like, do you find it difficult to actually assess people’s quality practices?
[00:08:54] Like maybe what made you write this book?
[00:08:56] We’ve both been asked to do assessments for organizations in our past.
[00:09:01] And Janet was asked by a particular client to, I think, just better formalize what that
[00:09:07] might look like.
[00:09:08] We came across a couple of other models and Janet said, Hey, would you want to help me
[00:09:13] build something out?
[00:09:14] And then we suddenly had our quality practices assessment model and said, well, now what?
[00:09:18] We think we have something of value to share with the industry.
[00:09:22] So we hummed and hawed and tried on some ideas and landed on, you know, the easiest thing
[00:09:26] to do, which actually turned out to be more work than we thought was to write a book about
[00:09:31] it so that we can get it out and share it with, you know, the industry, with our colleagues
[00:09:35] so that we weren’t a bottleneck that people could just pick it up and run with it as they
[00:09:38] wanted to.
[00:09:39] But is it common that many people poorly conducting their culture or, you know, like withholding
[00:09:46] their quality?
[00:09:48] And hence, I think, should every team try to think about doing this kind of assessment
[00:09:53] to actually know where they’re at and actually how to improve?
[00:09:57] I would like to see every team do an assessment like this.
[00:10:02] There’s lots of them out there, but if they know where they’re at, then they can start
[00:10:07] moving forward, right?
[00:10:09] If you never stop and assess whatever level you want to be at or whatever, you can assess
[00:10:17] small portion.
[00:10:18] What you’re doing, for example, and say, where are we?
[00:10:22] And if you never stop and look at that, then sometimes teams just start spinning their
[00:10:27] wheels and get in a rut and really don’t know what.
[00:10:31] They try fixing that and fixing that, but they don’t understand why or where they’re
[00:10:36] trying to even go.
[00:10:38] Yeah.
[00:10:39] So in the past, I’ve tried my own assessment as well, my own version of assessment.
[00:10:44] And actually it’s really hard.
[00:10:45] First is to define the model, right?
[00:10:47] Or like the questions to ask.
[00:10:49] And then after that, you probably have different kind of stages to actually represent where
[00:10:53] team at.
[00:10:54] I think it’s really hard kind of problem.
[00:10:56] So I really thank you for coming up with this kind of assessment, which I think is quite
[00:11:00] holistic.
[00:11:01] So maybe let’s move to the actually QPAM itself, right?
[00:11:04] The quality practices assessment model.
[00:11:07] So there are 10 practices that you include in this model and also four kind of stages
[00:11:13] for people to actually know where they’re at.
[00:11:15] So maybe let’s start with all the 10 practices.
[00:11:16] Okay.
[00:11:17] So let’s start with the 10 different practices first, like in the high level, what are those?
[00:11:21] And yeah, maybe we can talk from there.
[00:11:23] Yeah.
[00:11:24] So we’ll call the top level quality aspects.
[00:11:28] And then within each of the aspects, we have a number of practices identified.
[00:11:32] So the quality aspects that we’ve identified, the 10 are feedback loops and culture and
[00:11:37] learning and improvement.
[00:11:39] Those are the social ones.
[00:11:40] Development approach, quality and test ownership are the social technical.
[00:11:45] And testing breadth.
[00:11:46] Okay.
[00:11:47] So the technical debts, automation and tools, deployment pipelines and defect management
[00:11:53] are the non-social.
[00:11:54] So those are the technical ones.
[00:11:57] So there’s a lot in there.
[00:11:58] And I think a lot of people would be surprised that like we did a lot of work and we reached
[00:12:03] out to colleagues to kind of prioritize them in terms of what are the most important ones
[00:12:07] to focus on first to get your organization in better shape.
[00:12:13] And I think a lot of people would be surprised that we don’t focus on the technical ones
[00:12:16] first.
[00:12:17] We’re focusing on the social ones first.
[00:12:19] We start to get feedback loops in place early on, for example, like within our tests, within
[00:12:24] our code, our feedback loops within our team, feedback loops with our stakeholders, feedback
[00:12:28] loops for their customers.
[00:12:30] That really just helps us move everything else out for the other aspects down the road.
[00:12:36] What would you add to that, Janet?
[00:12:37] Yeah.
[00:12:38] How do we communicate?
[00:12:39] Those feedback loops is really about how do we communicate with each other?
[00:12:44] And if you don’t have those, then you end up with throwing it away.
[00:12:46] Yeah.
[00:12:47] You end up with throwing it over the wall and a lot of silos.
[00:12:50] So that was why we kind of chose that number one.
[00:12:53] It’s interesting.
[00:12:54] I often see in organizations, I do consulting and coaching working companies.
[00:12:59] I often see that the feedback loops is the first one to get cut when they’re feeling
[00:13:03] pressured for time, where they’re too busy.
[00:13:06] So trying to focus on getting that in healthy shape, I don’t think that’s a compromise that’s
[00:13:12] worth making.
[00:13:14] And that alone, focusing on that and doing that really well is going to help you get
[00:13:17] better.
[00:13:18] It’s going to help reduce costs, improve quality, increase value, all of the things that organizations
[00:13:23] want to have happen.
[00:13:25] That one conversation, it can take away so many misunderstandings.
[00:13:30] I think intuitively when people understand that software development is a socio-technical
[00:13:34] kind of systems involved, right?
[00:13:36] They actually understand that they should probably prioritize social aspect more, you
[00:13:41] know, all these feedback loops, communication, collaboration, psychological safety, and things
[00:13:45] like that.
[00:13:46] Yeah.
[00:13:47] And that’s why people do not have this same understanding about social technical.
[00:13:51] And that’s why probably intuitively for them is to actually focus more on the technical
[00:13:54] practices first.
[00:13:55] So things like, you know, CICD or automated testing and things like that.
[00:14:00] So I think I like in the book that you actually sequence these technical practices in the
[00:14:04] order of importance, right?
[00:14:06] And actually just Celina mentioned, you start with social and then some technical aspects
[00:14:10] at the end.
[00:14:11] So maybe a little bit here, like explain why do you think this order of importance is actually
[00:14:16] very important?
[00:14:17] Well, so one of the things that I’ll just say right off the bat, because this is a question
[00:14:23] I get really often is we put defect management at the very bottom, right?
[00:14:29] Because it’s really the lowest, it really is the lowest priority because if you get
[00:14:33] all the other things right, all of a sudden, you don’t have to worry about defect management.
[00:14:37] It becomes a non-issue.
[00:14:38] We debated about putting it in at all, but we put it in priority as if you have this,
[00:14:45] it makes the next thing.
[00:14:46] It makes the next thing easier, right?
[00:14:49] Because if you don’t have those social aspects, if you don’t have a quality culture, if you
[00:14:53] don’t have good feedback loops, if you don’t have a good way to learn and improve without
[00:14:59] those three social aspects, it doesn’t matter how good you are at your technical aspects,
[00:15:05] your quality is going to suffer the quality of the product, the quality of the process.
[00:15:11] So that’s kind of why we put those ones first for sure.
[00:15:14] Do you want to take the other one, Celina?
[00:15:16] I think you kind of hit it on the head, Janet, like to your point, you know, once you get
[00:15:22] the next things in place, the next things go, like if your development approach is in
[00:15:25] good shape, then quality and test ownership can more easily follow, right?
[00:15:31] And everything just kind of follows through one after another with more ease if we’re
[00:15:36] getting them in order one at a time.
[00:15:38] That said, there’s no requirements for any team to work through improvements in their
[00:15:45] team in that order.
[00:15:46] Yeah.
[00:15:46] They might want to focus on the areas that they’re struggling the most in also, right?
[00:15:50] But if you’re really starting fresh and you want to do a complete overall, by all means,
[00:15:56] work through the priority that, you know, we felt was more, more relevant and important.
[00:16:01] Yeah.
[00:16:02] And so just going to say that team may take, for example, development approach and just
[00:16:07] do an assessment on that to start with.
[00:16:10] Yeah.
[00:16:11] I like the way that you explained, like if you focus on the first few practices first,
[00:16:16] that the remaining practices will become easier to implement.
[00:16:19] I think this, again, intuitively makes sense if you understand this socio-technical aspect.
[00:16:24] But for people that also struggle to actually thinking about how to improve this quality
[00:16:28] assessment, I think this model is kind of like good, right?
[00:16:31] But I have one question because typically if we find this kind of model somewhere and
[00:16:37] you have different stages, they will call this maturity model.
[00:16:40] Why in your book, you specifically mentioned this is not actually a maturity model.
[00:16:44] So maybe explain a little bit.
[00:16:45] Neither of us are a big fan of maturity models.
[00:16:50] That’s probably the starting point because it indicates that if you’re doing things right,
[00:16:56] you have to be at a certain state, like you have to go to the highest level, like you’re
[00:17:00] not doing a good job otherwise.
[00:17:02] And we don’t think that’s true.
[00:17:03] We think that each organization and, you know, a team is unique and the environment that
[00:17:10] they’re in and the desires that they want for themselves should be respected.
[00:17:14] Right.
[00:17:15] And I think for one team, like hitting, moving from beginning to unifying to practicing might
[00:17:20] be enough in terms of their testing breadth and development approach, for example.
[00:17:24] And I say, you know what, we’re really happy here and we don’t need to go into the innovation
[00:17:29] stage.
[00:17:30] It doesn’t make sense for a product.
[00:17:31] It doesn’t make sense for our people and our customers don’t care enough.
[00:17:34] You know, like it could be something like that, right?
[00:17:37] So really it’s about respecting the uniqueness of each environment and the people and the
[00:17:43] product.
[00:17:44] And they’re able to make those decisions for themselves.
[00:17:45] The key is being aware of what possibilities might be there for them and really making
[00:17:52] a conscious choice rather than just blindly landing somewhere and saying, we’re the best
[00:17:57] because we’re here, but we don’t know where here is and we don’t know, you know, what
[00:18:02] the other options are.
[00:18:03] Right.
[00:18:04] So we want teams to, and organizations ideally, to make conscious decisions about what’s best
[00:18:09] for them rather than blindly landing somewhere.
[00:18:12] Yeah.
[00:18:13] So you’re explaining that, I like the way that you phrased, there are options that you
[00:18:17] can aspire to become, right?
[00:18:19] And then you consciously make decision rather than maturity model typically is kind of like
[00:18:23] released by, I don’t know, like consulting or some industry best practices and management.
[00:18:29] Once they see it, they just say, okay, we want to be the most mature organization ever,
[00:18:34] right?
[00:18:35] By following this maturity model.
[00:18:36] Sorry, can I just add to that?
[00:18:37] Yeah.
[00:18:38] Like you hit it on the head, like it’s not a best practices, right?
[00:18:41] It’s options available.
[00:18:42] Yeah.
[00:18:43] It’s options available.
[00:18:44] And when people are striving to maturity model, people will start to gain the system so that
[00:18:49] they believe that they’re high performing or world-class when they’re actually not because
[00:18:55] they need to work good to management, right?
[00:18:57] So moving against that is important and so that people can be honest with themselves
[00:19:02] and they feel safe to be honest with where they’re at and where do they want to land.
[00:19:06] And it’s okay.
[00:19:07] It’s okay that they don’t want to be at innovation, which kind of goes in part with the psychological
[00:19:12] safety aspect of culture.
[00:19:13] Right?
[00:19:14] And if we can promote a model that espouses that, I think that’s healthier for everybody
[00:19:19] too.
[00:19:20] Yeah.
[00:19:21] I think the spirit is the most important thing, right?
[00:19:22] So I find all this kind of assessment typically is like what you said, right?
[00:19:26] People will just game it because once you make it like a KPI, right?
[00:19:30] You will just find the metrics, right?
[00:19:32] What are the metrics that are being assessed and try to game that, you know, and try to
[00:19:36] actually make people feel that they’re actually improving.
[00:19:40] So I think the spirit is very important and we should not kind of like blame, right?
[00:19:43] So you mentioned about psychological safety, like not blame the people for the bad practices
[00:19:47] that they’re doing, simply because if it works for their context, like why not?
[00:19:52] So I think you define the four different dimensions you mentioned, right?
[00:19:55] It’s not like maturity models again.
[00:19:57] So there are four different dimensions and it’s kind of like the different terms that
[00:20:01] you use, maybe a little bit explaining here as well.
[00:20:04] What are the four different dimensions?
[00:20:05] Perfect.
[00:20:07] So we lengthen the idea of beginning because we’ve tossed around a lot of terms.
[00:20:12] We ask them up a lot.
[00:20:14] We decided everybody knows where they are.
[00:20:17] So beginning, it’s kind of where are you starting?
[00:20:21] And usually that’ll happen and it still amazes me how many teams are still moving to Agile
[00:20:27] that aren’t.
[00:20:28] Maybe they’ve come from Waterfall, maybe they’ve come from some chaos or something else, who
[00:20:33] knows.
[00:20:34] But the beginning, where are you starting?
[00:20:36] You know that you have issues, you want to get better.
[00:20:40] So we just said, okay, we’re beginning.
[00:20:42] And then the unifying is the idea that the team is starting to adapt to Agile practices.
[00:20:50] Do you call it Agile or not?
[00:20:51] I really don’t care.
[00:20:53] But they’re starting to think about collaboration more.
[00:20:56] They’re trying to get those communication lines better.
[00:20:59] So we’re talking about unifying.
[00:21:01] They’re doing the practices.
[00:21:03] They’re trying to understand.
[00:21:06] And then practicing is really our light.
[00:21:10] We understand.
[00:21:11] Yeah.
[00:21:12] We’re delivering our products on a regular basis, whatever regular basis means.
[00:21:17] We’re not having very many defects and we’ve got a continuous integration and delivery
[00:21:23] cycle going.
[00:21:24] We’re practicing, we’re doing well.
[00:21:28] And then innovating is those teams that, and I’ve been on a few and they’re so much fun,
[00:21:32] but they’re a lot of stress too, because you’re always pushing the edge and you’re innovating.
[00:21:38] You work together as that really great team.
[00:21:41] And you’re just constantly looking for how to better yourself, right?
[00:21:48] And so that’s innovating.
[00:21:49] Did I miss anything, Selina?
[00:21:50] Oh, no, you did great.
[00:21:51] And I would just add, you know, Janet said you could be Agile or we don’t care what you
[00:21:57] call it.
[00:21:58] I mean, we’ve titled the book Assessing Agile Quality Practices with QPAM, but it doesn’t
[00:22:03] have to be Agile.
[00:22:04] I mean, the particular practices embodied are just healthy practices for any software,
[00:22:10] technical organization, right?
[00:22:11] Irregardless of Agile, whatever comes after it, it doesn’t really matter.
[00:22:12] Like these are just healthy practices for any software organization.
[00:22:13] If we had them in place when I started my career a long, long time ago, we didn’t work
[00:22:14] in Agile then.
[00:22:15] But boy, oh boy, we would have been in better shape if we had been applying this stuff, right?
[00:22:16] I think the same could be said now, or 15 years from now.
[00:22:17] Thanks for explaining all these different dimensions.
[00:22:18] I think it kind of like makes sense, right?
[00:22:19] People always begins when they learn about new practice, right?
[00:22:20] So I think that’s a really good point.
[00:22:21] I think that’s a really good point.
[00:22:22] I think that’s a really good point.
[00:22:23] I think that’s a really good point.
[00:22:24] I think that’s a really good point.
[00:22:25] I think that’s a really good point.
[00:22:26] So, you have the beginning, you have then the unifying, right?
[00:22:41] Trying to gather the thoughts, adapting to the practices and understanding, actually,
[00:22:49] the jizz of the practices.
[00:22:51] And then you move into practices, maybe when those practices becomes a habit, right, you
[00:22:55] kind of like do all the time.
[00:22:57] And then innovating is always pushing the boundary,
[00:22:59] continuously improve from what you understand.
[00:23:02] I think this is probably the Suhari thing, right?
[00:23:04] So in Agile, you have this also loop, right?
[00:23:07] So yeah, I think I like this dimensions.
[00:23:09] So maybe if we can, right,
[00:23:11] let’s just touch on a little bit on a few of the practices
[00:23:14] to give people flavor,
[00:23:15] how they actually use this quality aspects and dimensions.
[00:23:19] So maybe let’s start with the most important one for sure, right?
[00:23:22] The feedback loops.
[00:23:22] We kind of like touch on a little bit here and there about it.
[00:23:25] So feedback loops is also one thing
[00:23:28] that kind of like brought up quite a lot recently
[00:23:30] about developer experience,
[00:23:32] about also the flow things, right?
[00:23:34] And also continuous delivery and all that
[00:23:36] all gathers around feedback loops.
[00:23:38] What is the most important thing here?
[00:23:40] How can people assess their feedback loops
[00:23:42] against all these four dimensions?
[00:23:45] All right, so we’ve divided it up.
[00:23:47] It was only tough to try to figure out practices
[00:23:50] around feedback loops.
[00:23:52] So we talked about within the delivery team themselves,
[00:23:55] things like how do programmers and testers talk to each other
[00:23:59] or anybody else on the team.
[00:24:02] So in a beginning,
[00:24:04] they probably have a throw it over the wall kind of thing.
[00:24:08] They don’t talk to each other.
[00:24:09] They report lots of defects, things like that.
[00:24:12] How are they actually talking to each other?
[00:24:15] Then we have the idea of between customers,
[00:24:19] stakeholders, and the delivery team.
[00:24:22] How do you interact with your customers?
[00:24:24] Do you?
[00:24:25] Do you even talk to them?
[00:24:27] Do you listen to what they have to say?
[00:24:29] Like if you were giving a demo,
[00:24:32] and I’ve watched teams, been on teams,
[00:24:34] where you give a demo,
[00:24:37] the stakeholders have lots of good things
[00:24:39] and nobody writes anything slow.
[00:24:41] They don’t pay attention.
[00:24:42] And then the third one that we talk about
[00:24:45] is between leadership and the delivery team.
[00:24:49] So in a beginning kind of thing,
[00:24:51] it’s probably hierarchical, one-dimensional.
[00:24:54] The leadership team says,
[00:24:57] hey, thou shalt do this.
[00:24:59] And everybody else goes running and does that.
[00:25:02] Changes priorities a million times
[00:25:04] because somebody tells them to.
[00:25:06] And so when you’re looking at those different aspects
[00:25:09] and those particular practices,
[00:25:11] you can kind of see that as, say,
[00:25:15] a practicing team will have multidirectional
[00:25:19] kind of feedback loops with their leadership team.
[00:25:21] They are not afraid
[00:25:23] because they have success.
[00:25:24] They’re not afraid to say,
[00:25:27] hey, you know what?
[00:25:28] That won’t work.
[00:25:30] Can we try this?
[00:25:31] And improve those kind of things.
[00:25:33] Whereas at the beginning,
[00:25:35] you’re really just doing what somebody tells you to do.
[00:25:37] So that’s how we’re assessing those feedback loops
[00:25:40] on those three dimensions,
[00:25:42] those three practices.
[00:25:45] Yeah, I think feedback loops,
[00:25:46] just like for any kind of leaders or management, right?
[00:25:50] Maybe again, like they are not aware about this concept.
[00:25:52] They think it’s fluffy, right?
[00:25:54] What do you mean?
[00:25:54] Feedback loops, right?
[00:25:55] So I think it’s not easily quantifiable.
[00:25:59] So maybe for management or people,
[00:26:01] when you want to explain,
[00:26:02] you should improve your feedback loop
[00:26:03] because like feedback loop,
[00:26:04] I think it’s very, very evident,
[00:26:06] you know, in agile practices, DevOps practices,
[00:26:08] this is probably the one thing that they focus a lot on, right?
[00:26:12] But for management to actually understand the concept of feedback loops,
[00:26:15] how would you explain this?
[00:26:17] So think of like when you do this assessment
[00:26:19] and you have to explain why feedback loop is the most important thing.
[00:26:23] Well, it’s all about
[00:26:24] check-in.
[00:26:24] Checking in and getting feedback.
[00:26:31] Right.
[00:26:32] So basically, we’re testing ideas, basically, as we’re going.
[00:26:36] So, you know, as a tester to a programmer, a business analyst to a tester, let’s say,
[00:26:43] like, we’re asking questions of each other frequently day to day about what’s going on
[00:26:48] and what’s our idea and what’s our design and checking those assumptions as we go, right?
[00:26:52] And then the programmer may be saying, hey, tester, come sit over here.
[00:26:56] I have some questions about what I’m doing and writing the code.
[00:26:59] Can you pair with me on that?
[00:27:01] So we’re constantly checking those things.
[00:27:03] And it’s the same as we’re communicating with stakeholders or customers or management
[00:27:08] to the team that we are ensuring that we’re constantly checking in to make sure that we’re
[00:27:14] on the optimal path to shorten our cycle time and do that with high quality so that our
[00:27:20] customers get high value.
[00:27:21] And so in the book, we’ve listed a lot of questions that could be asked for each of
[00:27:27] the specific areas.
[00:27:29] And some of them are a little qualitative, maybe a little fuzzier, really.
[00:27:34] But how does a tester or team member know what is ready for testing?
[00:27:38] I mean, like, what is your tool?
[00:27:39] What is your source?
[00:27:40] Is it a communication?
[00:27:41] Is it a sticky note on a wall?
[00:27:43] Is it something on the task board?
[00:27:44] Is it something that comes through where the code is stored?
[00:27:48] Like checking in on how do we know when something’s going on?
[00:27:51] When something’s ready to test?
[00:27:53] Does the team document decisions that they make?
[00:27:55] Right.
[00:27:56] So some of them are yes or no.
[00:27:57] And then where do we do that?
[00:27:58] It’s really like there’s a really good comprehensive, holistic set of questions that we’ve included.
[00:28:03] So really checking in on.
[00:28:05] And I think that the questions really get into the meat of what it is that you would
[00:28:09] want to be looking for, for any of the particular areas.
[00:28:12] But what it really comes down to, I think, if we’re talking to senior management, like
[00:28:17] what do they care about?
[00:28:18] Get it done faster.
[00:28:20] Save us money.
[00:28:22] Make our customers happy.
[00:28:23] Like there’s other things, of course, too.
[00:28:25] But those are often the predominant forces.
[00:28:28] And it seems a little counterintuitive that having more conversations and more check-ins
[00:28:33] is going to get us the fastest throughput with higher value and reduce our costs.
[00:28:38] But there’s good data that you could find in your own organization, never mind from
[00:28:44] other ones that show that that is the case.
[00:28:46] Because if we go too far down a wrong path, then you’re actually increasing time.
[00:28:51] You’re increasing costs.
[00:28:53] And down the road, maybe reducing the value that your customer is going to get because
[00:28:57] you didn’t give them what they actually wanted or needed.
[00:29:00] So those checkpoints are really helpful.
[00:29:02] It’s akin to like going on a road trip.
[00:29:05] And you have an idea of where you are and where you want to go, but you never check
[00:29:09] the map.
[00:29:10] And you don’t check the map anywhere along the route to know that maybe you’re 300 miles
[00:29:14] off track.
[00:29:15] You want to check in and know that you’re actually on the right path.
[00:29:18] And maybe that fits for people who drove without using a GPS.
[00:29:21] A GPS mapping tool back in the day, like I did once upon a time.
[00:29:26] But it’s the same thing.
[00:29:27] Yeah.
[00:29:28] Pardon?
[00:29:28] One of my family questions to ask to help them think about what feedback loops is, is
[00:29:34] how does leadership get the information they need to make decisions?
[00:29:40] That tells you lots about feedback loops between leadership and the delivery team.
[00:29:45] That’s what we’re talking about too.
[00:29:48] I love that question.
[00:29:49] So actually, that makes you reflect.
[00:29:51] I like a lot about how do you get the current situation of the team, right?
[00:29:55] Where the team at, what kind of struggle, challenges that they’re currently facing,
[00:29:59] right?
[00:29:59] And I like the way Selena mentioned, right?
[00:30:01] So it’s not about feedback loop of delivering features fast.
[00:30:05] It’s actually testing ideas, knowing the direction, right?
[00:30:08] Knowing that actually you are moving still in the right direction.
[00:30:11] I think these are all very, very important understanding that we have to have about
[00:30:15] feedback loops.
[00:30:16] So I think, thanks for sharing that.
[00:30:18] So maybe the other quality aspect that I want to touch on is about.
[00:30:21] It’s about development approach.
[00:30:23] So maybe when people talk about development approach, they look at the developers.
[00:30:27] How are you writing code or maybe things like architecture design.
[00:30:31] So maybe something more that you touch on in the book, right?
[00:30:33] Development approach is not just about writing code.
[00:30:36] So maybe brief us about this.
[00:30:39] So development approaches practice the team and we call it the delivery team.
[00:30:44] How do they work together?
[00:30:46] How is the team made up?
[00:30:48] Is it remote or is it together?
[00:30:50] Like those things.
[00:30:51] Things will affect how the team works.
[00:30:53] Doesn’t mean there’s a right or wrong.
[00:30:55] It just does.
[00:30:56] So things like quality ownership.
[00:30:59] Does the team really think that the testers own quality?
[00:31:03] And when we say delivery team, it’s programmers, testers, analysts, product
[00:31:08] owners, who is on your team and how do they do that?
[00:31:12] Thinking about what practices they do.
[00:31:14] Like if you have a, whatever agile framework you’re using will depend on the
[00:31:18] practices.
[00:31:19] If you’re doing Kanban versus.
[00:31:21] Scrum versus extreme programming versus whatever.
[00:31:25] So there will be different practices and we don’t specify those specifically.
[00:31:31] We just kind of, what’s important to you, trying to understand what that is.
[00:31:35] Do you have some kind of prioritization on your stories or your features you’re bringing in?
[00:31:42] How do you handle those sorts of things?
[00:31:44] Do you know when a story is done?
[00:31:47] What does that mean to you?
[00:31:49] How do you work with delivering it?
[00:31:51] How do you deliver it to production?
[00:31:52] How does the team work together?
[00:31:55] So sorts of things when we’re looking at a development approach is right from the
[00:31:59] very beginning, do you have a mechanism in there for asking the questions right
[00:32:04] through to delivery?
[00:32:06] How do you deliver it to production?
[00:32:07] What is that?
[00:32:09] Yeah.
[00:32:10] So I think it’s really important that the team actually have a mechanism or
[00:32:14] the practices in place, right?
[00:32:15] So first of all, it’s not like ad hoc, right?
[00:32:17] Like people just do whatever practice that they think they want to do.
[00:32:20] It’s like a method, an approach, right?
[00:32:22] And then like, you also holistically look at not just the code aspect, but also like
[00:32:27] things like gathering the requirements, right?
[00:32:29] The ownership about the code and the testing part, you know, sort of like how do you release
[00:32:33] that story or features to the customers, right?
[00:32:36] And then get the feedback loop again, back to the team.
[00:32:39] So I think development approach here is really important because I find so many
[00:32:43] teams probably, like they just take Agile framework, any Agile framework, and they think
[00:32:47] that’s the development approach, right?
[00:32:48] Take Scrum.
[00:32:49] So they just say, yeah.
[00:32:50] We do stand-up retrospectives and, you know, product backlog grooming and things
[00:32:53] like that, and they think that’s enough.
[00:32:55] So I think the point here is that it’s more beyond just the code and the framework,
[00:32:59] but it encompasses so many other aspects as well.
[00:33:03] So maybe the other quality aspects that I’d like to touch on is about the defect
[00:33:08] management, like interestingly, Jenna, in the beginning, you mentioned that you
[00:33:12] actually thought about removing this from the quality aspects, but for people
[00:33:16] actually to assess quality, normally again, intuitively, the first thing they will do
[00:33:20] is, how do we track defects?
[00:33:22] How do we lock defects?
[00:33:23] How do we know that they are closed?
[00:33:25] So maybe explain why defect management is put as a lowest order in the priority.
[00:33:31] Yeah.
[00:33:32] So with defect management, it’s A, why it’s lowest down is because if we’re doing
[00:33:37] really well in the other areas, we’ve improved in a way that the other areas are
[00:33:41] feeling really strong, the amount of defects that we have should be going down
[00:33:46] dramatically.
[00:33:47] So we will have less of a need to…
[00:33:50] track and manage those things.
[00:33:52] Certainly there may be some escapes, but we shouldn’t be going intentionally into
[00:33:58] releasing things with a lot of known issues, ideally.
[00:34:01] Right?
[00:34:01] So we’re doing really well in the other areas.
[00:34:04] Our need for management is reduced greatly.
[00:34:07] And one of the ways that we do that is when we find things, when we are in the
[00:34:12] process of developing, which includes the programming and the understanding and the
[00:34:16] testing, we fix them right away.
[00:34:18] We don’t consider things to be complete.
[00:34:20] We don’t consider things to be completed until we’ve addressed the issues that we
[00:34:23] found, right?
[00:34:24] In which case, what we have as defects out in the world are things that are being
[00:34:29] found by people outside of the team in general.
[00:34:32] And again, those should be a lot less if we’re doing the other things really well.
[00:34:36] So from there, we’re looking at issues around how do we report defects?
[00:34:40] How do we triage them and how do we report on them?
[00:34:45] Some organizations, they will choose to use defect management tools.
[00:34:49] Still.
[00:34:50] One of the things that we recommend that we’ve seen work really well is that defects
[00:34:54] get prioritized right alongside with new features.
[00:34:57] So we’re making decisions and we’re storing those things right alongside with any
[00:35:03] new features for the products that we have in the team’s backlog.
[00:35:07] So there’s a clear decision about what’s more valuable, this new feature fixing what
[00:35:12] a customer has reported.
[00:35:13] And that in itself takes away the need to have a separate defect management tool.
[00:35:17] That’s one option that we’ve both seen work really well.
[00:35:20] The time that’s invested in the reporting and fixing and triaging.
[00:35:24] I know early in my career, we spent a lot of time as testers, as programmers, as project
[00:35:30] managers, as all of these different roles in meetings and in sending emails and in
[00:35:35] dealing with the stuff, because there was a lot of problems, right?
[00:35:39] And at one point I was able to track in the organization I was in of several thousand
[00:35:44] people, 40 to 50% of people’s time is spent on these activities, right?
[00:35:49] So.
[00:35:49] It really, really, really is very helpful to focus on the other quality aspects and
[00:35:55] doing them well so that you can get that time back.
[00:35:58] Because just focusing on the defects and managing them and tracking them isn’t going to save
[00:36:03] you the time and the money that you want to save.
[00:36:05] You want to focus on fixing the other quality challenges you have in your process within
[00:36:11] your organization.
[00:36:13] And that in itself will address that and hopefully reduce it dramatically, which I did see happen
[00:36:18] in organizations that I worked with.
[00:36:19] Yeah, I’m only going to add one thing.
[00:36:22] I think if you measure the defects, what tells you how bad the quality is, it does not tell
[00:36:29] you how good it is.
[00:36:31] Excellent.
[00:36:32] Closing comments.
[00:36:33] Wow.
[00:36:33] I think that’s again, pretty deep, right?
[00:36:36] So you only know how bad you’re doing, but actually not how good you’re doing.
[00:36:40] And I think in your book, you mentioned that the goal is actually for defect prevention,
[00:36:44] right?
[00:36:44] Not actually defect detection.
[00:36:46] Exactly.
[00:36:46] I think many people try to detect a lot of defects.
[00:36:49] Right.
[00:36:49] Manage it, report that.
[00:36:51] I think when you mentioned about reporting defects, I kind of like laugh because in the
[00:36:54] past, I also used to experience those kinds of things and not just the report, but also
[00:36:59] the fighting in between, right?
[00:37:00] When the testers report something and developers don’t agree, you kind of like fight instead
[00:37:05] of prioritizing.
[00:37:06] So I think all this really important because in many teams’ culture, right, still defect
[00:37:11] management, you know, the Jira or whatever tools that they use is probably the center
[00:37:16] of their quality, so to speak, right?
[00:37:18] And I think very importantly in your book, you also mentioned that we should actually
[00:37:22] spend more time in understanding the kind of problems, the features that we’re doing
[00:37:27] and the kind of testing that we want to build from the very beginning, right?
[00:37:31] Not just something that you throw over the wall or you think about it at the end.
[00:37:35] So maybe a little bit on this part, how can people start thinking about understanding
[00:37:39] the problem first and, you know, like building the test cases in the earlier phase rather
[00:37:43] than the last?
[00:37:44] Right.
[00:37:45] So I’m going to tell a story here.
[00:37:48] And people don’t think about this necessarily as testing, but I do because it’s funny.
[00:37:54] But I was working with the team one time and they were all ready to start on a feature.
[00:37:59] They were going to have their kickoff meeting and just start going with this new feature.
[00:38:04] Product owner had done a bunch of work, they had stories ready to go.
[00:38:08] And I had been trying to coach them a little bit on maybe let’s start a little earlier
[00:38:14] thinking about some of these things.
[00:38:16] So I said, before we do that.
[00:38:18] Can we just take time, do it to the half an hour and let’s build a mind map.
[00:38:23] The product owner understands this feature really, really well.
[00:38:27] The team hasn’t seen it so much.
[00:38:30] So they’ll be learning about it, but asking questions, trying to understand.
[00:38:34] So can we just have a quick half hour meeting and build a mind map on what this feature
[00:38:39] is?
[00:38:40] And so, you know, the product owner was a little not sure, but he agreed and the team
[00:38:46] said, okay.
[00:38:47] We’re, they’re willing to try something new.
[00:38:49] So we had this half and we spent half an hour building a mind map.
[00:38:54] It was remote.
[00:38:55] We did it online.
[00:38:56] Before we finished that half hour, there were so many questions that had come up from the
[00:39:01] team, build it, bringing up risks, asking questions that had not been thought about
[00:39:07] before.
[00:39:09] That they actually took that feature off the table and said, we’re not ready.
[00:39:15] So that half hour.
[00:39:16] Prevented.
[00:39:17] Probably weeks and weeks and weeks of blocking stories, trying to understand this.
[00:39:23] They just took it off the table and went back and started asking some of those questions.
[00:39:28] That to me is testing early.
[00:39:31] We’re testing our ideas.
[00:39:33] We’re testing our understanding.
[00:39:35] We’re exposing risks.
[00:39:38] It’s just not testing the software.
[00:39:39] I think that’s a really powerful story that we can learn from, right?
[00:39:44] So I think this also comes back to the holistic testing approach.
[00:39:46] I think we covered that in the previous episode as well, right?
[00:39:50] So testing is not just testing the code, testing the product, the feature, but also testing
[00:39:54] the risk, the ideas, right?
[00:39:56] And make it early, as early as possible.
[00:39:58] I think it’s very, very important.
[00:40:00] So I really love the story, right?
[00:40:02] So I think it’s quite typical in many software teams, right?
[00:40:05] Some product owners or stakeholders will just say, okay, build me this feature, like in
[00:40:09] a few sentences.
[00:40:10] Maybe better if they have as a something, I want something.
[00:40:13] So that’s something, right?
[00:40:14] But it’s always not enough.
[00:40:15] Right.
[00:40:16] You miss the details, the edge cases, and maybe the variance of how the user journey
[00:40:21] will look like by using those features.
[00:40:24] So I think thanks for raising that.
[00:40:26] So knowing all these quality aspects, then the next important thing is actually to conduct
[00:40:31] the assessment itself, right?
[00:40:32] For teams who probably don’t have expertise, they cannot hire coaches like yourself.
[00:40:38] How would you advise people?
[00:40:39] Because I know that you are also writing the second book, right?
[00:40:42] To actually help people facilitate this kind of assessment.
[00:40:45] Yeah.
[00:40:46] So in the first book, the assessing agile quality practices, we do have a chapter devoted
[00:40:52] to how to conduct a quality assessment.
[00:40:54] It’s a brief synopsis of what we would recommend.
[00:40:58] And our second book, a guide for facilitating quality assessments goes into super detail.
[00:41:04] Everything that we would recommend, whether you are an external consultants, assessor,
[00:41:10] facilitator or internal, or you were a team member, and there’s actually a whole chapter
[00:41:14] devoted to I’m a team member.
[00:41:15] How do we want to do this?
[00:41:16] Yeah.
[00:41:16] How do we want to do this as a team?
[00:41:17] So the steps are gather information.
[00:41:20] How do we prepare for the assessments?
[00:41:23] Then we actually conduct the assessment.
[00:41:24] We gather everything that we need to do.
[00:41:27] And then how do we put all the information together in a way that is meaningful and assess
[00:41:32] all of the information that we pulled together.
[00:41:35] And then how do we want to report on the results for the team’s benefit, for the organization’s
[00:41:39] benefits, and then any follow up activities that you might want to do.
[00:41:43] So there’s a lot involved with each of those steps.
[00:41:46] Whether it’s how do you identify who to include in the assessment?
[00:41:50] Do you have examples of like stories or tests of code media that you want to look at?
[00:41:55] Do they do things like mind mapping?
[00:41:57] If so, like see examples of that.
[00:41:59] I don’t know if there are examples.
[00:42:00] I used to work for a company who we did these, that sort of thing.
[00:42:03] And I know Janet has too, but that’s not true everywhere, right?
[00:42:06] But if they have examples, you want to see that.
[00:42:09] And setting up all of the meeting invitations, getting the right people in the room, whether
[00:42:12] you’re in a room physically or doing it virtually.
[00:42:15] Yeah.
[00:42:16] And there’s also things like doing one-on-one discussions with people, like a selection
[00:42:21] of subset of people, or you’re doing group interviews with say, maybe all of the testers
[00:42:26] or all of the managers who are involved or whomever it makes sense to have those discussions
[00:42:31] with.
[00:42:32] But there’s a lot of sources that we’re pulling together.
[00:42:35] One of the really bigger activities that we would do in terms of conducting is what is
[00:42:39] called a process retrospective.
[00:42:41] So understanding from the very beginning of a project that we work on, all the way through
[00:42:46] to delivery, what are all of the steps and stages and activities that take place to understand
[00:42:53] like how it all fits together with all of the people involved in the room together.
[00:42:58] And oftentimes you will see light bulbs or questions or confusion pieces go off around.
[00:43:06] I didn’t know that was happening or what do you mean that’s what you do next?
[00:43:09] Like it’s a really great way for the whole team and the project team or the delivery
[00:43:15] team to get on the same page.
[00:43:16] But what is actually going on?
[00:43:19] Because they don’t often all know that, right?
[00:43:22] Yeah.
[00:43:23] And then pulling it together into reports.
[00:43:25] We actually provide a tool that you can do that with for those who’ve purchased the book,
[00:43:29] they can get the link.
[00:43:30] It’s a free downloadable spreadsheet that you can use to help you due to your assessment
[00:43:37] and analyze your results.
[00:43:40] So yeah, for us, it’s like, yes, we can do assessments.
[00:43:44] I know Janet’s not anymore.
[00:43:45] She’s in her retirement.
[00:43:46] Yeah.
[00:43:46] Yeah.
[00:43:46] She’s in her retirement case, but like myself and her other co-author, Lisa Crispin and
[00:43:51] like anyone who’s picked up the book, like frankly, you can pick it up and run with it
[00:43:55] and be able to facilitate an assessment, but you’re more than welcome to involve people
[00:44:01] who have done this before too, to learn from others who’ve done this a few times.
[00:44:07] I like the way that you, when you describe a process retrospectives, right?
[00:44:10] So it sounds like a value stream mapping by the way, right?
[00:44:13] It is.
[00:44:14] Kind of.
[00:44:15] It’s a little bit.
[00:44:16] It’s a little bit more similar.
[00:44:17] Yeah.
[00:44:18] Yeah.
[00:44:19] So it’s always a very revealing session where people start to understand, oh, what do you
[00:44:23] mean that we have this?
[00:44:24] Or what do you mean we are doing those kinds of stuff that are unnecessary, right?
[00:44:28] And it’s not just for the team, right?
[00:44:30] The leaders also, they sometimes don’t actually have a good understanding of what the team
[00:44:34] is actually doing and what are the process involved, or maybe those kinds of habitual
[00:44:39] things that are probably legacy or bureaucracy.
[00:44:42] So actually they may not even need to do that anymore, right?
[00:44:45] You can take the decision.
[00:44:46] They actually remove that from the process.
[00:44:49] Maybe any kind of interesting learning.
[00:44:51] I know that probably you have done this assessments a few times with your customers, any kind
[00:44:56] of revealing insights or revelation from your customers after they’re doing this kind of
[00:45:02] assessment.
[00:45:03] Maybe if you can share one or two.
[00:45:06] One of the biggest things that I’ve seen the most often actually is the programmers saying,
[00:45:13] I didn’t know that you did all of those things.
[00:45:16] Because testing is always, it seems to be one of those hidden things.
[00:45:22] It’s just magic.
[00:45:24] And so when you start putting up all of the little things that is being done, you will
[00:45:29] get a lot of that.
[00:45:30] I just didn’t know.
[00:45:31] We can help you with that.
[00:45:33] That’s what you want to hear, right?
[00:45:37] Yeah.
[00:45:38] I mean, I’ve heard that one too.
[00:45:40] Absolutely.
[00:45:41] I mean, the biggest aha really is.
[00:45:45] having it all there together in one visual picture is really just a big aha for everyone in the room
[00:45:53] about what it really takes to do the work that we’ve been asked to do that sounds like it’s just
[00:45:57] oh just put this change in really quickly la da da and it’s all beautiful right it’s not that
[00:46:02] simple it’s not that easy and when we know all of the steps that are involved yeah there are ways
[00:46:07] that we can find a way to accelerate in healthy ways some of those things by automating the right
[00:46:14] things at the right time but we have to be mindful about how we build quality into those
[00:46:19] steps too right so it’s not just we snap our fingers and it’s out the door right even with
[00:46:25] things like bringing ai in into organizations now like there are things that can be beneficial and
[00:46:31] helpful which i mean we didn’t touch on in this book but it’s the same it’s automation right how
[00:46:36] do we bring automation and to support us and help us and there are considerations that we need to
[00:46:40] make to ensure that we’re making good choices so regardless
[00:46:44] of how we look at the process and see how big it is and how involved it is and how complicated and
[00:46:50] all of these lines of communication and feedback loops that need to happen all over ensuring that
[00:46:56] we are front loading addressing quality rather than leaving it to the end is is really what we
[00:47:02] want to see people doing so that they are able to be as effective as possible and in my experience
[00:47:10] an activity like this helps people to start to make that connection
[00:47:14] and understand how to get there because now that they’ve looked at it they’ve had an assessment
[00:47:19] they’ve gotten their results back and they’re like oh okay we’re really struggling and like
[00:47:24] development approach testing reps and feedback loops let’s start with feedback loops first and
[00:47:29] then maybe the other ones will kind of domino effect down right and then they can make conscious
[00:47:35] choices about that but like having those results and really understanding where they are and why
[00:47:40] they’re having the problems that they’re having yeah that’s the big option and that’s the big
[00:47:44] aha that i see with teams and organizations and i would say with management in particular
[00:47:49] because they often don’t understand everything that’s involved and this gets them something
[00:47:54] tangible to hold on to to understand how they can get from this unknown like chaotic i don’t know
[00:48:01] why we are here but we’re still consistently not doing a good job into this is why we are here and
[00:48:08] these are the steps that we can take to get to where we want to go sorry there’s a lot of ideas
[00:48:12] but all of them there’s a few
[00:48:14] things there who gets to know yeah i think it would be really cool if every team has this kind
[00:48:19] of assessment you know they put together all these kind of quality aspects where they are at i think
[00:48:24] again a lot of teams probably don’t even understand where they’re at right and putting this together
[00:48:29] putting it as a map right and i think the most important thing also like the roadmap where
[00:48:34] for example if you’re in the beginning stage you know there’s a roadmap that you can aspire to to
[00:48:38] become the innovating dimension right so i think having this kind of roadmap that you put in
[00:48:44] place for a continuous improvement process within the team will be really really something that is
[00:48:49] cool i tried to build that i know it’s probably fail more than a successful one but i think yeah
[00:48:54] it’s really hard to come up with this kind of assessment model and i encourage people to check
[00:48:58] out janet’s and selena’s book right and maybe try to use that to assess where you are so we reach
[00:49:04] the end of our conversation but before i let you go i have one last question janet maybe you still
[00:49:09] remember last time i asked this i call this three technical leadership wisdom so think of it like
[00:49:14] last advice that you want to give to the listeners maybe if you can share the version of three
[00:49:19] technical leadership wisdom from any of you okay i’ll give one because it comes right out of what
[00:49:26] we just said what selena just said visualize visualize whatever whenever you possibly can
[00:49:34] because for example if you can see a process then you can discuss it and decide where you
[00:49:40] want to take it from now so i’m a big fan of
[00:49:44] visualizing and trying to draw a picture of it whether it’s a mind map a flow diagram
[00:49:49] whatever makes sense there’s so many things that we could add i would say working with reality
[00:49:56] i think is really important and the tools the model and some of the tools that we talked about
[00:50:02] around assessments i’m including visual like visualizing but where you are like if you just
[00:50:09] got dropped out of a plane with a blindfold on and you don’t know where you are in the world
[00:50:12] you want to figure out where you
[00:50:14] are before you make plans about what to do next right and it’s the same within the context of
[00:50:18] your organization and the teams and your products like taking some time and it feels like a slow
[00:50:25] down and you’re too busy to do that but taking that time to know where you are know your options
[00:50:30] for getting to where you want to go is really important and i see a lot of teams and organizations
[00:50:37] struggle to do that and that’s why consultants and coaches have jobs because we help facilitate
[00:50:44] them
[00:50:44] from a source to understand where you are and where do you want to go next so
[00:50:47] yeah that’s really important to do i’m not going to add a third one which i’m sure selena will agree
[00:50:53] with don’t be afraid to question especially if something is making you uncomfortable or is
[00:51:00] pushing your own ethics your own morals sometimes it’s it’s just something and you’re scared to ask
[00:51:07] that question because you think it’s a dumb question but often that’s the thing that can make
[00:51:14] a team just stop and say we never thought of that so break out of your comfort zone and ask the
[00:51:21] question one reframer that is like of courage but it’s not even that it’s you know having the
[00:51:27] willingness to sit in the uncomfortable space that you’re in and sitting in uncertainty
[00:51:31] and with the questions that you’re asking don’t ask why questions yeah go a little deeper than that
[00:51:39] yeah it kind of like also shows vulnerability right i think bernie brown has this concept
[00:51:44] right so the courage is actually the courage to actually be vulnerable right show vulnerability
[00:51:48] and ask these kind of questions right so i think it’s really important in a team where you have a
[00:51:53] lot of psychological safety this will be probably a key thing where you can start being a high
[00:51:58] performing one so really beautiful tech lead wisdom so for people who want to check out the
[00:52:03] book or they want to learn more any kind of resources that you have available online
[00:52:07] yeah so we have um both books are available for purchase on lean pub so
[00:52:14] you can look up our names or qpam both books will be coming up so assessing actual quality
[00:52:19] practices with qpam and a guide for facilitating quality assessments both books are also available
[00:52:25] for purchase on amazon worldwide as a physical copy so if you want a physical one that’s where
[00:52:30] you can go buy that otherwise website is horse coming that is our focus for the next few months
[00:52:36] and yeah otherwise you can find more about me on linkedin that’s selena delessi and janice
[00:52:43] bye
[00:52:44] my website is janic rickery’s ca.ca so yeah from there you can find me anywhere but if you google
[00:52:55] me i usually come up pretty quick so yeah i can attest to that so your name and lisa crispin are
[00:53:01] always like the thought leaders in the testing space so i’ll make sure to put all those resources
[00:53:06] you mentioned on the show notes right thank you again so much for this conversation i hope people
[00:53:11] learn a lot of things about quality aspects and the dimensions
[00:53:14] and people start to assess where they’re at so that you can improve from where you’re at to
[00:53:18] something that you can aspire to become in the future so thanks again for your time thank you
[00:53:23] very much for having us yes yes
[00:53:26] you
[00:53:44] you
[00:53:46] you
[00:53:48] you
[00:53:50] you
[00:53:52] you
[00:53:54] you
[00:53:56] you
[00:53:58] you
[00:54:00] you
[00:54:02] you
[00:54:04] you
[00:54:06] you
[00:54:08] you
[00:54:10] you
[00:54:12] you