Dev Interrupted

Helping Government Software Teams Move Faster | Carnegie Mellon's Robin Yeman

• Season 4 • Episode 31

Share episode

Both the aerospace and defense sectors are renowned for long project timelines rife with silos and hurdles that get in the way of productivity. With over 20 years of experience at Lockheed Martin and elsewhere, Robin Yeman literally wrote the book Industrial DevOps on how to implement DevOps principles at traditional behemoths to build faster, safer systems.

As Space Domain Lead at Carnegie Mellon's Software Engineering Institute, Robin’s pioneering work reveals how applying DevOps principles can significantly improve speed, quality, and collaboration at traditional enterprises. She emphasizes the importance of cross-functional teams, modular architectures, and a growth mindset in driving innovation and overcoming the challenges of digital transformation within the aerospace and defense sectors.

Tune in to gain practical insights about the application of DevOps in large-scale systems, the role of organizational design in fostering communication, and how these principles have helped government software teams.


Episode Highlights:

- 01:12 Robin’s book Industrial DevOps
- 04:00 How did Robin’s work at Lockheed Martin lead to Carnegie Mellon?
- 05:46 How should you get started thinking about industrial DevOps?
- 08:01 How Robin’s research came together across varied experiences
- 10:25 What patterns can you adapt to be more successful?
- 16:54 Quantitative vs. qualitative data when making long term plans
- 20:27 Shifting left in Industrial DevOps

Show Notes: 


Support the show:

Offers:

Robin Yeman:

The Department of Defense and the intelligence communities really started looking at the environment and saying, Hey, we, we have to go faster, right? Currently today, if we were to launch a satellite, it takes 90 months. That's just too long, right? And we've got new entrants into the market, like SpaceX or Relativity, and they can do it so much faster. why is that? How is that? my goals and objectives are really to enable, um, government organizations to become digital so that they can do Agile. They can do DevOps. They can do model based systems engineering. They can build digital twins. They can bring in artificial intelligence and machine learning. I usually refer to this as all of the tools in your digital engineering value stream.

How can you unlock the full potential of generating AI for your engineering team? Getting started with your gen AI transformation initiative isn't enough unless you understand the impact you can have. With insights from industry leading CTOs and VPs of engineering, linear B is generic impact report highlights how you can measure Denny I's impact on your software development life cycle. Discover which use cases are delivering the most value and learn how you can track adoption, evaluate benefits and mitigate risks. Don't miss your chance to stay ahead of the curve. Head to the show notes to download the gen AI impact report today, and start measuring the impact of AI on your develop processes.

Conor Bronsdon:

Hey everyone, welcome back to Dev Interrupted. I'm your co host Conor Bronsdon and today I'm delighted to be joined by Robyn Yeeman, Space Domain Lead at Carnegie Mellon's Software Engineering Institute. Robyn, great to have you with us. I hear you're also a former TEDx speaker.

Robin Yeman:

Yes, it was actually a fantastic experience. Um, I did a talk on human collisions, so it was right during that time during COVID. where we're like, yeah, everybody's working from home. Um, but what things weren't happening, right? We hadn't actually gotten good at this whole, working, virtually really engaging with one another. And so really talked about making sure that we can still do that, right? Because some of the best ideas happened when I ran into you at the lunchroom, not necessarily when I was like banging away at my keyboard.

Conor Bronsdon:

We have to be really intentional about the social design, the social circuitry of these hybrid or remote organizations because you can create that kind of extemporaneous collaboration, but if you're not intentional about it, you fall into these negative patterns as you bring up. And this, I'm sure, spans from your expertise over the years. You've spent Over 28 years in software engineering with a focus on digital engineering, DevSecOps, Agile, and you've been a leader, you've built large, complex solutions across multiple domains. In fact, you have a pioneering research that you've done in collaboration with ITRevolution and a number of other industry experts over the last five years, and you're releasing a book now is my understanding of, but with a set of proven success patterns. And it's under the title Industrial DevOps. Can you tell us a little bit about the book?

Robin Yeman:

Yeah, absolutely. So we've seen huge success over the last, let's say, two decades with, with Agile, right? Things have, you know, people have been able to go faster. They've reduced lead time, improved communication. DevOps Expanded that so now we've got the, the tooling and the automation to really, um, deliver capabilities at the speed of relevance. I wanted to be able to do that for safety critical cyber physical systems, right? So during my time in engineering, I've spent a long time building systems like submarines or satellites or aircraft, things of that nature. and they also need to be able to. deliver faster, have high morale, build quality in. So what happens when we take those practices, that are best for, let's say, software with Agile and DevOps, and we apply them in the arena of cyber physical systems. And what we've seen is actually even bigger results than, than maybe just the software alone.

Conor Bronsdon:

And your book's coming out October 12th, so it should be live by the time this episode is. Where's the best place for folks to find your book?

Robin Yeman:

Amazon. The best place, go, go to Amazon. It's on a pre order right now. Or IT Revolution, you can also get it right from Gene's website.

Conor Bronsdon:

Perfect, and as you mentioned, this book has been an offshoot of your work over the last 25 years at Lockheed Martin, you know, designing everything from secure submarines to satellites, and you also have a co author who has experience in Northrop Grumman and beyond, correct?

Robin Yeman:

Absolutely. Dr. Suzette Johnson, is a fellow at Northrop Grumman and, um, actually we started collaborating together almost a decade ago. Uh, and I was working at Lockheed, she was at Northrop Grumman, and a lot of people said, hey, how, you know, how unlikely collaborators, right? Because inherently Competitors. We're competitors. Um, but we really refer to it as competimates because in these large systems, say F 22 or F 35, that's not a Lockheed alone system. That is a Lockheed, Northrop Grumman, Raytheon, right? All of these contractors are in there because these systems are competitive. Huge.

Conor Bronsdon:

So this is really fascinating. I'm curious how it led to your work today at Carnegie Mellon.

Robin Yeman:

So I spent a long time, became a Senior Technical Fellow at Lockheed and, and, and I really liked it. However, um, I wasn't done yet. And a lot of times you get to that senior technical fellow role. It's, it's kind of the, it's kind of the last role or it has been, you know, for, for many technical leaders, but I, I wasn't done yet. in 2018, some really interesting things happened. So. The Department of Defense and the intelligence communities really started looking at the environment and saying, Hey, we, we have to go faster, right? Currently today, if we were to launch a satellite, it takes 90 months. That's just too long, right? And we've got new entrants into the market, like SpaceX or Relativity, and they can do it so much faster. So, so why is that? How is that? And, and that was a perfect opportunity to come work at Carnegie Mellon. Why? Because it's an FFRDC. Um, and for those of you who don't know what an FFRDC is, it's, it's really an organization that works with the government, right? They, they support the government needs. And my goals and objectives there are really to enable, um, government organizations to become digital so that they can do Agile. They can do DevOps. They can do model based systems engineering. They can build digital twins. They can bring in artificial intelligence and machine learning. They can make sure that they bring in cyber. And so I usually refer to this as all of the tools in your digital engineering value stream.

Conor Bronsdon:

And this is exactly what you're doing with the book as well, right? As you're expanding on this and going in depth. I know you're giving a talk later today around this and the approach to industrial DevOps. Where should folks get started as they start to approach these concepts?

Robin Yeman:

There's a couple things. One, have a growth mindset. We always say that, but what does that mean? if you were to look, let's say, Five years ago, even 10 years ago, we may not have had some of the tools to do this, but now we're in a place where we can take physical systems and put them into cyberspace, right? Which means that I can build them at the same speed and get the feedback loops that I could in software. So the first thing you want to do is, is recognize what didn't exist before these new tools in your toolbox. Ensure that you understand it, the tooling that is required, meaning I have to have test labs much earlier, right? If I wait for tests to the end of the system, it's, it's way too long and it's actually a lot of rework. So, um, we need to invest earlier. Look at your org structure. Inherently, when we have organizations, um, especially from like industrial age, They're all functional based, right? And you know, you have a functional organizational structure. If you've got program managers, software engineers, systems engineers, test engineers, um, here's the interesting thing. They, they don't speak the same language and communication. Follows the org structure inherently, right? So if I'm a program manager, I know maybe something about lean, lean startup. If I'm a systems engineer, probably have heard of things like system thinking and design thinking. If I am a software engineer, I have probably been engaged in the agile community or leveraged DevOps. Um, if you talk to folks within the test community, they're going to talk about things like ShiftLeft.

Conor Bronsdon:

Right.

Robin Yeman:

Awesome. And Operations is big into what I would say, um, ITIL or IT Infrastructure Library. Now the interesting thing about all of these approaches is, is they all are designed to optimize the speed of delivery while maintaining quality through your systems. But we've got org structures where we're talking past each other, right? You're telling me about Lean Startup, I'm trying to tell you about Agile. We aren't talking the same language, meaning we can't actually deliver around that value stream.

Conor Bronsdon:

So how does the research for your book come together into this concept? I mean, there's, there's clear variety here.

Robin Yeman:

there's a couple things. So my experience, You know, at Lockheed was building really large systems. So I can tell you with pretty good accuracy how to build a satellite, right? I can tell you, you know, you've got command and control, you've got telemetry, you've got mission management, et cetera. one of the things that I got to see, um, with these large systems is actually this, this lack of communication, right? So, um, interesting, I could see, you know, Program managers, while they were talking about this thing, software was talking about this, and we weren't actually collaborating. You'd be amazed at how many times there would be a gap in the system. Obviously, it got fixed during, you know, let's say rework cycles, or overlap, meaning multiple teams, right, addressed it from a different approach. Most. People, many people, never actually see across these large systems. You know, Orion is another good example. It's a space vehicle. There are hundreds of teams. It's a very large system. And so a day in the life of most engineers is going to be My piece, right? But, uh, the day in the life of a tech fellow is I'm going to see how all of the pieces come together. And that experience allowed me to see how the problems with the organizational structure, as well as our architecture, was leading to us, you know, not delivering systems as fast or as best as we could.

Conor Bronsdon:

Do you feel like the industry, particularly in the defense side of things, the government side of things, is still struggling to respond to? kind of address these challenges? It sounds like you do.

Robin Yeman:

Oh, yeah. I mean, this is a huge challenge. Now, I would say it's a Department of Defense thing, except that I've had the opportunity to do some collaboration with some automotive companies and some healthcare companies and things like that. And actually, what I'm seeing is anybody who's building cyber physical systems in highly regulated environments, whatever that is, are experiencing very similar problems, right? Making that leap from, you know, industrial and manufacturing to this digital age has resulted in a whole new way of working and, and they haven't adjusted how we communicate, how we organize, and the tool set to be able to do that.

Conor Bronsdon:

So what are some of the proven success patterns that you've discovered where companies or organizations can adapt and be successful?

Robin Yeman:

first one, and this is pretty well known, really look at your work structure. Right, cross functional teams. Now, I usually would say that that's kind of a grid, meaning I've got teams that are focused on delivering the outcomes, and I've got folks that are concerned about their specialization, right? So, I really want my software engineers to have the latest and greatest understanding of test driven development, things like that.

Conor Bronsdon:

So is that what you, what you're referencing here is like a platform team, a developer productivity, DevEx team, that kind of thing?

Robin Yeman:

Absolutely. Okay. And so the benefits of that are huge. Now, you could say that you're creating a different silo, right? Because we're saying, hey, we're creating a silo around the stream of value. Um, but it's a trade off. You're never going to have the right and perfect organization. You have to understand what your goals are. And in a conversation with Adrian Kocroft from Amazon, he said they make a very intentional approach to organize around their value streams, to organize around. Even though they know in some cases they probably run into things like redundancy or repeats because their number one focus is their customer, right? So that's really what many organizations number one focus today is too.

Conor Bronsdon:

Or often should be.

Robin Yeman:

Or often should be, or you're probably not going to be in business a super long time.

Conor Bronsdon:

Okay, so this reframing of organizational structures, what would be your advice to an engineering leader that's looking at their organizational structure and maybe building a cyber physical system as you as you bring up about how to reframe and refactor it so that it is more effective?

Robin Yeman:

the first thing is don't do it in a silo, right? So when I look at the org structure, I could just say, Hey, all right, everybody go to cross functional teams.

Conor Bronsdon:

Right, but it becomes a buzzword, right? Well, and

Robin Yeman:

the system's just going to break down, right? I have dependencies. So I have to do two things. I've got to look at my org structure in lockstep with my system architecture.

Conor Bronsdon:

So a contextual piece.

Robin Yeman:

Yes. And so when I look at my architecture and my organizational structure, I'm going to do a step You know, we would call it a reverse Conway's Law maneuver. A step where I refactor the architecture to remove some of those monoliths, maybe create some more modularity, and then change that particular piece of the organization. So it's got to be done incrementally. If I was to just go into, you know, um, Any large company, probably Boeing, and just say, Hey, everybody, cross functional teams, they're not going to get planes out anytime soon. Because currently, I guarantee they have a lot of legacy architecture. And so there's a reason why we're organized the way we are. So you've got to do that in lockstep. Look at those two pieces.

Conor Bronsdon:

Interesting, okay. So this is a, this is a fascinating question of organizational design, and I know, Gene Kim and, um, Steve Spear, his co author on their, their new book, uh, Wiring the Winning Organization, have been using this phrase, uh, social circuitry to kind of talk about rewiring. Is this a concept that you see as really crucial here when you're creating these, like, new alchemy of teams?

Robin Yeman:

yeah, absolutely. I had the opportunity, uh, two days ago to go to their, um, workshop. Right? Rewarding the organization. And as much as I think that I already knew, I learned some stuff and I was like, okay, so this, you know, approach of simplicity, slowification, amplification are things that I actually need. So while I think I kind of get to some of those items, They articulated it brilliantly. so actually I was on a conversation with a customer this morning and I said, Hey, I want to refactor this workshop to bring in some of these concepts that I learned, from, Gene and Dr. Spear's book to basically make sure that we're doing those things, right? Because these systems need exactly that. Simplification? Well, we need modularity, linearity, all the things that they bring up. And that's really hard. Slowification, we really need to have more time to, to practice and understand. a lot of these big systems, we never have any time to slow down. Um, we have a tight schedule, we got to get it done. But the funniest thing happens, we don't have any time to do it the right way the first time, but we always got time to do it the right way, the second time, right? Because we do rework. and so I think that some of those concepts, kind of galvanize the things that we have in, in our book. So I'm anxious to be able to take some of that and, and actually bring it together.

Conor Bronsdon:

What are the other architectural approaches that are in your industrial DevOps book?

Robin Yeman:

other approaches are going to be planning. so if you talk to a Pure, I don't want to say purist, but maybe pure agilist, they're going to say, Hey, we're going to plan every two weeks. We're going to plan two weeks at a time, right? So if I'm building something like Fleet Ballistic Missile, which is a 50 year program, it's not going to work for me, right? It's, it's, it's just not, and the system won't come together. we refer to what we call multiple horizons of planning, meaning I, I need that big long plan. I might need a 10 year plan. I might need a five year plan. and typically I do. And then I take that and I break it down into an annual plan, which then I break down into a quarterly plan and a sprint plan. Now the coolest thing happens when you do that. I get empirical data, right? So I get empirical data on how fast I'm able to do things. I get empirical data on, um, the rework metrics, all kinds of things. And I can take that data and I can use it to inform that next horizon, right? So for easy math, if you plan to get. 10 things done today and you got six done and you did it tomorrow for me. I think, right. I have those days. and you did that tomorrow. Well in one week I can pretty much say that it's likely you're going to be 60 percent complete. Now, if you keep planning to do 10 things a day, you're basically providing inaccurate data, right? People that are waiting on you.

Conor Bronsdon:

You're eliminating predictability.

Robin Yeman:

Exactly. So I can't predict the future, but I can constantly use that empirical data to inform my plan. And so I can do that with the annual plan, the five year plan, etc.

Conor Bronsdon:

So when you're looking at these long term plans that you're then breaking down into like bite sized chunks, so to speak, are you largely using quantitative data or are qualitative forms?

Robin Yeman:

Both. But quantitative. I see the, the gap primarily around the quantitative. Um, I have seen many a projects where, um, what we call the integrated master scheduler, so the person that's got the, the keys to the schedule, is completely divorced from, let's say, the teams who have the work in their, their sprint schedules. They don't match, they don't even slightly match. Um, and the, the problem is a traditional, let's say master scheduler, um. isn't in the Agile tools. They've never, they've never done that. And a traditional Agilist or Scrum Master, they're in the tools, but they don't actually recognize the gold in the data they have. And so that

Conor Bronsdon:

solidification effect you talked about that we need to address. And that's a org design issue in part.

Robin Yeman:

Absolutely. So bringing them together to have these multiple planning horizons, and then understanding the importance of that data. Um, and using it to further inform has been huge.

Conor Bronsdon:

And it sounds like you think this particularly applies when you're doing industrial scale work. Because to, I guess, to look at a counter example, like a startup, for example, can't plan on a five year horizon. They can maybe think about it a bit, but they have to be much more iterative early on. Because Usually pre product market fed or still trying to figure out the approach.

Robin Yeman:

Right. So the thing you got to take away from there is context matters.

Conor Bronsdon:

Definitely.

Robin Yeman:

Absolutely. So there are cases if I'm building an update and, you know, for, for an application, even, even something within the government, a small application every couple of days, that's, that's perfect. I can get fast feedback. Yeah. If I'm trying to build an aircraft, I have to order hardware. I have to make sure that I've got the supply chain coming in at the right order so that I can do the assembly line. There's, there's other factors. Many

Conor Bronsdon:

dependencies.

Robin Yeman:

Exactly. So context matters. It depends on what you're building.

Conor Bronsdon:

Yeah. And while software obviously has plenty of dependencies, especially if you're bringing open source projects, all these things that Uh, Software Security Supply Chain is much easier to pull from the cloud to move much faster on. So this is a good explanation, I appreciate it. What other patterns are you seeing in organizations? So the first two have been fascinating. What else was there?

Robin Yeman:

Uh, the other patterns that we're seeing is, is technology has exploded in many areas, right? So materials. Um, now I can do additive manufacturing or what we call 3D printing to get fast feedback. digital twins. A digital twin is technically a cyber system connected to a physical system with sensors and data.

Conor Bronsdon:

And this is a full simulation in a cyber environment, similar to, I'll use an example, Amazon does this with their warehouses, for example.

Robin Yeman:

Yes! Absolutely. And so does, uh, you know, Tesla. Each and every Tesla coming off the line has a digital twin. got a chance to talk to Joe Justice, who worked there for a while, and he said not only that, they take those digital twins, they create a digital ecosystem, multiple digital twins, and then they use artificial intelligence to find patterns. Why? Because in these large safety critical systems, like a car, There's something called emergent behavior. And when I bring together a lot of these different digital twins and let it run through a lot of different scenarios. I find things that I, as a human, would take hundreds of years to find.

Conor Bronsdon:

Interesting. This is a fascinating concept. Are there other key concepts in the book you want to share with us?

Robin Yeman:

So some other things are going to be We say shift left. Um, we have provided some real practical examples on how to shift left. Great. And how to begin with test. Um, in my world, at one point in time, I had the opportunity, well, many opportunities to, to respond to what we call RFPs, or Requests for Proposals. And, these proposals, they have, they have these requirements. And I read through one, and it looked perfectly reasonable. Like, all the requirements. Um, and so, just for grins and giggles, I thought I would go through each requirement and try to validate how I would test it. How would I, how would I prove that this happened? Um, on the front end, before I even started building anything, and the interesting thing was 32 percent of the requirements weren't testable at all. They seemed reasonable, but when you said the system shall be performant, or it shall be scalable, to what? Right? And those are the things, which allowed us to really reduce that rework, to build it right the first time. There's other things like we, we talked about is that architecting for speed, modularity, standardized interfaces, huge, both in software and hardware. Um, and really bringing together kind of the, the different cultures because we're continuously learning. My youngest son just graduated, uh, computer science degree from UCF.

Conor Bronsdon:

Congratulations.

Robin Yeman:

Yeah. And so, you know, he's, he's out looking, you know, getting his first job, things like that. And he's like, Oh, it's like, you have to keep on learning. And I'm like, yep, that's called the suck it up buttercup concept there, dude, that you pick technology. It changes every day. It's the fun part. But like, yeah.

Conor Bronsdon:

Fantastic. Well, Robin, I, I really enjoyed this conversation. Thank you so much for sharing all these insights from your book, industrial DevOps. I'm looking forward to checking it out. Awesome. And, uh, thanks for coming on the podcast. It's been great. you're listening to us here, check us out on YouTube too. You can see absolute Robin and I in the midst of this giant plastic dome and

Robin Yeman:

Very cool.

Conor Bronsdon:

Fast. Thank you. Yes. This, yes. We'll, we'll, we'll clip that. That's very cool. That's perfect. Um, and thanks again, Robin. All right, cool. Thanks.

People on this episode

Head Shot

Dan Lines

Host
Head Shot

Conor Bronsdon

Co-host