Failure Files 2.0: Macondo

Chris:
Whoo, we’re back story time with Rhett. I’m your co -host Chrisopher De Leon and I’m super excited back by popular demand We have Failure files 2 .0. I have to admit Rhett. I know I know I’ve been Reluctant to jump right back in because of all the successes. We had a Failure Files1 .0, but we’re here.
Rhett:
It was nearly two years ago.
Chris:
 But we’re here the floor is yours.
Rhett:
So today Chris we are going to do something that all of our audience, definitely old enough to remember, but it was in a different industry, a sister industry, but close. I just give you the stakes, 210 million gallons of crude oil into the Gulf of Mexico, an oil slick that covered 60,000 square miles of ocean. It’s larger than most U .S. states. The state of Georgia is actually 59 ,000 square miles and in a well located 48 miles off the coast of Louisiana. So what we’re talking about today is the BP -Macando oil spill from April of 2010. Most of us old enough to remember it, but I question whether or not we’ve understood the lessons and their very direct application to pipeline integrity. It’s going to be a fun episode. Stick around. Welcome Back to the failure files.
-Intro-
Rhett:
Chris, do you remember where you were on April 20^th, 2010. That’s a question for the audience too, but let’s start with you.
Chris:
Yeah, I was at 54 44 West Timer working as an integrity engineer for energy transfer.
Rhett:
So most people probably don’t associate, you know.
Chris:
Actually depending on the time of this I might have been thinking about lunch. It was there wasn’t this around like 10 o ‘clock ish? Around there, it was mid-morning?
Rhett:
Maybe by the time you heard about it, definitely probably so – The reason I asked that question to Chris, the significance of that date, and I’m starting to think there are a lot of dates in April, Waco, Oklahoma City Wyoming. I’m like, what is going on in April? April 20th, 2010 is when the Deepwater Horizon experienced a blowout. For those of you who are listening to this episode, Deepwater Horizon is the name you probably remember it. BP Oil Spill is the other name you probably remember it as, I actually knew it as Macondo because I knew the well name, right? So you’re going to hear me throughout this episode talk a lot about Macondo. Macondo was actually the name of the well that they were drilling and fewer people actually know it as that name. But we’re talking about the Deepwater Horizon incident. I remember it distinctly because I was working at stress engineering at the time and I was still pretty heavily involved in the offshore industry. Stress got drug in like every single company and Houston got drug into that, right? And then you remember it affecting, I don’t know, at that point in time, we had vacations planned to Gulf Shores. There were questions about whether or not you could go to Gulf Shores, the oil started coming up on the shore, and that’s what most of us remember from this event, miles, the images, the duration that it went on, right? So this story begins on August 20th, whenever the rig is on fire, it burns for two days before it eventually sinks to the seafloor, they had, you know, people were jumping off of the rig. We didn’t get a lot of those images, but I mean, people appreciate this rigs like 125 feet above the, the sea, the seafloor, not seafloor, I’m sorry, mean sea level, right? So it’s not just jumping off of a big high. This is a lot.
Chris:
If you want to, if you want to reflect, like we were on the, on that day in the building, I was on the ninth floor. That’s, and I distinctly remember how high that was. And so you’re going to tell me I’m going to jump out of there into the ocean.
Rhett:
Yep. While the rig is on fire, I mean, that was a level, but fortunately, and I say fortunately of the 126 people on board, 115 were rescued, which does, I think, speak to, uh, the safety protocols that they have in place, but we did lose 11. Um, again, the vessel eventually sank. And most of us remember the aftermath because it didn’t end the vessel sank, but then the saga continued for 87 more days. While they actually are, I remember they had ROV footage. If you were, if you remember that time, you could actually log on to the internet and just watch the ROV.
Chris:
You could watch it spewing.
Rhett:
You could see it spewing, you were getting all these experts. Every news channel was filled with experts talking about stories about how long the recovery would be. Would we ever be able to do this? What was going to happen? It was a, um, it was an amazing time. I will say.
Chris:
Maybe amazing is not the right word.  I don’t think amazing is a right word. It’s still frustrating to be quite a bit like I’m getting a little bit frustrated again.
Rhett:
And look, we were like many of the aspects still  remain unsolved to this day.
Chris:
 I mean, like one of my favorite parts about failure files is story time with Rhett. And I can already project how I’m gonna feel like I know that as you start getting some of the consequences of what thishappened, like—
Rhett:
So it’s great, I’m glad I got you worked up. –
Yeah, I’m probably gonna be a little bit fussing.
Rhett:
But I hope that you see yourself in the decision. So let’s get to what happened here. So, and I’m gonna break this down. If we have anybody that’s really, really deep into wells, maybe if we have some petroleum engineers out there, you’re probably going to take exception to some of what I say.
Chris:
 Or people like you that ran away from upstream to come into midstream.
Rhett:
Yes. Uh, but I didn’t work in the in the well side, actually worked about everything above the well, right? But it’s still, I understood the technology, understood what was going on. So I’m going to try and explain it a level, Chris, so that you understand that what’s happening in the decision, cause it’s the decisions that, that matter in the story, right? So when you drill a well, obviously, it’s like drilling a hole in your ground or just a straight hole, right? And one of the things that you have to do is you have to case that hole to keep it from falling, to keep the material from falling on itself to stabilize the hole. So you can go down like 30 ,000 feet, you actually have to stabilize the hole, right? And so that’s number one. Then inside of that, we actually put another production casing. So you imagine you have an outer casing that keeps, you know, the well intact, if you will, then you have an inner casing that is actually where your production fluids come up at the end of the day. Yeah. One of the things that you have to do, right? So, and again, I’m simplifying the process greatly is eventually at the bottom, you need the production fluids to come through and they actually only need to come through that production casing and then also imagine two tubes, right?
Chris:
Yeah, I’m hoping not through the space between the annulus between the earth and the outer casing.
Rhett:
That’s one very important annulus. And the second important annulus is between that, what’s called the well casing and the production casing. The annulus between the well casing and the production casing isn’t the critical part here. It’s actually between that one on the outer that you’re talking about. So-
Chris:
Earth and what we’re gonna call the overall well.
Rhett:
Yes, in order, we don’t want it to come up that way. So what we actually do is they pump cement down through the bottom. And so we actually, we pump cement all the way through the bottom of the production, it comes up, and then it goes and it sills that annulus around the edge.
Chris:
Okay, so again, visual guys, we wanna see what you’re saying. Once you have the well, you’ve tapped into the formation that you’re trying to get oil and gas out of, you then pump cement into that same formation and it’s going to rise and kind of create a sealing element from the inside to keep oil and gas from going anywhere through the straw in the production,
Rhett:
Other than the production.
Chris:
The straw that we’ve now put in the earth. So, this is so good. So, hold on. So, it’s like you take a, you take a Coke bottle. Yeah, and you you drill a little hole on the screw on the plastic cap You put a straw in it after quickly after you’ve shaken up the coke bottle And you’re hoping that all the coke comes up through the straw and not through the the hole in the in the cap Because it’s in pressure, you’re shaking up the coke and you’re trying to contain it.
Rhett:
You’re bringing up another thing which I need to discuss which is how wells in general work. ‘Cause when we drill them, we actually drill into these high pressure formations and the pressure in the formation enough is high enough to push the fluid all the way back up through the pipe.
Chris:
That’s what I’m thinking.
Rhett:
We counteract that with drilling mud, right? So drilling mud is something that is heavier than water. And what we do is we fill the pipe with when we’re drilling that whole production casing or that center casing is filled with drilling mud, which is heavy enough to put a pressure at the bottom that prevents the fluid from coming back up, right? But at the end of the day, we can’t produce with drilling mud in the pipe. So we have to seal it up and then we eventually will remove that drilling mud. And when we remove that drilling mud, we can get a controlled environment where the pressure and the formation is high enough in order to push the fluids back up there, which is what we want, right? But it’s really critical because when you make the decision to remove that drilling mud from the pipe, you’ve now removed the protection from the bottom. Right. So before you do that, you need to make sure that the wells integrity is all good. There are no gaps for the production.
Chris:
There’s no pump in the formation, right? This is, this is pressure within the formation that’s driving the fluids up.
Rhett:
So as you can imagine here, What they do is when they push the, the, the mud through the bottom, it has to come back up around the annulus and you need it to come up evenly. So a couple of things are really important here. That casing has to be centralized. If it’s not centralized, you can get an effect known as channeling where the mud wants to go up the wide side sooner than it wants to go up the narrow side. And you can get this belief like, Hey, we pumped in all the mud. We should have pumped. but in reality, you’ve got a channel for the gas to go up through one side. There are numerous other ways that this can go wrong. I’m not gonna get into all of those. Suffice it to say that at the end of the day, what we need is some type of a test to make sure that the well is okay. Before you remove those, The drilling mud before you remove that last barrier and you say hey, I’m gonna take off all the pressure I need to make sure this is okay. There’s a few ways that they can do this. This is where the decision starts to happen.
Chris:
I feel like I’m at the foot I’m to the point of where we’ve just laid the pipe and we’re about to do the post construction hydro test.
Rhett:
So what ultimately happened is the cementing job. I just described to you failed. It failed, and what that permitted production fluids to come back up through places we didn’t expect it to all the way up to the BOP eventually when it blew out. Because the cementing job wasn’t good enough.
Chris:
And this is, this is after testing or before testing. Like what you just said,
Rhett:
Oh, let’s get into that. So, they complete the well, like everything here, Chris. This story is over -budget and behind schedule.
Chris:
 No.
Rhett:
Yep.
Chris:
Hold on, hold on. I love story time with Rhett. I don’t know if we’ve painted the picture good enough for our audience. I mean, isn’t this like, aren’t they also working like a couple miles away?
Rhett:
What do you mean?
Chris:
Like, because this is a floating vessel, right? And isn’t it like, I don’t know one or two miles before they get to the surface and then they’re they’re they then have to drill like so many feet under the ground. What I’m getting to is like this isn’t like I’m in I’m west of San Antonio, where I can put my hands on deck right like this is a floating vessel. That is probably two miles or so above your surface.
Rhett:
You’re operating so above your 5000 feet of water. So you’re a you’re a mile above the sea floor and then you’re 13 ,000 feet below the sea floor. That was the depth of the well at the time.
Chris:
So, there’s a lot of calculation and engineering happening here.
Rhett:
There is, there’s a lot.
Chris:
Like you can’t just put your eyes on this and say, hey, how’s this going? This is all date acquisition systems and calculations and how much product have we used. There’s a lot of uncertainty that has been dealt with.
Rhett:
It gets deep. So let me, let me.
Chris:
What I wanna do is, I’m trying to and the big picture here, right? Which is it’s like, I’ve got a mile between depth of water.
Rhett:
Then you’ve got 13 ,000 feet below that.
Chris:
And then I’ve got 13 ,000 feet before that.
Rhett:
And at the very bottom of that, you’ve got a formation that you’ve tapped into that is high pressure that wants to push fluid back up all those pipes, right? And so before your protection against that is you’ve got this heavy weight drilling bud. So, the pressure from the vessel top down keeps that from happening like putting your finger-
Chris:
 Your fingers on a plug, Your your fingers on the water hose
Rhett:
because you’re your your weight of all that drilling mud 13 because they’re 18 ,000 feet of drilling mud Exceeds the pressure at the bottom. Yeah, so the drilling mud would want to go into the formation But not come back up and perfectly balanced and that whole drilling mud thing is complex.
Chris:
I think now we’ve kind of painted the picture of-
Rhett:
But you pull that drilling mud out It’s like pulling your finger off the things you need to make sure your other safety mechanisms are in place. There’s two ways they do this, right? So I told you, you can get this concept of channeling when you push in the mud through the outer annulus. If you’re not careful, well, so when they finished drilling the well, the first thing they did is they declared the well was a success.
Chris:
How do they do that?
Rhett:
Because they were done. So, the question came up, we can do a cement bond log test. The cement bond log is one way that we can actually test the adequacy of the cement. Problem is, it takes 18 hours to complete and cost $128 ,000.
Chris:
Ooh, 128 grand.
Rhett: Plus the time for the vessels to be on.
Chris:
In 2010.
Rhett:
So you’re behind schedule, over budget. We don’t need the cement bond log test. That’s an
extra test.
Chris:
Just pause, what is the easiest way to describe a cement bond log test?
Rhett:
It would actually, I’m gonna say it’s a method of inspecting the adequacy of the Siemens. It’s similar almost to like what we would do with inline inspection to some extent, right? You go down there and you actually, you do a test to measure that the Siemens job is adequate.
Chris:
And that’s to mitigate leaking through channeling through the outermost annulus between the earth and the well.
Rhett:
Correct.
Chris:
That sounds pretty important to me, Rhett.
Chris:
They decided they didn’t need it because they had another test.
Chris:
Are they allowed to do that?
Rhett:
They are allowed to do that.
Chris:
Is that regulatory?
Rhett:
No, there’s not a regulatory requirement on that.
Chris:
That is just best practice.
Rhett:
Second thing.
Chris:
If you don’t do that, what do you do then?
Rhett:
There’s a well integrity test, right? So what the well integrity test implies is test the integrity of the well, right? So I’m going to tell you what they do is what they do in here is they insert a production tubing through the center of the casing, right? Because I told you it’s the mud that holds back the pressure, right? And in this production tubing, they don’t have the mud. So it’s like a way to kind of test it. And what they do is they tap all the way into the bottom, they let it pressure up, And then they close the valve at the top of the, of the, um, of the, the production vessel, right? And then what they do is they bleed off the pressure. And in theory, if, if they bled off the pressure and there’s nothing leaking through the formation, this tubing that they’ve installed shouldn’t get pressurized, right? I mean, somebody in this audience is going to get like floored because I’m simple. I’m oversimplifying it.
Chris:
We get it.
Rhett:
They, They they tap in something that’s into the pressure part of the formation bleed off the pressure and they’re like if it’s sealed it shouldn’t see pressure again and what do you think happens?
Chris:
That that passes?
Rhett:
Oh, no, they see pressure So they see pressure in the tubing where they shouldn’t have seen pressure. That’s a it’s a blatant failure
Chris:
So now you go back and you do the cement test.
Rhett:
No, so they decided to repeat the test because they’re like something’s got to be wrong. So this is where it gets, this is where the challenge comes in. They repeat the test and it fails again. Nobody on board. So there is a team on board here and I want to say there are five people that are involved in this decision making process. Two of them work for BP, three of them work for TransOcean. None of them had ever seen a well integrity test fail, not once in their combined careers have they ever seen one. In fact, the well integrity test, you would think it’s to measure well integrity. You would think it’s like a stage gate. They didn’t view the test as that because the test was so routine, because It had always passed. They didn’t view it. I’m describing it to you as a measure of integrity. They didn’t view it that way. They viewed it as just something you checked the box to move on before. Like this test always passes. Why are we doing this test?
Chris:
But that’s a red flag. I mean, that’s an obvious red flag.
Rhett:
To you and me? To somebody who’s done this multiple times again and again throughout their year, throughout their careers, they just see this as, this is just check the box. This test doesn’t fail. You must have done something wrong.
Chris:
So did it again.
Rhett:
Did it again. You must be doing something wrong. So what they began to do is they began to invent theories for why this happened. And one of them was called a, oh my gosh, it was pretty funny, it was called a bladder effect, which makes absolutely no sense. They invented something. They said, Oh, that’s the bladder effect. That’s causing this to happen. So they tried to come up with a secondary test and they ran that test. And then that test gave them the results they wanted and they were like, oh, it’s good, but there were problems with that test. And we think there was a blocked line. Again, I’m leaving stuff out. Problem is they never endeavored to explain why they had the two failures. And when they did. They invented a scenario that technically made no sense, but gave them the justification they need.
Chris:
It’s beginning to sound like a little bit of like confirmation bias, right? It’s like we don’t have the intended result that we want, so let me see if I can confirm what I’m hoping is true. That’s what it almost begins to sound like here,
Rhett:
So it’s it’s confirmation bias is one way to think about it But you actually define by us in a little bit of a different way than most people do, or most people think about it. Are you doing the test to measure the Wells integrity?
Chris:
Nope. The second test that they did, they did it to-
Rhett:
Or to confirm the Wells integrity.
Chris:
Yeah, I was thinking about, again, my statement was more around it’s like they had a result they didn’t want, so they tried to rationalize why that happened by doing some secondary activity that confirms that their idea may be true.
Rhett:
So in this sense, confirmation bias as a psychological phenomenon that refers to the preference people have for information that confirms rather than disconfirms their beliefs. Right. So you have a preference for information that confirms what you want to see rather than disconfirms. And the challenge here-
 Chris:
 
An example that was then the integrity test didn’t fail because the well is bad, it failed because you don’t know what you’re doing.
Rhett:
Yes, or it failed. No, it’s the test you got failed right because of this effect. That we we concocted in our mind, right? And so it what happens is again, I want to take you back. The well was confirmed successful for the test was ever done. The test actually happened. The well is safe. They’re operating under the mode that the well is safe. Yeah, this test is just to check the box. Wait, you’re gonna stop this whole operation now. We’ve never even had this happen I’ve been doing this for 30 years, bro.
Chris:
Never seen this.
Rhett:
This doesn’t fail. You must have done so. Oh, this is the effect you have.
Chris:
This is easy. We’re only a mile above It’s only 5 ,000 feet of water depth.
Rhett:
And so exactly making this real to you think about it this way a lot of times when we’re dealing with data, whether it’s ILI or pipeline or another one that’s going to make it. You have multiple explanations for why something might have happened. And use one of those explanations is usually benign.
Chris:
Yeah.
Rhett:
You can tend to want to normalize that anomaly or the other anomalous explanations, which are problematic, right? It’s like one explanation for why the test failed is the well isn’t sealed. The consequences of the well not being sealed are dramatic and huge. The other alternative is oh, it’s this bladder effect. But that bladder effect helps me. It confirms what I want to be. So I tend to want to gravitate towards that explanation.
Chris:
So again, obviously, I know very little about wells. And but I almost want to create a parallel in our world. And one of the questions that it makes me wonder is like, you know, and don’t answer this yet. It’s, I wonder how significant the integrity test failing actually was. So for example, right? It’s like, in our normal day to day, we deal with a lot of problems. But I think you and I are generally aware of which problems are significant and which ones are ones that can be put on hold for a minute or are of lower consequence to us. And so me being ignorant to the like this whole process that is happening You know, I really want it because I’ve seen myself in this situation now, especially on ILI projects.
Rhett:
Exactly, that’s where you need to go.
Chris:
And this is what I’m getting to it’s like and I’ll try to create one ad hoc. How many times have I been on a crack inspection truly? And I have speed excursions, but it’s an inferred calculation of the tool speed because I don’t know what it’s actually doing. Cause I haven’t gotten the data yet, but I know if I operate outside the boundaries of the tools essential variables, I’m not good at data, which means I don’t get good crack data, but a lot of times we’ll overlook speed excursion cause we think we can deal with it later. I’m curious when they, when this happens, are they thinking I can deal with this later or are they thinking, oh my God, this is significant. We’re already behind schedule, we’re already over budget.
Rhett:
These are the real drivers. These are the real drivers, right? Dude, you’re doing the test to confirm the well. It’s supposed to show that it’s good. Why is it showing that it’s bad?
Chris:
We know the well’s good because we use the right amount of mud, we use the right amount of cement. We didn’t have any problems when we were drilling. None of the other KPIs are telling us that There’s a problem.
Rhett:
So I’ll give you, you know, where this brought it up for me and we’re getting ahead, but it’s okay. I want the audience to know where we’re going to go after the break. I really thought of what happened with Bellingham. Remember the guys like, “Dude, I’ve dug these tuba scope features before.”
Chris:
There’s not, that’s not going to be able to.
Rhett:
The benign explanation is there’s nothing here. This is a tool artifact. The alternative is we have something significant in the pipeline. If that example is not real for our audience in terms of how many times are you looking at situations where you have something that might legitimately be a concern? I mean, I’ll create the suspense. How many times as an analyst, as an ILI analyst, have you seen something that you either over rely on your experience or you say this has never been a problem?
Rhett:
I got one more Thing I want to bring out of this failure before the break and it’s a concept this one struck me It’s a concept of group think. So, the first one that we’re gonna come back one after the break and talk about his confirmation bias and normalization. The second one is group think so I told you there were five people involved in the decision to BP company men a Trainee and two long time drillers the long team the long-term drillers are the guys that have the experience here, right? The person responsible for the decision was the BP guy, 100%. That’s a buck stopped with him. The reality is they viewed it as a consensus decision. We’re all five of them needed to be on board. I never considered this, but I want people to think of their behavioral norms here.
Chris:
I get it.
Rhett:
We are not comfortable with a small group of people having one person dissent. We actually expect small group decisions to be unanimous. So the BP guy was uncomfortable with what was going on. He was uncomfortable with the bladder effect situation, but he had two experienced guys who created a peer pressure scenario where he was the only one out unwilling to accept the benign explanation and he caves because he can’t, they can’t, they can’t be like, hey, this decision’s four to one. They all need to get on board with the decision. We’re gonna see this play out in another failure we’re gonna tackle too. And I want you guys to think about that. When y ‘all are in a decision-making room, how come through are you with one person who’s like, I won’t make this decision? We have this tendency to expect, even you and I sometimes, in our company, Chris, when we lead it, you and I make decisions together but we usually make sure we’re getting the other person on board ’cause the decision needs to be both of us. It needs to be unanimous. But what that creates is a situation called risky shift. Not a term you’re familiar with. This is a term where groups are inclined to make more risky decisions than any one of the individuals themselves because of the group think mentality. And we all have to get to the same place.
Chris:
Yeah. You know, Rhett, the first thing that comes to mind is, um, conflict management, right? And, and the reason why I say that is it’s, I think, I think you and I are very comfortable, um, telling each other what we think
Rhett:
And disagreeing.
Chris:
And yeah, well, I was getting to that. And oftentimes we will disagree, right? But it’s kind of like, it’s kind of like the principles, if you’ve ever read the five disfunctions of a team, right? It’s kind of the idea of like, there is healthy conflict, as long as it’s debate and not arguing. Right. And I would say that healthy conflict often comes if there is an underlying trust. Right. Oftentimes when we disagree with Rhett, I trust that your heart’s in the right place and you’re trying to achieve the right goal.
Rhett:
Nine times out of 10.
Chris:
Nine. Well, I know. Well, anybody who knows you knows that. Yeah. But the intent is it’s, I think that that ability to disagree and to debate for the right reason ultimately does produce better work. But if, if decision makers are not willing to disagree and manage conflict for the right reasons, then, then that’s a problem, right? It’s not just about being an engineer, it’s also about understanding how to deal with the social sides of making decisions.
Rhett:
So when we come back after the break, hopefully you’re thinking about this, I want to pick up these decisions or pick up these discussions on confirmation bias, maybe conflict management, group think, but stay tuned. We’ll be right back.
-break-
Rhett:
So, as we continue our first episode on failure files Look, we went through the failure. This is the point where I want to flip and I want you to put yourself I want to make it applicable if you feel like we just did an episode on a topic That’s not relevant to you. I think I think you missed it. And I need you to get it because I see myself in these decisions. I mentioned over budget, over schedule, we always deal with economic and timelines on the decisions that we have to make. Those are real decisions. I think the confirmation bias I want to start with, because I think that’s more relevant to us than we realize. So that theme of the well integrity test was viewed as a test to confirm rather than to investigate, which completely flips how you view the results of that test. Yeah. I’m not expecting that test to give me negative results. It’s only there just to confirm that the well is good. It doesn’t tell me whether the well is bad. It confirms that the well is good. Those are different mindsets. My first question to the audience, we deal with ILI a lot. Do you view ILI as something, as a test to confirm the pipe’s integrity or is it to investigate the pipe’s integrity?
Chris:
And I’ll follow that question up. And if you’re a gas operator, be careful what you think. because we can now do an MEOP reconfirmation via ECA, which is a suite of appropriate ILI tools. So are you confirming or are you investigating?
Rhett:
And Chris, that’s really good, ’cause how you enter into that process matters.
Chris:
Matters.
Rhett:
If you enter into the process—
Chris:
Well, I’ve never had a hardspot failure, Rhett. So why Why do I need to run a hardspot tool?
Rhett:
I am doing all of these ILIs to confirm the pipes integrity ’cause it was grandfathered a long time ago and it’s been operating for years.
Chris:
Yeah, this is extra, I’m already doing extra. We’ve never had to do this before. I’m only putting these tools in this pipeline to confirm what I already know. So did the budget? Oh, that’s even, that’s, oh my God, this is so good. So did the MLP ECA budget, was it based on a philosophy of confirming that it’s good? IE, two digs per tool run? Or was that budget set up for investigation where it’s, you know what? Each of these ILI tools, we’re going to have to make sure we investigate that the tool performance is valid and that we’ve learned enough about the pipeline such that we feel comfortable continuing to operate this way.
Rhett:
All right, let me, so there’s a bunch of our audience that’s like, dude, I don’t do anything with MAOP reconfirmation be easy, I just turned out. So let me speak to you. When’s the last time you normalized a strange ILI signal? When’s the last time? When is the last time you were in a meeting? I’ve literally been told this. Let me tell you something Rhett when I run that tool. I’m not here to validate your tool because I’m tired of digging things and finding nothing.
Chris:
Yeah, it’s Bellingham, buddy.
Rhett:
I agree.
Chris:
That was our first episode of failure files 1.0.
Rhett:
When I saw the whole thing around confirmation bias I was like, oh my god Confirmation bias I think is more real in the subject of ILI and ILI response Then potentially we realize. Let me let me give you guys another one. Bending strain geohazards I find a major bending strain feature First response the features been in the ground forever, does that mean it’s not critical?
Chris:
Does it mean it hasn’t changed?
Rhett:
Is past performance a guarantee of future success now? And again what always drives these decisions in the audience and I know can appreciate this, if we had infinite budget and no economic constraints, we dig everything we find.
Chris:
Even then we wouldn’t, but still continue.
Rhett:
I would hope we would. You don’t think we would?
Chris:
You know, I think no.
Rhett:
Infinite budget and no, no, no economic constraints. Why aren’t you digging things?
Chris:
Yeah. Because someone will find purpose and say that doesn’t make sense. It’s human behaviors. But what I would say this is I want to bring this back around, guys, I think the intent of what we’re saying is it’s the takeaway from this is if you’re in a position as either a consultant, a service provider, or an operator, and you are associated with the pipeline safety or pipeline integrity process, be careful of the mindset in which you’re mean your task such that you’re aware of confirmation bias. It’s always been like this and I’m confirming it. Are you testing for compliance or testing to confirm what you think past performance was, or the big or underlined it in bold, or is it to investigate? Because in integrity, the integrity loop. The intent there is to understand the integrity, not to necessarily confirm it. And we’re creating a little bit of a chasm there intentionally for the purpose of this discussion of saying, we need to make sure we’re performing an investigation. In this case, when they had their first test failure of the well, it should have turned into an investigation. And some would say, ah, but it did. And that’s fine. It’s less about Macondo and more about our day to day, right? And I think that’s what we’re trying to drive home.
Rhett:
And for me, it is very real ’cause we’ve been in that situation where either we’re asked to review a signal. And a lot of times when you’re asked to give a second opinion, it’s because they don’t like the first opinion they got, which usually tells them that they need to do something.
Chris:
Can I give you a real one?
Rhett:
Yeah, you can give me a real one.
Chris:
Hey, so we got a set of Sharpie results. Two of them are very favorable, one of them weren’t. Rhett:
Oh, yeah.
Chris:
I just take the average, right?
Rhett:
No Max.
(laughing)
Chris:
I mean, guys, come on. How many times does this happen more recently, right?
Rhett:
Do you know how many features I’ll have to dig if I do that?
Chris:
So I’m calling you because you want me to confirm that yes, you could be less conservative? You’re calling me ’cause you want me to confirm your bias?
Rhett:
And again, those integrity decisions are real, right? Like we see it play out on EMAT, you know? I’m not gonna, there’s no reason to dig type B calls on EMAT. I’ve dug a bunch of them, and I’m sick of finding blank calls. I’m sick of digging dry holes.
Chris:
Why do you think there’s a B? Lower confidence, you’re not supposed to dig them. That’s the whole point of this.
Rhett:
I mean, yeah.
Chris:
There’s a reason why when you get these calls, I mean, it started all the way back when you had TFI’s, there were A’s and B’s, right? It’s that point, it’s A, focus on the A’s, understand what you’re finding in the ditch. And then dependent on that, then you can begin to consider the B’s. And anybody that’s been doing ILS for at least 20 years understands that.
Rhett:
Yes, you don’t dig B’s. In fact, you don’t run crack tools ’cause you don’t want to get results you don’t want to have to deal with. So it’s easier if you don’t run them at all ’cause then you don’t even have to deal with the bees because you don’t deal with anything.
Chris:
Well, I would say, yeah, I mean, the whole idea of the bees is those are the monitored ones though, Rhett, like the whole intent is that’s why we have immediate, scheduled, and monetized.
Rhett:
I think the audience right now was trying to figure out what just happened and they’re like, what do we…
Chris:
So we hope you realize we just tried to do the whole group think process on you and type B’s because that is totally wrong.
Rhett:
Which brings us to the next point, I think that is important to me, which is that subject of groupthink, right? So again, there are two major applications that I really hope our audience took. The first one I want you to think about is the confirmation bias. The second is groupthink, because most integrity decisions are not made in a vacuum, right? It’s usually not a single person that just makes a decision all on his own. Anytime there’s usually some type of challenge or question about the data, it usually ends up with a few people from the operator.
Chris:
Well, yeah, you wanna call, so the operations crew says, “Man, are you really gonna make me dig this on Farmer Joe’s land again? We were just there two weeks ago.”
Rhett:
Right, and so, I wanna give you another way to think about groupthink is jumping on the bandwagon, right, that’s another way. Point is, is people, we all have some degree of wanting to conform and fall in line, and it’s very difficult to stand out when you have four people staring at you and you’re the only one that thinks the action that’s being taken is incorrect. Very few people have the stomach, the ability, the willingness to be that uncomfortable in public.
Chris:
Well, it’s exhausting, too.
Rhett:
It is. It is. And it puts you get that knot in your stomach. Like, am I about to be the one that’s about to speak up and say that I don’t actually agree with the direction that we’re taking? I’ve had to do that before, and it is like you can feel you feel that you feel the sweat on your brow, your palms get sweaty, you get that sick feeling in your stomach.
Chris:
No, see, you’re making it about how you feel. But there’s another component to this. I won’t get a promotion. I will not get a promotion if I’m the difficult guy. I will not get a promotion if I’m the one that’s always trying to investigate, or if I keep asking my boss questions, I will not get a promotion, Rhett. Why would I speak up. Fall in line.
Rhett:
You know, what you’re saying, you’re bringing up an economic aspect to it that I didn’t even intend to go into, but that is also very real. And that speaks to the culture of the organization and the culture of the managers, hopefully that we’re talking to. But I want them to appreciate, and I have to give credit to Ms. Producer for bringing this up. There is a famous study that was done. It’s called Solomon Ash’s Conformity Study. And in this study, what they did is they brought in people. I don’t know if you ever participated in psychology experiments at U of H. I did at A &M and when I took psychology. They bring you in and Chris, they do something like, in this particular case, they asked you to match the length of a line to several comparison lines. So like choose A, B, C, or D, here’s the length of line. It’s a very, very easy task to do. What they did is they had one person who was the subject test study and then they had three other people who were paid actors. The actors all chose the wrong line even though it was obviously wrong.
Chris:
The same one or different ones?
Rhett:
I don’t know that but assuming they chose the same one.
Chris:
So they all agreed the same one. So it’s clearly a three against one.
Rhett:
Yes and the one guy’s looking 75 % of the time the study participant would go along with the other three and choose the wrong line, even though they knew it was wrong. ‘Cause it speaks to the power. So that’s 75%. So I said most people don’t have the stomach. This study is telling a 75 % of the people don’t have the stomach. And I think us as integrity managers, or I like me out there, need to recognize that. When you have four people on a call discussing a bizarre ILI signal, or discussing the results of a pressure test, And you’re trying to figure out, why is this happening? You more than likely out of that seven, four of those people, three of them are going to be unwilling to be the one to just stand up and say,
Chris:
I think this is wrong.
Rhett:
That’s me. Yeah. I think that’s wrong. Right. Think about the control room episodes that we did. How does that play out? Remember, because we had that one person who kind of assumed the role of expert in the, the inbridge failure, right? If you have one person who assumes the role of expert provides an explanation that everybody can get behind the one person who’s sitting there and says this is horse crap might not be willing to stand up and say this is horse crap
Chris:
And to be clear our whole little episode not episode that whole little skit we just did with the type B’s was exactly what we were trying to do here right if you and I are viewed as experts for a second you might have thought yeah I agree and then we flipped it on you were like no that’s wrong like We have numerous times found blatant cracks in type B calls. And, and we just, we were trying to, to model that there. And, and I also want to, want to say this to the audience guys, um, rely on your fundamentals. Right. That’s why you’re hired, right? If you go back to your interview, when you were interviewed, they, we, we asked you things like, what, why should I hire you? Right. You’re like, cause I’m a problem solver, right? I investigate, I’m diligent. I want to be part of a team where my voice matters. Don’t forget about that stuff. Right. Ask questions. If it doesn’t make sense, ask more questions. If it doesn’t make sense, ask more questions. Right. If the math is not adding up, keep asking questions. Right. And the second thing is it’s this also kind of makes you think a little about, and not that we need to get into it, the idea of personality tests, right? And formal training, right? If this study showing that on average, 75 % of people won’t have the personality to ask the right questions or to create conflict, then you almost need to start looking at your team construct and not that you want one type versus another, but maybe that’s the value of having a little bit of that involved in your team making.
Rhett:
So you’re bringing up some good lessons. I’m gonna leave two others that actually were directly out of the book I read related to this and they talk about decision makers. So what you need to realize is again, I wanna go back in the BP incident, there were five people, but the BP company man was the decision maker, but he was not able to be effective in his decision-making process because of the peer pressure scenario created by the group. I think you can see obvious parallels to integrity engineers, maybe integrity managers. And we need to recognize look, we all might want to think we’re that 25%, but studies so at least three out of four of us are not. So don’t say, oh, I’m the 25%. I’m the person that’s willing to do it. Or walk into the office tomorrow and all of a sudden try to become the 25%. You might get walked out the door. Just be careful. The challenge here is it says for the decision maker to be effective, they need some isolation from the group, right? That in some sense, before they issue their decision, they need to be able to withdraw so that they can take all the information in without somebody telling them and make a decision. I think that there’s some merit to that. There’s some merit to think about, Hey, as a manager and integrity, if you have to make a truly impactful decision, maybe in an HCA related to an excavation or related to taking a line down, that decision maker might need some time alone to think about all the data he’s been presented with and make a decision free from influence. The second thing, and I think it’s neat, and companies do implement this, I’ve seen it, is the role of a devil’s advocate. So in a group of four, they assign one person to be the one that always comes up with an alternative view and has to defend it, right? They’re basically assigned the role of critic. Again, that’s something that not everybody’s going to have the stomach for. I mean, I might enjoy the role of critic because ’cause then it’s get to like poop on stuff and be a jerk. That’s not how they should actually be, right?
Chris:
Civil engineering.
Rhett:
That’s not how they should actually be, but the reality is a lot of people will feel that’s what they’re being asked to do, right? You’re asking me to disagree with the team. I don’t wanna be the person that has to disagree. But the reality is there is merit behind stopping and a decision-making progress and saying, hey, hey, we all seem to be in line. One person, give me the reason why we shouldn’t do this and make the case, right? Like give, give intentional space for the other viewpoint and something that doesn’t just fit with a normalization. So.
Chris:
We have seen that manifest sometimes and I’m gonna, I’m gonna turn it on, on us. I mean, our role presently is consultants and consultants play an important role here, right? You know, we have a couple of clients that reach out to us often and say, Hey, I just need a third party opinion. Kudos to you guys for recognizing that. And there may be different drivers for why y ‘all do that. Maybe it’s economic, maybe it’s cause you actually have some legitimate doubts. Um, obviously consultants play a good role there, but also as consultants, just be careful, right? I mean, we’ve talked about this before too, we’ll have our own biases and that, you know, our livelihood is founded on certain decisions that are made. Right. And so just that whole idea of it, if you are in that role of consultant and you’re brought into these rooms, just appreciate your duty of care of being fair. Right. Sometimes your role isn’t always to find a problem. Right. And so just be cautious there.
Rhett:
Sometimes it’s always not to pass off the problem either. So again, I want to, I’m going to wrap up this episode on, on the condo and hopefully to the audience and this is not a series like BP paid millions and fines. There were criminal convictions that followed, but the actions that directly led to it of the five people that were there, put yourselves in their decisions with their shoes, with the economic pressures that they faced, those are not altogether different than what we face. Those are not when we are faced with challenging information on interpreting ILI, or maybe even the results of a hydro test, or making an integrity decision about something that we think is critical. We find ourselves in similar situations, four or five people on a phone discussing the results. Recognize the role that group think can play there. Recognize the strong incentive to want to find a suitable alternative of explanation that isn’t an anomaly or doesn’t, doesn’t reflect that it’s not difficult to put yourselves there as an audience. Again, hopefully you reflect on that and really think about that in your roles as integrity engineers, integrity managers, maybe even I know we have like VPs and some more senior level people think about that role that it plays on your team and hopefully we can learn something.
Chris:
Yeah. And The consequences are clear right right? I mean this this failure parallels what we do so close. I think you said, you know there was a comparison. I think you said that the To redo the cement or was a hundred and twenty-eight thousand dollars roughly.
Rhett:
 To do the cement bond lock dust.
Chris:
Yep. 128 grand but I think the total cost of the beat this failure is now north of like 60 billion dollars roughly and 11 people lost their lives. I mean, how many times do we see that, right? Is it’s you don’t, you don’t run another pig, or you don’t do a couple more digs, or you use the different repair type, and to save a couple thousand bucks, which in the long term, again, all those things add up, and we agree it’s a business. But if that’s going to be the case, enable your team to ask questions, encourage them to investigate and not confirm.  Rhett:
All right, so hopefully you enjoyed this episode where we tackled something that was non pipeline with a lot of pipeline application When we come back next week, we’re gonna talk about our most recent failure and it’s a pipeline one. Thanks for joining us We’ll be back in two weeks.
-outro-
Rhett:
Credits for this episode go to Sarah Etier, our executive producer. This episode was written by myself, Rhett Dotson, like to give credit to the work lodge for the room that we use and also our source material, disastrous decisions, the human and organizational causes of the Gulf of Mexico blowout by Andrew Hopkins.