GEM 2019: How Can Governments Better Implement Policies?

Salimah Samji All right. Welcome to the fourth panel of the day. This morning, you heard in Lant’s panel on state capability being stuck. This panel should hopefully give you some exciting ways forward potentially. In Ricardo’s words, a secret sauce of how do you actually build capability. I’m here with both Jorrit and Matt, who both have been building capability with, whether it’s city governments or governments in many countries. And they’re here to talk about their work. So what I’d first like to start with is ask each of them to make some opening remarks before we get into some questions. Would you like to share the work that you do?  

Jorrit de Jong Thank you, Salimah And thank you for having me here. It’s I’ve been a big fan of the work of CID for a long time, and I’ve had conversations with you and with Matt and with Ricardo for a long time, but never in such a distinguished audience here. So it’s great to have a conversation. So I’ll keep my remarks very short and I can come back to some things if you’re interested, so feel free to ask. So the Bloomberg Harvard City Leadership Initiative, the mission is the strengthening the leadership and organizational capabilities of cities around the world. Thanks to our donor, Mike Bloomberg, who was a mayor himself in New York, of course, he felt like the private sector has many more opportunities to basically support leaders as they rise through the ranks. And the public sector doesn’t necessarily have those same opportunities for leadership development and organizational development. And so he felt that mayors who do arguably one of the hardest jobs in the public sector around the world should have access to that same kind of opportunity. And so the idea was to focus on leadership development and organizational development in close connection. So it’s not leadership removed from the actual work of problem solving. It’s leadership firmly anchored in the work of public problem solving. So there’s no navel gazing, there’s no kind of fluffy stuff. It’s it’s real hard work. But, you know, raises the question, well, what is the kind of capacity that cities need to do their work? As you all know, cities are at the forefront of social problem solving, especially when national governments fail to deliver or are arguably in the way of problem solving. Then it really comes down to mayors and their staff to do the best possible job with limited resources and limited formal authority. So the work of mayors is really to leverage everything you can to make sure that you understand the problems that your city is facing and that you generate actionable solutions to do something about it. And that requires a bunch of different capabilities. So we started by developing a theory of change for our own initiative that that started with the notion what is it that we that we’re looking for? What is the kind of what are the kind of capabilities that city and city leaders need to actually make progress? And, you know, we withdrew from a lot of different kind of bodies of academic work. But most importantly, we spoke with many city leaders and and kind of tried to design a curriculum that was ultimately responsive to what their needs are. And the big themes were, first of all, performance review capability. And that’s kind of a, you know, a technical term. What we mean by it is the ability to actually diagnose the problems that you have to this aggregate data set. So you really know where your problem is, where who it is, why the problem occurred and so forth. But then also to develop a theory of change that helps you articulate what you’re really trying to do and how you would know if you’re actually making progress towards that goal. And as simple as that sounds, we universally find that in most policy areas, even if it’s not just that the theory of change is imperfect, it’s like there hasn’t been a conversation about it. And so what does it take for an organization or for the leader of an organization to orchestrate that conversation and to build consensus around that and then to, of course, implement? And then the second set of capabilities all around collaboration. And collaboration is the easiest word to say, the hardest thing to do. And so we broke that down into a couple of things that are important for mayors. First of all, how do you lead a team even within your own organization that has so many different departments if you want to reduce. And crime in your city. You can just work with your police department. You have to engage the education department. You have to engage transportation, public works. It’s a city wide problem and it requires a city wide response. And that means then that leaders are looking for ways to form teams that are diverse enough to tackle that problem, to lead those teams in a way that brings out the best in those different, you know, in that collaboration and to kind of sustain the collaboration. So it will go beyond talking and actually deliver. And the final set of capabilities is all about data and evidence. And so, you know, many cities have a lot of data, but they don’t know how to use it. And data access to data, availability of data is a preliminary condition’s a necessary condition, but by no means a sufficient condition. So what do you need to do as a mayor to ask your staff at a meeting, you know, what question do you ask that forces them to actually look at the data, to run experiments, to see what works and what doesn’t, so that you can make decisions about what to continue and what to discontinue and what to innovate and what to improve and so forth. So those three sets of capabilities, performance capabilities, collaborative capabilities and data analytic capabilities can inform the heart of our curriculum. But then, of course, the leadership part requires many more things. So we include public narrative, the ability of using the bully pulpit as a mayor to engage others that don’t necessarily have to work for or with you in a collaborative problem solving. So that’s basically what informed the curriculum. And then what we do is we have a yearlong engagement with 40 cities each year from over the world. About half is American and the other half is from from all over the globe. We start with the mayor. The mayor participates in a three day in-person program that’s a very intensive pressure cooker, really focusing on the particular development challenges that she or he is facing. And then throughout the year, we engage that cohort in virtual sessions. We have a dedicated studio here on campus. It’s like Skype, but then it works and it’s really great. And so, so we, we actually have those 40 mayors on big screens and you can really talk to each other. It’s phenomenal technology and otherwise we wouldn’t be able to do this, but we convene that cohort. So it’s not just a one off. It’s not just you go there and, oh, that was fun. You know, maybe we got inspired, but we keep having that conversation with the mayors and we facilitate peer to peer conversation all the time. The second thing that we do is we ask them to nominate two of their top leaders in the organization. But because we know if the mayor is all enlightened and inspired and empowered, that still doesn’t mean that the rest of the organization is able to execute on that. So we ask them, nominate your two top agents of change and they get a whole separate program that is also starts with an in-person and then continues through virtual sessions. And then on top of that, we have a field support and so the field support is around three tracks and they can self-select into that. It’s data using data and evidence where we ask like, is there a particular problem where you feel that you are not data driven? And so they may choose homelessness or economic development and then we help them through technical assistance to kind of translate those ideas into practice and to use that particular project as an example of how you could get more data driven. The same thing with experimentation and innovation. We all know the public sector, you know, has a proclivity for long term planning and blueprints and master plans and policy. But the kind of agile approach to just trying to get to a minimum viable product and to innovate and to engage others in redesign, that doesn’t happen a lot. So we support them on that too. And there’s individual supports as well. The mayors get a 360 degrees assessment and we thought this would never fly, but we thought it would be worthwhile trying it out. Will a mayor, an elected politician, will she or he actually ask other people what they feel of their leadership style? So we do this analysis of discrepancy between the self-perceived capabilities and what other people perceive, and then they get debriefing of the discrepancies with a professional executive coach. Don’t ask me about the legal and technical safeguards that we had to build in to make this happen. But it was there’s a big take of 75% of our mayors do it. They really benefit from it. So we have this whole year of supports for the mayors. And then the third question, the last question is, how do we know this is working? And I think it’s really important that, of course, as an academic institution, we believe in what we do. And but we also need to apply the same principles of performance measurement, of using data and evidence to see if this kind of training and support works. So we have a dedicated impact assessment function, and we use a variety of different research methods to find out what the results are. And we keep coming back to the mayors. We do impact surveys, but also interviews, and then we have our graduate students who are phenomenal, who get placed into mayors offices as Summer Fellows, and so they’ll be working with the mayor on whatever their priority was. But they also serve as a feedback mechanism for us. They do some interviews and they they tell us as a more neutral observer, if those capabilities if there’s any evidence that those capabilities are actually taking root. Let me stop there.  

Salimah Samji Great. Thank you very much. Matt, would you like to share what you.  

Matt Andrews I’m almost breathless just listening to everything that you do. And the first thing that comes to mind is just to say there are so many things that go into getting things done in government. Right? Like just all of the things that you’ve just described, working with teams, coordinating across bureaus, working with leaders, working on day to day stuff, working on big picture stuff, managing politics. Like this is complex stuff. And that’s our experience as well. I started my career as a public sector economist. My dearest wish was to help policies get implemented in the world. And my understanding was that the closer you were to money, the better chance you had to getting policies implemented. Because governments implement things in two ways. They implement things through projects, or they implement things through budgets. Those are the two primary ways in which things get implemented. So then I joined the South African government. I worked on the budgetary side for a while, and what I saw on the budgetary side was we budget had a lot of great things and very few of them ever got done. So then I left that and I went and joined the World Bank where we worked on the project side and we did lots and lots of projects. And then I left there because I realized that very few of the projects ever got done. Then I came to the Kennedy School and I started looking at numbers and saying, Okay, what in the world is going on here? And my observation was, we’ve kind of sanitized this process that you’ve just described, all the mess. And to get a good idea, put it in a budget or put it in a project document plan, it really, really, really well and even a bit better after that and then tell people what to do. And I started to look at organizations and say, well, how well is that working? Was my experience a failure or just kind of the odd thing or was it common? Now, the interesting thing with policy is that there’s kind of two ways to measure success. And the one is that you deliver on the kind of stuff that you said you were delivering on. And the other one is, did that stuff solve the problem that prompted you to planet in the first place? And I’d say this in the most functional places that I find, the success rate on the first is about 30%, and the success rate on the second is about seven or eight. That’s the number of World Bank project successes. World Bank projects deliver what they say they’re going to do 30% of the time World Bank projects that deliver what they say they’re going to do, but that the World Bank teams actually think they’re going to solve the problem that they started off solving. So a lot less. It’s about six or 7%. So this issue was really hot. That’s the first thing and we haven’t solved it. So we kind of find ourselves in the same place. I think that that that you’re at that a long time ago before you started with Bloomberg and when he started with the Caprio Brigade, which it was trying to work with governments to say, okay, how do you do this little bit differently? How do we up those numbers? And when you’re working in some places, like we were mentioning in the previous session, if you’re working in Honduras, if you’re working in Albania, if some of those numbers are lower. But historically, whenever governments try to introduce new policies, the success rate was lower than that. And you started to read evaluations, which is we are beginning to say, well, why did we fail? And you’d see things like, well, it’s complex. Okay, well, you need to get people to work together. Okay. And the best one is, well, there was politics involved. It’s like, okay, like, you know. And I started to realize that the management mechanisms that we were suggesting were the ways we should do this assumed that there was no politics. And they assumed that people wanted to work together. And they assumed something even more staggering was that we actually knew what we were doing. Right. So in education policy, in my country of South Africa would say, we’re going to train teachers, we’re going to introduce a new syllabus, we’re going to do X, Y and Z and very confidently has like $50 million for the next five years to do all the stuff as if we know how to train teachers. As if the teachers who get trained attend school. As if we know how to execute a syllabus. All of these things are really just gigantic mysteries to all of us. And so we started to see that, you know, even if we were executing on these things, we weren’t solving the problem because we were pretending that we knew what we were doing in the first place. It’s like Lance was talking this morning about doing business indicators, and I know that I’ve been Will maybe speak about this later in Albania, that somewhere along the line some people said, if you do well on these indicators, you do these 14 things, businesses will come to you. My country did really well at this. And you know what? No one came because actually the mystery as to why, even when it was doing well, South Africa couldn’t attract business has not been solved yet, right? Ricardo mean, we think we have some answers, but they don’t want to do them. We don’t really know why. So we started to say, how do we develop a management mechanism that is less imposing and less based on assumptions and more empowering and facilitating of learning? And we started exploring in a bunch of different places, a bunch of different policy areas, a methodology which we now call PDIA, which stands for problem driven, iterative adaptation. It’s a very simple approach, which says instead of pretending that we know the solution, let’s just really start with the problem. It’s just really start by saying what’s going wrong and can we scrutinize that as much as possible? Because the only thing we really can lay any claim to understanding right now is what’s going wrong. And we probably not going to have a good understanding of that when we start either. So let’s understand the problem. Let’s try to mobilize teams within the context to address the problem rather than external experts, because external experts aren’t the ones whose capability we want to grow. Lant was talking about consultants and outsiders this morning as mosquitos. I don’t want to get there, but I do want to say if you’re running a government and your government can’t do stuff, I don’t really understand why you would hire a consulting firm to come and do that for you, because they just going to go back to their consulting firm. Why are you building their capability to do the thing that your people need to learn how to do? Your people need to learn how to do that. It’s fundamental. So whatever your management approach is, it needs to involve that. They need to be the ones to do the work. So we would say, let’s get this team together, let’s understand the problem, and let’s start to find some entry points where they can start working. And then let’s just get them to work and let’s coach them again and again and again very regularly in tight iterations so that whatever they try, they learn from. And let’s do that again and again and again, ad nauseum. Until something happens. And see if they can achieve some things that are surprising to them and see if maybe they can build some capabilities that they didn’t have at the beginning. So we’ve done this now for about ten years, experimenting with the model. The model now comes with tools, and we come with the tool kit because we’ve kind of worked up how we think it can be done. And we have worked directly with about 50, 60 teams across the world. And we find that they kind of achieve things. Probably 60 to 80% of the time. So consider earlier I said 70, 30% success rate. We’re kind of like way above that. And the thing that I would say is it’s not really because we do great stuff. It’s just because you’re taking these teams and you’re giving them, firstly, the responsibility to act in the face of their problems. And secondly, the autonomy, the space and the power. And you’re also putting them on a learning trajectory whereby they can try things out and learn and expand their ability. A lot of the things that Europe is all important are very key to this, too. One is they work as teams, so they’re collaborating from the beginning. We even find that collaborating within a ministry is difficult. Collaborating across cross boundaries is very hard. But even within the ministry, people in governments don’t like to work together. We find that building networks is crucial. We find that being able to go and speak to people is really important. Speaking to citizens, speaking to business, learning what’s really going wrong. Learning how to respond to that. We find that learning how to manage up to political masters is really important. We’re learning that the leadership of the political masters at the top is absolutely vital and that interaction matters. What we’re finding is that the specific solutions to these kind of issues of how to get people to manage better, to lead better, differ across different places. The ways in which you get people to work together, the ways in which you get people to mobilize themselves is not the same in Albania as it is in Sri Lanka, as it is in Honduras, as it is in Mozambique. They are different. But it is those solutions to those very same issues that are the ones that seem to unlock opportunity. And we’re finding that over time, when we’re working in these places, not only are they getting better results, not necessarily always solving every problem for forever. And I have some people who we’ve worked with in the room, and they can tell you about some of our failures and some of our successes. But we finding that the cohorts of people who are capable to do stuff are growing across governments, that when we going to a government, you’ll say to someone, you know, we want to work with a team of 50 people. And they say there aren’t that many people who can do stuff in the government. Think about that and think about how serious that is as an indictment of governments that hire 30 to 40 to 50,000 people, we only trust five. Well, we’re happy to say that wherever we work after two or three years, there’s maybe 200, maybe 300, there’s maybe 400. And we think that that’s how you build state capability. You empower people to take responsibility for their country’s future and to learn how to do that on the go. So that’s kind of how we work. We’ve worked in a bunch of places. As I say, there’s people in the room here. I think Auburn will tell some of the stories later about the work in in Albania. I think the approach is very similar to how you working with the mayors. We have different models. One of our models is we actually come and coach teams in the countries and we usually work with six or seven teams at the same time. I stand by my previous experience. I will never, ever assume that success will happen more than about 20% of the time. So I always insist that we work with five teams, so it guarantees at least one success. But we also find that working with these six or seven teams, we have a very short workshop with them at the beginning, maybe two days where we have them identify a problem, break the problem down, identify entry points, and then they start working. And every week we come back to them and we say, What did you do? What did you learn? What are you struggling with and what’s next? And every month they come together and they celebrate their progress as all six or seven teams, and they learn together. And we usually work on six or seven months kind of, uh, programs. And we’re finding that this is kind of really successful, really interesting, really dynamic, very different to the norm. We, we don’t, we have coaches rather than experts working with people. So even our students who want to come and work with them, one of the first things we say to them is never tell the government what to do, no matter if you have the answer or not. They need to find it. And some people say, why do we need to recreate the wheel? And I’ll say, because the wheel of a Mercedes is different from the wheel of a tractor. It’s different from the wheel of a Fiat and different countries, the wheel might be the same shape, but it is very, very different in most other respects and they need to find it themselves so that they can use it.  

Salimah Samji Great. Thank you. We do. In the agenda there are links to a blog that gives you more information and links to your work. There’s a link to the toolkit. So there’s a lot of information out there that you can already find. What I want to do is ask some questions to them and kind of get into the nuts and bolts of how these methods work. I’ve had the good fortune of working with Jorrit and his team on their Cross-Boundary collaboration in January. So I’ve actually worked with their program and I am the director of Building State Capability, so I work very closely with Matt and kind of from that vantage point, one of the things that is so striking to me is that both of you come to this building capability from four lenses. One, you always start with the problem both of you write and you all have heard it. Problem, problem, problem. Right. They both said problem, right. That’s kind of the stance. And then the second thing that you do is you talk about teams. None of you said, no, the mayor is going to do this or that. No, it’s the team. Right. That’s kind of the second thing that you guys talk about. You guys both talk about the learning. And I know I was that I remember at the opening dinner in New York when Amy Edmondson starts off to, you know, all the teams and they were from there was ten teams from ten different cities. And she says, I’m not here. We are not here to teach you anything. And I was really struck by that because that’s very much the kind of sentiments that we take. We are not the expert here telling you what you need to do. And so that is something that’s very common, this kind of shared learning. We’re here to learn from each other and to get things done. And then the fourth thing is the time, right? Neither of you talk about this one kind of short instant that you do. I’m going to come in, do a training and walk away and boom, you’re going to change your country. No, neither of you talk about that, right? You were talking about this continued engagement, whether it’s a year you’re talk we work six months, a year, three years in Albania, you know, depending on where we are and how long we work. And so those kinds of things were really striking in terms of teams. I kind of want to kind of ask some more questions. How did you guys come up with the idea of a team and what’s been your experience? If you could speak both to the challenges of having a team as well as how successful has it been? Would you kind of change that?  

Jorrit de Jong So. I think that the very first intuition with regards to why we focus so much in teams is that the most pressing problems span multiple boundaries, and if you want to do anything about it, you will have to collaborate with others that are very unlike, unlike yourself. They have different professional perspectives. They come from different organizational cultures, they have different political biases. There are different legal frameworks guiding the work. There are different budget streams that can or cannot be repurposed. So we have coming from an all the way Berrien view. You know, we’ve created public sector organizations that are all structured in a way that’s aligned with whatever the task was before. And now the world has changed and problems manifest themselves in ways that do not conveniently fit or are silos. And therefore it is almost like a premise that you will have to collaborate if you want to do anything about real world problems. And so if you if you even accept that premise, then the question is, well, what is the problem? Right. And so, you know, both through the program and through our research on this work, we find that there’s a fundamental Catch 22 situation. It’s like, do we start by defining the problem and then identifying who should be working on it? For example, if we say there is an achievement gap with regards to education in our cities, which is almost everywhere. Right. So that’s the problem. Okay. That makes sense to then invite, you know, the education departments, you know, maybe some employers, you know, maybe neighborhood organizations and so forth. And then you start working on it. But then, you know, people come to the conclusion, well, actually, the problem is that we’re facing, you know, hundreds and hundreds of years of systemic racism. Right. This is a symptom of another problem. And so we won’t be we don’t want to be fighting symptoms. But if that then becomes the problem definition, you need a much broader coalition of people to work on it and your interventions might look differently and so forth. So where do you start? Do you start by saying, this is the problem, we have to find a team? Or do you start with this is a good team. This coalition actually has the muscle and the will to do something. Let’s have them work on a problem. Now, I don’t think there’s an answer to that. But this is where this key concept in your work mat is. In your work Salima is iteration. You know, you have to go back and forth between problem definition and stakeholder identification. And that is why you can never do a one off. You can never do a one off. I mean, like, unless that’s what you want to do. But if you’re really interested in making progress, you’ll have to kind of accept that. Now, what we find in teams is that this ambiguity and this uncertainty and this kind of lack of permanency is excruciating to people. Like, they won’t sign up for your project. They won’t come to your program if they know that everything is going to be subject to change. Like, what do you want to do here? What are your deliverables, you know? And who’s going to be part of this? And like almost everything you say, anything you say will appeal to some and deter others, which is why this work requires, you know, a sustained engagement with the issue. Being comfortable with the anxiety that is inevitable when you put people in kind of in this kind of situation, and you have to kind of create an accountability structure around the learning process and that around individual tasks. This is why we use the mayors to say, like, mayor, you have to nominate these people and hold them accountable as a group for progress. Then, of course, that’s the accountability piece of it. Then the team process itself becomes really important because people respond differently to this kind of challenge. To be held accountable for something that you can only contribute to in part is really hard. Like, well, you know, I’m a, you know, I’m in the Department of Public Works and now I’m co responsible for solving systemic racism in the city. Like, I was never trained for it. I’m an engineer or, you know, like you can think of all the other examples. So we have brought in Amy, you mentioned her, who focused a lot on the psychological safety required to actually do this work. And it’s not the soft skills. I think those are the hardest skills, the skills of real empathy, listening, overcoming kind of completely conceptual differences. Like they talk back to each other for quite a while before they actually start listening and supporting teams. To do that work is incorporating what we do, but then pay continuous attention to the accountability structure that keeps that collaboration going because the pull back to their departments and to their own organizations and sectors is so much stronger than the poor. You know, that comes from the problem. It’s easier to kind of go back to your organization and keep doing the work that you did. And it’s like, Oh yeah, I collaborated for a little while. But then this is, I think, one of the biggest explanations why 6 to 8% of the projects do not really move the needle on something, right? Because there is no sustained accountability structure that has people focus on the task rather than on the department.  

Matt Andrews You know, when I before we started the idea about 15 years ago, I did some research on leadership. And I was really interested in in what leadership looked like when governments did special things. So people were mentioning Lee Kuan Yew earlier. I spent a lot of time engaging with people around what happened in Singapore. I spent a lot of time trying to understand what happened post 1995 in Rwanda, just a bunch of different areas. And what I really expected to find was people who worked within web in bureaucracies very effectively and kind of made stuff happen. What I found was something completely different. I found that in every single. Including Lee Kuan Yew. And I’m happy to speak to anybody about this. It was not about the individual. It was about a group. You know, even if you read his book, you’ll read that he’ll tell you there are all sorts of names in that book that nobody has ever heard of. Anybody heard of a chin, chin book? Okay. In Singapore, he’s a hero of the country. I encourage you to go from Third World to first by Lee Kuan Yew, because it will disabuse you of any idea that he thought that he was the hero of his country. And what we found was there were many people involved in the stories. And one of the reasons why there were many people is there were many roles to be played. You find some people who were the authorizes. You have some people who were motivators. You have some people who are identifying problems. You found some people who were coming up with ideas. You found some people who were funding resources. Then you find people who just implement it relentlessly implement it, and lo and behold, they were not all one person, right? All of these roles are required in getting started on them and they cannot be done. But we do not have to put people in the world. We are all mortal. And so when I started to realize that, I started to say, Well, how do these people work? Because our organizations aren’t really set up for these people to be working together like that. And you started to find that they were actually breaking out of the boundaries of their organizations and working in flexible arrangements, which is what we call teams. Right. They were they were working in these mechanisms that could go across organizations, could go down organizations where everybody had an equal voice and when things happened in a much more organic way. And that’s where the observation of teams came up. And we find that they are just tremendously vital. Now, people play different roles in teams, and so finding people to play those roles is truly, truly important, but very hard, which is why we to exactly like you said, you know, you don’t know who you’re going to start with. I would also say this, in most of the cases where we start, people will say, well, you have to have the right people in your team. And I say, you know what? Just find a team and find something to start with and just see what happens. Because you find that the people who aren’t meant to be on the team, they drop off the team, the people who are not on the team and who should be on the team, they kind of find their way to your team and they say, I want to be a part of this. And over time you kind of end up with the right thing. The key is learning, and I think both of us say it’s about learning is that most of the policies we dealing with our policies to do with novelty, the government’s trying to solve problems that haven’t been solved before, which means that they trying to find things that they haven’t done in the past. And the only way that you find things you haven’t done in the past is by learning how to do them right. And the learning that is required is what record labels as kind of know how. And it’s tacit learning. It’s the learning that happens when you’re on the journey. And so that is why we like action learning too. It’s not just about learning where you teach people in the classroom, it’s about people working together and learning from what they do now. We also found that the team is vital to that because people often don’t learn well and people on our own. I don’t think we realize how bad we are at learning from our experience. And I really mean this. We find that on the team, people will go through something they’ll do, they’ll do something in a week and they’ll come to the team meeting and you’ll say, Jim, what did you do this week? And Jim will say, Nothing. It was a bad week, everything was bad. And then Jane will say, But Jim, didn’t you call this organization? So yes, I called them. Well, what happened? They didn’t answer the call. So you learned something, Jim? What did I learn? Well, you learn that they don’t answer the phone and you literally have people on their say. But that can’t be a lesson. No, no, no. It really is a lesson. Right. And anybody who’s worked in a government knows that it’s a really important lesson that people don’t answer the phone. It’s like it’s like learning that I don’t respond to email because the next thing that the team member will say is, Jim, how about next week? You go and visit them and they’ll say, okay, I’ll try that. And the next week he goes and visits them. And what happened, Jim I went to visit them. What happened? I got all my answers. We’ve opened some doors, things are happening. But you need that dynamic of multiple people working with each other to let them see the hidden lessons for that to happen. And it’s not something that is easily done in a power relationship. It needs to be done in a relationship of peers, which is when done well, what a team is about.  

Salimah Samji Great. This morning, Lant talked about experts and consultants and compared them to mosquitoes. We’re talking about teams and learning, right? So I wanted to like a quick response on is there a role for experts and consultants? And if so, what is that role? Are they mosquitoes or another kind of bug?  

Jorrit de Jong Well, there is, there’s a large variety of mosquitos, some carry diseases. And, you know, I don’t know. You know, like you made a joke about the consultant. It’s the person that takes your watch and tells you what time it is. Right. And so and I think in academia, you know, there is the stereotype that, you know, we teach. And, you know, I think what we try to do is to not be consultants. So, you know, we rarely give advice on what to do. I mean, almost never we will connect them to resources of people who are really great at economic development or health in cities or zero carbon emission programs. Like we see ourselves as a broker role to connect them to people who really know, you know, deeply what needs to happen in certain steps of the various. But all when it comes to leadership and organizational development is to help them ask the right questions. And the right questions are questions that are better than the superficial questions and that kind of there’s a process of sustained inquiry. So you get to the bottom of what you need to do. Now, the other thing I would say is like, we don’t teach, but we facilitate learning. And the learning is between people in a team, but also from city to city and even from previous cohorts to the current cohort. So we’ve been writing about, I think, almost ten teaching cases. For those who don’t know, like the teaching case is kind of the Harvard case method is a, is a discussion based, participant centered pedagogy where there is a story that doesn’t contain analysis but presents a dilemma from the perspective of a protagonist. So for example, writing cases about the removal of the Confederate statues here in the South and many of our mayors in the first cohort, we’re dealing with the aftermath of the Charlottesville riots. And they were confronted with protests, and they were they were asked, like, what should I do? And even if you believe that the statue should go, that’s easier said than done. If you got a, you know, a large violent mob that wants it, you know, etc., you can imagine the dilemmas, the legal dilemmas, the political dilemmas, the cultural dilemmas that a mayor faces. And so we were writing three cases of mayors and the first cohort that responded very differently to that situation, and then we will be using that in our second cohort. So this is one way in which I think external people such as ourselves can be helpful because we, we get the learning from them, but we curated, right? We bring it to them in a way that actually helps them think and internalize the knowledge rather than just throw information at them. The second way in which I think we can be helpful and of course we’re measuring if that actually is true, because I don’t want to assume that it’s helpful, but it’s it’s the coaching model that we use here on campus for students as well as for executives, is that, you know, you have to do two things to help people learn and learn deeply back to match point one is you got to support them, meaning that you get to connect them to the information that they need. You got to, you know, almost hold steady a process where people kind of grapple with the issues, both the substantive issues and the team issues, and you have to challenge them. So we’re not all about Kumbaya and, oh, let’s all work together. You know, if people have kind of an artificial harmony and they are too polite to each other, they pretend like they’re on the same page with the problem definition. They pretend like they’re going to be working together. They pretend they’re committed. We will call them on it. And you actually did a great job on this with your team, but that’s kind of the spirit like, you know, it’s if you don’t challenge a team, they, you know, they get complacent. If you don’t support them, they get burned out. And so, you know, I think I see your job as to kind of get that balance right. And that balance may be different from team to team, but all of our teachers all over or I should say facilitators, they all focus on, you know, noticing when a team needs to be challenged more and noticing when it needs to be supported more. And then we try to have a kind of a tailored approach to that.  

Salimah Samji Great. It’s interesting. We use the same terminology. We use facilitator, we use coach. We never use PDIA consultant or PDIA expert. It’s always facilitator or coach. Matt, did you want to comment? 

Matt Andrews To I think what I would say is you need to differentiate between expertize and expert expertize matters. The thing that is always interesting. To me in the places where we work is we working with teams where things have been tried before. Usually outside experts have been brought in before. Usually they’ve offered really, really good ideas that have sat on the shelf somewhere. So even if you get someone who really knows what they’re doing, you need the expertize within the government to use those experts properly. And that is often not there. And so, you know, even where those people are not mosquitos, but they are whatever and what they eat, there needs to be a it’s actually a skill and a capability to use a consultant. It’s not an easy thing to do. So we think it’s about expertize. The other thing that I’d say is in most of these places, there is subject matter expertize and there is country expertize. And, you know, the expertize required to to deal with the electricity problem in Honduras is partly about electricity, but much more about Honduras. Right. So an expert in electricity is going to give you a small part of that solution. But the people who you’re working with in the government are the ones who have most of that expertize. And what we try to do is we are trying to lay on the response, the responsibility on them, that they are the experts for a significant part of the problem. When they run into a part of the problem that requires the subject matter expertize that they do not have, then I say absolutely, we need one of those people to come in, but we don’t want a mosquito. We want someone who’s going to come in, who’s going to give them the ideas that they need in a way that they can use it. And they need to be at a place where they can use it as well. And so we found that that is kind of a really important balance. I think there is absolutely a role for outside people to come in with smart ideas, but those smart ideas need to find fertile ground and they need to not crowd out the expertize of the people there. I think probably even more than that, they need to not be provided with the message to people in the government that, don’t worry, someone from the outside will come in and tell you what to do. And because we see that happening again and again and again, we have a problem. Let’s bring the expert in. And all they’re doing is that we’re was saying earlier, all they’re doing is they just they just avoiding the responsibility. Right. Avoiding the responsibility. But absolutely. I mean, they are people and hopefully we teach some of them in our programs who just have subject matter expertize. That is going to be important in specific situations, but it needs to be put into the process rather than come and subvert the process.  

Salimah Samji Great. I have several other questions, but I’m going to open it up because we have 30 minutes left for this panel and I’m sure you have a lot of questions. We’re going to take a set of three questions. If you could just raise your hand so we can bring the mikes to you. Yes.  

Attendee 1 Thank you. You’re preaching to the faithful. But the idea when you go to and you talk to the decision makers to two problems. First of all, the definition of the problem itself, it seems we talked last night about this. I mean, it seems kind of naive saying, oh, let’s agree on the problem. Take Syria, for example. If you get in, talk to people about reconstruction, for example, the definition of the problem itself, you might not get there. I mean, even you might not get to the definition itself of the problem. How do you deal with this? And number two, you’re talking to decision makers who have short periods of time. Maybe they’re elected officials they appointed for three years, four years. And you might come across as people who are doing this for the greater glory of knowledge. You know, let’s try. This seems like we don’t we have an unlimited time. We have unlimited resources. While the decision maker is boiling, he’s trying or she’s trying to solve a problem within a definitive parameters in terms of time and resources allocated to a problem. How do we deal with these two questions, if I may get your thoughts on this. Thank you.  

Salimah Samji Great. Any other questions? Sorry. Back there, Amar.  

Attendee 2 Thank you. So earlier this month, 100 Resilient Cities flagship project, the Rockefeller Foundation abruptly announced that they were going to stop implementing this $160 million program around the world. It came as a surprise to many, but perhaps not to others. If you look at the evaluation report, it basically finds that they tried to create a private sector marketplace for goods and service providers. That failed. They tried to institutionalize Brazilians and silo how people think about resilience. It has created a, you know, appointed chief resilience officers and created this these plans. So the idea was that around this problem, we will, you know, bring people together. But what they found was that the actual benefit to a lot of the mayors, a sort of the chief resilience officers, was this network of resilience practitioners that was created that they didn’t really anticipate happening. But of course, this was a major, major investment. And the idea that sitting in New York, you can start a global movement that percolates around the world. Do you think that was destined for failure or were there elements here that you teach perhaps to your mayors that could be replicated in future programs?  

Salimah Samji Great. And the other question. I’ll take those two.  

Jorrit de Jong Did anyone? I can start with the last question and then touch on some of them, then turn over back to Matt. So I heard that that program is going to be defunded. That’s bad news. I don’t know them very well, but I admire the work that they do that they did and still do, I hope. Well, I don’t know why it was defunded specifically, but I do know that these networks of people in similar roles are extremely important. So what we do with our program is the mayors when they get together. It’s started with when we started designing this program, we had five worries. The first worry was, would we build it? Will the mayors come? And secondly, if they come, will they do their homework because we actually have them read the cases, otherwise you don’t ever. Secondly, even if they’ve written the cases, will they be on their phone all the time under the table? And then thirdly, you know, will they actually stay in the room or will they be called out for crisis and emergencies? And, you know, will they find this useful? Can they take this back home? That’s the most important question. And so what we found is that, yes, they showed up, all of them twice because we’re in their second year now. They did the reading. Apparently, we had given them that stern message that we really have to do. Thirdly, yes, they engage with each other. We have a purpose built classroom that it’s almost impossible to kind of, you know, to check out because there’s peer to peer accountability. Fourth is that they hardly ever left the room. And fifth, they said that this program was extremely useful in a very practical way. But what we didn’t know, and this gets to your point, is that one of the things that they value most is the ability to talk to people who are in the exact same situation that don’t need anything from them except learning. And so it’s a you could call it the sanctuary if you like, but it’s definitely a peer support network. We have a group of African-American mayors that created a WhatsApp group and that started kind of an advocacy coalition. And we have a group of mayors that focus on education reform. They’re in touch with each other. We have mayors from coastal cities that are in touch with each other because they face similar issues. And so that network is is extremely valuable. And it doesn’t that network doesn’t necessarily happen in these events where there’s a lot of a lot of vendors trying to kind of lobby them. There’s a lot of convenings for mayors, but the fact that this is mayors only is helpful. But it’s true for the chiefs of staff, too, because we convene them and the budget directors and so forth. And the chief resilience officers, I think, had a similar support network. And then to your point of how do you deal with the problem if it’s clear what the problem is? It’s easy to say, well, you got to agree on a problem. So one example that I’ve been involved in, in another effort to organized crime field lab, which is against organized crime, it’s not supporting them. It’s just I have to say that is that we were focusing on human trafficking in Amsterdam. I’m from the Netherlands. And the problem was. We can’t get to these organized criminals because human trafficking is a very lucrative crime and it’s international, it’s extremely well-organized, and it’s some of their capabilities that we talked about performance, orientation, collaboration, agility, data driven. They’re better. They’re like two or three steps ahead of the government. So what do you do if you you know, how do you define the problem in a way that makes it actionable? Well, if you look at the root cause, it’s deep global inequality and, you know, and economic disparity. And this is what drives a lot of crime and immigration and so forth. That’s if you’re a prosecutor in Amsterdam, that’s not something you can work on necessarily. Now, does that mean that everything else that you choose to work on is just fighting symptoms? Well, maybe and maybe not, because the way we can convince them to actually take this on is to say, well, look for an entry point, let’s try and, you know, start somewhere in kind of sabotaging and frustrating the enabling infrastructure in Amsterdam, the hotels, the that the lawyers, the accountants basically all the car rental companies, the tax lawyers, everybody. That’s part of the infrastructure that that that enables basically human trafficking. And if you start destroying and putting grit in and a machine of that infrastructure, then you’re not addressing the root cause, but you make it a lot harder for them to succeed. And if you only did that, it wouldn’t be enough because you want to see if you really are effective in sabotaging those enabling factors. See what happens in the system. Make sure to give a learning orientation and see, well, where does it move? Now, let’s say we take one city and we make it really hard for modern slavery because that’s what it is to really occur there. Then see where these traffickers go, see if the prostitution moves, see where the the forced labor moves, and then, you know, see what next intervention you have within your scope of authority and capability that you can do. And that way, it doesn’t feel like you’re doing something that’s not effective. But it also doesn’t feel completely overwhelming to take on this huge global problem. And I think that kind of approach we’ve taken with many of the teams is even if you can’t address a root cause, that doesn’t mean you can work on it, just good at something that you can work on and then see what happens if you do.  

Matt Andrews So let me raise the two issues. The one is about the problem. These intractable problems that I want is about leadership. I’ll get back to the problem in a second. How do you speak to authorizers? I have a few of them in the room actually, who are convinced to try this out. So you could speak to them afterwards and see what their thought process was. My sense is this we say you want to fund it, authorize it within it. An authorizer with an edge. So someone who is basically saying, I only have three or four years, but I need to try something different because that’s what this is. And usually I think, you know, I started off by saying the success rate is somewhere between 8% and 30%. What I’m looking for is somebody who has a position of power, who wants to do a little bit better than that, and who’s actually done things the old way multiple times and realizes it doesn’t work because they are they taking a bet on this rut? And I think it’s explicit. This is different. This is out of the norm. It looks fine. It feels smells different. So I’m looking for an authorizer who is looking for something that is different? And there are lots of them. There are lots of them. We engage with the authorizes a lot very regularly. A key thing that the authorizes buy into is the idea that they not just getting delivery, but they’re getting capability and that we’re going to be coming back to them again and again and saying, this is what we’re learning about your people. This is what we’re learning about your context. This is what we’re learning about information. This is what we are learning. This is and they’re going to be kind of part of a process that is very aggressive, where it’s not something that in three years time we’re going to see, did we work or not? It’s going to be regular, regular conversations about what’s going on in terms of the problem. You know, man, I don’t want to start with saying kind of what’s going on with Syria. Who’s part of the problem with Syria is that that’s where the problem is being discussed right now. The problem is Syria. Hey, let’s break that thing down. Ricardo has a great saying, why make something difficult when you can make it impossible? Right. So I say many civil servants, they deal with something and we want to have the conversation at the level of impossibility. Kids aren’t learning in your country. I don’t know what to do about that, but. So why don’t we just break this thing down? Why don’t we fund it? Why don’t we make it smaller? Every big problem is just. It’s just the combination of many, many, many smaller problems. And if you can break them down and you can find an entry point and you can start in that entry point, you can start building confidence in your system. You can start building confidence in your people. You can start penetrating the problem. I like the idea of being a little bit subversive and you start to kind of almost work your way into it. So we have teams that take these big problems and then they use very simple tools like problem trees and things, and they just identify what the roots of the problem. And then we say to them, Where do you think you can begin? And you know what often happens? They often say, we’ve already started on a bunch of these things. So that’s fantastic. How do we just go faster? Right. And then they start going faster and then they start to build confidence and they say, we can jump to the next thing. We can jump to the next thing. It’s about starting where you are empowering your people. Having them take responsibility to actually invade the problem where it is learning and moving to the heart of things as we move along. We worked with the Albanians and Orban has just stepped outs and I can tell the story just in case it’s he questions that a team that was working in the clothing sector in in Albania and the clothing sector was struggling a little bit. And the government said, our problem is that they are our major source of foreign exchange. We need them to bring we need them to increase their performance so that we have a good reservoir of foreign exchange because they were coming out of a financial struggle, which I think he will describe. When we got together with a bunch of government of government officials, they were three or four in the room. He said, well, the clothing sector isn’t producing enough foreign exchange. And they said, Why are you coming to us with this problem? I don’t even work on the clothing sector. I know nothing about the clothing sector. They said, Well, let’s just start breaking it down. They started breaking the thing down. And they identified firstly by going to business, they identified over 50 different problems. Right. And you might say, Oh, that sounds terrible. But it wasn’t terrible because the thing of it, increased performance of the clothing sector is intractable. You can’t get there. But then there’s 50 things you say. What do you think about these? I said, Well, we are already doing ten of them. And so they initially sort of they said we are already doing ten of them, so we need to make sure we get them done. And then they said the next ten, we think we can really get on two. And then they. And then they left five or six and they said, We can’t do those. So we said, okay, why don’t we just. Why don’t we just start with the ones you’re doing? After seven months, they cleared the decks. Everything had been done. Everything. Even the ones that at the beginning, they said, we can’t get there. And part of it was that their capability grew. Because capability is not just ability. It’s not just competence. It’s also power. Right. And power is something that is bestowed on you and something that comes from inside you. And what we found is that as they started to kind of see successes and see that they could do things, they just became more and more empowered. And that connection of power plus stability just allowed them to clear the decks with everything. But you can’t start with. Syria is in a mess. It’s asked the question why it’s asked a thousand times. Let’s break it down to many, many of the areas and let’s find entry points. And it’s mobilize people to work on those entry points and let’s see how long it takes them to invade the whole thing. That’s, I think, a common belief that we have. Don’t start with the thing that that today is impossible. Find the thing that is possible that makes the impossible possible over time.  

Jorrit de Jong I can add to that. So it’s funny. Many of the problems that we’re talking about are so complex that people feel often paralyzed, that they don’t have enough information or understanding of what the problem is. And like, we’ve been trained in public policy schools, including this one. It’s like you can’t act unless you understand, you know, like it would be foolish and irresponsible to engage without having done all your analysis. And then the problem is we don’t have the data because, you know, we don’t have those data capabilities and so forth. And even if we had the data, would we really agree on the analysis? Right. So a quick example of from the same organized ground field lab, there was a $20 billion from a particular area in going every year. You know, it was it was an informal economy around synthetic drugs. And they knew that there were going to be deposits. And there was an international financial network that was, you know, channeling all this money laundering and so forth. And it was a group, a team put together of 12 different people from different agencies. And they and they were doing this analysis. They were trying to put all their data together. They were really willing to come up with a better approach. And they couldn’t get there. They didn’t have real time data. They couldn’t agree on data standards. They had a lot of expertize, you know, among them, but nobody really knew where it was coming from. And they didn’t have the capacity to investigate because the problem is hidden, and particularly with hidden problems, including human trafficking and, you know, large scale organized crime, you know, it’s kind of a convenience not to look. And so, you know, this problem was in equilibrium. So what they did is ultimately this took like months of hard work. We said, why don’t you go try deposit $25,000 in Rotterdam in one of those money transfer? Why didn’t you just do that and see, first of all, if they’re in compliance, if they because they’re legally obligated to report unusual transactions and like they were looking at us like, what do you think it was? You could get $25,000 somewhere. And it it’ll be a good investment even if you lose it. Right. But you won’t lose it because you can arrest people if they. And so in one day, in one day, they closed down three of those money transfer offices. They arrested 11 people. They confiscated a ton of laptops with crucial information. And that led them to very specific investigations. And but it took like three months to actually get out there, get a, you know, a gym bag with unmarked bills and then and then go to two, you know, and sometimes you just need to try something clumsily and then really focus all your attention on the learning that comes from that first clumsy intervention and then take it from there. The, you know, the analytic approach may come later than the action.  

Salimah Samji Great. Ricardo, you had a question, Frank. Oh, sorry, Frank. Ricardo and that lady.  

Attendee 3 So I have a question. You know, yesterday we started with a discussion of Clovis and the, you know, the Germanic tribes had a practice. And, um, and here we are today. And that’s, that’s why we do the things. That’s why some countries do things one way and other countries do things another way so that some of these practices are are very long term and in general, etc.. In a you people can read you as at different levels. If you are the head of a department in a ministry that is going to solve a problem. How do you empower your people in your department to solve the problem? Or suppose I’m the minister. And so how do I implement this? Or suppose I’m the president. How do I implement this? And and how do I beyond beyond sort of like actually doing the work. A change the culture so that, um, you know, people start thinking about organization and good practice and performance and merit in a different way.  

Salimah Samji Great. Thank you, Frank.  

Attendee 4 So I am curious how interventionist you are with PDIA. So I’ve had experience running my own program where your teams and we saved, you know, set a problem and come up with solutions and so forth. A lot of times they will come up with a completely absurd problem. Usually it’s too big. It cannot simply cannot be solved. Or sometimes they come up with a hobbyhorse solution like they’ve got some technology they want to use, and you just know that it’s not going to work. Right. So do you just let them make mistakes until they figure it out for themselves? Or do you actually say, actually, you know, you got to go back to the drawing board because we kind of the outside experts sort of know that this sort of thing doesn’t work in other cases. And.  

Salimah Samji Thank you. Why me? You back that up. Thanks. I want to ask Matt a question and maybe it’s similar to what Ricardo saying. I’m not sure, but not everyone can afford your services. So what does it mean if I’m trying to organize a department or a city or a presidency? How do I think about this? And what would you change? So often you’re intervening in, let’s call it a very big in top bureaucracy of one kind or another. What are the implications of what you’re saying? Which makes a lot of sense to me. Well, how you would organize an institution or a bureaucracy from, you know, not from de novo, but what does it mean in terms of the theory of how you organize the state?  

Matt Andrews Can I go first? So just a couple of things. Firstly, Frank. I agree. I think it costs us when we intervene because I think this is a learned practice whereby people welcome the intervention and they therefore they will they will defer to you. And we’re trying to create the culture whereby people take responsibility. So it is costly. There are times where we will intervene, though, because sometimes we intervene because I’m concerned that people are going to. One of the things we have to be careful of is we’re working in environments that don’t always facilitate the autonomy and the empowerment that we’re thinking of. And the team might actually want to go faster than we think is good for them. And so the time when I really will intervene is when I’m trying to say to them, Slow down, right? Because I’m concerned I first do no harm. If they come up with silly definitions of the problem, we have mechanisms in our problem diagnostic that kind of deal with that. So in the diagnostic, you know, if they say our problem is this, I’ll come back to them and I’ll say, apply the tool again, apply the tool again, apply it, because they always start here and essentially we just funnel them down. Right and so I can deconstruction also. Yeah. So, so it’s kind of we have a process that is called a deconstruction process where we, we just apply. They just ask why many, many, many times. And within the scope of their tools, those tools, I’m not going to them and saying you’re wrong. I’m going to them and saying, Do it again, do it again, do it again. We have a bias towards action, though. So our workshops, if at the end of a workshop, they aren’t where we want them to be, which I used to really fret about, and I would say this team isn’t going to get there. And then I’d say, maybe we should come back for another workshop. The best thing to do is push them to action because they will learn if the problem is too big and they choose something to do at the end of the first week. Remember we iterating correctly so we don’t come back after six months or even three months. The first planning with them is going to be after the first week they’ll say, Oh, we did this and we learned that we couldn’t do it. Why? Because we got the problem wrong. So we went back and we and there’s something about kind of having some faith in the process that this iteration actually works. So you, as the as the coach, don’t need at any given time to be at the right place, you need to have faith that it’s going to happen now. It does mean, though, that if I think that they’re kind of doing something that looks like it’s not going to work, I’m going to be very aware of that in the coaching process and I’m going to be looking for that and I’m going to be trying to find a way to coach into that, but not in a way that takes the responsibility away from them. So it’s a little bit of a kind of a dance, you know, and but one of the main things we’ve done is created tools that allow us to engage in ways that don’t look like intervention. We never, ever want to say to them, No, you’re wrong. We would rather say to them, Try it again. Ask the question in this way. Have you thought about it like this? Never saw them know you’re wrong because they need to be the masters of their own destiny. And if you’re in a place where you know they aren’t where you want them to be, just go with it. That’s the other thing that’s interesting to me in many of the experiments we’ve done. In many of them when I thought that idea is really silly and they’ve done it, it’s turned out to be less silly than I thought. And when they’ve been team members that I’ve thought, why are those people on the team? I don’t think they should be there. They’ve turned out to be a lot better than I thought. So there’s kind of surprises along the way. Your questions are hard. We work with people at different levels. There’s different ways in which people consume this at different levels. There’s different things that prime ministers are looking for, ministers are looking for, people in ministries are looking for from this. There’s different things that are required from them. I’m. We have free online opportunities for people to process and learn the tools and the tools. We have a very, very simple. So we’re trying to make it something people can access. If someone were to say, look, we want to do this. How do I organize myself to do this? The first thing I’d say is there is no such thing in a government as organizing from scratch. You’re always working within what you have. Which is why I like teams, because I can say to someone, you don’t have to reorganize your organization, just create a team. And a team is a a a flexible entity. And then we’d say, now bring the team into this process with you. So if you know, if anyone at any level of government said, I want to do this because here’s has the problem, I’d say we need a problem and we need a team. And that’s it. And we need and we need you to commit to do this for six, four, six months. That’s it. And I think maybe the last one may be the hardest one, because the commitment needs to be even longer. We’ve been working in Albania for a long time, and I can tell you, you know, whenever I go to Albania, there’s people who are thinking in different ways. And it’s not just because we’ve been there, it’s because the government has been the government has been, in my opinion, and we have some Albanians who are on a journey of rethinking how they engage with their people. That is way beyond our program for a long period. And it’s been patient and it’s been willing to say we don’t necessarily know the answers and we don’t necessarily need to see the result tomorrow. We’re going to work with people. So for us, if someone were to come and say, we want to do this, I’d say, Do you have an ice problem? Gee, are you willing to work as a team and are you willing to kind of put real effort into it over a period of time and be patient? And those three things are the key. I think. But we can talk more after.  

Salimah Samji Thank you. Or do you have anything to add?  

Jorrit de Jong You know, thanks to my Bloomberg’s generosity, we’re able to offer these services for free so the cities don’t pay anything. And so we select based on their engagement. So I really like the phrase authorizer within it. We’re looking for authorizers within which we have the mayor of Freetown, Sierra Leone, in our program. And she is an authorizer with an issue, and she doesn’t have the kind of capabilities in her organization to do what she wants. But that doesn’t hold us back. We’re like, we’ll work with an authorized within it. And then another example, the mayor of Fortaleza, Mayor Claudio, is part of a program. And to Ricardo’s point of how do you engage everybody from top to bottom and have them work together and play their part, which makes it less important for you to stay engaged, right? Ultimately, you want to create a dynamic where everybody plays their part. And so what he thought when he came in, he had run on a platform where he was going to dramatically improve the medicine distribution in Fortaleza. And this is, you know, constitutional rights to have access to medicine there. And he wanted to deliver. He’s a doctor, so he thought he came into this program. Now I will know how to manage my people to do this. And then he came back from the program and his senior leaders that were in the program said like, well, he didn’t change the rest, but he got he came back a changed mayor because he had been leading his meetings where he was just giving orders and, you know, holding people accountable for targets and so forth. He had never been asking them questions. So he came back from the program with a whole set of questions that he had never asked his staff that empowered his staff. And that changed kind of and he changed the team around a little bit. And so he reported back. He said, like the big difference was in myself. And because I became a more effective leader, my teams became more effective. Because they became more effective. They were they felt more empowered to try different things. And because they tried different things, he was able to deliver on that promise. And so I think on our best days we do that right. It doesn’t always work. It takes a leader with an itch and it takes kind of some level of support. And, you know, where are we going to live? We’re in our second year. So in this fall will present the first impact assessment because everything we facilitate or teach, we apply to ourselves to and we’ll find out, you know, what the correlation is between the level of support and the level of success and see is to look for patterns and see where we can best put our efforts.  

Salimah Samji Great. Thank you very much. We’re out of time. Jorrit and Matt will be around if you have any further questions. Thank you very much. A round of applause. Thank you.