Transcript of https://www.youtube.com/watch?v=qX57Mob_eZY, published 2025-07-01. Auto-generated captions, lightly corrected; the video is the canonical version.
AI governance is not just important for those companies that are selling AI based products. It is just as important for companies adopting those AI based products because the risks are similar. The risks don't care whether you're producing the product or using the product. The risks are the same. Gather around as we spill the tea on cyber security. We're talking about the topic in a way that everyone can understand. I'm your host, Jara Rowe Row, giving you just what you need. This is The Tea on Cybersecurity, a podcast from Trava. AI is a huge topic right now and we've already talked about AI compliance on the podcast, but now we're diving into AI governance, what that
really means, and why it's becoming essential for any organization using AI. On this episode, we'll explore how governance differs from compliance, who should own it inside a company, and how businesses can build ethical frameworks for AI use. Plus, we'll talk about the real risks and challenges companies are facing right now. But as we know, I am not the expert, but I have my favorite cyber security expert with me today, Jim Goldman. Hi, Jim. Hey, Jara. Good to see you again. Nice seeing you. I know you've been on the podcast a couple of times, but just in case this is someone's first episode with you tuning in, can you go ahead and introduce yourself? Sure. I'm
Jim Goldman, co-founder of Trava, formerly CEO. I've stepped aside as CEO and now I'm doing what I love best, which is working with current and prospective customers on their security and compliance needs. Fantastic. Okay, so let's go ahead and dive right on in. So, just to kick things off, can you explain what AI governance means in simple terms for cyber security newbies like myself? You bet. It's almost like we have to take a historical perspective somewhat jar on this in order to we'll kind of get to AI governance but we need to talk about governance in general first and you know here's the thing this uh notion of security GRC security governance risk management and
compliance has been around for quite a while but what I would say is subjectively speaking the R and the C the risk management and the compliance have and more widely adopted are more mature than the governance part than the G part. The reason I say that in my experience the governance part of you know security cyber security etc fell mostly to the people in charge of information technology or security and compliance. Where it hadn't gotten to is it hadn't gotten to the senior leadership of the companies to the board of directors to the risk and audit committee of board of directors etc. So it was like this exception. So cyber governance was this exception over
here that the senior most leadership I don't want to say wash their hands of but it wasn't something they were staying on top of on a regular basis. That's now all changed. I think as a reaction to the realization that when there is a large cyber incident, a large data breach, it's not just an IT problem. Now it's a reputational problem. Now it negatively affects their stock price and they're a publicly held company. So now it's sort of got the attention of the board of directors and regulatory agencies or agencies that put together security frameworks and so forth saying, "Okay, we've been too casual about this. We haven't been prescriptive enough. So now we're going
to get a little bit more prescriptive." The best example of that is with the NIST cyber security framework, the NIST CSF that we've talked about in the past. And you know the original NIS CSF version 1.0 was created in 2014. NIS CSF 2.0 came out in 2024 and significantly they added this whole chunk about governance. That was the big difference. So it's like okay we're not just going to leave this to chance. We're not going to assume everybody's doing it correctly. It's like no this has to be done this way. and they're very specific about the requirements for senior most management to be involved. It's not just a technical problem. It is a corporate governance
problem. So AI to get to your question, AI piggybacks on top of that and saying AI is not a data science problem. It is not a problem of the engineering team to worry about solely. Certainly, you know, they have responsibilities as they should, but governance is a top-down function and what they're saying if AI governance has to start at the very highest layers of organizational leadership. We did actually just talked about that on a different episode about how everything kind of starts top down and it's not just on one person to care about. It's like a team effort. The most successful, whatever you want to call it, cyber security, information security organizations are the ones
where everybody buys in and they don't just say, "Oh, the nerds over in IT, you know, take care of that and I can just go about my business and not worry about it." All right. So, as I mentioned at the beginning, we've already had an episode about AI compliance, but can you tell us how AI governance is different from AI compliance? Absolutely. So as I alluded to previously, a lot of it has to do with the organizational aspect of it in that seniormost uh layers of management are responsible primarily responsible for AI governance. You know, the buck stops with them. Whereas the compliance end of it is more with organizational layers, the doers if you
will. So in other words, we set the philosophy at the highest level, but then the implementation of that philosophy has to be done by the people actually doing the work, right? The AI experts, the machine learning experts, the engineers, you know, the deployment people, etc. They're the ones that have to gather the evidence to say, "Here's our policy. Here's our process. Here's our procedure. Here's our control. Here's the evidence that we're doing these things according to our policies, procedures, and controls." Yeah. I will admit, especially since I'm still learning this industry, I always kind of got confused by governance and compliance. But you putting it that way definitely like makes more sense. Right. Right. So,
you know, the other the way to think of it is in almost like a public sector context is governance is where our representatives pass laws, but then law enforcement is the compliance part. That is super helpful. I just wrote that down. Yeah, I just made that up. Well, it's helpful. So, okay. Next. Why is AI governance necessary for organizations even using AI? And that's a very good distinction. AI governance is not just important for those companies that are selling AI based products. It is just as important for companies adopting those AI based products because the risks are similar. the risks don't care whether you're producing the product or using the product. The risks are the
same, you know, and then there's the obvious ones. It's sort of like risks are the antithesis of the desirable characteristics that we want from AI. So for example, the thing that the that most people are worried about is because of a lack of quality control on the data that AI is based on. And and you know this is fundamental like AI is not magic, right? AI is this super intense processing but what's a processing? It's processing data, right? And so there used to be an old expression back in the when we called it computer programming. It was garbage in garbage out. Your output is only good as the quality of your input. And so I
think in the rush to implement AI, I think that and this is just my opinion. I think that there's been a lack of clarity on the importance of data management and more specifically data quality. That is the foundation is the fundamental layer of any AI system that's going to sit on top of it. So getting back to my previous point, the worry about poor data quality is you're making bad recommendations, right? And those bad recommendations could even lean into bias, etc. Because if you think about how AI works, it sort of builds on itself, right? And so if you've got some kind of incomplete data or something in your data layer, that first iteration that
goes through it produces a biased or slightly wrong recommendation or something like that. That path now becomes data for the next generation, right? So it's like the bias or the error perpetuates and could potentially even accelerate. Yeah. We want to make sure that there's no biases. No. Yeah. So, it's got to be unbiased. It's got to be ethical. It's got to be accurate. That's really what all of this comes down to. And then there's many different risks that could potentially contribute to that. But that's really what we're trying to avoid that at in the end of this the recommendations that come out of any AI based system are responsible, accurate, unbiased, ethical. Okay. So I
know earlier you were you started talking about like NIST and things like that, but what are the key components of an AI governance framework? Really what it comes down to and you know we'll use NIST as a good example. The govern function in the NIST AI risk management framework basically it wants to outline the policies the processes the procedures and the practices across an organization top to bottom you know from the leaders to the doers kind of thing. And really what we're trying to do is we want to map and measure and manage the risks. In other words, risks are inherent. It's sort of like we need to start there. You're never going to eliminate
risk. There's never going to be a risk proof or a zerorisk system. There's always risks. Risks have to be properly managed. How we manage risk is not unique to AI. It's like you can mitigate the risk, you can accept the risk, you can transfer the risk, you know, that type of thing. That's effective risk management. The key here is that you're doing all that risk management not off the cuff, not in any kind of ad hoc manner, inconsistent manner, but you're doing it consistently according to the policies in the practices and the procedures that you've laid out. So if we look at for instance govern in the NIST AI riskmanagement framework, you know, it starts
with the legal and regulatory environment in which you operate. That's the govern 1.1 area. All right? And so what that's talking about is there's a difference between standards and legal requirements, right? So in the EU there's the EU AI act. That's a law. This is not voluntary like I really think we ought to try to be ISO 420001 compliant. That's not law. That's a choice, right? Same thing. Or I really think it would be smart if we were compliant to the NIST AI risk management framework. You have to understand that's sort of standards, but then there's the laws. There's the EU law. In the United States, we already have AI laws either already, you know,
ratified or in the process from California, Colorado, Illinois, Maryland, New York, Virginia, probably others. What's interesting is there's also a law at the federal level saying we need a 10-year moratorum on states being able to pass their own AI laws. So, you know, I'm not going to weigh in on which is which or which is better or preferable or whatever. But governance starts with you better understand the legal environment in which you're operating. Speaking of that, who is responsible for all of these things like at an organization when it comes to AI governance? It's a really good question and it sort of goes back to something that we said earlier on with you know what's
the formula the secret formula to doing anything in GRC successfully and that's that it's a total team effort. every layer of the organization knows what their role is, knows what their responsibilities are, and they're managing and monitoring that and being able to produce evidence. They're following procedures and the managing and monitoring comes in and that if they're not, those exceptions pop up. So, in other words, the bottom line to governance is things don't go undetected. In other words, if anywhere along the line something is not compliant with a policy, a process, a procedure or control or whatever, it doesn't go unnoticed. Alarm goes off, a warning email comes out, something like that. That's the key
to governance is nothing goes unnoticed. So, everybody's responsible. That's really what it comes down to. Absolutely. Yeah. And no one person is responsible top to bottom, but everybody has a role to play, right? Okay. So, I feel like you were going over this briefly before, but how can companies develop ethical guidelines for AI use? Fortunately, both ISO 420001 and the NIST AI risk management framework have a very strong perspective on ethical development and ethical use. And so it's in some ways it's like any other software development life cycle, right? And so it starts at the very beginning with ethical AI development. So right from the beginning, it has to be designed with respect for human
rights and privacy and dignity. So that's that kind of ethical use, unbiased, truthful, etc. kind of thing. No ulterior motives, I guess, is one one way to say it. You know, no purposefully introduced biases, that type of thing. And then the other key part of that is and this sometimes gets missed. I think this is maybe the wrongly placed trust that some people are putting in AI in that transparency and explainability is a key desirable characteristic of AI systems that both ISO 420001 and NIST AI risk management framework highlight. And if you stop and think about that, if you and I were having a conversation and you said something to me and it was obviously
that you felt like this was a fact, it was absolute truth, we would probably have a conversation that says, "Well, why'd you reach that conclusion? Where, you know, where'd you get the the data to come to that recommendation or that opinion or whatever?" Well, we need to be able to do the same thing with AI systems. It's like you're telling me here's your answer to this query. Well, show me how you reach that conclusion. I don't know how many AI systems are equipped to do that, but that's exactly that transparency and explanability characteristic is absolutely key to us being able to use AI in the future in an effective and fair manner and trustworthy manner.
for sure. And then you know beyond that it's the accountability. So we have to have clear lines of responsibility. There's no pass in the bot. This goes back to what I said before the clear policies practices and from our other talks and compliance. Anytime you have a control, there's a name next to it, right? Here's the person responsible for this control. Here's the person responsible for that control. And it's never like a department or a role. You know what I mean? It's unambiguous. It's a person's name. That's the accountability part. And then, you know, we talk a lot about fairness and non-discrimination. We have to guard against any inherent or purposeful biases. You know, that
got introduced. Again, where's that start? It starts with the data. Like what if data was only from one part of the world and then someone wants to ask a question and make a recommendation and the system extrapolates it and says well across the world here's what's true and it's like well that was based on data from only this small part of the world so how can that be crude you know what I mean that kind of thing and then finally it's risk management it's like and again you know this because we've talked about risk management so much risk is inherent right we can't eliminate risks. What we have to do is if there's no excuse
for not knowing what your risks are. That's where why risk assessment is so important. So you have to assess the risks and risk assessment on a AI system is a little bit different. It's there's some similarities but it's a little bit different than like we do an you know an ISO 27001 risk assessment or a SOC 2 risk assessment or something like that. And so, you know, it's different, but it's the same in that the risk, effective risk management always starts with a thorough and objective risk assessment. So, how can an organization ensure accountability and like AI decision making? We've talked a lot about GRC, you know, in these podcasts and GRC platforms, etc. And
so in some ways what we want to assure people is just because it says AI in front of it doesn't mean you have to forget everything you knew about security governance or security compliance etc. A lot of those same practices still apply. It's just like instead of looking over there now we're looking over here. So those same kind of things where you know well-defined policies and processes and procedures are put in place control owners are identified and then really you need that platform that provides that managing and monitoring and evidence gathering and doing the internal audits and so forth and so on. And for some companies it's going to make sense uh probably largely driven
by their customer demands that they get certified. So you know they get an external audit against the ISO 420001 standard and even the interesting thing Jarrett is as new as that standard is we're seeing quite a big uptake in companies large and small you know going through that audit process and getting ISO 420001 certified I think it's going to be a big deal. Yeah, for sure. Especially AI is everywhere now. Even when I go to some fast food places, there's like a little robot that's picking my order now and not a person. So, totally makes sense. Well, it's probably a pendulum swing. Hopefully, it swings back a little bit. But, as you know, we have
customers now coming to us saying that their customer because they're putting the big push on their publicity. Hey, we're AI based. We're introducing all this new AI stuff. our customers are saying, "Hey, our customers or potential customers are coming to us asking about AI risk management." So, there's a growing awareness. It's kind of like a timeout. And again, that's a good thing. That's that governance thing because somewhere high in that organization, that potential customer organization said, "Don't be just adopting AI systems willy-nilly. We need to know is this system that you're you know going to start basing our business decisions on reliable unbiased etc. So let's go ahead and dive into implementation a little bit.
What are some common challenges in implementing AI governance? It's almost like where we kind of started the conversation. I think the biggest challenge that isn't spoken about enough is the lack of focus on the importance of data management and data quality. And it's almost like the chief data officer role is a fairly new role. But you know in my experience it's like well where's the chief data officer report to you know is it a business function? Is it a security function? Is it a compliance function? Who do they report to? If you're a multi-billion dollar enterprise and every little department in every acquisition has owned their own data for the past 15 or 20 years,
how do you corral all that and tell people, well, we got to get rid of all this redundant data and we have to have a single data catalog and it's it's a big deal and it costs a lot of money. And people say, well, what's the benefit? And you know, there's nothing wrong with our data. Everything seems to be working fine. If it ain't broke, don't fix it. Well, it actually is very broken, especially when you try to sit an AI system on top of a data layer that has never been cataloged, managed, data redundancy hasn't been taken care of, or data discrepancy hasn't been taken care of, that kind of thing. So, there's a
saying in the Bible about a house built on sand versus a house built on stone. If you don't take care of the data, you're building the house on sand. Yeah. I don't want that. Especially with the rain we've had in Indiana lately. For sure. Oh my goodness. Again, I know with AI, there are some people that love it and there's some people that are like side eyeing it a little bit. So, how can AI governance help address societal concerns about AI? Yeah, it's very interesting. And early this morning, I was looking and I found a survey from the Pew Research Center. You know, it's pretty reliable. They've been doing kind of public sentiment on a
variety of topics. And I found this pretty interesting. It did US adults in generals. The question was what does the public have to say about will AI have a positive effect on the United States over the next 20 years or not? So the general population of US adults 35% negative 17% positive AI experts and it has a definition of who those are only 15% negative and 56% positive. It's almost like you have two views of the world that don't seem to align. So it's like where is the truth lie? It's probably where does the truth lie? I don't know. That's the question. Where does the truth lie? So if you think about it in any
new innovation right or issue or whatever how do people feel comfortable right I think it starts with transparency and interestingly enough transparency and explainability we just talked about it that is one of those key traits key desirable outcomes or sometimes they're called objectives that both ISO 420001 and the NIST AI risk management framework put out there as a pri priority as a priority outcome of any system. And so I think if companies producing AI systems want to lower the anxiety then they need to kind of go full gas. You know sort of sometimes we call it like an overcorrection or something like that. You know what I mean? Overcompensate on the transparency and explainability. For
sure. Yeah, that totally makes sense. And yeah, I'm I'm like skeptical of it sometimes as well, but there are also some tools that like help me get through like the day or make tasks a little easier for me. So yeah, I usually use it as a starting point. Jar, for sure. So I'll ask a question, I'll get an answer, I'll say, "Okay." But then I'll dig after that. So, you know, because the transparency and explanability isn't in that answer, I'll go look for that transparency and explainability myself afterwards. Yeah, for sure. All right, Jim, we covered a lot of information here, but is there anything that we missed or do you have any other tips
for our listeners before we wrap the episode? Well, not to end on a negative note, but like any new technology that gets introduced, the criminal element of our society, and by society, I mean global society, not US society, is always going to do their best to take advantage of that new technology, take advantage of the fact that it is new, people don't understand it, etc., etc., you know, to use it for their own nefarious purposes. It's just human nature. We are not going to be able to stop that. But that's where the challenge in my opinion is, you know, having been in the FBI and so forth, is for law enforcement to quickly get up
to speed on, you know, in this case it's AI. You know, 20 years from now, it'll be something different, right? 20 years ago it was something different. But law enforcement has to, and I think you could say corporate leadership has to get up to speed on both the positive and potential negative impacts of this newest technology, which in this case happens to be AI. We cannot be naive about that fact. At the same time, we don't want to slam the door on it. For example, with the amount of data that is available in the health care field, medical diagnosis, etc. And you think about how there's a shortage of doctors in some in many local,
not just in the United States, but around the world. Think about the impact that a well-designed, well-built, fair, unbiased, ethical AI system could have if it was able to look at all the medical information that's available across every health care provider in the entire world and provide that to somebody, you know, in a remote part of the United States or the world. Just saying could be great, right? So, what I'm trying to say is we shouldn't clamp down on this and try to regulate it or stop it. There's no stopping it, but you know what I mean. We shouldn't stifle the creativity and the ingenuity because there are criminals out there that might use it
for criminal purposes. It could be a fantastic thing. It could make a huge positive difference in society. Absolutely. And I don't think that we're ending it on a negative note. I just think we're ending it on food for thought. Yeah. Yeah. Exactly. Well, we're all about trying to raise awareness and trying to raise consciousness. That's why we do these podcasts, right? And so what we're trying to say is don't freak out, but you know, ask good questions. Exactly. All right, Jim. I appreciate your time and expertise and thanks for joining me on another episode of the tea on cyber security. As always, Jarro, it was my pleasure. Thank you very much. Now that we've spilled
the tea on AI governance, it's time to go over the receipts. Jim covered a lot of information here, but there were a handful of things that really stuck out to me. The first being what is AI governance and AI governance alike security governance is a shared responsibility across all layers of an organization's leadership and I feel like my next receipt also does a pretty good job at explaining that which is what's the difference between AI compliance and AI governance and I really think that Jim did a great job putting this in simple terms for me to understand. So if we want to think about it as AI governance is like setting a law or passing
a law and then AI compliance is really that law enforcement and those people that have to be in the dayto-day activities of that similar to like a police officer. The next receipt I have is why AI governance is necessary. As we all know AI provides many benefits but with those benefits are also a lot of risks. Those risks are there for the people that create the AI system, but also the people that use the AI systems. And some of those risks include poor data, biased information, inadequate data, and other things that could potentially make the AI system untrustworthy. And the final receipt I have for this episode can probably sum up a lot of the
episodes lately. Like cyber security and compliance and anything else, it all starts top down. So leadership really needs to be the ones that are driving this effort. However, it is a team effort to make sure that our policies and things like that are in place. So, we all need to do our share in making sure not only our personal data, but our employers data is safe as well. Thanks for tuning in to another episode of the tea on cyber security. And that's the tea on cyber security. If you like what you listen to, please leave a review. If you need anything else from me, head on over to Trava Security. Follow wherever you get your
podcasts.