Transcript of https://www.youtube.com/watch?v=wb1eDXGEfW4, published 2025-06-17. Auto-generated captions, lightly corrected; the video is the canonical version.

Don’t Overtrust the Robots: The Real Tea on AI Compliance

Do not overrust AI models. Think about all the different aspects that you mentioned, the transparency, the bias, the fairness, the security, the data privacy regulation acts. Because if business organizations fail to think about and address all of these aspects before plugging in these AI models into their production environments, they could be getting themselves into a lot of legal and ethical issues. So, it's important to be aware of these things. And the good news is that help is available. Gather around as we spill the tea on cyber security. We're talking about the topic in a way that everyone can understand. I'm your host, Jara Rowe, giving you just what you need. This is the tea on

cyber security, a podcast from Trava. Hey everyone and welcome back to the tea on cyber security. I'm sure in all of our conversations nowadays, AI is always mentioned. And that's exactly what we will be talking about on this episode, specifically AI compliance. So, we'll be diving into what it is, why it matters, and how businesses can stay ahead. AI is truly changing everything, and with great power comes great responsibility. So, we need to make sure that we all stay on the right side of the rules. So let's go ahead and dive into it with our guest Marwan. Hi Marwan. Hello. So can you go ahead and introduce yourself to our listeners? Absolutely. Thank you Jara

for the opportunity. I'm Marwan Omar. I'm the chief AI officer for Insight Assurance where we help companies with AI strategy, governance and risk. And we also do AI red team. So anything related to compliance and security aspects of AI, we can do that. I have a PhD in AI and I'm super excited to be here today. All right. So let's go ahead and just dive straight into it. In simple terms, what is AI compliance? Absolutely. So in simple terms, AI compliance basically means we are follow we're going to follow laws and regulations and also ethical standards and guidelines to ensure that AI systems or AI models are being used in a safe, fair and transparent

manner. This may involve uh meeting the expectations of the legal, ethical and also security standards so that we can prevent harm or bias or misuse from using AI systems and technologies. Yeah, we don't we don't want any harm coming our way, right? Definitely not. Yep. So why is AI compliance important for businesses and organizations? Absolutely. AI is very critical for business organizations of all kinds and of all kinds of industries. First of all, they need to make sure that they avoiding legal risk. For example, somebody could be fined. Somebody could face some companies or business organizations, they could face lawsuits or penalties for non-compliance with AI regulations and and rules as you mentioned. And we

have to ensure fairness and trust so that we can reduce the AI bias and discrimination. Imagine there is an AI system that is in charge of approving mortgage applications. And if this AI system is not fair, is not trustworthy and not reliable and makes the wrong decisions on behalf of mortgage officers, then companies could face legal issues. And uh also we have to protect consumer privacy when we handle the data. As you know these AI systems we interact with them on a regular daily basis and they could be collecting and handling our personal private data as they call it PII personally identifiable information. So this is why it's also important and finally maybe just the

business reputation because again AI decisions have to be ethical and transparent simply because these AI systems in many cases they could be making decisions on behalf of people whether it's again finance industry, mortgage industry, even healthcare industry. So for all of these reasons it's important for companies and business organizations to stay in compliance with AI. For sure. from me being like a novice when it comes to AI before I knew very much about it. I didn't know that anytime I would like throw in information that it was just really collecting it. And so that's one thing I want other people to be aware of as well that the more you give it information, it collects

it, it learns and it grows. So don't throw out your the PII like you mentioned and things like that. Right. And just to your point Jer whenever we are using an AI system whether it's chat GPT or any other AI system or chatbot especially if you are signing up with them as a piece of advice we need to uh read their policies right I know the policies are usually in fine print and nobody wants to read all of these pages but in their policy they will tell you how they store your data how they handle your data whether or not they will use your data for training their system so always be mindful of that

and do not overrust these AI systems. Just because it's an AI system, we should not overrust them and just give them our PII. Absolutely. So you were just mentioning things about like different industries and laws. So what are the key laws and regulations currently governing AI use? So there are some major AI regulations and guidelines even though this field is evolving and more regulations and you might even see more laws and acts coming up especially from the US since we are here in the US. One notable one which is outside of the US interestingly it's the EU AI act. So EU for the Europe. So EU AI act this is the European act for AI

and it's actually the very first this is considered the very first comprehensive AI law in the world believe it or not and this classifies AI risks and sets the strict compliance rules. We know that many other people might know this as well. Europe is way more stricter when it comes to privacy regulations, customer data, customer privacy. And so this is not a surprise that Europe is the first one to come up with an AI act. It's called EU AI act. There's also another act which has been around for a while but also relevant here and and current to this conversation is the GDPR. GDPR stands for General Data Protection Regulation Act. again came out of

Europe and it regulates AI data processing. We just talked about AI data processing. These AI systems chat GPT how do they handle your data? Do they use it for retraining their models? Do they encrypt your data when they communicate it? And there's also the US AI executive order and NIST National Institute of Standard and Technology. So there's US AI executive order and AI risk framework. So this is a framework for managing AI risk specifically and we also have China's AI regulation which is less relevant here because that's a different part of the world. Fantastic. You mentioned something that I actually was not even aware of. So definitely learning a lot so far. So, how does

AI compliance differ from traditional technology compliance or is there even a difference? Great question. Yes. So, a traditional IT compliance focuses on security, how to make our systems more secure, data protection, how to protect customer data, business data, business plans and software, you know, regulations as you mentioned like GDPR regulation. We also have the HIPPA. So HIPPA is mainly for the healthcare industry. It's a law from the US congress that basically requires companies to protect our customer data. As you know, if we are patients and you go to our healthcare provider, they have to abide by HIPPA. HIPPA is a big deal here in the US. So that's traditional IT compliance and auditing. On the

other side, for AI compliance, AI compliance goes one step further because they have more things to address. Now we talked about bias. If an AI system makes a decision and denies my mortgage application, the AI system could be biased. Maybe it's not fair. Maybe it was just trained on limited number of data. So in this case, AI bias is a big aspect of AI compliance. So this differs completely from the traditional IT compliance. As we mentioned again, fairness, the AI system or the model has to be fair. Imagine an applicant tracking system. If I apply for a job, most of the companies these days, they use AI to pick the best candidate. But how do

we know that this AI system is fair? Maybe it just picks male rather than female because it's maybe to strain on more male than female or people with certain skill sets. This was a case that actually happened with Amazon a few years ago. Amazon designed an AI model for applicant tracking to to recruit applicants. And it turns out that it was very unfair because it was trained on more males than female. So it was picking more male actually for the interviews than female and they figured this out. So this is a huge aspect of AI compliance. It talks about fairness. So bias and fairness are very very important aspects when it comes to AI compliance.

And of course the ethical risk. Again, automated decision making is a big deal these days as we mentioned. You know, I could go to a doctor and the doctor might just use an AI system to tell me whether or not I have pneumonia. So relying on an automated decision making process is important. So it's very important to understand the ethical aspects, the fairness and the bias aspects of AI models and to make sure that you are in compliance with them. Yeah, definitely want everything to be fair, right? That totally makes sense. All right, so what are the basic steps to start implementing AI compliance in an organization? So the first step is understanding the laws

and regulations that are relevant and current to our industry. We already mentioned the EU AI act. We already mentioned the GDPR and the US executive order and the NS. So these are some of the regulations and laws to be aware of. Number two, I would say conduct an AI risk assessment. Understand what is the risk on AI system because these AI system as I mentioned many companies they just overrust them. Just because there's an AI model out there for free on hugging face or any other resource, it doesn't mean that we should trust them. We should actually do our own diligence and test stress test these AI systems to make sure they are transparent, they

are robust, they are fair, they're not biased. Remember the Amazon applicant tracking system? It turned out that it was not fair. It was biased towards male. So if Amazon had done their due diligence and you think that a company like Amazon would do their due diligence, unfortunately sometimes companies they just go with the market pressure. They don't take enough time to stress test their systems and create a framework for AI compliance. And then also maybe equally important, train AI employees. We just mentioned that there are a few things that we just learned in this conversation. Many employees may not be aware of fairness issues, bias, transparency, ethical aspects of AI, automated decision- making. So, it's

important to train our employees on AI ethics and the requirements for uh AI compliance. So, one thing I've learned through hosting this podcast is that cyber security is like a team effort. But when it comes to AI compliance specifically, what roles or department should be involved in these AI compliance efforts? So AI compliance is a crossf functional effort. That means there is more than one department that needs to be involved. We just mentioned you know the AI regulations. So that means the legal and compliance team needs to be involved to make sure that they are adhering to the laws because if we break laws then we could go out of business and that will hurt

the reputation. So we need the legal team. We need data scientists and AI engineers because without these people we will not be able to develop an AI model that is responsible, that is ethical, that is not biased and that is fair and transparent. And you just mentioned cyber security. These are people that we definitely need. I'm one of them to ensure again data protection to ensure that our models are robust against attacks and you also need the HR and the ethics officers those people again along with the legal department they need to monitor the model to make sure it's not biased you know the decision that's making that they are fair and transparent and explainable

and of course the executive leadership team upper management definitely needs to be involved in all of this because without support and buy in from upper management, none of this will be possible. We've talked a couple times on the podcast as well. It's like leadership has to buy in first before anyone else under them. So that also makes sense. So how can companies ensure their AI systems are transparent and explainable? Excellent question. So the good news is that there are ways for us to ensure the transparency, the ethical aspects and responsible aspects of AI systems. So there's a field called explainable AI. This field involves a lot of techniques that we can use. One of them

is called SHAP and I know this because I'm in academia and I use this myself and I teach that to my students as well. This is a framework that we can use to understand the explanability of AI systems. If an AI system tells a patient that they have pneumonia, how did this AI model make that decision based on what? Is there a justification for this or is it just a model that is just maybe hallucinating and just telling us okay this person has a disease or this person should be denied for this loan application or the stock market is going to go up tomorrow. So all of these aspects should have explanability. So there's an

entire field in AI called AI explanability or explainable AI for short we say XAI transparent AI. So this is one aspect. We can also look at audit logs. So if an AI system denied an applicant or denied a loan uh application for an applicant, a person, we need to go back to the logs again. How did this AI model make this decision? Maybe it was the wrong decision. So we need to go back and look at the log files and we have to have again userfriendly explanations again for end users so that they can understand why these decisions are being made. Sometimes we also need to conduct third-party assessments independent assessments or audits. This is

where a company like Insight Assurance can come in and they can independently with their AI experts audit the AI systems to make sure that they meet all of these standards. Fantastic. So I do know implementing things like that typically have challenges. So what are common AI compliance challenges that organizations face? First of all, the regulations. Remember talk about regulations and laws. EU AI act, the US AI act, the executive order. Unfortunately, some companies may not be aware of these laws and regulation and this is where a problem might arise. So if a company does not have a clear understanding of the regulation landscape, the legal landscape they don't know about the bias, fairness, explanability issues

of AI that we talked about. If they don't understand data privacy risks, right? Where is the data coming from? If I have an AI model like Chad GPT for example, I need to train it on data sets, millions of data points, where do I get that data from? Do I get it from open source from out in the internet? If I do, then how do I ensure that that data is actually secure? Because just because it's on hugging face on some other platform, it doesn't mean that it is safe and secure. This is where these issues might come from. So companies need to be aware of the legal landscape, the bias in AI models and

the data privacy issues. Great. So you're just mentioning data privacy. So how does data privacy factor into AI compliance, right? So AI must comply with the rules and regulations and laws that are related to data privacy. We just mentioned GDPR, general data protection regulation from Europe. And then in California, there's a US version of that. It's called CCPA. CCPA, which stands for California Compliance Protection Act. And of course, we have HIPPA. So, AI systems must comply with these regulations to ensure that data is handled in a manner that is secure and responsible. Remember, we mentioned a company like OpenAI that is a creator of Chad GPT. They need to have policies and these policies need

to be communicated with users and consumers telling them how do they handle my data again if I type in my personal data or PII with chat GPT maybe looking up things or processing something do they encrypt my data do they send it to a third party do they send it to the cloud or does it stay locally here on my system so this is why it's important for AI to comply with those regulations again GDPR CCPA and HIPPA And all AI models should be secure against you know data leaks and also adversarial attacks. Adversal attacks where somebody can come in and do prompt engineering and see what the model can reveal. So AI systems should

be secure against those attacks. Definitely. So how can organizations stay updated on like these evolving AI regulations? This is a very dynamic field. It's evolving rapidly. So organizations can follow government agencies. We just mentioned the US executive order and the Nest framework. So Nest is part of the government here in the US. So companies can follow what Nest has as a regulation body. We also have the EU Commission for example that's another body in the U in the European Union. And they can also companies can also engage with AI ethics organizations. For example, they can partnership on AI. There is also ILE E AI standards. So ILE E is an engineering organization that has their

own AI standards. They can participate in AI compliance conferences in workshops. They can go through training and they can also subscribe to any legal or tech policy newsletters for AI regulations and updates. Fantastic. Those are great resources for us all to dive into a little more. So, Marwan, we definitely covered a lot of information about AI compliance. But before we wrap the episode, do you have anything else you would like to mention or drive home to our listeners? Thank you, Jar. I really enjoyed this podcast conversation, one thing I would say maybe just again, do not overrust AI models. If you are going to use that AI model either as a company or as an

individual, think about all the different aspects that you mentioned, the transparency, the bias, the fairness, the security, the data privacy regulation acts. All of these things are important aspects especially for business organizations because if business organizations fail to think about and address all of these aspects before plugging in these AI models into their production environments they could be getting themselves into a lot of legal and ethical issues not to mention the reputation. So it's important to be aware of these things and the good news is that help is available and inside assurance is one company that can help with AI security and implementation and thank you Jera. Yeah, thank you. I appreciate your time and

expertise. All right, it was great having this conversation with you. Have a wonderful day. [Music] Now that we've spilled the tea on AI compliance, it's time to go over the receipts. Obviously, the tea on cyber security is full of experts, but Marwan was so incredibly knowledgeable on AI. It was a great conversation for me. So, let's go ahead and dive into receipt number one. Marwan mentioned several times during the episode that we should not overrust any AI models. It's important that we know and understand that AI should be transparent, but there are times where it may exhibit bias. So, we should not overrust the information that it is giving us. So, I asked about implementation

of AI compliance. And when it comes to implementing AI compliance, it's important to first understand your industry's regulations and then second, you should perform an AI risk assessment. And a part of that, Marwan stressed how important it is to train our employees on using or even creating AI models to ensure that it's transparent and unbiased. And the final receipt I have for this episode is about who should be involved in AI compliance. And Marwan emphasized that it is crossfunctional, which we've talked about several times that cyber security in general is a team effort. So when it comes to implementing AI compliance, it's important to have the legal and compliance team involved, leadership, of course, the

data scientist and AI engineers, as well as the cyber security team. Again, Marwan covered so much information during this episode, so you might want to go back and listen to it again. But that wraps another episode on the tea on cyber security. And that's the tea on cyber security. If you like what you listen to, please leave a review. If you need anything else from me, head on over to Trava Security. Follow wherever you get your podcasts.