Cybertech Texas Mission · Mar 14 to 21, 2027 · Austin, San Antonio, Houston. Join the waitlist
Back to blog
By Thiago Vieira 11 min read

AI Explainability Compliance for Startups Selling in the US and EU

Founder explaining a low-ranked AI security alert

AI explainability compliance comes down to one thing: when someone asks why your system produced an output, you can show an answer and prove it with records. No single law demands one technique. In the EU, specific transparency duties already apply. In the US, a patchwork of agency expectations, state laws and buyer questionnaires does the same work. For a founder, the winning move is to build evidence into the product from the start. We built our program around that idea: proof that holds up with US buyers and in court.

TL;DR

  • Explainability, interpretability and transparency are three different things in regulation. Explaining a model does not by itself satisfy a transparency duty.
  • The EU AI Act’s Article 50 transparency obligations apply from 2 August 2026: tell people when they deal with AI, and mark AI-generated content in machine-readable form.
  • The US has no single federal AI law. Expect FTC enforcement, voluntary-but-expected NIST AI RMF alignment, state laws and buyer due diligence.
  • No technique (LIME, SHAP, heatmaps) counts as compliance on its own. Documented design choices, logs and impact assessments are what regulators and buyers read.
  • Founders who log decisions and keep tamper-evident records answer almost every explainability question with the same evidence.

Table of Contents

Three words regulators use differently

Most confusion starts with vocabulary. The European Data Protection Supervisor’s November 2025 guidance for EU institutions separates three ideas:

  • Interpretability: how far a human can understand a model or its decisions.
  • Explainability: clear explanations of specific predictions, including after-the-fact techniques such as LIME heatmaps.
  • Transparency: the duty to give people meaningful information about the system and how it affects them.

The same guidance states that interpretability and explainability alone do not meet the transparency threshold. A beautiful SHAP chart in your data science notebook does not tell a loan applicant they were scored by a model, and it does not tell a user the chatbot is not a person.

Definitions also differ by authority. A 2025 Financial Stability Institute paper from the Bank for International Settlements notes that US regulatory agencies define explainability as how an AI approach uses inputs to produce outputs, while the OECD AI Principles take a customer-centric view: enabling affected people to understand how an outcome was reached. Build for both. Your engineers need the input-to-output trace, and your customers need a plain statement the affected person can follow.

What the EU requires: Article 50 and the high-risk rules

If you sell into Europe, or a US customer deploys your product there, Article 50 of the AI Act is the transparency rule to know. According to the EU AI Act Service Desk, it covers:

  • AI systems that interact directly with people, who must be told they are dealing with AI.
  • Machine-readable marking of AI-generated or manipulated content.
  • Exposure to emotion-recognition or biometric-categorisation systems, where deployers must inform the people exposed.
  • Deepfakes, which must be disclosed as artificially generated.
  • AI-generated text published to inform the public on matters of public interest, unless a human reviewed it under editorial control.

Information has to be clear and distinguishable, at the latest at the first interaction or exposure. The European Commission’s guidelines on transparency obligations say these duties apply from 2 August 2026. The same guidelines add that providers of generative AI who do not follow the Code of Practice on Transparency of AI-generated Content must show compliance with marking and labelling through other, equally adequate means. For duties such as telling users they are talking to AI, providers choose adequate measures in light of the guidelines. Enforcement sits with national market surveillance authorities, the AI Office, and the EDPS for EU institutions.

For a practical walk-through of what these duties mean when you build a SaaS product, this overview of AI transparency requirements under the EU AI Act is a good companion. Our own guide, EU AI Act for startups, covers what you must do and when.

Where explainability techniques fit

The Act does not mandate a specific explainability method. The Practical AI Act Guide notes that techniques such as feature attribution map onto Articles 13, 14 and 86(3): they help humans interpret outputs, support oversight, and back the right to an explanation of individual decision-making. They aid compliance, but they are a bridge, not the destination.

If you want a taxonomy to organize your methods, ISO/IEC TS 6254:2025 describes approaches and methods for meeting different stakeholders’ explainability objectives for ML models and AI systems, across the system life cycle. Use it to match the method to the audience: an auditor, a clinician and an end user need different explanations of the same output.

What the US expects instead

There is no single comprehensive federal AI law. Practitioners describe US obligations as running through agency guidance, executive orders, sector rules, consumer-protection and privacy laws, and state legislation. The same guide notes that the NIST AI Risk Management Framework is voluntary but widely used, and that the FTC holds companies accountable for deceptive AI claims under its existing authority.

In practice that gives you four pressures:

  1. FTC exposure. If you say your system is explainable, unbiased or human-reviewed, you must be able to back it up. Disclosures should be easy to see and sit next to the AI content they relate to.
  2. State laws. Several states have passed AI transparency or high-risk AI statutes, with effective dates and legal challenges that keep shifting. Check each state where your customers operate before you commit to a launch plan.
  3. Sector rules. Lending, hiring, insurance and healthcare buyers already carry explanation duties for individual decisions, and they push those duties down to vendors.
  4. Procurement. Enterprise security and legal teams send questionnaires. Federal contractors and their suppliers increasingly align with NIST’s framework. Our guide to NIST AI RMF for startups shows how to map your product to it.

For most early-stage founders, the buyer’s questionnaire is the first real test, long before a regulator calls.

What the UK adds

If you also sell in the UK, the Information Commissioner’s Office has the most concrete guidance on explaining AI-assisted decisions. Its four principles are: be transparent, be accountable, consider context, and reflect on impact. It expects a designated human contact where people can query or contest a decision, and it expects you to justify and evidence your design choices. That last point is the key to the whole subject: the regulator wants proof that you considered explainability, not just an explanation.

Who asks what: a quick map

Who is asking What they usually want Evidence that answers it
End user or affected person To know AI is involved and why an outcome happened Plain-language notice, per-decision reason, contact for appeals
Enterprise buyer How the model behaves, what data it used, how you monitor it Model documentation, impact assessment, monitoring records
Regulator That you considered risk and can reproduce a decision Technical documentation, logs, design rationale
Court or opposing counsel What the system did on a specific date for a specific person Tamper-evident logs, preserved inputs and outputs, chain of custody

The bottom row is the one most startups forget. A decision your system made two years ago may need to be reconstructed in a dispute. An explanation produced today, from a model version that has since changed, is not the same as a record made at the time.

Building the evidence trail

Across frameworks, the pattern is the same: one principle turns into a policy, a method, a control and a record. A 2026 arXiv study reconciling ISO/IEC 42001, the NIST AI RMF and the EU AI Act shows how protection against discriminatory outcomes descends into a fairness policy, a bias-measurement method, a control obligation and a documented bias evaluation, with the model card and impact-assessment record forming the audit trail. It proposes a shared control library so one body of evidence supports several frameworks.

For a startup, that means building the following once and reusing it:

  1. Decision logs. For every consequential output, record the model version, inputs, output, confidence or score, any human override, and a timestamp. Our guide to AI audit trails covers what to log and how to prove it.
  2. Impact assessments. ISO/IEC 42005:2025 gives guidance on running and documenting AI impact assessments so affected parties can be assured that impacts were considered.
  3. Explanation methods mapped to audiences. Keep a short table: which technique explains which output to which reader, and its known limits.
  4. Disclosure copy. Store the exact wording and placement of every AI notice, with version history, so you can show what a user saw and when.
  5. Content provenance. If your product generates or edits media, mark it in machine-readable form, and make sure detection and authentication hold up. Our deepfake evidence guide goes deeper.
  6. Human oversight records. Note who can override the system, how, and how often they did.

A worked example

Imagine a startup selling an AI triage tool that ranks security alerts for a US hospital network, which also serves patients in Germany. A buyer asks: why was alert 4471 ranked low last March?

With evidence built in, the answer takes minutes. The log shows the model version, the input fields, the score, the top contributing features from the explanation method, and the analyst who reviewed and closed it. The disclosure copy on file shows the notice users saw. The impact assessment shows that this class of error was considered and monitored.

Analyst reviewing a low-ranked hospital security alert

Without it, the team retrains the model, loses the old version, and offers a chart generated from today’s weights. That chart explains nothing about last March, and the deal stalls in legal review.

Common mistakes

  • Treating a technique as compliance. A heatmap helps a human interpret an output. It is not a disclosure, a log or a risk assessment.
  • Explaining after the fact only. Reconstructed explanations can differ from what the system actually did. Capture the trace at decision time.
  • Burying notices. Disclosures placed in terms of service or footnotes do not meet the expectation of clear notice near the AI content.
  • Planning for one jurisdiction. A US founder with one European user falls under EU duties for that user. A European founder with one US enterprise buyer meets US procurement questions. Build one evidence base and map it to each regime.
  • Assuming the rule will stay put. Effective dates for several rules have moved, and some US state laws are in litigation. Re-check the official page for each rule before every major release.

Where trade-offs are acknowledged

Regulators do not demand that every model be fully transparent. The BIS paper observes that few financial authorities have issued AI-specific explainability guidance, that existing model-risk rules mostly embed explainability within governance, documentation, validation and independent review, and that authorities may accept trade-offs between explainability and performance so long as risks are properly assessed and managed. The lesson for founders: the assessment and the documentation carry the weight. A powerful model with a well-evidenced risk process is easier to defend than a simple model with nothing on file.

What we tell founders in the program

Most founders we meet treat explainability as a feature to add before a sales call. Buyers and courts treat it as a record. The question that decides the deal is rarely “can your model explain itself?” It is “can you show what the system did, on that date, for that person, and prove nobody altered the record?”

That is why our program is built around evidence-grade proof. Founders work with exited founders, CISOs, MSSPs and a court-appointed digital forensics expert to build proof into their products that holds up with US buyers and in court. For a solo or early-team founder, that is a lot of decisions to carry alone, and these are the ones we help make: first hires, pricing, US entry, and how your product’s evidence trail is designed.

If you are choosing between spending your next sprint on another explanation widget or on tamper-evident logging, choose the logging. The widget explains one output. The log defends all of them.

 

Thiago Vieira

FAQ

Is explainability legally required for AI systems?

Not as a single universal rule. The EU AI Act does not mandate a specific explainability method, though techniques help meet its transparency, oversight and right-to-explanation provisions. In the US, expectations come from the FTC, sector rules, state laws and buyer requirements rather than one federal statute.

What is the difference between AI transparency and explainability?

Transparency is the duty to give people meaningful information, such as telling them they are interacting with AI. Explainability is about making a specific output understandable. The EDPS guidance states that explainability alone does not satisfy transparency, so you need both.

Which AI transparency obligations apply from August 2026?

The EU’s Article 50 duties apply from 2 August 2026. They cover informing people when they interact with AI, machine-readable marking of AI-generated content, and disclosure for deepfakes, emotion recognition and biometric categorisation. Check the Commission’s page for the current guidance before you ship.

Does a technique like SHAP or LIME make me compliant?

No. Such techniques can support human oversight and individual explanations, but regulators and buyers look for documented design choices, impact assessments, logs and clear notices. ISO/IEC TS 6254 offers a taxonomy for choosing methods by audience.

Sources

About the author

Thiago Vieira

Founder of Cybertech Acceleration Inc and digital forensics expert, leading the forensic component of every startup in the program.

Author website →

Related articles

US soft landing by country