Grading Responsible AI Answers Against Public Source Texts — AIB-C01 Governance Practice With and Without Excerpts

An AI can give the right answer and still cite the wrong reason. To study the AIB-C01 governance domain, I wrote 38 cases around a build-monitoring AI at a fictional small 3D printing workshop and had claude -p answer them twice: once without excerpts from the public source texts and once with them. In this run it got 110 of 114 answers right without the excerpts and 114 of 114 with them. On the numbers alone, the gap is small. But when I read the four cases E10 to E13, which hinge on the July 2026 amendment to the EU AI Act, the picture changed: without the excerpts, all 12 answers were correct, yet for 9 of them (E11 to E13) the rationale did not rest on the text the amendment introduced.
This article contains affiliate links. As an Amazon Associate, we earn from qualifying purchases.
The exam guide for AIB-C01 (AWS Certified AI Business Strategist) gives Domain 3, “AI Governance and Responsible AI Leadership”, 24% of the scored content. I covered the certification as a whole in What the AWS AI Business Strategist certification proves — preparing over 13 weeks for the February 15 early-adopter badge (published 2026-11-16). This article describes how to practise this domain against public source texts, and what happened when I measured it. I ran the same comparison with the local model gemma4:12b: it scored 29 of 38 without the excerpts and 36 of 38 with them, and its two errors with the excerpts were completely different in kind. The takeaways come down to three habits: check the rationale, not just the answer; check which version of the provision the rationale relies on; and the more a provision has been amended, the more important it is to hand the AI the public source text.
The verdicts in this article are study notes for an exam, not legal advice. Take real-world decisions to a qualified professional. The workshop in the cases is fictional, and this article does not say whether any reader’s business or workshop falls under any of these rules. I have not taken this exam, and I have not seen the actual exam questions. Everything about the exam and the sources reflects the situation as of November 2026.
- What does the AIB-C01 governance domain test?
- Frameworks for reading Domain 3 — AWS dimensions, NIST’s four functions, three types of actors
- Different verdict wording for each source
- 38 cases in a fictional workshop, and what the AI was given
- Results — how the source bundles changed the number of correct answers
- A correct answer does not mean the rationale came from the amended text — reading E10 to E13
- What was inside the answers counted as wrong — gemma4:12b’s J09
- Mixing up NIST functions — N02 and N07
- A procedure for reading AI answers while studying
- Judgments the source text alone cannot settle
- What this run cannot tell you
- Summary — study AIB-C01 governance through the rationale
- Sources
What does the AIB-C01 governance domain test?
Domain 3 of the exam guide consists of three tasks and twelve skills. Task 3.1 is “Apply responsible AI principles to business decisions.” Task 3.2 is “Establish AI governance structures and ensure regulatory compliance.” Task 3.3 is “Identify enterprise AI risks and direct mitigation strategies.” The guide is also published in Japanese; I compared its wording with the English original in Checking the Japanese AIB-C01 exam guide against the English original — using AI to find where the translation changes the meaning across 58 skills (published 2026-11-17).
The skills in this domain are largely about holding a case up against a norm. Skill 3.1.1 asks you to “Apply responsible AI principles and dimensions (for example, fairness, explainability, privacy, safety, transparency, robustness) to business scenarios.” Because the guide says “for example”, those six words are not a complete list of the dimensions. Skill 3.2.2 is “Identify and address regulatory compliance risks for business processes that use AI”, and Skill 3.2.4 is “Apply AI risk classification frameworks to prioritize governance and compliance decisions across the AI lifecycle.”
Note, however, that these two skills do not name any framework. A full-text search of the exam guide PDF (18 pages) returned zero hits for NIST, EU AI Act, European and GDPR (2026-11-07). The NIST AI RMF, the EU AI Act and the Japanese documents used here were chosen by this site as public source texts; the exam does not prescribe them. Among the AWS frameworks the exam guide does name, the one for governance best practices is the Responsible AI Lens of the Well-Architected Framework.
It is also worth checking the scope boundaries. Shadow AI is Skill 1.2.4, and ISO/IEC 42001 and 23053 fall under Skill 1.1.6; both belong to Domain 1, not Domain 3. So in this article shadow AI appears in only one case, and the ISO standards appear only by title and summary. Whether to adopt AI at all is covered in Can AI answer “don’t use AI”? Testing AIB-C01 adoption decisions with cases where rules are enough (published 2026-11-18), and the technical permission design for a different exam is covered in Training agent permissions and governance with design problems — having AI propose designs and grading them against official best practices (published 2026-10-24). This article is about organizational and accountability decisions.
Domain 3 asks how you would set up AI governance in a real organization. For a hands-on guide to risk assessments, policies, oversight workflows and monitoring, this new Packt book fits.
Frameworks for reading Domain 3 — AWS dimensions, NIST’s four functions, three types of actors
First, a quick tour of the sources used to judge the cases. All of them were chosen by this site; none are specified by the exam guide.
AWS’s number of dimensions depends on the document
Counting the responsible AI dimensions across AWS documents does not give a single number. The AWS overview page and the Generative AI Lens list eight, while the Responsible AI Lens, which the exam guide names, splits privacy from security and veracity from robustness for a total of ten. Both the overview page and the Lens say the dimensions will keep being updated. So it is better not to memorize any one count.
| Source | Count | What is being counted |
|---|---|---|
| AWS overview page, Generative AI Lens | 8 items | Dimensions. Privacy and Security, and Veracity and Robustness, are each combined into one item |
| Responsible AI Lens | 10 items | Dimensions. The two pairs above are listed separately |
| Responsible AI Lens | 8 | The number of focus areas, not dimensions |
| Exam guide, Skill 3.1.1 | 6 words | Examples, not a complete list |
| AIF-C01 exam guide | 6 words | Examples from a different document (bias, fairness, inclusivity, robustness, safety, veracity) |
The definitions also differ between documents. On the overview page, Explainability is about understanding and evaluating system outputs; in the Lens, it is about having mechanisms to understand system behavior. Only four dimensions are defined in the same words across all three documents: Controllability, Fairness, Governance and Transparency. Cases W01 to W03, used later, rely on these four alone. As I noted in AIF-C01 Domains 4 and 5 — locking in 28% with responsible AI, security and governance (published 2026-06-12), the six AIF-C01 words are examples from a different exam guide.
The four functions of the NIST AI RMF
AI RMF 1.0 (released January 26, 2023) is a voluntary framework, and its Core consists of four functions: GOVERN, MAP, MEASURE and MANAGE. NIST’s page says 1.0 is under revision (as of November 2026). Because it is not law, the questions in this article are limited to “which function’s subcategory does this correspond to?”
| Function | Example subcategories |
|---|---|
| GOVERN | 1.6 Mechanisms to inventory AI systems, resourced accordingly / 2.1 Documented roles, responsibilities and lines of communication |
| MAP | 4.1 Approaches for mapping the technology and legal risks of components, including third-party data or software |
| MEASURE | 2.10 Privacy risk identified in the map function is examined and documented / 2.11 Fairness and bias are evaluated and results documented |
| MANAGE | 2.4 Mechanisms and assigned responsibilities to supersede, disengage or deactivate AI systems whose outcomes are inconsistent with intended use |
NIST states: “Actions do not constitute a checklist, nor are they necessarily an ordered set of steps.” So do not read the subcategories as a to-do list to be worked through from the top. Separately, NIST lists seven characteristics of trustworthy AI, but these are a different list from the AWS dimensions, and the two should not be mapped one to one. NIST also publishes a Japanese translation of the PDF, but its closing note says the translation is not guaranteed to be complete or accurate.
Three types of actors in Japan’s AI Guidelines for Business
Version 1.2 of Japan’s AI Guidelines for Business (Ministry of Internal Affairs and Communications and Ministry of Economy, Trade and Industry, March 31, 2026) is the latest version as of November 2026. It is non-binding soft law, and most of the passages used here are phrased as things to “consider” or “take note of”. The Appendix passage used in case J05 says that “responses such as notifying or publicly announcing the purpose of use become necessary” (our translation). Read every passage in its own wording, and do not turn it into “obligation” or “prohibition”. The common guiding principles are ten: human-centric, safety, fairness, privacy protection, ensuring security, transparency, accountability, education and literacy, ensuring fair competition, and innovation (items 8 to 10 are matters that actors are expected to address in cooperation with society).
The guidelines distinguish three types of actors. AI developers are businesses that develop AI systems; AI providers are businesses that provide services incorporating AI systems to AI business users and others; and AI business users are businesses that use AI systems or AI services in their business activities. One business can play more than one of these roles. The EU AI Act’s “provider” and “deployer”, on the other hand, are defined differently, so do not equate them just because the words look similar. Also, the term “shadow AI” appears in neither the main text nor the Appendix of Version 1.2. Employees using generative AI outside company control is discussed in the Appendix, and case J06 uses that passage.
The camera guidebook and the ISO/IEC standards
The Camera Image Utilization Guidebook ver3.0 (March 2022) is published by the IoT Acceleration Consortium, the Ministry of Internal Affairs and Communications and the Ministry of Economy, Trade and Industry as a reference for businesses’ voluntary efforts. It was the latest version I could find in November 2026. The use cases it examines are mainly shops and similar settings; the words “factory” and “workshop” do not appear. It does, however, contain considerations for cases where employees are captured on camera, so this article uses those passages as reading practice.
ISO/IEC 42001:2023 and ISO/IEC 23053:2022 are both paid standards. I have not read their full text, so I did not give it to the AI either. According to the official summaries, 42001 specifies requirements and provides guidance for establishing, implementing, maintaining and continually improving an AI management system within an organization (as shown on the IEC store page; I could not open the iso.org page). 23053 establishes a conceptual framework and common terminology for describing AI systems that use machine learning.
The AWS dimensions of responsible AI turn into engineering work once a system is built. Chip Huyen’s AI Engineering explains evaluation, guardrails and the trade-offs behind them.
Different verdict wording for each source
Before grading the cases, I fixed the wording of the verdict for each source. If a legally binding regulation, a voluntary framework and a reference for voluntary efforts are all judged in the same words, their relative strength gets distorted.
| Source | Nature | Verdict wording used in this article |
|---|---|---|
| EU AI Act | Legally binding regulation | “Falls / does not fall within the wording of Article N” |
| European Commission guidelines on prohibited practices (2025-07-29) | Not binding; the authoritative interpretation lies with the Court of Justice of the EU | “In the guidelines’ examples” |
| NIST AI RMF 1.0 | Voluntary; under revision | “Which function’s subcategory it corresponds to” |
| AI Guidelines for Business, Version 1.2 | Non-binding soft law | “The guidelines say to consider …” |
| Camera Image Utilization Guidebook ver3.0 | Reference for businesses’ voluntary efforts | “The considerations include …” |
| AWS list of dimensions | AWS’s own definitions; subject to updates | “In the AWS Responsible AI Lens” |
For the EU AI Act, too, the question was limited to “does this fall within the wording?” Being listed in Annex III is not the same as being high-risk. Article 6(3) provides a derogation: an Annex III AI system is not considered high-risk if it does not pose a significant risk of harm to the health, safety or fundamental rights of natural persons. The derogation applies where any of the following conditions is met (paraphrased):
- (a) the system is intended to perform a narrow procedural task
- (b) it is intended to improve the result of a previously completed human activity
- (c) it is intended to detect decision-making patterns or deviations from prior decision-making patterns, and is not meant to replace or influence the previously completed human assessment without proper human review
- (d) it is intended to perform a preparatory task to an assessment relevant to the use cases listed in Annex III
However, the last subparagraph of the same paragraph says that an Annex III AI system “shall always be considered to be high-risk where the AI system performs profiling of natural persons.” Article 3(52) defines profiling by reference to Article 4(4) of the GDPR: automated processing that uses personal data to evaluate personal aspects of an individual, with analysing or predicting performance at work among the examples. Which AI systems fall into these categories is a legal judgment, so this article does not decide it.
Being a safety component is likewise not the same as being high-risk. So this article never applies the words “illegal” or “high-risk” to the workshop. It only says things like “falls within the wording of Article 5(1)(f)” or “falls within the wording of the listing in point 4(b) of Annex III”.
I also drew a line around the Commission’s guidelines. The correct answers for E02, E03, E05 and E06 are not settled by the text of Article 5(1)(f) alone; they rest on how the Commission’s guidelines on prohibited practices read it. For example, E03, “infer the writer’s mood from the text of daily work reports”, looks as if it could fall within the article’s wording, but the guidelines read the provision as limited to inferences based on biometric data. In the answer column, such cases are marked “reading of the guidelines”.
It also helps to know exactly where the amendment landed. Regulation (EU) 2026/1744 (the Digital Omnibus on AI), published in the Official Journal on July 24, 2026, replaced Article 4, added paragraphs 1a to 1c to Article 6, added a sentence to the definition of a safety component in Article 3(14), and changed the application dates in Article 113. The text of Article 5(1)(f) did not change. I covered the application date of Article 50 and the December 2 transitional arrangements in Where do AI-generated labels and marks disappear in images and 3D data? Counting 2,862 article images before the December 2 deadline, and framing 3D data as open questions (published 2026-11-09), so I will not repeat it here.
38 cases in a fictional workshop, and what the AI was given
The setting is a fictional small 3D printing workshop introducing a build-monitoring AI. At the center is an AI that uses a camera to detect failed prints; around it are related decisions such as monitoring workers, assigning work, a quotation chatbot, outsourcing analysis and training on design data. By source, there are 14 EU AI Act cases, 8 NIST AI RMF cases, 10 AI Guidelines for Business cases, 3 camera guidebook cases and 3 AWS dimension cases, for a total of 38.
The questions were limited to “does it fall within the wording?”, “which provision?”, “which function?”, “which type of actor?” and “what does the source text require?”, and the AI had to choose one answer from a list of options. Of the 38 cases, 22 had two options, 4 had three, 9 had four and 3 had ten. Here are some representative cases (the cases themselves were written in Japanese):
| ID | Case | Question | Correct answer | Deciding text |
|---|---|---|---|---|
| E01 | Reads workers’ facial expressions from camera footage, infers irritation or dissatisfaction, and notifies the workshop owner. No medical or safety purpose | Does it fall within the wording of Article 5(1)(f)? | Yes | Article 5(1)(f) |
| E03 | Infers the writer’s mood from the text of daily work reports. No biometric data is used | Is it within the scope of Article 5(1)(f)? | No | Reading of the guidelines |
| E12 | The AI detects failed prints and stops the build. It is used only for quality control, has nothing to do with human safety, and its failure would not endanger human health or safety | Is it a “safety component” under Article 6? | No | Article 6(1a) |
| N07 | Examine and document the privacy risks of a camera-based AI | Which function’s subcategory does it correspond to? | MEASURE | MEASURE 2.10 |
| J09 | The monitoring AI itself keeps malfunctioning and is causing harm | Which of these does the Appendix list as an initial response? | Stop the AI system | Appendix, page 178 |
| K01 | Only the workshop’s own employees appear on camera | Does the guidebook say no consideration is needed if only employees are captured? | No | Consideration item 12 |
| W02 | Let customers know that AI is involved and choose whether to use it | Which Lens dimension is this? | Transparency | Lens definition |
E12 is about stopping a print, while J09 and N02 (which comes up later) are about stopping the AI. Because the wording is similar, the case texts keep the two apart. Also, E14 only asks whether the subject of the sentence in Article 50(1) is the provider or the deployer; it does not ask which of the two the workshop is.
There were two conditions. “Without excerpts” gives only the name of the source. “With excerpts” gives every case the same collection of excerpts for its source (the “source bundle” from here on). Each bundle is a set of rows taken from the tables this site built during its pre-publication fact check, not the full text of the provisions. The row headings were written in Japanese by this site, and the old and new versions of Article 4 are distinguished by those headings. Only for NIST did I extract the text of all 72 Core subcategories automatically from the official HTML version. In that extraction, MAP 1.1 was cut off partway, and GOVERN 6.2, MAP 5.2 and MEASURE 4.3 picked up part of the following section, but none of the text that decides N01 to N08 was missing. The bundles were 7,043 characters for the EU AI Act, 14,520 for NIST, 983 for the AI Guidelines for Business, 659 for the camera guidebook and 869 for the AWS dimensions.
Here is what I did not give the AI. Not the full text of the aws.amazon.com overview page: for the AWS dimension definitions, only the Responsible AI Lens text on docs.aws.amazon.com. Not the ISO standards, which I do not have. Not the full camera guidebook, only the passages needed. Not the draft guidelines on high-risk classification. And under no condition any official AIB-C01 questions or course material. The AI received only my own cases and excerpts from public source texts.
The prompt is below (translated from the Japanese original). The line “Answer on the basis of the following excerpts.” and the “Excerpts” section were included only in the with-excerpts condition.
(System instruction) You help with studying for a certification exam. You do not give legal advice. Answer only in the specified format.
For the following case, answer the question from the wording of (name of source). This is study practice for a certification exam, not a legal judgment.
Answer on the basis of the following excerpts.
Excerpts:
(source bundle)
Case: (case text)
Question: (question text)
Options: (options)
Return only the following JSON. In answer, write the option text exactly as given.
{"answer": "option text", "basis": "rationale in one sentence"}
One detail matters: the “name of source” slot. For the EU AI Act cases, even the without-excerpts prompt said “Regulation (EU) 2024/1689 and the amending Regulation (EU) 2026/1744”. In other words, even without excerpts, the model was told that an amending regulation exists.
The runs took place on 2026-11-07. claude -p was Claude Code 2.1.319 with --model sonnet, configured without tools or MCP. The recorded model ID was claude-sonnet-5-5. Each case was answered three times per condition. The local gemma4:12b ran on Ollama 0.40.15 with think: false and temperature 0, and answered each case once per condition. After the first with-excerpts run, I found a problem in how the source bundles were built, so I fixed the bundles and reran that condition with the same number of runs. All with-excerpts numbers and quotes in this article come from the fixed bundles. As decided before the runs, scoring only checked whether the answer text exactly matched an option. In other words, the rationale (basis) was not scored.
Results — how the source bundles changed the number of correct answers
| Source | claude -p without excerpts | claude -p with excerpts | gemma4:12b without excerpts | gemma4:12b with excerpts |
|---|---|---|---|---|
| EU AI Act | 42/42 | 42/42 | 10/14 | 14/14 |
| of which E10 to E13 | 12/12 | 12/12 | 3/4 | 4/4 |
| NIST AI RMF | 20/24 | 24/24 | 4/8 | 7/8 |
| AI Guidelines for Business | 30/30 | 30/30 | 9/10 | 9/10 |
| Camera Image Utilization Guidebook | 9/9 | 9/9 | 3/3 | 3/3 |
| AWS dimensions | 9/9 | 9/9 | 3/3 | 3/3 |
| Total | 110/114 | 114/114 | 29/38 | 36/38 |
The denominators are 38 cases × 3 runs, or 114, for claude -p, and 38 cases × 1 run, or 38, for gemma4:12b. gemma4:12b’s without-excerpts run on E01 was invalid: its rationale degenerated into the same phrase repeated over and over and broke the JSON. It is not counted as correct in the table.
For claude -p, the source bundles added only four correct answers, all in NIST. The EU AI Act, the AI Guidelines for Business, the camera guidebook and the AWS dimensions were already perfect without excerpts. gemma4:12b went from 29/38 to 36/38, with 4 of the gain in the EU AI Act and 3 in NIST. It stayed at 9/10 on the AI Guidelines for Business, but the case it got wrong changed from J05 without excerpts to J09 with excerpts.
Reading only this table, it looks as if “with claude -p, handing over the source text makes little difference.” On top of that, 22 cases have only two options, so the scores have limited room to differ. The core finding of this run lies outside the table. You can score the answer text, but you cannot tell which sentence the answer came from without reading the rationale.
A correct answer does not mean the rationale came from the amended text — reading E10 to E13
E10 to E13 were built so that text introduced by the July 2026 amendment would be the basis for the answer. However, only for E10 would the pre-amendment text produce a different answer; for E11 to E13, as the without-excerpts runs show, you can reach the same answer without relying on the newly introduced text. claude -p got all 12 right both with and without excerpts. So, separately from the answers, I read every rationale and classified whether it rested on text introduced by the amendment.
| Case | Question | Without excerpts (3 runs) | With excerpts (3 runs) |
|---|---|---|---|
| E10 | As of November 2026, do Sections 1 to 3 of Chapter III apply to high-risk AI systems in Annex III? | Correct all 3 times. All 3 cited the postponement to 2027-12-02 and added a caveat such as “based on memory” | Correct all 3 times. All 3 cited the amended Article 113 |
| E11 | Does Article 4 require a guarantee that each individual reaches a specific level of AI literacy? | Correct all 3 times. All 3 relied on the pre-amendment wording of Article 4 (“sufficient level”, “to their best extent”) | Correct all 3 times. All 3 relied on the amended text of Article 4 |
| E12 | Is an AI used only for quality control a “safety component” under Article 6? | Correct all 3 times. All 3 relied on the definition in Article 3(14) and did not mention the newly added Article 6(1a) | Correct all 3 times. All 3 relied on Article 6(1a) |
| E13 | Is an AI whose failure would endanger human health and safety a “safety component” under Article 6? | Correct all 3 times. All 3 relied on Article 3(14). One stated that it had not checked the content of the amending regulation | Correct all 3 times. All 3 relied on the newly added paragraph |
E10 — it named the postponement, but said it was from memory
The amendment moved the application dates for high-risk systems: for high-risk AI systems in Annex III, the date became December 2, 2027 (I summarized all the dates in Update the law and ethics for Japan’s G Certificate to current rules with AI before memorizing them — mapping primary sources on Japan’s AI Act, the AI Basic Plan, copyright and the EU AI Act to the syllabus (published 2026-10-04)). The three without-excerpts runs all named this postponement, and the first, for example, added the caveat “the content of the amendment is based on my memory and needs to be checked against the text of the provisions” (translated from the Japanese output). The three with-excerpts runs relied on the Article 113 text in the source bundle. The answer is the same, but the source of the rationale moved from memory to the source text.
E11 — it answered with the pre-amendment wording of Article 4
The text of Article 4 changed with the amendment. Before the amendment, it required measures to ensure, to the best extent, a sufficient level of AI literacy among staff. After the amendment, it requires measures to support the development of AI literacy among staff and others, and states explicitly that it does not require any specific level to be guaranteed for any individual. All three without-excerpts runs answered using the pre-amendment wording. The third rationale, for example, contains this passage (translated from the Japanese output):
it is an obligation requiring measures to ensure, ‘to their best extent (best extent)’, a ‘sufficient level’ of AI literacy among the company’s own staff and others
For this question, relying on the pre-amendment wording still leads to the answer “No”, so nothing shows up in the score. But what a learner absorbs from reading this explanation is a text that has already been replaced.
The three with-excerpts runs relied on the amended text of Article 4. The first rationale, for example, began “the amended Article 4 only requires measures to support the development of AI literacy” (translated from the Japanese output) and quoted the newly added sentence verbatim: “This obligation does not require providers or deployers to guarantee any specific level of AI literacy of any individual.”
E12 and E13 — no mention of the newly added paragraphs
E12 and E13 are questions about “safety components” under Article 6. The newly added paragraph 1a says that AI systems used solely for non-safety purposes (such as quality control) are not safety components, and paragraph 1b says that, notwithstanding paragraph 1a, AI systems whose failure or malfunction would endanger health and safety are safety components. The six without-excerpts runs did not mention these paragraphs and relied on the definition in Article 3(14) instead. That part of Article 3(14) survives the amendment, so it is not a wrong basis. Still, none of the six mentioned the new paragraphs of Article 6 or the sentence added to Article 3(14). One E13 run ended with this caveat (translated from the Japanese output):
Note that I have not checked the content of amending Regulation (EU) 2026/1744.
With excerpts, on the other hand, the three E12 runs relied on Article 6(1a), and the three E13 runs relied on the newly added paragraph. All three E13 runs called it paragraph 1c, but in the Official Journal this sentence is paragraph 1b. The source bundle only refers to paragraphs 1a to 1c collectively and does not number each sentence, so the classification did not compare paragraph numbers. What I checked here was only how the wording was read. The text of paragraph 1a applies to this Regulation as a whole, including Article 6(1), and Article 6(1) is the route for products covered by Annex I. And to repeat: being a safety component is not the same as being high-risk.
Not every 12/12 is the same
To sum up: claude -p without excerpts got all 12 answers on E10 to E13 right, but the 3 E11 runs relied on a replaced text and the 6 E12 and E13 runs relied on the surviving text of Article 3(14), so none of those 9 rested on text introduced by the amendment. The only amendment content that appeared in the without-excerpts rationales was the postponed application date in E10. With excerpts, the score was also 12/12, the same value in the table. But with excerpts, the rationales switched to the newly introduced text. The accuracy rate alone does not show this difference.
What was inside the answers counted as wrong — gemma4:12b’s J09
The local gemma4:12b went from 29/38 without excerpts to 36/38 with them. On the amendment-dependent E10 to E13 it improved from 3/4 to 4/4, and the content of its rationales changed too. Without excerpts, it answered “Yes” to E10 without mentioning the application date. With excerpts, it answered “No” to E10 citing the amended Article 113, but its rationale got the chapters wrong (translated from the Japanese output):
Because, under the amendment to Article 113, the date from which Chapters I and II (including Sections 1 to 3) apply to high-risk AI systems under Annex III is December 2, 2027.
Under Article 113 as published in the Official Journal, what applies to Annex III high-risk AI systems from December 2, 2027 is Sections 1 to 3 of Chapter III (except Article 6(5)); Chapters I and II have, with some exceptions, applied since February 2, 2025. The answer text is correct, but the chapters it says apply are not.
With excerpts, its E11 relied on the content of the sentence the amendment added to Article 4, its E12 on Article 6(1a), and its E13 on the newly added paragraph. Like the three with-excerpts runs of claude -p, its E13 called that paragraph 1c.
With excerpts, it made two errors, and they are completely different in nature.
| Case | Correct answer | Answer given | What happened |
|---|---|---|---|
| J09 | Stop the AI system | Stop the AI system (kill switch) | Answered in the wording of the source bundle, so it did not exactly match the option text. The content is the same as the correct answer |
| N07 | MEASURE | MAP | Read the text of MEASURE 2.10 as “MAP 2.10” |
J09 was counted as wrong under the rule set before the runs: an answer is correct only if it exactly matches the option text. Its rationale was “the excerpts contain the phrase ‘stopping the AI system (kill switch)’” (translated from the Japanese output), and its content is the same as the correct answer. I did not change the rule afterwards and recount. N07, by contrast, is a case of mixing up the function itself, covered in the next section.
The lesson is that the score does not reflect what is inside the answers, in either direction. Tallying answer text alone counts a difference in wording like J09 and a substantive error like N07 as the same single error. It also counts as correct the runs whose rationale did not rest on the amended text, like the without-excerpts E11 to E13 of claude -p, and runs whose rationale got the chapters wrong, like gemma4:12b’s with-excerpts E10. Whether an answer was scored right or wrong, you only see what is inside by reading the rationale.
gemma4:12b ran locally through Ollama. To run open models yourself and compare answers with and without source excerpts, Hands-On Large Language Models is a good starting point.
Mixing up NIST functions — N02 and N07
claude -p made four errors without excerpts, and all of them were mix-ups of NIST functions: one on N02 and three on N07. With excerpts, NIST went to 24/24, including these two cases.
The N07 case is “examine and document the privacy risks of a camera-based AI”, and the correct answer is MEASURE. MEASURE 2.10 reads: “Privacy risk of the AI system – as identified in the map function – is examined and documented.” All three without-excerpts runs answered MAP and used the word “identify” in their rationale (translated from the Japanese output):
The activity of identifying and documenting potential risks and impacts, including privacy, in the context in which the AI system is used falls under a MAP subcategory, which deals with risk identification and understanding context.
The case says “examine and document”; it does not say “identify”. In the source text, identifying belongs to MAP and examining to MEASURE, yet all three without-excerpts runs made the same mistake in the same direction. What deserves attention is that the three answers agreed. Asking the same question repeatedly and getting the same answer was not evidence of correctness in this case.
N02 is a case about putting in place mechanisms to disengage or deactivate an AI that produces outcomes outside its intended use, and assigning responsibility for them; it is about stopping the AI. The correct answer is decided by the text of MANAGE 2.4. One without-excerpts run answered GOVERN and cited GOVERN 1.7. GOVERN 1.7 is a real subcategory, but its text is about processes for decommissioning and phasing out AI systems safely; the wording about outcomes inconsistent with intended use is in MANAGE 2.4. Another run that correctly answered MANAGE also listed MANAGE 4.2 alongside MANAGE 2.4. The text of MANAGE 4.2 is about measurable activities for continual improvement. A subcategory number existing is one thing; the number being paired with the right text is another.
Without excerpts, gemma4:12b got N01, N02, N04 and N07 wrong; with excerpts, only N07 remained. That rationale began “MAP 2.10 states that the privacy risk of the AI system”. The sentence it cited belongs to MEASURE 2.10, and there is no subcategory called MAP 2.10 in the source bundle. The original sentence contains the words “map function”, so it may have been pulled in by them, but with a single run I cannot say what caused it. The study habit to take from this result is to tell the functions apart by the text of each subcategory, not by the impression the function names give.
A procedure for reading AI answers while studying
Based on these results, here is a procedure for reading AI answers. All of this is about studying before the exam, not about using AI during the exam. Give the AI only public source texts and cases you wrote yourself; do not paste in official questions or course material.
- Write the cases yourself. Use a familiar setting such as a workshop, and build the conditions for the verdict into the case text
- Phrase questions as “does it fall within the wording?” or “which function?”. Do not ask the AI “is this illegal?” or “does this apply to my company?”
- Have the answer and the rationale returned in separate fields. The JSON format above works as is
- For any run that looks wrong, read the rationale and separate differences in wording from substantive errors
- Check that the text cited in the rationale actually exists in the source you provided, and that the number and the text match
- Check which version of the provision the rationale relies on. For law that has been amended, open the amending regulation and a consolidated version, and see whether pre-amendment wording lingers and whether added text is addressed
- Do not skip the caveats. Wherever the AI writes “based on memory” or “not in the excerpts”, open the source text yourself
The table shows which runs in this experiment correspond to steps 4 to 7.
| What to check | Runs where it showed up | What answer text alone suggests |
|---|---|---|
| Difference in wording or substantive error | gemma4:12b with excerpts, J09 | Looks like one error, the same as a substantive one |
| Number-text pairing (answer also wrong) | claude -p without excerpts, one N02 run; gemma4:12b with excerpts, N07 | Looks like nothing more than a function mix-up |
| Number-text pairing (answer correct) | With excerpts, E13 for claude -p and gemma4:12b (4 runs in total); gemma4:12b with excerpts, E10; claude -p without excerpts, the N02 run that cited MANAGE 4.2; and others | Looks correct |
| Version of the provision and added text | claude -p without excerpts, 9 runs on E11 to E13 | Looks correct |
| Caveats | claude -p without excerpts, 3 runs each on E10, J09 and K02; and others | Looks like a verified correct answer |
Caveats are easy to overlook, but they are useful. The three without-excerpts J09 runs also got the answer right while adding caveats such as “this is a summary from memory, not a verbatim quotation of the original” (translated from the Japanese output). A run with a caveat is one where the AI itself says it has not verified something, which makes it a good signal for which source text to open first. The reverse does not hold, though. The six without-excerpts runs on E11 and E12 had no caveats, yet their rationales did not rest on the amended text. The absence of a caveat is not evidence of verification.
The more a provision has been amended, the more you should hand the AI the public source text; that is how I read this run. The only amendment content that appeared in the without-excerpts rationales for E10 to E13 was the postponed application date; the amendments to Articles 4 and 6 never came up. For the EU AI Act, keeping in mind that only the Official Journal text is legally authentic, the consolidated version on the AI Act Service Desk (as of 2026-07-27) is also a useful aid. Its Article 6 page marks paragraphs 1a to 1c as NEW. That said, handing over the source text does not guarantee a correct reading: gemma4:12b’s with-excerpts N07 misread the subcategory number of the very text it was given.
Judgments the source text alone cannot settle
Beyond the 38 cases, there are cases I did not have the AI judge. While building the case table, I removed 12 cases whose answer could not be determined from the wording of the source text alone. The main ones are:
- Does the EU AI Act apply to the workshop at all? (Applying Article 2 is a legal judgment)
- Are the AI systems in E07 (assigning work based on each worker’s rework rate and response speed) and E08 (timing each worker’s setup and sending evaluation scores) high-risk? (The Article 6(3) derogation turns on whether there is a significant risk of harm and on conditions (a) to (d), and it is not available for AI systems that profile natural persons. Whether it applies is a legal judgment, so I do not decide it)
- Does inferring a worker’s emotions near machinery or solvents fall under the safety exception? (The guidelines say the exception is to be read narrowly, and their examples turn on whether there is a specific danger)
- May a print be stopped on the AI’s judgment alone, or should a human check be inserted? (The main text of the guidelines can only be read as “it depends on the use”)
- Is monitoring AI behavior in production MEASURE or MANAGE? (It fits both MEASURE 2.4 and MANAGE 4.1)
This list also marks the limits of studying by checking source texts. Skill 3.1.2, for example, is “Navigate tradeoffs when business objectives conflict with responsible AI principles.” Practising finding the right words in a source text does not train that kind of judgment. This article also gives no answer on whether a print may be stopped on the AI’s judgment alone. I worked out stop thresholds from material costs in Setting 3D print monitoring thresholds from material costs — where on the probability scale to stop and where to hand off to a human (published 2026-10-17), but that is a cost question, not a normative answer.
Some cases were also set aside. The European Commission’s guidelines on high-risk classification were still a draft as of November 7, 2026 (published 2026-05-19). Four cases that depend on a reading of the draft were left out of the run because their correct answers could change. The draft also describes how to read point 4(b) of Annex III, but it should not be read as a settled interpretation. Even with a public source text in hand, whether that text is final is a separate thing to check.
What this run cannot tell you
- The cases were written by this site and are not exam questions. The scores are no indicator of passing or failing. I have not checked which sources the actual exam draws on or how it asks
- The source bundles are rows from this site’s fact-check tables, not the full text of the provisions. They include the text that decides each answer, but for E10 only the date part of Article 113 was included, and the words “Sections 1 to 3 of Chapter III” appeared only in the E10 case text given to the AI. This may be related to gemma4:12b mixing up the chapters, but with a single run I cannot say what caused it. The row headings (such as old and new Article 4) were written by this site. The with-excerpts results include the ability to find words in the excerpts; they do not measure the ability to search and interpret the full text of the law
- The EU sources are in English, while the cases were written in Japanese. The effect of language cannot be separated from the effect of providing the sources
- The rationales for all of E10 to E13 were classified by the Claude model this site uses for production (Claude Opus 5.5), with no human review. The answering model is from the same company. Rationales for the other cases were not classified on the same basis
- When gemma4:12b’s with-excerpts condition was rerun after fixing the format of the source bundles, answers such as E10 changed. Since the answers moved with the bundle format even at temperature 0, the change from 3/4 to 4/4 cannot be attributed solely to the presence of the sources
claude -pran three times per condition and gemma4:12b once, and only two models were tested. This does not support generalizations such as “AI does not know about amendments”- The without-excerpts prompt still named the source, so it was not a condition with no information at all
- The verdicts in this article are not legal judgments. E12 is a wording exercise; I have not found any official example of whether real-world build monitoring counts as quality control. Do not apply the workshop cases to your own business
- NIST AI RMF 1.0 is under revision, and the guidelines on high-risk classification are a draft. Both statements reflect November 2026, and if the sources change, the basis of the cases will need to be reviewed
Summary — study AIB-C01 governance through the rationale
The AIB-C01 governance domain lists skills for applying principles, regulations and risk frameworks to cases, but the exam guide does not name NIST or the EU AI Act. This article graded 38 cases about a fictional workshop against public source texts chosen by this site. In this run, claude -p scored 110/114 without excerpts and 114/114 with them, and gemma4:12b scored 29/38 and 36/38.
The rationale mattered more than the numbers. claude -p without excerpts got all 12 answers on E10 to E13 right, but for 9 of them (E11 to E13) the rationale did not rest on the text introduced by the amendment. With the excerpts, the rationales switched to the newly introduced text. gemma4:12b’s two errors with excerpts were different in kind: one was a difference in wording, the other a misread subcategory number. The errors of claude -p without excerpts were concentrated in mixing up NIST functions.
If you want to bring one thing back to your AIB-C01 governance study, start by trying a single provision. Have the AI return an answer and a rationale in separate fields for a case you wrote yourself, and read the rationale side by side with the public source text. Check whether the provision it relies on is the pre-amendment or post-amendment version, and for amended parts, hand over the source text and ask again. Along the way, you will also see which judgments the source text alone cannot settle.
Sources
- AWS, AIB-C01 exam guide, Content Domain 3
- AWS, AIB-C01 exam guide, Content Domain 1
- AWS, AIB-C01 exam guide, introduction page
- AWS, AIB-C01 exam guide (PDF)
- AWS, Responsible AI overview page
- AWS Well-Architected, Responsible AI Lens
- AWS Well-Architected, Generative AI Lens, Responsible AI section
- NIST, AI Risk Management Framework
- NIST AI 100-1, Artificial Intelligence Risk Management Framework (AI RMF 1.0) (PDF)
- NIST AI 100-1, Japanese translation (PDF)
- NIST AI Resource Center, AI RMF 1.0 text (Core section)
- NIST AI Resource Center, AI RMF 1.0 text (trustworthy AI characteristics section)
- Regulation (EU) 2024/1689 (AI Act), EUR-Lex, European Union
- Regulation (EU) 2026/1744 (Digital Omnibus on AI), EUR-Lex, European Union
- Regulation (EU) 2016/679 (GDPR), EUR-Lex, European Union
- AI Act Service Desk, Article 6 (European Union)
- European Commission, Commission Guidelines on prohibited artificial intelligence practices, C(2025) 5052 final (PDF, European Union)
- European Commission, Guidelines for providers and deployers of AI high-risk systems (draft announcement)
- Ministry of Internal Affairs and Communications and Ministry of Economy, Trade and Industry, AI Guidelines for Business (Version 1.2), main text (PDF, in Japanese)
- Ministry of Internal Affairs and Communications and Ministry of Economy, Trade and Industry, AI Guidelines for Business (Version 1.2), Appendix (PDF, in Japanese)
- Ministry of Internal Affairs and Communications, AI Guidelines for Business page (in Japanese)
- IoT Acceleration Consortium, Ministry of Internal Affairs and Communications and Ministry of Economy, Trade and Industry, Camera Image Utilization Guidebook ver3.0 (PDF, in Japanese)
- ISO, ISO/IEC 23053:2022
- IEC Webstore, ISO/IEC 42001:2023
The descriptions of the EU sources and of the AI Guidelines for Business in the body are excerpts and summaries edited by this site. For EU law, only the text published in the Official Journal is legally authentic. Translations of Japanese sources and of the Japanese model outputs are our own. Quotations of NIST text: Republished courtesy of the National Institute of Standards and Technology.





