Green, amber, red: what the confidence of a Compliance Agent answer means, where the sources come from and when the answer is withheld

Key takeaways

• Green means, under the instruction: directly grounded in the provided sources or in unambiguously applicable law, no legal interpretation required, all facts needed for the answer present. Amber means: complex or contested in supervisory practice, several defensible readings or incomplete source coverage; an answer is possible but flagged with explicit uncertainty.
• Red means: no defensible answer is possible, because sources are missing or because the question requires an individual legal interpretation within the meaning of § 2 RDG. In that case the model is to give no substantive answer but to escalate. In the interface the levels appear as high, medium and low.
• Every source carries a type from four values: regulation, company document, standard or supervisory guidance, and an identifier. Company documents are the sections that a similarity search retrieved from the documents in the workspace database; what is not uploaded cannot be cited.
• The escalation field is set under four rules: if the answer would constitute legal advice under § 2 RDG, if conflicting supervisory decisions or high-court rulings apply, if a wrong answer would be fineable and source coverage is unclear, or if an individual contractual interpretation is asked for. It is a note with a reason, not a handover to a law firm.
• The confidence is the model's assessment under a fixed rubric, not a measured hit rate. The reason the model must supply with it is the part against which the assessment can be checked.
• CIVAC is not a law firm. A green answer is a well-grounded answer, not legal advice; the decision on what follows from it lies with the officer and the company.

What goes into an answer

The agent does not answer the question alone. The instruction it works under places a fixed context before every question: the active role in its German label and in its long form with legal basis, the applicable jurisdictions, the output language, the company profile from the setup wizard and the workspace database, the best-matching sections from the uploaded company documents, and the last six turns of the conversation, oldest first. The document sections come from a similarity search over the database: privacy policy, records of processing activities, technical and organisational measures, data processing agreements, impact assessments, contracts, internal policies and training material, insofar as they are uploaded and processed. What is not in the database is not before the model, and the answer cannot cite it.

What comes out: the answer schema

The answer is a fixed schema with required fields. The answer text is limited to six hundred words. Added to it are the confidence with exactly three possible values, a reason for the confidence, the list of sources, the escalation field as yes or no with a reason, suggestions for follow-up questions, and a field recording whether the company profile was used in the answer. Free text without those fields is not accepted. The three levels are internally called green, amber and red; the interface shows them as high, medium and low.

The three levels, as the instruction defines them

The rubric is part of the instruction to the model and applies without exception. Green: the answer is directly grounded in the provided sources or in unambiguously applicable regulation, no legal interpretation is required, and all the facts needed for the answer are present. Amber: the question is complex or contested in supervisory practice, there are several defensible readings, or the sources do not fully cover the question; an answer is possible but is flagged with explicit uncertainty. Red: no defensible answer is possible, because sources are missing or because the question requires an individual legal interpretation within the meaning of § 2 RDG. On red the model is to give no substantive answer but to escalate.

Two things follow for reading. A green answer is a statement about the sources, not about importance: a trivial question with a clear statute is green, a grave question with a clear statute is green as well. And the confidence is the model's assessment under this rubric, not a measured hit rate. There is no statistic behind the colours. What there is, is the model's obligation to give a reason for the rating. That reason is the part of the answer against which an officer can check the assessment: does it name the source that carries the answer, or does it name an uncertainty that fits the question?

The sources, and where they come from

Every source in the answer carries a label, a type and an identifier. The type has four values: regulation, company document, standard, supervisory guidance. Regulations and standards come from what the model knows of the law, supplemented by what the role brings as its legal basis. Company documents come exclusively from the sections that the similarity search retrieved from the workspace database. That means two things: an answer that cites a company document cites a section that is actually uploaded and can be read in the database. And an answer that cites no company document although the question concerns the company says that the search found nothing fitting; the field recording whether the company profile was used is then the place to look.

The escalation field: four rules, one note

The field is set to yes if one of four conditions holds: the answer would constitute legal advice within the meaning of § 2 RDG; conflicting supervisory decisions or high-court rulings apply; the consequence of a wrong answer would be potentially fineable and source coverage is unclear; or an individual interpretation of a contract is being asked for. For the third rule the instruction names the fining provisions expressly, among them Art. 83 GDPR and § 130 OWiG. Every yes comes with a reason.

What the field is and what it is not should be clear. It is a note in the answer that, under these rules, the question belongs to a law firm, with the reason. It is not a handover: the agent sends nothing, instructs nobody and knows no law firm. Whoever passes the question on is the officer, and the answer with its sources and reason is the brief. The rules are deliberately broad. A question that meets both conditions of the third rule, fineable and unclear sources, is escalated even if the model could phrase an answer. That is the price of the agent not providing a legal service.

How an officer reads an answer

First the colour and its reason, not the text. On green: which source carries the answer, and is it a regulation or one of the company's own documents? On amber: which uncertainty does the reason name, and is it the one you would have seen yourself? On red: the answer contains no substance, and that is as it should be; the reason says why, and whether it is a matter of missing documents that can be uploaded or of the kind of question. Then the sources: open the company documents in the database and read the section the identifier points to. Then the escalation field: if it says yes, the question goes to the legal department or a law firm with the answer as the brief, no matter how convincing the text sounds.

What the agent is not

CIVAC is not a law firm. The agent provides no legal advice, and its instruction is built so that questions requiring it lead to red and escalation rather than to an answer. A green answer is a well-grounded answer with verifiable sources, not legal advice, and what follows from it for the company is decided by the officer with management. The agent also knows only what sits in the workspace: a contract that is not uploaded, a resolution that exists only by email, a works agreement in the filing cabinet do not exist for it, and the answer is then as good as the law but not as good as the company. And the colours are an assessment under a rubric, not a measurement. Reading them as a hit rate is reading them wrong.

What does a green answer mean?

Under the rubric in the instruction: directly grounded in the provided sources or in unambiguously applicable law, without legal interpretation, with all the facts needed present. It is a statement about the sources, not about the significance of the question, and an assessment by the model, not a statistic.

Why does a red answer contain no substance?

Because the instruction requires it: on red no defensible answer is possible, either because sources are missing or because the question requires an individual legal interpretation under § 2 RDG. The model is then not to answer but to escalate and give the reason.

Where do the cited company documents come from?

From a similarity search over the documents in the workspace database. What is cited are sections that are uploaded and processed and can be read there. What is not uploaded cannot be cited.

Does the agent hand escalated questions to a law firm?

No. The escalation field is a note with a reason that, under the four rules, the question belongs to a law firm. The agent sends nothing and instructs nobody; passing it on is the officer's task, with the answer as the brief.

Is the confidence a measured hit rate?

No. It is the model's assessment under a fixed rubric, with a reason the model must supply. There is no statistic behind the three levels; what can be checked is the reason, not the colour.