How to Build an Agentic SAP Assistant, Layer by Layer

A chatbot with your logo on it can answer questions about invoices but getting it to release a blocked invoice in SAP S/4HANA without bypassing segregation of duties, leaking bank details or slowing everything down at month end takes seven layers.

In this walkthrough, Chris Carpenter builds a procure-to-pay (P2P) assistant one layer at a time. It starts where almost every team starts, with an API wrapper around a large language model and a persona on top, and adds each layer only when a real problem forces it.

The seven layers of an agentic SAP assistant

Grounded. The model has never seen your tolerance configs, your policies or your approval matrix, so it fills the gap with a plausible guess. In finance, a confidently wrong answer is worse than no answer. Retrieval-augmented generation (RAG) pulls the right passages from your own P2P documents first and cites them in every answer.

Connected. The clerk wants the work done too. Tool calling through MCP servers and OData APIs lets the assistant look up invoices, create POs from quotes, check goods receipts and kick off approvals in Teams, Slack or email, all visible in Fiori.

Authorized. Handing the agent a broad technical service user gets the change through and quietly bypasses every SoD rule, and the logs only show the service user. Principal propagation passes the clerk's own identity on every call, so SAP checks their real authorizations, routes a blocked change to a second approver and logs it against their user ID.

Guarded. Identity covers the person asking. Documents carry instructions too. A supplier invoice with white text on a white background can tell the agent to change the remit-to bank account. Scan documents and inputs for prompt injection on the way in, require human approval on anything that touches payments, and mask bank and tax IDs in every response.

Efficient. At month end, thousands of people ask the same question, and every one of them is a full round trip through every layer plus an SAP API call. Semantic caching recognizes repeat questions, and model routing sends status lookups to a small, cheap model and disputes and exceptions to a frontier model.

Observable. When finance asks a week later why an invoice posted to cost center 4410, the final answer alone can't tell them. Tool call traces, prompt logs and an audit trail tied to the user who made the request explain every action and give your SOX auditors the evidence they'll ask for.

Tested. New prompts, new model versions and SAP support packs all change behavior. An evaluation suite with known P2P answers, prompt injection attempts and authorization cases that should be denied runs alongside automated Fiori and OData regression, and nothing reaches users until both suites pass. The video shows what happens when that gate gets skipped: an invoice released with tax code I0 when it should have been I1.

Where the work actually is

Picking the model is the easy part. The work is in the layers around it, and they map to the three disciplines Merito works in. Application security covers identity and SAP authorization design for agents, AI threat modeling, prompt injection testing, guardrails and data redaction. Quality covers evaluation suites, automated SAP regression across Fiori and OData, and test data that keeps production data private. DevSecOps covers release gates in your pipeline, observability, audit evidence and cost routing across the models you use. If you're also rolling Claude out across the business, this post on keeping control of Claude adoption covers the governance side, and our test automation practice covers the regression suites that sit behind every release gate.

Map your assistant against the seven layers

Where's your assistant today, still in chat mode or actually completing tasks in SAP? Spend 15 minutes with us mapping it against the seven layers, and we'll find the gaps before your auditors or your suppliers do. Book a conversation HERE.

Merito is a Value-Added Partner delivering AI-powered, secure, efficient and lasting enterprise solutions in AppSec, Quality, and DevSecOps.