AI is moving inside the enterprise trust boundary
According to Bloomberg Law, Latham decided three years ago to build its own Nvidia server infrastructure, has bought multiple H200 GPUs, houses the servers in a facility only Latham employees can access, and is experimenting with fine-tuning open-weight models developed by Nvidia. The firm also keeps using commercial AI tools (Bloomberg Law). As Legal IT Insider puts it, owning the infrastructure means the firm can potentially run certain AI workloads without sending client data to a legal AI vendor, model provider, or hyperscaler (Legal IT Insider). The private stack is an additional path for sensitive work, not a replacement.
This article is not about law firms. Latham is useful as evidence of a broader pattern. Organizations that handle highly sensitive information are willing to own substantial AI infrastructure to keep data inside a boundary they control. Private models, open-weight fine-tuning, and enterprise-controlled inference now sit alongside external model APIs as serious deployment options. The regional version of this trend is covered in sovereign AI needs sovereign engineering.
Private inference solves part of the problem
Two different questions tend to get merged:
- Data governance asks where information is allowed to go.
- AI governance asks what an AI system is allowed to do.
Running a model inside the trust boundary answers the first question well. It addresses data residency, third-party exposure, and source-code egress. It says nothing about the second. A privately hosted model driving a coding agent can still use a framework the organization retired, ignore an architectural decision, cross a boundary the security team drew, repeat an approach that was already rejected, or make a structural change nobody approved. The model running on your hardware does not make it aware of your decisions.
The missing piece is decision context
Every large engineering organization holds thousands of decisions spread across ADRs, standards, security policies, tickets, repositories, and the memory of senior engineers. A private coding agent does not know them because it runs locally. It knows what it was trained on and what it was handed in context.
So the private AI stack has a gap between the model and the action:
Private model
|
v
Coding agent
|
v
Applicable decisions <-- usually missing
|
v
Checked constraints <-- usually missing
|
v
ChangeRetrieval is necessary, but it is not governance
The first step is letting an agent ask which decisions apply to the change it is about to make. Protocols like MCP make that possible in a vendor-neutral way, so the same decision source can serve different agents and different models, hosted or private.
Retrieval alone still leaves the agent free to ignore what it retrieved. Governance needs the next steps too: determine which decisions actually apply, check the change against the ones that can be checked mechanically, and record which decision produced the verdict. The limits of context-only approaches are covered in why context alone doesn't prevent architectural drift.
Governance has to stay inside the same boundary
This is the enterprise argument. If an organization has invested heavily to keep models, prompts, and proprietary code inside infrastructure it controls, its governance layer cannot require shipping the repository to another SaaS provider for analysis. That would reopen the boundary the infrastructure was bought to close.
Governance for private AI therefore needs to be able to:
- analyze repositories locally,
- store decisions locally,
- run without network access where required,
- evaluate changes without source code leaving the environment, and
- keep the resulting verdicts inside the same boundary.
The same shift toward repository-local governance shows up at the device level, as argued in the governance perimeter is moving to the endpoint.
From private AI to accountable AI
Regulated organizations will eventually need more than "the governance control ran." They will need to show which decision applied, which rule represented it, what change was evaluated, and what the outcome was. Deterministic verdicts are what make that possible. A model's opinion about whether it complied is not evidence. A rule evaluated against a change, traced to the decision it came from, can be.
Where Mneme fits
Mneme applies to one class of decision in this stack: architectural decisions in a repository. The open-source engine and CLI run on your own machines, and the local audit never sends your code anywhere. Accepted decisions become scoped rules that return deterministic PASS, WARN, or FAIL verdicts against proposed changes. Its MCP server, shipped in 0.9.0, runs as a local process and lets agents, whether backed by a hosted or a private model, retrieve the decisions that apply to the paths they are changing. See the MCP integration and the six-tool reference.
Mneme does not govern model behavior in general, and a complete, audit-grade enforcement evidence trail is direction rather than a shipped feature. The hosted Architecture Audit processes repositories on Mneme infrastructure, which is documented in security and data handling. Organizations that cannot send code outside their boundary can run the same analysis locally with the open-source CLI.
The enterprise AI stack is becoming layered
Enterprise-controlled compute, private and commercial models, agents, the decisions they must follow, deterministic checks, and the systems they change are separating into distinct layers. The first two get most of the attention and budget today. The decision layer is what connects them to the organization's actual architecture.
Owning the model infrastructure gives an enterprise control over where AI runs. Owning the decisions gives it control over how AI acts.