FDEs in the Dark

Many years ago, I had the chance to lead Project Frontline at Palantir, a program that sent over 250 software engineers into live customer deployments to become Forward Deployed Engineers. Those engineers are now at OpenAI, Anduril, xAI, and dozens of companies building important things.

I devoted much of my career to this model. But I’ve been thinking lately about where it breaks, and why.

The original problem

The FDE emerged from a specific frustration. For decades, enterprise software followed the same pattern: build the product, close the deal, hand it off, hope for the best. What customers usually got was shelfware, expensive software that never quite fit and eventually got abandoned. The FDE was a correction. Instead of throwing software over the wall, you embedded an engineer inside the customer’s environment. They learned the workflows, owned the failures, built the feedback loop that product teams never had.

At Palantir, the best FDEs became indistinguishable from the customer’s own team. They weren’t there to manage a relationship, but to make the thing actually work.

AI made deployment harder, not easier

You can’t hand an LLM to a compliance team and walk away. So every serious AI company started hiring FDEs. Made sense. The role was built for exactly this kind of complexity.

Except it wasn’t.

What the role was built for:

• Software failed in ways that were diagnosable: a bad pipeline, a misconfigured integration, a wrong assumption about how data would arrive
• Trust was relational; a person in the room with real accountability was part of what got purchased
• The feedback loop between builders and users was broken; FDEs closed it

What’s different now:

• The failure modes are probabilistic, not deterministic. You can’t trace a wrong answer the way you’d trace a broken pipeline
• An FDE can watch where outputs fail and iterate. They still can’t tell you why the model was wrong on a specific query
• Nobody has figured out how to assign accountability when the system can’t explain its own reasoning

FDEs in the dark

At Palantir, when something broke, the FDE could pull the logs, find the failure, and explain it. That explanation was part of the value. Not just fixing the problem, but being able to say why it happened and what you’d changed. That’s what built trust over time. The customer saw you diagnose it. They watched you close the loop. They renewed.

That loop is broken in most AI deployments right now. The FDE is in the room, but the room is darker. They’re accountable for outcomes they can’t fully audit, explaining decisions the system can’t articulate.

The role is getting more important and harder to do well at the same time. Someone still needs to own what happens when the product meets reality. But the old debugging toolkit doesn’t work. You can watch where outputs fail. You can iterate on prompts. You cannot trace a wrong answer the way you’d trace a bad pipeline.

An FDE without traceability is just another person in the room who can’t find the right answer.