Prompts are source code — treat them like it

A small note before this starts: the examples here are intentionally generic.

The actual system that made me think about this has a few implementation details I can’t really get into, so I’m using a simplified fictional setup instead. The architecture and problems are the same; the names and examples are not.

I was recently working on a system where the same LLM behaviour needed to vary depending on a few things.

Maybe one deployment needs a shorter response style. Another has a different set of tools. One interface supports rich UI, while another only supports plain text. Some workflows need extra instructions that others should never see.

Initially, prompts are easy to manage.

You have a string. Maybe two.

Then the combinations start multiplying.

copy-paste works until it quietly doesn’t

The most obvious approach is to maintain separate prompts.

prompt-web.txt
prompt-chat.txt
prompt-client-a.txt
prompt-client-b.txt

This is perfectly reasonable when the system is small.

The problem starts when most of those prompts contain the same instructions with a few small differences.

You update an important rule in four files but miss a fifth.

Someone fixes behaviour for one interface but the same bug still exists elsewhere.

Two prompts that were supposed to be nearly identical slowly stop being nearly identical.

The annoying part isn’t even the duplication itself.

It’s that eventually you can’t easily answer:

What exactly is this model being told in this particular configuration?

The opposite approach is usually one giant template full of conditionals.

{% if interface == "chat" %}
...
{% endif %}

{% if feature_x_enabled %}
...
{% endif %}

That removes some duplication, but after enough conditions the prompt becomes difficult to reason about.

You haven’t really removed the complexity.

You’ve moved it.

prompts started looking more like source code

At some point, treating these prompts as strings stopped making much sense to me.

They had:

shared modules
configuration
conditional behaviour
variables
build-time transformations
multiple final outputs

That sounds much closer to source code than a text file.

Once I started looking at it that way, the architecture became fairly straightforward:

┌───────────┐     ┌──────────┐     ┌─────────────────┐
│ fragments │ ──▶ │ compiler │ ──▶ │ compiled prompt │
└───────────┘     └──────────┘     └─────────────────┘

Instead of maintaining complete prompts for every combination, you maintain smaller pieces.

For example:

base
interface
feature
deployment

The compiler picks the relevant fragments, combines them in a defined order, resolves build-time configuration, and produces one flat prompt.

Something like:

base
+ terminal
+ code-review
─────────────────────────────
compiled/code-review-terminal

The important part is that the application doesn’t need to understand how this composition works.

By runtime, it just receives a normal prompt.

That made the whole thing much easier to reason about.

build-time and runtime variables need separate boundaries

One small problem appeared quickly.

There were two completely different kinds of substitution happening.

Some values should be resolved when generating the prompt:

{{product_name}}

Others only exist when the actual model call happens:

<<user_request>>

So the distinction became:

{{...}}   → build time
<<...>>   → runtime

The {{...}} syntax here wasn’t an arbitrary choice.

The prompts in this setup were stored in Langfuse and pulled through Langfuse, where prompt variables already use that format. Keeping the same representation made the integration much simpler than inventing another syntax around it.

The second delimiter, <<...>>, gives runtime variables their own namespace.

It looks like a tiny distinction, but I like it quite a lot.

When reading a prompt, you immediately know which stage owns a variable.

More importantly, the generation step can resolve everything it knows about while deliberately leaving runtime values untouched.

Undefined values should also fail loudly wherever possible.

If a required build-time value doesn’t exist, I’d rather fail while generating the prompt than quietly produce something broken and discover it during an actual model call.

the compiled prompt is worth keeping

This was probably the less obvious part for me.

Generated files are usually things we throw away.

For prompts, keeping them can actually be useful.

Suppose you change one shared fragment.

That might affect fifteen compiled prompt variants.

If those outputs are treated as golden files, the change becomes much easier to inspect.

Instead of reviewing:

changed one shared instruction

you’re reviewing:

these 15 actual prompts changed because of it

Suddenly the blast radius is visible.

Reviewers don’t need to mentally execute a template system to figure out what the model will eventually receive.

They can just read the final prompt.

And because the output is plain text, normal software tooling works surprisingly well here.

Diffs. History. Reviews. CI.

Nothing particularly fancy.

a small example

Imagine this base fragment:

You are a coding assistant.

Explain important assumptions before suggesting changes.

Then an interface-specific fragment:

Keep responses concise because this interface has limited space.

And a feature fragment:

When reviewing code, identify correctness issues before style issues.

Project name: {{project_name}}

Current diff:
<<code_diff>>

During generation, {{project_name}} is resolved:

You are a coding assistant.

Explain important assumptions before suggesting changes.

Keep responses concise because this interface has limited space.

When reviewing code, identify correctness issues before style issues.

Project name: Example Project

Current diff:
<<code_diff>>

<<code_diff>> survives because that value only exists at runtime.

That’s the final prompt I actually care about.

So that’s also the thing I want to inspect and test.

where the generated prompts live is a separate decision

In the system that led me here, the generated prompts were pushed back to Langfuse and organised using labels.

That worked well for the surrounding setup, but it isn’t really a requirement of this architecture.

The compiler’s job ends once it produces the final prompt.

After that, you could store the output in a database, keep it in memory, write static text files, bundle it directly with the application, or push it to whatever prompt management system you’re already using.

I think separating those two concerns is useful.

How prompts are built and where prompts are stored don’t need to be the same problem.

this is not really about building a prompt framework

There are obvious tradeoffs.

Fragment precedence has to be well defined.

Too many layers can become difficult to follow.

Some combinations should probably be invalid and rejected during generation.

And keeping generated outputs around can create some noise.

But I think that’s acceptable if the alternative is reconstructing the final prompt mentally every time something behaves strangely.

The useful mental shift for me was fairly simple:

don’t treat a growing prompt system as a collection of strings.

Treat it as something that gets built.

Fragments are source.

The compiler resolves configuration.

The compiled prompt is the artifact.

And the artifact is what you inspect.

I’m also working on turning the general idea into a small reusable open-source template for similar problems. There are still a few things I want to clean up before that, but if it becomes useful enough outside this specific setup, I might write a follow-up with the actual implementation.