PromptJIT

The Story

Every production AI system today shares one universal, slightly embarrassing property: the ten-thousandth request costs exactly as many tokens and exactly as much money as the first. The system never learns that it has seen this kind of request before. It pays full tuition for the same lesson, forever.

Java solved this problem decades ago. HotSpot made Java fast by noticing what runs often and compiling it down. PromptJIT makes AI cheap the same way — except what gets compiled is thought.

In PromptJIT, CLOUD_ADVANCED is the optimising compiler, LOCAL is the CPU, and an autoboxing failure is a deoptimisation trap. The cost of serving a request decays as a function of how often the system has seen its kind — asymptotically approaching local inference, and ultimately plain Java.

How It Is Built

The analogy is not decoration; it is a structural correspondence, and every component exists in PrOOPt today:

• The profiler — the plan cache with PlanCacheStrategy.INTENT clusters requests by intent, not bytes, and its hit statistics reveal the hot paths of cognition.
• The compiler — a CLOUD_ADVANCED function (with thinking enabled) studies successful traces of a hot intent and writes a rule-prompt that a small local model can follow.
• The promotion gate — a deterministic @CodeFunction replays the candidate rule against a golden set and promotes it only above a pass-rate threshold.
• The deopt trap — if the local tier's output fails autoboxing or schema validation, the request falls back up the ladder and the rule goes back to the shop.

@PromptOrchestrator(
        prompt = "Resolve the customer-operations request using registered tools.",
        planMode = PlanMode.STATIC,
        planCacheStrategy = PlanCacheStrategy.INTENT,
        planCacheSize = 2000,
        planCacheTtl = -1,
        allowDynamic = true,
        dynamicFunctionModel = ModelTier.CLOUD_FAST,
        maxDynamicFunctions = 3
)
public Resolution handle(String request) { return null; }

public class PromptJitCompiler {

    @PromptFunction(model = ModelTier.CLOUD_ADVANCED, thinking = true,
            prompt = "You are an optimising compiler. For hot intent {profile}, "
                   + "study these successful cloud-handled traces: {exemplars}. "
                   + "Write a rule-prompt that a small local model can follow...")
    public String compileRuleForLocal(IntentProfile profile, String exemplars,
                                      String deoptCases) { return null; }

    @CodeFunction(description = "Run the compiled rule against the golden "
            + "replay set on the LOCAL tier; promote only if pass-rate >= 0.98")
    public PromotionVerdict verifyAndPromote(String candidateRule, String intent) { /* … */ }
}

Why It Is Unprecedented

Vendor prompt caching caches tokens. PromptJIT caches and recompiles strategy — what to do, in what order, with which tools, under which rules. Distillation transfers capability by gradient descent over weeks; PromptJIT transfers it by writing legible instructions in minutes — reviewable in a diff, reversible by cache eviction. It is distillation without gradient descent, and the cost-decay curve falls straight out of the audit log you were already keeping. The endgame is lovely: when a compiled rule becomes stable enough, it is a candidate for transcription into an actual @CodeFunction. Cognition crystallises into software.

The Fine Print

The ladder's bottom rungs only fit recurring, structured work; genuinely novel and open-ended requests will live at cloud tiers forever, and that is fine — the promise is about the distribution's fat head. Compiled rules inherit the local model's ceiling, so the promotion gate, not optimism, decides what they carry. And intent drift is the moral equivalent of profile pollution: drift detection and evidence-based eviction are mandatory, not optional.