Prompt versioning

Register prompt template versions, link them to traces, and compare performance metrics side-by-side to understand which prompt produces the best outcomes.

Prompt versioning lets you track which prompt template produced which agent outcomes. As you iterate on your prompts, you register each version in AgentVista, link it to the traces it produces, and then compare versions side-by-side on success rate, cost, and latency.

Prompt template fields

Each prompt template version has the following fields:

Field	Type	Description
name	string	Identifier for the prompt, shared across all versions (e.g., "support-classifier")
version	integer	Version number within the named prompt (e.g., 1, 2, 3)
template_text	string	The full text of the prompt template
description	string (optional)	Human-readable notes about what changed in this version
is_active	boolean	Whether this version is the current default

The combination of name and version must be unique per account. If you try to register a duplicate, the API returns HTTP 409.

Workflow

1. Register a prompt version

   Call POST /api/v1/dashboard/prompts/ to register a new version of your prompt:

   import httpx
   
   response = httpx.post(
       "https://api.agentvista.dev/api/v1/dashboard/prompts/",
       headers={"Authorization": "Bearer av_xxxxx"},
       json={
           "name": "support-classifier",
           "version": 1,
           "template_text": "You are a support classifier. Categorize the following ticket: {ticket_text}",
           "description": "Initial version — direct classification instruction"
       }
   )
   # Returns HTTP 201 with the created prompt template
   print(response.json())
   

   When you ship an updated prompt, register it as the next version:

   response = httpx.post(
       "https://api.agentvista.dev/api/v1/dashboard/prompts/",
       headers={"Authorization": "Bearer av_xxxxx"},
       json={
           "name": "support-classifier",
           "version": 2,
           "template_text": "You are an expert support classifier. Think step by step before categorizing: {ticket_text}",
           "description": "v2 — added chain-of-thought instruction"
       }
   )
   
2. Run your agent

   Run your agent as normal. Your existing SDK instrumentation captures the trace automatically. Make a note of the trace_id for each run so you can link it to the prompt version that produced it.

   If you are using the AgentVista Python SDK, the trace_id is available on the completed run object.

3. Record prompt usage

   After each trace completes, call POST /api/v1/dashboard/prompts/usage to link the trace to the prompt version that produced it:

   response = httpx.post(
       "https://api.agentvista.dev/api/v1/dashboard/prompts/usage",
       headers={"Authorization": "Bearer av_xxxxx"},
       json={
           "prompt_name": "support-classifier",
           "prompt_version": 2,
           "trace_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
       }
   )
   # Returns HTTP 201: {"status": "recorded"}
   

   This call is idempotent — submitting the same (prompt_name, prompt_version, trace_id) combination more than once is safe.

   You can optionally include a span_id to link to a specific span within the trace rather than the trace header.

4. Compare versions in the dashboard

   Once you have traces linked to multiple versions, open the prompt detail view in the dashboard. You will see a side-by-side table with per-version stats.

   You can also fetch comparison data directly from the API:

   response = httpx.get(
       "https://api.agentvista.dev/api/v1/dashboard/prompts/support-classifier/compare",
       headers={"Authorization": "Bearer av_xxxxx"}
   )
   print(response.json())
   

   Example response:

   {
     "name": "support-classifier",
     "versions": [
       {
         "version": 1,
         "template_text": "You are a support classifier...",
         "total_traces": 120,
         "success_rate": 0.72,
         "avg_cost_usd": 0.0031,
         "avg_latency_ms": 1840,
         "p95_latency_ms": 3200
       },
       {
         "version": 2,
         "template_text": "You are an expert support classifier...",
         "total_traces": 95,
         "success_rate": 0.89,
         "avg_cost_usd": 0.0048,
         "avg_latency_ms": 2210,
         "p95_latency_ms": 4100
       }
     ]
   }
   

   In this example, version 2 achieves a 17-point higher success rate at the cost of a slightly higher average latency and per-run cost.

Viewing all versions

To list all registered versions of a specific prompt, call GET /api/v1/dashboard/prompts/{name}/versions. To list all prompts across all names, call GET /api/v1/dashboard/prompts/. Versions are returned newest first within each name.

Per-version metrics

The comparison endpoint computes the following metrics for each version, derived from all traces linked to that version:

Metric	Description
total_traces	Number of traces linked to this version
success_rate	Fraction of linked traces where success = true
avg_cost_usd	Mean total_cost_usd across linked traces
avg_latency_ms	Mean duration_ms across linked traces
p95_latency_ms	95th percentile duration_ms across linked traces

Metrics are computed from the success, total_cost_usd, and duration_ms fields on traces linked to each prompt version. Make sure your SDK is recording outcome signals and cost data on each run — otherwise success_rate and cost metrics will be null.