Prompt Engineering & Management
Prompt engineering tools treat prompts as versioned, testable artifacts: playgrounds to iterate, version control and deployment separate from code, evaluation to catch regressions, and collaboration so non-engineers can contribute. Many overlap with observability platforms that also manage prompts.
| Agenta Open-source LLMOps: prompt playground, evals, and observability. | Open source (MIT) | Managed, Self-hosted | Open source (free) | Yes | Yes | Yes | Yes |
| Braintrust Eval-first platform for iterating on LLM quality. | Proprietary | Managed, Hybrid | Free tier + paid | — | Yes | — | — |
| Langfuse Open-source LLM observability, evals, and prompt management. | Open source (MIT core (some enterprise features separate)) | Managed, Self-hosted | Open source (free) | — | Yes | — | — |
| LangSmith Tracing, evals, and prompt management from the LangChain team. | Proprietary | Managed, Hybrid | Free tier + paid | — | Yes | — | — |
| Latitude Open-source prompt engineering and evaluation platform. | Open source (LGPL-3.0) | Managed, Self-hosted | Open source (free) | Yes | Yes | Yes | Yes |
| Mirascope Pythonic library for prompts as typed functions. | Open source (MIT) | Self-hosted | Open source (free) | Via git | No | No | No |
| Portkey AI gateway with observability, guardrails, and prompt management. | Open source (Apache-2.0 (gateway core); platform is managed) | Managed, Self-hosted | Free tier + paid | — | — | — | — |
| Promptfoo Open-source, CLI-first prompt testing and LLM evals. | Open source (MIT) | Self-hosted | Open source (free) | Via git | Yes | Yes | No |
| PromptLayer Prompt management, versioning, and evaluation platform. | Proprietary | Managed | Free tier + paid | Yes | Yes | Yes | Yes |
| Vellum Development platform for prompts, workflows, and evals. | Proprietary | Managed | Tiered | Yes | Yes | Yes | Yes |
Compiled from public docs and vendor sites; verify current pricing with the vendor. No vendor pays for placement. Each tool links to a fuller profile.
How to choose
If you want prompts decoupled from deploys with a UI for non-engineers, look at PromptLayer, Vellum, or Latitude. For CI-friendly, code-first evaluation use Promptfoo or Mirascope. Open-source options (Latitude, Agenta, Promptfoo) let you self-host. Teams already using an observability tool may get prompt management bundled in.
FAQ
Why manage prompts outside my codebase?
So you can iterate, A/B test, and roll back prompts without a full deploy, and let product or domain experts edit them safely with versioning and evals as guardrails.
What is prompt evaluation?
Running a prompt against a dataset of inputs and scoring the outputs — with assertions, model graders, or human review — to measure quality and catch regressions before release.
Do these replace observability tools?
They overlap. Some observability platforms include prompt management; dedicated prompt tools go deeper on iteration, versioning, and evaluation workflows.