Production AI agents
Teams that need to see multi-step reasoning, retrieval, tool calls, cost, latency, failures, and user outcomes in one trace.
Independent tool overview
LangWatch at a glance
LangWatch is an open-core LLMOps platform for testing, evaluating, observing, and governing AI applications and agents. It combines OpenTelemetry-native tracing, multi-turn agent simulations, offline and production evaluations, prompt versioning, red teaming, an AI gateway, and cloud, hybrid, or self-hosted deployment. The free cloud tier is useful for a small evaluation; Growth starts at €29 per core seat per month and adds event and extended-retention charges. LangWatch can improve evidence and debugging, but an LLM judge score is not proof of safety or correctness.
Visit the official LangWatch site ↗
- Product type
- LLMOps, AI-agent testing, evaluation, observability, prompt management, and governance platform
- Instrumentation
- OpenTelemetry-native tracing plus Python, TypeScript, Go, framework, API, and LiteLLM options
- Testing
- Multi-turn text and voice scenarios, CI runs, red teaming, mocked tools, and trace-level judges
- Evaluation
- Code metrics, model judges, custom workflows, datasets, human annotation, pairwise and online evaluation
- Deployment
- Managed cloud, hybrid, self-hosted Docker, Kubernetes or Helm, and on-premises options
- Cloud regions
- LangWatch advertises EU, US, UK, and APAC cloud availability; confirm the contracted region and subprocessors
- Open-source license
- Apache 2.0 core, MIT SDKs and MCP, with commercially licensed enterprise modules
- Free cloud plan
- 50,000 events per month, 14-day data access, two users, and limited scenarios, simulations, and custom evals
- Growth price
- €29 per core seat monthly, 200,000 included events, then €5 per 100,000 events
- Security posture
- ISO 27001 certified and GDPR compliance claimed; enterprise reports and DPA are available through security review
- Reviewed
- August 31, 2026 from current pricing, product, documentation, trust, privacy, and source-code materials
Overview
What LangWatch is
LangWatch has expanded well beyond its original broad promise to help teams build AI safely. The current product is an engineering platform for the full AI-agent quality loop: instrument production behavior, turn requirements and failures into repeatable tests, run simulations, compare versions, evaluate outputs, manage prompts, monitor cost and latency, and govern model access.
Its observability layer captures LLM calls, tool calls, retrieval, user interactions, tokens, cost, timing, and metadata as traces and spans. LangWatch is OpenTelemetry-native and supports framework-specific SDKs plus APIs and proxy-based collection, making it relevant to teams that do not want their telemetry trapped in a single agent framework.
Agent testing is the differentiator. Scenario tests can simulate multi-turn text or voice users, mock or fixture tools, run locally and in CI, probe unsafe behavior, and inspect the full trace. LangWatch also offers Langy, an automated workflow that can draft a test plan from a product goal and propose prompt changes through pull requests. Those proposals still need code, security, and domain review.
Evaluation can use deterministic code, human annotation, model-based judges, datasets, pairwise comparisons, online scoring, and custom workflows. This flexibility matters because no single metric catches every failure. It also creates a governance requirement: prompts, judge models, rubrics, sampling, thresholds, and ground truth must be versioned and validated like production code.
LangWatch Cloud has a useful free tier and transparent event-based pricing. The platform can also run on customer infrastructure with PostgreSQL, ClickHouse, Redis, object storage, application workers, and evaluation services. Most of the repository is Apache 2.0, SDKs are MIT, and enterprise modules such as SCIM and audit logs require a commercial license for production.
Use cases
Who LangWatch is best for
The strongest fit depends on the job you need the product to complete, not the size of its feature list.
Regression testing
Engineering teams that want repeatable multi-turn scenarios to run locally and in CI before prompt, model, tool, or policy changes ship.
Cross-functional evaluation
Product managers and domain experts who need to define behavior and review quality without living entirely in notebooks.
Voice-agent testing
Teams that need simulated voice conversations and adversarial cases before customer-facing calls.
OpenTelemetry environments
Organizations standardizing telemetry across application and AI workloads and trying to reduce framework-specific lock-in.
Controlled deployment
Regulated or data-sensitive organizations evaluating cloud, hybrid, or self-hosted observability with enterprise identity and audit controls.
Capabilities
Core LangWatch features
Agent traces
Captures conversations, LLM calls, tool and MCP activity, retrieval, spans, tokens, timing, errors, costs, metadata, and evaluation results.
OpenTelemetry support
Uses GenAI-oriented OTel tracing so data can arrive from supported frameworks, SDKs, Go, custom setups, and compatible pipelines.
Scenario testing
Runs repeatable multi-turn simulated users against an agent through white-box or black-box interfaces.
Voice simulations
Exercises voice agents with simulated conversations before live customer traffic, subject to the selected speech and judge systems.
Red teaming
Generates adversarial interactions for jailbreaks, policy violations, unsafe content, data exposure, and risky tool behavior.
Offline experiments
Runs prompts, models, configurations, or pipelines across datasets and compares quality, latency, and cost.
Online evaluations
Scores sampled or selected production interactions and makes results available for monitoring and investigation.
Configurable evaluators
Supports deterministic metrics, custom code, LLM-as-a-judge, workflow evaluators, safety checks, RAG metrics, and human annotations.
Prompt management
Versions prompts across code, UI, API, templates, a playground, deployments, A/B tests, and GitHub synchronization.
Trace analytics
Provides saved views, natural-language search, topic clustering, waterfall, flame, topology, and sequence views plus custom charts.
AI governance
Offers virtual provider keys, budgets, routing policies, cost attribution, and audit-oriented controls depending on edition.
Langy automation
Turns a plain-language product goal into proposed scenarios and judge criteria, then can draft prompt-change pull requests for human review.
Flexible hosting
Supports managed multi-tenant cloud, hybrid data-plane patterns, Docker evaluation, and production Kubernetes or Helm deployments.
Process
How the LangWatch workflow works
Step 1
Define product behavior
Write explicit requirements for helpfulness, task completion, refusal, escalation, privacy, latency, cost, and tool authorization before choosing metrics.
Step 2
Classify sensitive telemetry
Map prompts, responses, retrieved documents, tool arguments, audio, files, identifiers, secrets, regulated data, and customer consent.
Step 3
Choose the deployment boundary
Select cloud, hybrid, or self-hosted based on data flow, region, retention, operational capability, certifications, and subprocessor requirements.
Step 4
Instrument a narrow path
Trace one representative workflow first, preserve correlation IDs and user outcomes, and avoid collecting fields that are not needed for debugging.
Step 5
Redact before export
Remove secrets and sensitive data in the application or collector before telemetry leaves the trust boundary; validate redaction against realistic payloads.
Step 6
Build an incident dataset
Convert confirmed failures, edge cases, policy violations, and successful counterexamples into a versioned regression set with expected behavior.
Step 7
Use multiple evaluators
Combine deterministic assertions, tool-state checks, human labels, domain rules, and calibrated model judges rather than trusting a single score.
Step 8
Calibrate every judge
Compare judge decisions with qualified human reviewers by segment, measure disagreement and false-pass rates, and version model, prompt, rubric, and threshold.
Step 9
Simulate agent behavior
Run ordinary, ambiguous, adversarial, multilingual, long-context, tool-failure, and authorization scenarios through the whole workflow.
Step 10
Gate the release
Require material regressions to block deployment, but keep manual review for high-impact changes and never let an automated judge approve its own safety standard.
Step 11
Monitor production carefully
Track quality, outcomes, refusal errors, latency, spend, tool failures, drift, and incidents with sampling designed for rare but severe failures.
Step 12
Review automated changes
Treat Langy-generated tests and pull requests as untrusted proposals; review code, prompts, data exposure, policy behavior, and benchmark leakage before merging.
Step 13
Audit cost and retention
Monitor billable events per interaction, evaluator fan-out, storage beyond 30 days, core seats, external model cost, and whether retained traces still have a purpose.
Cost
LangWatch pricing and free plan
LangWatch Cloud combines seats, events, and retained storage. Developer is free with 50,000 monthly events and 14-day access. Growth is €29 per core seat monthly, includes 200,000 events and 30-day retention, then charges €5 per additional 100,000 events and €3 per GB kept beyond 30 days. One user interaction can create many events because each LLM call, tool call, retrieval, evaluation, or simulation step counts. Enterprise, supported self-hosting, and advanced governance are custom.
Developer
€0 forever
For individual developers and small proofs of concept in LangWatch Cloud.
- 50,000 events per month
- 14-day data access
- Two users
- Three scenarios, three simulations, and three custom evaluations
- Community support through GitHub and Discord
- No credit card required
Growth
€29/core seat/month
For teams testing and monitoring production AI applications.
- 200,000 events included monthly
- €5 per additional 100,000 events
- 30-day retention included
- €3 per GB for data retained beyond 30 days
- Unlimited lite users
- Unlimited simulations, evaluations, and prompts
- Private Slack or Microsoft Teams support
- Volume discounts advertised above 20 users
Enterprise
Custom quote
For regulated, higher-scale, hybrid, self-hosted, or on-premises deployments.
- Negotiated usage pricing and custom retention
- Hybrid, self-hosted, and on-premises options
- Custom SSO and RBAC
- Audit logs, SLAs, and security review
- ISO 27001 report and DPA access
- Custom terms and forward-deployed engineering
- Marketplace billing options
Community self-managed
Open-source core; infrastructure not included
For teams able to operate LangWatch on their own systems without enterprise-only production modules.
- Most core code is Apache 2.0 and SDKs are MIT
- Enterprise modules such as SCIM, audit logs, license, and billing management require a commercial license for production
- Customer operates application workers, PostgreSQL, ClickHouse, Redis, object storage, backups, TLS, monitoring, and upgrades
- Docker Compose is documented for evaluation and small teams but lacks built-in high availability, TLS, and automated backups
- Production Kubernetes requires material infrastructure and operations capacity
- External LLM and embedding provider charges remain separate
Pricing checked . Check current pricing at the source ↗
Assessment
LangWatch strengths and limitations
Where it stands out
- Unifies agent testing, evaluations, observability, prompt management, and governance in one workflow
- Multi-turn scenarios exercise stateful agents more realistically than isolated prompt tests
- Full-trace evaluation can include retrieval and tool behavior instead of scoring only the final sentence
- OpenTelemetry-native instrumentation reduces dependence on one application framework
- Supports deterministic checks, human review, code, model judges, and custom workflows
- Scenarios can run locally and in CI for regression control before deployment
- Cloud, hybrid, and self-hosted options support different data-boundary requirements
- Free plan is large enough to evaluate a modest instrumented application
- Current pricing publishes the seat, event, and extended-retention units clearly
- Open-core repository provides source visibility and a usable Apache 2.0 community core
- Trace visualizations and search help engineers diagnose complex multi-tool runs
- Prompt versioning and GitHub sync can connect evaluation evidence to change review
- Trust center documents encryption, identity, backup, recovery, and privacy controls
What to consider
- LangWatch adds instrumentation, test design, evaluator maintenance, sampling, data governance, and operational work; installing an SDK does not create a quality program
- Every LLM call, tool call, retrieval, evaluation, and simulation step can be billable, so a single user request may create many events
- Model-based evaluations also incur external model cost and can create latency, privacy, and availability dependencies
- LLM judges can prefer verbosity or familiar model styles, miss subtle factual errors, and share correlated blind spots with the system under test
- A high aggregate score can hide rare but severe safety, privacy, authorization, financial, medical, or legal failures
- Red-team simulations do not prove security; generated attacks are limited by the simulator, model, scope, and threat assumptions
- Synthetic users do not perfectly reproduce real human goals, accents, emotions, adversarial creativity, or production context
- Topic clustering, AI search, trace summaries, and Langy plans are generated analyses that require verification
- Automatically drafted prompt changes can optimize to a narrow benchmark, break unmeasured segments, or introduce unsafe tool behavior
- Production traces may contain prompts, responses, files, audio, personal data, retrieved secrets, API arguments, and proprietary business context
- Automatic PII detection is not a guarantee; secrets and sensitive fields should be minimized or redacted before telemetry export
- The general privacy notice says it does not process sensitive data, but an observability customer's traces can contain sensitive customer content; the processor contract and telemetry design must govern that reality
- Cloud retention is short on lower tiers, while longer retention adds storage cost and privacy exposure
- Self-hosting moves security, backups, upgrades, capacity, TLS, secrets, incident response, and availability to the customer
- The documented self-hosted stack includes several stateful components and outbound evaluator providers, so 'nothing leaves' depends on configuration and enabled features
- Enterprise modules are not Apache 2.0 and require a commercial license for production use
- LangWatch Cloud is multi-tenant; buyers should verify certification scope, penetration tests, tenant-isolation controls, contracted region, subprocessors, and deletion
- Growth pricing is per core seat plus events plus extended retention, which complicates forecasting across high-fan-out agents and always-on evaluations
- Framework integrations and OpenTelemetry schemas evolve quickly and can require SDK upgrades or mapping changes
- Observability can encourage overcollection; teams need field allowlists, role controls, audit logs, export restrictions, retention limits, and deletion tests
- No evaluation platform can decide the acceptable risk of a consequential product without accountable human owners and domain-specific evidence
Compare
LangWatch alternatives
The right alternative depends on the specific output, workflow, controls and budget your project requires.
Coding
Helicone
A focused LLM observability and gateway alternative for teams prioritizing request logging, cost, latency, caching, routing, and operational analytics over LangWatch's broader simulation-testing workflow.
Explore Helicone →Questions
LangWatch FAQs
What is LangWatch?
LangWatch is an open-core platform for tracing, testing, evaluating, and governing LLM applications and AI agents. It combines production observability with datasets, model judges, agent simulations, prompt management, and deployment controls.
Is LangWatch open source?
Mostly. LangWatch says its core is Apache 2.0 and its SDKs and MCP server are MIT. Enterprise modules such as SCIM, audit logs, and license or billing management have a separate commercial license for production use.
Is LangWatch free?
Yes. The cloud Developer plan includes 50,000 events per month, 14-day data access, two users, and limited scenarios, simulations, and custom evaluations. A community self-managed core is also available, but infrastructure and operations are not free.
How much does LangWatch cost?
As checked August 31, 2026, Growth costs €29 per core seat monthly, includes 200,000 events and 30-day retention, then charges €5 per 100,000 events and €3 per GB retained beyond 30 days. Enterprise is custom.
What counts as a LangWatch event?
Each LLM call, tool call, retrieval, evaluation, or simulation step counts. A single end-user interaction can therefore produce several billable events, especially for agents and online evaluations.
What does LangWatch observe?
Depending on instrumentation, it can capture traces and spans for conversations, model calls, retrieval, tools, MCP, tokens, cost, latency, errors, metadata, and evaluation results. Teams should deliberately minimize and redact telemetry.
What is an LLM-as-a-judge evaluation?
It asks a model to score or classify another system's output against a rubric. It is scalable but fallible, so calibrate it against qualified humans and pair it with deterministic, state-based, and domain-specific checks.
Can LangWatch test AI agents?
Yes. It can run multi-turn text or voice scenarios, inspect full traces, mock or fixture tools, run tests locally and in CI, and perform generated red-team exercises.
Can LangWatch guarantee that an agent is safe?
No. It can reveal failures and provide repeatable evidence, but test coverage, synthetic users, evaluators, judges, and production sampling all have blind spots. High-impact systems still need security testing, domain review, monitoring, and incident response.
Can LangWatch be self-hosted?
Yes. Docker Compose is documented for evaluation and small teams; Kubernetes or Helm is the production path. Customers must operate the application, workers, PostgreSQL, ClickHouse, Redis, object storage, secrets, TLS, backups, monitoring, and upgrades.
Does self-hosting keep all data inside my network?
The core data plane can remain on customer infrastructure, but enabled model-based evaluators can call external providers. Review every outbound integration, telemetry setting, object store, support workflow, and hybrid control-plane connection.
Is LangWatch secure?
LangWatch advertises ISO 27001, GDPR compliance, encryption, MFA, role controls, backups, incident response, and enterprise SSO, SCIM, and audit options. Verify the current reports, scope, region, architecture, pen test, DPA, subprocessors, and customer configuration.
What is Langy?
Langy is LangWatch's automated AI-engineering workflow for turning product goals and traces into proposed tests and prompt-change pull requests. Treat its outputs as untrusted drafts that require engineering, security, and domain review.
Bottom line
Our LangWatch verdict
LangWatch is a strong fit for teams that need more than request logs: its multi-turn scenarios, full-trace evaluation, prompt versioning, OpenTelemetry support, and flexible deployment can connect a production failure to a repeatable test and reviewed change. The value depends on rigorous implementation. Minimize telemetry, calibrate judges, preserve human approval, price event fan-out and external model calls, and validate the self-hosted or cloud boundary. Use the platform to produce better evidence—not to replace accountable decisions with a dashboard score.
Visit LangWatch website ↗Similar Data Analysis Tools
View all Data Analysis AI tools →
AI training for the future of work.
Get access to all our AI courses, hundreds of real-world AI use cases, live expert-led workshops, an exclusive network of AI early adopters, and more.
AI Courses
Get unlimited access to all of our current & upcoming industry-specific AI courses for the duration of your subscription.
Daily Guides
To keep up with the rapid pace of AI, our team publishes AI implementation guides daily. Our library contains 300+ practical use cases to automate real-world work.
Workshops
Join weekly, live, interactive sessions with industry leaders who are at the forefront of AI for hands-on implementation guidance and exclusive insights.
Community
Network with an exclusive community of AI-first professionals who are working smarter with AI. Learn how early adopters are using AI in their work and businesses.