Process & Understanding¶
Tools for observability, evaluation, data extraction, and understanding agentic workflows and LLM performance.
Contents¶
| Tool | What it does |
|---|---|
| AgentOps | Observability platform for agent monitoring and debugging |
| Arize AI | LLM observability and evaluation platform (Phoenix) |
| Braintrust | Enterprise-grade tool for eval, proxy, and logging |
| ClickHouse | OLAP database for high-performance log analytics |
| Comet Opik | Open-source platform for LLM evaluation and observability |
| Crawl4AI | Web crawling and data extraction for LLMs |
| Datadog | Cloud-scale observability and security for AI apps |
| Docling | Layout-aware document parsing and extraction |
| Docling MCP | Model Context Protocol server for Docling |
| Fiddler AI | Enterprise model monitoring and guardrails |
| Firecrawl | Turn websites into LLM-ready Markdown |
| Grafana Cloud | Managed observability for metrics, logs, and traces |
| Helicone | Open-source LLM observability proxy |
| Langfuse | Open-source observability and analytics for LLM apps |
| LastMile AI | Evaluation and guardrails for generative AI |
| New Relic AI | AI-powered observability for full-stack monitoring |
| Ocrmypdf | Adds OCR text layer to scanned PDF files |
| OpenDataLoader PDF | PDF extraction and normalization tool |
| OpenTelemetry Collector | Telemetry data processing and export |
| PageIndex | Contextual search and indexing for web pages |
| Parea | Observability and evaluation platform |
| PostHog | Product analytics and session replay for AI apps |
| Ragas | Framework for reference-free evaluation of RAG pipelines |
| RAGFlow | Deep document understanding based RAG engine |
| Sentry | Application monitoring and error tracking |
| Snowflake | Data warehouse for AI and analytics workloads |
| Tesseract CLI | Open-source OCR engine |
| W&B Weave | Observability and evaluation for agentic workflows |
| Webhook | Standardized ingestion and event handling |