mirror of https://github.com/github/awesome-copilot.git synced 2026-04-11 02:35:55 +00:00

Files

Jim Bennett d79183139a Add Arize and Phoenix LLM observability skills (#1204 )

* Add 9 Arize LLM observability skills

Add skills for Arize AI platform covering trace export, instrumentation,
datasets, experiments, evaluators, AI provider integrations, annotations,
prompt optimization, and deep linking to the Arize UI.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add 3 Phoenix AI observability skills

Add skills for Phoenix (Arize open-source) covering CLI debugging,
LLM evaluation workflows, and OpenInference tracing/instrumentation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Ignoring intentional bad spelling

* Fix CI: remove .DS_Store from generated skills README and add codespell ignore

Remove .DS_Store artifact from winmd-api-search asset listing in generated
README.skills.md so it matches the CI Linux build output. Add queston to
codespell ignore list (intentional misspelling example in arize-dataset skill).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add arize-ax and phoenix plugins

Bundle the 9 Arize skills into an arize-ax plugin and the 3 Phoenix
skills into a phoenix plugin for easier installation as single packages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fix skill folder structures to match source repos

Move arize supporting files from references/ to root level and rename
phoenix references/ to rules/ to exactly match the original source
repository folder structures.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fixing file locations

* Fixing readme

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-04-02 09:58:55 +11:00

6.6 KiB

Raw Permalink Blame History

name, description, license, compatibility, metadata

name

description

license

compatibility

metadata

phoenix-tracing

OpenInference semantic conventions and instrumentation for Phoenix AI observability. Use when implementing LLM tracing, creating custom spans, or deploying to production.

Apache-2.0

Requires Phoenix server. Python skills need arize-phoenix-otel; TypeScript skills need @arizeai/phoenix-otel.

author	version	languages
oss@arize.com	1.0.0	Python, TypeScript

Phoenix Tracing

Comprehensive guide for instrumenting LLM applications with OpenInference tracing in Phoenix. Contains reference files covering setup, instrumentation, span types, and production deployment.

When to Apply

Reference these guidelines when:

Setting up Phoenix tracing (Python or TypeScript)
Creating custom spans for LLM operations
Adding attributes following OpenInference conventions
Deploying tracing to production
Querying and analyzing trace data

Reference Categories

Priority	Category	Description	Prefix
1	Setup	Installation and configuration	`setup-*`
2	Instrumentation	Auto and manual tracing	`instrumentation-*`
3	Span Types	9 span kinds with attributes	`span-*`
4	Organization	Projects and sessions	`projects-`, `sessions-`
5	Enrichment	Custom metadata	`metadata-*`
6	Production	Batch processing, masking	`production-*`
7	Feedback	Annotations and evaluation	`annotations-*`

Quick Reference

1. Setup (START HERE)

setup-python - Install arize-phoenix-otel, configure endpoint
setup-typescript - Install @arizeai/phoenix-otel, configure endpoint

2. Instrumentation

instrumentation-auto-python - Auto-instrument OpenAI, LangChain, etc.
instrumentation-auto-typescript - Auto-instrument supported frameworks
instrumentation-manual-python - Custom spans with decorators
instrumentation-manual-typescript - Custom spans with wrappers

3. Span Types (with full attribute schemas)

span-llm - LLM API calls (model, tokens, messages, cost)
span-chain - Multi-step workflows and pipelines
span-retriever - Document retrieval (documents, scores)
span-tool - Function/API calls (name, parameters)
span-agent - Multi-step reasoning agents
span-embedding - Vector generation
span-reranker - Document re-ranking
span-guardrail - Safety checks
span-evaluator - LLM evaluation

Reference Files

fundamentals-overview - Traces, spans, attributes basics
fundamentals-required-attributes - Required fields per span type
fundamentals-universal-attributes - Common attributes (user.id, session.id)
fundamentals-flattening - JSON flattening rules
attributes-messages - Chat message format
attributes-metadata - Custom metadata schema
attributes-graph - Agent workflow attributes
attributes-exceptions - Error tracking

Common Workflows

Quick Start: setup-{lang} → instrumentation-auto-{lang} → Check Phoenix
Custom Spans: setup-{lang} → instrumentation-manual-{lang} → span-{type}
Session Tracking: sessions-{lang} for conversation grouping patterns
Production: production-{lang} for batching, masking, and deployment

How to Use This Skill

Navigation Patterns:

# By category prefix
references/setup-*              # Installation and configuration
references/instrumentation-*    # Auto and manual tracing
references/span-*               # Span type specifications
references/sessions-*           # Session tracking
references/production-*         # Production deployment
references/fundamentals-*       # Core concepts
references/attributes-*         # Attribute specifications

# By language
references/*-python.md          # Python implementations
references/*-typescript.md      # TypeScript implementations

Reading Order:

Start with setup-{lang} for your language
Choose instrumentation-auto-{lang} OR instrumentation-manual-{lang}
Reference span-{type} files as needed for specific operations
See fundamentals-* files for attribute specifications

References

Phoenix Documentation:

Python API Documentation:

Python OTEL Package - arize-phoenix-otel API reference
Python Client Package - arize-phoenix-client API reference

TypeScript API Documentation:

TypeScript Packages - @arizeai/phoenix-otel, @arizeai/phoenix-client, and other TypeScript packages

6.6 KiB Raw Permalink Blame History