mirror of https://github.com/github/awesome-copilot.git synced 2026-04-12 03:05:55 +00:00

Files

Jim Bennett d79183139a Add Arize and Phoenix LLM observability skills (#1204 )

* Add 9 Arize LLM observability skills

Add skills for Arize AI platform covering trace export, instrumentation,
datasets, experiments, evaluators, AI provider integrations, annotations,
prompt optimization, and deep linking to the Arize UI.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add 3 Phoenix AI observability skills

Add skills for Phoenix (Arize open-source) covering CLI debugging,
LLM evaluation workflows, and OpenInference tracing/instrumentation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Ignoring intentional bad spelling

* Fix CI: remove .DS_Store from generated skills README and add codespell ignore

Remove .DS_Store artifact from winmd-api-search asset listing in generated
README.skills.md so it matches the CI Linux build output. Add queston to
codespell ignore list (intentional misspelling example in arize-dataset skill).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add arize-ax and phoenix plugins

Bundle the 9 Arize skills into an arize-ax plugin and the 3 Phoenix
skills into a phoenix plugin for easier installation as single packages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fix skill folder structures to match source repos

Move arize supporting files from references/ to root level and rename
phoenix references/ to rules/ to exactly match the original source
repository folder structures.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fixing file locations

* Fixing readme

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-04-02 09:58:55 +11:00

3.4 KiB

Raw Blame History

RETRIEVER Spans

Purpose

RETRIEVER spans represent document/context retrieval operations (vector DB queries, semantic search, keyword search).

Required Attributes

Attribute	Type	Description	Required
`openinference.span.kind`	String	Must be "RETRIEVER"	Yes

Attribute Reference

Query

Attribute	Type	Description
`input.value`	String	Search query text

Document Schema

Attribute Pattern	Type	Description
`retrieval.documents.{i}.document.id`	String	Unique document identifier
`retrieval.documents.{i}.document.content`	String	Document text content
`retrieval.documents.{i}.document.score`	Float	Relevance score (0-1 or distance)
`retrieval.documents.{i}.document.metadata`	String (JSON)	Document metadata

Flattening Pattern for Documents

Documents are flattened using zero-indexed notation:

retrieval.documents.0.document.id
retrieval.documents.0.document.content
retrieval.documents.0.document.score
retrieval.documents.1.document.id
retrieval.documents.1.document.content
retrieval.documents.1.document.score
...

Document Metadata

Common metadata fields (stored as JSON string):

{
  "source": "knowledge_base.pdf",
  "page": 42,
  "section": "Introduction",
  "author": "Jane Doe",
  "created_at": "2024-01-15",
  "url": "https://example.com/doc",
  "chunk_id": "chunk_123"
}

Example with metadata:

{
  "retrieval.documents.0.document.id": "doc_123",
  "retrieval.documents.0.document.content": "Machine learning is a method of data analysis...",
  "retrieval.documents.0.document.score": 0.92,
  "retrieval.documents.0.document.metadata": "{\"source\": \"ml_textbook.pdf\", \"page\": 15, \"chapter\": \"Introduction\"}"
}

Ordering

Documents are ordered by index (0, 1, 2, ...). Typically:

Index 0 = highest scoring document
Index 1 = second highest
etc.

Preserve retrieval order in your flattened attributes.

Large Document Handling

For very long documents:

Consider truncating document.content to first N characters
Store full content in separate document store
Use document.id to reference full content

Examples

Basic Vector Search

{
  "openinference.span.kind": "RETRIEVER",
  "input.value": "What is machine learning?",
  "retrieval.documents.0.document.id": "doc_123",
  "retrieval.documents.0.document.content": "Machine learning is a subset of artificial intelligence...",
  "retrieval.documents.0.document.score": 0.92,
  "retrieval.documents.0.document.metadata": "{\"source\": \"textbook.pdf\", \"page\": 42}",
  "retrieval.documents.1.document.id": "doc_456",
  "retrieval.documents.1.document.content": "Machine learning algorithms learn patterns from data...",
  "retrieval.documents.1.document.score": 0.87,
  "retrieval.documents.1.document.metadata": "{\"source\": \"article.html\", \"author\": \"Jane Doe\"}",
  "retrieval.documents.2.document.id": "doc_789",
  "retrieval.documents.2.document.content": "Supervised learning is a type of machine learning...",
  "retrieval.documents.2.document.score": 0.81,
  "retrieval.documents.2.document.metadata": "{\"source\": \"wiki.org\"}",
  "metadata.retriever_type": "vector_search",
  "metadata.vector_db": "pinecone",
  "metadata.top_k": 3
}

3.4 KiB Raw Blame History