Tracing

LLM apps use increasingly complex abstractions (chains, agents with tools, advanced prompts). The nested traces in Varex help to understand what is going on and get to the root cause of problems.

Introduction

• Each backend execution is logged with a single trace.
• Each trace can contain multiple observations to log the individual steps of the execution.
  • Observations are of different types:
    • Events are the basic building block. They are used to track discrete events in a trace.
    • Spans represent durations of units of work in a trace.
    • Generations are spans which are used to log generations of AI models. They contain additional attributes about the model, the prompt/completion. For generations, token usage and costs are automatically calculated.
  • Observations can be nested.

Follow the integration docs to send traces to Varex. You can use:

Working with traces

Tags enable you to flexibly add metadata to your traces. You can filter for tags in the Varex UI. You can also group traces in the UI for comparison and pattern identification. Use descriptive tags and leverage the filtering and grouping capabilities of the Varex UI to enhance trace management and analysis workflows.

Use API to create traces

curl --location 'http://cloud.varex.ai/api/v1/traces' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header "Authorization: Bearer {Varex_public_key}"
--data '{
  "traceId": "<string>",
  "name": "<string>",
  "value": "<integer>",
  "observationId": "<optional_string>"
}'