User Feedback

User feedback is a great source to evaluate the quality of an LLM app's output. In Varex, feedback is collected as a score and attached to an execution trace or an individual LLM generation.

Types of Feedback

Depending on the type of the application, there are different types of feedback that can be collected that vary in quality, detail, and quantity.

• Explicit Feedback: Directly prompt the user to give feedback, this can be a rating, a like, a dislike, a scale or a comment. While it is simple to implement, quality and quantity of the feedback is often low.
• Implicit Feedback: Measure the user's behavior, e.g., time spent on a page, click-through rate, accepting/rejecting a model-generated output. This type of feedback is more difficult to implement but is often more frequent and reliable.