# `ReqLLM.Evaluation`
[🔗](https://github.com/agentjido/req_llm/blob/main/lib/req_llm/evaluation.ex#L1)

Evaluates one text, JSON-compatible, or context state against named questions.

This is a model call, not a test run. Providers handle their own request and
answer formats. An evaluation model does not need to support chat generation.

# `state`

```elixir
@type state() :: String.t() | map() | list() | ReqLLM.Context.t()
```

# `evaluate`

```elixir
@spec evaluate(ReqLLM.model_input(), state(), map(), keyword()) ::
  {:ok, ReqLLM.Response.t()} | {:error, term()}
```

Evaluates one state against a map of named questions.

    questions = %{
      department: %{
        type: :choice,
        instructions: "Which team should handle this?",
        criteria: %{billing: "Billing and refunds", support: "Other requests"}
      },
      urgent: %{type: :boolean, instructions: "Is this urgent?"}
    }

    {:ok, result} = ReqLLM.evaluate("typesafe:jev-latest", "Please refund me", questions)
    result.object["department"]["choice"]
    result.object["urgent"]["probability"]

Choice and score answers may include probabilities and confidence. Providers
may support more question types. OpenRouter evaluations accept routing preferences
such as `provider_options: [openrouter_provider: %{zdr: true}]`. Eligible OpenAI
models use the Decisions API. They support boolean, choice, and score questions;
`provider_options: [openai: [safety_identifier: "tenant-123"]]` adds the optional
safety identifier. OpenAI refusals stay in the result as `%{"type" => "refusal"}`.
OpenAI Decisions also accepts a `ReqLLM.Context` state of system, user, and
assistant text, with images in user messages:

    context =
      ReqLLM.Context.new([
        ReqLLM.Context.user([
          ReqLLM.Message.ContentPart.text("Inspect the product in this photo."),
          ReqLLM.Message.ContentPart.image(png_bytes, "image/png")
        ])
      ])

    ReqLLM.evaluate("openai:gpt-6-luna", context, questions)

Other providers accept only text or JSON state. Azure Microsoft Decision
models use a Foundry resource root as `base_url` and
`provider_options: [azure: [deployment: "my-decision-deployment"]]` to select
the deployment. They support boolean, choice, and score questions.
The response stores named answers in `object` with string keys.
`provider_meta.raw_response` keeps the original provider data.

# `evaluate!`

```elixir
@spec evaluate!(ReqLLM.model_input(), state(), map(), keyword()) ::
  ReqLLM.Response.t() | no_return()
```

Same as `evaluate/4`, but raises on error.

# `models`

```elixir
@spec models() :: [String.t()]
```

Lists model specs that `evaluate/4` can call with an installed adapter.

Catalog evaluation metadata can also describe models for providers that
ReqLLM does not yet support. Those models are excluded.

---

*Consult [api-reference.md](api-reference.md) for complete listing*
