arcane@prod : ~/oracle
online

// oracle

The Oracle

arcane@prod:~$ oracle --ask

An AI assistant that answers as this site's operator — ask it anything about Gabriel's experience, the projects, or how this platform works.

What keeps it honest:

  • grounded — every claim traces back to a curated public corpus: the résumé, the projects, the platform docs
  • tool-using — it can call read-only tools against gipc.dev's own live APIs, the same numbers you see on /system
  • transparent — it cites its sources (including this site's own source code), refuses to fabricate, and shows its working in a live trace panel
  • bounded — a hard daily budget and a bot check; when the budget is spent, it rests honestly

The tabs above the console each do one thing:

  • Ask — the conversation itself
  • Analyze a JD — reads a job description and maps it honestly against the record
  • Tailor résumé — reorders the real facts for a specific role, never inventing new ones
  • Theme studio — lets the model restyle the site within strict guard-rails
  • Local model — runs a small self-hosted model on my own hardware, with live latency and cost beside it

The eval panel at the bottom publishes real measured scores.

    a quick bot check appears when you start typing.

    // trace — watch it think

    retrievals & tool calls appear here as the oracle works.

      // evals — measured, not vibes

      metricscorenmethod
      retrieval hit@60.81216 questionsgold question → expected chunk in the oracle's top-6
      retrieval MRR0.56516mean reciprocal rank of the first expected chunk
      faithfulness0.925146 claims / 16 answersLLM-judged: every claim supported by the retrieved chunks
      JD label accuracy0.7516 labelled requirementsstrong/partial/gap vs hand-labelled expectations (2 sample JDs)
      evidence lexical grounding131 evidence stringsstrict token-stem overlap with the résumé — a LOWER BOUND, not a fabrication rate

      measured 2026-07-16 · model claude-haiku-4-5 · embedder BAAI/bge-small-en-v1.5 · corpus 15e52a75 · k=6, code_cap=2; answers at temp 0, JD mapping runs the production path

      answers by claude-haiku-4-5, judged for faithfulness by claude-sonnet-5 — an independent model, so the judge doesn't grade its own words; not comparable with earlier same-model-judged runs (claim-splitting varies by judge)

      A small-n snapshot (16 retrieval questions; 146 judged claims; 16 labelled JD requirements) against a pinned corpus and model — a regression tripwire for this site's oracle, not a general benchmark. Faithfulness evaluates the retrieval-grounded answer path at temperature 0 — no live tools, no chat history; tool-mediated answers are out of scope. Lexical grounding counts a token as grounded when it (or its stem: s/es/ed/ing/ion/ment stripped) appears in the résumé text — connective vocabulary and true-but-unmentioned words deflate it, and every failing token is published in the results file, so read it as a strict floor.

      unsupported claims the judge flagged: "AKS is Azure Kubernetes Service" · "At WealthGoal, he likely worked with managed Kubernetes as part of the AWS environment" · "The IEEE Access publication is in his PhD domain"

      audit it: the harness · gold questions · gold JDs · raw results