artifact_id: observable-live
rubric_version: 2.8.0
formula: baseline
judges:
- openai:gpt-5.6-terra
rows:
- criterion_id: DEL-01
  category: delivery
  checker: program
  verdict: 'yes'
  points_possible: 10
  points_earned: 10.0
  evidence: ''
  agreement: null
  votes: 0
- criterion_id: DEL-02
  category: delivery
  checker: program
  verdict: 'yes'
  points_possible: 0
  points_earned: -4.0
  evidence: 1 console errors
  agreement: null
  votes: 0
- criterion_id: DEL-03
  category: delivery
  checker: program
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: ''
  agreement: null
  votes: 0
- criterion_id: COR-01
  category: correctness
  checker: program
  verdict: na
  points_possible: 0
  points_earned: 0.0
  evidence: the brief has no reference numbers
  agreement: null
  votes: 0
- criterion_id: BRF-01
  category: answers_brief
  checker: judge
  verdict: 'yes'
  points_possible: 0
  points_earned: -15.0
  evidence: '“Where do things stand?” — metric cards: “Open issues,” “Opened / closed (28d),” “Awaiting
    triage,” “Answered within 48h (28d),” “Median days to close (28d),” and “Customer-reported open” |
    “Is the backlog shrinking?” — “Open issues over time” and “Opened vs closed per week” | “Are we keeping
    up with triage?” — “Open issues by triage status and age,” “New issues answered within 48 hours,”
    and “Triage queue — oldest first” | “Where is the work?” — “Open issues by area” and “Open issues
    by adapter” | “How close are the epics?” — “Open epics by % of sub-issues closed” | “What should we
    work on, and who is on it?” — “Most-requested open issues” and “Open issues by assignee”'
  agreement: 1.0
  votes: 1
- criterion_id: BRF-02
  category: answers_brief
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Compared the visible page space used by requested dashboard content with incidental elements
    not requested in the brief, such as navigation, metadata, refresh control, and footer.
  agreement: 1.0
  votes: 1
- criterion_id: WRK-01
  category: working
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Reviewed the full page and all visible dashboard sections for page-level errors, loading states,
    and widespread empty panels.
  agreement: 1.0
  votes: 1
- criterion_id: WRK-02
  category: working
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Reviewed each visible KPI card, chart, and table panel for blank states, errors, placeholders,
    or spinners.
  agreement: 1.0
  votes: 1
- criterion_id: HON-01
  category: honesty
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Checked value-axis baselines on the area and bar charts, including open issues over time,
    opened vs closed, triage age, area, adapter, and assignee charts.
  agreement: 1.0
  votes: 1
- criterion_id: HON-02
  category: honesty
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Compared rate/share/% wording with displayed values and axes, including “New issues answered
    within 48 hours” and “Open epics by % of sub-issues closed.”
  agreement: 1.0
  votes: 1
- criterion_id: HON-03
  category: honesty
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: 'Compared the repeated issue-type color legend across charts: Feature, Bug, Task, and Other
    use consistent colors where repeated.'
  agreement: 1.0
  votes: 1
- criterion_id: HON-04
  category: honesty
  checker: judge
  verdict: na
  points_possible: 0
  points_earned: 0.0
  evidence: Reviewed visible charts for dual independent value axes and series-to-axis ambiguity.
  agreement: 1.0
  votes: 1
- criterion_id: HON-05
  category: honesty
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Compared the side-by-side “Open issues by area” and “Open issues by adapter” charts. Their
    count-axis scales differ, but each chart visibly provides its own labeled axis ticks.
  agreement: 1.0
  votes: 1
- criterion_id: HON-06
  category: honesty
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Reviewed line charts for lines connecting unordered categories. The visible line chart, “New
    issues answered within 48 hours,” is ordered by week/date.
  agreement: 1.0
  votes: 1
- criterion_id: FRM-01
  category: form
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Reviewed all chart forms and the comparisons they support.
  agreement: 1.0
  votes: 1
- criterion_id: FRM-02
  category: form
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Checked whether two or more charts use forms that hide their main comparisons.
  agreement: 1.0
  votes: 1
- criterion_id: FRM-03
  category: form
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Reviewed multi-series/category charts, including the stacked issue-type charts, triage-age
    chart, area, adapter, and assignee charts, for unmatched or indistinguishable series.
  agreement: 1.0
  votes: 1
- criterion_id: FRM-04
  category: form
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: 'Checked ordering in ranked categorical comparisons: triage queue, open issues by area, adapter,
    epic completion, requested issues, and assignee.'
  agreement: 1.0
  votes: 1
- criterion_id: FRM-05
  category: form
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Checked KPI cards and charts for a single value unnecessarily rendered as a gauge or one-bar
    chart.
  agreement: 1.0
  votes: 1
- criterion_id: FRM-07
  category: form
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Compared chart subjects and forms for duplicate visualizations that add no new information.
  agreement: 1.0
  votes: 1
- criterion_id: FRM-08
  category: form
  checker: judge
  verdict: 'yes'
  points_possible: 0
  points_earned: -8.0
  evidence: '"Opened vs closed per week" — the roughly 430 opened-issues spike forces a 0–450 scale, making
    most weekly opened/closed bars comparatively difficult to read'
  agreement: 1.0
  votes: 1
- criterion_id: FRM-06
  category: form
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Checked the first and last points of the time series for unexplained incomplete-period spikes
    or drops; the weekly opened/closed chart explicitly states "Complete weeks."
  agreement: 1.0
  votes: 1
- criterion_id: LEG-01
  category: legibility
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Reviewed chart titles, legends, axis ticks, table headers, and plotted marks across the full-resolution
    tiles.
  agreement: 1.0
  votes: 1
- criterion_id: LEG-02
  category: legibility
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Reviewed all charts for overlapping text.
  agreement: 1.0
  votes: 1
- criterion_id: LEG-03
  category: legibility
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Reviewed card boundaries, chart/table titles, labels, and table columns in each tile.
  agreement: 1.0
  votes: 1
- criterion_id: LEG-04
  category: legibility
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Reviewed text at the full-resolution tile scale, including chart ticks, legends, notes, and
    table text.
  agreement: 1.0
  votes: 1
- criterion_id: LEG-05
  category: legibility
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Reviewed chart sizes and whether plotted bars, lines, and table/progress cells can be visually
    distinguished.
  agreement: 1.0
  votes: 1
- criterion_id: LBL-01
  category: labeling
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Headline tiles, chart axes, legends, and table columns were checked for missing units on counts,
    rates, and durations.
  agreement: 1.0
  votes: 1
- criterion_id: LBL-02
  category: labeling
  checker: judge
  verdict: 'yes'
  points_possible: 0
  points_earned: -4.0
  evidence: '"Data as of [object Ob"'
  agreement: 1.0
  votes: 1
- criterion_id: LBL-03
  category: labeling
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: The page scope and visible time references were checked.
  agreement: 1.0
  votes: 1
- criterion_id: LBL-04
  category: labeling
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: All six headline tiles were checked for a tile-level or page-level timeframe; the page also
    states it was queried live and several tiles explicitly state "(28d)".
  agreement: 1.0
  votes: 1
- criterion_id: LBL-05
  category: labeling
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: All visible charts and tables were checked for a descriptive title or label.
  agreement: 1.0
  votes: 1
- criterion_id: LBL-07
  category: labeling
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Visible percentages were checked for their basis, including "Answered within 48h (28d)" and
    the epic "% complete" column.
  agreement: 1.0
  votes: 1
- criterion_id: LBL-08
  category: labeling
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Chart titles, axes, legends, and plotted measures were compared, including the opened-versus-closed
    weekly bars and the open-issue stacked series.
  agreement: 1.0
  votes: 1
- criterion_id: LBL-09
  category: labeling
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Headline metrics were compared with related chart/table values and page timestamps/notes were
    checked for conflicts.
  agreement: 1.0
  votes: 1
- criterion_id: LAY-01
  category: layout
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Space between and around dashboard cards across the full page.
  agreement: 1.0
  votes: 1
- criterion_id: LAY-02
  category: layout
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Alignment of metric cards, chart cards, tables, and row gaps.
  agreement: 1.0
  votes: 1
- criterion_id: LAY-04
  category: layout
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Whether detailed tables or secondary content appear before the headline metrics and primary
    charts.
  agreement: 1.0
  votes: 1
- criterion_id: LAY-05
  category: layout
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Visual weight of card borders, fills, legends, and other decoration relative to the data.
  agreement: 1.0
  votes: 1
- criterion_id: LAY-06
  category: layout
  checker: judge
  verdict: 'no'
  points_possible: 0
  points_earned: 0.0
  evidence: Whether chart colors have meaningful, consistently legend-defined encodings.
  agreement: 1.0
  votes: 1
- criterion_id: LAY-07
  category: layout
  checker: judge
  verdict: 'yes'
  points_possible: 0
  points_earned: -4.0
  evidence: '"Data as of [object Ob"'
  agreement: 1.0
  votes: 1
- criterion_id: EXC-01
  category: excellence
  checker: judge
  verdict: 'no'
  points_possible: 10
  points_earned: 0.0
  evidence: Page title, section headings, and chart annotations for a written statement of the dashboard's
    main finding supported by a chart.
  agreement: 1.0
  votes: 1
- criterion_id: EXC-03
  category: excellence
  checker: judge
  verdict: 'yes'
  points_possible: 3
  points_earned: 3.0
  evidence: 'Open issues: “511” with “+52 in 28 days”'
  agreement: 1.0
  votes: 1
- criterion_id: EXC-04
  category: excellence
  checker: judge
  verdict: 'no'
  points_possible: 6
  points_earned: 0.0
  evidence: Charts for an in-place annotation, target/reference line, or a singled-out highlighted mark/series.
  agreement: 1.0
  votes: 1
- criterion_id: EXC-06
  category: excellence
  checker: judge
  verdict: 'no'
  points_possible: 5
  points_earned: 0.0
  evidence: Chart color treatment for mostly neutral-gray data with a single accent reserved for the focal
    subject.
  agreement: 1.0
  votes: 1
- criterion_id: EXC-07
  category: excellence
  checker: judge
  verdict: 'no'
  points_possible: 4
  points_earned: 0.0
  evidence: Multi-series charts for direct labels on series or slices rather than reliance on a legend.
  agreement: 1.0
  votes: 1
- criterion_id: EXC-08
  category: excellence
  checker: judge
  verdict: 'yes'
  points_possible: 5
  points_earned: 5.0
  evidence: “Complete weeks; when closed > opened the backlog shrank” beside “Opened vs closed per week”
    | “By week opened; unanswered issues count as misses” beside “New issues answered within 48 hours”
    | “Issues with several areas count in each” beside “Open issues by area” | “(none) = adapter-agnostic”
    beside “Open issues by adapter”
  agreement: 1.0
  votes: 1
score: 0.347
category_scores:
  delivery: 0.6
  excellence: 0.242
agreement: 1.0
judge_errors: []
votes:
- criterion_id: BRF-01
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'yes'
  evidence: '“Where do things stand?” — metric cards: “Open issues,” “Opened / closed (28d),” “Awaiting
    triage,” “Answered within 48h (28d),” “Median days to close (28d),” and “Customer-reported open” |
    “Is the backlog shrinking?” — “Open issues over time” and “Opened vs closed per week” | “Are we keeping
    up with triage?” — “Open issues by triage status and age,” “New issues answered within 48 hours,”
    and “Triage queue — oldest first” | “Where is the work?” — “Open issues by area” and “Open issues
    by adapter” | “How close are the epics?” — “Open epics by % of sub-issues closed” | “What should we
    work on, and who is on it?” — “Most-requested open issues” and “Open issues by assignee”'
  round: 1
- criterion_id: BRF-02
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Compared the visible page space used by requested dashboard content with incidental elements
    not requested in the brief, such as navigation, metadata, refresh control, and footer.
  round: 1
- criterion_id: WRK-01
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Reviewed the full page and all visible dashboard sections for page-level errors, loading states,
    and widespread empty panels.
  round: 1
- criterion_id: WRK-02
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Reviewed each visible KPI card, chart, and table panel for blank states, errors, placeholders,
    or spinners.
  round: 1
- criterion_id: HON-01
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Checked value-axis baselines on the area and bar charts, including open issues over time,
    opened vs closed, triage age, area, adapter, and assignee charts.
  round: 1
- criterion_id: HON-02
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Compared rate/share/% wording with displayed values and axes, including “New issues answered
    within 48 hours” and “Open epics by % of sub-issues closed.”
  round: 1
- criterion_id: HON-03
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: 'Compared the repeated issue-type color legend across charts: Feature, Bug, Task, and Other
    use consistent colors where repeated.'
  round: 1
- criterion_id: HON-04
  judge: gpt-5.6-terra
  sample: 0
  verdict: na
  evidence: Reviewed visible charts for dual independent value axes and series-to-axis ambiguity.
  round: 1
- criterion_id: HON-05
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Compared the side-by-side “Open issues by area” and “Open issues by adapter” charts. Their
    count-axis scales differ, but each chart visibly provides its own labeled axis ticks.
  round: 1
- criterion_id: HON-06
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Reviewed line charts for lines connecting unordered categories. The visible line chart, “New
    issues answered within 48 hours,” is ordered by week/date.
  round: 1
- criterion_id: FRM-01
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Reviewed all chart forms and the comparisons they support.
  round: 1
- criterion_id: FRM-02
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Checked whether two or more charts use forms that hide their main comparisons.
  round: 1
- criterion_id: FRM-03
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Reviewed multi-series/category charts, including the stacked issue-type charts, triage-age
    chart, area, adapter, and assignee charts, for unmatched or indistinguishable series.
  round: 1
- criterion_id: FRM-04
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: 'Checked ordering in ranked categorical comparisons: triage queue, open issues by area, adapter,
    epic completion, requested issues, and assignee.'
  round: 1
- criterion_id: FRM-05
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Checked KPI cards and charts for a single value unnecessarily rendered as a gauge or one-bar
    chart.
  round: 1
- criterion_id: FRM-07
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Compared chart subjects and forms for duplicate visualizations that add no new information.
  round: 1
- criterion_id: FRM-08
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'yes'
  evidence: '"Opened vs closed per week" — the roughly 430 opened-issues spike forces a 0–450 scale, making
    most weekly opened/closed bars comparatively difficult to read'
  round: 1
- criterion_id: FRM-06
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Checked the first and last points of the time series for unexplained incomplete-period spikes
    or drops; the weekly opened/closed chart explicitly states "Complete weeks."
  round: 1
- criterion_id: LEG-01
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Reviewed chart titles, legends, axis ticks, table headers, and plotted marks across the full-resolution
    tiles.
  round: 1
- criterion_id: LEG-02
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Reviewed all charts for overlapping text.
  round: 1
- criterion_id: LEG-03
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Reviewed card boundaries, chart/table titles, labels, and table columns in each tile.
  round: 1
- criterion_id: LEG-04
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Reviewed text at the full-resolution tile scale, including chart ticks, legends, notes, and
    table text.
  round: 1
- criterion_id: LEG-05
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Reviewed chart sizes and whether plotted bars, lines, and table/progress cells can be visually
    distinguished.
  round: 1
- criterion_id: LBL-01
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Headline tiles, chart axes, legends, and table columns were checked for missing units on counts,
    rates, and durations.
  round: 1
- criterion_id: LBL-02
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'yes'
  evidence: '"Data as of [object Ob"'
  round: 1
- criterion_id: LBL-03
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: The page scope and visible time references were checked.
  round: 1
- criterion_id: LBL-04
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: All six headline tiles were checked for a tile-level or page-level timeframe; the page also
    states it was queried live and several tiles explicitly state "(28d)".
  round: 1
- criterion_id: LBL-05
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: All visible charts and tables were checked for a descriptive title or label.
  round: 1
- criterion_id: LBL-07
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Visible percentages were checked for their basis, including "Answered within 48h (28d)" and
    the epic "% complete" column.
  round: 1
- criterion_id: LBL-08
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Chart titles, axes, legends, and plotted measures were compared, including the opened-versus-closed
    weekly bars and the open-issue stacked series.
  round: 1
- criterion_id: LBL-09
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Headline metrics were compared with related chart/table values and page timestamps/notes were
    checked for conflicts.
  round: 1
- criterion_id: LAY-01
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Space between and around dashboard cards across the full page.
  round: 1
- criterion_id: LAY-02
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Alignment of metric cards, chart cards, tables, and row gaps.
  round: 1
- criterion_id: LAY-04
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Whether detailed tables or secondary content appear before the headline metrics and primary
    charts.
  round: 1
- criterion_id: LAY-05
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Visual weight of card borders, fills, legends, and other decoration relative to the data.
  round: 1
- criterion_id: LAY-06
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Whether chart colors have meaningful, consistently legend-defined encodings.
  round: 1
- criterion_id: LAY-07
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'yes'
  evidence: '"Data as of [object Ob"'
  round: 1
- criterion_id: EXC-01
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Page title, section headings, and chart annotations for a written statement of the dashboard's
    main finding supported by a chart.
  round: 1
- criterion_id: EXC-03
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'yes'
  evidence: 'Open issues: “511” with “+52 in 28 days”'
  round: 1
- criterion_id: EXC-04
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Charts for an in-place annotation, target/reference line, or a singled-out highlighted mark/series.
  round: 1
- criterion_id: EXC-06
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Chart color treatment for mostly neutral-gray data with a single accent reserved for the focal
    subject.
  round: 1
- criterion_id: EXC-07
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'no'
  evidence: Multi-series charts for direct labels on series or slices rather than reliance on a legend.
  round: 1
- criterion_id: EXC-08
  judge: gpt-5.6-terra
  sample: 0
  verdict: 'yes'
  evidence: “Complete weeks; when closed > opened the backlog shrank” beside “Opened vs closed per week”
    | “By week opened; unanswered issues count as misses” beside “New issues answered within 48 hours”
    | “Issues with several areas count in each” beside “Open issues by area” | “(none) = adapter-agnostic”
    beside “Open issues by adapter”
  round: 1