Briefs · Taipei time (GMT+8)

Sakana AI releases JFinBench, a Japanese finance benchmark

一句重點:JFinBench makes the gap between open-weight and proprietary models easier to see.

發生什麼事

@SakanaAILabs says JFinBench draws on the 2026 editions of six Japanese financial qualification exams, including the Certified Public Accountant exam, the Certified Tax Accountant exam and Grade 1 of the FP Skills Test. Its 321 questions include multiple-choice items, tax calculations, written responses and chart interpretation. A coding agent converted the exam PDFs, and people checked each question by hand in about 12 hours (🗣 Self-report).

The benchmark was used to evaluate 12 of the latest LLMs. Sakana AI says the main reason for lost points was a lack of knowledge about Japan’s systems, rather than calculation errors (🗣 Self-report). Open-weight models are catching up with proprietary ones on major benchmarks, but the gap remains large on JFinBench.

01Sources

  • @SakanaAILabs: Sakana AI’s official account; post published 2026-11-16 at 08:53 UTC