Financial Language Understanding And Semantic Reasoning

via “multi-step numerical reasoning over financial documents”

8.3K financial reasoning questions over real S&P 500 earnings reports.

Unique: Combines real SEC filing documents (not synthetic) with crowdsourced questions requiring multi-step arithmetic, creating a hybrid dataset that tests both domain knowledge extraction and quantitative reasoning in a single evaluation task. Unlike generic math word problems, answers require locating figures within 10+ page documents first.

vs others: More challenging than DROP or SVAMP because it requires financial domain knowledge AND document retrieval before arithmetic, whereas generic math benchmarks assume figures are already extracted