What is the best code generation product?

SWE-bench is the top-rated code generation product with a score of 48/100 on Unfragile.

How many code generation products are there?

Unfragile indexes 9 code generation products, ranked by real usage data.

Code Generation

9 tools · Browse 9 code generation AI artifacts on Unfragile.

Avg score: 35/100

9 free

9 open source

1

SWE-bench

48/100Free

Real-world software engineering task evaluation suite

HumanEval

47/100Free

OpenAI's standard for evaluating code generation models

Benchmark

3

MBPP

43/100Free

Mostly Basic Programming Problems (beginner-friendly code)

Benchmark

4

LiveCodeBench

43/100Free

Live coding benchmark with recent LeetCode problems

Benchmark

5

EvalPlus

43/100Free

Extended code evaluation with harder test cases for HumanEval

Benchmark

6

Qwen 2.5 Coder (1.5B, 3B, 7B, 32B)

23/100Free

Alibaba's Qwen 2.5 specialized for code generation and understanding — code-specialized

Model

7

CodeLlama (7B, 13B, 34B, 70B)

23/100Free

Meta's CodeLlama — Llama-based model specialized for code — code-specialized

Model

8

StarCoder 2 (3B, 7B, 15B)

21/100Free

BigCode's StarCoder 2 — multilingual code generation model — code-specialized

Model

9

DeepSeek Coder V2 (16B, 236B)

20/100Free

DeepSeek's Coder V2 — specialized for code generation and understanding — code-specialized

Model

← All categories Search tools

API Docs

Company

nfragile