llm-checker

Analyzes system hardware specifications (CPU, GPU, RAM, VRAM, architecture type) by querying OS-level APIs and device information to build a hardware profile. The tool detects GPU presence (NVIDIA CUDA, Apple Metal, AMD ROCm), measures available memory, identifies CPU architecture (x86, ARM), and determines system constraints that impact LLM inference performance. This profiling data becomes the input for model recommendation algorithms.

Uses an LLM (likely Claude or GPT via API) to analyze the hardware profile and recommend optimal open-source models from registries like Ollama, Hugging Face, or GGUF repositories. The engine considers hardware constraints (VRAM, CPU cores, GPU type), user preferences (latency vs quality), and model characteristics (parameter count, quantization format, inference speed benchmarks) to generate ranked recommendations with justifications. Recommendations are filtered by compatibility (e.g., only suggesting GGUF-quantized models if the system lacks GPU acceleration).

Queries the Ollama model registry (or compatible GGUF model repositories) to fetch available models, their parameter counts, quantization formats, and estimated VRAM requirements. The integration parses model metadata (e.g., 'mistral:7b-instruct-q4_0') to extract quantization level and architecture, then cross-references this against the hardware profile to filter compatible models. This enables real-time model availability checking and prevents recommending models that are unavailable or incompatible with the user's setup.

Maps hardware capabilities (GPU type, VRAM, CPU architecture) to compatible quantization formats (GGUF Q4, Q5, Q6, FP16, etc.) and determines which formats will run efficiently on the target system. For example, systems with limited VRAM (4-6GB) are matched to Q4 quantization, while systems with 16GB+ VRAM can run higher-quality Q6 or FP16 formats. The matching considers GPU acceleration support (CUDA for NVIDIA, Metal for Apple Silicon) and falls back to CPU inference for unsupported quantization formats.

Orchestrates a multi-step CLI workflow that guides users through hardware detection, preference input, model recommendation, and model selection. The workflow uses interactive prompts (e.g., 'What is your priority: speed or quality?') to gather user preferences, then chains together hardware analysis, LLM-powered recommendation, and registry lookup to produce a final model suggestion with download/run instructions. The workflow is designed for non-technical users and includes explanatory text at each step.

Detects Apple Silicon (M1, M2, M3, M4) architecture and identifies optimized model variants and inference engines that leverage Metal GPU acceleration. The detection checks for ARM64 architecture, Metal framework availability, and recommends models with Metal-optimized GGUF quantizations or inference engines like llama.cpp with Metal support. This enables Apple Silicon users to achieve near-GPU performance on CPU-only inference without requiring NVIDIA CUDA.

Integrates or estimates performance benchmarks (tokens per second, latency) for recommended models on the target hardware. The tool may query external benchmark databases (e.g., LLM benchmarks from Hugging Face or community sources) or use heuristic estimation based on model size, quantization level, and hardware specs (e.g., 'a 7B Q4 model on RTX 4090 typically achieves 100 tokens/sec'). Benchmarks help users understand real-world inference speed and make informed tradeoffs between model quality and latency.

Generates platform-specific, copy-paste-ready commands and instructions for downloading and running recommended models. For Ollama models, it generates 'ollama pull' and 'ollama run' commands; for GGUF models, it generates llama.cpp or other inference engine setup instructions. Instructions include environment variable configuration, GPU acceleration setup (CUDA, Metal, ROCm), and optional Docker commands for containerized deployment. The output is tailored to the user's OS (macOS, Linux, Windows) and detected hardware.

Abstracts LLM API calls across multiple providers (OpenAI, Anthropic, Ollama local, etc.) with a unified interface for the recommendation engine. The abstraction handles provider-specific authentication, request formatting, and response parsing, allowing the recommendation logic to remain provider-agnostic. This enables users to choose their preferred LLM provider for recommendations without changing the tool's code, and supports fallback to local Ollama if API keys are unavailable.