ChatGLM-4

Generates contextually-aware responses in Chinese and English through a stateful chat interface that maintains conversation history across multiple turns. The model.chat(tokenizer, prompt, history) method encodes the full dialogue history into the transformer's context window, enabling coherent multi-turn conversations with relative position encoding that theoretically supports unlimited context length, though performance degrades beyond the 2048-token training length.

Reduces model memory footprint through post-training quantization via model.quantize(bits) method, supporting both INT4 (6GB minimum) and INT8 (8GB minimum) precision levels. The quantization process converts the 6.2B parameter FP16 model to lower-bit representations, enabling deployment on consumer-grade GPUs while maintaining inference quality through careful bit-width selection and calibration.

Supports inference on Apple Silicon (M1/M2/M3) and Intel-based Macs through Metal GPU acceleration, automatically routing computation to the GPU when available while falling back to CPU. The implementation leverages PyTorch's Metal backend to achieve 2-5x speedup over pure CPU inference on Apple Silicon while maintaining compatibility with standard PyTorch code.

Provides evaluation utilities to measure fine-tuned model performance on validation datasets using standard metrics (BLEU, ROUGE, exact match) and custom metrics. The evaluation pipeline handles batch processing of test examples, computes aggregate statistics, and generates detailed reports comparing fine-tuned vs base model performance to quantify adaptation effectiveness.

Manages model checkpoints and fine-tuning artifacts through PyTorch's save/load mechanisms, enabling persistence of model weights, tokenizer state, and training configuration. The checkpoint system supports resuming interrupted training, loading fine-tuned models for inference, and maintaining version history of model iterations through organized directory structures.

Enables domain-specific model adaptation through P-Tuning v2 implementation in the ptuning/ directory, which adds learnable prompt embeddings to the input layer while freezing the base model weights. This approach reduces fine-tuning memory requirements to 7-9GB (vs 14GB for full fine-tuning) and requires only 5-10% of the parameters to be trainable, allowing rapid adaptation to specialized tasks without catastrophic forgetting.

Exposes the ChatGLM-6B model as an HTTP endpoint through api.py, accepting JSON-formatted requests containing prompts and conversation history, and returning JSON responses with generated text and updated history. The API service handles tokenization, inference, and response formatting automatically, enabling integration with web applications, microservices, and third-party tools without requiring direct Python model access.

Provides a cli_demo.py interface for real-time dialogue interaction, accepting user input from stdin and streaming model responses character-by-character to stdout. The CLI maintains conversation history automatically, handles tokenization transparently, and supports interactive mode where users can continue conversations across multiple turns without reloading the model.

Offers two browser-based UI implementations (web_demo.py using Gradio and web_demo2.py using Streamlit) that wrap the ChatGLM-6B model in interactive web applications. Both interfaces handle model loading, tokenization, and inference transparently, providing chat-like UX with conversation history display, and can be deployed locally or on cloud platforms without code modification.

Implements the ChatGLMForConditionalGeneration class, a 6.2 billion parameter transformer model based on the General Language Model (GLM) framework that combines bidirectional and autoregressive attention patterns. The architecture uses relative position encoding to handle variable-length sequences, enabling both understanding and generation tasks through a unified conditional generation objective that masks different portions of the input during training.

Implements ChatGLMTokenizer that encodes and decodes text in both Chinese and English using a unified vocabulary optimized for bilingual content. The tokenizer handles Chinese characters as individual tokens while using subword tokenization (BPE-style) for English, enabling efficient representation of mixed-language inputs and maintaining semantic coherence across language boundaries.

Supports deployment across multiple GPUs through model parallelism, where different layers of the 6.2B parameter model are distributed across GPUs to reduce per-GPU memory requirements. The implementation automatically handles tensor communication between GPUs during forward passes, enabling inference on systems with multiple consumer-grade GPUs rather than requiring a single high-memory GPU.

Enables inference on CPU-only systems through INT4 quantization combined with memory-mapped file loading, where model weights are stored on disk and loaded into RAM on-demand. This approach trades inference speed (10-50x slower than GPU) for accessibility, allowing ChatGLM-6B to run on laptops and servers without dedicated GPUs by keeping only active layers in memory.