javaparser vs WMDP

Converts Java source code into a complete Abstract Syntax Tree using a recursive descent parser that handles Java language features from version 1.0 through Java 25 including preview features. The parser generates a hierarchical node structure (CompilationUnit → ClassOrInterfaceDeclaration → MethodDeclaration, etc.) that preserves all syntactic information including comments, annotations, and modifiers. The parsing pipeline tokenizes input, applies grammar rules, and constructs typed AST nodes that can be traversed and manipulated programmatically.

Unique: Supports Java 1-25 with preview features through a metamodel-driven parser generator (javaparser-core-metamodel-generator) that auto-generates AST node classes from a grammar specification, enabling rapid adaptation to new Java language features without manual node class creation

vs alternatives: More comprehensive Java version support (1-25) than ANTLR-based parsers and includes built-in symbol resolution, whereas generic parser generators require separate semantic analysis layers

Provides visitor pattern implementations (GenericVisitor, ModifierVisitor, VoidVisitor) that enable traversal and transformation of the AST without modifying the node classes themselves. The visitor pattern allows developers to define custom logic that executes on specific node types (e.g., MethodDeclaration, FieldDeclaration) as the tree is walked. ModifierVisitor enables in-place AST transformation by returning modified nodes, while VoidVisitor supports side-effect operations like analysis and reporting.

Unique: Implements three distinct visitor variants (GenericVisitor for read-only traversal, ModifierVisitor for in-place transformation, VoidVisitor for side-effects) generated from a metamodel, allowing developers to choose the appropriate pattern without boilerplate

vs alternatives: More flexible than tree-walking interpreters because visitors are composable and can be chained; more type-safe than reflection-based AST manipulation because visitor methods are generated with correct node type signatures

Extracts and analyzes Java annotations from AST nodes, providing access to annotation values, targets, and metadata. Developers can query annotations on classes, methods, fields, and parameters, and extract annotation values (strings, numbers, arrays, nested annotations) for use in code analysis and generation. This enables tools to leverage annotation-driven development patterns and extract configuration from annotated code.

Unique: Provides direct AST-level access to annotations through AnnotationExpr nodes, enabling extraction of annotation values without reflection or runtime processing, making it suitable for static analysis and code generation

vs alternatives: More flexible than reflection-based annotation processing because it works with source code; more complete than regex-based annotation matching because it understands annotation syntax and values

Resolves method calls and field accesses to their definitions by analyzing method signatures, parameter types, and inheritance hierarchies to determine which overloaded method is being invoked. The resolution system handles method overloading, varargs, type erasure, and inheritance-based method lookup (including interface default methods). It returns ResolvedMethodDeclaration objects that provide access to the method's signature, return type, and declaring class.

Unique: Implements overload resolution that respects Java's method selection rules (exact match, widening conversion, boxing, varargs) and handles inheritance-based lookup including interface default methods, enabling accurate determination of which method is invoked

vs alternatives: More accurate than name-based matching because it considers parameter types and inheritance; more complete than simple signature matching because it handles overloading and method overriding

Preserves original source formatting, whitespace, and comments during parsing and AST manipulation through a lexical preservation system that tracks token positions and associates comments with AST nodes. When the AST is modified and pretty-printed, the original formatting is maintained where possible, and comments are reattached to their corresponding code elements. This is critical for tools that need to preserve developer intent and code style during transformations.

Unique: Uses a token-position tracking system (Range objects) that maps AST nodes to their source locations and associates comments through proximity analysis, enabling round-trip preservation where code can be parsed, modified, and printed with original formatting intact

vs alternatives: Preserves formatting better than ANTLR-based parsers which typically discard whitespace; more accurate comment attribution than regex-based comment matching because it uses syntactic context

Resolves Java symbols (types, methods, fields, variables) to their definitions across multiple compilation units using a context-based resolution system (javaparser-symbol-solver-core). The symbol solver uses type resolvers (ReflectionTypeSolver, JavaParserTypeSolver, CombinedTypeSolver) to locate symbol definitions in the classpath, source code, or runtime reflection. It performs type inference on expressions and method calls, handling generics, inheritance hierarchies, and method overloading to determine the exact symbol being referenced.

Unique: Implements a pluggable type resolver architecture (TypeSolver interface) that combines multiple resolution strategies (reflection, source parsing, classpath scanning) through CombinedTypeSolver, enabling resolution across heterogeneous codebases mixing compiled and source code

vs alternatives: More accurate than simple name-based matching because it respects Java scoping rules and inheritance; more flexible than IDE-specific symbol tables because it works with arbitrary codebases without IDE integration

Generates Java source code from AST structures using a builder pattern API (CompilationUnitBuilder, ClassOrInterfaceBuilder, MethodBuilder, etc.) that constructs AST nodes programmatically without parsing. Developers can fluently build AST hierarchies by chaining builder methods, then pretty-print the resulting AST to Java source code. This enables code generation tools to create Java code dynamically based on templates, configurations, or runtime decisions.

Unique: Provides a fluent builder API (CompilationUnitBuilder, ClassOrInterfaceBuilder) that mirrors the AST structure, allowing developers to construct code programmatically without parsing, with type-safe method chaining and automatic node hierarchy management

vs alternatives: More type-safe and discoverable than string-based code generation because builders enforce valid AST construction; more maintainable than template strings because changes to code structure are refactored automatically

Serializes parsed AST structures to JSON format and deserializes JSON back into AST objects through the javaparser-core-serialization module. This enables AST persistence, transmission over networks, and integration with tools that work with JSON representations of code structure. The serialization preserves all AST node information including types, positions, and metadata.

Unique: Provides bidirectional JSON serialization that preserves all AST node types and metadata, enabling round-trip conversion (AST → JSON → AST) without information loss, unlike generic JSON serialization which would lose type information

vs alternatives: More complete than generic JSON serialization because it preserves AST node types; more efficient than re-parsing because deserialization is faster than parsing for cached ASTs

Evaluates LLM outputs against curated question sets spanning three distinct hazard domains (biosecurity, cybersecurity, chemical security) using domain-expert-validated benchmarks. The assessment framework maps model responses to risk levels within each domain, enabling quantitative measurement of dangerous capability presence. Responses are scored against rubrics developed by security domain experts to identify whether models can produce actionable harmful information.

Unique: Combines expert-validated questions across three distinct security domains (biosecurity, cybersecurity, chemical) into a unified benchmark framework, rather than treating each domain separately. Uses domain-expert rubrics for scoring rather than automated classifiers, ensuring nuanced assessment of harmful capability presence.

vs alternatives: More comprehensive than single-domain safety benchmarks (e.g., ToxiGen for toxicity) because it measures dangerous knowledge across multiple hazard categories simultaneously, enabling holistic safety evaluation.

Provides standardized evaluation infrastructure to measure the effectiveness of unlearning techniques (methods that remove dangerous capabilities from trained models) by comparing model performance before and after unlearning interventions. The framework isolates the impact of unlearning by holding the benchmark constant while varying the model state, enabling quantitative assessment of whether dangerous knowledge has been successfully suppressed.

Unique: Provides a standardized evaluation harness specifically designed for unlearning research, with built-in comparison logic and side-effect detection. Unlike generic benchmarks, it explicitly measures delta between model states and flags unintended capability loss.

vs alternatives: More rigorous than ad-hoc unlearning evaluation because it enforces consistent benchmark administration, statistical testing, and side-effect measurement across all methods being compared.

Implements a structured scoring framework where model responses to dangerous knowledge questions are evaluated against expert-developed rubrics that assess the degree of hazard (e.g., specificity, actionability, completeness of harmful information). Responses are scored on multi-point scales (typically 0-4 or 0-5) rather than binary pass/fail, capturing nuance in how dangerous a model's output actually is. Rubrics are domain-specific (biosecurity, cybersecurity, chemical) and developed by subject matter experts to ensure validity.

Unique: Uses domain-expert-developed multi-point rubrics rather than automated classifiers or binary labels, enabling nuanced assessment of dangerous knowledge severity. Rubrics are calibrated to distinguish between vague, incomplete, and highly actionable harmful information.

vs alternatives: More interpretable and defensible than black-box classifiers because rubric criteria are explicit and expert-validated; enables stakeholders to understand why a response received a particular score.

Analyzes patterns in how dangerous knowledge correlates across the three benchmark domains (biosecurity, cybersecurity, chemical security), identifying whether models that excel at suppressing one type of hazard tend to suppress others. The analysis uses statistical correlation and clustering techniques to reveal whether dangerous capabilities are independent or coupled in model behavior. This enables understanding of whether unlearning interventions have domain-specific or global effects.

Unique: Explicitly analyzes relationships between dangerous knowledge across domains rather than treating each domain independently. Enables discovery of whether hazards are coupled or independent in model behavior.

vs alternatives: Provides deeper insight than single-domain benchmarks by revealing how safety properties interact across different hazard categories, informing more effective unlearning strategies.

Manages the creation, validation, and versioning of benchmark questions and rubrics through a structured curation pipeline involving domain experts, adversarial testing, and iterative refinement. The pipeline ensures questions are sufficiently difficult to elicit dangerous knowledge without being unrealistic, and rubrics are calibrated through inter-rater agreement studies. Version control enables tracking of benchmark evolution and ensures reproducibility across research papers.

Unique: Implements a formal curation pipeline with expert validation and inter-rater agreement checks, rather than ad-hoc question collection. Versioning enables reproducible research and transparent tracking of benchmark evolution.

vs alternatives: More rigorous than informal benchmarks because it enforces expert review, inter-rater validation, and version control, reducing bias and enabling reproducible comparisons across papers.

Provides a unified interface for evaluating diverse LLM architectures (open-source models, API-based models, fine-tuned variants) by abstracting away implementation differences. The abstraction handles API calls (OpenAI, Anthropic, etc.), local inference (Hugging Face, Ollama), and custom model serving, enabling consistent benchmark administration across heterogeneous model types. This enables fair comparison between models with different deployment modalities.

Unique: Abstracts away differences between API-based, local, and custom-deployed models through a unified interface, enabling fair comparison without reimplementing benchmark logic for each model type.

vs alternatives: More flexible than model-specific benchmarks because it supports any LLM architecture without code changes, reducing friction for researchers evaluating new models.

Implements rigorous statistical testing to determine whether differences in dangerous knowledge scores between models or unlearning methods are statistically significant or due to random variation. Uses techniques like bootstrap confidence intervals, permutation tests, and effect size estimation to quantify uncertainty in benchmark results. This prevents overconfident claims about safety improvements that may not be robust.

Unique: Integrates formal statistical testing into the benchmark evaluation pipeline rather than relying on point estimates, ensuring claims about safety improvements are statistically justified.

vs alternatives: More rigorous than informal comparisons because it quantifies uncertainty and prevents overconfident claims about safety improvements that may not be robust to sampling variation.

Employs adversarial testing techniques to validate that benchmark questions reliably elicit dangerous knowledge and cannot be easily circumvented by prompt engineering. Red-teamers attempt to find questions that fail to elicit dangerous knowledge or rubric edge cases, and the benchmark is iteratively refined based on findings. This ensures the benchmark is robust to adversarial adaptation and captures genuine dangerous capabilities rather than surface-level patterns.

Unique: Incorporates formal red-teaming into the benchmark validation pipeline rather than assuming questions are robust, ensuring the benchmark remains effective against adversarial adaptation.

vs alternatives: More robust than static benchmarks because it actively searches for evasion techniques and iteratively refines questions, reducing the risk that models can circumvent the benchmark through prompt engineering.