OpenRewrite language module for Pascal

Recursive descent parser with lossless round-trip fidelity for pure Pascal syntax (ISO 7185 / Turbo Pascal). Includes LST node definitions, visitor, printer, remote serialization support, and a standalone LST generator for Moderne CLI integration. 65 parser tests covering programs, units, control flow, expressions, types, and more.
2026-03-10 23:32:06 +00:00
commit 8915c205cd
@@ -0,0 +1,31 @@
+# Build output
+build/**
+build/*
+build/
+.gradle/
+out/
+
+# IDE
+.idea/
+.vscode/
+*.iml
+*.ipr
+*.iws
+.project
+.classpath
+.settings/
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Local reference material (cloned rewrite-python)
+.gitignore-ref/
+
+# Claude Code / AI agent config (local only)
+.claude/
+AGENTS.md
+rewrite-pascal.md
+
+# Moderne CLI output
+.moderne/
@@ -0,0 +1,137 @@
+# rewrite-pascal
+
+An [OpenRewrite](https://docs.openrewrite.org/) language module for Pascal. Enables lossless semantic tree (LST) parsing, round-trip printing, visitor-based analysis, and recipe-driven refactoring of Pascal source code.
+
+## Motivation
+
+This project exists to bring the Moderne platform's tooling - [Prethink](https://docs.moderne.io/user-documentation/agent-tools/prethink/) for AI-assisted code comprehension, [TriGrep](https://docs.moderne.io/) for structural search, and OpenRewrite recipes for automated refactoring - to Pascal codebases.
+
+The immediate goal is to support analysis of two Free Pascal projects:
+
+- [Asuro OS](https://gitea.spexeah.com/Spexeah/Asuro) - a hobby operating system written in Free Pascal (~660 source files, ~100K lines)
+- [Wasuro](https://gitea.spexeah.com/Spexeah/Wasuro) - a WebAssembly virtual machine, also in Free Pascal
+
+### Current Status
+
+- The parser handles pure Pascal syntax and successfully parses 659 of 660 files in the Asuro codebase with full round-trip fidelity (the remaining file uses an unsupported Free Pascal extension)
+- The `rewrite-pascal-generator` module produces Moderne-compatible V2 LST artifacts that the Moderne CLI (`mod run`) accepts
+- Prethink runs against the generated LSTs, though its built-in discovery recipes are Java/JS-focused - Pascal-specific discovery recipes (unit dependency graphs, procedure signatures, record types) are a planned next step
+
+## Language Support
+
+The parser currently supports pure Pascal syntax (ISO 7185 / Turbo Pascal style). It does not yet support Delphi or ObjFPC language extensions such as:
+
+- Classes and objects (`class`, `object`, `interface` as OOP construct)
+- Generics (`TList<T>`)
+- Anonymous methods and closures
+- Advanced RTTI and attributes
+- Operator overloading
+- `try`/`except`/`finally` exception handling
+- String interpolation
+
+Support for Free Pascal's `{$mode objfpc}` and `{$mode delphi}` extensions is planned for future phases. Compiler directives are parsed and preserved but not interpreted.
+
+### Supported Constructs
+
+- Programs and units with `uses` clauses, `interface`/`implementation` sections
+- Declarations: `var`, `const`, `type` sections
+- Types: records (packed/bitpacked), enums, sets, arrays, pointers, variant records, subranges
+- Procedures and functions with parameters (`var`, `const`, `out`), local declarations, nested routines
+- Control flow: `if`/`then`/`else`, `for`/`to`/`downto`, `while`/`do`, `repeat`/`until`, `case`/`of`, `with`
+- Expressions: arithmetic, comparison, boolean, bitwise (`shl`/`shr`/`xor`), field access, indexing, type casts, set values
+- Comments: `{ }`, `(* *)`, `//` line comments
+- Compiler directives: `{$mode}`, `{$H+}`, etc.
+- Inline assembly blocks (`asm ... end`)
+
+All whitespace and comments are preserved in the LST for lossless round-trip fidelity.
+
+## Project Structure
+
+| Module | Description |
+|--------|-------------|
+| `rewrite-pascal` | Core language module: parser, printer, visitor, and LST node definitions |
+| `rewrite-pascal-remote` | Remote serialization support (Sender/Receiver/Validator) for distributed LST processing |
+| `rewrite-pascal-generator` | Standalone tool to parse Pascal projects and produce Moderne-compatible LST artifacts |
+
+## Building
+
+Requires Java 17+ and uses Gradle 8.12 (wrapper included).
+
+```bash
+./gradlew build
+```
+
+Run tests:
+
+```bash
+./gradlew test
+```
+
+## Generating LST Artifacts
+
+The `rewrite-pascal-generator` module produces Moderne-compatible `.ast.jar` artifacts from Pascal source trees. It requires the [Moderne CLI](https://docs.moderne.io/user-documentation/moderne-cli/getting-started/cli-intro) to be installed (for its bundled Java 25 runtime and serialization library).
+
+```bash
+# Build the generator fat JAR
+./gradlew :rewrite-pascal-generator:fatJar
+
+# Run it against a Pascal project (use ; as classpath separator on Windows, : on Linux/macOS)
+~/.moderne/cli/dist/jre/bin/java \
+  -cp "rewrite-pascal-generator/build/libs/*-all.jar;~/.moderne/cli/dist/lib/moderne-cli.jar" \
+  org.openrewrite.pascal.generator.PascalLstGenerator /path/to/pascal/project
+```
+
+The output is written to `/path/to/pascal/project/.moderne/build/` as a V2-format LST artifact. This artifact can then be used with `mod run` to execute OpenRewrite recipes:
+
+```bash
+mod run /path/to/pascal/project --recipe=<RecipeName>
+```
+
+## Using the Parser Programmatically
+
+```java
+import org.openrewrite.pascal.PascalParser;
+import org.openrewrite.SourceFile;
+
+PascalParser parser = PascalParser.builder().build();
+Stream<SourceFile> trees = parser.parse(
+    """
+    program Hello;
+    begin
+      WriteLn('Hello, World!');
+    end.
+    """
+);
+```
+
+## Architecture
+
+The LST reuses the Java tree infrastructure (`Pas extends J`) following the same pattern as [rewrite-python](https://github.com/openrewrite/rewrite-python). This means:
+
+- `PascalVisitor` extends `JavaVisitor` - Java-based visitors can traverse Pascal trees
+- Padding types (`JLeftPadded`, `JRightPadded`, `JContainer`) preserve whitespace
+- The `Markers` system stores keyword casing and other source-level metadata
+
+The parser is a hand-written recursive descent parser with a custom lexer - no external parser generator dependency.
+
+## Dependencies
+
+- OpenRewrite BOM 8.45.0 (`rewrite-core`, `rewrite-java`)
+- `rewrite-remote` 0.28.0 (for the remote module)
+- Lombok 1.18.34 (compile-only)
+- JUnit 5.11.4 (test)
+
+## AI Disclosure
+
+This project was built in part with the assistance of AI (Claude Opus 4.6). The development workflow followed a tight feedback loop:
+
+1. Write or generate parser code for a Pascal construct
+2. Compile and run the test suite
+3. Analyze failures by comparing printed output against the original source
+4. Fix the parser or printer and repeat
+
+This loop is a natural fit for OpenRewrite's lossless parsing model. The correctness criterion is simple and absolute - the printed output of a parsed file must be identical to the original input, byte for byte. There is no ambiguity about whether the parser is correct: either the round-trip produces the exact original source (including all whitespace, comments, and casing), or it doesn't. That binary signal made it straightforward for AI to identify what was wrong, propose a fix, and verify it in the next iteration.
+
+## License
+
+[Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0)
@@ -0,0 +1 @@
+// Root project — subprojects apply their own plugins
@@ -0,0 +1,7 @@
+distributionBase=GRADLE_USER_HOME
+distributionPath=wrapper/dists
+distributionUrl=https\://services.gradle.org/distributions/gradle-8.12-bin.zip
+networkTimeout=10000
+validateDistributionUrl=true
+zipStoreBase=GRADLE_USER_HOME
+zipStorePath=wrapper/dists
@@ -0,0 +1,130 @@
+#!/bin/sh
+
+#
+# Copyright © 2015-2021 the original authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+##############################################################################
+#
+#   Gradle start up script for POSIX generated by Gradle.
+#
+##############################################################################
+
+# Attempt to set APP_HOME
+# Resolve links: $0 may be a link
+app_path=$0
+while
+    APP_HOME=${app_path%"${app_path##*/}"}  # leaves a trailing /; empty if no leading path
+    [ -h "$app_path" ]
+do
+    ls=$( ls -ld -- "$app_path" )
+    link=${ls#*' -> '}
+    case $link in             #(
+      /*)   app_path=$link ;; #(
+      *)    app_path=$APP_HOME$link ;;
+    esac
+done
+
+APP_HOME=$( cd "${APP_HOME:-./}" > /dev/null && pwd -P ) || exit
+
+APP_NAME="Gradle"
+APP_BASE_NAME=${0##*/}
+
+# Use the maximum available, or set MAX_FD != -1 to use that value.
+MAX_FD=maximum
+
+warn () {
+    echo "$*"
+} >&2
+
+die () {
+    echo
+    echo "$*"
+    echo
+    exit 1
+} >&2
+
+# OS specific support (must be 'true' or 'false').
+cygwin=false
+msys=false
+darwin=false
+nonstop=false
+case "$( uname )" in                #(
+  CYGWIN* )         cygwin=true  ;; #(
+  Darwin* )         darwin=true  ;; #(
+  MSYS* | MINGW* )  msys=true   ;; #(
+  NonStop* )        nonstop=true ;;
+esac
+
+CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
+
+# Determine the Java command to use to start the JVM.
+if [ -n "$JAVA_HOME" ] ; then
+    if [ -x "$JAVA_HOME/bin/java" ] ; then
+        JAVACMD=$JAVA_HOME/bin/java
+    else
+        die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
+
+Please set the JAVA_HOME variable in your environment to match the
+location of your Java installation."
+    fi
+else
+    JAVACMD=java
+    if ! command -v java > /dev/null 2>&1 ; then
+        die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
+
+Please set the JAVA_HOME variable in your environment to match the
+location of your Java installation."
+    fi
+fi
+
+# Increase the maximum file descriptors if we can.
+if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
+    case $MAX_FD in #(
+      max*)
+        MAX_FD=$( ulimit -H -n ) ||
+            warn "Could not query maximum file descriptor limit"
+      ;;
+    esac
+    case $MAX_FD in  #(
+      '' | soft) :;; #(
+      *)
+        ulimit -n "$MAX_FD" ||
+            warn "Could not set maximum file descriptor limit to $MAX_FD"
+      ;;
+    esac
+fi
+
+# Collect all arguments for the java command, stracks://gnu.org/software/
+# temporary files.
+if "$cygwin" || "$msys" ; then
+    CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" )
+fi
+
+# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
+DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
+
+# Collect all arguments for the java command
+set -- \
+    -classpath "$CLASSPATH" \
+    org.gradle.wrapper.GradleWrapperMain \
+    "$@"
+
+# Stop when "xargs" is not available.
+if ! command -v xargs > /dev/null 2>&1 ; then
+    die "xargs is not available"
+fi
+
+exec "$JAVACMD" "$@"
@@ -0,0 +1,84 @@
+@rem
+@rem Copyright 2015 the original author or authors.
+@rem
+@rem Licensed under the Apache License, Version 2.0 (the "License");
+@rem you may not use this file except in compliance with the License.
+@rem You may obtain a copy of the License at
+@rem
+@rem      https://www.apache.org/licenses/LICENSE-2.0
+@rem
+@rem Unless required by applicable law or agreed to in writing, software
+@rem distributed under the License is distributed on an "AS IS" BASIS,
+@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+@rem See the License for the specific language governing permissions and
+@rem limitations under the License.
+@rem
+
+@if "%DEBUG%"=="" @echo off
+@rem ##########################################################################
+@rem
+@rem  Gradle startup script for Windows
+@rem
+@rem ##########################################################################
+
+@rem Set local scope for the variables with windows NT shell
+if "%OS%"=="Windows_NT" setlocal
+
+set DIRNAME=%~dp0
+if "%DIRNAME%"=="" set DIRNAME=.
+@rem This is normally unused
+set APP_BASE_NAME=%~n0
+set APP_HOME=%DIRNAME%
+
+@rem Resolve any "." and ".." in APP_HOME to make it shorter.
+for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi
+
+@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
+set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m"
+
+@rem Find java.exe
+if defined JAVA_HOME goto findJavaFromJavaHome
+
+set JAVA_EXE=java.exe
+%JAVA_EXE% -version >NUL 2>&1
+if %ERRORLEVEL% equ 0 goto execute
+
+echo. 1>&2
+echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2
+echo. 1>&2
+echo Please set the JAVA_HOME variable in your environment to match the 1>&2
+echo location of your Java installation. 1>&2
+
+goto fail
+
+:findJavaFromJavaHome
+set JAVA_HOME=%JAVA_HOME:"=%
+set JAVA_EXE=%JAVA_HOME%/bin/java.exe
+
+if exist "%JAVA_EXE%" goto execute
+
+echo. 1>&2
+echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2
+echo. 1>&2
+echo Please set the JAVA_HOME variable in your environment to match the 1>&2
+echo location of your Java installation. 1>&2
+
+goto fail
+
+:execute
+@rem Setup the command line
+
+set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar
+
+@rem Execute Gradle
+"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %*
+
+:end
+@rem End local scope for the variables with windows NT shell
+if %OS%==Windows_NT endlocal
+
+:omega
+@exit /b %ERRORLEVEL%
+
+:fail
+@exit /b 1
@@ -0,0 +1,66 @@
+plugins {
+    id("java")
+    id("application")
+}
+
+group = "org.openrewrite"
+version = "0.1.0-SNAPSHOT"
+
+java {
+    toolchain {
+        languageVersion.set(JavaLanguageVersion.of(17))
+    }
+}
+
+repositories {
+    mavenCentral()
+    maven {
+        url = uri("https://oss.sonatype.org/content/repositories/snapshots/")
+    }
+}
+
+val rewriteVersion = "8.45.0"
+
+dependencies {
+    implementation(project(":rewrite-pascal"))
+
+    implementation(platform("org.openrewrite:rewrite-bom:${rewriteVersion}"))
+    implementation("org.openrewrite:rewrite-core")
+    implementation("org.openrewrite:rewrite-java")
+
+    // The Moderne CLI JAR provides TreeSerializer and related classes at runtime.
+    // It must be on the classpath when running the generator (via the launch script).
+    compileOnly(files(findModerneCli()))
+}
+
+application {
+    mainClass.set("org.openrewrite.pascal.generator.PascalLstGenerator")
+}
+
+tasks.withType<JavaCompile> {
+    options.encoding = "UTF-8"
+    options.compilerArgs.addAll(listOf("-parameters"))
+}
+
+// Create a fat JAR with all project dependencies (excluding the CLI JAR)
+tasks.register<Jar>("fatJar") {
+    archiveClassifier.set("all")
+    duplicatesStrategy = DuplicatesStrategy.EXCLUDE
+    from(sourceSets.main.get().output)
+    dependsOn(configurations.runtimeClasspath)
+    from({
+        configurations.runtimeClasspath.get()
+            .filter { it.name.endsWith("jar") }
+            .map { zipTree(it) }
+    })
+    manifest {
+        attributes["Main-Class"] = "org.openrewrite.pascal.generator.PascalLstGenerator"
+    }
+}
+
+fun findModerneCli(): String {
+    val cliJar = file("${System.getProperty("user.home")}/.moderne/cli/dist/lib/moderne-cli.jar")
+    if (cliJar.exists()) return cliJar.absolutePath
+    // Fallback: allow building without the CLI JAR (runtime-only dependency)
+    return ""
+}
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+# Launch script for the Pascal LST Generator.
+# Uses the Moderne CLI's bundled JRE (Java 25) and TreeSerializer.
+#
+# Usage: ./generate-lst.sh <source-dir> [output-dir]
+# Example: ./generate-lst.sh /g/Asuro/Asuro
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
+
+# Moderne CLI location
+CLI_HOME="${HOME}/.moderne/cli/dist"
+CLI_JRE="${CLI_HOME}/jre/bin/java"
+CLI_JAR="${CLI_HOME}/lib/moderne-cli.jar"
+
+if [ ! -f "$CLI_JRE" ]; then
+    echo "Error: Moderne CLI JRE not found at $CLI_JRE"
+    echo "Install it with: curl -s https://docs.moderne.io/moderne-cli/cli-intro | bash"
+    exit 1
+fi
+
+if [ ! -f "$CLI_JAR" ]; then
+    echo "Error: Moderne CLI JAR not found at $CLI_JAR"
+    exit 1
+fi
+
+# Build the generator fat JAR if needed
+FAT_JAR="${SCRIPT_DIR}/build/libs/rewrite-pascal-generator-0.1.0-SNAPSHOT-all.jar"
+if [ ! -f "$FAT_JAR" ]; then
+    echo "Building generator fat JAR..."
+    (cd "$PROJECT_ROOT" && ./gradlew :rewrite-pascal-generator:fatJar)
+fi
+
+echo "Running Pascal LST Generator..."
+"$CLI_JRE" -cp "$FAT_JAR:$CLI_JAR" \
+    org.openrewrite.pascal.generator.PascalLstGenerator "$@"
@@ -0,0 +1,35 @@
+plugins {
+    id("java-library")
+}
+
+java {
+    toolchain {
+        languageVersion.set(JavaLanguageVersion.of(17))
+    }
+}
+
+repositories {
+    mavenCentral()
+    maven {
+        url = uri("https://oss.sonatype.org/content/repositories/snapshots/")
+    }
+}
+
+val rewriteVersion = "8.45.0"
+
+dependencies {
+    compileOnly("com.google.auto.service:auto-service-annotations:1.1.1")
+    annotationProcessor("com.google.auto.service:auto-service:1.1.1")
+
+    implementation(platform("org.openrewrite:rewrite-bom:${rewriteVersion}"))
+    implementation("org.openrewrite:rewrite-remote:0.28.0") {
+        exclude(group = "org.openrewrite", module = "rewrite-pascal")
+    }
+    implementation("org.openrewrite:rewrite-core")
+    implementation("org.openrewrite:rewrite-java")
+    implementation(project(":rewrite-pascal"))
+}
+
+tasks.withType<JavaCompile> {
+    options.encoding = "UTF-8"
+}
@@ -0,0 +1,37 @@
+package org.openrewrite.pascal.remote;
+
+import com.google.auto.service.AutoService;
+import org.openrewrite.pascal.tree.Pas;
+import org.openrewrite.remote.Receiver;
+import org.openrewrite.remote.Sender;
+import org.openrewrite.remote.SenderReceiverProvider;
+import org.openrewrite.remote.Validator;
+
+@SuppressWarnings("rawtypes")
+@AutoService(SenderReceiverProvider.class)
+public class PascalSenderReceiverProvider implements SenderReceiverProvider<Pas> {
+
+    static {
+        Pascal.register();
+    }
+
+    @Override
+    public Class<Pas> getType() {
+        return Pas.class;
+    }
+
+    @Override
+    public Sender<Pas> newSender() {
+        return new PascalSender();
+    }
+
+    @Override
+    public Receiver<Pas> newReceiver() {
+        return new PascalReceiver();
+    }
+
+    @Override
+    public Validator newValidator() {
+        return Validator.fromVisitor(new PascalValidator<>());
+    }
+}
@@ -0,0 +1,4 @@
+@NullMarked
+package org.openrewrite.pascal.remote;
+
+import org.jspecify.annotations.NullMarked;
@@ -0,0 +1,57 @@
+plugins {
+    id("java-library")
+    id("maven-publish")
+}
+
+group = "org.openrewrite"
+version = "0.1.0-SNAPSHOT"
+
+java {
+    toolchain {
+        languageVersion.set(JavaLanguageVersion.of(17))
+    }
+}
+
+repositories {
+    mavenCentral()
+    maven {
+        url = uri("https://oss.sonatype.org/content/repositories/snapshots/")
+    }
+}
+
+val rewriteVersion = "8.45.0"
+val lombokVersion = "1.18.34"
+
+dependencies {
+    implementation(platform("org.openrewrite:rewrite-bom:${rewriteVersion}"))
+
+    implementation("org.openrewrite:rewrite-core")
+    implementation("org.openrewrite:rewrite-java")
+
+    compileOnly("org.projectlombok:lombok:${lombokVersion}")
+    annotationProcessor("org.projectlombok:lombok:${lombokVersion}")
+
+    testCompileOnly("org.projectlombok:lombok:${lombokVersion}")
+    testAnnotationProcessor("org.projectlombok:lombok:${lombokVersion}")
+
+    testImplementation("org.openrewrite:rewrite-test")
+    testImplementation("org.junit.jupiter:junit-jupiter:5.11.4")
+    testRuntimeOnly("org.junit.platform:junit-platform-launcher")
+}
+
+tasks.test {
+    useJUnitPlatform()
+}
+
+tasks.withType<JavaCompile> {
+    options.encoding = "UTF-8"
+    options.compilerArgs.addAll(listOf("-parameters"))
+}
+
+publishing {
+    publications {
+        create<MavenPublication>("maven") {
+            from(components["java"])
+        }
+    }
+}
				`@@ -0,0 +1 @@`
				`// Root project — subprojects apply their own plugins`