Asuro/doc/planning/x86_64.md

x86-64 migration

Status

The x86-64 target boots into Pascal with exception handling, physical/virtual allocation, the shared heap, PIC/PIT interrupts and tested kernel-context preemption. Fatal faults report over COM1. This is a bootstrap kernel, not the complete Asuro desktop or storage stack. The existing x86 target remains the default.

Architecture boundaries and reuse

Follow the namespace convention in namespace.md: architecture units live under src/arch/<architecture>/ and use arch.<architecture>.* names. The compiler-required system.pas retains its special name.

• src/arch/x86_64/boot/stub.asm: 32-bit Multiboot entry and long-mode transition.
• src/arch/x86_64/arch.x86_64.boot.pas: bootstrap entry and serial self-checks.
• src/arch/x86_64/system.pas: minimal compiler runtime definitions.
• src/arch/x86_64/arch.x86_64.isr.pas: IDT/TSS setup, frame validation, and allocation-free exception reporting.
• src/arch/x86_64/isr/entry.asm: generated entry stubs, register/FX save and restore, iretq, emergency stacks, and controlled fault probes.
• src/debug/debug.early.pas: reusable architecture-independent diagnostic formatting, supplied with a character sink by arch.x86_64.debug.
• src/arch/common/system.types.inc: shared scalar/compiler definitions extracted from the existing system unit; native widths follow the selected CPU target.
• toolchain/target.sh: target selection shared by the existing build steps.
• toolchain/x86_64/linker.script: bootstrap address layout.

Do not copy the i386 kernel, drivers, heap, scheduler policy, or runtime wholesale. As each subsystem is enabled, retain portable logic in its existing namespace and put architecture-specific operations behind the appropriate architecture interface. External lvgl/ and wasuro/ sources must remain unchanged.

Build and boot

Rebuild the builder image when first adding the native x86-64 FPC compiler:

docker-compose down --remove-orphans
docker-compose build builder

Build x86-64 using the normal build command with an explicit target:

$env:ASURO_ARCH = 'x86_64'
docker-compose run builder | Tee-Object -FilePath .\build.log
Remove-Item Env:ASURO_ARCH
Select-String -Path .\build.log -Pattern "Error|Fatal|Success|No errors|Failed" -CaseSensitive

Outputs are Asuro-x86_64.iso, bin/x86_64/kernel.bin, and lib/x86_64/. Without ASURO_ARCH, the existing x86 build still produces Asuro.iso, bin/kernel.bin, and lib/. Cleaning and linking only consider the selected target's objects. The x86 compiler search path excludes src/arch/x86_64.

Use VirtualBox for runtime validation. With the configured Asuro VM stopped, select Asuro-x86_64.iso as its optical medium, then start it through the normal VM workflow and read C:\Temp\asuro.log. Restore the original optical medium after testing. The bootstrap does not access storage controllers or disks.

Expected serial output:

ASURO x86_64: IDT/TSS ready; DF/NMI/MC emergency stacks installed
ASURO x86_64: entered Pascal
PASS: long mode, Multiboot handoff, 64-bit Pascal types
PASS: INT3 frame, GPRs, RFLAGS, x87/SSE and IRETQ
PASS: page-fault error code, CR2 and IRETQ recovery
EXCEPTION_TESTS_PASSED=0000000000000002
EXCEPTION_TESTS_FAILED=0000000000000000
ASURO x86_64: bootstrap complete; interrupts disabled

An early single C means CPUID/long-mode support is unavailable; B means an invalid Multiboot magic value. The Pascal checks validate the handoff magic, a non-null info pointer, native/fixed type widths, wide arithmetic, and EFER.LMA.

Repeatable VirtualBox fault checks

The default build runs two returning self-tests. INT3 verifies the saved frame, all 15 general registers including high halves, CF/DF, x87 and XMM0/XMM15 state, and iretq. The handler intentionally destroys FX state before the return path restores it. A write to unmapped 0x100000000 verifies hardware error code 2, CR2, and recovery by changing RIP only for the exact expected probe.

Use the bounded test harness with the configured VM powered off:

.\toolchain\x86_64\test_boot.ps1 -Test none

For deliberately fatal tests, select ud, pf, or df when building, then use the matching harness option. For example:

$env:ASURO_ARCH = 'x86_64'
$env:ASURO_BOOT_TEST = 'df'
docker-compose run builder | Tee-Object -FilePath .\build.log
Remove-Item Env:ASURO_BOOT_TEST
Remove-Item Env:ASURO_ARCH
.\toolchain\x86_64\test_boot.ps1 -Test df

The harness uses localenv.json, temporarily attaches the ISO, starts the VM through virtualbox-wrapper.ps1 -command up -Headless -NoMonitor, validates serial output, captures registers, stops the VM, and restores the original optical medium in a finally block. The new wrapper switches are optional; the existing interactive workflow is unchanged. Storage disks are not changed.

Rebuild without ASURO_BOOT_TEST after testing to restore the normal image.

Bootstrap layout and limitations

• GRUB loads the kernel at physical/virtual 1 MiB using explicit Multiboot v1 load-address fields; the payload is ELF64 with a 32-bit assembly entry.
• Six reserved 4 KiB pages hold the PML4, PDPT, and four page directories. They identity-map the first 4 GiB with 2 MiB leaves.
• A reserved 16 KiB stack is aligned for the SysV AMD64 calling convention. These bounded early allocations exist before any heap and must remain reserved when the physical allocator is introduced.
• Paging, PAE, long mode, x87, and SSE2 support are enabled before calling Pascal.
• A 256-entry IDT uses normalized 64-bit frames. TSS IST1 supplies a 16 KiB double-fault stack; IST2 and IST3 supply separate 8 KiB NMI and machine-check stacks. The 4 KiB IDT, 104-byte TSS, and emergency stacks are bounded early reservations before the allocator exists. Stack guard pages remain future work.
• Entry preserves GPRs and captures CR2 before Pascal code. It clears DF, aligns the call stack, and saves/restores x87/SSE state using FXSAVE64/FXRSTOR64. AVX/OSXSAVE and lazy-FPU switching are not enabled. Future changes must extend this state policy before enabling those features.
• IRQ0 is enabled during timer and context tests, then masked before the final halt. PIC acknowledgement and spurious IRQ7/15 handling are implemented. General driver hooks, APIC/SMP, Pascal unit-init dispatch remain future work.
• The low identity map is temporary. It is not a final process address-space design and does not enforce page-level W^X. New heap mappings use 4 KiB pages at 0xFFFF800000000000, spanning up to 16 GiB with NX when supported. Their backing frames can reside above 4 GiB.
• Unsupported runtime operations intentionally fail to link rather than silently delegating to a hosted runtime or pretending to succeed.

Remaining migration work

Area	Existing assumptions / next work
Runtime	Keep compiler metadata shared; implement required x86-64 helpers and initialization dispatch as consumers are ported.
Common kernel entry	src/asuro.pas currently imports x86 units; separate common boot sequencing from architecture setup before sharing full initialization.
Exceptions	IDT/TSS, IST stacks, normalized frames, diagnostics, and tested iretq paths are implemented. NMI/MC injection, user transitions, guard pages, and general IRQ dispatch remain.
ABI	Bootstrap interrupt alignment and x87/SSE preservation are tested. Audit red-zone assumptions for each added compiler/library and extend preservation before enabling AVX or lazy-FPU switching.
Memory	PMM, VMM and shared heap implemented. Final process layout, guards, MMIO and DMA APIs remain.
Scheduler/loader	Shared scheduling, kernel preemption and process lifecycle tested. Process address spaces and executable loading remain.
Drivers/storage	Audit uint32(pointer) conversions, MMIO and DMA translations, and hardware address limits. Keep disk/USB/controller structures fixed-width.
Graphics	VESA calls into V86, which cannot operate in long mode. Initially use a bootloader framebuffer; rebuild LVGL and verify Pascal/C bindings later.
Validation	Add deliberate exception tests, allocator checks, preemption checks, and storage tests as their implementations become available.

Next milestone: port filesystem/console services and complete driver address/DMA audits. Kernel process lifecycle now uses the shared manager. Do not enable the desktop/storage stack before its dependencies are ported and tested.

Validation performed — 2026-11-07

• x86-64 compile, link, GRUB header validation, and ISO generation passed.
• The existing Asuro VirtualBox VM printed all three expected serial lines.
• First-milestone VirtualBox registers showed CS=0x18, CR0=0x80000013, CR3=0x103000, CR4=0x620, and EFER=0x500 (LME and LMA set).
• Disassembly confirmed 64-bit Pascal code, wide arithmetic, and aligned calls in the bootstrap. This does not establish interrupt safety for future code.
• The full default x86 build passed, including compilation, linking, ISO, and documentation generation, with the x86-64 output directory still present.
• The VM was stopped and its original optical medium restored after testing.
• VirtualBox passed the normal boot tests and deliberate invalid-opcode, page-fault, and double-fault builds. The final write-fault test preserved error code 2 and CR2 0x100000000.
• The exception milestone's normal boot captured TR=0x20, confirming the installed TSS selector, with long mode still active (EFER=0x500).
• Double-fault injection set RSP to unmapped 0x100001000. Diagnostics reported vector 8, error 0, the original RSP, and a frame inside the IST1 stack.
• Fatal exception reports include all 15 GPRs, RIP/CS/RFLAGS/RSP/SS, error code, CR2, and frame address. CR2 is the last page-fault address and may be stale for other exception types.

Runtime evidence is saved locally in the ignored bin/x86_64/boot.log and bin/x86_64/registers.log files for the first milestone, and boot-<test>.log / registers-<test>.log for the exception milestone. Unsupported-CPU boot checks and NMI/MC injection have not yet been performed.

The interrupt-frame and IST layout follows the AMD64 System Programming manual, chapter 8.

Memory, timer and shared scheduling progress

The PMM uses 4 KiB frames and dynamically placed bitmaps, with a 64 GiB management cap. Boot metadata, modules, kernel ranges, low bootstrap memory and allocator metadata stay reserved. VMM allocates high virtual arenas and rolls back both page tables and backing frames on failure. Page tables remain below 4 GiB; backing frames may reside above it.

memory.heap is shared between targets, with native-width pointer/size APIs, 16-byte alignment, overflow rejection and large-allocation rollback. memory.bitmap supplies the common bitmap operations. boot.multiboot owns fixed-width protocol records, with aliases retained in the x86 boot unit. arch.common.legacyirq shares PIC programming and PIT constants.

io.types separates data contracts from console initialization. proc.types uses native saved-stack addresses. The existing round-robin/quantum policy now lives in proc.scheduler, called by both the i386 process manager and x86-64 preemption tests. Process creation, messaging, return/exit, resource cleanup and reaping now use proc.mgr on x86-64. Descriptor tables are now active; console commands, executable loading and full filesystem/controller integration remain unavailable in the bootstrap.

VirtualBox validation has passed at 64 MiB, 2 GiB and 6 GiB. The 6 GiB run mapped, wrote, read and released RAM above 4 GiB. Tests cover PMM accounting and invalid frees, VMM rollback under injected allocation failures, heap alignment/zeroing/ large allocations, timer delivery, spurious IRQ7/15, IF preservation and CLI. Two heap-backed kernel contexts preserve distinct 64-bit registers, x87 and XMM state across timer preemption, then release their stacks after returning to the bootstrap context. Only the BSP is active; IRQ exclusion is not an SMP lock.

.\toolchain\x86_64\test_boot.ps1 -Test none -MemoryMB 64
.\toolchain\x86_64\test_boot.ps1 -Test none -MemoryMB 6144

The harness restores original VM RAM and optical media. Logs use bin/x86_64/boot-<test>-<MemoryMB>.log and matching register filenames; zero means use the configured RAM. All subsystem *_TESTS_FAILED fields must be zero. PIC handling follows the Intel 8259A datasheet.

The final i386 regression build passed compilation, linking, ISO and documentation steps. VirtualBox reached Boot complete - halting into IDLE; reported boot test suites had zero failures. Evidence: bin/x86_64/i386-regression.log. Fatal UD, write-PF and broken-stack DF tests also passed after the context-return path was extended. Final normal images are built without fault injection.

Shared process lifecycle

proc.mgr now runs on both targets with native process-pointer slots, stable monotonic IDs, suspended creation, messaging, exit and resource ownership. Initial stack layout lives in the architecture context units. x86-64's scheduler adapter saves the current stack and calls the same scheduler/manager as i386. A shared reaper process handles destruction on both targets outside IRQ context. Timer selection and kill requests coalesce worker wakeups; cleanup runs after registry/resource detachment. The reaper sleeps while there is no pending work.

core.ds.dynamic contains the common dynamic-list implementation; existing core.ds.lists exports compatible aliases/wrappers. Native addresses, checked capacity arithmetic, overlap-safe insertion and empty-list regrowth replace 32-bit arithmetic. Deletion retains capacity; explicit DL_ShrinkToFit frees excess storage without allocating inside every scheduler deletion. memory.raw supplies allocation-free scalar fill/move operations.

Process tests cover suspended publication, high-address lookup, killed tasks never running, normal return, explicit exit, child notification waking an awaiting parent, callback reentry without double cleanup, command argument/ stream publication, sleep using the PIT divisor, and restored heap accounting. The full i386 build and VirtualBox boot passed after these shared changes; evidence is in bin/x86_64/i386-lifecycle-regression.log.

The dedicated reaper revision passed all 15 process/container checks at 64 MiB and 6 GiB, including the corrected check-before-await notification protocol. The final i386 build and VirtualBox boot also passed with the shared reaper. VM RAM and original optical media were restored and the VM was powered off.

Descriptor tables

The shared driver.storage.fdtable now uses native memory operations and balanced IRQ locking. Every x86-64 process, including system tasks, owns its normal FD table. Storage protocol types depend on shared record definitions rather than bringing in the linked-list implementation and its console dependencies.

Close reserves an entry until callbacks and owned-string cleanup finish, rejects active async references and reference-count overflow, and prevents callback reentry from freeing it twice. Table teardown atomically rejects new users before process removal. An exited process with active FD references remains alive; final completion only wakes the shared reaper, which performs cleanup in process context. Waiting does not repeatedly scan descriptors on every timer tick.

VirtualBox passes 15 descriptor checks plus 16 lifecycle/container checks, including an exited child held alive by an async FD reference until another task releases it. No disk I/O is performed by these bootstrap tests. This validates FD ownership, not full VFS operation or controller DMA support in long mode.

The descriptor-table revision also passes the complete i386 build and VirtualBox boot, including the existing storage/VFS suites and normal volume discovery. Evidence: bin/x86_64/i386-fd-regression.log. Original VM settings are restored.

Shared strings and console command infrastructure

core.util memory helpers accept native addresses and sizes, reusing scalar memory.raw operations while retaining i386's existing large-copy REP path. core.strings no longer imports the x86 utility or linked-list implementation; all string copy addresses are native. Its common tests cover signed 64-bit minimum formatting, overflowing substring/match bounds, and terminator size rejection. The x86-64 system unit supplies Double-to-int64 truncation using CVTTSD2SI without changing rounding control state.

io.stdio now builds on both targets: growable buffers, bounded command parsing, case-insensitive allocation-free lookup, registry growth, ANSI output, and ECHO/CLEAR/HELP share one implementation. Buffer appends reserve space in bulk and support self-append across relocation. Allocation failures retain the old buffer; parser failures release all partially built parameters. Completion detaches the halt owner before invoking callbacks, allowing safe reentry.

VFS working-directory delegation and VERSION/TIME/REBOOT registration remain gated until their platform dependencies are integrated. This is command infrastructure, not yet an interactive keyboard/display/serial shell. No VFS or storage-controller support is implied by compiling the console unit.

The 64-bit bootstrap runs 162 shared string tests and 17 stdio tests. The process suite also launches the registered ECHO through runCommand with real parsed arguments and a growable output buffer, bringing it to 17 lifecycle/container checks. The 6 GiB VirtualBox run verifies scheduled ECHO and physical RAM above 4 GiB. The complete i386 build and boot also pass, including 162 string tests, stdio tests, normal mounts, and the existing storage/VFS suites. Evidence: bin/x86_64/i386-console-regression.log and the normal x86-64 boot logs.

Build-selected platform adapters

Shared Pascal units no longer select CPUs with CPU64/CPU386 conditionals. ASURO_ARCH selects a single src/arch/<target>/platform directory in the compiler search path. Both targets implement the same arch.platform.cpu, arch.platform.memory, arch.platform.process, and arch.platform.diagnostics unit names. Their declarations are shared through src/arch/common/platform.*.inc; implementations delegate to the existing architecture units. Calls are resolved at compile/link time, with no runtime architecture selection or function-pointer dispatch.

The bitmap heap, allocation headers, process registry, scheduling policy, strings, console buffers/parser, and descriptor lifecycle remain single shared implementations. Native scalar widths live in each selected system unit. memory.raw builds its fill pattern from native word width without a CPU branch.

arch.x86.startup now registers heap/process/console startup with the boot manager. arch.x86.console owns VERSION/TIME/REBOOT commands and their driver dependencies. The x86-64 bootstrap retains its explicit startup sequence. Unused working-directory wrappers moved from io.stdio into io.directory, which explicitly depends on VFS. Existing in-repository callers need no changes.

Existing DEBUG_LMM allocator guards and architecture-local fault-injection build flags remain; these are feature/test options, not architecture selection.

Validation: complete i386 build and VirtualBox boot, including 17 stdio checks, 162 string checks, storage/VFS suites and normal mounts. Evidence: bin/x86_64/i386-platform-regression.log. x86-64 boot validation covers heap, IRQs, contexts, descriptors, strings, console and scheduled command execution.

The final adapter build also passes the 6 GiB VirtualBox bootstrap, including physical memory above 4 GiB and all 162 string, 17 console, 15 descriptor, and 17 lifecycle/container checks. VM RAM and optical media were restored; the VM is powered off.

Source search paths are discovered recursively from shared roots and the selected architecture tree, with other architectures excluded. Both targets use the same compiler invocation; target.sh loads compiler options and entry points from toolchain/targets/<architecture>.sh. Directory arguments are NUL-delimited, sorted, and passed as an array so new subdirectories and spaces in names do not require compile-script edits. Full i386 and x86-64 builds pass using this discovery path.

All shared build steps now consume the selected profile: pipeline stages, assembler format and source roots, C/Pascal compiler options, linker format, entry points, output locations, optional compiler runtime, and image validation. target.sh loads the profile by filename instead of keeping an architecture switch. Assembly files, object files, and static libraries are discovered; the configured assembly entry is linked first to preserve the Multiboot header. Initialization-table, icon, LVGL, version, and checksum generation use target settings too. Full i386 and x86-64 builds and VirtualBox boot checks validate this pipeline; the i386 boot includes normal mounts and existing storage tests.