# x86-64 migration ## Status The x86-64 target boots into Pascal with exception handling, physical/virtual allocation, the shared heap, PIC/PIT interrupts and tested kernel-context preemption. Fatal faults report over COM1. This is a bootstrap kernel, not the complete Asuro desktop or storage stack. The existing x86 target remains the default. ## Architecture boundaries and reuse Follow the namespace convention in [namespace.md](namespace.md): architecture units live under `src/arch//` and use `arch..*` names. The compiler-required `system.pas` retains its special name. - `src/arch/x86_64/boot/stub.asm`: 32-bit Multiboot entry and long-mode transition. - `src/arch/x86_64/arch.x86_64.boot.pas`: bootstrap entry and serial self-checks. - `src/arch/x86_64/system.pas`: minimal compiler runtime definitions. - `src/arch/x86_64/arch.x86_64.isr.pas`: IDT/TSS setup, frame validation, and allocation-free exception reporting. - `src/arch/x86_64/isr/entry.asm`: generated entry stubs, register/FX save and restore, `iretq`, emergency stacks, and controlled fault probes. - `src/debug/debug.early.pas`: reusable architecture-independent diagnostic formatting, supplied with a character sink by `arch.x86_64.debug`. - `src/arch/common/system.types.inc`: shared scalar/compiler definitions extracted from the existing system unit; native widths follow the selected CPU target. - `toolchain/target.sh`: target selection shared by the existing build steps. - `toolchain/x86_64/linker.script`: bootstrap address layout. Do not copy the i386 kernel, drivers, heap, scheduler policy, or runtime wholesale. As each subsystem is enabled, retain portable logic in its existing namespace and put architecture-specific operations behind the appropriate architecture interface. External `lvgl/` and `wasuro/` sources must remain unchanged. ## Build and boot Rebuild the builder image when first adding the native x86-64 FPC compiler: ```powershell docker-compose down --remove-orphans docker-compose build builder ``` Build x86-64 using the normal build command with an explicit target: ```powershell $env:ASURO_ARCH = 'x86_64' docker-compose run builder | Tee-Object -FilePath .\build.log Remove-Item Env:ASURO_ARCH Select-String -Path .\build.log -Pattern "Error|Fatal|Success|No errors|Failed" -CaseSensitive ``` Outputs are `Asuro-x86_64.iso`, `bin/x86_64/kernel.bin`, and `lib/x86_64/`. Without `ASURO_ARCH`, the existing x86 build still produces `Asuro.iso`, `bin/kernel.bin`, and `lib/`. Cleaning and linking only consider the selected target's objects. The x86 compiler search path excludes `src/arch/x86_64`. Use VirtualBox for runtime validation. With the configured Asuro VM stopped, select `Asuro-x86_64.iso` as its optical medium, then start it through the normal VM workflow and read `C:\Temp\asuro.log`. Restore the original optical medium after testing. The bootstrap does not access storage controllers or disks. Expected serial output: ```text ASURO x86_64: IDT/TSS ready; DF/NMI/MC emergency stacks installed ASURO x86_64: entered Pascal PASS: long mode, Multiboot handoff, 64-bit Pascal types PASS: INT3 frame, GPRs, RFLAGS, x87/SSE and IRETQ PASS: page-fault error code, CR2 and IRETQ recovery EXCEPTION_TESTS_PASSED=0000000000000002 EXCEPTION_TESTS_FAILED=0000000000000000 ASURO x86_64: bootstrap complete; interrupts disabled ``` An early single `C` means CPUID/long-mode support is unavailable; `B` means an invalid Multiboot magic value. The Pascal checks validate the handoff magic, a non-null info pointer, native/fixed type widths, wide arithmetic, and EFER.LMA. ### Repeatable VirtualBox fault checks The default build runs two returning self-tests. `INT3` verifies the saved frame, all 15 general registers including high halves, CF/DF, x87 and XMM0/XMM15 state, and `iretq`. The handler intentionally destroys FX state before the return path restores it. A write to unmapped `0x100000000` verifies hardware error code `2`, CR2, and recovery by changing RIP only for the exact expected probe. Use the bounded test harness with the configured VM powered off: ```powershell .\toolchain\x86_64\test_boot.ps1 -Test none ``` For deliberately fatal tests, select `ud`, `pf`, or `df` when building, then use the matching harness option. For example: ```powershell $env:ASURO_ARCH = 'x86_64' $env:ASURO_BOOT_TEST = 'df' docker-compose run builder | Tee-Object -FilePath .\build.log Remove-Item Env:ASURO_BOOT_TEST Remove-Item Env:ASURO_ARCH .\toolchain\x86_64\test_boot.ps1 -Test df ``` The harness uses `localenv.json`, temporarily attaches the ISO, starts the VM through `virtualbox-wrapper.ps1 -command up -Headless -NoMonitor`, validates serial output, captures registers, stops the VM, and restores the original optical medium in a `finally` block. The new wrapper switches are optional; the existing interactive workflow is unchanged. Storage disks are not changed. Rebuild without `ASURO_BOOT_TEST` after testing to restore the normal image. ## Bootstrap layout and limitations - GRUB loads the kernel at physical/virtual 1 MiB using explicit Multiboot v1 load-address fields; the payload is ELF64 with a 32-bit assembly entry. - Six reserved 4 KiB pages hold the PML4, PDPT, and four page directories. They identity-map the first 4 GiB with 2 MiB leaves. - A reserved 16 KiB stack is aligned for the SysV AMD64 calling convention. These bounded early allocations exist before any heap and must remain reserved when the physical allocator is introduced. - Paging, PAE, long mode, x87, and SSE2 support are enabled before calling Pascal. - A 256-entry IDT uses normalized 64-bit frames. TSS IST1 supplies a 16 KiB double-fault stack; IST2 and IST3 supply separate 8 KiB NMI and machine-check stacks. The 4 KiB IDT, 104-byte TSS, and emergency stacks are bounded early reservations before the allocator exists. Stack guard pages remain future work. - Entry preserves GPRs and captures CR2 before Pascal code. It clears DF, aligns the call stack, and saves/restores x87/SSE state using FXSAVE64/FXRSTOR64. AVX/OSXSAVE and lazy-FPU switching are not enabled. Future changes must extend this state policy before enabling those features. - IRQ0 is enabled during timer and context tests, then masked before the final halt. PIC acknowledgement and spurious IRQ7/15 handling are implemented. General driver hooks, APIC/SMP, Pascal unit-init dispatch remain future work. - The low identity map is temporary. It is not a final process address-space design and does not enforce page-level W^X. New heap mappings use 4 KiB pages at `0xFFFF800000000000`, spanning up to 16 GiB with NX when supported. Their backing frames can reside above 4 GiB. - Unsupported runtime operations intentionally fail to link rather than silently delegating to a hosted runtime or pretending to succeed. ## Remaining migration work | Area | Existing assumptions / next work | |---|---| | Runtime | Keep compiler metadata shared; implement required x86-64 helpers and initialization dispatch as consumers are ported. | | Common kernel entry | `src/asuro.pas` currently imports x86 units; separate common boot sequencing from architecture setup before sharing full initialization. | | Exceptions | IDT/TSS, IST stacks, normalized frames, diagnostics, and tested `iretq` paths are implemented. NMI/MC injection, user transitions, guard pages, and general IRQ dispatch remain. | | ABI | Bootstrap interrupt alignment and x87/SSE preservation are tested. Audit red-zone assumptions for each added compiler/library and extend preservation before enabling AVX or lazy-FPU switching. | | Memory | PMM, VMM and shared heap implemented. Final process layout, guards, MMIO and DMA APIs remain. | | Scheduler/loader | Shared scheduling, kernel preemption and process lifecycle tested. Process address spaces and executable loading remain. | | Drivers/storage | Audit `uint32(pointer)` conversions, MMIO and DMA translations, and hardware address limits. Keep disk/USB/controller structures fixed-width. | | Graphics | VESA calls into V86, which cannot operate in long mode. Initially use a bootloader framebuffer; rebuild LVGL and verify Pascal/C bindings later. | | Validation | Add deliberate exception tests, allocator checks, preemption checks, and storage tests as their implementations become available. | Next milestone: port filesystem/console services and complete driver address/DMA audits. Kernel process lifecycle now uses the shared manager. Do not enable the desktop/storage stack before its dependencies are ported and tested. ## Validation performed — 2026-11-07 - x86-64 compile, link, GRUB header validation, and ISO generation passed. - The existing Asuro VirtualBox VM printed all three expected serial lines. - First-milestone VirtualBox registers showed `CS=0x18`, `CR0=0x80000013`, `CR3=0x103000`, `CR4=0x620`, and `EFER=0x500` (LME and LMA set). - Disassembly confirmed 64-bit Pascal code, wide arithmetic, and aligned calls in the bootstrap. This does not establish interrupt safety for future code. - The full default x86 build passed, including compilation, linking, ISO, and documentation generation, with the x86-64 output directory still present. - The VM was stopped and its original optical medium restored after testing. - VirtualBox passed the normal boot tests and deliberate invalid-opcode, page-fault, and double-fault builds. The final write-fault test preserved error code `2` and CR2 `0x100000000`. - The exception milestone's normal boot captured `TR=0x20`, confirming the installed TSS selector, with long mode still active (`EFER=0x500`). - Double-fault injection set RSP to unmapped `0x100001000`. Diagnostics reported vector `8`, error `0`, the original RSP, and a frame inside the IST1 stack. - Fatal exception reports include all 15 GPRs, RIP/CS/RFLAGS/RSP/SS, error code, CR2, and frame address. CR2 is the last page-fault address and may be stale for other exception types. Runtime evidence is saved locally in the ignored `bin/x86_64/boot.log` and `bin/x86_64/registers.log` files for the first milestone, and `boot-.log` / `registers-.log` for the exception milestone. Unsupported-CPU boot checks and NMI/MC injection have not yet been performed. The interrupt-frame and IST layout follows the [AMD64 System Programming manual, chapter 8](https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/programmer-references/24593.pdf). ## Memory, timer and shared scheduling progress The PMM uses 4 KiB frames and dynamically placed bitmaps, with a 64 GiB management cap. Boot metadata, modules, kernel ranges, low bootstrap memory and allocator metadata stay reserved. VMM allocates high virtual arenas and rolls back both page tables and backing frames on failure. Page tables remain below 4 GiB; backing frames may reside above it. `memory.heap` is shared between targets, with native-width pointer/size APIs, 16-byte alignment, overflow rejection and large-allocation rollback. `memory.bitmap` supplies the common bitmap operations. `boot.multiboot` owns fixed-width protocol records, with aliases retained in the x86 boot unit. `arch.common.legacyirq` shares PIC programming and PIT constants. `io.types` separates data contracts from console initialization. `proc.types` uses native saved-stack addresses. The existing round-robin/quantum policy now lives in `proc.scheduler`, called by both the i386 process manager and x86-64 preemption tests. Process creation, messaging, return/exit, resource cleanup and reaping now use `proc.mgr` on x86-64. Descriptor tables are now active; console commands, executable loading and full filesystem/controller integration remain unavailable in the bootstrap. VirtualBox validation has passed at 64 MiB, 2 GiB and 6 GiB. The 6 GiB run mapped, wrote, read and released RAM above 4 GiB. Tests cover PMM accounting and invalid frees, VMM rollback under injected allocation failures, heap alignment/zeroing/ large allocations, timer delivery, spurious IRQ7/15, IF preservation and CLI. Two heap-backed kernel contexts preserve distinct 64-bit registers, x87 and XMM state across timer preemption, then release their stacks after returning to the bootstrap context. Only the BSP is active; IRQ exclusion is not an SMP lock. ```powershell .\toolchain\x86_64\test_boot.ps1 -Test none -MemoryMB 64 .\toolchain\x86_64\test_boot.ps1 -Test none -MemoryMB 6144 ``` The harness restores original VM RAM and optical media. Logs use `bin/x86_64/boot--.log` and matching register filenames; zero means use the configured RAM. All subsystem `*_TESTS_FAILED` fields must be zero. PIC handling follows the [Intel 8259A datasheet](https://www.cs.umb.edu/cs341/Intel8259/I8259APIC.pdf). The final i386 regression build passed compilation, linking, ISO and documentation steps. VirtualBox reached `Boot complete - halting into IDLE`; reported boot test suites had zero failures. Evidence: `bin/x86_64/i386-regression.log`. Fatal UD, write-PF and broken-stack DF tests also passed after the context-return path was extended. Final normal images are built without fault injection. ## Shared process lifecycle `proc.mgr` now runs on both targets with native process-pointer slots, stable monotonic IDs, suspended creation, messaging, exit and resource ownership. Initial stack layout lives in the architecture context units. x86-64's scheduler adapter saves the current stack and calls the same scheduler/manager as i386. A shared reaper process handles destruction on both targets outside IRQ context. Timer selection and kill requests coalesce worker wakeups; cleanup runs after registry/resource detachment. The reaper sleeps while there is no pending work. `core.ds.dynamic` contains the common dynamic-list implementation; existing `core.ds.lists` exports compatible aliases/wrappers. Native addresses, checked capacity arithmetic, overlap-safe insertion and empty-list regrowth replace 32-bit arithmetic. Deletion retains capacity; explicit `DL_ShrinkToFit` frees excess storage without allocating inside every scheduler deletion. `memory.raw` supplies allocation-free scalar fill/move operations. Process tests cover suspended publication, high-address lookup, killed tasks never running, normal return, explicit exit, child notification waking an awaiting parent, callback reentry without double cleanup, command argument/ stream publication, sleep using the PIT divisor, and restored heap accounting. The full i386 build and VirtualBox boot passed after these shared changes; evidence is in `bin/x86_64/i386-lifecycle-regression.log`. The dedicated reaper revision passed all 15 process/container checks at 64 MiB and 6 GiB, including the corrected check-before-await notification protocol. The final i386 build and VirtualBox boot also passed with the shared reaper. VM RAM and original optical media were restored and the VM was powered off. ## Descriptor tables The shared `driver.storage.fdtable` now uses native memory operations and balanced IRQ locking. Every x86-64 process, including system tasks, owns its normal FD table. Storage protocol types depend on shared record definitions rather than bringing in the linked-list implementation and its console dependencies. Close reserves an entry until callbacks and owned-string cleanup finish, rejects active async references and reference-count overflow, and prevents callback reentry from freeing it twice. Table teardown atomically rejects new users before process removal. An exited process with active FD references remains alive; final completion only wakes the shared reaper, which performs cleanup in process context. Waiting does not repeatedly scan descriptors on every timer tick. VirtualBox passes 15 descriptor checks plus 16 lifecycle/container checks, including an exited child held alive by an async FD reference until another task releases it. No disk I/O is performed by these bootstrap tests. This validates FD ownership, not full VFS operation or controller DMA support in long mode. The descriptor-table revision also passes the complete i386 build and VirtualBox boot, including the existing storage/VFS suites and normal volume discovery. Evidence: `bin/x86_64/i386-fd-regression.log`. Original VM settings are restored. ## Shared strings and console command infrastructure `core.util` memory helpers accept native addresses and sizes, reusing scalar `memory.raw` operations while retaining i386's existing large-copy REP path. `core.strings` no longer imports the x86 utility or linked-list implementation; all string copy addresses are native. Its common tests cover signed 64-bit minimum formatting, overflowing substring/match bounds, and terminator size rejection. The x86-64 system unit supplies Double-to-int64 truncation using CVTTSD2SI without changing rounding control state. `io.stdio` now builds on both targets: growable buffers, bounded command parsing, case-insensitive allocation-free lookup, registry growth, ANSI output, and ECHO/CLEAR/HELP share one implementation. Buffer appends reserve space in bulk and support self-append across relocation. Allocation failures retain the old buffer; parser failures release all partially built parameters. Completion detaches the halt owner before invoking callbacks, allowing safe reentry. VFS working-directory delegation and VERSION/TIME/REBOOT registration remain gated until their platform dependencies are integrated. This is command infrastructure, not yet an interactive keyboard/display/serial shell. No VFS or storage-controller support is implied by compiling the console unit. The 64-bit bootstrap runs 162 shared string tests and 17 stdio tests. The process suite also launches the registered ECHO through `runCommand` with real parsed arguments and a growable output buffer, bringing it to 17 lifecycle/container checks. The 6 GiB VirtualBox run verifies scheduled ECHO and physical RAM above 4 GiB. The complete i386 build and boot also pass, including 162 string tests, stdio tests, normal mounts, and the existing storage/VFS suites. Evidence: `bin/x86_64/i386-console-regression.log` and the normal x86-64 boot logs. ## Build-selected platform adapters Shared Pascal units no longer select CPUs with `CPU64`/`CPU386` conditionals. `ASURO_ARCH` selects a single `src/arch//platform` directory in the compiler search path. Both targets implement the same `arch.platform.cpu`, `arch.platform.memory`, `arch.platform.process`, and `arch.platform.diagnostics` unit names. Their declarations are shared through `src/arch/common/platform.*.inc`; implementations delegate to the existing architecture units. Calls are resolved at compile/link time, with no runtime architecture selection or function-pointer dispatch. The bitmap heap, allocation headers, process registry, scheduling policy, strings, console buffers/parser, and descriptor lifecycle remain single shared implementations. Native scalar widths live in each selected system unit. `memory.raw` builds its fill pattern from native word width without a CPU branch. `arch.x86.startup` now registers heap/process/console startup with the boot manager. `arch.x86.console` owns VERSION/TIME/REBOOT commands and their driver dependencies. The x86-64 bootstrap retains its explicit startup sequence. Unused working-directory wrappers moved from `io.stdio` into `io.directory`, which explicitly depends on VFS. Existing in-repository callers need no changes. Existing `DEBUG_LMM` allocator guards and architecture-local fault-injection build flags remain; these are feature/test options, not architecture selection. Validation: complete i386 build and VirtualBox boot, including 17 stdio checks, 162 string checks, storage/VFS suites and normal mounts. Evidence: `bin/x86_64/i386-platform-regression.log`. x86-64 boot validation covers heap, IRQs, contexts, descriptors, strings, console and scheduled command execution. The final adapter build also passes the 6 GiB VirtualBox bootstrap, including physical memory above 4 GiB and all 162 string, 17 console, 15 descriptor, and 17 lifecycle/container checks. VM RAM and optical media were restored; the VM is powered off. Source search paths are discovered recursively from shared roots and the selected architecture tree, with other architectures excluded. Both targets use the same compiler invocation; `target.sh` loads compiler options and entry points from `toolchain/targets/.sh`. Directory arguments are NUL-delimited, sorted, and passed as an array so new subdirectories and spaces in names do not require compile-script edits. Full i386 and x86-64 builds pass using this discovery path. All shared build steps now consume the selected profile: pipeline stages, assembler format and source roots, C/Pascal compiler options, linker format, entry points, output locations, optional compiler runtime, and image validation. `target.sh` loads the profile by filename instead of keeping an architecture switch. Assembly files, object files, and static libraries are discovered; the configured assembly entry is linked first to preserve the Multiboot header. Initialization-table, icon, LVGL, version, and checksum generation use target settings too. Full i386 and x86-64 builds and VirtualBox boot checks validate this pipeline; the i386 boot includes normal mounts and existing storage tests.