Compare commits
14
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
36ec7bfdb6 | ||
|
|
6c3c0e63d5 | ||
|
|
df71c59aea | ||
|
|
97244a2b95 | ||
|
|
ecca496978 | ||
|
|
e52b4a9763 | ||
|
|
f78eaa3811 | ||
|
|
acb4a15e41 | ||
|
|
d3581eea2e | ||
|
|
1fc00e3e7c | ||
|
|
2f837ace6e | ||
|
|
f8182b24b3 | ||
|
|
f0531ff861 | ||
|
|
3d9c4cb49a |
No files matched your search
@@ -14,5 +14,6 @@ localenv.json
|
|||||||
/lvgl/
|
/lvgl/
|
||||||
dockerout.txt
|
dockerout.txt
|
||||||
AGENTS.md
|
AGENTS.md
|
||||||
|
lessons_learnt.md
|
||||||
*.log
|
*.log
|
||||||
/doc/*.md
|
/doc/*.md
|
||||||
-157
File diff suppressed because it is too large.
Load diff
+1
-1
@@ -10,4 +10,4 @@ for dir in $(find src wasuro -type d 2>/dev/null); do
|
|||||||
FU_PATHS="$FU_PATHS -Fu$dir"
|
FU_PATHS="$FU_PATHS -Fu$dir"
|
||||||
done
|
done
|
||||||
|
|
||||||
fpc -Aelf -gw -g -gl -n -v0ew -O3 -OpPENTIUM3 -Si -Sc -Sg -Xd -CX -XXs -CfSSE -CfSSE2 -Rintel -Pi386 -Tlinux -FElib/ $FU_PATHS src/kernel.pas
|
fpc -Aelf -gw -g -gl -n -v0e -O3 -OpPENTIUM3 -Si -Sc -Sg -Xd -CX -XXs -CfSSE -CfSSE2 -Rintel -Pi386 -Tlinux -FElib/ $FU_PATHS src/kernel.pas
|
||||||
@@ -5,3 +5,4 @@ echo " "
|
|||||||
echo "Compiling Stub..."
|
echo "Compiling Stub..."
|
||||||
echo " "
|
echo " "
|
||||||
nasm -f elf src/stub/stub.asm -o lib/stub.o
|
nasm -f elf src/stub/stub.asm -o lib/stub.o
|
||||||
|
nasm -f elf src/stub/splash_tga.asm -o lib/splash_tga.o
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 1.0 MiB |
Binary file not shown.
@@ -1,40 +0,0 @@
|
|||||||
# Lessons Learnt
|
|
||||||
|
|
||||||
## 1. File Pickers Must Be Proper Reusable OS Components
|
|
||||||
Inline pickers hacked into app code are broken and not reusable. Build file pickers as a standalone unit (`filepicker.pas`) that any program can call via a callback.
|
|
||||||
|
|
||||||
## 2. All Errors Need Visible In-App Feedback
|
|
||||||
Never use `syslog.logln` as the only error output. Always show the user a dialog, msgbox, or status bar message. Syslog is supplemental only.
|
|
||||||
|
|
||||||
## 4. Shell Command Output Goes to stdout_buf, Not syslog
|
|
||||||
Use `stdio.bufWriteStr(stdout_buf, ...)` for all shell command output. `syslog` is for kernel diagnostics only.
|
|
||||||
|
|
||||||
## 5. Unit Test Convention
|
|
||||||
New subsystem units get a `procedure UnitTest` (called at boot from kernel.pas) and a `procedure init` that registers a CLI command so tests can be re-run at runtime. Tests use a nested `Assert(condition, name)` that increments `passed`/`failed` and logs failures via `syslog.logln`. At-boot tests must not do disk I/O — only test in-memory/virtual structures.
|
|
||||||
|
|
||||||
## 6. ISR-Visible Spin-Wait Flags Must Use Pointer Dereferencing
|
|
||||||
FPC has no `volatile` keyword. Any `while flag = 0 do asm hlt end` loop where `flag` is set by an ISR will hang forever because FPC hoists the load into a register before the loop and never re-reads memory. Always access the flag through a pointer: `while puint32(@flag)^ = 0 do proc_yield()`. This applies everywhere in the kernel that spins waiting for ISR-written state (e.g. `submit_io_wait`, `proc_await`-style loops, DMA done flags).
|
|
||||||
|
|
||||||
## 7. Never Spawn a Process Per I/O Operation
|
|
||||||
Spawning a new kernel process for each FAT32 read/write/dir operation is pure overhead: process creation (kalloc 8 KB stack, fake IRET frame, DList insertion), an extra context switch, and a double-wait (VFS spins on the worker, worker spins on AHCI). Instead, submit the I/O directly and park the calling process via `psAwaiting` until the AHCI ISR fires the completion callback — zero extra processes, zero wasted scheduler ticks.
|
|
||||||
|
|
||||||
## 8. Use psAwaiting to Park a Process Pending Async I/O (not proc_yield spin)
|
|
||||||
The correct way to block a process on an async I/O result is: (1) set up a completion callback that sets `State := psReady` then `Done := 1`, (2) atomically check if the operation is already done (under CLI) and set `CurrentProcess^.State := psAwaiting` if not, (3) spin on `Done` via `puint32(@wait.Done)^` per Lesson #6 with `asm hlt end` inside the loop. The process is removed from the runnable set while hardware works. **Do NOT call `proc_await` from `submit_io_wait`** — `proc_await` spins on `CurrentProcess^.State = psAwaiting` without pointer indirection and FPC caches the value in a register, causing it to return immediately before the I/O completes. The calling code then reads garbage from the DMA buffer, causing invalid pointer dereferences and page faults.
|
|
||||||
|
|
||||||
## 9. VFS GetDirectoryListing Must Return a Snapshot, Not a Live Tree Reference
|
|
||||||
`GetDirectoryListing` for `otVDIRECTORY` nodes previously returned `PHashMap(Obj^.Reference)` — a direct pointer into the live VFS tree. When callers called `FreeDirectoryListing` on the returned map, it freed the live VFSObjects and their name strings, destroying the VFS tree. Any subsequent `GetObjectFromPath` call would then dereference the trashed memory and page fault. **Fix**: always return a heap-allocated snapshot copy of the directory (new `PHashMap`, new `PVFSObject` entries with `stringCopy`'d keys). `FreeDirectoryListing` can then safely own and free the snapshot without touching the live tree. The rule: any function that returns a `PHashMap` to a caller who will call `FreeDirectoryListing` must allocate a fresh map — never return a raw interior pointer to a live data structure.
|
|
||||||
|
|
||||||
## 10. Never Call Synchronous Disk I/O From an LVGL Timer or Event Callback
|
|
||||||
LVGL timer callbacks (`lv_timer_create`) and event callbacks run synchronously inside the LVGL render loop. Calling any function that parks the current process waiting for disk I/O (e.g. `vfs.FileSize`, `vfs.OpenFile`, `vfs.ReadFile`) from inside a timer or event callback will hang the entire UI — the process blocks mid-callback and the LVGL loop never resumes. All disk I/O that must occur in a UI context must either: (a) use the async VFS API (`OpenFileAsync`, `WriteFileAsync`) with a completion callback that triggers a deferred LVGL update, or (b) be deferred to a dedicated worker process that signals back via a flag the UI can poll. `GetDirectoryListingFrom` is the one bulk exception — it was already present in the original picker and works because it loads into a snapshot synchronously as a single operation; per-file calls like `FileSize` in a loop are the problem.
|
|
||||||
|
|
||||||
## 11. Never Allocate Large Arrays as Local Variables in Deep Kernel Call Chains
|
|
||||||
Local variables are stack-allocated in Pascal. Kernel processes have a fixed, limited stack (≈8 KB). In deep LVGL callback chains (kernel → LVGL render loop → timer callback → app code), there may only be a few KB of usable stack remaining by the time app code executes. Declaring local arrays larger than ~200–400 bytes (e.g. `array[0..255] of pchar` = 1024 bytes) inside any procedure that runs in this context will silently overflow the stack, corruptung adjacent memory and causing a page fault with a garbled call trace. **Rule**: any local variable array larger than ~128 bytes in a UI callback or procedure called from one must be moved to the heap with `kalloc`/`kfree`. The corruption pattern is a page fault with a call stack full of completely unrelated function names.
|
|
||||||
|
|
||||||
## 12. Pascal Silently Ignores Everything After `end.` — Dead Code Masquerades as Live Code
|
|
||||||
FPC compiles everything up to and including the first `end.` and silently discards the rest of the file. If a file is partially rewritten by prepending a new implementation but the old implementation is not deleted, the old code survives after `end.`. Forward declarations resolve to the **last** implementation before `end.` — so if the new code was added before the old `end.` but the old procedures were in the "dead" zone they have no effect. However the reverse is dangerous: if the old `end.` was replaced/removed and the new code placed before it, the old procedures in the dead zone still compile to nothing but pollute the source. **Rule**: after any large-scale replacement of a Pascal unit, explicitly check where `end.` falls with a text search and PowerShell-truncate the file at the first `end.`. Never assume a multi-step replacement tool correctly removed all old code — always verify with `Select-String -Pattern "^end\.$"`.
|
|
||||||
|
|
||||||
## 13. `for i := 0 to uint32_count - 1` Wraps to 4 Billion When Count is Zero
|
|
||||||
In Free Pascal, `uint32` arithmetic wraps: if `dir_count` is `uint32` and equals `0`, then `dir_count - 1 = 0xFFFFFFFF`. A `for i := 0 to dir_count - 1 do` loop with zero count does NOT skip — it runs 4 billion iterations, immediately accessing invalid memory and page faulting. FPC does not emit a warning for this. **Rule**: always guard `uint32`-bounded for loops: `if count > 0 then for i := 0 to count - 1 do`. Alternatively, use `sint32` for loop counters that might legitimately be zero.
|
|
||||||
|
|
||||||
## 14. FAT32 Deleted Entries ($E5) Must Be Filtered in getDirEntries
|
|
||||||
When a FAT32 directory entry is deleted, `fileName[0]` is overwritten with `$E5` but the rest of the entry (including the cluster pointer) remains intact on disk. `getDirEntries` must explicitly skip entries where `fileName[0] = char($E5)` — stopping only at `char(0)` (end-of-directory) is not enough. If `$E5` entries are returned to callers, they appear in directory listings with a garbage first character, and navigating into them follows a stale/freed cluster chain, which causes reads into arbitrary memory or FAT data, leading to a triple fault. Fix location: the `while i < maxEntries` loop in `fat32.getDirEntries`.
|
|
||||||
-585
File diff suppressed because it is too large.
Load diff
@@ -1,58 +0,0 @@
|
|||||||
# AHCI Driver Notes
|
|
||||||
|
|
||||||
## Pre-scheduling / Process Kill Hazards
|
|
||||||
|
|
||||||
### 1. Dangling DMA Buffer (critical)
|
|
||||||
When a process is killed, its `kalloc`'d buffers get freed. But the AHCI hardware
|
|
||||||
is still DMA'ing into that physical memory. The DMA write lands on whatever now
|
|
||||||
occupies that memory, silently corrupting the kernel heap or another process.
|
|
||||||
|
|
||||||
**Fix:** Before freeing a process's memory, either:
|
|
||||||
- Wait for all its pending AHCI slots to drain (`cmd_issue` bits clear), or
|
|
||||||
- Issue a port reset to abort in-flight commands, *then* free memory.
|
|
||||||
|
|
||||||
### 2. Dangling Completion Callback / Userdata Pointer (critical)
|
|
||||||
In `storage_read`/`storage_write`, the completion callback writes to `@done` —
|
|
||||||
a **stack-local variable**. If the scheduler preempts or kills the process while
|
|
||||||
it's in the `hlt` loop, that stack frame is gone. The ISR fires and writes to a
|
|
||||||
dangling stack address, corrupting whatever now uses that stack page.
|
|
||||||
|
|
||||||
**Fix:** The `done` flag and completion context must be heap-allocated (or in a
|
|
||||||
kernel-owned structure that outlives the process). A per-request struct like:
|
|
||||||
```pascal
|
|
||||||
type TPendingIO = record
|
|
||||||
done : uint32;
|
|
||||||
ownerPID : uint32; { skip delivery if process is dead }
|
|
||||||
end;
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Command Slot Leak
|
|
||||||
If a process is killed with pending I/O, `pending[slot].inUse` stays `true`
|
|
||||||
forever. Eventually all 32 slots fill up and the device becomes unusable.
|
|
||||||
|
|
||||||
**Fix:** When killing a process, scan all AHCI devices for `pending[]` entries
|
|
||||||
belonging to that process, and either:
|
|
||||||
- Let the hardware finish but null out the completion callback (ISR just frees slot), or
|
|
||||||
- Abort the command (port reset) and clear the slot.
|
|
||||||
|
|
||||||
### 4. Interrupt-Disabled Preemption
|
|
||||||
The `asm sti` before the `hlt` loop is fine, but if the timer fires during setup
|
|
||||||
(between `readCallbackAsync(...)` and `asm sti`), the process could be switched
|
|
||||||
out with interrupts in an unexpected state. `hlt` is also not preemption-safe.
|
|
||||||
|
|
||||||
**Fix:** Move to a proper wait-queue/sleep mechanism instead of `hlt`.
|
|
||||||
|
|
||||||
## Recommended Path Forward
|
|
||||||
Implement a small **I/O request queue** owned by the kernel (not the process stack):
|
|
||||||
1. Allocate a request struct
|
|
||||||
2. Put it on a per-device queue
|
|
||||||
3. ISR completes it
|
|
||||||
4. Process kill walks the queue and cancels entries for that PID
|
|
||||||
|
|
||||||
## Other Bugs Fixed (2026-03-03)
|
|
||||||
- **vtop mask bug:** `$FFFFFF` (24-bit) → `$3FFFFF` (22-bit) for 4MB page offset
|
|
||||||
- **find_cmd_slot:** Now checks `pending[].inUse` in addition to `cmd_issue` bit
|
|
||||||
- **Missing lba5:** Both `send_read_dma_async` and `send_write_dma_async` now write all 48 LBA bits
|
|
||||||
- **IOAPIC routing:** PCI interrupts routed through IOAPIC so AHCI interrupts actually reach the CPU
|
|
||||||
- **LAPIC EOI:** `ISR_N` now sends LAPIC EOI for IOAPIC-delivered interrupts
|
|
||||||
- **PxIS pre-clear removed:** No longer blanket-clears port interrupt status before issuing commands (was eating other slots' completions)
|
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
// Copyright 2021 Kieron Morris
|
// Copyright 2021 Aaron Hance
|
||||||
//
|
//
|
||||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
// you may not use this file except in compliance with the License.
|
// you may not use this file except in compliance with the License.
|
||||||
|
|||||||
@@ -1,3 +1,16 @@
|
|||||||
|
// Copyright 2021 Aaron Hance
|
||||||
|
//
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
{
|
{
|
||||||
Driver->Storage->MBR Master boot record
|
Driver->Storage->MBR Master boot record
|
||||||
|
|
||||||
|
|||||||
@@ -1,3 +1,16 @@
|
|||||||
|
// Copyright 2021 Aaron Hance
|
||||||
|
//
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
{
|
{
|
||||||
Driver->Storage->FDTable - Per-process file descriptor table.
|
Driver->Storage->FDTable - Per-process file descriptor table.
|
||||||
|
|
||||||
|
|||||||
@@ -1,3 +1,16 @@
|
|||||||
|
// Copyright 2021 Aaron Hance
|
||||||
|
//
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
{
|
{
|
||||||
Driver->Storage->GPT - GUID Partition Table support.
|
Driver->Storage->GPT - GUID Partition Table support.
|
||||||
|
|
||||||
|
|||||||
@@ -1,3 +1,16 @@
|
|||||||
|
// Copyright 2021 Aaron Hance
|
||||||
|
//
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
{
|
{
|
||||||
Driver->Storage->IOBuffer - Safe IO buffer allocation helpers.
|
Driver->Storage->IOBuffer - Safe IO buffer allocation helpers.
|
||||||
|
|
||||||
|
|||||||
@@ -1,3 +1,16 @@
|
|||||||
|
// Copyright 2021 Aaron Hance
|
||||||
|
//
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
{
|
{
|
||||||
Driver->Storage->IORequest - I/O request allocation and lifecycle helpers.
|
Driver->Storage->IORequest - I/O request allocation and lifecycle helpers.
|
||||||
|
|
||||||
|
|||||||
@@ -1,12 +1,21 @@
|
|||||||
{ ************************************************
|
// Copyright 2021 Aaron Hance
|
||||||
* Asuro
|
//
|
||||||
* Unit: Drivers/storage/partitiontable
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
* Description: partionTable
|
// you may not use this file except in compliance with the License.
|
||||||
*
|
// You may obtain a copy of the License at
|
||||||
************************************************
|
//
|
||||||
* Author: Aaron Hance
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
* Contributors:
|
//
|
||||||
************************************************ }
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
{
|
||||||
|
Driver->Storage->PartitionTable - MBR partition table parsing and management.
|
||||||
|
|
||||||
|
@author(Aaron Hance <[email protected]>)
|
||||||
|
}
|
||||||
|
|
||||||
unit partitiontable;
|
unit partitiontable;
|
||||||
|
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
// Copyright 2021 Kieron Morris
|
// Copyright 2021 Aaron Hance
|
||||||
//
|
//
|
||||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
// you may not use this file except in compliance with the License.
|
// you may not use this file except in compliance with the License.
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
// Copyright 2021 Kieron Morris
|
// Copyright 2021 Aaron Hance
|
||||||
//
|
//
|
||||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
// you may not use this file except in compliance with the License.
|
// you may not use this file except in compliance with the License.
|
||||||
|
|||||||
@@ -1,3 +1,16 @@
|
|||||||
|
// Copyright 2021 Aaron Hance
|
||||||
|
//
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
{
|
{
|
||||||
Driver->Bus->USB->USB_Storage - USB Mass Storage Class Driver (stub).
|
Driver->Bus->USB->USB_Storage - USB Mass Storage Class Driver (stub).
|
||||||
|
|
||||||
|
|||||||
@@ -1,3 +1,16 @@
|
|||||||
|
// Copyright 2021 Aaron Hance
|
||||||
|
//
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
{
|
{
|
||||||
Driver->Storage->VolumeManager - Partition and volume management.
|
Driver->Storage->VolumeManager - Partition and volume management.
|
||||||
|
|
||||||
|
|||||||
Loaded 20 of 36 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user