diff --git a/.planning/phases/02-vault-core-and-rendering/02-RESEARCH.md b/.planning/phases/02-vault-core-and-rendering/02-RESEARCH.md new file mode 100644 index 0000000..6e3f83f --- /dev/null +++ b/.planning/phases/02-vault-core-and-rendering/02-RESEARCH.md @@ -0,0 +1,999 @@ +# Phase 2: Vault Core and Rendering - Research + +**Researched:** 2026-02-28 +**Domain:** Markdown parsing (pulldown-cmark), ratatui widget system, syntax highlighting (syntect), scrolling, CGA color palette, file I/O +**Confidence:** HIGH + + +## User Constraints (from CONTEXT.md) + +### Locked Decisions + +#### Color palette & text styling +- Classic CGA 16-color palette for accent elements (headers, code, links, emphasis markers) +- Body text uses terminal default foreground/background — respects user's TERM color scheme +- CGA accent colors only override for specific elements, not body text +- Heading levels (H1-H6) distinguished by both color AND decorators (e.g. H1 = bright cyan + ══ underline, H2 = bright yellow, H3 = green, etc.) +- Bold uses terminal bold attribute, italic uses terminal italic attribute, inline code uses terminal styling — modern terminals support these natively +- No need to fall back to color-only for bold/italic since SSH users have modern terminal emulators + +#### Screen layout & chrome +- No content borders — content goes edge-to-edge, maximizing reading space +- Single status bar at the bottom only (vim-style) +- Status bar content: current filename on the left, keyboard hints (q:Quit j/k:Scroll) on the right +- Status bar style: reverse video (inverted colors) — classic terminal status bar look +- No top bar, no title bar, no breadcrumb bar (breadcrumb comes in Phase 3) + +#### Code blocks & tables +- Fenced code blocks get syntax highlighting using CGA palette colors (cyan keywords, green strings, etc.) +- Code blocks framed with indent + colored background + rounded-corner box-drawing borders (╭─╮│╰─╯) +- GFM tables rendered with full box-drawing grid (┌┬┐├┼┤└┴┘) — classic DOS table look +- Table header row styled with bold + CGA accent color (e.g. bright cyan or yellow) + +#### Scrolling +- j/k and arrow keys scroll one line at a time +- PgUp/PgDn scroll a full page height +- No splash screen on startup — go straight to content + +### Claude's Discretion +- Exact CGA color assignments per heading level (H1-H6) +- Exact decorator style per heading level (═══ vs ─── vs none) +- Syntax highlighting color mapping (which CGA color for keywords, strings, comments, etc.) +- Blockquote visual style +- Horizontal rule style +- Image placeholder styling +- List bullet/number styling +- Scroll boundary behavior (stop or bounce) + +### Deferred Ideas (OUT OF SCOPE) +None — discussion stayed within phase scope + + + +## Phase Requirements + +| ID | Description | Research Support | +|----|-------------|-----------------| +| REND-01 | User sees H1-H6 headers with visual hierarchy and styling | pulldown-cmark Tag::Heading { level: HeadingLevel, .. } event; ratatui Line/Span with Color::LightCyan + Modifier::BOLD + decorator spans | +| REND-02 | User sees bold, italic, and inline code rendered with terminal styling | pulldown-cmark Tag::Strong, Tag::Emphasis, Event::Code; ratatui Modifier::BOLD, Modifier::ITALIC, Color::LightCyan for inline code | +| REND-03 | User sees fenced code blocks with syntax highlighting | pulldown-cmark CodeBlockKind::Fenced with language name; syntect 5.3 HighlightLines; CGA color mapping function | +| REND-04 | User sees ordered and unordered lists with proper indentation | pulldown-cmark Tag::List(Option), Tag::Item; u64=None is unordered, u64=Some(n) is ordered starting at n | +| REND-05 | User sees blockquotes with visual distinction | pulldown-cmark Tag::BlockQuote; render with left-margin │ character in CGA color + indented content | +| REND-06 | User sees horizontal rules as visual separators | pulldown-cmark Event::Rule; render as a full-width line of ─── characters in dim/DarkGray | +| REND-07 | User sees [IMAGE: alt text] placeholders for images | pulldown-cmark Tag::Image { .. }; collect alt text from nested Text events; render as Span "[IMAGE: {alt}]" | +| REND-08 | User sees GFM tables rendered with aligned columns | pulldown-cmark Tag::Table(Vec) with Options::ENABLE_TABLES; two-pass rendering: collect cells then emit box-drawing grid | +| REND-09 | User sees box-drawing borders on content panels | Code blocks use BorderType::Rounded (╭╮╰╯); tables use manual box-drawing chars; no borders on main content area per decision | +| REND-10 | User sees CGA-era retro color theme (cyan/magenta/green on dark) | ratatui Color::LightCyan, Color::Cyan, Color::LightGreen, Color::Green, Color::LightMagenta, Color::Yellow, Color::White | +| NAV-05 | User can scroll content with j/k, arrows, PgUp/PgDn | App state scroll_offset: u16; Paragraph::scroll((y, 0)); compute max_scroll from line_count(width) - viewport_height | +| NAV-06 | User lands on index.md as the entry point | On init: vault_path.join("index.md"); if not exists show BBS error screen; use std::fs::read_to_string | +| NAV-07 | User sees graceful error page when a linked file is not found | BBS-styled error widget with box-drawing border; message + vault path + create hint | +| NAV-08 | User sees keyboard hints in status bar | Reverse-video single-line Paragraph at bottom; Layout split [Min(0), Length(1)] | +| NAV-09 | App handles terminal resize without breaking layout | Event::Resize in crossterm event loop; ratatui autoresize in draw() handles buffer resize automatically for Viewport::Fullscreen | + + +--- + +## Summary + +Phase 2 is primarily a markdown parsing and rendering pipeline problem. The critical path runs: read file from disk → parse with pulldown-cmark → convert events to `Vec` (ratatui's styled text lines) → store in App state → render with `Paragraph::scroll()` → handle scroll keys. Every requirement maps cleanly to a specific pulldown-cmark event type and a ratatui rendering primitive. + +The ratatui 0.30 Widget trait has a verified, stable signature: `fn render(self, area: Rect, buf: &mut Buffer) where Self: Sized`. Custom widgets implement either consuming (`impl Widget for MyWidget`) or reference-based (`impl Widget for &MyWidget`) patterns. For Phase 2, the recommended approach is to build a `DocumentWidget` struct holding `Vec` and implementing `Widget` on a reference to it, since the document content is stable between renders and only scroll_offset changes. + +Syntax highlighting uses syntect 5.3.0 + syntect-tui 3.0. The `HighlightLines` API returns `Vec<(syntect::highlighting::Style, &str)>` per line; `syntect-tui::into_span()` converts each tuple to a `ratatui::text::Span`. Since the user specified CGA 16-color output, a custom RGB-to-CGA mapping function is needed: syntect themes produce RGB values and these must be quantized to the nearest of the 16 ratatui named colors. The ~200KB binary overhead from embedded syntax definitions is acceptable. + +GFM table rendering is the most complex single requirement: pulldown-cmark delivers table events in streaming order (TableHead → TableCell* → TableRow* → TableCell*), requiring a two-pass approach — first collect all cell content with column widths, then emit the full box-drawing grid as `Line` objects. This is purely Rust data transformation, no additional library needed. + +**Primary recommendation:** Build a `MarkdownRenderer` struct that converts a markdown string to `Vec>` (owned strings, no lifetime leakage). Store the rendered lines in `App` state. Use `Paragraph::new(lines).scroll((offset, 0))` for display. Add `syntect` and `syntect-tui` to Cargo.toml. + +--- + +## Standard Stack + +### Core +| Library | Version | Purpose | Why Standard | +|---------|---------|---------|--------------| +| pulldown-cmark | 0.13.1 | Markdown event-based parser | Already decided pre-Phase 1; CommonMark + GFM extensions; iterator-based, no allocation tree | +| ratatui | 0.30.0 | TUI rendering, Widget/Paragraph/Line/Span | Already a dependency; 0.30 stable Widget trait | +| syntect | 5.3.0 | Syntax highlighting for code blocks | De-facto Rust standard; embedded syntax definitions; HighlightLines API | +| syntect-tui | 3.0 | Convert syntect Style to ratatui Span | Lightweight bridge; avoids manual color conversion for most cases | + +### Supporting +| Library | Version | Purpose | When to Use | +|---------|---------|---------|-------------| +| std::fs | stdlib | Read markdown files from vault_path | Opening index.md and linked files | +| std::path::PathBuf | stdlib | Resolve file paths within vault | Join vault_path with relative file paths | + +### Alternatives Considered +| Instead of | Could Use | Tradeoff | +|------------|-----------|----------| +| syntect | Manual keyword regex highlighter | Manual approach avoids ~200KB binary overhead but requires per-language keyword lists; syntect has 80+ languages for free | +| syntect | tree-sitter | tree-sitter is more accurate but requires C compilation and is heavier; overkill for a terminal reader | +| pulldown-cmark | comrak | Pre-Phase 1 decision locked pulldown-cmark; comrak builds a DOM tree which uses more memory | +| tui-markdown crate | Custom renderer | tui-markdown is experimental PoC (0.3.7); cannot produce BBS-specific CGA aesthetics without heavy forking; building from pulldown-cmark events gives full control | + +**Installation:** +```toml +[dependencies] +ratatui = "0.30.0" # already present +pulldown-cmark = "0.13.1" +syntect = { version = "5.3.0", default-features = false, features = ["default-fancy"] } +syntect-tui = "3.0" +``` + +Note: `default-features = false, features = ["default-fancy"]` uses the pure-Rust fancy-regex engine instead of the Oniguruma C library. This avoids build complexity on non-standard systems (including some SSH server environments). The `default-fancy` feature still includes all embedded syntax definitions. + +--- + +## Architecture Patterns + +### Recommended Project Structure +``` +src/ +├── main.rs # unchanged from Phase 1 +├── config.rs # unchanged from Phase 1 +├── signals.rs # unchanged from Phase 1 +├── terminal.rs # unchanged from Phase 1 +├── app.rs # extended: add document state, scroll, key handling +├── vault.rs # NEW: file loading, index.md resolution, error states +├── renderer.rs # NEW: pulldown-cmark → Vec conversion pipeline +└── highlighter.rs # NEW: syntect integration, CGA color mapping +``` + +### Pattern 1: Markdown-to-Lines Conversion Pipeline + +**What:** Convert a markdown string to `Vec>` using pulldown-cmark events. This is the core of Phase 2. + +**When to use:** Called once when a document is loaded. The result is stored in App state and reused on every frame render. + +**Key insight:** Use `'static` lifetime for the output (owned `String` content in each `Span`) so there are no lifetime dependencies on the input markdown string. This simplifies App state dramatically. + +```rust +// Source: https://docs.rs/pulldown-cmark/latest/pulldown_cmark/ + +use pulldown_cmark::{Event, Options, Parser, Tag, TagEnd, HeadingLevel, CodeBlockKind, Alignment}; +use ratatui::text::{Line, Span}; +use ratatui::style::{Color, Modifier, Style}; + +pub struct MarkdownRenderer { + pub lines: Vec>, + // Internal state during parsing + current_spans: Vec>, + current_style: Style, + in_code_block: bool, + code_lang: String, + code_buf: String, + list_depth: u32, + list_counters: Vec>, // None=unordered, Some(n)=ordered at n + in_table: bool, + table_columns: Vec, + table_rows: Vec>, // raw cell text (two-pass for tables) + in_table_head: bool, +} +``` + +**Event loop skeleton:** +```rust +// Source: https://docs.rs/pulldown-cmark/latest/pulldown_cmark/ + +pub fn render_markdown(input: &str) -> Vec> { + let mut opts = Options::empty(); + opts.insert(Options::ENABLE_TABLES); + opts.insert(Options::ENABLE_STRIKETHROUGH); + opts.insert(Options::ENABLE_TASKLISTS); + opts.insert(Options::ENABLE_GFM); // blockquote alerts + + // TextMergeStream coalesces consecutive Text events + let parser = pulldown_cmark::TextMergeStream::new(Parser::new_ext(input, opts)); + + let mut state = RenderState::new(); + for event in parser { + state.handle(event); + } + state.finish() +} +``` + +### Pattern 2: pulldown-cmark Event Handling — Complete Event Map + +All events relevant to Phase 2 requirements: + +```rust +// Source: https://docs.rs/pulldown-cmark/latest/pulldown_cmark/enum.Event.html +// Source: https://docs.rs/pulldown-cmark/latest/pulldown_cmark/enum.Tag.html + +match event { + // ── HEADING ────────────────────────────────────────────────────────────── + Event::Start(Tag::Heading { level, .. }) => { + // level: HeadingLevel (H1..H6) + // Flush current line, push heading style onto style stack + // After End(TagEnd::Heading), add underline decorator line + } + Event::End(TagEnd::Heading(_)) => { + // Emit the heading Line, then emit a decorator Line (═══ or ───) + } + + // ── PARAGRAPH ──────────────────────────────────────────────────────────── + Event::Start(Tag::Paragraph) => { /* nothing — spans accumulate */ } + Event::End(TagEnd::Paragraph) => { + // Flush current_spans → push Line to lines; push empty Line for spacing + } + + // ── INLINE FORMATTING ──────────────────────────────────────────────────── + Event::Start(Tag::Strong) => { push_modifier(Modifier::BOLD); } + Event::End(TagEnd::Strong) => { pop_modifier(Modifier::BOLD); } + Event::Start(Tag::Emphasis) => { push_modifier(Modifier::ITALIC); } + Event::End(TagEnd::Emphasis) => { pop_modifier(Modifier::ITALIC); } + + // ── INLINE CODE ────────────────────────────────────────────────────────── + Event::Code(text) => { + // Render as Span with distinct style (e.g., Color::LightCyan + DIM background) + let span = Span::styled(text.to_string(), inline_code_style()); + current_spans.push(span); + } + + // ── TEXT ───────────────────────────────────────────────────────────────── + Event::Text(text) => { + // If in_code_block: accumulate to code_buf + // Otherwise: push Span with current_style + if in_code_block { + code_buf.push_str(&text); + } else { + current_spans.push(Span::styled(text.to_string(), current_style)); + } + } + + // ── CODE BLOCK ─────────────────────────────────────────────────────────── + Event::Start(Tag::CodeBlock(CodeBlockKind::Fenced(lang))) => { + code_lang = lang.to_string(); + in_code_block = true; + code_buf.clear(); + } + Event::Start(Tag::CodeBlock(CodeBlockKind::Indented)) => { + code_lang = String::new(); // no language, no highlighting + in_code_block = true; + code_buf.clear(); + } + Event::End(TagEnd::CodeBlock) => { + // Emit code block: rounded borders + highlighted lines + emit_code_block(&code_buf, &code_lang, &mut lines); + in_code_block = false; + } + + // ── LISTS ──────────────────────────────────────────────────────────────── + Event::Start(Tag::List(start_num)) => { + list_counters.push(start_num); // None = bullet, Some(n) = ordered from n + } + Event::End(TagEnd::List(_)) => { + list_counters.pop(); + // Push blank line after list + } + Event::Start(Tag::Item) => { + // Render bullet/number based on list_counters.last() + let prefix = match list_counters.last() { + Some(None) => " • ".to_string(), + Some(Some(n)) => format!(" {}. ", n), + None => " • ".to_string(), + }; + // Increment counter for ordered lists + } + + // ── BLOCKQUOTE ─────────────────────────────────────────────────────────── + Event::Start(Tag::BlockQuote(_kind)) => { + // Push quote context: prepend "│ " to subsequent lines in CGA color + } + Event::End(TagEnd::BlockQuote(_)) => { /* pop quote context */ } + + // ── HORIZONTAL RULE ────────────────────────────────────────────────────── + Event::Rule => { + let rule = Line::from(Span::styled( + "─".repeat(width), // width available at render time, not parse time + Style::default().fg(Color::DarkGray), + )); + lines.push(rule); + lines.push(Line::default()); + } + + // ── IMAGE ───────────────────────────────────────────────────────────────── + Event::Start(Tag::Image { .. }) => { /* begin collecting alt text */ } + Event::End(TagEnd::Image) => { + let placeholder = format!("[IMAGE: {}]", collected_alt_text); + current_spans.push(Span::styled(placeholder, image_style())); + } + + // ── TABLE (two-pass) ───────────────────────────────────────────────────── + Event::Start(Tag::Table(alignments)) => { + table_columns = alignments; + table_rows.clear(); + in_table = true; + } + Event::End(TagEnd::Table) => { + emit_table(&table_columns, &table_rows, &mut lines); + in_table = false; + } + Event::Start(Tag::TableHead) => { in_table_head = true; } + Event::End(TagEnd::TableHead) => { in_table_head = false; } + Event::Start(Tag::TableRow) => { table_rows.push(Vec::new()); } + Event::Start(Tag::TableCell) => { /* begin cell accumulation */ } + Event::End(TagEnd::TableCell) => { + // Push accumulated cell text to current row + table_rows.last_mut().unwrap().push(current_cell_text.clone()); + current_cell_text.clear(); + } + + // ── BREAKS ─────────────────────────────────────────────────────────────── + Event::SoftBreak => { current_spans.push(Span::raw(" ")); } + Event::HardBreak => { + flush_line(&mut lines, &mut current_spans); + } + + _ => {} // FootnoteReference, Html, InlineHtml, TaskListMarker handled as needed +} +``` + +**IMPORTANT:** `Tag::List` contains `Option` not `Option`. Ordered lists start at the `u64` value. `None` means unordered. This is the actual type as of pulldown-cmark 0.13.1. + +### Pattern 3: ratatui Widget Trait — Verified 0.30 Signature + +The Widget trait signature in ratatui 0.30.0 is: + +```rust +// Source: https://docs.rs/ratatui/0.30.0/ratatui/widgets/trait.Widget.html + +pub trait Widget { + fn render(self, area: Rect, buf: &mut Buffer) + where Self: Sized; +} +``` + +**Breaking change from 0.28/0.29:** WidgetRef no longer has a blanket impl of Widget. The correct pattern for a reusable widget stored in state is to implement Widget for a reference: + +```rust +// Source: https://ratatui.rs/recipes/widgets/custom/ + +pub struct DocumentWidget { + lines: Vec>, + scroll_offset: u16, +} + +impl Widget for &DocumentWidget { + fn render(self, area: Rect, buf: &mut Buffer) { + let paragraph = Paragraph::new(self.lines.clone()) + .scroll((self.scroll_offset, 0)); + paragraph.render(area, buf); + } +} +``` + +For widgets that need to modify their own state during render (e.g., update scroll limits after layout is known), implement `Widget for &mut MyWidget`. + +### Pattern 4: Scrolling with Paragraph + +```rust +// Source: https://docs.rs/ratatui/0.30.0/ratatui/widgets/struct.Paragraph.html + +// Scroll signature: +pub const fn scroll(self, offset: (u16, u16)) -> Paragraph<'a> +// (y, x) — y is lines, x is chars. NOTE: unusual (y, x) order, not (x, y) + +// Computing max scroll: +let content_height = paragraph.line_count(area.width) as u16; +let viewport_height = content_area.height; +let max_scroll = content_height.saturating_sub(viewport_height); + +// Clamping scroll: +scroll_offset = scroll_offset.min(max_scroll); + +// Page scroll: +let page_size = viewport_height; +scroll_offset = scroll_offset.saturating_add(page_size).min(max_scroll); // PgDn +scroll_offset = scroll_offset.saturating_sub(page_size); // PgUp +``` + +**Critical note on line_count:** `Paragraph::line_count(width)` computes total lines accounting for word wrapping. Since the document lines are pre-rendered as discrete `Line` objects (one per logical line), and the content area has no wrap by default in Paragraph, `line_count` will equal `lines.len()` unless you enable `Wrap`. Keep wrapping disabled for pre-rendered markdown lines. + +### Pattern 5: Status Bar Layout + +```rust +// Source: https://docs.rs/ratatui/0.30.0/ratatui/prelude/struct.Layout.html + +// Split the frame: content area above, status bar at bottom +let chunks = Layout::default() + .direction(Direction::Vertical) + .constraints([Constraint::Min(0), Constraint::Length(1)]) + .split(frame.area()); + +let content_area = chunks[0]; +let status_area = chunks[1]; + +// Status bar: reverse video, filename left, hints right +let status = Paragraph::new(Line::from(vec![ + Span::styled( + format!(" {} ", filename), + Style::default().add_modifier(Modifier::REVERSED), + ), + Span::styled( + " q:Quit j/k:Scroll PgUp/PgDn:Page ", + Style::default().add_modifier(Modifier::REVERSED), + ), +])) +.style(Style::default().add_modifier(Modifier::REVERSED)); + +frame.render_widget(status, status_area); +``` + +### Pattern 6: Code Block Rendering with Rounded Borders + +Code blocks use ratatui's `Block` with `BorderType::Rounded` (produces ╭╮╰╯): + +```rust +// Source: https://docs.rs/ratatui/0.30.0/ratatui/widgets/struct.Block.html + +use ratatui::widgets::{Block, BorderType, Borders}; + +// Emit a Block frame around the highlighted code +let code_block_border = Block::default() + .borders(Borders::ALL) + .border_type(BorderType::Rounded) // ╭─╮ / ╰─╯ corners + .border_style(Style::default().fg(Color::DarkGray)) + .title(Span::styled(lang, Style::default().fg(Color::Yellow))); + +// The inner area of the block holds the Paragraph with highlighted lines +``` + +When building `Vec`, code blocks are NOT rendered inside a live `Block` widget (that would require nested rendering). Instead, emit the border characters as raw `Span` objects forming the box-drawing frame: + +``` +╭─ rust ──────────────────╮ +│ let x = 1; │ +│ println!("{}", x); │ +╰─────────────────────────╯ +``` + +This is simpler: each line of the code block becomes a `Line` with a leading `│` span and a trailing `│` span. The top/bottom borders are separate `Line` objects. This keeps everything in the flat `Vec` model and avoids nested widget rendering complexity. + +### Pattern 7: Syntax Highlighting with syntect + syntect-tui + +```rust +// Source: https://docs.rs/syntect/latest/syntect/easy/struct.HighlightLines.html +// Source: syntect-tui 3.0 (into_span function) + +use syntect::easy::HighlightLines; +use syntect::highlighting::ThemeSet; +use syntect::parsing::SyntaxSet; + +// Load once at program start (not per-document): +lazy_static! { + static ref SYNTAX_SET: SyntaxSet = SyntaxSet::load_defaults_newlines(); + static ref THEME_SET: ThemeSet = ThemeSet::load_defaults(); +} + +pub fn highlight_code(code: &str, lang: &str) -> Vec> { + let syntax = SYNTAX_SET.find_syntax_by_token(lang) + .or_else(|| SYNTAX_SET.find_syntax_by_name(lang)) + .unwrap_or_else(|| SYNTAX_SET.find_syntax_plain_text()); + + // Use base16-ocean.dark as starting theme; map RGB to CGA afterwards + let mut highlighter = HighlightLines::new( + syntax, + &THEME_SET.themes["base16-ocean.dark"], + ); + + let mut result = Vec::new(); + for line in code.lines() { + // highlight_line signature: + // pub fn highlight_line<'b>(&mut self, line: &'b str, syntax_set: &SyntaxSet) + // -> Result, Error> + let ranges = highlighter.highlight_line(line, &SYNTAX_SET).unwrap_or_default(); + let spans: Vec> = ranges.into_iter() + .map(|(style, text)| { + let fg = syntect_color_to_cga(style.foreground); + Span::styled( + text.to_string(), // 'static via owned String + Style::default().fg(fg), + ) + }) + .collect(); + result.push(Line::from(spans)); + } + result +} +``` + +**Note:** `syntect_tui::into_span()` can be used instead of manual conversion, but it passes through RGB colors as `Color::Rgb(r, g, b)`. For strict CGA 16-color output, the manual mapping function below is needed. + +### Pattern 8: RGB to CGA 16-Color Mapping + +Syntect themes use RGB colors. Map to nearest CGA 16 color for terminal compatibility: + +```rust +// Source: ratatui Color variants documented at +// https://docs.rs/ratatui/0.30.0/ratatui/style/enum.Color.html + +use syntect::highlighting::Color as SyntectColor; +use ratatui::style::Color; + +/// Map a syntect RGB color to the nearest CGA 16-color ratatui Color. +/// Uses Euclidean distance in RGB space. +pub fn syntect_color_to_cga(c: SyntectColor) -> Color { + // CGA 16 palette: (r, g, b, Color variant) + const CGA: &[(u8, u8, u8, Color)] = &[ + (0, 0, 0, Color::Black), + (170, 0, 0, Color::Red), + (0, 170, 0, Color::Green), + (170, 170, 0, Color::Yellow), + (0, 0, 170, Color::Blue), + (170, 0, 170, Color::Magenta), + (0, 170, 170, Color::Cyan), + (170, 170, 170, Color::Gray), + (85, 85, 85, Color::DarkGray), + (255, 85, 85, Color::LightRed), + (85, 255, 85, Color::LightGreen), + (255, 255, 85, Color::LightYellow), // also Color::Yellow bright variant + (85, 85, 255, Color::LightBlue), + (255, 85, 255, Color::LightMagenta), + (85, 255, 255, Color::LightCyan), + (255, 255, 255, Color::White), + ]; + + let mut best = Color::White; + let mut best_dist = u32::MAX; + for &(r, g, b, color) in CGA { + let dr = (c.r as i32 - r as i32).pow(2) as u32; + let dg = (c.g as i32 - g as i32).pow(2) as u32; + let db = (c.b as i32 - b as i32).pow(2) as u32; + let dist = dr + dg + db; + if dist < best_dist { + best_dist = dist; + best = color; + } + } + best +} +``` + +**Alternative approach:** Assign CGA colors by syntect scope name rather than RGB proximity. Scope names like `keyword`, `string.quoted`, `comment` can be mapped directly to desired CGA colors without going through RGB at all. This is more predictable for the BBS aesthetic but requires scope inspection from syntect's `ScopeRegionIterator`. For Phase 2, RGB proximity is simpler and sufficient. + +### Pattern 9: Terminal Resize Handling + +```rust +// Source: https://ratatui.rs/faq/ +// Source: https://docs.rs/ratatui/0.30.0/ratatui/struct.Terminal.html + +// In the event loop (extending Phase 1 pattern): +match event::read()? { + Event::Key(key) => { self.handle_key(key); } + Event::Resize(_w, _h) => { + // For Viewport::Fullscreen: Terminal::draw() calls autoresize() internally. + // Buffer is resized automatically on next draw call. + // No explicit action needed — just let the next draw() handle it. + // HOWEVER: recompute max_scroll after resize since content_area.height changes. + // This happens naturally because max_scroll is computed inside draw(). + } + _ => {} +} +``` + +**Key finding:** `Viewport::Fullscreen` + `Terminal::draw()` automatically resizes the internal buffer when terminal size changes. No explicit `terminal.autoresize()` call is needed. The draw closure receives a `Frame` whose `frame.area()` reflects the current terminal size. Scroll offset recomputation happens inside the draw closure naturally. + +### Pattern 10: GFM Table Two-Pass Rendering + +Tables require knowing all column widths before emitting any lines: + +```rust +// Phase 1: Collect all cells during event parsing +// Phase 2: Emit box-drawing grid + +pub fn emit_table( + alignments: &[Alignment], + rows: &[Vec], // rows[0] = header row + lines: &mut Vec>, +) { + let n_cols = alignments.len(); + + // Compute column widths + let mut col_widths: Vec = vec![3; n_cols]; + for row in rows { + for (i, cell) in row.iter().enumerate() { + if i < n_cols { + col_widths[i] = col_widths[i].max(cell.len() + 2); // padding + } + } + } + + // Top border: ┌──────┬──────┐ + let top = build_table_row('┌', '┬', '┐', '─', &col_widths); + lines.push(Line::from(Span::styled(top, Style::default().fg(Color::Cyan)))); + + // Header row with bold CGA styling + if let Some(header) = rows.first() { + let row_line = build_table_data_row(header, &col_widths, alignments, + Style::default().fg(Color::LightCyan).add_modifier(Modifier::BOLD)); + lines.push(row_line); + // Header separator: ├──────┼──────┤ + let sep = build_table_row('├', '┼', '┤', '─', &col_widths); + lines.push(Line::from(Span::styled(sep, Style::default().fg(Color::Cyan)))); + } + + // Data rows + for row in rows.iter().skip(1) { + let row_line = build_table_data_row(row, &col_widths, alignments, + Style::default()); + lines.push(row_line); + } + + // Bottom border: └──────┴──────┘ + let bottom = build_table_row('└', '┴', '┘', '─', &col_widths); + lines.push(Line::from(Span::styled(bottom, Style::default().fg(Color::Cyan)))); + lines.push(Line::default()); // spacing after table +} +``` + +### Pattern 11: File Loading and Error State + +```rust +// Source: stdlib std::fs + +pub enum VaultDocument { + Loaded { path: PathBuf, content: String }, + Missing { path: PathBuf }, + ReadError { path: PathBuf, reason: String }, +} + +pub fn load_document(vault_path: &Path, relative: &str) -> VaultDocument { + let full_path = vault_path.join(relative); + match std::fs::read_to_string(&full_path) { + Ok(content) => VaultDocument::Loaded { path: full_path, content }, + Err(e) if e.kind() == std::io::ErrorKind::NotFound => { + VaultDocument::Missing { path: full_path } + } + Err(e) => VaultDocument::ReadError { + path: full_path, + reason: e.to_string(), + }, + } +} +``` + +The missing index.md error screen is a custom `ErrorScreenWidget` that renders a BBS-styled box with box-drawing borders: + +``` +┌─────────────────────────────────────────┐ +│ *** SYSTEM ERROR *** │ +│ │ +│ No index.md found in vault: │ +│ /path/to/vault │ +│ │ +│ Create index.md to begin. │ +└─────────────────────────────────────────┘ +``` + +### Anti-Patterns to Avoid + +- **Lifetime-borrowing `Line<'a>` from input markdown string:** The markdown string goes out of scope or is reloaded. Always convert to `Line<'static>` (owned `String` in each `Span`) during rendering. This costs a small allocation but avoids pervasive lifetime annotations through `App`. + +- **Calling `SyntaxSet::load_defaults_newlines()` per document:** Loading the embedded syntax set takes ~23ms. Call it once at startup. Use `lazy_static!` or pass a `&SyntaxSet` reference. + +- **Forgetting `Options::ENABLE_TABLES`:** Without this flag, `Parser::new_ext()` will not emit `Tag::Table` events. Table markdown is silently treated as plain text with pipe characters. Always set GFM extension flags explicitly. + +- **Not using `TextMergeStream`:** pulldown-cmark may split a single logical text run across multiple `Event::Text` events when inline formatting markers exist nearby (e.g., text before an emphasis marker). `TextMergeStream::new(parser)` coalesces adjacent `Text` events before your handler sees them. + +- **Rendering live `Block` widgets inside the lines Vec:** `Block` is a ratatui widget that requires a `Rect` and `Buffer` to render. You cannot embed it inside `Vec`. All box-drawing for code blocks and tables must be emitted as raw character spans (│, ╭, ─, etc.) in the Line sequence. + +- **Scroll offset as `i32` or signed:** Use `u16` throughout. `saturating_sub` handles underflow. `saturating_add` + `min(max)` handles overflow. Paragraph::scroll() takes `(u16, u16)` — passing `usize` requires explicit cast. + +- **Using `Paragraph::scroll()` with `Wrap` enabled:** When Paragraph wraps lines, `line_count()` reflects wrap expansion, but the input `Line` objects do not change. The scroll offset applies to rendered lines, not input lines. For pre-rendered markdown (one `Line` per logical line), disable `Wrap` to keep scroll math simple and predictable. + +--- + +## Don't Hand-Roll + +| Problem | Don't Build | Use Instead | Why | +|---------|-------------|-------------|-----| +| Markdown parsing | Custom parser | pulldown-cmark 0.13.1 | CommonMark spec compliance, edge cases (nested emphasis, escaped chars, HTML entities) | +| Syntax tokenization | Keyword regex lists | syntect 5.3.0 | 80+ languages, Sublime text definitions, maintained separately | +| Color conversion (syntect→ratatui) | Manual RGB conversion | syntect-tui 3.0 `into_span()` | Handles style flag mapping (bold, italic from theme); only override for CGA quantization | +| Table column widths | Screen-width allocation | Two-pass cell collection + max() | Tables need all cell content before first border char can be emitted | + +**Key insight:** The pulldown-cmark event stream is a correct CommonMark parser; building on top of it is always correct. The only custom logic needed is converting events to ratatui `Line` objects — this is inherently application-specific and there is no general library that produces BBS-CGA-styled output. + +--- + +## Common Pitfalls + +### Pitfall 1: Widget Trait Signature Changed in 0.29/0.30 + +**What goes wrong:** Code written for ratatui <= 0.28 using `WidgetRef` directly fails to compile with 0.30. Code that `impl WidgetRef for Foo` instead of `impl Widget for &Foo` no longer works. + +**Why it happens:** In 0.30, `WidgetRef` no longer has a blanket impl of `Widget`. The relationship inverted: `impl Widget for &W` is now the canonical pattern for immutable reference-based widgets. + +**How to avoid:** Use `impl Widget for &MyWidget` (consuming a reference) for stored, reusable widgets. Use `impl Widget for MyWidget` (consuming ownership) for one-shot widgets created in the draw closure. Do NOT implement `WidgetRef` directly. + +**Warning signs:** Compiler error "the trait `Widget` is not implemented for `MyWidget`" when passing a stored widget to `frame.render_widget()`. + +### Pitfall 2: pulldown-cmark Table Events Not Firing + +**What goes wrong:** Markdown tables render as plain text with pipe characters instead of structured tables. + +**Why it happens:** `Options::ENABLE_TABLES` is not set. Tables are a GFM extension, not CommonMark. They require explicit opt-in. + +**How to avoid:** Use `Parser::new_ext(input, opts)` with `opts.insert(Options::ENABLE_TABLES)`. Verify with a test document containing a table. + +**Warning signs:** `| col1 | col2 |` appears verbatim in output instead of a box-drawing grid. + +### Pitfall 3: Consecutive Text Events Splitting Styled Runs + +**What goes wrong:** Text like `Hello *world* there` produces four separate `Event::Text` events — "Hello ", then inside the emphasis a "world" text, then "world" again, then " there" — causing broken styled spans. + +**Why it happens:** pulldown-cmark's internal state machine may emit multiple adjacent `Text` events at inline formatting boundaries. + +**How to avoid:** Wrap the parser in `TextMergeStream::new(parser)` before iterating events. This coalesces adjacent `Text` events into single events. + +**Warning signs:** Text appears duplicated or styling bleeds into adjacent text. + +### Pitfall 4: Syntect Load on Every Document + +**What goes wrong:** Loading `SyntaxSet::load_defaults_newlines()` takes ~23ms. If called on each document load, it adds perceptible lag on navigation. + +**Why it happens:** The embedded syntax definitions are a ~200KB compressed binary blob that requires decompression and linking on each call. + +**How to avoid:** Initialize `SyntaxSet` and `ThemeSet` once at program startup. Store in a module-level `OnceLock` or pass as shared references. Do not call `load_defaults_*` inside document rendering loops. + +**Warning signs:** Noticeable lag when switching documents (Phase 3 concern, but architecture must support it from Phase 2). + +### Pitfall 5: Scroll Math with Paragraph::scroll() is (y, x) Not (x, y) + +**What goes wrong:** Passing `(horizontal, vertical)` produces sideways scrolling instead of vertical scrolling. + +**Why it happens:** The `Paragraph::scroll()` method takes `(y, x)` — vertical first, horizontal second. This is the opposite of typical (x, y) coordinate convention. The ratatui documentation explicitly flags this as a deliberate deviation from crate convention. + +**How to avoid:** Always write `paragraph.scroll((self.scroll_offset, 0))` where the first element is vertical scroll and the second is always `0` (no horizontal scroll for a markdown reader). + +**Warning signs:** Content scrolls horizontally but not vertically when pressing j/k. + +### Pitfall 6: Line<'static> vs Line<'a> Lifetime Confusion + +**What goes wrong:** Storing `Vec>` in `App` where `'a` is tied to the markdown string lifetime causes the borrow checker to reject the code because `App` must outlive the borrowed string. + +**Why it happens:** If `Span` content is borrowed `&str` from the markdown string, the `Line` borrows from that string. When the string is dropped or replaced, all the `Line` objects become invalid. + +**How to avoid:** Always use `.to_string()` or `.into_owned()` when creating `Span` content during rendering: `Span::styled(text.to_string(), style)`. This makes every `Span` own its content (`'static`). `Vec>` can be stored in `App` without lifetime parameters. + +**Warning signs:** Compiler error "cannot infer an appropriate lifetime for borrow expression" in the renderer function. + +### Pitfall 7: Missing index.md Should NOT Panic + +**What goes wrong:** `std::fs::read_to_string(vault_path.join("index.md")).unwrap()` panics if index.md is missing, leaving the terminal in raw mode. + +**Why it happens:** Missing files are an expected user-facing state (new vault), not a bug. `unwrap()` on `Result` converts expected errors into panics. + +**How to avoid:** Use `match` or `.map_err()` to convert `io::Error` to a `VaultDocument::Missing` state. The App then renders the BBS error screen widget instead of a document widget. Never unwrap file I/O operations. + +**Warning signs:** Terminal stuck in raw mode after starting with an empty vault directory. + +--- + +## Code Examples + +Verified patterns from official sources: + +### CGA Color Assignments for Headings (Claude's Discretion) + +Based on CGA palette and BBS aesthetic — assigned by researcher: + +| Level | Color | Decorator | +|-------|-------|-----------| +| H1 | `Color::LightCyan` + `Modifier::BOLD` | `══════` (double-line, full width) | +| H2 | `Color::LightYellow` + `Modifier::BOLD` | `──────` (single-line, full width) | +| H3 | `Color::LightGreen` + `Modifier::BOLD` | none (color only) | +| H4 | `Color::Green` | none | +| H5 | `Color::Cyan` | none | +| H6 | `Color::DarkGray` | none | + +### CGA Colors for Syntax Highlighting (Claude's Discretion) + +For syntax scopes mapped to CGA colors: + +| Scope Category | CGA Color | +|----------------|-----------| +| Keywords (`fn`, `let`, `if`, etc.) | `Color::LightCyan` | +| Strings | `Color::LightGreen` | +| Comments | `Color::DarkGray` | +| Numbers | `Color::LightYellow` | +| Types/classes | `Color::Yellow` | +| Functions/methods | `Color::LightMagenta` | +| Operators | `Color::Gray` | +| Default/text | `Color::Gray` | + +### Blockquote Style (Claude's Discretion) + +``` +│ This is a blockquote line +│ with a second line. +``` + +- Left border `│` in `Color::Yellow` +- Content in `Color::Gray` (slightly dimmed from default) +- One blank line before and after + +### Horizontal Rule Style (Claude's Discretion) + +``` +──────────────────────────────────────── +``` + +Full-width `─` characters in `Color::DarkGray`. One blank line before and after. + +### Image Placeholder Style (Claude's Discretion) + +``` +[IMAGE: alt text here] +``` + +`Color::DarkGray` + `Modifier::DIM`. Surrounded by blank lines if block-level. + +### List Bullet Style (Claude's Discretion) + +- Unordered: ` • ` in `Color::Cyan`, content in default color +- Ordered: ` 1. ` in `Color::Cyan`, content in default color +- Nested (each level adds 2 spaces): ` ◦ ` for depth 2, ` ▪ ` for depth 3 + +### Scroll Boundary Behavior (Claude's Discretion) + +**Stop at boundaries** (not bounce): `scroll_offset = scroll_offset.saturating_sub(1)` for up, `scroll_offset = (scroll_offset + 1).min(max_scroll)` for down. Scroll stops silently at top/bottom — no visual indicator needed in Phase 2. + +### Complete pulldown-cmark Parser Setup + +```rust +// Source: https://docs.rs/pulldown-cmark/latest/pulldown_cmark/ + +use pulldown_cmark::{Options, Parser, TextMergeStream}; + +pub fn make_parser(input: &str) -> impl Iterator> { + let mut opts = Options::empty(); + opts.insert(Options::ENABLE_TABLES); + opts.insert(Options::ENABLE_STRIKETHROUGH); + opts.insert(Options::ENABLE_TASKLISTS); + opts.insert(Options::ENABLE_GFM); // blockquote alert syntax + opts.insert(Options::ENABLE_HEADING_ATTRIBUTES); // {#id .class} on headings + TextMergeStream::new(Parser::new_ext(input, opts)) +} +``` + +### syntect Initialization (Once at Startup) + +```rust +// Source: https://docs.rs/syntect/latest/syntect/ + +use std::sync::OnceLock; +use syntect::highlighting::ThemeSet; +use syntect::parsing::SyntaxSet; + +static SYNTAX_SET: OnceLock = OnceLock::new(); +static THEME_SET: OnceLock = OnceLock::new(); + +pub fn init_highlighter() { + SYNTAX_SET.get_or_init(SyntaxSet::load_defaults_newlines); + THEME_SET.get_or_init(ThemeSet::load_defaults); +} + +pub fn syntax_set() -> &'static SyntaxSet { + SYNTAX_SET.get().expect("init_highlighter() not called") +} + +pub fn theme_set() -> &'static ThemeSet { + THEME_SET.get().expect("init_highlighter() not called") +} +``` + +### App State Extension for Phase 2 + +```rust +// Extending Phase 1 App struct +pub struct App { + // Phase 1 fields (unchanged) + is_login_shell: bool, + ctrl_c_pressed_at: Option, + show_quit_prompt: bool, + should_quit: bool, + config: Config, + + // Phase 2 additions + document: DocumentState, + scroll_offset: u16, +} + +pub enum DocumentState { + Rendering, // initial state during parse + Loaded { + filename: String, + lines: Vec>, + }, + Missing { + path: PathBuf, + }, + Error { + path: PathBuf, + reason: String, + }, +} +``` + +--- + +## State of the Art + +| Old Approach | Current Approach | When Changed | Impact | +|--------------|------------------|--------------|--------| +| `impl WidgetRef for Foo` | `impl Widget for &Foo` | ratatui 0.29/0.30 (2024-2025) | Previous WidgetRef impl pattern no longer compiles; must update if migrating from older ratatui | +| `Parser::new()` only | `Parser::new_ext(input, opts)` with Options flags | pulldown-cmark 0.9+ | Extensions like tables require explicit opt-in via Options struct | +| syntect with Oniguruma C lib | `default-features = false, features = ["default-fancy"]` | syntect 4.0 (2022) | Avoids C compilation; pure-Rust regex engine; simpler build on Linux servers | +| Manual scroll state management | `Paragraph::scroll((y, 0))` | ratatui stable | Paragraph handles the offset; app only needs to track the u16 offset value | + +**Deprecated/outdated:** +- `tui-rs`: The predecessor crate to ratatui. All docs using `tui::widgets::*` are from this era. Not applicable. +- `syntect::highlighting::HighlightIterator`: The lower-level iterator API. `easy::HighlightLines` is the preferred high-level wrapper. +- `pulldown_cmark::html::push_html`: The HTML output function. Phase 2 never produces HTML; ignore this API. + +--- + +## Open Questions + +1. **syntect-tui 3.0 ratatui 0.30 compatibility** + - What we know: syntect-tui 3.0 targets ratatui. The crate list shows syntect-tui 3.0 most recently updated. + - What's unclear: Whether syntect-tui 3.0 has been tested specifically against ratatui 0.30.0. The Widget trait change in 0.30 only affects widget implementations, not `Span`/`Style` types which syntect-tui converts. The risk is LOW. + - Recommendation: Add `syntect-tui = "3.0"` and `syntect = "5.3"` to Cargo.toml. If compilation fails due to ratatui version mismatch, use manual syntect-style-to-ratatui-style conversion (the Pattern 8 RGB-to-CGA function handles the whole pipeline anyway). + +2. **Horizontal rule width at parse time** + - What we know: `Event::Rule` fires during markdown parsing before we know the terminal width. Full-width `─────` requires knowing `area.width` which is only available inside `frame.render_widget()`. + - What's unclear: Best approach to defer width-dependent rendering. + - Recommendation: Emit a sentinel `Line` with a special marker during parsing (e.g., `Line::from(Span::raw("--- RULE ---"))`). In the widget's `render()` method, detect this sentinel and replace it with `"─".repeat(area.width)`. Alternatively, emit a fixed 80-char rule and accept that it won't be exact width. + +3. **Table rendering width constraint** + - What we know: Tables with many wide columns may exceed terminal width. The two-pass approach computes column widths from content, not from available terminal width. + - What's unclear: Whether to truncate, wrap, or overflow table cells. + - Recommendation: For Phase 2, emit tables at natural content width and let the terminal clip. Horizontal scrolling is not required (NAV-05 only specifies vertical scroll). Address overflow in Phase 3 if reported. + +4. **OnceLock for syntect vs. passing as parameter** + - What we know: `OnceLock` is in stable `std` since Rust 1.70. The project uses edition 2024. + - What's unclear: Whether to initialize `SyntaxSet`/`ThemeSet` in `main.rs` and pass down, or use module-level `OnceLock`. + - Recommendation: Initialize in `main.rs` before `App::new()`, store `SyntaxSet` in `App`. Avoids global state and is more testable. `SyntaxSet` is `Send + Sync` so storing in App is fine. + +--- + +## Sources + +### Primary (HIGH confidence) +- `https://docs.rs/pulldown-cmark/latest/pulldown_cmark/enum.Event.html` — 13 Event variants documented, including all Phase 2 relevant types +- `https://docs.rs/pulldown-cmark/latest/pulldown_cmark/enum.Tag.html` — Complete Tag enum: Heading{level, id, classes, attrs}, List(Option), CodeBlock(CodeBlockKind), Table(Vec), TableHead, TableRow, TableCell +- `https://docs.rs/pulldown-cmark/latest/pulldown_cmark/struct.Options.html` — 15 Options flags documented, including ENABLE_TABLES, ENABLE_STRIKETHROUGH, ENABLE_TASKLISTS, ENABLE_GFM +- `https://docs.rs/ratatui/0.30.0/ratatui/widgets/trait.Widget.html` — Verified Widget trait: `fn render(self, area: Rect, buf: &mut Buffer) where Self: Sized` +- `https://docs.rs/ratatui/0.30.0/ratatui/widgets/struct.Paragraph.html` — scroll((y, x): (u16, u16)), line_count(width: u16) -> usize +- `https://docs.rs/ratatui/0.30.0/ratatui/style/struct.Style.html` — fg(), bg(), add_modifier(), Modifier constants: BOLD, ITALIC, DIM, UNDERLINED, REVERSED +- `https://docs.rs/ratatui/0.30.0/ratatui/style/enum.Color.html` — Named colors: Black, Red, Green, Yellow, Blue, Magenta, Cyan, Gray, DarkGray, LightRed, LightGreen, LightYellow, LightBlue, LightMagenta, LightCyan, White, Reset +- `https://docs.rs/ratatui/0.30.0/ratatui/widgets/struct.Block.html` — BorderType::Rounded produces ╭╮╰╯ corners +- `https://ratatui.rs/highlights/v030/` — Breaking change: WidgetRef no longer blanket-implements Widget; use `impl Widget for &Foo` pattern +- `https://docs.rs/syntect/latest/syntect/easy/struct.HighlightLines.html` — highlight_line<'b>(&mut self, line: &'b str, &SyntaxSet) -> Result, Error> +- `https://docs.rs/syntect/latest/syntect/` — Version 5.3.0 confirmed; embedded syntax set ~200KB; load time ~23ms +- `https://ratatui.rs/faq/` — Resize: Event::Resize captured via crossterm; Viewport::Fullscreen autoresizes buffer in draw() + +### Secondary (MEDIUM confidence) +- `https://github.com/chanq-io/syntect-tui` — syntect-tui 3.0: `into_span()` converts `(syntect::Style, &str)` to `ratatui::Span`; MIT license +- `https://ratatui.rs/recipes/widgets/custom/` — Widget impl patterns for 0.30: consuming vs reference-based +- `https://github.com/joshka/tui-markdown` — Experimental PoC; not suitable for BBS aesthetic without heavy forking; reference only + +### Tertiary (LOW confidence, flag for validation) +- syntect-tui 3.0 compatibility with ratatui 0.30.0 — not explicitly verified against 0.30; Style/Span API compatibility inferred from no known breaking changes in those types between 0.28 and 0.30 +- CGA color assignments for syntax scopes (keywords, strings, etc.) — researcher recommendation based on classic BBS aesthetics, not from an authoritative source; adjust during implementation + +--- + +## Metadata + +**Confidence breakdown:** +- Standard stack: HIGH — all versions verified via docs.rs; pulldown-cmark 0.13.1, ratatui 0.30.0, syntect 5.3.0 confirmed +- Architecture: HIGH — Widget trait signature verified from official ratatui 0.30 docs; pulldown-cmark Event and Tag enums verified; Paragraph::scroll() API verified +- Pitfalls: HIGH for ratatui 0.30 Widget breaking change (confirmed from release notes); HIGH for pulldown-cmark Options flags; MEDIUM for syntect-tui compatibility +- CGA color assignments: LOW — researcher recommendation, no authoritative source; safe to adjust during implementation + +**Research date:** 2026-02-28 +**Valid until:** 2026-03-30 (stable libraries; pulldown-cmark and ratatui move slowly)