docs(quick-3): plan remote markdown page linking with domain whitelist
This commit is contained in:
@@ -0,0 +1,278 @@
|
||||
---
|
||||
phase: quick-3
|
||||
plan: 01
|
||||
type: execute
|
||||
wave: 1
|
||||
depends_on: []
|
||||
files_modified:
|
||||
- Cargo.toml
|
||||
- src/config.rs
|
||||
- src/vault.rs
|
||||
- src/app.rs
|
||||
- src/renderer.rs
|
||||
autonomous: true
|
||||
requirements: [REMOTE-01, REMOTE-02, REMOTE-03, REMOTE-04]
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "User can follow an HTTP/HTTPS link to a whitelisted domain and see the remote markdown rendered in the TUI"
|
||||
- "User sees a BBS-themed error screen when following an HTTP link to a non-whitelisted domain"
|
||||
- "User sees a BBS-themed error screen when the remote URL fails to fetch or returns non-markdown content"
|
||||
- "Config file accepts an allowed_remote_domains list in bbs.toml"
|
||||
- "Remote pages participate in browser-style history (back/forward navigation works)"
|
||||
artifacts:
|
||||
- path: "Cargo.toml"
|
||||
provides: "ureq HTTP client dependency"
|
||||
contains: "ureq"
|
||||
- path: "src/config.rs"
|
||||
provides: "allowed_remote_domains config field"
|
||||
contains: "allowed_remote_domains"
|
||||
- path: "src/vault.rs"
|
||||
provides: "fetch_remote_markdown function"
|
||||
contains: "fn fetch_remote_markdown"
|
||||
- path: "src/app.rs"
|
||||
provides: "HTTP link detection and remote page navigation in follow_selected_link"
|
||||
contains: "fetch_remote_markdown"
|
||||
key_links:
|
||||
- from: "src/app.rs"
|
||||
to: "src/vault.rs"
|
||||
via: "fetch_remote_markdown call in follow_selected_link"
|
||||
pattern: "vault::fetch_remote_markdown"
|
||||
- from: "src/app.rs"
|
||||
to: "src/config.rs"
|
||||
via: "allowed_remote_domains whitelist check"
|
||||
pattern: "allowed_remote_domains"
|
||||
- from: "src/renderer.rs"
|
||||
to: "src/app.rs"
|
||||
via: "HTTP links styled differently (LightMagenta) to indicate remote"
|
||||
pattern: "starts_with.*http"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Add remote markdown page linking: when a user follows an HTTP/HTTPS link in a markdown document,
|
||||
the app checks if the domain is whitelisted in bbs.toml, fetches the URL, validates it returns
|
||||
markdown content, and renders it in the TUI like a local page.
|
||||
|
||||
Purpose: Enable the BBS to link to external markdown content (documentation sites, wikis, READMEs)
|
||||
while maintaining security through domain whitelisting.
|
||||
|
||||
Output: Working remote link following with config, fetching, validation, rendering, error states,
|
||||
and history integration.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@/Users/ruohki/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@/Users/ruohki/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/STATE.md
|
||||
@Cargo.toml
|
||||
@src/config.rs
|
||||
@src/vault.rs
|
||||
@src/app.rs
|
||||
@src/renderer.rs
|
||||
@src/main.rs
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Add config field, ureq dependency, and remote fetch function</name>
|
||||
<files>
|
||||
Cargo.toml
|
||||
src/config.rs
|
||||
src/vault.rs
|
||||
</files>
|
||||
<action>
|
||||
1. **Cargo.toml**: Add `ureq = "2.12"` to `[dependencies]`. ureq is a synchronous HTTP client
|
||||
with no async runtime requirement -- fits the project's "no tokio/async runtime" decision.
|
||||
|
||||
2. **src/config.rs**: Add `allowed_remote_domains` field to the `Config` struct:
|
||||
```rust
|
||||
#[serde(default)]
|
||||
pub allowed_remote_domains: Vec<String>,
|
||||
```
|
||||
Also add the field to the `Default` impl with an empty Vec default.
|
||||
IMPORTANT: The struct uses `#[serde(deny_unknown_fields)]` -- this field MUST be added to
|
||||
the struct, not just documented. The `#[serde(default)]` attribute ensures existing configs
|
||||
without this field still parse correctly.
|
||||
|
||||
3. **src/vault.rs**: Add a new public enum and function:
|
||||
|
||||
```rust
|
||||
/// Result of attempting to fetch a remote markdown document.
|
||||
pub enum RemoteDocument {
|
||||
/// Successfully fetched and content appears to be markdown.
|
||||
Loaded { url: String, content: String },
|
||||
/// Domain is not in the whitelist.
|
||||
DomainNotAllowed { domain: String },
|
||||
/// HTTP request failed (network error, timeout, non-2xx status).
|
||||
FetchError { url: String, reason: String },
|
||||
/// Response content does not appear to be markdown (e.g. HTML page, binary).
|
||||
NotMarkdown { url: String, content_type: String },
|
||||
}
|
||||
```
|
||||
|
||||
Add function `pub fn fetch_remote_markdown(url: &str, allowed_domains: &[String]) -> RemoteDocument`:
|
||||
- Extract the domain from the URL using simple string parsing (split on `://`, then take
|
||||
everything before the first `/` or end of string, then strip any port number with rfind(':'))
|
||||
- Check domain against `allowed_domains` (case-insensitive comparison). Also check if the
|
||||
URL domain ENDS WITH a whitelisted domain prefixed by `.` -- this allows `sub.example.com`
|
||||
to match whitelist entry `example.com`.
|
||||
- If not whitelisted, return `RemoteDocument::DomainNotAllowed`.
|
||||
- Use `ureq::get(url).call()` with a 10-second timeout via `.timeout(Duration::from_secs(10))`.
|
||||
- On success, check the Content-Type header: accept `text/markdown`, `text/plain`, `text/x-markdown`,
|
||||
or any response where the URL path ends in `.md`. If Content-Type indicates HTML (`text/html`)
|
||||
or binary, return `RemoteDocument::NotMarkdown`.
|
||||
- Read the response body as a string (with a 5MB size limit to prevent memory exhaustion:
|
||||
use `.into_reader().take(5_000_000)` and read to string).
|
||||
- Return `RemoteDocument::Loaded` with the URL and content.
|
||||
- On any ureq error, return `RemoteDocument::FetchError` with the error message.
|
||||
</action>
|
||||
<verify>
|
||||
`cargo check` passes with no errors. The new `allowed_remote_domains` field exists in Config.
|
||||
The `fetch_remote_markdown` function compiles and is public.
|
||||
</verify>
|
||||
<done>
|
||||
Config struct has `allowed_remote_domains: Vec<String>` with serde default.
|
||||
`vault::fetch_remote_markdown` function exists, handles domain whitelisting, HTTP fetching,
|
||||
content-type validation, and returns appropriate RemoteDocument variants.
|
||||
ureq dependency is in Cargo.toml.
|
||||
</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Wire remote link navigation into app event loop with history and error screens</name>
|
||||
<files>
|
||||
src/app.rs
|
||||
src/renderer.rs
|
||||
</files>
|
||||
<action>
|
||||
1. **src/renderer.rs**: In the `Event::Start(Tag::Link { .. })` handler, add HTTP link detection.
|
||||
Currently all non-wiki links get `Style::default().fg(Color::LightCyan)`. Add a check:
|
||||
if `dest_url.starts_with("http://") || dest_url.starts_with("https://")`, use
|
||||
`Style::default().fg(Color::LightMagenta)` instead of `LightCyan` to visually distinguish
|
||||
remote links from local links. This goes in the `else` branch (non-wiki links) around line 603.
|
||||
|
||||
2. **src/app.rs** -- Add a `current_url` field to `App` struct:
|
||||
```rust
|
||||
/// When viewing a remote document, stores the source URL for display in breadcrumb.
|
||||
/// None when viewing a local vault document.
|
||||
current_url: Option<String>,
|
||||
```
|
||||
Initialize to `None` in `App::new()`. Set to `Some(url)` when navigating to a remote page,
|
||||
set back to `None` when navigating to a local page.
|
||||
|
||||
3. **src/app.rs** -- Modify `follow_selected_link()`: In the `else` (non-wiki) branch, BEFORE
|
||||
the existing `resolve_standard_link` logic, add a check:
|
||||
```rust
|
||||
if dest.starts_with("http://") || dest.starts_with("https://") {
|
||||
self.navigate_to_remote(&dest);
|
||||
return;
|
||||
}
|
||||
```
|
||||
|
||||
4. **src/app.rs** -- Add new method `navigate_to_remote(&mut self, url: &str)`:
|
||||
- Save current state to history (same pattern as `navigate_to`).
|
||||
- Truncate forward history.
|
||||
- Call `crate::vault::fetch_remote_markdown(url, &self.config.allowed_remote_domains)`.
|
||||
- Match on the result:
|
||||
- `RemoteDocument::Loaded { url, content }`:
|
||||
- Derive filename from the URL path (last segment, or the domain if path is `/`).
|
||||
- Render with `crate::renderer::render_markdown(&content, width, None)` -- pass `None`
|
||||
for vault_path since wiki-links in remote content cannot resolve to the local vault.
|
||||
- Set `self.document = DocumentState::Loaded { filename, lines }`.
|
||||
- Set `self.raw_content = Some(content)`.
|
||||
- Set `self.current_url = Some(url)`.
|
||||
- Set `self.current_path` to the URL string (for breadcrumb display).
|
||||
- Update link_records, copyable_blocks, reset scroll, push history entry.
|
||||
- Do NOT call `self.rewatch_for_current_page()` (no file to watch).
|
||||
- But DO stop the watcher from watching irrelevant dirs: unwatch is optional,
|
||||
just skip the rewatch call.
|
||||
- `RemoteDocument::DomainNotAllowed { domain }`:
|
||||
- Show an error screen: set `self.document = DocumentState::Error` with
|
||||
`path: PathBuf::from(url)` and
|
||||
`reason: format!("Domain '{}' is not in the allowed remote domains list. Add it to allowed_remote_domains in bbs.toml.", domain)`.
|
||||
- Clear link_records, copyable_blocks, raw_content. Set current_url to None.
|
||||
- Do NOT push to history (same pattern as Missing/ReadError in navigate_to).
|
||||
- `RemoteDocument::FetchError { url, reason }`:
|
||||
- Show error: `DocumentState::Error { path: PathBuf::from(&url), reason }`.
|
||||
- Same cleanup as DomainNotAllowed. Do NOT push to history.
|
||||
- `RemoteDocument::NotMarkdown { url, content_type }`:
|
||||
- Show error: `DocumentState::Error { path: PathBuf::from(&url), reason: format!("Remote content is not markdown (Content-Type: {})", content_type) }`.
|
||||
- Same cleanup. Do NOT push to history.
|
||||
|
||||
5. **src/app.rs** -- Update `navigate_back()` and `navigate_forward()`: When restoring a history
|
||||
entry whose path starts with "http://" or "https://", call `navigate_to_remote` logic instead
|
||||
of `load_document`. The simplest approach: check `if target_path.starts_with("http://") || target_path.starts_with("https://")`,
|
||||
then re-fetch the remote content (same as initial navigation -- no caching, consistent with
|
||||
the existing "re-load from disk" pattern for local docs).
|
||||
|
||||
6. **src/app.rs** -- Update `handle_resize()`: When `self.current_url.is_some()`, re-render
|
||||
using the stored `raw_content` (same as local docs) but pass `None` for vault_path.
|
||||
Currently handle_resize always passes `Some(&vault_path)` -- add a conditional:
|
||||
```rust
|
||||
let vault_ref = if self.current_url.is_some() { None } else { Some(&*vault_path) };
|
||||
```
|
||||
|
||||
7. **src/app.rs** -- Update `reload_current_document()`: When `self.current_url.is_some()`,
|
||||
skip reload entirely (remote content does not live-reload from filesystem).
|
||||
Add early return: `if self.current_url.is_some() { return; }`.
|
||||
|
||||
8. **src/app.rs** -- Update `build_breadcrumb()` or the status bar: When `current_path` starts
|
||||
with `http`, show a truncated URL in the breadcrumb instead of path components. The existing
|
||||
`build_breadcrumb` function will naturally split on `/` which works acceptably for URLs
|
||||
(e.g. "https: > example.com > docs > page.md"). This is acceptable BBS behavior. No change
|
||||
needed unless it looks terrible -- use your judgment.
|
||||
|
||||
9. **src/app.rs** -- In `navigate_to()` (the local navigation method), set
|
||||
`self.current_url = None` to clear remote state when navigating back to a local page.
|
||||
</action>
|
||||
<verify>
|
||||
`cargo build` succeeds. Test manually:
|
||||
1. Create a `bbs.toml` with `allowed_remote_domains = ["raw.githubusercontent.com"]`
|
||||
2. Create a vault markdown file with a link like `[Test](https://raw.githubusercontent.com/some/repo/main/README.md)`
|
||||
3. Run `cargo run -- --config bbs.toml`, navigate to the link, press Enter
|
||||
4. Verify the remote markdown renders in the TUI
|
||||
5. Verify pressing Backspace returns to the previous local page
|
||||
6. Test a non-whitelisted domain link shows the domain-not-allowed error
|
||||
7. Verify `cargo clippy` has no errors (warnings acceptable)
|
||||
</verify>
|
||||
<done>
|
||||
HTTP/HTTPS links in markdown are detected and visually distinguished (LightMagenta color).
|
||||
Following a whitelisted remote link fetches, validates, and renders the markdown content.
|
||||
Non-whitelisted domains show a clear error with instructions to add to config.
|
||||
Fetch failures and non-markdown content show appropriate error screens.
|
||||
Remote pages participate in back/forward history navigation.
|
||||
Resize re-renders remote content correctly. Live reload is skipped for remote pages.
|
||||
</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<verification>
|
||||
1. `cargo build` compiles without errors
|
||||
2. `cargo clippy` passes (warnings acceptable)
|
||||
3. Remote link to whitelisted domain renders markdown in TUI
|
||||
4. Remote link to non-whitelisted domain shows error screen with domain name
|
||||
5. Remote link to non-existent URL shows fetch error screen
|
||||
6. Remote link to HTML content shows "not markdown" error screen
|
||||
7. Back/forward navigation works through remote pages
|
||||
8. Terminal resize re-renders remote content correctly
|
||||
9. Existing local vault navigation is completely unaffected
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
- bbs.toml `allowed_remote_domains = ["example.com"]` config field works
|
||||
- HTTP/HTTPS links render in LightMagenta (visually distinct from local LightCyan links)
|
||||
- Following whitelisted HTTP links fetches and displays remote markdown
|
||||
- Non-whitelisted domains show BBS-themed error with domain name and config hint
|
||||
- Fetch errors and non-markdown responses show appropriate error screens
|
||||
- Browser history (back/forward) works through local and remote pages
|
||||
- No regressions in local vault navigation, wiki-links, or directory listing
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
After completion, create `.planning/quick/3-implement-remote-markdown-page-linking-w/3-SUMMARY.md`
|
||||
</output>
|
||||
Reference in New Issue
Block a user