Ensure web edits preserve comments as much as possible #34

Closed
opened 2026-07-07 14:42:59 +01:00 by danny · 2 comments
danny commented 2026-07-07 14:42:59 +01:00 (Migrated from gitlab.keyop.co.uk)

Ideally any edits via the Web UI should attempt to preserve surrounding comments as well as any preamble comments.

Ideally any edits via the Web UI should attempt to preserve surrounding comments as well as any preamble comments.
danny commented 2026-07-07 14:43:47 +01:00 (Migrated from gitlab.keyop.co.uk)

changed the description

changed the description
Collaborator

Plan

Found (by reading the actual code, not assumed) that comments are destroyed in two
independent places, not one:

  1. resolveSubmittedYAML's yaml.Marshal(req.Config) (structured-form save path) - a fresh
    config.Configuration struct has no memory of the original file's comments at all.
  2. config_edit.js's switchToRaw() unconditionally overwrites the raw textarea with
    server-marshaled (comment-free) YAML on every form→raw toggle - comments are lost just by
    glancing at the structured form and switching back, even without ever saving.

Root cause of both: config.Configuration carries no comment/structure metadata at all - plain
Go structs, only json/yaml tags. gopkg.in/yaml.v3 (already a dependency) supports exactly
this via its yaml.Node tree (HeadComment/LineComment/FootComment per node) - confirmed
empirically against the real library: a preamble comment lands in HeadComment on the key
node, an inline comment lands in LineComment on the value node, and a comment before a
sequence item lands in HeadComment on the item node itself - which is your suggested
preamble-block convention; it's already how yaml.v3 represents it, nothing extra needed to honor
that.

Design

New internal/config/comments.go: MergeIntoYAML(baseYAML []byte, newConfig *Configuration)
marshals newConfig the ordinary way into a comment-free "shape" tree, then splices the base
document's comments onto matching nodes of that shape tree (the shape's own values are always
kept - they're already correct, only comments move):

  • Scalar fields (including nested defaults/web_ui): copy head/line/foot comments directly
    onto the matching key/value nodes.
  • Sequences (connections, defaults.by_network, forwarded_ports): match items by natural
    identity, not position
    , so a reorder doesn't strand a comment on the wrong entry -
    connections by name (falling back to queue_number, mirroring Connection.Label()'s own
    precedence), by_network by cidr, forwarded_ports by the port value itself. An added item
    gets no comments (correct - freshly marshaled); a removed item's comments simply go with it -
    deliberately not reattached to some unrelated neighbor, since that would be more
    misleading than losing them.
  • Falls back to a plain yaml.Marshal (today's behavior) if the base YAML doesn't parse - never
    hard-fails an edit just because the base became unparseable.

Wiring: configEditRequest gains base_yaml alongside the existing base_hash - the client
must resend whatever YAML text it last received as the merge base every time it submits a
structured Config, since a validate call never touches disk and has no other way to know what
comments to preserve. Each round trip's returned yaml becomes the next round's base, so
comments survive arbitrarily many mode toggles within a session, not just a single save. The
save path itself merges onto the already-hash-verified on-disk bytes rather than trusting the
client-sent base, so it can never be stale or forged.

Net effect: comments are only ever edited by typing in the raw YAML textarea (structured-form
fields have nowhere to put a comment) - but they now survive being carried through a
structured-form edit session and back out again, instead of being destroyed the moment the
structured form is touched.

Branch: issue34_preserve_comments.

## Plan Found (by reading the actual code, not assumed) that comments are destroyed in **two** independent places, not one: 1. `resolveSubmittedYAML`'s `yaml.Marshal(req.Config)` (structured-form save path) - a fresh `config.Configuration` struct has no memory of the original file's comments at all. 2. `config_edit.js`'s `switchToRaw()` unconditionally overwrites the raw textarea with server-marshaled (comment-free) YAML on **every** form→raw toggle - comments are lost just by glancing at the structured form and switching back, even without ever saving. Root cause of both: `config.Configuration` carries no comment/structure metadata at all - plain Go structs, only `json`/`yaml` tags. `gopkg.in/yaml.v3` (already a dependency) supports exactly this via its `yaml.Node` tree (`HeadComment`/`LineComment`/`FootComment` per node) - confirmed empirically against the real library: a preamble comment lands in `HeadComment` on the **key** node, an inline comment lands in `LineComment` on the **value** node, and a comment before a sequence item lands in `HeadComment` on the **item** node itself - which *is* your suggested preamble-block convention; it's already how yaml.v3 represents it, nothing extra needed to honor that. ### Design New `internal/config/comments.go`: `MergeIntoYAML(baseYAML []byte, newConfig *Configuration)` marshals `newConfig` the ordinary way into a comment-free "shape" tree, then splices the base document's comments onto matching nodes of that shape tree (the shape's own values are always kept - they're already correct, only comments move): - Scalar fields (including nested `defaults`/`web_ui`): copy head/line/foot comments directly onto the matching key/value nodes. - Sequences (`connections`, `defaults.by_network`, `forwarded_ports`): match items by **natural identity, not position**, so a reorder doesn't strand a comment on the wrong entry - `connections` by `name` (falling back to `queue_number`, mirroring `Connection.Label()`'s own precedence), `by_network` by `cidr`, `forwarded_ports` by the port value itself. An added item gets no comments (correct - freshly marshaled); a removed item's comments simply go with it - deliberately **not** reattached to some unrelated neighbor, since that would be more misleading than losing them. - Falls back to a plain `yaml.Marshal` (today's behavior) if the base YAML doesn't parse - never hard-fails an edit just because the base became unparseable. Wiring: `configEditRequest` gains `base_yaml` alongside the existing `base_hash` - the client must resend whatever YAML text it last received as the merge base every time it submits a structured `Config`, since a validate call never touches disk and has no other way to know what comments to preserve. Each round trip's returned `yaml` becomes the next round's base, so comments survive arbitrarily many mode toggles within a session, not just a single save. The save path itself merges onto the already-hash-verified on-disk bytes rather than trusting the client-sent base, so it can never be stale or forged. Net effect: comments are only ever *edited* by typing in the raw YAML textarea (structured-form fields have nowhere to put a comment) - but they now survive being carried through a structured-form edit session and back out again, instead of being destroyed the moment the structured form is touched. Branch: `issue34_preserve_comments`.
danny closed this issue 2026-07-10 08:06:23 +01:00
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
keyop-go/nfq_forwarder#34
No description provided.