Redact sensitive information from repo #24

Closed
opened 2026-07-05 19:24:30 +01:00 by danny · 8 comments
danny commented 2026-07-05 19:24:30 +01:00 (Migrated from gitlab.keyop.co.uk)

We currently have commits (especially in the notes/ directory which contains Claude chat exports) which include gitlab access tokens.

We should check the entire repository for any other sensitive information, and force a history rewrite to redact this information from the repo without removing the files entirely.

For now notes/ has been excluded via .gitignore, but this does not remove the history. Once redactions are made (as have been done manually in some of the current files), the .gitignore exclusion can be removed.

It may make sense to include a Makefile target to handle automatically redacting this information, for use prior to committing any further exports.

We currently have commits (especially in the notes/ directory which contains Claude chat exports) which include gitlab access tokens. We should check the entire repository for any other sensitive information, and force a history rewrite to redact this information from the repo without removing the files entirely. For now notes/ has been excluded via .gitignore, but this does not remove the history. Once redactions are made (as have been done manually in some of the current files), the .gitignore exclusion can be removed. It may make sense to include a Makefile target to handle automatically redacting this information, for use prior to committing any further exports.
danny commented 2026-07-05 19:27:16 +01:00 (Migrated from gitlab.keyop.co.uk)

changed the description

changed the description
danny commented 2026-07-05 21:41:08 +01:00 (Migrated from gitlab.keyop.co.uk)

Findings

Scanned the full history (gitleaks git --log-opts="--all", 101 commits) and the current
working tree (gitleaks dir), plus targeted manual checks for anything a regex-based scanner
might miss (private key blocks, AWS-style keys, password assignments, accidentally-committed
config.production.yaml, hardcoded secrets in .gitlab-ci.yml/.goreleaser.yaml).

Two distinct GitLab Personal Access Tokens are exposed in history, both already manually
redacted (to glpat-<redacted>) in the current working-tree copies, but still fully present
and retrievable in earlier commits:

File Commit
notes/issue3_export.txt bcf6133093d9
notes/issue5_webUI.txt f0701257c5fa
notes/issue14_export.txt 30722b03512d
notes/issue19_export.txt 9871067144f0
notes/issue20_simplify_config.txt (formerly nodes/issue20_simplify_config.txt) 4ce5fcfc9754

One of the two tokens is (as of this scan) still the active token in use - flagged directly to
Danny separately as needing immediate revocation/rotation, independent of this ticket
: rewriting
history doesn't undo prior exposure to GitLab's own server-side copy or anything that already
cloned/fetched the repo.

Nothing else found: no private keys (BEGIN ... PRIVATE KEY), no AWS-style keys, no other
API keys/passwords, and config.production.yaml (the real, gitignored prod config) was never
accidentally committed. Every hit was one of the two token values above, in notes/ only - the
actual source tree is clean.

Not a secret, but worth a call on: emails appearing in commit authorship/history are all
Danny's own, across several past employers (gmail, keyop.co.uk, cinesite.com, framestore.com)
plus bot addresses (goreleaser/openssh/anthropic) - no third-party PII, but worth deciding
whether that history is fine to have public.

Also worth checking, separate from git history: whether either token was ever echoed into a
GitLab CI job log (this repo's CI runs shell commands against GITLAB_TOKEN) - job logs are a
separate exposure surface a git history rewrite doesn't touch.

Plan

  1. (Blocking, outside this ticket's scope) Both tokens treated as compromised; the active one
    revoked/rotated in GitLab before or independent of the steps below.
  2. Build an exact replacement list (the two literal token values, redacted to
    glpat-<redacted> - matching the redaction convention already used in the working tree) for
    git filter-repo --replace-text (the modern, git-recommended tool for exactly this - rewrites
    matching text within blobs across all history without deleting the files themselves, unlike
    BFG's file-deletion-oriented approach).
  3. Run it against a fresh mirror clone, never the primary working copy directly.
  4. Verify: re-run gitleaks against the rewritten mirror (expect zero hits), spot-check a few
    rewritten commits by hand, confirm commit messages/authorship/timestamps/graph structure are
    otherwise unchanged (only blob text and the resulting SHAs change).
  5. Coordinate the force-push: every commit from the first affected one onward gets a new SHA,
    which means every branch and every existing tag (v1.0.0-v1.0.5) needs re-tagging too.
    No open MRs currently exist (checked), which simplifies this. Danny's own local clone(s), and
    this session's local clone, will need a hard reset or re-clone after the rewrite - a stale
    clone can't be reconciled, only discarded. The stale issue17_unknown_host_keys remote
    branch (already merged, not auto-deleted) should be deleted as part of this, since it still
    points at pre-rewrite history.
  6. Once verified clean on the remote, remove the notes/ exclusion from .gitignore (per the
    ticket) so future exports track normally again.
  7. Add a make target that runs a redaction script (matching the glpat-[A-Za-z0-9_\-\.]+
    shape generically, not just today's two known values) across notes/*.txt, so future chat
    exports get scrubbed automatically before git add rather than relying on manual edits.

No changes have been made yet - this comment is the plan only, per Danny's request, awaiting
sign-off before the history rewrite (step 2 onward) proceeds.

## Findings Scanned the full history (`gitleaks git --log-opts="--all"`, 101 commits) and the current working tree (`gitleaks dir`), plus targeted manual checks for anything a regex-based scanner might miss (private key blocks, AWS-style keys, password assignments, accidentally-committed `config.production.yaml`, hardcoded secrets in `.gitlab-ci.yml`/`.goreleaser.yaml`). **Two distinct GitLab Personal Access Tokens are exposed in history**, both already manually redacted (to `glpat-<redacted>`) in the *current* working-tree copies, but still fully present and retrievable in earlier commits: | File | Commit | |---|---| | `notes/issue3_export.txt` | `bcf6133093d9` | | `notes/issue5_webUI.txt` | `f0701257c5fa` | | `notes/issue14_export.txt` | `30722b03512d` | | `notes/issue19_export.txt` | `9871067144f0` | | `notes/issue20_simplify_config.txt` (formerly `nodes/issue20_simplify_config.txt`) | `4ce5fcfc9754` | One of the two tokens is (as of this scan) still the active token in use - **flagged directly to Danny separately as needing immediate revocation/rotation, independent of this ticket**: rewriting history doesn't undo prior exposure to GitLab's own server-side copy or anything that already cloned/fetched the repo. **Nothing else found**: no private keys (`BEGIN ... PRIVATE KEY`), no AWS-style keys, no other API keys/passwords, and `config.production.yaml` (the real, gitignored prod config) was never accidentally committed. Every hit was one of the two token values above, in `notes/` only - the actual source tree is clean. **Not a secret, but worth a call on**: emails appearing in commit authorship/history are all Danny's own, across several past employers (gmail, keyop.co.uk, cinesite.com, framestore.com) plus bot addresses (goreleaser/openssh/anthropic) - no third-party PII, but worth deciding whether that history is fine to have public. **Also worth checking, separate from git history**: whether either token was ever echoed into a GitLab CI job log (this repo's CI runs shell commands against `GITLAB_TOKEN`) - job logs are a separate exposure surface a git history rewrite doesn't touch. ## Plan 0. **(Blocking, outside this ticket's scope)** Both tokens treated as compromised; the active one revoked/rotated in GitLab before or independent of the steps below. 1. Build an exact replacement list (the two literal token values, redacted to `glpat-<redacted>` - matching the redaction convention already used in the working tree) for `git filter-repo --replace-text` (the modern, git-recommended tool for exactly this - rewrites matching text within blobs across all history without deleting the files themselves, unlike BFG's file-deletion-oriented approach). 2. Run it against a **fresh mirror clone**, never the primary working copy directly. 3. Verify: re-run `gitleaks` against the rewritten mirror (expect zero hits), spot-check a few rewritten commits by hand, confirm commit messages/authorship/timestamps/graph structure are otherwise unchanged (only blob text and the resulting SHAs change). 4. Coordinate the force-push: every commit from the first affected one onward gets a new SHA, which means every branch and **every existing tag (`v1.0.0`-`v1.0.5`) needs re-tagging** too. No open MRs currently exist (checked), which simplifies this. Danny's own local clone(s), and this session's local clone, will need a hard reset or re-clone after the rewrite - a stale clone can't be reconciled, only discarded. The stale `issue17_unknown_host_keys` remote branch (already merged, not auto-deleted) should be deleted as part of this, since it still points at pre-rewrite history. 5. Once verified clean on the remote, remove the `notes/` exclusion from `.gitignore` (per the ticket) so future exports track normally again. 6. Add a `make` target that runs a redaction script (matching the `glpat-[A-Za-z0-9_\-\.]+` shape generically, not just today's two known values) across `notes/*.txt`, so future chat exports get scrubbed automatically before `git add` rather than relying on manual edits. No changes have been made yet - this comment is the plan only, per Danny's request, awaiting sign-off before the history rewrite (step 2 onward) proceeds.
danny commented 2026-07-05 21:52:55 +01:00 (Migrated from gitlab.keyop.co.uk)

Status update

Token rotated by Danny - confirmed working with a freshly-issued PAT. Proceeding with the plan now:

  • Item 5 (removing notes/ from .gitignore) already done by Danny - skipping.
  • Item 6 (Makefile redaction target) expanded per Danny's request to also include an automated gitleaks run, blocking commits that contain secrets rather than just a manual redaction script.

Proceeding with steps 1-4 (build replacement list, rewrite history in a fresh mirror clone,
verify, then force-push) now. Will post progress/results here as each stage completes.

## Status update Token rotated by Danny - confirmed working with a freshly-issued PAT. Proceeding with the plan now: - Item 5 (removing `notes/` from `.gitignore`) already done by Danny - skipping. - Item 6 (Makefile redaction target) expanded per Danny's request to also include an automated `gitleaks` run, blocking commits that contain secrets rather than just a manual redaction script. Proceeding with steps 1-4 (build replacement list, rewrite history in a fresh mirror clone, verify, then force-push) now. Will post progress/results here as each stage completes.
danny commented 2026-07-06 07:53:54 +01:00 (Migrated from gitlab.keyop.co.uk)

mentioned in commit dec19a6119

mentioned in commit dec19a61194ca27262fe103182121fbd5f915290
danny commented 2026-07-06 07:54:24 +01:00 (Migrated from gitlab.keyop.co.uk)

mentioned in merge request !67

mentioned in merge request !67
danny commented 2026-07-06 07:56:13 +01:00 (Migrated from gitlab.keyop.co.uk)

History rewrite complete

  • Rewrote the full repo history with `git filter-repo --replace-text` (fresh mirror clone, not
    the working copy) redacting the exposed token pattern everywhere it appeared.
  • Verified: `gitleaks` against the rewritten history reports zero leaks; spot-checked
    several previously-affected commits by hand to confirm the redaction landed cleanly and
    surrounding content is otherwise untouched; `git fsck --full` clean.
  • Force-pushed `main` and all 6 tags (`v1.0.0`-`v1.0.5`) to the rewritten history. This
    required temporarily relaxing `main`'s "no force push" protection and removing the `v*`
    protected-tag rule (both restored by Danny immediately after - please double check they're
    back the way you want them).
  • As expected/discussed beforehand, force-pushing the tags re-triggered their release pipelines
    (#353-356). These are failing - benignly, for an unrelated reason: the project's
    `GITLAB_TOKEN` CI/CD variable is currently missing (`missing GITHUB_TOKEN, GITLAB_TOKEN and
    GITEA_TOKEN` in the job log), so `goreleaser release` can't even authenticate. Nothing was
    re-published or corrupted - it fails before doing anything. Worth restoring that CI/CD
    variable (with the newly-rotated token) before the next real release tag is pushed, though,
    or that'll fail the same way.
  • Local clones (mine, and presumably yours) need a hard reset/re-clone against the new history -
    a pre-rewrite clone can't be merged or rebased against it.

Prevention (MR !67)

Item 6 implemented and expanded per your request:

  • `make check-secrets` (full-history `gitleaks` sweep) is now part of `make lint`, so it's
    covered by CI's existing lint job automatically.
  • `make install-git-hooks` installs an opt-in pre-commit hook (`gitleaks protect --staged`)
    that blocks committing a likely secret before it's even committed, not just before it's merged.

MR: !67 - https://gitlab.keyop.co.uk/keyop/go/nfq_forwarder/-/merge_requests/26

Once that's reviewed/merged, this ticket should be ready to close.

## History rewrite complete - Rewrote the full repo history with \`git filter-repo --replace-text\` (fresh mirror clone, not the working copy) redacting the exposed token pattern everywhere it appeared. - Verified: \`gitleaks\` against the rewritten history reports **zero leaks**; spot-checked several previously-affected commits by hand to confirm the redaction landed cleanly and surrounding content is otherwise untouched; \`git fsck --full\` clean. - Force-pushed \`main\` and all 6 tags (\`v1.0.0\`-\`v1.0.5\`) to the rewritten history. This required temporarily relaxing \`main\`'s "no force push" protection and removing the \`v*\` protected-tag rule (both restored by Danny immediately after - please double check they're back the way you want them). - As expected/discussed beforehand, force-pushing the tags re-triggered their release pipelines (#353-356). These are failing - **benignly, for an unrelated reason**: the project's \`GITLAB_TOKEN\` CI/CD variable is currently missing (\`missing GITHUB_TOKEN, GITLAB_TOKEN and GITEA_TOKEN\` in the job log), so \`goreleaser release\` can't even authenticate. Nothing was re-published or corrupted - it fails before doing anything. Worth restoring that CI/CD variable (with the newly-rotated token) before the *next real* release tag is pushed, though, or that'll fail the same way. - Local clones (mine, and presumably yours) need a hard reset/re-clone against the new history - a pre-rewrite clone can't be merged or rebased against it. ## Prevention (MR !67) Item 6 implemented and expanded per your request: - \`make check-secrets\` (full-history \`gitleaks\` sweep) is now part of \`make lint\`, so it's covered by CI's existing lint job automatically. - \`make install-git-hooks\` installs an opt-in pre-commit hook (\`gitleaks protect --staged\`) that blocks committing a likely secret before it's even committed, not just before it's merged. MR: !67 - https://gitlab.keyop.co.uk/keyop/go/nfq_forwarder/-/merge_requests/26 Once that's reviewed/merged, this ticket should be ready to close.
danny commented 2026-07-06 08:17:24 +01:00 (Migrated from gitlab.keyop.co.uk)

mentioned in commit 4a3a9b06ff

mentioned in commit 4a3a9b06ffbaa92f60f745627f1113ef5faea3e5
danny commented 2026-07-06 08:42:26 +01:00 (Migrated from gitlab.keyop.co.uk)

Fixed via MR !67 (merge commit cdeebc12), on top of the history rewrite already pushed directly to main. Summary of everything done:

  • Rewrote the full git history (git filter-repo) to redact the exposed GitLab tokens - verified with gitleaks (zero leaks) and git fsck. Tokens were also rotated by Danny.
  • Force-pushed the rewritten main and all 6 tags; branch/tag protections that had to be temporarily relaxed for that were restored afterward.
  • Added automated prevention: make check-secrets (a full-history gitleaks sweep, now part of make lint and therefore CI's existing lint job) and make install-git-hooks (an opt-in pre-commit hook blocking a secret before it's ever committed).

No release tag needed for this one - the history rewrite already happened directly against main, outside the normal tag/release flow. Closing.

Fixed via MR !67 (merge commit `cdeebc12`), on top of the history rewrite already pushed directly to `main`. Summary of everything done: - Rewrote the full git history (`git filter-repo`) to redact the exposed GitLab tokens - verified with `gitleaks` (zero leaks) and `git fsck`. Tokens were also rotated by Danny. - Force-pushed the rewritten `main` and all 6 tags; branch/tag protections that had to be temporarily relaxed for that were restored afterward. - Added automated prevention: `make check-secrets` (a full-history `gitleaks` sweep, now part of `make lint` and therefore CI's existing lint job) and `make install-git-hooks` (an opt-in pre-commit hook blocking a secret before it's ever committed). No release tag needed for this one - the history rewrite already happened directly against `main`, outside the normal tag/release flow. Closing.
danny (Migrated from gitlab.keyop.co.uk) closed this issue 2026-07-06 08:42:56 +01:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
keyop-go/nfq_forwarder#24
No description provided.