my gt research: the paper's basically done, and the one number i wasn't proud of just got a better story (week four)

week four, in two sentences: the 76.7% cell from two weeks ago, the one i wrote up as a limitation because a single rewrite pass couldn't suppress enough at the widest clearance gap, is now 0% under an iterative suppress-verify-repair loop, with confidence intervals to back it. and a third external review forced a genuinely better explanation of what my deterministic stage is for, which turned out to matter more than the number itself.

last time the choice on the table was offense or defense: spend the week chasing the bad cell, or lock scope and anchor the judge against human labels instead. i picked defense. this week both things happened anyway, just not in the order i expected.

the mls result, closed

two weeks ago the multi-level clearance table had one row i wasn't rounding off: a recipient cleared for almost nothing, reading a message centered on the diagnosis, where one rewrite pass had to suppress too much at once and the clinical implication survived 76.7% of the time. the fix wasn't a smarter single pass, it was removing the "single" — suppress, verify, and if the check fails, repair and check again.

all three cells at zero, n=60, with clopper-pearson intervals attached this time instead of a bare percentage. and because a system scoring 0% on its own judge is a little too convenient to trust blind, i reran it under a second, harsher independent judge (mistral, not the model that built the gate) and it holds at 0% / 0% / 10%. the 10% residual is at the exact boundary that broke last time, secret information reaching a recipient cleared for almost nothing, and i'm keeping that number in the paper rather than only reporting the friendlier judge.

a third review, and the number that mattered more than a number

i ran the draft through a third detailed review and synthesized it against two earlier ones, then went and measured the three things reviewers would actually push on instead of just responding in prose.

the sharpest one: what if you just give a single llm call the full disallowed set, literal values included, in one shot? call that C1. it turns out C1 matches ciguard's two-stage semantic leak rate exactly, 17.3% versus 17.3%. for about a day that looked like a real problem — if a single call ties the two-stage system, what's the second stage buying you? the answer was sitting in the data i'd already collected: C1 leaves an 8.0% literal residual, because nothing stops an llm from just emitting the ssn verbatim if it decides to. the deterministic stage was never supposed to lower the semantic number, it's what turns "usually redacts literal pii" into a guaranteed, auditable floor. that's a better sentence than the one i had before, and i only found it because a reviewer forced the comparison i'd been avoiding.

the rest of the hardening pass was less dramatic but closed real gaps: re-running the deterministic deletion after the rewrite (D1) restores the 0.0% exact-match floor by construction, which explains a 0.4% residual a reviewer had flagged. two grader models agree with each other 79.2% of the time (kappa 0.58, D2), which is why i now read success as an off-to-on delta within one fixed grader instead of trusting either grader's absolute number. utility holds up on both channels i measured, 91.8% and 82.3% retention. and a full number audit against the results log caught three p-values i'd stated too tightly and one significance test attributed to the wrong comparison — small, but exactly the kind of thing that should never survive to a submission.

one change i'm calling out plainly rather than folding into the audit: i dropped the human-labeling step i was planning two weeks ago. instead the semantic judge is anchored on the benchmark's canary ground truth, which is objective and recall-safe by construction, and the human-annotated validation moved to future work. that's a narrower claim than "a human checked this," and i'd rather ship the narrower true claim than the broader one i can't fully back yet.

where it actually stands

the venue moved from ieee access to ieee open journal of the computer society per my advisor's call, so main.tex is now on the ojcs class, all three architecture figures are redrawn at v3 in the ojcs style, and all 23 measurements (E1 through E23) are done, ahead of the july 15 completeness gate. the paper sits at 12 pages, or 13 right now because my bio and photo are still a placeholder.

what's actually left is almost entirely on me, not the research: bio and photo, confirming the final title (whether to foreground "privacy" more directly), and a last consistency read before submission on july 24.

next

bio, photo, title sign-off, submit. the honest-limitation columns stay in the paper — the 10% boundary case under the harsher judge, the human-validation-as-future-work note — because a clean paper isn't one with no residuals, it's one where every residual is measured and explained instead of rounded away.