Appsec

Background The risk-based framework I built scores and gates test suites. The AI extension does the same for agents and LLM features. Both answer how much testing, and where it blocks a merge. Neither answers when. The Problem Lately, when has been the problem. A feature ships, the PR lands, and that’s the first time QA sees it. Testing becomes archaeology: reconstruct what the feature was supposed to do from the diff, then work out what wasn’t tested. ...

Background Last month I wrote about the risk-based testing framework I built to keep QA on pace with agentic coding: twenty test categories five phases a scoring rubric that decides where the effort goes. Then the products started shipping AI features. They were the same products as before, handling PII and other regulated, person-level business data. The Problem The original framework breaks in four places. You can’t enumerate the input space. 80% line coverage means something. “80% of prompts” means nothing. There is no denominator. ...

Background Over the last week I’ve been building a risk-based framework for testing software products and documenting evidence for audits. The products handle PII, worker and employer data, payrolling, staffing and placement, benefits, prescreening, compliance, Vendor Management Systems (VMSs), Managed Service Providers (MSPs), and all kinds of other fun stuff. They serve clients in the US and in jurisdictions across the EU, the UK, and Canada. The Problem Testing had to keep pace with developers who had just picked up agentic coding tools, and it couldn’t. ...