Jeevan Surya Maddu

Compliance Monitoring at Scale

Building detection from nothing, after a blind spot in my own instrumentation taught me how

PM ยท 2021 to 2022

TL;DR

Goldman Sachs classified any access to a production database from a non-production host as a violation, and had no way to see when it happened. I built the detection for the Technical Risk team from nothing. It surfaced roughly one million such accesses, each one with enough detail attached that someone could actually go and fix it.

The problem

The rule was clear and the visibility was zero. Nobody could answer how often it happened, which processes were responsible, or where they ran. Without that, the policy was a statement of intent rather than a control.

What I owned

The detection, end to end: enumerating every access path, bringing every access method into one unified log rather than several partial views nobody could reconcile, and an automated report on top that gave Technical Risk what they needed to act: which process, where it ran, and when. That last part is the difference between a tool and a product. Acting on each finding was Technical Risk's call.

The hard part

Coverage before code. The first thing I did was not write anything. It was to enumerate every way a connection to these databases could be initiated, so that no access path sat outside the net. I knew to start there because of the previous project. The year before, I led the migration of 16,000 databases onto secure infrastructure after an internal audit finding, and built a logging system to find out what accessed them. The Japan businesses never appeared in those logs. I ran the migration out of hours with a revert path ready, the Japan processes failed exactly as the risk anticipated, and we reverted in minutes with no business impact. Those processes reached the databases in other languages, which is why my instrumentation had not seen them. The lesson was that instrumentation you have not proven complete is evidence of nothing.

Outcome

Roughly one million accesses to production databases from non-production hosts surfaced, each either shut down or moved onto production infrastructure. The policy became a control.