A couple weeks ago, I ranted about one of our teams causing an issue by testing stuff in a prod environment.
Well, that came back up last night and blew the fuck up.
Yesterday afternoon, someone was looking at one of the reports we put together, and went "Hey, this little piece here? This isn't right." So we look at the system of record (that had tests done in prod), and it matches what's on our report. It's correct. So we peel back another layer, and look at the data that feeds our system of record.
And sure enough, he's correct. It's wrong. Our system of record has data inaccuracies in it.
Fuck.
The reason that's a big deal, is because those reports have already been distributed and are about to be presented in 2 hours to a bunch of regulators, the boards of directors, and just overall a pile of people making 7 and 8 figures. And while the actual inaccuracies themselves are very minor, it calls into question the data and legitimacy of everything else in the report.