For B2B SaaS teams in healthcare, fintech, insurance, legal & accounting
Your AI works in the demo.
Is it still working in production?
AI assurance and operations for regulated SaaS. I get your production AI under control on cost, reliability, and the evidence you need to prove it works.
- 30% identified or the audit is free
- One week, fixed fee
Serhii Panchyshyn
ex‑PwC compliance · production AI engineer
Meet Serhii
I did not start in code. I started in tax compliance at PwC: filings, audits, and rooms where someone senior asks why a number is wrong and “the system did it” is not an answer.
Then seven years of software. Junior to senior at a logistics SaaS handling thousands of transactions a day, backend architecture, and the testing strategy that stopped regressions from reaching customers. The last three years have been production LLM systems: retrieval, multi‑agent orchestration, evals, and cost work.
Since going independent in 2025, I keep walking into the same scene. An AI feature shipped in a hurry. A bill climbing quietly. No one who can prove the thing still works. Everyone built. Nobody is keeping it alive.
That is the job I do now. Engineer enough to fix your system. Compliance enough to defend it.
When your AI gets it wrong, you should hear it from a dashboard, not a customer.
On working with me
Pulled from my public LinkedIn recommendations. Every name links to the source.
“Serhii is one of those rare engineers who not only delivers great results but also raises the standard for everyone around him. I’d gladly work with him again anytime.”
Alexey Novak · Senior Software Engineer
“Consistently one of the most talented and reliable engineers I’ve had the pleasure of working with. He can take full ownership of large projects and see them through to production reliably.”
“He embraces and leans into the unknown, picking up new concepts and ideas incredibly quickly. One of the most productive engineers I know.”
Alicia Chin · ex-Snowflake, ex-IBM
“One of the most driven and impressive people I’ve ever met. An incredible collaborator, always willing to go above and beyond to get results.”
Derek Santos · AI Builder
“Grit, hustle, and total comfort in ambiguity. When something has to ship, Serhii is who I count on to get it done.”
Alexander Luksidadi · CTO & Co-Founder, Rose Rocket (YC S16)
The problem
Five ways production AI quietly fails
You already shipped the feature. These are the failure modes that follow it into production. If even two feel familiar, keep reading.
Silent drift
Your product breaks, and a customer tells you
The provider ships a new model version behind the same endpoint. Same name, same API call, slightly different behaviour. Nothing throws an error. Quality decays for weeks until a customer emails to complain. You learn about your own product from the outside.
Runaway cost
The bill nobody can break down
It went from $3K a month to $30K. Every single increase looked small, so nobody flagged it. Now a board member is asking about the line item. Nobody can answer “what does one user cost us?”
No owner
Shipped in a hurry, held by hope
Someone built the AI feature fast, shipped it, and moved on. Now a production system runs with no evals, no cost ceiling, no runbook, and no one responsible. Everyone is quietly hoping it holds.
No evidence
“How do we know it’s still accurate?”
There were evaluation tests once. They ran once. The person who wrote them is gone. If someone senior asks how you know the AI still works, the honest answer is: you don’t. In a regulated company, that is not an inconvenience. It is a liability.
The compliance gap
Consequences with no regulatory cover
You are in healthcare or finance. An AI output has consequences someone can be fined or sued over. The people who built it have zero regulatory background. Nobody in the building can say the system is defensible.
None of these throw an error. That is the whole problem. Your monitoring watches for crashes. AI systems don’t crash. They rot.
AI assurance and operations
I keep your production AI alive and defensible, on a retainer. Not vague advice. Five concrete things, running every week.
Evaluation suites on a schedule
Quality checks that run weekly, not once. Drift shows up on a dashboard before it shows up in a customer email.
Cost instrumentation and reduction
Every request tagged and attributed. You can finally answer what one user costs. Then we cut the number.
Model migration handling
When a provider deprecates a model, I run the migration and prove the replacement performs before it ships.
Incident response
When the AI breaks, someone owns it. Triage, fix, and a postmortem you can show your board.
The audit trail
The evidence file that proves outputs are correct. Built for the day a regulator, auditor, or enterprise customer asks.
Proof
The numbers hold up
$100K/mo down to $8K/mo
One client’s AI bill. Same product, same traffic, same quality bar. The spend was in the routing, the retries, and the models nobody had questioned.
35% projected infrastructure cut
ContactMonkey’s AI spend, analysed route by route. Same outputs on cheaper paths. The result: an itemised 35% cut in projected infrastructure cost, documented before anything shipped.
Evals before a single user
Also at ContactMonkey: their first generative AI feature, headed for real customers. I built the evaluation framework that verified output quality first. It launched with evidence, not hope.
The offer
Start with the audit
One week inside your AI stack. You get a complete picture of what it costs, what can fail, and what to do about it. Whether we continue or not, you keep everything.
20 minutes
Scope
A short call. You walk me through the stack, I tell you exactly what the audit will cover.
one week
Audit
I go through your AI systems. Cost, eval coverage, model risks, failure modes.
after that
Decide
You keep the written plan either way. Run it yourselves, or I stay on and run it for you.
The AI Assurance Audit
Five deliverables, in writing, in one week:
- Cost map: where every dollar of your AI bill goesEval coverage report: what is tested, what is hoped forModel risk register: what breaks when providers change thingsFailure mode inventory: how your system fails and who finds outA specific, itemised plan to cut spend
$5,000
One week. Fixed fee.
Credited in full toward the retainer if we continue. The retainer is the ongoing service: $15K to $20K a month, everything above, running every week.
I will identify at least 30% of your current AI infrastructure spend in itemised annual savings, or the audit is free.
The $100K-to-$8K result was a standout case — the guarantee is a floor, and I typically find more.
I guarantee what I find and document. Your team decides what to implement. No asterisks beyond that.
Not ready for a call? Email me — serhii.panchyshyn@animanovalabs.com
Who this is for
B2B SaaS teams, 50 to 500 people, Series A to C. You already shipped an AI feature. Your monthly AI bill has five figures in it and nobody can fully explain it. You need the system reliable, affordable, and defensible.
- healthcare
- fintech
- insurance
- legal
- accounting
Not for teams still deciding whether to build AI. This is for the ones who built it and now have to live with it.
FAQ
Questions worth asking first
What does the $5,000 audit include?
Five deliverables, in writing, in one week: a cost map of where every dollar of your AI bill goes, an eval coverage report, a model risk register, a failure mode inventory, and a specific, itemised plan to cut spend. It is a fixed fee, credited in full toward the retainer if we continue.
How do you find 30%+ savings?
By going through your AI spend route by route: which model handles which request, where retries and redundant calls pile up, and where a cheaper model produces the same output at the same quality bar. That is the same analysis that found a 35% projected cut at one company and took another’s bill from $100K to $8K a month, checked against an evaluation suite so the savings do not cost you quality.
Do I need this if we already have SOC 2?
SOC 2 tests your controls at a point in time. It does not tell you whether your AI’s output quality has drifted, what one user costs you, or who owns the system when it fails. AI systems do not crash, they rot quietly, and that gap matters most when an AI output has real regulatory consequences.
Who is this for?
B2B SaaS teams, 50 to 500 people, Series A to C, in healthcare, fintech, insurance, legal, or accounting. You already shipped an AI feature, your monthly AI bill has five figures in it, and you need the system reliable, affordable, and defensible. Not for teams still deciding whether to build AI — this is for the ones who built it and now have to live with it.
What happens after the audit?
You keep the written plan either way, no obligation to continue. If you want it run for you, the retainer picks up where the audit leaves off: everything above, running every week, for $15K to $20K a month, with the audit fee credited in full toward it.
The cost is climbing. The quality is unproven. And you are the one who will be asked to explain it.
Twenty minutes. We look at your production AI together and you leave with a straight answer on where you stand.
Not ready for a call? Email me — serhii.panchyshyn@animanovalabs.com