Building Got Easier. Understanding Was Never Optional.

Have you built something in a weekend that would have taken a quarter? 

A partner church had a list of priorities. Somewhere on it was an executive dashboard: adult and kids’ attendance, events, volunteer attendance, all in one view. It kept slipping down their list, the way any priority does when three other things are also priorities. So while we worked through what was in front of us, they didn’t wait. They vibe-coded the dashboard themselves. 

We opened it up, and it was good. They’d verified the numbers, checked them against what Rock actually had on file, and it worked. 

But there was a KPI on that dashboard for “active small group members,” and that’s the part that worried us. 

Active Wasn’t Defined 

What does “active” mean here? A group member with an active status? Someone who’s shown up for four of the last eight weeks? 

Take the latter. Someone attended four of the last eight weeks, and their record got archived last night because their leader moved them to a different group. The dashboard still counts them as active. The numbers are accurate. The record is real. The person building it just never had a reason to ask what “active” meant here, or to know that path exists in Rock. 

Verified isn’t the same as understood. 

You’ve probably seen a version of this already. Maybe not a dashboard, but a report or a workflow someone built fast to save everyone time. 

This Already Happened, and It Wasn’t AI 

The Data Views Problem

Rock had already produced its own version of this, years before anyone asked AI to build anything. You’ve probably run into a Data View already, Rock’s feature for saving a list of people or records that match criteria you set. A partner church gave every ministry staff member the ability to build their own, from attendance over 60 days to gifts toward a specific fund, whatever the staff person needed that day. The intentions were good: more self-serve, fewer requests sitting in a queue. 

Within a few months, everybody had their own version of “engaged.” Their own definition of “giving.” Reports that were each technically correct and quietly disagreed with each other. Leadership stopped trusting the numbers coming out of Rock. The problem wasn't Rock, it was the lack of a cohesive data strategy. 

This is a source-of-truth problem, and it’s the same one that follows you into how a team uses AI. It doesn’t stay contained to Data Views, either. 

It Happens Outside Church Tech, Too 

Airbnb ran into the same thing before they built a centralized metrics platform. Data Science and Finance would answer a question as simple as “which city had the most bookings last week” with two different numbers, pulled from slightly different tables using slightly different definitions. Confidence in the data fell. Trust from decision-makers went with it. The fix wasn’t a smarter tool. It was one place everyone agreed to pull from.   

Ask AI To Do The Same Work, and The Gap Shows Up Faster 

On Rock Cast episode 207, they used this as a go-to example for exactly where AI-written SQL breaks down: “give me all the accounts that someone has given to. It’s like, okay, which accounts are those? Which ones are considered giving?” 

It’s not limited to Rock, either. In 2023, two lawyers filed a legal brief full of fake case citations. ChatGPT had invented them, and they read like real court opinions. When the court pushed them to produce the actual cases, neither lawyer pulled them from a real legal database, the same tool every lawyer already has and is trained to use before filing anything. One of them told the judge, “I just never thought it could be made up.” The citations sounded finished, so he skipped the step that would have caught it. 

A tool gets faster. Trust in the output outruns anyone’s ability to check it. An experienced Rock Admin can spot the gap because they’ve spent years learning exactly where it lives. A staff member building fast on their own, without that fluency and without a reason to be suspicious, probably won’t even think to ask. 

The Risk That Costs You 

Multiply that first dashboard, the one built in a weekend, by every team and every well-meaning staff member doing the same thing, each with their own definitions and nobody stewarding any of it. 

Different data models per dashboard. Nobody left to fix it once the person who built it moves on. Definitions that quietly disagree with each other across ministries. 

That stops looking like a few people moving fast. It starts looking like reporting and tools nobody trusts: numbers that don’t match, dashboards that disagree with each other. If you’re the executive who must stand behind those numbers, that’s the cost that lands on you. 

This Isn’t An Argument Against Building 

We want to be clear about what we’re not saying. 

We’re not saying slow down. We’re not saying stop letting your team build. Shipping a dashboard in a weekend instead of waiting a quarter, verifying the numbers, getting more confident along the way, that’s a win. We don’t want to talk anyone out of it. 

Building without anyone owning the definitions underneath it is what causes the damage. AI didn’t create that risk. It just took away the last bit of friction that used to force somebody to ask the question first. 

The Fix

1. Get the right people in a room and turn the words into a source of truth. What counts as an active member? What counts as engaged? What counts as giving? Agree on one definition, not a different one for every ministry, and write it down somewhere everyone can find it. Call it a Ministry Dictionary: one shared list of terms and definitions that any dashboard, report, or AI-built dataset has to agree with. Then build the foundational data views and datasets that carry it, so anyone building on top of Rock, staff or AI, is building on something true.
2. Match the training to what’s at stake. Not everyone on your team needs to go through Rock’s Master Class, but a few people should, because someone needs to know what’s actually possible in Rock before the rest of the team builds around a gap that isn’t real. Sometimes the real gap isn’t Rock knowledge at all, it’s AI fluency. Triumph runs an AI Agentic Workflows Cohort for exactly that: a small group working through one real workflow together, with guidance and review along the way.
3. Test understanding, not just verification. Geoffrey Litt makes this distinction plainly: verification asks whether something works; understanding asks whether the person can explain it, extend it, and catch it the next time it’s wrong. Something that runs correctly still isn’t safe to build on top of if the person who made it can’t explain the decisions baked into it

Skip the order and you end up training people to navigate a mess, or handing them a foundation nobody knows how to use. 

Before You Hand Your Team, Or An Agent, More Speed 

Ask one question first: can the person who built it explain it, not just whether it ran? 

A build can be technically accurate and still wrong, because “wrong” isn’t always a bad number. Sometimes it’s a good number nobody agreed to. 

The building got easier. It was always going to. 

Understanding what you’re building on top of was never optional. It just got easier to skip.