Skip to main content

When to Override the AI: Patterns the Machine Gets Wrong

· 4 min read
Gabriel Paunescu
Founder CTO Neologic

AI coding assistants are remarkably good at following patterns. They're remarkably bad at knowing when a pattern doesn't apply. Here are three categories where you should always take the wheel.

The Premise​

AI generates code by pattern matching against its training data and your project's skill files. For 80% of Logic Bee hooks, this produces correct, convention-following code. But certain categories of logic require reasoning that current AI models consistently get wrong — not because they're broken, but because the problems require domain knowledge that never appears in code patterns alone.

The Story​

A senior developer tracked every AI-generated hook over three months and categorized the corrections. Three patterns emerged: financial calculations, multi-tenant edge cases, and transaction session management. Each required the same type of intervention — the developer understanding why, not just how.

Pattern 1: Financial Calculations​

What the AI Does​

AI uses JavaScript arithmetic because it looks correct:

// ❌ AI-generated: floating point disaster
const taxAmount = subtotal * taxRate
const total = subtotal + taxAmount + shippingCost

The problem isn't obvious until you see: 199.99 * 0.08 = 15.999200000000001.

What the Developer Does​

// ✅ mathChain for precision
const taxAmount = naoUtils.mathChain(subtotal).multiply(taxRate).done()
const total = naoUtils.mathChain(subtotal)
.add(taxAmount)
.add(shippingCost)
.done()

Why AI Gets This Wrong​

AI sees * and + operators in countless training examples. mathChain() is Logic Bee-specific. Even with skill files telling the AI to use mathChain, it sometimes falls back to raw operators for "simple" calculations — but there's no such thing as a simple financial calculation.

Rule: Any arithmetic involving currency → naoUtils.mathChain(). No exceptions.

Pattern 2: Multi-Tenant Edge Cases​

What the AI Does​

AI handles the happy path correctly — querying documents with .user(bob.flowUser). But it fails on edge cases where tenant boundaries interact in unexpected ways:

// ❌ AI-generated: cross-tenant data leak in batch processing
const allVendors = await vendorCollection.docs.flowQuery()
.user(bob.flowUser)
.flowOptions(eventOptions.vendorNaoQueryOptions)
.query({ 'data.type': 'preferred' })
.getMany(undefined, 'PurchasingInterface.Vendor')

// AI then uses vendor IDs to query another collection
// But what if a vendor was shared across tenants?
const vendorBills = await billCollection.docs.flowQuery()
.user(bob.flowUser)
.flowOptions(eventOptions.billNaoQueryOptions)
.query({ 'data.vendorId': { $in: allVendors.map(v => v.docId) } })
.getMany(undefined, 'FinanceInterface.Bill')

What the Developer Does​

The developer adds business-unit scoping:

// ✅ Add explicit business unit filtering
const vendorBills = await billCollection.docs.flowQuery()
.user(bob.flowUser)
.flowOptions(eventOptions.billNaoQueryOptions)
.query({
'data.vendorId': { $in: allVendors.map(v => v.docId) },
'data.businessUnitId': bob.flowUser.getBusinessUnitId() // explicit scope
})
.getMany(undefined, 'FinanceInterface.Bill')

Why AI Gets This Wrong​

Multi-tenancy is an architectural concern, not a code pattern. The AI knows to add .user(bob.flowUser) because the skill file says so. But it doesn't understand why — that in a shared-vendor scenario, FlowQuery's default scoping may not be sufficient and additional business-unit filtering is needed.

Rule: Any cross-collection join → verify tenant and business-unit boundaries manually.

Pattern 3: Complex Transaction Sessions​

What the AI Does​

AI includes bob.dbSession() on write operations because the skill file says so. But it doesn't understand transaction design:

// ❌ AI-generated: partial transaction safety
try {
// Operation 1: create credit memo
await creditMemoCollection.docs.flowQuery()
.user(bob.flowUser)
.flowOptions(eventOptions.creditMemoNaoQueryOptions)
.insertOne(creditMemoDoc, bob.dbSession())

// Operation 2: update invoice status
await invoiceCollection.docs.flowQuery()
.user(bob.flowUser)
.flowOptions(eventOptions.invoiceNaoQueryOptions)
.docId(invoiceId)
.updateOne({ 'data.status': 'credited' }, bob.dbSession())

// Operation 3: adjust vendor balance (NO SESSION!)
await vendorCollection.docs.flowQuery()
.user(bob.flowUser)
.flowOptions(eventOptions.vendorNaoQueryOptions)
.docId(vendorId)
.updateOne({ 'data.balance': newBalance }) // ← missing bob.dbSession()
}

What the Developer Does​

The developer ensures all writes within a logical transaction share the session — and adds rollback awareness:

// ✅ All writes in the same session, with failure awareness
const session = bob.dbSession()

await creditMemoCollection.docs.flowQuery()...insertOne(creditMemoDoc, session)
await invoiceCollection.docs.flowQuery()...updateOne({ 'data.status': 'credited' }, session)
await vendorCollection.docs.flowQuery()...updateOne({ 'data.balance': newBalance }, session)

Why AI Gets This Wrong​

The AI treats bob.dbSession() as a "should include" parameter, not a "must include for atomicity" requirement. It doesn't reason about what happens when operation 3 fails — operation 1 and 2 commit, but the vendor balance is stale.

Rule: If you're touching more than one collection → every write operation must use bob.dbSession(). Audit this manually every time.

The Decision Framework​

SituationAI or Developer?
Hook scaffolding and boilerplate✅ AI
Single-collection CRUD operations✅ AI
Standard FlowQuery chains✅ AI (with review)
Financial arithmetic❌ Developer
Cross-collection joins with tenant concerns❌ Developer
Multi-document transaction design❌ Developer
Error message wording❌ Developer
Schema-specific validation rules❌ Developer
Performance optimization (batching, indexing)❌ Developer

The Takeaway​

The AI is your fastest pair programmer — for patterns it has seen before. For patterns that require understanding, not just matching, the developer must intervene.

Three questions to ask about every AI-generated hook:

  1. Is there money involved? → Verify all math uses mathChain()
  2. Are there cross-collection queries? → Verify tenant and business-unit scoping
  3. Are there multiple writes? → Verify every one uses bob.dbSession()

Know when to let the AI drive. Know when to take the wheel.