Skip to main content
CloudBit Logo
Data and AI

Generative AI: what changes between the pilot and production

The demo works in an afternoon; taking it to production requires curated data, permissions, evaluation and cost control.

· 7 min read · CloudBit team

The pilot tests the idea, not the solution

An assistant that answers well over ten hand-picked documents guarantees nothing about ten thousand real documents, with old versions, duplicates and different permissions.

Permissions by design

The assistant should only answer with information the user can already see. That means carrying permissions from the source systems into the search index, not adding them at the end.

Measure answer quality

A set of test questions with expected answers lets you catch regressions every time the model, the prompt or the data changes.

Control the cost per query

Choosing the model based on question complexity and caching frequent answers can cut the monthly cost significantly without hurting the experience.

Frequently asked questions

Want to apply this at your company?

Book a free 30-minute technical consultation.

Talk to a specialist