There is a gigantic gap between the AI chatbot that works in a demo and one that can scale to 1000+ conversations per day without issues. I've worked on the production side of this gap and most of the work is not about the AI at all.
Demo vs Production gap
Demo requirements:
- An LLM call that returns a good response
- Nice UI
Production requirements: