Questions
How do you build AI roadmap
- How do you prioritise AI hypotheses?
- ML development steps and specifics in each of them
- How do you choose architecture?
- Embeddings / RAG / Finetune, when choose each of them
- Model choice (accuracy, cost, latency, long context)
- What are the biggest risks in LLM products?
- How do you fight hallucinations?
- How do you fight model drift?
- What are major leverages to achieve good quality of the product?
- Metrics? Post-launch metrics?
- How do you choose architecture?
- When do you decide model is ready for launch?
- How do you make sure model is usable in real human-driven processes?
How do you personally use LLMs as a professional?
- What is your experience in prompt engineering?
Criteria
1) Project lifecycle understanding
2) AI Architecture fluency
3) Evaluation process & risks mitigation
4) Process-centric design (holistic view - how does model impact other processes and areas of organisation)