Hi, Here are some example questions for you to practise on, I used a LLM to generate them and could use a similar approach to generate some more. I'd also recommend looking at some of the exam prep content on udemy, there are a few courses on there with many example questions. That's what I did to prepare for my exam, it helped me to nail down the concepts I needed to understand better.
1) You’re building a customer-support RAG chatbot on Databricks. New PDFs arrive hourly to a Bronze Delta table. You need low-latency retrieval with up-to-the-minute content. What’s the best architecture? A) Nightly batch embed PDFs to a Delta table and query with LIKE filters B) Stream Bronze → Silver, chunk + embed in a streaming job, sync to a Vector Search index, and query via the index C) Write all chunks to Parquet and use approximate nearest neighbor in a Python UDF D) Use MLflow to store embeddings and query with Delta ZORDER
2) Your LLM endpoint experiences sudden 5–10x traffic spikes during product launches, causing timeouts. You want to keep costs modest during normal traffic and scale up automatically during spikes. What should you prioritize? A) Disable autoscaling to avoid scale-up delays B) Use serverless Model Serving with autoscaling and warm pool settings tuned to expected bursts C) Run a single large GPU node 24/7 to avoid cold starts D) Move to batch inference for all traffic
3) You fine-tuned an instruction model for your internal knowledge base, but it often fabricates answers. You want to reduce hallucinations without another fine-tune. What’s the best next step? A) Introduce retrieval-augmented generation using a curated Vector Search index and include citations in the prompt B) Increase temperature to encourage more diverse reasoning C) Switch to a larger model only D) Remove system prompts to avoid bias
4) You’re onboarding a multi-tenant RAG app across business units with strict data separation. What’s the primary control to enforce isolation of embeddings and source documents? A) Model endpoint tokens B) Unity Catalog permissions on source tables and vector indexes, + service principals scoped per tenant C) Notebook ACLs only D) Row-level security in notebooks via Python conditionals
5) You notice high retrieval latency from your Vector Search index. Chunks are 2,500 tokens and documents contain mixed topics. What is the most impactful remediation? A) Increase chunk size further to reduce index size B) Introduce semantic chunking with smaller, coherent chunks and add metadata filters for doc_type and product C) Remove metadata to simplify the index D) Use only keyword search because vector search is slower by default
6) A product manager requests “Why did the model choose this answer?” for every chat response in production. You also need to compare retriever performance over time. What should you implement first? A) Only prompt logs in MLflow B) Model Serving request/response logging with retrieved contexts, and offline evaluations on retrieval quality (e.g., MRR/Recall@k) with versioned datasets C) A/B test two model sizes without logging D) Disable logging due to PII concerns
7) Your RAG system sometimes returns irrelevant context due to ambiguous queries. You want to improve retrieval without changing the model. What should you try? A) Hybrid retrieval combining vector similarity with BM25 or metadata filters B) Increase temperature to encourage variety C) Remove metadata to reduce conflicts D) Use only embeddings trained on code
😎 You’re migrating from a prototype to production. The team wants reproducible experiments, prompt versioning, and offline evaluations over a fixed test set. Which combination fits best? A) Store prompts and results as notebook markdown B) Track prompts, parameters, and metrics with MLflow, and run eval notebooks regularly against a Unity Catalog curated dataset C) Save everything in CSVs on DBFS D) Add comments to the model endpoint
9) A finance team needs a scheduled job to generate structured summaries (JSON) from new transactions daily. The priority is consistent schema and downstream parsing, not chat. What is the best approach? A) Use batch inference with a structured output schema and store results in a Delta table B) Use interactive chat endpoints with human supervision C) Log raw model text to a JSON column and parse downstream D) Use notebooks only, without any model serving
10) Your endpoint costs have doubled, and CPU utilization is low while GPU is medium. Most prompts are long, with repeated instructions. What should you optimize first? A) Compress or template the system prompt; leverage prompt templates and caching where feasible B) Scale up GPUs to reduce latency C) Increase context window size D) Disable autoscaling and run a fixed large cluster
11) You added tool/function-calling to let the model query an internal REST API for order status. Sometimes the LLM hallucinates tool parameters. How can you improve reliability? A) Allow free-form text for tool arguments B) Provide JSON schemas for tool inputs and validate before execution; include few-shot examples of correct tool usage C) Increase temperature D) Remove function calling and hardcode API calls
12) Your legal team requires removal of sensitive PII in prompts and model outputs. You want minimal developer friction. What should you deploy? A) Manual developer checklist B) Pre/post-processing policies that redact PII at the gateway or serving layer, with audit logs C) Remove logs entirely D) Ask users not to enter PII
13) You notice retrieval quality degrades after frequent schema changes in source tables. Some embeddings don’t match expected vector dimensions. What’s the most robust fix? A) Re-embed only new rows B) Enforce a contract for embedding model + vector dimension; store model metadata alongside vectors; rebuild index when changing model C) Convert vectors to the new dimension by zero-padding D) Switch to keyword-only search
14) You’re asked to launch an A/B test of two prompts for the same endpoint to reduce hallucinations. You need traffic splitting and win-rate measurement. What’s the best plan? A) Manually alternate between prompts in the notebook B) Use a gateway or routing layer to split traffic between prompt variants, and log outcomes for statistical comparison C) Launch both in separate workspaces and compare logs by hand D) Switch models instead of prompts
15) Your chatbot’s first-token latency is high after periods of inactivity. You cannot afford constant overprovisioning. What should you try? A) Increase autoscaling cooldown to scale down faster B) Configure a minimum number of warm instances and adjust scale-to-zero behavior for expected idle windows C) Use larger models to produce tokens faster D) Disable logging
16) A stakeholder wants the bot to answer only from approved sources and refuse otherwise, with a clear “I don’t know” when evidence is weak. What’s the best approach? A) Lower temperature and hope for the best B) Retrieval gating: require a minimum relevance threshold and include a refusal policy in the system prompt C) Increase top_p for diversity D) Use only embeddings without prompts
17) You must support multilingual queries on an English corpus and return English answers with citations. What is the safest approach? A) Translate corpus to all possible languages B) Use multilingual embeddings for retrieval, translate the query to English if needed, and instruct the model to answer in English with citations C) Force user to ask in English D) Use monolingual embeddings and increase k
18) You’re backfilling embeddings for 50M documents. Index build speed is too slow and blocking launch. What will help most? A) Single-threaded local job B) Distributed embedding generation using Spark, write to Delta in batches, and build the vector index incrementally with parallelism C) Embed on the serving endpoint D) Switch to a larger model first
19) The team wants to evaluate end-to-end task success (not just BLEU/ROUGE) for a claims-processing agent that calls multiple tools. What should you implement? A) Only measure token counts B) Task-level success metrics with golden tasks, plus step-level traces for tool calls and failures C) Per-token probabilities D) Context window utilization
20) After adding richer context, the model sometimes exceeds token limits and truncates answers. What’s the best immediate mitigation? A) Increase temperature B) Apply retrieval budget: limit number/size of chunks by dynamic relevance, compress or summarize context before generation C) Use a smaller model D) Remove system prompts
21) You’ve deployed a content-generation job that runs nightly with stable load, and latency is not critical. How can you reduce costs? A) Switch to batch inference on scheduled jobs and right-size compute to cheaper instances B) Force serverless real-time endpoints C) Always keep two warm GPUs D) Add more replicas to reduce duration
22) Your RAG answers include stale prices for SKUs that change daily. You already re-embed content nightly. What else should you do? A) Add a tool/function call to fetch live pricing for cited SKUs and instruct the model to prioritize tool data over retrieved chunks B) Increase vector dimension C) Disable retrieval and use the tool only D) Reduce k in retrieval
23) The security team wants visibility into who called which model, with what data classes, and when. What should you enable? A) Random sampling of prompts in notebooks B) Centralized access logs and lineage across data sources and serving endpoints, with Unity Catalog tags for sensitive data C) Save logs to a local file D) Disable access to reduce risk
24) Your PDF-heavy corpus includes long tables that are poorly parsed into text, hurting retrieval quality. What’s the best path? A) Ignore tables and rely on the model B) Add a table-aware extraction step that preserves structure; store both text and structured table data with metadata for retrieval C) Increase chunk size to include entire tables in each chunk D) Use only OCR text
25) You need to roll out a new model version but want a safe migration with minimal risk to production users. What should you do? A) Replace the model immediately B) Run shadow or canary traffic for the new version, monitor metrics and feedback, then gradually increase traffic C) Force all users to test in dev first D) Disable logging during rollout
Good luck with your exam!
