AI Ultimate AI Engineer 5

Question 1

What is 'prompt injection' in the context of deployed LLM applications?

Accepted Answer

Malicious input that overrides system instructions to hijack model behavior

Answer

Prompt injection attacks embed adversarial instructions in user input that override the developer's system prompt, causing the model to follow attacker directives.

Question 2

Which evaluation framework is specifically designed to assess LLM applications end-to-end using LLM-as-a-judge scoring?

Accepted Answer

RAGAS for RAG pipelines or LangSmith evals

Answer

RAGAS and LangSmith provide LLM-as-judge metrics like faithfulness, answer relevancy, and context precision tailored to generative AI pipelines.

Question 3

In the context of AI engineering, what does 'guardrails' refer to?

Accepted Answer

Input/output validation layers that enforce safety and policy constraints around LLM calls

Answer

Guardrails are programmatic checks (classifiers, rules, or another LLM) applied to inputs and outputs to detect and block policy violations.

Question 4

What is the primary benefit of using structured output (e.g., JSON mode) when calling an LLM in a production pipeline?

Accepted Answer

It guarantees the model's output conforms to a parseable schema, removing brittle regex parsing

Answer

Structured output constrains the model's decoding to valid JSON (or another schema), making downstream parsing reliable and eliminating format-related failures.

Question 5

Which strategy best handles a user query that exceeds an LLM's context window in a document QA system?

Accepted Answer

Use hierarchical summarization or a retrieval step to select only relevant chunks

Answer

Hierarchical summarization or retrieval-based selection ensures the most relevant content fits in the window without losing critical information through naive truncation.

AI - Engineer Practice Test

AI - Engineer Practice Test

AI Ultimate AI Engineer 5