Quality Engineering Center of Excellence AI Best Practices
Prompt engineering frameworks, techniques, and best practices for AI-augmented Quality Assurance.
The Five Pillars of Prompt Excellence
C
Clarity
Use precise, unambiguous language without unclear references.
C
Context
Provide tech stack, framework conventions, business rules.
C
Constraints
Define what should NOT happen as much as what should.
C
Completeness
Include API specs, schemas, sample responses, test files.
C
Correctability
Structure for easy verification. Request step-by-step reasoning.
Prompt Mnemonics Frameworks
Feature
RACE (General & Agile)
TRACE (Audience-Centric)
CREATE (Detailed & Complex)
TRICE+ (QA-Optimized)
Best For
Fast, everyday tasks like emails or summaries.
Educational content or technical documentation for specific groups.
Complex projects requiring nuanced reasoning and strict output formats.
Data-driven QA tasks requiring concrete artifacts and dynamic test data.
Key Strength
Minimalist and quick to implement.
High precision and tailored tone for a specific audience.
Comprehensive coverage with built-in evaluation.
Built-in data strategy for scalable, isolated test generation.
QA Application
Quick test case drafts, bug summaries, sprint reports.
Test documentation for devs vs. stakeholders, training materials.
Test framework design, automation architecture, complex test plans.
API testing, data-driven automation, test data generation, CI/CD integration.
RACE Agile Standard
Designed for speed and clarity. The "go-to" for quick, solid responses.
RRole: Who should the AI be? (e.g., "Expert QA Engineer")
AAction: What should it do? (e.g., "Write test cases")
CContext: What is the background? (e.g., "For a REST API endpoint")
EExecute: What format or goal? (e.g., "Gherkin format, cover edge cases")
TRACE Targeted
Adds Audience to ensure the AI speaks to the right people.
TTask: The job to be done.
RRole: The AI's persona.
AAudience: Who is reading this? (e.g., "Junior QA" vs. "Tech Lead")
CContext: Relevant background details.
EExpectation: Desired output format.
CREATE Comprehensive
A "deep-dive" framework for complex instructions requiring reasoning.
CCharacter: Detailed persona description.
RRequest: Clear, specific task.
EExamples: Real-world samples to guide style.
AAdditions: Refining with constraints or POV.
TType of Output: Exact format and structure.
EEvaluate: Criteria to ensure quality.
TRICE+ QA-Optimized
Designed for QA with built-in data strategy for scalable, isolated tests.
TTask: Define the action precisely. (e.g., "Generate API test cases")
RRole: Assign a persona. (e.g., "Senior SDET with REST expertise")
IInput: Provide concrete artifacts. (e.g., OpenAPI spec, schemas, examples)
CConstraints: Define boundaries. (e.g., "No hardcoded IDs, follow RBAC rules")
EExpectation: Define output format. (e.g., "Gherkin scenarios with data tables")
+Data Strategy: Dynamic data approach. (e.g., "Use faker for emails, UUIDs for IDs")
Prompting Techniques for QA
1
Zero-Shot Prompting
Direct instruction with no examples provided. The model relies entirely on its pre-trained knowledge to complete the task. Best for straightforward requests where the desired output format is clear and commonly understood. Works well when the task aligns with standard patterns the model has seen during training.
QA Use: Manual test case design for functional testing
User Prompt
Task: Create functional test cases for user registration
Feature: User Registration Form
Fields: Email, Password, Confirm Password, First Name, Last Name
Business Rules:
- Email must be unique in the system
- Password minimum 8 characters with 1 uppercase, 1 number
- All fields are required
Output: Generate 5 test cases in Given/When/Then format covering positive and negative scenarios
Expected LLM Output
TC001: Successful Registration
Given I am on the registration page
When I enter valid email "john@example.com"
And I enter password "SecurePass1" and confirm it
And I fill First Name "John" and Last Name "Doe"
And I click Register
Then I should see "Registration successful" message
TC002: Duplicate Email Rejection
Given "existing@test.com" is already registered
When I attempt to register with "existing@test.com"
Then I should see "Email already exists" error
TC003: Password Validation - Missing Uppercase
Given I am on the registration page
When I enter password "lowercase1"
Then I should see "Password must contain uppercase" error
TC004: Required Field Validation
Given I am on the registration page
When I leave First Name empty and click Register
Then I should see "First Name is required" error
TC005: Password Mismatch
Given I enter password "SecurePass1"
When I enter confirm password "DifferentPass1"
Then I should see "Passwords do not match" error
2
Few-Shot Prompting
Provide 2-3 examples of desired input-output patterns before the actual task. This technique teaches the model your specific format, naming conventions, and style preferences. Particularly valuable for maintaining consistency across team artifacts and ensuring outputs match existing project standards.
QA Use: API automation test generation following team standards
User Prompt
Context: Our team uses SuperTest with Jest. Follow these examples exactly.
Example 1:
describe('POST /api/login', () => {
it('should return 200 and token for valid credentials', async () => {
const response = await request(app)
.post('/api/login')
.send({ email: 'test@example.com', password: 'valid123' });
expect(response.status).toBe(200);
expect(response.body.token).toBeDefined();
});
});
Example 2:
describe('GET /api/users/:id', () => {
it('should return 404 for non-existent user', async () => {
const response = await request(app)
.get('/api/users/99999');
expect(response.status).toBe(404);
expect(response.body.message).toBe('User not found');
});
});
Task: Generate 3 tests for PUT /api/products/:id endpoint (update product)
Instruct the model to think step-by-step through intermediate logical steps before arriving at a conclusion. This technique improves reasoning accuracy for complex problems by making the thinking process explicit. Essential for root cause analysis, impact assessment, and multi-variable debugging scenarios.
QA Use: Bug re-test verification and regression analysis
User Prompt
Bug ID: BUG-1234
Original Issue: Shopping cart total shows incorrect value when applying discount code
Root Cause: Percentage calculation was applied before tax instead of after
Fix Applied: Updated calculateTotal() to apply discount after tax calculation
Task: Think step-by-step about:
1. What scenarios need re-testing
2. What regression areas might be affected
3. Generate a verification test plan
Expected LLM Output
Step 1: Analyzing Original Bug Impact
- The bug affected price calculation order
- Users saw incorrect totals (lower than expected)
- Financial accuracy was compromised
Step 2: Re-test Scenarios Required
1. Apply 10% discount code → verify total = (subtotal + tax) - 10%
2. Apply fixed-amount discount → verify correct order
3. Apply multiple discounts → verify cumulative calculation
4. Zero discount code → verify no change to total
Step 3: Regression Risk Areas
- Invoice generation (uses same calculation)
- Order history displays
- Email receipt totals
- Analytics revenue reporting
Step 4: Verification Test Plan
| Test Case | Input | Expected Result |
|-----------|-------|-----------------|
| TC1 | $100 item + 8% tax + 10% discount | Total: $97.20 |
| TC2 | $50 item + 8% tax + $5 off | Total: $49.00 |
| TC3 | Edge: $0 discount code | No calculation change |
| TC4 | Verify invoice matches cart | Values identical |
4
Role Prompting
Assign a specific persona or expert role to the model such as Security Tester, Performance Engineer, or Accessibility Specialist. This focuses the response through the lens of domain expertise, surfacing specialized knowledge and best practices relevant to that role. Combines well with other techniques.
QA Use: Front-end automation with accessibility focus
User Prompt
Role: You are a Senior SDET specializing in accessibility automation testing with expertise in WCAG 2.1 guidelines.
Context: We have a React login form that needs accessibility testing.
Requirements:
- Use Playwright with @axe-core/playwright
- Check for WCAG 2.1 Level AA compliance
- Verify keyboard navigation works correctly
- Test screen reader compatibility
Task: Generate a comprehensive accessibility test suite for the login form
Use multi-turn conversations to progressively improve outputs. Start broad, then narrow down with specific feedback: add edge cases, improve formatting, enhance coverage. This technique mimics the natural review cycle and produces polished, production-ready artifacts through incremental enhancement.
QA Use: Exploratory testing session documentation
User Prompt
Turn 1: I'm testing a file upload feature. Help me create an exploratory testing charter.
Turn 2: Good. Now add specific test ideas for edge cases around file types and sizes.
Turn 3: Perfect. Format this as a session-based testing report with time boxes and findings sections.
Expected LLM Output
EXPLORATORY TESTING SESSION REPORTCharter: Explore file upload functionality focusing on boundary conditions and error handling
Time Box: 60 minutes
Tester: [Name]
Date: [Date]
TEST IDEAS EXPLORED:
□ Upload exactly at max size limit (10MB)
□ Upload 1 byte over limit (10MB + 1 byte)
□ Upload 0 byte empty file
□ Upload file with no extension
□ Upload .exe disguised as .jpg
□ Upload file with special characters in name
□ Upload during network interruption
□ Concurrent uploads (5 files simultaneously)
FINDINGS:
| ID | Severity | Description |
|----|----------|-------------|
| F1 | Medium | No validation for empty files |
| F2 | High | .exe uploads accepted |
| F3 | Low | Unicode filenames truncated |
COVERAGE:
- File types tested: 8/10
- Size boundaries: Complete
- Error messages: Partial
NOTES FOR FOLLOW-UP:
- Need to test with slow 3G network
- Verify virus scan integration
6
Self-Consistency Sampling
Request multiple reasoning paths to the same problem, then identify the consensus or best approach. This technique reduces errors by cross-validating different solution strategies. Useful for complex decisions where there's no single obvious answer, helping to find the most robust solution.
QA Use: Bug severity classification and prioritization
User Prompt
Bug Description:
Users cannot complete checkout when cart contains more than 15 items. The "Place Order" button becomes unresponsive.
Task: Classify this bug's severity using 3 different analysis approaches:
1. User Impact perspective
2. Business Revenue perspective
3. Technical Risk perspective
Then provide a consensus severity rating with justification.
Expected LLM Output
APPROACH 1: User Impact Analysis
- Affected users: ~5% (those with 15+ items)
- Workaround exists: Split into multiple orders
- Frustration level: High
- Severity: MEDIUMAPPROACH 2: Business Revenue Analysis
- Large orders = high-value customers
- Average order >15 items: $450
- Potential daily revenue loss: $2,700
- Customer churn risk: Moderate
- Severity: HIGHAPPROACH 3: Technical Risk Analysis
- Root cause: Likely JavaScript memory issue
- Could indicate deeper performance problems
- May worsen over time
- Severity: MEDIUM-HIGHCONSENSUS SEVERITY: HIGHJustification: 2 of 3 approaches indicate HIGH/MEDIUM-HIGH. The revenue impact on high-value customers and the technical debt risk outweigh the relatively small percentage of affected users. This bug blocks a critical business flow (checkout) with no seamless workaround.
Recommendation: Priority 1 - Fix in current sprint
7
Tree-of-Thought (ToT)
Explore multiple branching reasoning paths like a decision tree, evaluate each branch's viability, and prune unpromising ones. This technique enables systematic evaluation of alternatives before committing to a solution. Ideal for strategic decisions like framework selection or test architecture planning.
QA Use: Test automation framework evaluation and selection
User Prompt
Context: We need to choose a mobile testing framework.
Requirements: iOS + Android, CI/CD integration, team knows JavaScript
Task: Evaluate these 3 options using Tree-of-Thought:
1. Appium
2. Detox
3. WebDriverIO with Appium
For each branch, analyze: Learning curve, CI integration, Cross-platform support, Community support
Then recommend the best option with justification.
Expected LLM Output
BRANCH 1: Appium
├── Learning Curve: Moderate (familiar WebDriver API)
├── CI Integration: Excellent (all major CI tools)
├── Cross-platform: ✓ iOS + Android + Web
├── Community: Large, mature ecosystem
└── Score: 8/10BRANCH 2: Detox
├── Learning Curve: Easy (React Native focus)
├── CI Integration: Good (needs configuration)
├── Cross-platform: ⚠️ Best for React Native only
├── Community: Growing, RN-specific
└── Score: 6/10 (PRUNED - not universal)
BRANCH 3: WebDriverIO + Appium
├── Learning Curve: Easy (JS-based, good docs)
├── CI Integration: Excellent (built-in reporters)
├── Cross-platform: ✓ iOS + Android + Web
├── Community: Active, modern approach
└── Score: 9/10RECOMMENDATION: WebDriverIO + AppiumJustification:
- Team knows JS → minimal ramp-up
- Same framework for web + mobile
- Best-in-class CI/CD integration
- Active community with modern practices
8
ReAct (Reason + Act)
Interleave reasoning with action steps in an iterative loop: Think about the problem, take an action, observe the result, then repeat. This technique mirrors how skilled testers approach debugging and investigation. Excellent for troubleshooting flaky tests, investigating bugs, and exploratory analysis.
QA Use: Debugging flaky tests with systematic investigation
User Prompt
Problem: This API test passes locally but fails in CI 40% of the time:
test('should fetch user profile', async () => {
const response = await api.get('/users/profile');
expect(response.data.name).toBe('John Doe');
});
Task: Use ReAct pattern to diagnose and fix. Show your Think/Act/Observe loop.
Expected LLM Output
THINK: Flaky in CI suggests environment difference. Could be:
- Database state not reset
- Race condition with test data setup
- Network timeout differences
ACT: Check if test depends on pre-existing data
OBSERVE: Test assumes 'John Doe' exists but doesn't create it
THINK: CI runs tests in isolation/parallel. Data may not exist.
ACT: Add beforeEach to create test user
OBSERVE: Still fails sometimes
THINK: User creation might not complete before GET runs
ACT: Add proper await and verification
OBSERVE: ✓ Test now passes consistently
FIXED CODE:
test('should fetch user profile', async () => {
// Setup: Ensure user exists
await api.post('/users', { name: 'John Doe' });
// Add small delay for eventual consistency
await new Promise(r => setTimeout(r, 100));
const response = await api.get('/users/profile');
expect(response.data.name).toBe('John Doe');
});
ROOT CAUSE: Missing test isolation - test depended on global state
9
Meta-Prompting
Use the LLM to generate, improve, or optimize prompts themselves. This creates reusable templates that can be shared across teams, ensuring consistent high-quality outputs. Think of it as creating prompt libraries or test generation tools that multiply your productivity.
QA Use: Creating reusable test case generation templates
User Prompt
Task: Create a reusable prompt template that our QA team can use to generate test cases for any CRUD API endpoint.
Requirements:
- Template should have clear placeholders
- Include positive and negative scenarios
- Cover authentication and authorization
- Output in our standard format
Deliverable: The template plus one example of how to use it
Expected LLM Output
CRUD API TEST CASE TEMPLATE
---
Generate test cases for: {{ENDPOINT_NAME}}
Method: {{HTTP_METHOD}}
Authentication: {{AUTH_TYPE}}
Required Fields: {{REQUIRED_FIELDS}}
Business Rules: {{BUSINESS_RULES}}
Include test cases for:
1. Happy path with valid data
2. Missing required fields (one per field)
3. Invalid data types
4. Authentication failures (missing/invalid/expired token)
5. Authorization (wrong role accessing endpoint)
6. Boundary values for numeric/string fields
7. Duplicate data handling (if applicable)
Output Format: Test ID | Scenario | Expected Status | Expected Response
---
EXAMPLE USAGE:
Generate test cases for: POST /api/orders
Method: POST
Authentication: Bearer Token (JWT)
Required Fields: customerId, items[], shippingAddress
Business Rules: Minimum order $10, Max 50 items
Generated Output:
| TC001 | Valid order creation | 201 | Order ID returned |
| TC002 | Missing customerId | 400 | "customerId required" |
| TC003 | Empty items array | 400 | "At least 1 item required" |
| TC004 | No auth token | 401 | "Unauthorized" |
| TC005 | Expired token | 401 | "Token expired" |
10
Structured Output
Explicitly request responses in specific formats: JSON, YAML, CSV, Gherkin, Markdown tables, or custom schemas. This ensures outputs can be directly imported into test management tools, CI pipelines, or documentation systems. Critical for automation workflows where parsing and consistency matter.
QA Use: Generating test data in specific formats for automation
User Prompt
Task: Generate test data for e-commerce order testing
Output Format: JSON array that can be directly used in our test fixtures
Requirements:
- 5 order objects
- Include: orderId, customerId, items (array), total, status
- Mix of statuses: pending, shipped, delivered, cancelled
- Realistic product names and prices
- Properly formatted dates (ISO 8601)
Constraint: Output ONLY valid JSON, no explanations
"As a security engineer, think step-by-step about how an attacker might exploit this API endpoint." Combines specialized knowledge with rigorous analysis.
Few-Shot+Structured Output
Scalable Test Generation
Provide 2 example test cases in JSON format, then ask for 20 more in the same structure. Gets consistent, parseable output at scale.
Role+Self-Consistency
Unbiased Bug Classification
Ask the model to classify a bug's severity from three roles (developer, PM, end user), then take consensus. Reduces individual bias.
Meta+Iterative
Template Library Building
Generate a prompt template, test it, refine across iterations until it consistently produces high-quality test artifacts for your team.
RACE+Role
Quick Expert Reviews
Combine RACE structure with a specialist role for rapid, expert-level analysis. "As a performance tester (Role), analyze this query (Action) for our e-commerce DB (Context), list bottlenecks in bullets (Execute)."
CREATE+CoT
Framework Architecture
Use CREATE's comprehensive structure with explicit "think through each decision" instructions for designing test automation frameworks with justified choices.
Anti-Patterns to Avoid
Anti-Pattern
Problem
Correct Approach
"Write API tests"
Vague, no context
"Write Supertest assertions for /users POST validating 201 with JSON schema"
