← Back to categories

Data Reality

Assumptions That Break

7 questions

Multiple Choice

Stationarity in Time Series

Data Independence Assumption

Feature Distribution Shift

Normality Assumption Violations

Open-Ended

When the IID Assumption Breaks

Distribution Shift Between Training and Production

Correlated Samples and Grouped Data

Sampling & Selection Bias

7 questions

Multiple Choice

Convenience Sampling Pitfalls

Survivorship Bias in Analytics

Non-Response Bias

Selection Bias in Training Data

Open-Ended

How Data Collection Skews Your Model

Missing the Failures You Never See

Self-Selection and Non-Random Sampling

Leakage & Contamination

7 questions

Multiple Choice

Feature Leakage Detection

Preprocessing Leakage

Entity-Level Leakage

Subtle Target Leakage

Open-Ended

User and Entity Leakage in Train/Test Splits

Features That Leak Future Information

When Targets Contaminate Features

Labels Are Not Ground Truth

7 questions

Multiple Choice

Delayed Labels

Proxy Labels Problem

Labeler Bias

Label Noise Impact

Open-Ended

When Labels Approximate the True Target

Handling Label Noise and Disagreement

Systematic Errors in Labeling Processes

Missingness & Data Gaps

7 questions

Multiple Choice

Imputation Dangers

Types of Missing Data

Structural vs Random Missing Data

Informative Missingness

Open-Ended

Types of Missingness: MCAR, MAR, and MNAR

When Missingness Is Informative

Pitfalls of Filling in Missing Data

Time & Non-Stationarity

7 questions

Multiple Choice

Proper Temporal Validation

Seasonality in Model Training

Concept Drift Detection

Trend vs Seasonality

Open-Ended

Why Random Splits Fail for Temporal Data

When Patterns Change Over Time

Handling Seasonality and Trends

Granularity & Aggregation Effects

7 questions

Multiple Choice

Aggregation Hides Segments

Ecological Fallacy

Temporal Aggregation

Simpson's Paradox

Open-Ended

Simpson's Paradox and Aggregation Traps

The Ecological Fallacy: Group vs Individual Inference

Temporal Aggregation and Lost Signals

Correlation ≠ Causation

7 questions

Multiple Choice

Confounding Variables

Reverse Causality

Spurious Correlations

Collider Bias

Open-Ended

Spurious Correlations and Confounding

When Prediction Isn't Enough: Causal Questions

Confounders, Colliders, and Causal Diagrams

Data Size Illusions

7 questions

Multiple Choice

Diminishing Returns

Big Data Fallacy

Small Data Validity

Class Imbalance vs Data Size

Open-Ended

Big Data Doesn't Fix Bad Data

Diminishing Returns and Data Efficiency

Valid Inference from Small Datasets

Production Reality

7 questions

Multiple Choice

Feedback Loops

Train-Serve Skew

Data Quality Monitoring

Silent Model Failures

Open-Ended

Monitoring Data Quality in Production

Train-Serve Skew: When Training Doesn't Match Production

Feedback Loops and Self-Fulfilling Predictions