Developing ML solutionsFull notesSummaryRingkasanStoriesPracticeMachine Learning Models Performance Evaluation12 exam-style questions on this lesson.Question 1 of 12Which dataset is used to check generalization while the team is still improving the model?ATest setBProduction dataCValidation setDTraining setCheck answerQuestion 2 of 12On the bullseye analogy, the shots are tightly grouped but far from the center. What does that indicate?AHigh bias, high varianceBLow bias, low varianceCLow bias, high varianceDHigh bias, low varianceCheck answerQuestion 3 of 12A spam filter sends important customer emails to the spam folder, and the business says this is very costly. Which metric should it maximize?APrecisionBMean squared errorCAccuracy on imbalanced dataDRecallCheck answerQuestion 4 of 12A medical screening model must catch as many patients with a serious disease as possible. Missing a sick patient is the worst outcome. Which metric should it maximize?APrecisionBRecallCR squaredDSpecificity onlyCheck answerQuestion 5 of 12A fraud dataset has 99.5% legitimate transactions. A model that labels everything "legitimate" gets 99.5% accuracy. What does this show?AThe model is excellent and ready for productionBRegression metrics such as MSE are neededCAccuracy is misleading here; use precision, recall, or F1DAccuracy is the best metric for rare fraud casesCheck answerQuestion 6 of 12A team needs one metric that balances precision and recall. Which metric fits?AAUC-ROCBMean squared errorCR squaredDF1 scoreCheck answerQuestion 7 of 12A team wants to compare classification models across all decision thresholds and choose a threshold. Which metric fits?AAUC-ROCBBLEUCMean squared errorDR squaredCheck answerQuestion 8 of 12What is the formula for recall?ATN / (TN + FP)BTP / (TP + FN)CTP / (TP + FP)D(TP + TN) / all predictionsCheck answerQuestion 9 of 12. Select two.Which TWO metrics are used to evaluate regression models? (Select TWO.)AR squaredBPrecisionCMean squared error (MSE)DF1 scoreERecallCheck answer0 of 2 selectedQuestion 10 of 12A regression model has an R squared of 0.92. What does that mean?AThe model is wrong on 92% of predictionsBThe average prediction error is 0.92 unitsCThe model explains about 92% of the variance in the targetDThe model has 92% precision on the test setCheck answerQuestion 11 of 12. Select two.A company wants to compare a new model against the current one using live production traffic. Which TWO approaches fit? (Select TWO.)AA/B testingBDimensionality reductionCData augmentationDCanary deploymentECross-validation on training dataCheck answer0 of 2 selectedQuestion 12 of 12A model has great precision and recall, but the business wants to know if it's worth the investment. What should the team do?ARetrain on the test set to raise the scoresBSwitch to a larger model with better metricsCReport only accuracy, since executives know itDTie model metrics to the business KPIs and the cost of each errorCheck answerSources of ML Models7 exam-style questions on this lesson.Model Deployment8 exam-style questions on this lesson.