NTJ-Python_Model_Evaluation

This repository introduces and summarizes common model evaluation indicators and methods.

The notes contain the modules outlined below:

No.	Module	Gist
1	Overview	1.1 What is Model Evaluation 1.2 Type of Model Evaluation 1.3 Over Fitting and Under Fitting 1.4 Model Generalization Ability
2	Classification Model	2.1 Confusion Matrix 2.2 Accuracy, Recall, Precision, etc. 2.3 F1 Score 2.4 AUC and its Index 2.5 PR Curve
3	Regression Model	3.1 Vector Distance 3.2 Mean Absolute Error (MAE) 3.3 Mean Square Error (MSE) 3.4 Root Mean Square Error (RMSE) 3.5 Explain Variation 3.6 Determination Coefficient
4	Clustering Model	4.1 Rand Index 4.2 Mutual Information 4.3 Profile Coefficient

Overview

1.1 What's mdoel evaluation

Classification Model

2.1 Confusion matrix

predicted	actual	values
⬇️	➕	➖
➕	TP (1−β)	FP (α) type I error
➖	FN (β) type II error	TN (1−α)

A type I error corresponds to convicting an innocent defendant; misdiagnosis

A type II error corresponds to acquitting a criminal; missed diagnosis

Table of error types

In inferential statistics, the null hypothesis (often denoted $H_0$)is that two (possibilities) are the same.
☝🏿 Hypothesis: "The patient has pneumonia."
👌🏿 Null hypothesis $H_0$ ✅: "The patient is healthy."

error types	$H_0$ ✅	$H_0$❌
🙇🏻‍♀️	TN(1−α)	FN (β) type II error
🙅‍♀️	FP (α) type I error	TP (1−β)

$H_0$ ✅

😁 The people is healthy, but the physician judges the patient was ill.

🙅‍♀️ A type I error is the mistaken rejection of a null hypothesis as the result of a test procedure.

$H_0$ ❌

🤧 The patient has symptoms of fever, cough, expectoration and chest pain, but the physician judges the patient was ok.

🙇🏻‍♀️ A type II error is the mistaken failure to reject the null hypothesis as the result of a test procedure.

2.2 Accuracy, Recall, Precision

term	formula	definition
accuracy
recall
precision