F1지표(F1 score) 예시 2

 

F1지표(F1 score) 예시 2

F1 score on multiclass classification

 

 

 

 

Step by Step

·         Recall - MODEL 1

·         Precision - MODEL 1

·         F1 score - MODEL 1

·         Recall - MODEL 2

·         Precision - MODEL 2

·         F1 score - MODEL 2

·         F1 score Comparison

 

 

 

 

MODEL 1

 

 

Recall - MODEL 1

MODEL 1	Predictions	output	output	output	output	Predict data
Actual class		A	B	C	D	total
input	A	110	80	10	20	220
input	B	1	9	0	0	10
input	C	0	1	8	1	10
input	D	0	1	0	9	10
Actual data	total	111	91	18	30	250

 

 

index	TP	FP	Recall
A	110	110	110/220=0.5
B	9	1	9/10=0.9
C	8	2	8/10=0.8
D	9	1	9/10=0.9

 

 

A, B, C, D로 class가 4개 이므로 4로 나누어서 average Recall을 구합니다.

구분	MODEL 1
Average Recall	(0.5+0.9+0.8+0.9)/4=0.775

 

 

 

 

Precision - MODEL 1

MODEL 1	Predictions	output	output	output	output	Predict data
Actual class		A	B	C	D	total
input	A	110	80	10	20	220
input	B	1	9	0	0	10
input	C	0	1	8	1	10
input	D	0	1	0	9	10
Actual data	total	111	91	18	30	250

 

	A	B	C	D
TP	110	9	8	9
FP	1	82	10	21
Average Precision	110/111=0.991	9/91=0.0989	8/18=0.4444	9/30=0.3

 

 

A, B, C, D로 class가 4개 이므로 4로 나누어서 average precision을 구합니다.

구분	MODEL 1
Average Recall	(0.5+0.9+0.8+0.9)/4=0.775
Average Precision	(0.991+0.0989+0.4444+0.3)/4=0.4586

 

 

 

 

F1 score - MODEL 1

구분	MODEL 1
Average Recall	(0.5+0.9+0.8+0.9)/4=0.775
Average Precision	(0.991+0.0989+0.4444+0.3)/4=0.4586
F1 score	2x( (0.775x0.4586)/(0.775+0.4586) )=0.5762

 

 

 

 

 

 

 

MODEL 2

 

Recall - MODEL 2

MODEL 2	Predictions	output	output	output	output	Predict data
Actual class		A	B	C	D	total
input	A	217	2	0	1	220
input	B	7	1	0	2	10
input	C	7	1	1	1	10
input	D	2	4	3	1	10
Actual data	total	233	8	4	5	250

 

 

 

	A	B	C	D
TP	217	1	1	1
FP	3	9	9	9
Recall	217/220=0.9864	1/10=0.1	1/10=0.1	1/10=0.1

 

 

index	TP	FP	Recall
A	217	3	217/220=0.9864
B	1	9	1/10=0.1
C	1	9	1/10=0.1
D	1	9	1/10=0.1

 

 

A, B, C, D로 class가 4개 이므로 4로 나누어서 average Recall을 구합니다.

구분	MODEL 2
Average Recall	(0.9864+0.1+0.1+0.1)/4=0.3216

 

 

 

 

 

Precision - MODEL 2

MODEL 2	Predictions	output	output	output	output	Predict data
Actual class		A	B	C	D	total
input	A	217	2	0	1	220
input	B	7	1	0	2	10
input	C	7	1	1	1	10
input	D	2	4	3	1	10
Actual data	total	233	8	4	5	250

 

 

	A	B	C	D
TP	217	1	1	1
FP	16	7	3	4
Precision	217/233=0.9313	1/8=0.125	1/4=0.25	1/5=0.2

 

 

A, B, C, D로 class가 4개 이므로 4로 나누어서 average Recall을 구합니다.

구분	MODEL 2
Average Recall	(0.9864+0.1+0.1+0.1)/4=0.3216
Average Precision	(0.9313+0.125+0.25 +0.2 )/4=0.3766

 

 

 

 

F1 score - MODEL 1

구분	MODEL 2
Average Recall	(0.9864+0.1+0.1+0.1)/4=0.3216
Average Precision	(0.9313+0.125+0.25 +0.2 )/4=0.3766
F1 score	2x( (0.3216 x0.3766 )/(0.3216+0.3766 ) )=0.3469

 

 

 

 

 

 

 

 

F1 score Comparison

average precision/average recall 통해 구한 F1 score를 구해 보면

MODEL1은 0.5762 이고 MODEL2는 0.3469 으로 나와

MODEL1 > MODEL2보다 큰 것을 확인할 수 있습니다.

각 측정항목이 동일한 크기의 dataset을 가지지 못한 imbalanced data에서 F1 score를 계산했을때 MODEL1 이 MODEL2보다 더 좋다라고 나왔습니다.

imbalanced data에서 Accuracy를 볼 경우에는 MODEL2가 MODEL1보다 Accuracy가 더 커서 더 좋은 모델이라고 나왔습니다.

 

index	MODEL1	MODEL2
Accuracy	0.544	0.88
F1 score	0.5762	0.3469

 

Model 1 predicts well on multiple class classification on imbalanced given data, and F1 score is the metric to quantify its performance.