F1지표(F1 score) 예시 2

CODEDRAGON Development/Big Data, R, ...

반응형



 

F1지표(F1 score) 예시 2

F1 score on multiclass classification

 

 

 

 

Step by Step

·         Recall - MODEL 1

·         Precision - MODEL 1

·         F1 score - MODEL 1

·         Recall - MODEL 2

·         Precision - MODEL 2

·         F1 score - MODEL 2

·         F1 score Comparison

 

 

 

 

MODEL 1

 

 

Recall - MODEL 1

MODEL 1

Predictions

output

output

output

output

Predict data

Actual class

 

A

B

C

D

total

input

A

110

80

10

20

220

input

B

1

9

0

0

10

input

C

0

1

8

1

10

input

D

0

1

0

9

10

Actual data

total

111

91

18

30

250

 

 

 

 

 

 

 

 

 

index

TP

FP

Recall

A

110

110

110/220=0.5

B

9

1

9/10=0.9

C

8

2

8/10=0.8

D

9

1

9/10=0.9

 

 

A, B, C, D class 4 이므로 4 나누어서 average Recall 구합니다.

구분

MODEL 1

Average Recall

(0.5+0.9+0.8+0.9)/4=0.775

 

 

 

 

Precision - MODEL 1

MODEL 1

Predictions

output

output

output

output

Predict data

Actual class

 

A

B

C

D

total

input

A

110

80

10

20

220

input

B

1

9

0

0

10

input

C

0

1

8

1

10

input

D

0

1

0

9

10

Actual data

total

111

91

18

30

250

 

 

 

 

 

 

 

 

 

A

B

C

D

TP

110

9

8

9

FP

1

82

10

21

Average Precision

110/111=0.991

9/91=0.0989

8/18=0.4444

9/30=0.3

 

 

A, B, C, D class 4 이므로 4 나누어서 average precision 구합니다.

구분

MODEL 1

Average Recall

(0.5+0.9+0.8+0.9)/4=0.775

Average Precision

(0.991+0.0989+0.4444+0.3)/4=0.4586

 

 

 

 

F1 score - MODEL 1

구분

MODEL 1

Average Recall

(0.5+0.9+0.8+0.9)/4=0.775

Average Precision

(0.991+0.0989+0.4444+0.3)/4=0.4586

F1 score

2x( (0.775x0.4586)/(0.775+0.4586) )=0.5762

 

 

 

 

 

 

 

MODEL 2

 

Recall - MODEL 2

MODEL 2

Predictions

output

output

output

output

Predict data

Actual class

 

A

B

C

D

total

input

A

217

2

0

1

220

input

B

7

1

0

2

10

input

C

7

1

1

1

10

input

D

2

4

3

1

10

Actual data

total

233

8

4

5

250

 

 

 

 

A

B

C

D

TP

217

1

1

1

FP

3

9

9

9

Recall

217/220=0.9864

1/10=0.1

1/10=0.1

1/10=0.1

 

 

index

TP

FP

Recall

A

217

3

217/220=0.9864

B

1

9

1/10=0.1

C

1

9

1/10=0.1

D

1

9

1/10=0.1

 

 

A, B, C, D class 4 이므로 4 나누어서 average Recall 구합니다.

구분

MODEL 2

Average Recall

(0.9864+0.1+0.1+0.1)/4=0.3216

 

 

 

 

 

Precision - MODEL 2

MODEL 2

Predictions

output

output

output

output

Predict data

Actual class

 

A

B

C

D

total

input

A

217

2

0

1

220

input

B

7

1

0

2

10

input

C

7

1

1

1

10

input

D

2

4

3

1

10

Actual data

total

233

8

4

5

250

 

 

 

A

B

C

D

TP

217

1

1

1

FP

16

7

3

4

Precision

217/233=0.9313

1/8=0.125

1/4=0.25

1/5=0.2

 

 

A, B, C, D class 4 이므로 4 나누어서 average Recall 구합니다.

구분

MODEL 2

Average Recall

(0.9864+0.1+0.1+0.1)/4=0.3216

Average Precision

(0.9313+0.125+0.25 +0.2 )/4=0.3766

 

 

 

 

F1 score - MODEL 1

구분

MODEL 2

Average Recall

(0.9864+0.1+0.1+0.1)/4=0.3216

Average Precision

(0.9313+0.125+0.25 +0.2 )/4=0.3766

F1 score

2x( (0.3216 x0.3766 )/(0.3216+0.3766 ) )=0.3469

 

 

 

 

 

 

 

 

F1 score Comparison

average precision/average recall 통해 구한 F1 score 구해 보면

MODEL1 0.5762 이고 MODEL2 0.3469 으로 나와

MODEL1 > MODEL2보다 것을 확인할 있습니다.

측정항목이 동일한 크기의 dataset 가지지 못한 imbalanced data에서 F1 score 계산했을때 MODEL1 MODEL2보다 좋다라고 나왔습니다.

imbalanced data에서 Accuracy 경우에는 MODEL2 MODEL1보다 Accuracy 커서 좋은 모델이라고 나왔습니다.

 

index

MODEL1

MODEL2

Accuracy

0.544

0.88

F1 score

0.5762

0.3469

 

Model 1 predicts well on multiple class classification on imbalanced given data, and F1 score is the metric to quantify its performance.

 

 

반응형

'Development > Big Data, R, ...' 카테고리의 다른 글

데이터 교정  (0) 2020.01.29
다중 공선성(multicollinarity)  (0) 2020.01.29
기존 프로그래밍 vs 머신러닝  (0) 2020.01.29
배열의 산술연산자 (.+, .-, .*, ./, .\, .^)  (0) 2020.01.28
cost  (0) 2020.01.27