달력

1

« 2020/1 »




 

 

말뭉치(Corpus)

·         코퍼스

·         정형이나 비정형인 단어나 표현의 묶음입니다.

·         자연어 분석 작업을 위해 만든 샘플 문서 집합을 지칭합니다.

·         소설, 신문 등의 문서를 단순히 모아놓은 것도 있지만 품사, 형태소, 등의 보조적 의미를 추가하고 쉬운 분석을 위해 구조적인 형태로 정리해 놓은 것도 포함됩니다.

 

 


'Development > Big Data, R, ...' 카테고리의 다른 글

6.Summary - 6. 학습결과 적용하기  (0) 2020.01.27
텍스트 데이터 전처리  (0) 2020.01.26
말뭉치(Corpus)  (0) 2020.01.25
신뢰도(conviction)  (0) 2020.01.24
Moravec's Paradox (모라벡의 역설)  (0) 2020.01.23
과소적합(underfitting)  (0) 2020.01.23
Posted by codedragon codedragon

댓글을 달아 주세요

2020. 1. 25. 12:32

apply() 적용 Development/Python


 

 

apply() 적용

·         행이나 단위로 복잡한 처리를 하고 싶을 사용합니다.

·         apply() axis로 행 또는 열을 지정하면 각 열(또는 행)을 반복하여 해당 함수를 적용시킵니다.

 

method

description

apply(|열을 받는 함수, axis=0)

axis 인수로 행 또는 열을 지정하면 각 열(또는 행)을 반복하여 해당 함수를 적용시킵니다.

apply(pd.value_counts)

열에 대해 어떤 값이 얼마나 사용되었는지 알려줍니다.

 

 


Posted by codedragon codedragon

댓글을 달아 주세요



 

 

일반 함수 vs 익명함수

 

구분

def  이름이 있는 함수 정의

lambda  익명 함수 정의

함수 정의

def my_2times(x):

     result = x*2

     return result

 

my_2times = lambda x: x*2

 

 

 

함수 호출

my_2times(3)

# 6

my_2times(3)

# 6

 

 



Posted by codedragon codedragon

댓글을 달아 주세요



 

 

 

mlxtend 설치시 에러

ERROR: Could not install packages due to an EnvironmentError: [WinError 5] Access is denied

[WinError 5] Access is denied: 'c:\\python\\python36\\lib\\site-packages\\~umpy\\.libs\\libopenblas.BNVRK7633HSX7YVO2TADGR4A5KEKXJAW.gfortran-win_amd64.dll'

Consider using the `--user` option or check the permissions.

ERROR: Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: 'c:\\python\\python36\\lib\\site-packages\\~andas\\io\\msgpack\\_packer.cp36-win_amd64.pyd'

Consider using the `--user` option or check the permissions.

 

 

 

 

해결방법

numpy, scipy, joblib, pandas, matplotlib, scikit-learn, mlxtend 함께 사용되는 해당 패키지 버전이 맞지 않아 에러가 발생하며

반복적으로 명령어를 수행하면 이전 패키지를 삭제 다시 설치해 주는 작업을 해주므로 에러가 나오지 않을 때까지 반복 줍니다.

 

C:\CodeLab>

 

C:\CodeLab>pip install mlxtend

Collecting mlxtend

  Downloading https://files.pythonhosted.org/packages/52/04/c362f34f666f0ddc7cf593805e64d64fa670ed96fd9302e68549dd48287d/mlxtend-0.17.0-py2.py3-none-any.whl (1.3MB)

     |████████████████████████████████| 1.3MB 252kB/s

Requirement already satisfied: setuptools in c:\python\python36\lib\site-packages (from mlxtend) (39.0.1)

Collecting scipy>=1.2.1

  Downloading https://files.pythonhosted.org/packages/8d/2f/fcb6150813b89d628749784370132e431f687ebab5a1063eb298cc941f76/scipy-1.4.1-cp36-cp36m-win_amd64.whl (30.8MB)

     |████████████████████████████████| 30.8MB 177kB/s

Collecting joblib>=0.13.2

  Downloading https://files.pythonhosted.org/packages/28/5c/cf6a2b65a321c4a209efcdf64c2689efae2cb62661f8f6f4bb28547cf1bf/joblib-0.14.1-py2.py3-none-any.whl (294kB)

     |████████████████████████████████| 296kB 145kB/s

Collecting numpy>=1.16.2

  Downloading https://files.pythonhosted.org/packages/53/74/b997e4c7b4abc668e99f4c3dba87ee2c6f7559319af756cc1ede37665a8d/numpy-1.18.1-cp36-cp36m-win_amd64.whl (12.8MB)

     |████████████████████████████████| 12.8MB 192kB/s

Collecting pandas>=0.24.2

  Downloading https://files.pythonhosted.org/packages/f0/ac/92c3d2f0b627efbd1a7b2156faa697f9c2bbd7b0fe83ba8a9d36f982156f/pandas-0.25.3-cp36-cp36m-win_amd64.whl (9.0MB)

     |████████████████████████████████| 9.0MB 1.3MB/s

Collecting matplotlib>=3.0.0

  Downloading https://files.pythonhosted.org/packages/44/fb/132de6a4b803d8ce909a89043b7d3f775f64e0a39398fc98c02e3e144b61/matplotlib-3.1.2-cp36-cp36m-win_amd64.whl (9.1MB)

     |████████████████████████████████| 9.1MB 49kB/s

Collecting scikit-learn>=0.20.3

  Downloading https://files.pythonhosted.org/packages/21/d9/7da9d5afdf901ab069226853c84a432c5db80fd616849ebcee29fd8a04e0/scikit_learn-0.22.1-cp36-cp36m-win_amd64.whl (6.3MB)

     |████████████████████████████████| 6.3MB 60kB/s

Requirement already satisfied: python-dateutil>=2.6.1 in c:\python\python36\lib\site-packages (from pandas>=0.24.2->mlxtend) (2.7.3)

Requirement already satisfied: pytz>=2017.2 in c:\python\python36\lib\site-packages (from pandas>=0.24.2->mlxtend) (2018.4)

Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (2.2.0)

Requirement already satisfied: cycler>=0.10 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (0.10.0)

Requirement already satisfied: kiwisolver>=1.0.1 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (1.0.1)

Requirement already satisfied: six>=1.5 in c:\python\python36\lib\site-packages (from python-dateutil>=2.6.1->pandas>=0.24.2->mlxtend) (1.11.0)

Installing collected packages: numpy, scipy, joblib, pandas, matplotlib, scikit-learn, mlxtend

  Found existing installation: numpy 1.14.5

    Uninstalling numpy-1.14.5:

      Successfully uninstalled numpy-1.14.5

ERROR: Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: 'c:\\python\\python36\\lib\\site-packages\\~umpy\\.libs\\libopenblas.BNVRK7633HSX7YVO2TADGR4A5KEKXJAW.gfortran-win_amd64.dll'

Consider using the `--user` option or check the permissions.

 

 

C:\CodeLab>pip install mlxtend

Collecting mlxtend

  Using cached https://files.pythonhosted.org/packages/52/04/c362f34f666f0ddc7cf593805e64d64fa670ed96fd9302e68549dd48287d/mlxtend-0.17.0-py2.py3-none-any.whl

Collecting pandas>=0.24.2

  Using cached https://files.pythonhosted.org/packages/f0/ac/92c3d2f0b627efbd1a7b2156faa697f9c2bbd7b0fe83ba8a9d36f982156f/pandas-0.25.3-cp36-cp36m-win_amd64.whl

Requirement already satisfied: setuptools in c:\python\python36\lib\site-packages (from mlxtend) (39.0.1)

Collecting scipy>=1.2.1

  Using cached https://files.pythonhosted.org/packages/8d/2f/fcb6150813b89d628749784370132e431f687ebab5a1063eb298cc941f76/scipy-1.4.1-cp36-cp36m-win_amd64.whl

Collecting joblib>=0.13.2

  Using cached https://files.pythonhosted.org/packages/28/5c/cf6a2b65a321c4a209efcdf64c2689efae2cb62661f8f6f4bb28547cf1bf/joblib-0.14.1-py2.py3-none-any.whl

Collecting scikit-learn>=0.20.3

  Using cached https://files.pythonhosted.org/packages/21/d9/7da9d5afdf901ab069226853c84a432c5db80fd616849ebcee29fd8a04e0/scikit_learn-0.22.1-cp36-cp36m-win_amd64.whl

Collecting matplotlib>=3.0.0

  Using cached https://files.pythonhosted.org/packages/44/fb/132de6a4b803d8ce909a89043b7d3f775f64e0a39398fc98c02e3e144b61/matplotlib-3.1.2-cp36-cp36m-win_amd64.whl

Requirement already satisfied: numpy>=1.16.2 in c:\python\python36\lib\site-packages (from mlxtend) (1.18.1)

Requirement already satisfied: pytz>=2017.2 in c:\python\python36\lib\site-packages (from pandas>=0.24.2->mlxtend) (2018.4)

Requirement already satisfied: python-dateutil>=2.6.1 in c:\python\python36\lib\site-packages (from pandas>=0.24.2->mlxtend) (2.7.3)

Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (2.2.0)

Requirement already satisfied: kiwisolver>=1.0.1 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (1.0.1)

Requirement already satisfied: cycler>=0.10 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (0.10.0)

Requirement already satisfied: six>=1.5 in c:\python\python36\lib\site-packages (from python-dateutil>=2.6.1->pandas>=0.24.2->mlxtend) (1.11.0)

Installing collected packages: pandas, scipy, joblib, scikit-learn, matplotlib, mlxtend

  Found existing installation: pandas 0.23.1

    Uninstalling pandas-0.23.1:

      Successfully uninstalled pandas-0.23.1

ERROR: Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: 'c:\\python\\python36\\lib\\site-packages\\~andas\\io\\msgpack\\_packer.cp36-win_amd64.pyd'

Consider using the `--user` option or check the permissions.

 

 

C:\CodeLab>pip show mlxtend

WARNING: Package(s) not found: mlxtend

 

C:\CodeLab>pip install mlxtend

Collecting mlxtend

  Using cached https://files.pythonhosted.org/packages/52/04/c362f34f666f0ddc7cf593805e64d64fa670ed96fd9302e68549dd48287d/mlxtend-0.17.0-py2.py3-none-any.whl

Collecting scipy>=1.2.1

  Using cached https://files.pythonhosted.org/packages/8d/2f/fcb6150813b89d628749784370132e431f687ebab5a1063eb298cc941f76/scipy-1.4.1-cp36-cp36m-win_amd64.whl

Requirement already satisfied: pandas>=0.24.2 in c:\python\python36\lib\site-packages (from mlxtend) (0.25.3)

Requirement already satisfied: numpy>=1.16.2 in c:\python\python36\lib\site-packages (from mlxtend) (1.18.1)

Requirement already satisfied: setuptools in c:\python\python36\lib\site-packages (from mlxtend) (39.0.1)

Collecting scikit-learn>=0.20.3

  Using cached https://files.pythonhosted.org/packages/21/d9/7da9d5afdf901ab069226853c84a432c5db80fd616849ebcee29fd8a04e0/scikit_learn-0.22.1-cp36-cp36m-win_amd64.whl

Collecting matplotlib>=3.0.0

  Using cached https://files.pythonhosted.org/packages/44/fb/132de6a4b803d8ce909a89043b7d3f775f64e0a39398fc98c02e3e144b61/matplotlib-3.1.2-cp36-cp36m-win_amd64.whl

Collecting joblib>=0.13.2

  Using cached https://files.pythonhosted.org/packages/28/5c/cf6a2b65a321c4a209efcdf64c2689efae2cb62661f8f6f4bb28547cf1bf/joblib-0.14.1-py2.py3-none-any.whl

Requirement already satisfied: python-dateutil>=2.6.1 in c:\python\python36\lib\site-packages (from pandas>=0.24.2->mlxtend) (2.7.3)

Requirement already satisfied: pytz>=2017.2 in c:\python\python36\lib\site-packages (from pandas>=0.24.2->mlxtend) (2018.4)

Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (2.2.0)

Requirement already satisfied: kiwisolver>=1.0.1 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (1.0.1)

Requirement already satisfied: cycler>=0.10 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (0.10.0)

Requirement already satisfied: six>=1.5 in c:\python\python36\lib\site-packages (from python-dateutil>=2.6.1->pandas>=0.24.2->mlxtend) (1.11.0)

Installing collected packages: scipy, joblib, scikit-learn, matplotlib, mlxtend

  Found existing installation: scipy 1.1.0

    Uninstalling scipy-1.1.0:

      Successfully uninstalled scipy-1.1.0

ERROR: Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: 'c:\\python\\python36\\lib\\site-packages\\~cipy\\cluster\\_hierarchy.cp36-win_amd64.pyd'

Consider using the `--user` option or check the permissions.

 

 

C:\CodeLab>pip install mlxtend

Collecting mlxtend

  Using cached https://files.pythonhosted.org/packages/52/04/c362f34f666f0ddc7cf593805e64d64fa670ed96fd9302e68549dd48287d/mlxtend-0.17.0-py2.py3-none-any.whl

Collecting scikit-learn>=0.20.3

  Using cached https://files.pythonhosted.org/packages/21/d9/7da9d5afdf901ab069226853c84a432c5db80fd616849ebcee29fd8a04e0/scikit_learn-0.22.1-cp36-cp36m-win_amd64.whl

Requirement already satisfied: scipy>=1.2.1 in c:\python\python36\lib\site-packages (from mlxtend) (1.4.1)

Requirement already satisfied: numpy>=1.16.2 in c:\python\python36\lib\site-packages (from mlxtend) (1.18.1)

Requirement already satisfied: pandas>=0.24.2 in c:\python\python36\lib\site-packages (from mlxtend) (0.25.3)

Collecting joblib>=0.13.2

  Using cached https://files.pythonhosted.org/packages/28/5c/cf6a2b65a321c4a209efcdf64c2689efae2cb62661f8f6f4bb28547cf1bf/joblib-0.14.1-py2.py3-none-any.whl

Collecting matplotlib>=3.0.0

  Using cached https://files.pythonhosted.org/packages/44/fb/132de6a4b803d8ce909a89043b7d3f775f64e0a39398fc98c02e3e144b61/matplotlib-3.1.2-cp36-cp36m-win_amd64.whl

Requirement already satisfied: setuptools in c:\python\python36\lib\site-packages (from mlxtend) (39.0.1)

Requirement already satisfied: pytz>=2017.2 in c:\python\python36\lib\site-packages (from pandas>=0.24.2->mlxtend) (2018.4)

Requirement already satisfied: python-dateutil>=2.6.1 in c:\python\python36\lib\site-packages (from pandas>=0.24.2->mlxtend) (2.7.3)

Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (2.2.0)

Requirement already satisfied: kiwisolver>=1.0.1 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (1.0.1)

Requirement already satisfied: cycler>=0.10 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (0.10.0)

Requirement already satisfied: six>=1.5 in c:\python\python36\lib\site-packages (from python-dateutil>=2.6.1->pandas>=0.24.2->mlxtend) (1.11.0)

Installing collected packages: joblib, scikit-learn, matplotlib, mlxtend

  Found existing installation: scikit-learn 0.19.1

    Uninstalling scikit-learn-0.19.1:

      Successfully uninstalled scikit-learn-0.19.1

ERROR: Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: 'c:\\python\\python36\\lib\\site-packages\\~klearn\\cluster\\_dbscan_inner.cp36-win_amd64.pyd'

Consider using the `--user` option or check the permissions.

 

 

C:\CodeLab>pip install mlxtend

Collecting mlxtend

  Using cached https://files.pythonhosted.org/packages/52/04/c362f34f666f0ddc7cf593805e64d64fa670ed96fd9302e68549dd48287d/mlxtend-0.17.0-py2.py3-none-any.whl

Requirement already satisfied: pandas>=0.24.2 in c:\python\python36\lib\site-packages (from mlxtend) (0.25.3)

Requirement already satisfied: joblib>=0.13.2 in c:\python\python36\lib\site-packages (from mlxtend) (0.14.1)

Requirement already satisfied: setuptools in c:\python\python36\lib\site-packages (from mlxtend) (39.0.1)

Requirement already satisfied: scikit-learn>=0.20.3 in c:\python\python36\lib\site-packages (from mlxtend) (0.22.1)

Requirement already satisfied: scipy>=1.2.1 in c:\python\python36\lib\site-packages (from mlxtend) (1.4.1)

Collecting matplotlib>=3.0.0

  Using cached https://files.pythonhosted.org/packages/44/fb/132de6a4b803d8ce909a89043b7d3f775f64e0a39398fc98c02e3e144b61/matplotlib-3.1.2-cp36-cp36m-win_amd64.whl

Requirement already satisfied: numpy>=1.16.2 in c:\python\python36\lib\site-packages (from mlxtend) (1.18.1)

Requirement already satisfied: pytz>=2017.2 in c:\python\python36\lib\site-packages (from pandas>=0.24.2->mlxtend) (2018.4)

Requirement already satisfied: python-dateutil>=2.6.1 in c:\python\python36\lib\site-packages (from pandas>=0.24.2->mlxtend) (2.7.3)

Requirement already satisfied: kiwisolver>=1.0.1 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (1.0.1)

Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (2.2.0)

Requirement already satisfied: cycler>=0.10 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (0.10.0)

Requirement already satisfied: six>=1.5 in c:\python\python36\lib\site-packages (from python-dateutil>=2.6.1->pandas>=0.24.2->mlxtend) (1.11.0)

Installing collected packages: matplotlib, mlxtend

  Found existing installation: matplotlib 2.2.2

    Uninstalling matplotlib-2.2.2:

      Successfully uninstalled matplotlib-2.2.2

ERROR: Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: 'c:\\python\\python36\\lib\\site-packages\\~atplotlib\\backends\\_backend_agg.cp36-win_amd64.pyd'

Consider using the `--user` option or check the permissions.

 

 

C:\CodeLab>pip install mlxtend

Collecting mlxtend

  Using cached https://files.pythonhosted.org/packages/52/04/c362f34f666f0ddc7cf593805e64d64fa670ed96fd9302e68549dd48287d/mlxtend-0.17.0-py2.py3-none-any.whl

Requirement already satisfied: matplotlib>=3.0.0 in c:\python\python36\lib\site-packages (from mlxtend) (3.1.2)

Requirement already satisfied: numpy>=1.16.2 in c:\python\python36\lib\site-packages (from mlxtend) (1.18.1)

Requirement already satisfied: pandas>=0.24.2 in c:\python\python36\lib\site-packages (from mlxtend) (0.25.3)

Requirement already satisfied: setuptools in c:\python\python36\lib\site-packages (from mlxtend) (39.0.1)

Requirement already satisfied: joblib>=0.13.2 in c:\python\python36\lib\site-packages (from mlxtend) (0.14.1)

Requirement already satisfied: scikit-learn>=0.20.3 in c:\python\python36\lib\site-packages (from mlxtend) (0.22.1)

Requirement already satisfied: scipy>=1.2.1 in c:\python\python36\lib\site-packages (from mlxtend) (1.4.1)

Requirement already satisfied: python-dateutil>=2.1 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (2.7.3)

Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (2.2.0)

Requirement already satisfied: kiwisolver>=1.0.1 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (1.0.1)

Requirement already satisfied: cycler>=0.10 in c:\python\python36\lib\site-packages (from matplotlib>=3.0.0->mlxtend) (0.10.0)

Requirement already satisfied: pytz>=2017.2 in c:\python\python36\lib\site-packages (from pandas>=0.24.2->mlxtend) (2018.4)

Requirement already satisfied: six>=1.5 in c:\python\python36\lib\site-packages (from python-dateutil>=2.1->matplotlib>=3.0.0->mlxtend) (1.11.0)

Installing collected packages: mlxtend

Successfully installed mlxtend-0.17.0

 

C:\CodeLab>

 

 

 

 

 


Posted by codedragon codedragon

댓글을 달아 주세요

2020. 1. 25. 00:30

ndarray vs list Development/Python



 

ndarray vs list

ndarray

list

[10 20 30 40]

[50, 60, 70, 80]

리스트 내 요소를 구분하는 쉼표(,) 없습니다.

리스트 내 요소를 구분하는 쉼표(,) 있습니다.

가지 타입의 데이터만을 저장할 수 있습니다.

정수, 문자열, 리스트 등 다양한 데이터 타입을 담을 수 있습니다.

 

 


Posted by codedragon codedragon

댓글을 달아 주세요