首先執(zhí)行
pip install pyspider
此時(shí)系統(tǒng)提示
<span style="font-size: 16px;">Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-Lau0Qp/pycurl/
You are using pip version 9.0.1, however version 9.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
</span>
這是pip版本的問(wèn)題,執(zhí)行
<span style="font-size: 16px;">sudo python -m pip install --upgrade pip</span>
升級(jí)pip
繼續(xù)執(zhí)行
<span style="font-size: 16px;"> sudo pip install pyspider</span>
來(lái)安裝pyspider,此時(shí)報(bào)錯(cuò):
<span style="font-size: 16px;">Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-szUHrj/pycurl/
</span>
原因是pyspider的依賴庫(kù)未安裝,需要執(zhí)行
<span style="font-size: 16px;">sudo apt-get install python python-dev python-distribute python-pip libcurl4-openssl-dev libxml2-dev libxslt1-dev python-lxml</span>
命令來(lái)安裝以下支持類庫(kù)
本來(lái)以為可以正常安裝pyspider了,可繼續(xù)執(zhí)行 sudo pip install pyspider時(shí)系統(tǒng)報(bào)錯(cuò)信息為:
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
span style="color: rgb(255, 0, 0);">Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-B8gZjb/pycurl/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-aDBSCP-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-B8gZjb/pycurl/</span>
原因是因?yàn)榘惭b工具包時(shí)需要抓取網(wǎng)頁(yè)因而要處理 https,而處理 https 又依賴加解密算法(即 cryptography 包),而 cryptography 又依賴傅立葉變換的算法以及相應(yīng)的編譯環(huán)境。Ubuntu 16.04 默認(rèn)沒(méi)有安裝 libffi-dev 和 libssl-dev,gcc 也不一定安裝,而目標(biāo)安裝包又沒(méi)有將相關(guān)軟件包記到依賴列表里,因此需要先手動(dòng)安裝:
sudo apt-get install libssl-dev libffi-dev build-essential
此時(shí)再執(zhí)行 sudo pip install pyspider 就OK了!
因此Linux下安裝要執(zhí)行以下幾個(gè)命令,(劃重點(diǎn)?。?/p>
sudo python -m pip install --upgrade pip
sudo apt-get install python python-dev python-distribute python-pip libcurl4-openssl-dev libxml2-dev libxslt1-dev python-lxml
sudo apt-get install libssl-dev libffi-dev build-essential
sudo pip install pyspider
補(bǔ)充: