I am trying to install lxml to install scrapy on my Mac (v 10.9.4)
╭─ ~ ╰─➤ pip install lxml Downloading/unpacking lxml Downloading lxml-3.4.0.tar.gz (3.5MB): 3.5MB downloaded Running setup.py (path:/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/setup.py) egg_info for package lxml /System/Library/Frameworks/ UserWarning: Unknown distribution option: 'bugtrack_url' warnings.warn(msg) Building lxml version 3.4.0. Building without Cython. Using build configuration of libxslt 1.1.28 warning: no previously-included files found matching '*.py' Installing collected packages: lxml Running setup.py install for lxml /System/Library/Frameworks/ UserWarning: Unknown distribution option: 'bugtrack_url' warnings.warn(msg) Building lxml version 3.4.0. Building without Cython. Using build configuration of libxslt 1.1.28 building 'lxml.etree' extension cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/src/lxml/includes -I/System/Library/Frameworks/ -c src/lxml/lxml.etree.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -w -flat_namespace cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.macosx-10.9-intel-2.7/lxml/etree.so building 'lxml.objectify' extension cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/src/lxml/includes -I/System/Library/Frameworks/ -c src/lxml/lxml.objectify.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.objectify.o -w -flat_namespace cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.objectify.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.macosx-10.9-intel-2.7/lxml/objectify.so error: could not create '/lib/python2.7/site-packages/lxml': Permission denied Complete output from command /usr/bin/python -c "import setuptools, tokenize;__file__='/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip-_xm20j-record/install-record.txt --single-version-externally-managed --compile: /System/Library/Frameworks/ UserWarning: Unknown distribution option: 'bugtrack_url' warnings.warn(msg) Building lxml version 3.4.0. Building without Cython. Using build configuration of libxslt 1.1.28 running install running build running build_py creating build creating build/lib.macosx-10.9-intel-2.7 creating build/lib.macosx-10.9-intel-2.7/lxml copying src/lxml/__init__.py -> build/lib.macosx-10.9-intel-2.7/lxml copying src/lxml/_elementpath.py -> build/lib.macosx-10.9-intel-2.7/lxml copying src/lxml/builder.py -> build/lib.macosx-10.9-intel-2.7/lxml copying src/lxml/cssselect.py -> build/lib.macosx-10.9-intel-2.7/lxml copying src/lxml/doctestcompare.py -> build/lib.macosx-10.9-intel-2.7/lxml copying src/lxml/ElementInclude.py -> build/lib.macosx-10.9-intel-2.7/lxml copying src/lxml/pyclasslookup.py -> build/lib.macosx-10.9-intel-2.7/lxml copying src/lxml/sax.py -> build/lib.macosx-10.9-intel-2.7/lxml copying src/lxml/usedoctest.py -> build/lib.macosx-10.9-intel-2.7/lxml creating build/lib.macosx-10.9-intel-2.7/lxml/includes copying src/lxml/includes/__init__.py -> build/lib.macosx-10.9-intel-2.7/lxml/includes creating build/lib.macosx-10.9-intel-2.7/lxml/html copying src/lxml/html/__init__.py -> build/lib.macosx-10.9-intel-2.7/lxml/html copying src/lxml/html/_diffcommand.py -> build/lib.macosx-10.9-intel-2.7/lxml/html copying src/lxml/html/_html5builder.py -> build/lib.macosx-10.9-intel-2.7/lxml/html copying src/lxml/html/_setmixin.py -> build/lib.macosx-10.9-intel-2.7/lxml/html copying src/lxml/html/builder.py -> build/lib.macosx-10.9-intel-2.7/lxml/html copying src/lxml/html/clean.py -> build/lib.macosx-10.9-intel-2.7/lxml/html copying src/lxml/html/defs.py -> build/lib.macosx-10.9-intel-2.7/lxml/html copying src/lxml/html/diff.py -> build/lib.macosx-10.9-intel-2.7/lxml/html copying src/lxml/html/ElementSoup.py -> build/lib.macosx-10.9-intel-2.7/lxml/html copying src/lxml/html/formfill.py -> build/lib.macosx-10.9-intel-2.7/lxml/html copying src/lxml/html/html5parser.py -> build/lib.macosx-10.9-intel-2.7/lxml/html copying src/lxml/html/soupparser.py -> build/lib.macosx-10.9-intel-2.7/lxml/html copying src/lxml/html/usedoctest.py -> build/lib.macosx-10.9-intel-2.7/lxml/html creating build/lib.macosx-10.9-intel-2.7/lxml/isoschematron copying src/lxml/isoschematron/__init__.py -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron copying src/lxml/lxml.etree.h -> build/lib.macosx-10.9-intel-2.7/lxml copying src/lxml/lxml.etree_api.h -> build/lib.macosx-10.9-intel-2.7/lxml copying src/lxml/includes/c14n.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes copying src/lxml/includes/config.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes copying src/lxml/includes/dtdvalid.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes copying src/lxml/includes/etreepublic.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes copying src/lxml/includes/htmlparser.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes copying src/lxml/includes/relaxng.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes copying src/lxml/includes/schematron.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes copying src/lxml/includes/tree.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes copying src/lxml/includes/uri.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes copying src/lxml/includes/xinclude.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes copying src/lxml/includes/xmlerror.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes copying src/lxml/includes/xmlparser.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes copying src/lxml/includes/xmlschema.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes copying src/lxml/includes/xpath.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes copying src/lxml/includes/xslt.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes copying src/lxml/includes/etree_defs.h -> build/lib.macosx-10.9-intel-2.7/lxml/includes copying src/lxml/includes/lxml-version.h -> build/lib.macosx-10.9-intel-2.7/lxml/includes creating build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources creating build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/rng copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/rng creating build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl creating build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1 copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1 copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1 copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1 copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1 copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1 copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1 running build_ext building 'lxml.etree' extension creating build/temp.macosx-10.9-intel-2.7 creating build/temp.macosx-10.9-intel-2.7/src creating build/temp.macosx-10.9-intel-2.7/src/lxml cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/src/lxml/includes -I/System/Library/Frameworks/ -c src/lxml/lxml.etree.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -w -flat_namespace cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.macosx-10.9-intel-2.7/lxml/etree.so building 'lxml.objectify' extension cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/src/lxml/includes -I/System/Library/Frameworks/ -c src/lxml/lxml.objectify.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.objectify.o -w -flat_namespace cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.objectify.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.macosx-10.9-intel-2.7/lxml/objectify.so running install_lib creating /lib/python2.7/site-packages/lxml error: could not create '/lib/python2.7/site-packages/lxml': Permission denied ---------------------------------------- Cleaning up... Command /usr/bin/python -c "import setuptools, tokenize;__file__='/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip-_xm20j-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml Storing debug log for failure in /Users/ishaantaylor/Library/Logs/pip.log If somebody could help me I would really appreciate it! I have a suspicion that the issue has to do with multiple versions of python but I'm not sure how to check or fix that.
Edit 1: When I try sudo pip install lxml:
╭─ ~ ╰─➤ sudo pip install lxml Traceback (most recent call last): File "/bin/pip", line 7, in <module> from pip import main ImportError: No module named pip Edit 2: When I try STATIC_DEPS=true sudo pip install lxml
╭─ ~ ╰─➤ STATIC_DEPS=true sudo pip install lxml Traceback (most recent call last): File "/bin/pip", line 7, in <module> from pip import main ImportError: No module named pip Also, I tried installing another package with pip and that didn't work either.. see below:
╭─ /lib/python2.7/site-packages ╰─➤ pip install requests Downloading/unpacking requests Downloading requests-2.4.1-py2.py3-none-any.whl (458kB): 458kB downloaded Installing collected packages: requests Cleaning up... Exception: Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/", line 122, in main status = self.run(options, args) File "/usr/local/lib/python2.7/site-packages/", line 283, in run requirement_set.install(install_options, global_options, root=options.root_path) File "/usr/local/lib/python2.7/site-packages/", line 1435, in install requirement.install(install_options, global_options, *args, **kwargs) File "/usr/local/lib/python2.7/site-packages/", line 671, in install self.move_wheel_files(self.source_dir, root=root) File "/usr/local/lib/python2.7/site-packages/", line 901, in move_wheel_files pycompile=self.pycompile, File "/usr/local/lib/python2.7/site-packages/", line 215, in move_wheel_files clobber(source, lib_dir, True) File "/usr/local/lib/python2.7/site-packages/", line 205, in clobber os.makedirs(destdir) File "/System/Library/Frameworks/", line 157, in makedirs mkdir(name, mode) OSError: [Errno 13] Permission denied: '/lib/python2.7/site-packages/requests' Storing debug log for failure in /Users/ishaantaylor/Library/Logs/pip.log Edit 4: Tried to download virtualenv, people recommended this as an alternative to sudo. Result is below:
╭─ ~ ╰─➤ pip install virtualenv 127 ↵ Downloading/unpacking virtualenv Downloading virtualenv-1.11.6-py2.py3-none-any.whl (1.6MB): 1.6MB downloaded Installing collected packages: virtualenv Cleaning up... Exception: Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/", line 122, in main status = self.run(options, args) File "/usr/local/lib/python2.7/site-packages/", line 283, in run requirement_set.install(install_options, global_options, root=options.root_path) File "/usr/local/lib/python2.7/site-packages/", line 1435, in install requirement.install(install_options, global_options, *args, **kwargs) File "/usr/local/lib/python2.7/site-packages/", line 671, in install self.move_wheel_files(self.source_dir, root=root) File "/usr/local/lib/python2.7/site-packages/", line 901, in move_wheel_files pycompile=self.pycompile, File "/usr/local/lib/python2.7/site-packages/", line 215, in move_wheel_files clobber(source, lib_dir, True) File "/usr/local/lib/python2.7/site-packages/", line 209, in clobber shutil.copy2(srcfile, destfile) File "/System/Library/Frameworks/", line 130, in copy2 copyfile(src, dst) File "/System/Library/Frameworks/", line 83, in copyfile with open(dst, 'wb') as fdst: IOError: [Errno 13] Permission denied: '/lib/python2.7/site-packages/virtualenv.py' Storing debug log for failure in /Users/ishaantaylor/Library/Logs/pip.log Edit 5:
╭─ ~ ╰─➤ sudo pip install virtualenv 1 ↵ Password: Traceback (most recent call last): File "/bin/pip", line 7, in <module> from pip import main ImportError: No module named pip ╭─ ~ ╰─➤ pip install virtualenv 1 ↵ Downloading/unpacking virtualenv Downloading virtualenv-1.11.6-py2.py3-none-any.whl (1.6MB): 1.6MB downloaded Installing collected packages: virtualenv Cleaning up... Exception: Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/", line 122, in main status = self.run(options, args) File "/usr/local/lib/python2.7/site-packages/", line 283, in run requirement_set.install(install_options, global_options, root=options.root_path) File "/usr/local/lib/python2.7/site-packages/", line 1435, in install requirement.install(install_options, global_options, *args, **kwargs) File "/usr/local/lib/python2.7/site-packages/", line 671, in install self.move_wheel_files(self.source_dir, root=root) File "/usr/local/lib/python2.7/site-packages/", line 901, in move_wheel_files pycompile=self.pycompile, File "/usr/local/lib/python2.7/site-packages/", line 215, in move_wheel_files clobber(source, lib_dir, True) File "/usr/local/lib/python2.7/site-packages/", line 209, in clobber shutil.copy2(srcfile, destfile) File "/System/Library/Frameworks/", line 130, in copy2 copyfile(src, dst) File "/System/Library/Frameworks/", line 83, in copyfile with open(dst, 'wb') as fdst: IOError: [Errno 13] Permission denied: '/lib/python2.7/site-packages/virtualenv.py' Storing debug log for failure in /Users/ishaantaylor/Library/Logs/pip.log Edit 6: In response to hd1:
╭─ ~ ╰─➤ curl -kLso /tmp/get-pip.py ╭─ ~ ╰─➤ sudo python get-pip.py python: can't open file 'get-pip.py': [Errno 2] No such file or directory Edit 7: In Response to hd1:
╭─ / ╰─➤ curl -kLso /tmp/get-pip.py ╭─ / ╰─➤ cd ~ ╭─ ~ ╰─➤ sudo python /tmp/get-pip.py Requirement already up-to-date: pip in /usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg Cleaning up... ╭─ ~ ╰─➤ sudo pip install virtualenv Downloading/unpacking virtualenv Downloading virtualenv-1.11.6-py2.py3-none-any.whl (1.6MB): 1.6MB downloaded Installing collected packages: virtualenv Successfully installed virtualenv Cleaning up... ╭─ ~ ╰─➤ mkdir -p ~/.virtualenvs ╭─ ~ ╰─➤ virtualenv ~/.virtualenvs/lxmltest Traceback (most recent call last): File "/bin/virtualenv", line 7, in <module> from virtualenv import main ImportError: No module named virtualenv ╭─ ~ ╰─➤ which virtualenv 1 ↵ /bin/virtualenv ╭─ ~ ╰─➤ echo -e ${PATH//:/\\n} /usr/local/bin /usr/bin /bin /usr/sbin /sbin /opt/X11/bin 3 Answers
It looks like you're having a permissions error, based on this message in your output: error: could not create '/lib/python2.7/site-packages/lxml': Permission denied.
One thing you can try is doing a user install of the package with pip install lxml --user. For more information on how that works, check out this StackOverflow answer. (Thanks to Ishaan Taylor for the suggestion)
You can also run pip install as a superuser with sudo pip install lxml but it is not generally a good idea because it can cause issues with your system-level packages.
Set up a virtualenv:
% curl -kLso /tmp/get-pip.py % sudo python /tmp/get-pip.py These commands install pip into the global site-packages directory.
% sudo pip install virtualenv and ditto for virtualenv:
% mkdir -p ~/.virtualenvs I like my virtualenvs under one tree in my home directory called .virtualenvs
% virtualenv ~/.virtualenvs/lxmltest Creates a virtualenv.
% . ~/.virtualenvs/lxmltest/bin/activate Removes the need to specify the full path to pip/python in this virtualenv.
% pip install lxml Alternatively execute ~/.virtualenvs/lxmltest/bin/pip install lxml if you chose not to follow the previous step. Note, I'm not sure how far along you are, so some of these steps can be safely skipped. Of course, if you mess something up, you can always rm -Rf ~/.virtualenvs/lxmltest and start again from a new virtualenv.
On a Mac, you need to use this command:
STATIC_DEPS=true sudo pip install lxml
3
