Sometimes, the installed version of Python on your linux system isn’t what you need, and you can’t easily install a newer version system-wide. Of course, you can download, extract, and configure and then make Python that lives in it’s own directory. But then you don’t have all the pointers to the Include directories that are useful for linking stuff against. And you don’t want to change symlinks at the system level, that might break things, a lot of things.
If you are using cython to speed up your code, then it needs to know where exactly to find those extra header files. Unfortunately, it needs both the C++ and the C versions, it seems.
Lets go through the steps of setting up a new python, and then linking it for compiling cythonized programs.
Download and Make
Download python, and navigate to where you saved it. I’ll assume we downloaded version 10 to ~/software
cd ~/software
tar -xzf Python-3.10.5.tgz
cd Python-3.10.5
./configure --enable-optimizations
makeSetup New Virtualenv With It
Make sure you have virtualenv installed, its necessary if you want to specify a specific version of python. Yes, you could probably get around this with conda or docker, but we are doing things the painful way.
virtualenv -p ~/software/Python-3.10.5/python ~/py10_venv
source ~/py10_venv/activateAdd It to Environment
export C_INCLUDE_PATH=/home/rmflight/software/Python-3.10.5:$C_INCLUDE_PATHInstall Something Cython
We are going to install a package that needs only a few python deps, our labs icikt package.
# check that you are using the right version in the venv
python3 --version
python3 -m pip install docopt
python3 -m pip install scipy
python3 -m pip install cythoncd ~/software
git clone git@github.com:MoseleyBioinformaticsLab/icikt.gitpython3 -m pip install --global-option=build_ext --global-option="-I/home/rmflight/software/Python-3.10.5/Include" ~/software/iciktNotice we’ve done two very important things overall:
- Added a path,
C_INCLUDE_PATH - Added two
--global-optionin the call to build our local package
For some reason we need both the environment variable path, and the global option to get this to work.
If you forget one or the other, you will see error messages concerning python.h
Reuse
Citation
@online{mflight2022,
author = {Robert M Flight},
title = {Compiling {Against} {Python.h}},
date = {2022-06-22},
url = {https://rmflight.github.io/posts/2022-06-22-compiling-against-pythonh},
langid = {en}
}