Update Python version to latest

[orome]

Currently it is impossible to use a Python version other than the one currently offered, making it impossible to share anything written for later versions of Python (e.g., currently 3.8 and 3.9).

It should be possible to select from several Python version, up through the latest stable release, at least; or to be able to install an arbitrary Python.

[jasonkhadka]

any update on this issue? Is there any plan/work going on to update to python 3.8 or 3.9?
This issue mentioned there were work going on for further updates from the one in Feb to Python 3.7?
#1422

[craigcitro]

It's still on our radar; #1880 tracks an ubuntu update (which would bump python along the way).

[jasonkhadka]

Do you know what is the expected date for this update? Is it Dec 26 as mentioned in the comment in the other issue?

It's still on our radar; #1880 tracks an ubuntu update (which would bump python along the way).

[craigcitro]

No, it won't be on Dec 26.

[jasonkhadka]

Is there any estimate for this? When can we roughly expect this?

No, it won't be on Dec 26.

[craigcitro]

@jasonkhadka No specific date.

[KelSolaar]

Hello,

I was wondering if it would make sense to follow the Scientific Python SPEC-0000: https://scientific-python.org/specs/spec-0000/

Cheers,

Thomas

[EwoutH]

This issue is becoming really important, with many packages dropping Python 3.7 support, following the Numpy depreciation policy (NEP 29). Python 3.7 support was dropped on December 26th, 2021 for NumPy.

Aside from NumPy 1.22+, Pandas 1.4+ and NetworkX 2.7+ also only support Python 3.8+.

I would suggest adopting either Python 3.8 and 3.9 as soon as possible, to be able to use the latest versions of those packages.

[EwoutH]

@colaboratory-team @blois @craigcitro @yilei and @katlyn-edwards,

This issue becomes more and more pressing each week that passes. So much packages now require Python 3.8 support. It sucks being limited by the environment, it harms compatibility. Colab really needs an Python 3.8+ update, preferably directly to 3.9.

[craigcitro]

As noted above, this is on our radar. I don't have a specific date to offer, unfortunately.

[EwoutH]

Aside from NumPy 1.22+, Pandas 1.4+ and NetworkX 2.7+ also only support Python 3.8+.

We can now add SciPy 1.8+ and scikit-learn 1.1+ to that list. Soon it will be joined by matplotlib 3.6+, statsmodels 0.14+ and probably PyTorch 1.13+.

Can we get a weekly update or something until the Python version is raised to at least 3.8?

If it's the same effort, I would suggest leapfrogging to Ubuntu 22.04 LTS with Python 3.10. See #1880.

But please do whatever is the fastest.

[namurphy]

If it's the same effort, I would suggest leapfrogging to Ubuntu 22.04 LTS with Python 3.10.

One consideration is that Python 3.10 has significantly improved error messages, which improves user experience, especially for people new to Python. Python 3.11 is going to have some improvements in error messages too.

[liamdugan]

+1 to updating to at least 3.8. The latest version of fairseq now fails to build because it requires numpy 1.23.3 (and thus python 3.8). This is incredibly inconvenient for ML dev

[MrAshRhodes]

Guessing this will never be updated...
Even if there was just a way to manually update the version... just anything to get to a higher python version.

[lisphilar]

I suppose that and I changed Jupyter Notebook environment. I have no idea to help and encourage the maintainers so that they can update Python version.

[ilia3101]

This really needs attention. Important packages are dropping support for 3.7 now.

[RyuuOujiXS]

Pandas pickles in current versions are no longer compatible with the latest available Pandas for Python 3.7

[jakirkham]

What is the ETA for updating Python?

Several libraries have dropped Python 3.7 like NumPy ( numpy/numpy#21558 ), Pandas ( pandas-dev/pandas#41989 ), and other projects following NEP 29.

In conda-forge, we plan to drop Python 3.7 when adding Python 3.11 ( conda-forge/conda-forge-pinning-feedstock#2623 ), which is coming out Monday.

[betaBison]

Can @colaboratory-team @craigcitro @cperry-goog please provide an update on Python/Ubuntu upgrades?

Last update regarding Python was 6 months ago from @craigcitro on Apr 24, 2022

As noted above, this is on our radar. I don't have a specific date to offer, unfortunately.

And the last update on the related #1880 to upgrade Ubuntu was from @cperry-goog on Oct 14, 2021

Thanks, no updates, this is on our radar.

[unnir]

Please update python to at least 3.9 :)

[pawel-smth]

Just in case, as of today, I get "Python 3.8.15" as default.

[leofang]

It seems the update to Python 3.8 is done recently: #3246

[unnir]

how long will it take for 3.9 🙃

[pawel-smth]

UPD 04/01/2022 I've updated it to what works for me currently.

Just in case, here it goes how I've installed 3.10. It survives Runtime restarts (but not the deletion). All the searchable "solutions" were either irrelevant or incomplete/outdated as of today (so mine may become like that later on))) so I've updated the one that was the closest.
I haven't tested it heavily, but basic 3.10 items work.

#The code below installs 3.10 (assuming you now have 3.8) and restarts environment, so you can run your cells.

import sys #for version checker
import os #for restart routine

if '3.10' in sys.version:
  print('You already have 3.10, nothing to install')
elif '3.8' in sys.version:
  print("Python version is: ", sys.version)

  print("Printing content of /usr/local/lib/python* to see available versions")
  !ls /usr/local/lib/python*

  #install python 3.10 and dev utils
  #you may not need all the dev libraries, but I haven't tested which aren't necessary.
  !sudo apt-get update -y > /dev/null
  !sudo apt-get install python3.10 python3.10-dev python3.10-distutils libpython3.10-dev > /dev/null
  !sudo apt-get install python3.10-venv binfmt-support  > /dev/null #recommended in install logs of the command above

  #change alternatives
  !sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 1 > /dev/null
  !sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 2 > /dev/null

  # install pip
  !curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10  > /dev/null

  #install colab's dependencies
  !python3 -m pip install ipython==7.9.0 traitlets==5.7.1 jupyter psutil matplotlib setuptools ipython_genutils ipykernel jupyter_console notebook prompt_toolkit httplib2 astor  > /dev/null

  #minor cleanup
  !sudo apt autoremove > /dev/null

  #link to the old google package
  !ln -s /usr/local/lib/python3.8/dist-packages/google /usr/local/lib/python3.10/dist-packages/google > /dev/null

  #!jupyter serverextension list  > /dev/null
  #!jupyter notebook --config=/usr/local/etc/jupyter/jupyter_notebook_config.json  > /dev/null
  
  #this is just to verify if 3.10 folder was indeed created
  print("Printing content of /usr/local/lib/python3.10/")
  !ls /usr/local/lib/python3.10/

  #restart environment so you don't have to do it manually
  os.kill(os.getpid(), 9)
else:
  print("Your out of the box Pythong is not 3.8, so probably the script will not work, so pls feel free to edit the script to ignore then check and re-run: ", sys.version)

[metrizable]

Colab is now on Python 3.8 (#3246) with plans to upgrade to Python 3.9+ in the future!

[jakirkham]

Thanks Eric! 🙏 Christmas came early 🎁

[KelSolaar]

Colab is now on Python 3.8 (#3246) with plans to upgrade to Python 3.9+ in the future!

This is great news, thank you!

[KelSolaar]

Just in case, here it goes how I've installed 3.10. It survives Runtime restarts (but not the deletion). All the searchable "solutions" were either irrelevant or incomplete/outdated as of today (so mine may become like that later on))) so I've updated the one that was the closest. I haven't tested it heavily, but basic 3.10 items work.

#The code below installs 3.10 (assuming you now have 3.8) and restarts environment, so you can run your cells.

import sys #for version checker
import os #for restart routine

if '3.10' in sys.version:
  print('You already have 3.10, nothing to install')
else:
  #install python 3.10 and dev utils
  #you may not need all the dev libraries, but I haven't tested which aren't necessary.
  !sudo apt-get update -y
  !sudo apt-get install python3.10 python3.10-dev python3.10-distutils libpython3.10-dev 
  !sudo apt-get install python3.10-venv binfmt-support #recommended in install logs of the command above

  #change alternatives
  !sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 1
  !sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 2

  # install pip
  !curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10
  !python3 get-pip.py --force-reinstall

  #install colab's dependencies
  !python3 -m pip install setuptools ipython ipython_genutils ipykernel jupyter_console prompt_toolkit httplib2 astor

  #minor cleanup
  !sudo apt autoremove

  #link to the old google package
  !ln -s /usr/local/lib/python3.8/dist-packages/google /usr/local/lib/python3.10/dist-packages/google
  #this is just to verify if 3.10 folder was indeed created
  !ls /usr/local/lib/python3.10/

  #restart environment so you don't have to do it manually
  os.kill(os.getpid(), 9)

@PavelGorbanj : This just stalls here for me:

WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be REMOVED:
  libnvidia-common-460
0 upgraded, 0 newly installed, 1 to remove and 20 not upgraded.
After this operation, 35.8 kB disk space will be freed.
(Reading database ... 125030 files and directories currently installed.)
Removing libnvidia-common-460 (460.106.00-0ubuntu1) ...

[unnir]

just checked, have the same issue as @KelSolaar

Before, it worked perfectly...

[pawel-smth]

@unnir , @KelSolaar or anyone in the future - just in case, if I understand it correctly, the thread is not the support thread for the script I've posted. You're free to debug/update it to your particular needs (just like I did for mine) and I suppose others will be thankfull if you post what works for you

[wenkesj]

This worked for me to install 3.9 on a 3.8 instance:

!sudo apt-get update -y && sudo apt-get install python3.9 python3.9-dev python3.9-distutils libpython3.9-dev python3.9-venv

# Link python3 to 3.9.
!sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 1
!sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 2

# Install pip.
!curl -sS https://bootstrap.pypa.io/get-pip.py | python3.9

# Install your dependencies. Pin IPython and traitlets versions, as traitlets version 5.8 contains a breaking change that prevents google-colab server extension from working properly.
# Some of these packages may not be required.
!python3 -m pip install ipython==7.9.0 traitlets==5.7.1 jupyter psutil setuptools ipython_genutils ipykernel jupyter_console prompt_toolkit httplib2 astor

# Copy the already installed `google-colab` package over. It depends on an _old version_ of pandas, so building the wheel won't work.
!ln -s /usr/local/lib/python3.8/dist-packages/google \
       /usr/local/lib/python3.9/dist-packages/google

# Reinstall jupyter server extension.
!jupyter serverextension list
!jupyter notebook --config=/usr/local/etc/jupyter/jupyter_notebook_config.json

Restart your runtime, then run:

import sys
print(sys.version)

It should output something like:

3.9.16 ...

Please note that this may not work tomorrow or the next day, as this is just a hack. Hopefully the Colab team could learn from some of the migration challenges that we get around with some of these hacks 🤷. I am only posting this to share support for those who are eager to work with 3.9 like I am 😄

[metrizable]

@orome Thanks for filing the request. We have upgraded Colab's runtime to Python 3.9 (#3450) with plans to upgrade to modern versions in the future.

[orome]

@metrizable Good to know. We're on 3.11 now so — will have to keep waiting.

[metrizable]

@orome One more version closer :). We've upgraded Colab's runtime to Python 3.10 (#3572) and we'll keep cadence with final regular bug fix releases with binary installers for Python versions (like 3.11) in the future.