Install Python Packages From Azure DevOps

Python Package Index pypi.org is not the only source for downloading python packages. It’s possible to host your own package index.

⟶

A Common Problem

Companies may hinder (knowingly or unknowingly) their employees productivity with restrictive policies and stacked layers of security. A common problem I face is the inability to install python dependencies using pip:

➜ pip install sklearn
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /simple/sklearn/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /simple/sklearn/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /simple/sklearn/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /simple/sklearn/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /simple/sklearn/
ERROR: Could not find a version that satisfies the requirement sklearn (from versions: none)
ERROR: No matching distribution found for sklearn

Solution: tldr.

And Another Challenge

Sharing code between different teams and departments in a decoupled but secure and private way.

The Solution

Using Azure DevOps Feeds, developers can publish, download and install public and private python packages, while security teams have the ability to configure upstream behavior, eliminating the need for developers to invent their own (potentially risky) ways of sharing/installing dependencies.

Create a devops project with a feed.
Enable “Upstream sources” (pypi.org for example):
Install keyring in order to connect to the DevOps feed:
```
pip install keyring artifacts-keyring
```
Got an error?

Try and install any package that you need using the feed url:

pip install sklearn \
  --index-url https://pkgs.dev.azure.com/<project>/<id>/_packaging/<feed>/pypi/simple/

Update your pip.ini file to avoid having to pass the index url:
```
pip config -v list
```
Open one of the above ini files, depending on your setup, and enter:
```
[global]
index-url = https://pkgs.dev.azure.com/<project>/<id>/_packaging/<feed>/pypi/simple/
```

Creating and Publishing Packages

Have a look at this post to see how you can create and publish packages to a feed.

Error

If you got the following error:

WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /simple/keyring/

Download keyring and artifacts keyring on a different computer:

pip download keyring artifacts-keyring -d dependencies
ls dependencies

Copy over, and install the .whl files on the computer that gives the error:

cd dependencies
pip install *.whl

Now proceed and install any other dependencies through our earlier created Azure DevOps Feed:

pip install sklearn \
  --index-url https://pkgs.dev.azure.com/<project>/<id>/_packaging/<feed>/pypi/simple/

If you’re still stuck, please let me know in the comments below.

Howto