26591

AssertionError: Multiple .dist-info directories on Data Science Experience

Question:

In a Python 3.5 notebook, backed by an Apache Spark service, I had installed BigDL 0.2 using pip. When removing that installation and trying to install version 0.3 of BigDL, I get this error: (linebreaks added for readability)

AssertionError: Multiple .dist-info directories: /gpfs/fs01/user/scbc-4dbab79416a6ec-4cf890276e2b/.local/lib/python3.5/site-packages/BigDL-0.3.0.dist-info, /gpfs/fs01/user/scbc-4dbab79416a6ec-4cf890276e2b/.local/lib/python3.5/site-packages/BigDL-0.2.0.dist-info

However, neither of these directories exists:

!ls -al /gpfs/fs01/user/scbc-4dbab79416a6ec-4cf890276e2b/.local/lib/python3.5/site-packages/ total 0 drwx------ 2 scbc-4dbab79416a6ec-4cf890276e2b users 4096 Nov 8 06:12 . drwx------ 3 scbc-4dbab79416a6ec-4cf890276e2b users 4096 Nov 8 06:12 ..

What's wrong? How can I install the new version of the package after removing the old one?

Answer1:

The directory paths in the error message are wrong. The Python 3.5 kernel on DSX specifies a build directory for pip by setting the environment variable PIP_BUILD. The multiple dist-info directories are there:

!printenv PIP_BUILD ; ls -l $PIP_BUILD/* /tmp/scbc-4dbab79416a6ec-4cf890276e2b/pip-build total 0 drwx------ 8 scbc-4dbab79416a6ec-4cf890276e2b users 117 Nov 7 02:02 bigdl drwx------ 2 scbc-4dbab79416a6ec-4cf890276e2b users 135 Nov 7 02:02 BigDL-0.2.0.dist-info drwx------ 2 scbc-4dbab79416a6ec-4cf890276e2b users 135 Nov 8 06:12 BigDL-0.3.0.dist-info

To fix the problem, remove the build directory:

!rm -rf $PIP_BUILD

After that, pip can install the package without problems:

!pip install --no-dependencies bigdl==0.3 Collecting bigdl==0.3 Using cached BigDL-0.3.0-py2.py3-none-manylinux1_x86_64.whl Installing collected packages: bigdl Successfully installed bigdl

Recommend

  • Covariance of Box type in Rust
  • Detecting foreign words
  • How can I grep through an array, while filtering out matches?
  • Python: Removing negatives from a list of numbers
  • Why does ng-mouseover not work with ng-if
  • Runtime exception when calling dll function with parameter in Inno Setup
  • How Align the Legend Items in Chart.js 2?
  • PHPUnit: mock all methods except some
  • How to set memory limit to ElasticSearch in Windows?
  • Why is this Python code running twice? [duplicate]
  • How to find out if a lazy relation isn't loaded yet, with SQLAlchemy?
  • Warning in Resharper “Return value of pure method is not used”
  • std::numeric_limits::digits10 = number of digits on the right or left to the radix point? [closed]
  • Iterator Inheritance and inheriting *this
  • MYSQL + PHP to display data from two rows as a single column
  • Combining javascript / jQuery and PHP
  • hide column based on cell font color in a row vba
  • Creating Actions for Personal use only
  • Using COUNT returns different results
  • Codename One - container.setScrollVisible(true) in the center of a BorderLayout
  • How to pull json data using java
  • How to loop through a combination only once?
  • How to get selected text using android spinner wheel?
  • Swift dictionary map - init in closure
  • How deleteLater() actually works in qt?
  • How to read config files with section in bash shell
  • Preventing Internet-accessing-method from delaying a toast popup
  • How to open a serial port with pyserial?
  • How to Export CSV file from ASP.NET core
  • How to populate a Mutation for a different Cassandra table in a trigger
  • Read specific column from Parquet without using Spark
  • How can i use JQuery fadeTo() in IE 7?
  • $this->a->b->c->d calling methods from a superclass in php
  • dmtracedump doesn't work, HELP!
  • Find all parks for a given zipcode with google maps
  • Computing the discrete fourier transform of audio data with FFTW
  • Using Service Component Runtime
  • convert json to excel in java
  • ReferenceError: TextEncoder is not defined
  • Create/delete users from text file using Bash script