8758

Logical indices in numpy throwing exception [duplicate]

<div class="question-status question-originals-of-duplicate">

This question already has an answer here:

    <li> Logical indexing in Numpy with two indices as in MATLAB <span class="question-originals-answer-count"> 1 answer </span> </li> </ul>

    I am trying to write some code that uses logical numpy arrays to index a larger array, similar to how MATLAB allows array indexing with logical arrays.

    import numpy as np m = 4 n = 4 unCov = np.random.randint(10, size = (m,n) ) rowCov = np.zeros( m, dtype = bool ) colCov = np.ones( n, dtype = bool ) >>> unCov[rowCov, rowCov] [] # as expected >>> unCov[colCov, colCov] [0 8 3 3] # diagonal values of unCov, as expected >>> unCov[rowCov, colCov] ValueError: shape mismatch: objects cannot be broadcast to a single shape

    For this last evaluation, I expected an empty array, similar to what MATLAB returns. I'd rather not have to check rowCov/colCov for True elements prior to indexing. Why is this happening, and is there a better way to do this?

    Answer1:

    As I understand it, numpy will translate your 2d logical indices to actual index vectors: arr[[True,False],[False,True]] would become arr[0,1] for an ndarray of shape (2,2). However, in your last case the second index array is full False, hence it corresponds to an index array of length 0. This is paired with the other full True index vector, corresponding to an index array of length 4.

    From the <strong>numpy</strong> manual:

    If the index arrays do not have the same shape, there is an attempt to broadcast them to the same shape. If they cannot be broadcast to the same shape, an exception is raised:

    In your case, the error is exactly due to this:

    --------------------------------------------------------------------------- IndexError Traceback (most recent call last) <ipython-input-1411-28e41e233472> in <module>() ----> 1 unCov[colCov,rowCov] IndexError: shape mismatch: indexing arrays could not be broadcast together with shapes (4,) (0,)

    MATLAB, on the other hand, automatically returns an empty array if the index array is empty along any given dimension.

    <hr>

    This actually highlights a fundamental difference between the logical indexing in MATLAB and numpy. In MATLAB, vectors in subscript indexing always slice out a subarray. That is, both

    arr([1,2],[1,2])

    and

    arr([true,true],[true,true])

    will return the 2 x 2 submatrix of the matrix arr. If the logical index vectors are shorter than the given dimension of the array, the missing indexing elements are assumed to be false. Fun fact: the index vector can also be longer than the given dimension, as long as the excess elements are all false. So the above is also equivalent to

    arr([true,true,false,false],[true,true])

    and

    arr([true,true,false,false,false,false,false],[true,true])

    for a 4 x 4 array (for the sake of argument).

    In numpy, however, indexing with boolean-valued numpy arrays in this way will try to extract a vector. Furthermore, the boolean index vectors should be the same length as the dimension they are indexing into. In your 4 x 4 example,

    unCov[np.array([True,True]),np.array([True,True])]

    and

    unCov[np.array([True,True,False,False,False]),np.array([True,True,False,False,False])]

    both return the two first diagonal elements, so not a submatrix but rather a vector. Furthermore, they also give the less-then-encouraging warning along the lines of

    /usr/bin/ipython:1: VisibleDeprecationWarning: boolean index did not match indexed array along dimension 0; dimension is 4 but corresponding boolean dimension is 5

    So, in numpy, your logical indexing vectors should be the same length as the corresponding dimensions of the ndarray. And then what I wrote above holds true: the logical values are translated into indices, and the result is expected to be a vector. The length of this vector is the number of True elements in every index vector, so if your boolean index vectors have a different number of True elements, then the referencing doesn't make sense, and you get the error that you get.

Recommend

  • Microsoft.Build.Evaluation.Project add folder and file force refresh of project in VS
  • Is a single constant value considered an expression?
  • SSLException: Connection has been shutdown: javax.net.ssl.SSLException: Tag mismatch
  • Howto count a variable inside of innerHTML?
  • How can I get new CSRF token in LARAVEL by using ajax
  • How to distinguish between attribute and element nodes returned from a Saxon XPathSelector
  • How to wait till webViewDidFinishLoad gets completed
  • What is lua_len() alternative in Lua 5.1?
  • How can I merge my files when the folder structure has changed using Borland StarTeam?
  • Cythonized function unexpectedly slow
  • How to reshape a 3D numpy array?
  • How to change placeholder text in an autocomplete activity of android google place?
  • Filter strings with regex before casting to numeric
  • Overlapping controls in Windows XP
  • Jackson Parser: ignore deserializing for type mismatch
  • MongoDb aggregation
  • How to use remove-erase idiom for removing empty vectors in a vector?
  • How to avoid particles glitching together in an elastic particle collision simulator?
  • Recording logins for password protected directories
  • Repeat a vertical line on every page in Report Builder / SSRS
  • Why is an OPTIONS request sent to the server?
  • Splitting given String into two variables - php
  • Nant, Vault & Windows Integrated Authentication
  • Why HTML5 Canvas with a larger size stretch a drawn line?
  • TFS: Get latest causes slow project reloading
  • How to convert from System.Drawing.Color to Excel.ColorFormat in C#? Change comment color
  • Controls, properties, events and timers running in design time
  • Why doesn't :active or :focus work on text links in webkit? (safari & chrome)
  • Check if a string to interpolate provides expected placeholders
  • Fill an image in a square container while keeping aspect ratio
  • When should I choose bucket sort over other sorting algorithms?
  • Rearranging Cells in UITableView Bug & Saving Changes
  • How do I rollback to a specific git commit
  • Unanticipated behavior
  • Windows forms listbox.selecteditem displaying “System.Data.DataRowView” instead of actual value
  • sending mail using smtp is too slow
  • Busy indicator not showing up in wpf window [duplicate]
  • Why is Django giving me: 'first_name' is an invalid keyword argument for this function?
  • Reading document lines to the user (python)
  • How can I use `wmic` in a Windows PE script?