Suppose I have the following list:
rays_all = [np.array(r11, r21, r31, r41), np.array(r12, r22, r32, r42), np.array(r13, r23, r33, r43), np.array(r14, r24, r34, r44)]
all the r11, r21, r31, etc are arrays with shape (3L,) (think of it as a vector in 3D space).
If I want to extract the (4L,3L) array of
np.array(r14, r24, r34, r44), I just simply use
rays_all[-1]. If I want to append a new array of
np.array(r15, r25, r35, r45), I just use
Now I arrange the above vectors (r11,r12, etc) in an alternative way:
ray1 = [r11, r12, r13, r14] ray2 = [r21, r22] ray3 = [r31, r32, r33] ray4 = [r41, r42, r43, r44]
Each 'ray' now has its own list with different lengths. If I want to extract the last element of each list in an array structure, i.e.
np.array([r14,r22,r33,r44]), what is the most efficient way to do so? On the other hand, if I want to add the elements in the array
np.array([r15,r23,r34,r45]) to the list such that I will have
ray1 = [r11, r12, r13, r14, r15] ray2 = [r21, r22, r23] ray3 = [r31, r32, r33, r34] ray4 = [r41, r42, r43, r44, r45]
what is the most efficient way? I know I can just a loop to do so, but I guess it is much slower than the
rays_append()? Are there any 'vectorized' way of doing this?
Be careful with mixing array and list operations.
Make some 3 element arrays and combine them as in your first case:
In : r1,r2,r3,r4=np.arange(3),np.ones(3),np.zeros(3),np.arange(3)[::-1] In : x1=np.array((r1,r2)) In : x2=np.array((r3,r4)) In : rays=[x1,x2] In : rays Out: [array([[ 0., 1., 2.], [ 1., 1., 1.]]), array([[ 0., 0., 0.], [ 2., 1., 0.]])]
rays is now a list contain two 2d array ((2,3) shape). As you say, you can select an item from that list or append another array to it (you can append anything to it, not just a similar array). Operations of
rays are list operations.
You could also create a 3d array:
In : ray_arr=np.array((x1,x2)) In : ray_arr Out: array([[[ 0., 1., 2.], [ 1., 1., 1.]], [[ 0., 0., 0.], [ 2., 1., 0.]]]) In : ray_arr.shape Out: (2, 2, 3) In : ray_arr[-1] Out: array([[ 0., 0., 0.], [ 2., 1., 0.]])
You can select from
ray_arr as with the list. But appending requires creating a new array via
np.concatenate (possibly hidden in the
np.append function). No 'in-place' append as on a list.
Efficient selection of the last elements of all component arrays, by indexing on the last dimension.
In : ray_arr[:,:,-1] Out: array([[ 2., 1.], [ 0., 0.]])
To get the corresponding values from the list
rays you have to a list comprehension (or other loop):
In : [r[:,-1] for r in rays] Out: [array([ 2., 1.]), array([ 0., 0.])]
There's no indexing shortcut as with arrays.
There are tools like
zip (and others in
itertools) that help you iterate through lists, and even rearrange values, e.g.
In : list(zip(['a','b'],['c','d'])) Out: [('a', 'c'), ('b', 'd')] In : list(zip(['a','b'],['c','d']))[-1] Out: ('b', 'd')
and with ragged sublists:
In : list(zip(['a','b','c'],['d'])) Out: [('a', 'd')] In : list(itertools.zip_longest(['a','b','c'],['d'])) Out: [('a', 'd'), ('b', None), ('c', None)]
But I don't see how those will help with extracting values from your ray vectors.<hr />
Something worth exploring is to collect the base vectors into one 2d array, and use indexing to extra groups for various purposes,
In : allrays=np.array([r1,r2,r3,r4]) In : allrays Out: array([[ 0., 1., 2.], [ 1., 1., 1.], [ 0., 0., 0.], [ 2., 1., 0.]])
The 'z' coor for all rays
In : allrays[:,-1] Out: array([ 2., 1., 0., 0.])
One subset of rays (since it is a slice it is a view)
In : allrays[0:2,:] Out: array([[ 0., 1., 2.], [ 1., 1., 1.]])
In : allrays[2:,:] Out: array([[ 0., 0., 0.], [ 2., 1., 0.]])
3 item subset, selected with a list - this is a copy
In : allrays[[0,1,2],:] Out: array([[ 0., 1., 2.], [ 1., 1., 1.], [ 0., 0., 0.]]) In : allrays[,:] Out: array([[ 2., 1., 0.]])
several subsets obtained by indexing:
In : ind=[[0,1,2],] In : [allrays[i] for i in ind] Out: [array([[ 0., 1., 2.], [ 1., 1., 1.], [ 0., 0., 0.]]), array([[ 2., 1., 0.]])]
If the groups are contiguous, you can use
In : np.split(allrays,) Out: [array([[ 0., 1., 2.], [ 1., 1., 1.], [ 0., 0., 0.]]), array([[ 2., 1., 0.]])]
The subarrays are views (check with the
It does, in effect, just move the ragged list problem up a level. Still, there is more flexibility. You could construct other indexing sublists, e.g.
In : ind1=[i[-1] for i in ind] # last of all groups In : ind1 Out: [2, 3] In : ind2=[i for i in ind] # first of all groups In : ind2 Out: [0, 3]
You could concatenate some new values on to
allrays. You may then have to rebuild the indexing lists. But I suspect this sort of building is done one, while access is repeated.
An earlier SO question about accessing values from the
img produced by
plt.pcolor) comes to mind. One maintains an image as a surface on the 2d mesh, the other, more general, is just a collection of quadrilaterals, each with a color and path defining its boundary.
In answer to your specific question, a list containing the last element of the four "ray" lists is, in general
Since your main concern here seems to be execution speed, I assume you have to perform this operation over and over again. Have you considered creating a little data structure that represents the last element, say,
last_element = [r1x,r2x,r3x,r4x] and maintaining its value as you step through the problem? Each time you change
last_element you append new data to the other lists as necessary. In other words, instead of repeatedly extracting the last element from the big lists, build the big lists step-by-step from the last element. That would have to be more efficient as long as you've got to build those big lists anyway. Would it work for your problem?