45344

Julia: use of pmap with matrices

Question:

I have a question about the use of pmap. I think it's a simple/obvious answer but still can't figure it out! I am currently running a loop where each of 50 iterations is separate and so running it in parallel should be possible and should improve speed. It uses a function that has multiple inputs and outputs, which are both a mixture of vectors and scalars. I need to save the outputs of the function for each of the 50 iterations for later use. Here are the basics of the code when not in parallel.

A=Array(Float64, 500,50) b=Array(Float64,50) for i in 1:50 A[:,i],b[i] = func(i,x,y,z) end

Any advice for how to implement this is parallel? I'm using v0.3 Julia.

Thanks in advance.

David

Answer1:

This worked for me.

@everywhere x,y,z = 1,2,3 @everywhere function f(i,x,y,z) sleep(1) return(ones(500)*i, i+x+y+z) end naive = @time map(i -> f(i,x,y,z), 1:50) parallel = @time pmap(i -> f(i,x,y,z), 1:50) A = [x[1] for x in parallel] b = [x[2] for x in parallel]

Let me know if anyone can suggest a more elegant way to get A and b out of the array of tuples that is produced by pmap.

The timings (when run on 8 processes) are as we would expect

elapsed time: 5.063214725 seconds (94436 bytes allocated) elapsed time: 0.815228485 seconds (288864 bytes allocated)

Recommend

  • Can a Matlab PARFOR loop be programmatically switched on/off?
  • Python - Difference Between Windows SystemParametersInfoW vs SystemParametersInfoA Function
  • Write a cell array in text file using Matlab
  • Hide characters in email address using an SQL query
  • Object Detection API Assertion failed: [maximum box coordinate value is larger than 1.01: ] for resn
  • Numpy: conditional sum
  • Creating multicolumn index in PostgreSQL, containing both scalar and array columns
  • Why does my GradientDescentOptimizer produce NaN?
  • Hash function to matrix
  • Using ifelse with random variate generation in a function applied to a vector
  • Convert from DataFrame to JavaPairRDD
  • Making a vectorized numpy function behave like a ufunc
  • Type definition: expected UnionAll, got TypeVar
  • Python error: len() of unsized object while using statsmodels with one row of data
  • How to access UIViewController's varaibles inside “func applicationWillResignActive”? Swift, iO
  • Auto Height of UICollectionView inside UITableViewCell
  • Convert Func to Func
  • Implementing HMAC-SHA256 for Keybase in Javascript
  • Program crashes when run outside IDE
  • Calling java project from Mathematica
  • Cythonized function unexpectedly slow
  • PHP file_exists() anomaly
  • Custom Nav Title offset ios 11
  • Most efficient way to replace lowest list values in dataframe in R
  • Insertion large number of Entities into SQL Server 2012 [duplicate]
  • Efficient & Pythonic way of finding all possible sublists of a list in given range and the minim
  • Recording values of radio buttons in ember
  • Add reference to ASP.NET 5 Class Library from Framework 4.5 Class Library Project
  • C#: Import/Export Settings into/from a File
  • How can I set a binding to a Combox in a UserControl?
  • AJAX Html Editor Extender upload image appearing blank
  • Functions in global context
  • Repeat a vertical line on every page in Report Builder / SSRS
  • Update CALayer sublayers immediately
  • Android screen density dpi vs ppi
  • Bug in WPF DataGrid
  • In LanguageTool, how do you create a dictionary and use it for spell checking?
  • Matrix multiplication with MKL
  • XCode 8, some methods disappeared ? ex: layoutAttributesClass() -> AnyClass
  • java string with new operator and a literal