42035

is it possible to get the R survey package's `svyby` function multicore= parameter working on w

Question:

being able to multithread on windows would be awesome, but perhaps this problem is harder than i had thought.. :(

inside of survey:::svyby.default there is a a block that's either lapply or mclapply depending on multicore=TRUE and your operating system. windows users get forced into the lapply loop no matter what, and i was wondering if there's any way to go down the mclapply path instead.. speeding up the computation.

i don't know too much about the innards of parallel processing, but i did some experiments to see if any of the windows-acceptable alternatives would work. first i tried overwriting mclapply with

mclapply <- function( X , FUN , ... ){ clusterApply( x = X , fun = FUN , cl = makeCluster( detectCores() ) , ... ) }

next i used fixInNamespace( svyby.default , "survey" ) to remove the line if (multicore) parallel:::closeAll()

but that only got me to the point where

> svyby(~api99, ~stype, dclus1, svymean , multicore=TRUE ) Error in checkForRemoteErrors(val) : 3 nodes produced errors; first error: object 'svymean' not found

Answer1:

quoting Dr. Thomas Lumley, author of the R survey package in response to my inquiry--

<blockquote>

No. This approach to parallelising relies on forking, which Windows doesn't support.

It would be necessary to rewrite it to use clusterApply(), and I'm pretty sure the communications overhead would eat the speed gain. With forking, the child process gets a copy of the parent process data for free -- it's all done by the virtual<->physical memory translation hardware -- but with the cluster approach R has to send data to the child process explicitly.

</blockquote>

Recommend

  • Silverlight 2 and large canvases?
  • R lattice 3d plot: ticks disappear when changing panel border thickness
  • pseudocolors in R
  • How to use XSL:Key to get the following-sibling in XSLT 2.0
  • midpoints returned by barplot function do not actually line up with midpoints of bars
  • In zend, how to print a mysql query properly? [duplicate]
  • Parallel cosine distance using clusterapply in R
  • Remove PNG plot margins
  • Correct use of Realloc
  • Javascript bubble sort of an array with objects
  • JQuery animation: Is it possible to change speed during the animation?
  • Android: Alternative to onSensorChanged? no ACCELEROMETER data when phone is lying on its back on a
  • Non alpha character arrowlabels on a diagram
  • Share session on subdomains in php
  • Is TWebBrowser dependant on IE version?
  • What is the difference between CacheStoreMode USE and REFRESH
  • Plot ROC curve and calculate AUC in R at specific cutoff info
  • several hexbin plots on one PDF page in R
  • Zipfile python module bytesize difference
  • Indexing k-d tree?
  • How to debug 'value of ESP was not saved across function call' error?
  • Error: java.util.Arrays$ArrayList cannot be cast to java.util.ArrayList
  • Is there a way to read an Excel file using Dataflow
  • Using Python objects in C++
  • How Can I Prevent Activation For Some ListView Items When The Selection Mode Is MultiChoiceModal?
  • TensorFlow C++, runtime issue
  • iOS - Is this a task for enums?
  • OpenMP and File I/O
  • configure: error: no acceptable C compiler found in $PATH
  • Do create extension work in single-user mode in postgres?
  • R: gsub and capture
  • jqPlot EnhancedLegendRenderer plugin does not toggle series for Pie charts
  • Data Validation Drop Down Box Arrow Disappearing
  • Comma separated Values
  • IndexOutOfRangeException on multidimensional array despite using GetLength check
  • costura.fody for a dll that references another dll
  • Observable and ngFor in Angular 2
  • UserPrincipal.Current returns apppool on IIS
  • java string with new operator and a literal
  • How to load view controller without button in storyboard?