40229

Compute in a new thread and refer to results later in R

Question:

In clojure I can do something like this:

<pre class="lang-lisp prettyprint-override">(def x ;; perform some expensive computation in a new thread ;; the repl is not blocked, so you can go on do something else (future (do (Thread/sleep 500) 3.14))) ;; ... do something else ;; now when you need x ;; just deref the future, get 3.14 @x

Is there something similar to this in R?

Answer1:

On Linux you can fork a process and then collect it later, as illustrated on the help page ?parallel::mccollect(); this is more of a hack than a robust feature like future.

> p <- mcparallel({ Sys.sleep(5); "done" }) > sqrt(1:5) [1] 1.000000 1.414214 1.732051 2.000000 2.236068 > mccollect(p) $`15666` [1] "done"

One can implement a similar strategy with snow::sendCall() / snow::recvResult(); see the implementation of snow::clusterCall() and note that a cluster can be subset, e.g., cl[1] to set a single node to work. (identically named functions are available in the parallel package, but not exported).

To check whether a job has finished, use wait = FALSE. Example:

require(parallel) p1 = mcparallel({Sys.sleep(90); list(tag = "job1", res = 1}) p2 = mcparallel({Sys.sleep(80); 2}) p3 = mcparallel({Sys.sleep(60); 3}) res = mccollect(list(p1, p2, p3), wait = FALSE) is.null(res)

If none of these 3 jobs has been finished, then res is NULL.

Recommend

  • Has anyone used the “selection preferences” provided by MKS Yacc?
  • How to call a generic method with type constraints when the parameter doesn't have these constr
  • DCG and left recursion
  • Sql Copy table from One Database to Another Database
  • Lua - How to find a substring with 1 or 2 characters discrepancy
  • Programatically modify Eclipse workspace and CDT options from a plugin
  • How to check if a Gmail label has a nested sub-label?
  • How to start server for Selenium grid Java Maven setup
  • Number of threads being used during Parallel.ForEach
  • FTS3 searches in ORMLite?
  • CSS bleed-through with cfinput type=“datefield”
  • Android changing fragment order inside FragmentPagerAdapter
  • Abort upload large uploads after reading headers
  • java.lang.NoClassDefFoundError: com.parse.Parse$Configuration$Builder on below Lollipop versions
  • What is Eclipse's Declaration View used for?
  • Bug in WPF DataGrid
  • htaccess rewriting URLs with multiple forward slashes
  • How to show dropdown in excel using jrxml (jasper api)?
  • Importing jscolor library in angular 2
  • Display Images one by one with next and previous functionality
  • Release, debug version and Authorization Google?
  • Websockets service method fails during R startup
  • Alternatives to the OPTIONAL fallback SPARQL pattern?
  • Web-crawler for facebook in python
  • How to get next/previous record number?
  • Apache 2.4 - remove | delete | uninstall
  • A cron job substitute?
  • WPF Applying a trigger on binding failure
  • Proper way to use connect-multiparty with express.js?
  • Trying to get generic when generic is not available
  • how does django model after text[] in postgresql [duplicate]
  • embed rChart in Markdown
  • How to get Windows thread pool to call class member function?
  • IndexOutOfRangeException on multidimensional array despite using GetLength check
  • costura.fody for a dll that references another dll
  • Observable and ngFor in Angular 2
  • How to Embed XSL into XML
  • UserPrincipal.Current returns apppool on IIS
  • Conditional In-Line CSS for IE and Others?
  • java string with new operator and a literal