27902

Why parallel code is slower? [duplicate]

Question:

This question already has an answer here:

<ul><li> <a href="/questions/6036120/parallel-foreach-slower-than-foreach" dir="ltr" rel="nofollow">Parallel.ForEach Slower than ForEach</a> <span class="question-originals-answer-count"> 5 answers </span> </li> </ul>

I created a simple test with big arrays, and I get a big difference between parallel for and normal for

using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; namespace ConsoleApplication1 { class Program { static void Main(string[] args) { var rn = new Random(541); var tmStart = DateTime.Now; int[,] a = new int[2048, 2048]; for (int i = 0; i < 2048; i++) for (int j = 0; j < 2048; j++) a[i, j] = rn.Next(); var tmEnd = DateTime.Now - tmStart; Console.WriteLine("Normal..: {0}",tmEnd.ToString()); tmStart = DateTime.Now; a = new int[2048, 2048]; Parallel.For(0, 2048, i => { for (int j = 0; j < 2048; j++) a[i, j] = rn.Next(); }); tmEnd = DateTime.Now - tmStart; Console.WriteLine("Parallel: {0}", tmEnd.ToString()); } } }

The time for process:

<blockquote>

Normal..: 00:00:00.1250071<br /> Parallel: 00:00:00.3880222

</blockquote>

Why is that difference so big?

I imagined that if you use several threads, it would be faster...

Answer1:

There are a few things wrong with your benchmark:

<ol><li>Your tasks are so small that the threading overhead is significant.</li> <li>You break the threading rules with the instance of Random. Who knows what impact this has?</li> <li>Your timing is, well, crude.</li> </ol>

Here's a better benchmark that shows the parallel version of the code being faster:

using System; using System.Diagnostics; using System.Threading.Tasks; namespace ConsoleApplication1 { class Program { const int N = 1024; static void Main(string[] args) { int[,] a = new int[N, N]; var stopwatch = Stopwatch.StartNew(); for (int i = 0; i < N; i++) for (int k = 0; k < N; k++) for (int j = 0; j < N; j++) a[i, j] = i+j; Console.WriteLine("Normal..: {0}", stopwatch.ElapsedMilliseconds); stopwatch = Stopwatch.StartNew(); Parallel.For(0, N, i => { for (int k = 0; k < N; k++) for (int j = 0; j < N; j++) a[i, j] = i+j; }); Console.WriteLine("Parallel: {0}", stopwatch.ElapsedMilliseconds); Console.ReadLine(); } } }

And here's one which uses random number generators:

using System; using System.Diagnostics; using System.Threading.Tasks; namespace ConsoleApplication1 { class Program { const int N = 512; static void Main(string[] args) { int[,] a = new int[N, N]; var stopwatch = Stopwatch.StartNew(); var rng = new Random(); for (int i = 0; i < N; i++) for (int k = 0; k < N; k++) for (int j = 0; j < N; j++) a[i, j] = rng.Next(); Console.WriteLine("Normal..: {0}", stopwatch.ElapsedMilliseconds); stopwatch = Stopwatch.StartNew(); Parallel.For(0, N, i => { var rngpar = new Random(); for (int k = 0; k < N; k++) for (int j = 0; j < N; j++) a[i, j] = rngpar.Next(); }); Console.WriteLine("Parallel: {0}", stopwatch.ElapsedMilliseconds); Console.ReadLine(); } } }

Answer2:

When you parallelize tasks you have some "context switch" time spent from the SO to manage various tasks. If the single task isn't doing something computationally interesting you will not perceive any benefit from multithreading. You should choose some other example in which each task is eating some CPU cycles ( even a loop would do as an exercise purpose ).

Recommend

  • Extract IP address from HTML document
  • Threading lock in python not working as desired
  • Does argparse support multiple exclusive arguments?
  • Does sql server minds the way records where inserted?
  • java swing resize
  • What's the essential difference between these two variadic functions?
  • how can I key rotate for google cloud storage service account?
  • Using constants or global variables in 3 tier console application
  • Share two different things in Windows Phone 8.1 C#
  • Runtime.exec() gives Error: Could not find or load main class
  • Problem with Django using Apache2 (mod_wsgi), Occassionally is “unable to import from module” for no
  • Passing variable arguments using PowerShell's Start-Process cmdlet
  • If statement skipping right to else after being called once?
  • Scipy Leastsq Optional Output Variable (Mesg)
  • Differences in dis-assembled C code of GCC and Borland?
  • how to find common suffix in java by using method
  • Display java JPanel in a JFrame
  • How to Cache Real-time Data?
  • ActiveRecord query for a count of new users by day
  • Algorithm for a smudge tool?
  • Scrapy recursive link crawler
  • Spring Data JPA custom method causing PropertyReferenceException
  • Illegal mix of collations for operation for date/time comparison
  • NetLogo BehaviorSpace - Measure runs using reporters
  • Possible to stop flickering java tooltip in heavyweight mode?
  • How to handle AllServersUnavailable Exception
  • sending/ receiving email in Java
  • How to delete a row from a dynamic generate table using jquery?
  • Proper way to use connect-multiparty with express.js?
  • using HTMLImports.whenReady not working in chrome
  • How get height of the a view with gone visibility and height defined as wrap_content in xml?
  • FormattedException instead of throw new Exception(string.Format(…)) in .NET
  • IndexOutOfRangeException on multidimensional array despite using GetLength check
  • Authorize attributes not working in MVC 4
  • EntityFramework adding new object to nested object collection
  • Checking variable from a different class in C#
  • Sorting a 2D array using the second column C++
  • failed to connect to specific WiFi in android programmatically
  • java string with new operator and a literal
  • How can I use threading to 'tick' a timer to be accessed by other threads?