60812

Powershell v2.0 Using multiple threads

Question:

Basic script idea: Hello. I've created a powershell script which I use to check the filesizes of certain executables, and then keep them in a text file. Next time the script runs, if a filesize differs it will replace the one in the text file with the new one.

The structure: I have a main script and a folder which contains many scripts, each for every executable of which I want to check the filesize. So the scripts in the folder will return a string containing the link to the executable, which will be fed to the main script.

The code:

$progdir = "C:\script\programms" $items = Get-ChildItem -filter *.ps1 -Path $progdir $webclient = New-Object System.Net.WebClient $filesizes = get-content C:\updatechecker\programms\filesizes if ($filesizes.length -ne $items.length) { if ($filesizes.length -eq $null) { Write-Host ("Building filesize database...") -nonewline } else { Write-Host ("Rebuilding filesize database...") -nonewline } clear-content C:\programms\filesizes for ($i=0; $i -le $items.length-1; $i++) { $command = "c:\programms\" + $items[$i].name $link = & $command $webclient.OpenRead($link) | Out-Null $filesize = $webclient.ResponseHeaders["Content-Length"] $filesize >> C:\programms\filesizes } echo "Done." } else { ...

Question: This for loop is the one I want to run in parallel. I need your advice on how to do this since I'm new to powershell. I tried to implement a few things I found but they didn't work correctly (took very long to finish, output errors, multiple entries of filesizes in my filesizes file). I suspect it's a synchronization issue and somehow I need to lock the critical parts. Isn't there anything like omp parallel for in powershell? :P

Any help,advice on how to achieve this would be appreciated :)

edit:

Get-Job | Remove-Job -Force $progdir = "C:\programms" $items = Get-ChildItem -filter *.ps1 -Path $progdir $webclient = New-Object System.Net.WebClient $filesizes = get-content C:\programms\filesizes $jobWork = { param ($MyInput) $command = "c:\programms\" + $MyInput $link = & $command $webclient.OpenRead($link) | Out-Null $filesize = $webclient.ResponseHeaders["Content-Length"] $filesize >> C:\programms\filesizes } foreach ($item in $items) { Start-Job -ScriptBlock $jobWork -ArgumentList $item.name | out-null } Get-Job | Wait-Job Get-Job | Receive-Job | Out-GridView | out-null echo "Done."

Edit 2: Used code I found here: <a href="http://ryan.witschger.net/?p=22" rel="nofollow">http://ryan.witschger.net/?p=22</a>

$mutex = new-object -TypeName System.Threading.Mutex -ArgumentList $false, “RandomGlobalMutexName”; $MaxThreads = 4 $SleepTimer = 500 $jobWork = { param ($MyInput) $webclient = New-Object System.Net.WebClient $command = "c:\programms\" + $MyInput $link = & $command $webclient.OpenRead($link) | Out-Null $result = $mutex.WaitOne(); $file = $webclient.ResponseHeaders["Content-Length"] $file >> C:\programms\filesizes $mutex.ReleaseMutex(); } $progdir = "C:\programms" $items = Get-ChildItem -filter *.ps1 -Path $progdir $webclient = New-Object System.Net.WebClient $filesizes = get-content C:\programms\filesizes Get-Job | Remove-Job -Force $i = 0 ForEach ($item in $items){ While ($(Get-Job -state running).count -ge $MaxThreads){ Start-Sleep -Milliseconds $SleepTimer } $i++ Start-Job -ScriptBlock $jobWork -ArgumentList $item.name | Out-Null }

Answer1:

You can run each iteration of the loop in a background job which is not the same a seperate thread in that it is a whole other PowerShell.exe process. Data is passed from the background processes through serialization.

To approach it using background jobs you'll need to define a script block that will do that actual work and then call the script block with parameters in each iteration of the loop. The script block can report back status via Write-Output or by throwing an exception.

You'll probably want to throttle how many concurrent background jobs are running. Here's an example of how to throttle:

$jobItems = "a", "b", "c", "d", "e" $jobMax = 2 $jobs = @() $jobWork = { param ($MyInput) if ($MyInput -eq "d") { throw "an example of an error" } else { write-output "Processed $MyInput" } } foreach ($jobItem in $jobItems) { if ($jobs.Count -le $jobMax) { $jobs += Start-Job -ScriptBlock $jobWork -ArgumentList $jobItem } else { $jobs | Wait-Job -Any } } $jobs | Wait-Job

As an alternative you might try eventing. Take a look at this thread for some examples of how to implement concurrency using events.

<a href="https://stackoverflow.com/questions/4926060/powershell-runspace-problem-with-downloadfileasync" rel="nofollow">PowerShell: Runspace problem with DownloadFileAsync</a>

You might be able to replace DownloadFileAsync with <a href="http://msdn.microsoft.com/en-us/library/ms144211.aspx" rel="nofollow">OpenReadAsync</a>

Recommend

  • Can we differentiate between frame and non-frame loads with Ci in firefox addon
  • Getting Data from Fetch Post that returns response data
  • NUnit crashing with NHibernate
  • Use Webclient to download several files
  • FTP upload corrupting files
  • Unzipping a file error
  • Zipping Files and Downloading them Using SaveAs Dialog
  • Extract zip entries to another Zip file
  • Stop Bash Script if Hive Fails
  • How to add git credentials to the build so it would be able to be used within a shell code?
  • Mysterious problem with floating point in LISP - time axis generation
  • How to know when stdin is empty if it contains EOF?
  • Ajax Loaded meta Tags
  • Xamarin Forms - UWP Fonts
  • DotNetZip - Calculate final zip size before calling Save(stream)
  • Arrow is showed instead of the material design version hamburger icon. Why doesn't syncState in
  • Display Images one by one with next and previous functionality
  • Upload files with Ajax and Jquery
  • Timeout for blocking function call, i.e., how to stop waiting for user input after X seconds?
  • Warning: Can't call setState (or forceUpdate) on an unmounted component
  • SVN: Merging two branches together
  • Hibernate gives error error as “Access to DialectResolutionInfo cannot be null when 'hibernate.
  • Arrays break string types in Julia
  • retrieve vertices with no linked edge in arangodb
  • Linker errors when using intrinsic function via function pointer
  • Windows forms listbox.selecteditem displaying “System.Data.DataRowView” instead of actual value
  • Load html files in TinyMce
  • Free memory of cv::Mat loaded using FileStorage API
  • FormattedException instead of throw new Exception(string.Format(…)) in .NET
  • How to CLICK on IE download dialog box i.e.(Open, Save, Save As…)
  • Change div Background jquery
  • Can Visual Studio XAML designer handle font family names with spaces as a resource?
  • LevelDB C iterator
  • apache spark aggregate function using min value
  • Reading document lines to the user (python)
  • Observable and ngFor in Angular 2
  • How can i traverse a binary tree from right to left in java?
  • How to Embed XSL into XML
  • UserPrincipal.Current returns apppool on IIS
  • Conditional In-Line CSS for IE and Others?