Start-RSJob is not running code as multi-threaded

[Scott Bass]

Do you want to request a feature or report a bug?
Bug

What is the current behavior?
I have a PS script which updates a SQL Server table via the BulkCopy class. I want to update 34 tables concurrently. The script creates a log file. I would expect all 34 jobs to start at once, with 34 log files created at roughly the same time. However the jobs appear to be running single threaded, i.e. consecutively not concurrently.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

$ScriptBlock = {
    Param($Name)
    $x=(Get-Random -min 45 -max 90)
    $m="$Name::$x"
    $m | Out-File C:\Temp\RSJob_temp_${Name}.txt
    Sleep $x
    $m
}

'A','B','C','D','E','F','G' | Start-RSJob -ScriptBlock $ScriptBlock -Name {$_} -Throttle 10

What is the expected behavior?
I would expect each script invocation to run concurrently, with the log files created by the script to be created at roughly the same time. Instead, each log is created after the end of the previous script, i.e. the time stamps in the logs show they are running consecutively rather than concurrently.

Which versions of Powershell and which OS are affected by this issue? Did this work in previous versions of our scripts?

PS C:\Powershell\Scripts> $PSVersionTable

Name                           Value
----                           -----
CLRVersion                     2.0.50727.5466
BuildVersion                   6.1.7601.17514
PSVersion                      2.0
WSManStackVersion              2.0
PSCompatibleVersions           {1.0, 2.0}
SerializationVersion           1.1.0.1
PSRemotingProtocolVersion      2.1

Please provide a code example showing the issue, if applicable:
This is the code I've submitted (minor edits for privacy):

$ScriptBlock = {
    Param($Table)
    $local:sw = [Diagnostics.Stopwatch]::StartNew()
    \\server\path\RunAll_SqlServer.ps1 -Tables $Table
    $local:sw.Stop()
    "{0,-32} completed in {1,12:N2} seconds" -f $Table, $local:sw.Elapsed.TotalSeconds
}

. \\server\path\RunAll_SqlServerParameters.ps1
<#
This creates a $Tables array similar to this code:

$Tables = @()
$Tables+='TABLE_1'
$Tables+='TABLE_2'
...
$Tables+='TABLE_34'
#>

$Tables | Start-RSJob -ScriptBlock $ScriptBlock -Name {$_} -Throttle 50

Get-RSJob | Wait-RSJob
Get-RSJob | Receive-RSJob
Get-RSJob | Remove-RSJob

Perhaps I should remove -Throttle? Although I would expect a "high" throttle and no throttle to have the same effect.

Perhaps I'm using Wait-RSJob wrong? Although I would expect that the Start-RSJob function would start all 34 jobs, then the Wait-RSJob function would wait for all 34 jobs to complete.

[Scott Bass]

Also, compare the above behavior to this:

1. Download this script: http://www.get-blog.com/?p=189

2. Run this code:

$ScriptBlock = {
    Param($Name)
    $x=(Get-Random -min 45 -max 90)
    $m="$Name::$x"
    $m | Out-File C:\Temp\RSJob_temp_${Name}.txt
    Sleep $x
    $m
}

# convert script block to script - Run-CommandMultiThreaded does not work with script blocks
$ScriptBlock | Out-File C:\Temp\script.ps1

'A','B','C','D','E','F','G' | `
\\server\path\Run-CommandMultiThreaded.ps1 `
-Command C:\Temp\script.ps1 `
-InputParam Name `
-MaxThreads 50 `
-MaxResultTime $(24*60*60) `
-SleepTimer $(10*1000)

I've had some issues with this script, where threads are mysteriously dying. But, at least it runs multi-threaded as I would expect.

There are also some good ideas in here, such as the progress bar and configurable update interval on the status of the threads. Your scripts seem more robust and actively developed. But perhaps you can add some of the features from this script as enhancements. (I assume this script is in the public domain).

[Scott Bass]

I've made some progress since my initial post. I'll leave it to the development team as to whether this is a code bug, documentation bug, or end user error.

This sample code gives me what I want:

$ScriptBlock = {
    Param($Name)
    $x=(Get-Random -min 20 -max 60)
    $m="$Name::$x"
    $m | Out-File C:\Temp\RSJob_log_${Name}_${x}.txt
    Sleep $x
    $m
}

1..50 | Start-RSJob -ScriptBlock $ScriptBlock -Name {$_} -Throttle 50 | Wait-RSJob -ShowProgress | Receive-RSJob
Get-RSJob | Remove-RSJob

However, I find it weird that, with the below minimal change, the code runs single threaded rather than multi-threaded:

$ScriptBlock = {
    Param($Name)
    $x=(Get-Random -min 20 -max 60)
    $m="$Name::$x"
    $m | Out-File C:\Temp\RSJob_log_${Name}_${x}.txt
    Sleep $x
    $m
}

1..50 | Start-RSJob -ScriptBlock $ScriptBlock -Name {$_} -Throttle 50 
Get-RSJob | Wait-RSJob -ShowProgress | Receive-RSJob
Get-RSJob | Remove-RSJob

I would think that Start-RSJob returns RSJob objects to the pipeline, and returns control to the console. And Get-RSJob returns the same job objects as Start-RSJob.
So to me, the two approaches should be equivalent.

It's not intuitively obvious that piping from Start-RSJob to Wait-RSJob is required in order to get multi-threaded processing.

(I don't know if PS 2.0 is supported? But Win 7 / PS 2.0 is all I've got at work. Perhaps someone can run the above and see if there's different behavior in PS 3.0+.)

[Max Kozlov]

There is an error for PSv2 when displaying running job at host
you can workaround it with this simple method (your first script)

$jobs = 'A','B','C','D','E','F','G' | Start-RSJob -ScriptBlock $ScriptBlock -Name {$_} -Throttle 10

i.e. you should send results of Start-RSJob to variable, not host
or you can try my latest fork where I seem to have fixed this error

[Boe Prox]

@scottbass I believe that this should have been resolved with the PR #159. Can you verify this?