net user name password

DO NOT USE IT! It doesn’t change the password, it resets it. When you reset the password this way, all things encrypted with user password are no longer accessible (unless you have the certificate backed up and want to go with manual decryption process). This includes EFS, user specific encrypted app configs, passwords, some Outlook settings etc.

So how do you do it? If you search a little longer you’ll probably find PowerShell commandlets for WinAPI methods to change AD password. Most likely this is not what you want as you have local user.

What works for me is the following C# code:

using (PrincipalContext ctx = new PrincipalContext(ContextType.Machine))
{
	UserPrincipal user = UserPrincipal.FindByIdentity(ctx, IdentityType.SamAccountName, userName);
	user.ChangePassword(oldPassword, newPassword);
}

You need to add reference to System.DirectoryServices. Works on Windows Server 2012 R2 with .NET Framework 4.5.

Some people suggest you should call user.Save(), however, it throws the following exception for me:

System.UnauthorizedAccessException: Access is denied.
   at System.DirectoryServices.Interop.UnsafeNativeMethods.IAds.SetInfo()
   at System.DirectoryServices.DirectoryEntry.CommitChanges()
   at System.DirectoryServices.AccountManagement.SDSUtils.ApplyChangesToDirectory(Principal p, StoreCtx storeCtx, GroupMembershipUpdater updateGroupMembership, NetCred credentials, AuthenticationTypes authTypes)
   at System.DirectoryServices.AccountManagement.SAMStoreCtx.Update(Principal p)
   at System.DirectoryServices.AccountManagement.Principal.Save()

I don’t know if that’s needed but I do it anyway. What happens if you pass incorrect old password? You get

Unhandled Exception: System.DirectoryServices.AccountManagement.PasswordException: The specified network password is not correct.
 ---> System.Runtime.InteropServices.COMException: The specified network password is not correct.
   --- End of inner exception stack trace ---
   at System.DirectoryServices.AccountManagement.SDSUtils.ChangePassword(DirectoryEntry de, String oldPassword, String newPassword)
   at System.DirectoryServices.AccountManagement.SAMStoreCtx.ChangePassword(AuthenticablePrincipal p, String oldPassword, String newPassword)
   at System.DirectoryServices.AccountManagement.PasswordInfo.ChangePassword(String oldPassword, String newPassword)
   at System.DirectoryServices.AccountManagement.AuthenticablePrincipal.ChangePassword(String oldPassword, String newPassword)

You can try wrapping this in some PowerShell code and you should be good.

This is the eighth part of the Availability Anywhere series. For your convenience you can find other parts in the table of contents in Part 1 – Connecting to SSH tunnel automatically in Windows

Imagine that you’d like to run an interactive application on some remote server. This application should have access to UI and ideally pop-up in the session of logged-in user.
There is a psexec tool from Sysinternals and it can run applications remotely. However, it didn’t work for me (I guess some permission issues for remote shares) so I did it with PowerShell.

Run this:

$username = "DOMAIN\USER"
$pass = "PASSWORD"
$pass = convertto-securestring -asplaintext $pass -force
$cred = new-object System.Management.Automation.PSCredential -argumentlist $username, $pass
invoke-command -scriptblock { PATH_TO_SYSINTERNALS\.\psexec -accepteula -s -i 1 notepad.exe } -computername IP -credential $cred

In line 1 you specify username (with domain if needed).
In line 2 you provide a password.
Lines 3-4 create a secure password object.
Line 5 does the magic. First, it uses PowerShell Remoting to connect to the machine over IP with given credentials (see Part 3 how to configure PS Remoting). Next, it executes a command.
In the command we use psexec to run application interactively in session 1. We need to use -s parameter to run the app as a system account, otherwise it will not have an access to the UI. You may also use -u "DOMAIN\USER" -p "PASSWORD" of the session owner instead.

This is the third part of the Availability Anywhere series. For your convenience you can find other parts in the table of contents in Part 1 – Connecting to SSH tunnel automatically in Windows

Today we will enable PowerShell Remoting in Windows. This sounds like a trivial task (you can google up an “easy” solution) but every time I’m doing that it happens that I need more and more commands because there are more and more settings to tweak. This note tries to summarize everything I captured over the last 10 years.

Server configuration — for receiving connections

First, run elevated PS and run this:

enable-psremoting -force -SkipNetworkProfileCheck -confirm

Confirm everything and this in theory enables PS Remoting. It’s is not enough, though, because you need to allow multiple other things to get connections over IP (not using domain name) and without encryption (which we take care of using tunneling or VPN). Start with this:

netsh advfirewall firewall add rule name='PSRemoting HTTP' dir=in action=allow protocol=TCP localport=5985
netsh advfirewall firewall add rule name='PSRemoting HTTPS' dir=in action=allow protocol=TCP localport=5986

This configures your firewall. Now, you want to enable Windows Remote Management (WinRM). Open gpedit.msc, browse to Computer Configuration > Administrative Templates > Windows Components > Windows Remote Management (WinRM) > WinRM Service and enable Open the Allow Remote Server management through WinRM policy setting. Set filters to * (unless you need something specific there).

Next, enable firewall rules for all profiles. In gpedit.msc browse to Computer Configuration> Administrative Templates > Network > Network Connections > Windows Firewall > Domain Profile and open Windows Defender Firewall: Define inbound port exceptions policy setting. Enable it, click Show button and add those port exceptions:

5985:TCP:*:enabled:WSMan
5986:TCP:*:enabled:WSMan

Finally, you need to give permissions for particular users to connect to the machine. Run

Set-PSSessionConfiguration -ShowSecurityDescriptorUI -Name Microsoft.PowerShell

and give permissions to users of your choice.

To verify your setting, run

winrm get winrm/config/listener?Address=*+Transport=HTTP

and you should get something like

Listener [Source="GPO"]
    Address = *
    Transport = HTTP
    Port = 5985
    Hostname
    Enabled = true
    URLPrefix = wsman
    CertificateThumbprint
    ListeningOn = 127.0.0.1, 169.254.148.232, 169.254.170.254, , ::1, fe80::24a9:613:3ca1:6de8%4

Client — for connecting to other machine over HTTP and IP address

You need to add trusted hosts using elevated PS:

Set-Item WSMan:\localhost\Client\TrustedHosts -Value "*"

Now you can connect using

etsn -computername IP_ADDRESS -port 5985 -credential DOMAIN\USER

This makes an unencrypted connection using IP address. Obviously, you want to open this connection over VPN/SSH tunnel. If you want encryption, go with

etsn -computername IP_ADDRESS -usessl -port 5986 -credential DOMAIN\USER

1.ps1

echo "1_1 STD OUT"
write-host "1_2 HOST OUT"
write-error "1_3 STD ERR"

.\2.ps1

2.ps1

echo "2_1 STD OUT"
write-host "2_2 HOST OUT"
write-error "2_3 STD ERR"

.\3.bat

3.bat

echo "3_1 STD OUT"
echo "3_2 STD ERR" 1>&2
echo "3_2 STD 3" 1>&3

cscript 4.vbs

4.vbs

Wscript.Echo "4_1 STD OUT"
Set fso = CreateObject ("Scripting.FileSystemObject")
Set stdout = fso.GetStandardStream (1)
Set stderr = fso.GetStandardStream (2)
stdout.WriteLine "4_2 STD OUT"
stderr.WriteLine "4_3 STD ERR"

Set WshShell = CreateObject("WScript.Shell")
Set WshShellExec = WshShell.Exec("5.exe")
WScript.StdOut.Write WshShellExec.StdOut.ReadAll
WScript.StdErr.Write WshShellExec.StdErr.ReadAll

5.exe

#include <iostream>
#include <cstdlib>

int main()
{
    std::cout << "5_1 STD OUT" << "\n";
    std::cerr << "5_2 STD ERR" << "\n";

    system("6.exe");
}

6.exe

#include <iostream>
#include <cstdlib>

int main()
{
    std::cout << "6_1 STD OUT" << "\n";
    std::cerr << "6_2 STD ERR" << "\n";
}

Let’s now run this with powershell:

PS C:\Afish\Playground> .\1.ps1
1_1 STD OUT
1_2 HOST OUT
C:\Afish\Playground\1.ps1 : 1_3 STD ERR
At line:1 char:1
+ .\1.ps1
+ ~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,1.ps1

2_1 STD OUT
2_2 HOST OUT
C:\Afish\Playground\2.ps1 : 2_3 STD ERR
At C:\Afish\Playground\1.ps1:5 char:1
+ .\2.ps1
+ ~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,2.ps1


C:\Afish\Playground>echo "3_1 STD OUT"
"3_1 STD OUT"

C:\Afish\Playground>echo "3_2 STD ERR"  1>&2
"3_2 STD ERR"

C:\Afish\Playground>echo "3_2 STD 3"  1>&3
"3_2 STD 3"

C:\Afish\Playground>cscript 4.vbs
Microsoft (R) Windows Script Host Version 5.812
Copyright (C) Microsoft Corporation. All rights reserved.

4_1 STD OUT
4_2 STD OUT
4_3 STD ERR
5_1 STD OUT
6_1 STD OUT
5_2 STD ERR
6_2 STD ERR

Okay, we have all STD OUT and STD ERR streams. Let’s now try redirecting this to file:

PS C:\Afish\Playground> .\1.ps1 > out.txt
1_2 HOST OUT
C:\Afish\Playground\1.ps1 : 1_3 STD ERR
At line:1 char:1
+ .\1.ps1 > out.txt
+ ~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,1.ps1

2_2 HOST OUT
C:\Afish\Playground\2.ps1 : 2_3 STD ERR
At C:\Afish\Playground\1.ps1:5 char:1
+ .\2.ps1
+ ~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,2.ps1

"3_2 STD ERR"
4_3 STD ERR
5_2 STD ERR
6_2 STD ERR

File content:

1_1 STD OUT
2_1 STD OUT

C:\Afish\Playground>echo "3_1 STD OUT" 
"3_1 STD OUT"

C:\Afish\Playground>echo "3_2 STD ERR"  1>&2 

C:\Afish\Playground>echo "3_2 STD 3"  1>&3 
"3_2 STD 3" 

C:\Afish\Playground>cscript 4.vbs 
Microsoft (R) Windows Script Host Version 5.812
Copyright (C) Microsoft Corporation. All rights reserved.

4_1 STD OUT
4_2 STD OUT
5_1 STD OUT
6_1 STD OUT

Okay, so standard output was redirected correctly but standard error and write-host was not. Let’s try this:

PS C:\Afish\Playground> .\1.ps1 2>&1 > out.txt
1_2 HOST OUT
2_2 HOST OUT

File content:

1_1 STD OUT
C:\Afish\Playground\1.ps1 : 1_3 STD ERR
At line:1 char:1
+ .\1.ps1 2>&1 > out.txt
+ ~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,1.ps1
 
2_1 STD OUT
C:\Afish\Playground\2.ps1 : 2_3 STD ERR
At C:\Afish\Playground\1.ps1:5 char:1
+ .\2.ps1
+ ~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,2.ps1
 

C:\Afish\Playground>echo "3_1 STD OUT" 
"3_1 STD OUT"

C:\Afish\Playground>echo "3_2 STD ERR"  1>&2 
.\3.bat : "3_2 STD ERR" 
At C:\Afish\Playground\2.ps1:5 char:1
+ .\3.bat
+ ~~~~~~~
    + CategoryInfo          : NotSpecified: ("3_2 STD ERR" :String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 

C:\Afish\Playground>echo "3_2 STD 3"  1>&3 
"3_2 STD 3" 

C:\Afish\Playground>cscript 4.vbs 
Microsoft (R) Windows Script Host Version 5.812
Copyright (C) Microsoft Corporation. All rights reserved.

4_1 STD OUT
4_3 STD ERR
4_2 STD OUT


5_1 STD OUT
6_1 STD OUT
5_2 STD ERR
6_2 STD ERR

Okay, we have almost whole output in the file. Only write-host was not redirected. Can we do better?

The trick is to use CMD:

PS C:\Afish\Playground> cmd /c powershell.exe -file "1.ps1" 4>&1 3>&1 2>&1 > out.txt

File content:

1_1 STD OUT
1_2 HOST OUT
cmd : C:\Afish\Playground\1.ps1 : 1_3 STD ERR
At line:1 char:1
+ cmd /c powershell.exe -file "1.ps1" 4>&1 3>&1 2>&1 > out.txt
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (C:\Afish\Playground\1.ps1 : 1_3 STD ERR:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException

    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,1.ps1

 

2_1 STD OUT
2_2 HOST OUT
C:\Afish\Playground\2.ps1 : 2_3 STD ERR

At C:\Afish\Playground\1.ps1:5 char:1

+ .\2.ps1

+ ~~~~~~~

    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException

    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,2.ps1

 


C:\Afish\Playground>echo "3_1 STD OUT" 
"3_1 STD OUT"

C:\Afish\Playground>echo "3_2 STD ERR"  1>&2 
"3_2 STD ERR" 


C:\Afish\Playground>echo "3_2 STD 3"  1>&3 
"3_2 STD 3" 

C:\Afish\Playground>cscript 4.vbs 
Microsoft (R) Windows Script Host Version 5.812
Copyright (C) Microsoft Corporation. All rights reserved.

4_1 STD OUT
4_2 STD OUT
4_3 STD ERR


5_1 STD OUT
6_1 STD OUT
5_2 STD ERR
6_2 STD ERR

Now we have everything in file. How to see it in the console as well?

PS C:\Afish\Playground> cmd /c powershell.exe -file "1.ps1" 4>&1 3>&1 2>&1 | Tee-Object -FilePath out.txt
1_1 STD OUT
1_2 HOST OUT
cmd : C:\Afish\Playground\1.ps1 : 1_3 STD ERR
At line:1 char:1
+ cmd /c powershell.exe -file "1.ps1" 4>&1 3>&1 2>&1 | Tee-Object -File ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (C:\Afish\Playground\1.ps1 : 1_3 STD ERR:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,1.ps1

2_1 STD OUT
2_2 HOST OUT
C:\Afish\Playground\2.ps1 : 2_3 STD ERR
At C:\Afish\Playground\1.ps1:5 char:1
+ .\2.ps1
+ ~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,2.ps1


C:\Afish\Playground>echo "3_1 STD OUT"
"3_1 STD OUT"

C:\Afish\Playground>echo "3_2 STD ERR"  1>&2
"3_2 STD ERR"

C:\Afish\Playground>echo "3_2 STD 3"  1>&3
"3_2 STD 3"

C:\Afish\Playground>cscript 4.vbs
Microsoft (R) Windows Script Host Version 5.812
Copyright (C) Microsoft Corporation. All rights reserved.

4_1 STD OUT
4_3 STD ERR
4_2 STD OUT
5_1 STD OUT
6_1 STD OUT
5_2 STD ERR
6_2 STD ERR

File output:

1_1 STD OUT
1_2 HOST OUT
cmd : C:\Afish\Playground\1.ps1 : 1_3 STD ERR
At line:1 char:1
+ cmd /c powershell.exe -file "1.ps1" 4>&1 3>&1 2>&1 | Tee-Object -File ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (C:\Afish\Playground\1.ps1 : 1_3 STD ERR:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException

    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,1.ps1

 

2_1 STD OUT
2_2 HOST OUT
C:\Afish\Playground\2.ps1 : 2_3 STD ERR

At C:\Afish\Playground\1.ps1:5 char:1

+ .\2.ps1

+ ~~~~~~~

    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException

    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,2.ps1

 


C:\Afish\Playground>echo "3_1 STD OUT" 
"3_1 STD OUT"

C:\Afish\Playground>echo "3_2 STD ERR"  1>&2 
"3_2 STD ERR" 


C:\Afish\Playground>echo "3_2 STD 3"  1>&3 
"3_2 STD 3" 

C:\Afish\Playground>cscript 4.vbs 
Microsoft (R) Windows Script Host Version 5.812
Copyright (C) Microsoft Corporation. All rights reserved.

4_1 STD OUT
4_3 STD ERR


4_2 STD OUT
5_1 STD OUT
6_1 STD OUT
5_2 STD ERR
6_2 STD ERR

Now it works correctly.

I had the the following problem problem — application X was constantly creating new instance of application Y (as desired) which in turn resulted in creating new window stealing focus and appearing on the screen. This is especially irritating when typing or when working using RDP since new window consumes some bandwidth which (especially painful over mobile connection).

Powershell hiding windows

You can use a powershell background script which would restore the focus if it was stole. Here is the code:

Add-Type -AssemblyName System.Windows.Forms
[System.Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms") 

$TypeDef = @"

using System;
using System.Text;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using System.Threading;
using System.Windows.Forms;

namespace Api
{
	public class ApiDef
	{
		[DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
		static extern int GetWindowText(IntPtr hWnd, StringBuilder lpString, int nMaxCount);

		[DllImport("user32.dll", SetLastError=true)]
		static extern IntPtr GetForegroundWindow();
		
		private static string ReadText(IntPtr hWnd){
			StringBuilder sb = new StringBuilder(256);
			int res = GetWindowText((IntPtr)hWnd, sb, 256);
			return sb.ToString();
		}

		public static void Activate(){
			var oldWindow = GetForegroundWindow();
			while(true){
				var foreground = GetForegroundWindow();
				var text = ReadText(foreground).ToLower();
				if(text.Contains("whatever title you like")){
					SendKeys.SendWait("%{ESC}");
				}else{
					oldWindow = foreground;
				}
				
				Thread.Sleep(250);
			}
		}

	}
}
"@

Add-Type -TypeDefinition $TypeDef -Language CSharpVersion3 -ReferencedAssemblies @("System.Windows.Forms")

[Api.Apidef]::Activate()

Every 250 milliseconds I get the foreground window and check its title. If it is something which I know steals the focus then I send CTRL + ESC sequence which ultimately switches application back.

I tried doing SetWindowPos or BringWindowToTop orSetForegroundWindow but none of those worked as expected. Finally I decided to go with the hotkey and it looks like this is good enough for me.

VirtuaWin

There is actually much easier solution for doing that. VirtuaWin can create multiple desktops and automatically move windows between them based on rules. Just capture the class or process name you want to move, configure rule and the window will be moved automatically.

Running in different session

You can open new session to the host (using MSTSC or Remote App Terminal Services) and then run application in that session. You won’t see its UI at all, however, it will run normally. You can do that using qwinsta to get session id and then psexec.exe -i session_id app.exe

Creating new desktop

You can create new desktop using Sysinternals Desktops utility. Next, run application stealing focus in new desktop and it will not pop up on other desktops anymore.

I think desktop switcher in Windows 10 may be using the same API. If you are using it then you may look at Windows 10 virtual desktop enhancer to get some more shortcuts.

There is also Dexpot which I was using for many years and it is great. Just watch the license restrictions if you want to use it commercially.

See Pushing the limits and Understanding windows at a deeper level articles about this part of OS.

This is the first part of the Parallel Execution for Contests series. For your convenience you can find other parts using the links below :
Part 1 — Powershell
Part 2 — C#
Part 3 — C++

Hello! Today we start a new series about running code in parallel. This is not intented to be neither a concurrent programming course nor a tutorial how to use MPI. I am going to show a few simple programs which are very useful when solving algorithmic tasks.

Introduction

When you take part in a contest similar to Deadline24 you need to solve multiple problems on your machine. These are algorithmic problems so you need to come up with a feasible solution, both in time and space complexity. However, when you have a slower algorithm, you might want to catch up using the power of your machine — if you have a powerfull cluster then you can easily solve tasks even using less optimized algorithms.

Since most of todays’ CPUs are multicore it is fairly easy to speed up execution time using multiple threads. Unfortunately, sometimes it is not easy to change algorithm to multithreaded and do not loose precious seconds on threads synchronization. You might try introducing interthreads communication using MPI but when there is not enough time you probably want to introduce parallelism with as few changes as possible.

Fortunately, contests like Deadline usualy consists of multiple test cases for each task. We can utilize this to simply execute multiple instances of our program in parallel. This approach doesn’t require any threads synchronization and is very effective because it is the operating system who will take care of most of the stuff. All we need to do is run multiple instances of application and pass different input files for each of them. Let’s see how to do that.

Powershell features

We assume that we have a single threaded program called Solver.exe which reads some data from standard input and writes to standard output. We also have multiple input files: input1.txt, …, input10.txt. All we want to do is run application, pass input file and capture output. We can do the following using Powershell:

cat input1.txt | .\Solver.exe | out-file -encoding ascii output1.txt

We can also solve all files with one command:

1..10 | %{ cat "input$_.txt" | .\Solver.exe | out-file -encoding ascii "output$_.txt" }

This is very easy, however, it runs one by one. Another instance of Solver will be run only after previous one finished its work. This means that most of our cores will stay idle during the execution.

Fortunately, there are simple tricks to run code in parallel using Powershell. Let’s see them.

Workflows

We start with workflows. 64 bit Powershell is able to run code in parallel using a feature called Workflow like this:

workflow OurWorkflow
{
	foreach -parallel ($i in 1..10){
		cat "input$i.txt" | .\Solver.exe | out-file -encoding ascii "output$i.txt"
	}
}

OurWorkflow

We can see two importants things here. First, we introduce a block of code using workflow keyword. Next, we use foreach -parallel option to run code in parallel. You can see some description here

This approach is simple and works out of the box. However, there are some drawbacks:

• we cannot simply stop the script since CTRL+C might not work
• we cannot write to the console so simple echo "I am here!" will not work
• we cannot easily control number of threads running the code
• requires 64 bit Powershell

So we can see that while it works it might not suit all our needs.

Background jobs

Powershell is capable of running some jobs in the background using start-job commandlet. We can use this to run all jobs in background and wait for them:

$jobs = @()

Function Run($scriptBlock, $arguments){
	$jobs += ,(start-job $scriptBlock -Arg $arguments)
}

Function Finish(){
	$jobs | % {wait-job $_}
}

1..10 | %{ Run { param($_); cat "input$_.txt" | .\Solver.exe | out-file -encoding ascii "output$_.txt" } @($_) } -end { Finish }

This works as expected, we can also query the job result and do some magic with it. We could also await for specific jobs and implement custom thread pooling. Unfortunately, there are drawbacks as well:

• it runs in different directory
• it is slow — even after all jobs are done I need to wait few seconds to see the results (when writing to files)
• we are unable to write to the console (since these are background jobs)

Parallel foreach

Previous methods were using only built-in features of Powershell. Right now we will take a look at different method which requires additional code.

If you google carefully you will find this script on Technet. It basically does everything we need to execute code in parallel. This code goes as follows:

function ForEach-Parallel { 
<# 
.SYNOPSIS 
A parallel ForEach that uses runspaces 
 
.PARAMETER ScriptBlock 
ScriptBlock to execute for each InputObject 
 
.PARAMETER ScriptFile 
Script file to execute for each InputObject 
 
.PARAMETER InputObject 
Object(s) to run script against in parallel 
 
.PARAMETER Throttle 
Maximum number of threads to run at one time.  Default: 5 
 
.PARAMETER Timeout 
Stop each thread after this many minutes.  Default: 0 
 
WARNING:  This parameter should be used as a failsafe only 
Set it for roughly the entire duration you expect for all threads to complete 
 
.PARAMETER SleepTimer 
When looping through open threads, wait this many milliseconds before looping again.  Default: 200 
 
.EXAMPLE 
(0..50) | ForEach-Parallel -Throttle 4 { $_; sleep (Get-Random -Minimum 0 -Maximum 5) } 
} 
 
Send the number 0 through 50 to scriptblock.  For each, display the number and then sleep for 0 to 5 seconds.  Only execute 4 threads at a time. 
 
.EXAMPLE 
$servers | Foreach-Parallel -Throttle 20 -Timeout 60 -sleeptimer 200 -verbose -scriptFile C:\query.ps1 
 
Run query.ps1 against each computer in $servers.  Run 20 threads at a time, timeout a thread if it takes longer than 60 minutes to run, give verbose output. 
 
.FUNCTIONALITY  
PowerShell Language 
 
.NOTES 
Credit to Tome Tanasovski 
http://powertoe.wordpress.com/2012/05/03/foreach-parallel/ 
#> 
    [cmdletbinding()] 
    param( 
        [Parameter(Mandatory=$false,position=0,ParameterSetName='ScriptBlock')] 
            [System.Management.Automation.ScriptBlock]$ScriptBlock, 
 
        [Parameter(Mandatory=$false,ParameterSetName='ScriptFile')] 
        [ValidateScript({test-path $_ -pathtype leaf})] 
            $scriptFile, 
 
        [Parameter(Mandatory=$true,ValueFromPipeline=$true)] 
            [PSObject]$InputObject, 
 
            [int]$Throttle=5, 
 
            [double]$sleepTimer = 200, 
 
            [double]$Timeout = 0 
    ) 
    BEGIN { 
         
        #Build the scriptblock depending on the parameter used 
        switch ($PSCmdlet.ParameterSetName){ 
            'ScriptBlock' {$ScriptBlock = $ExecutionContext.InvokeCommand.NewScriptBlock("param(`$_)`r`n" + $Scriptblock.ToString())} 
            'ScriptFile' {$scriptblock = [scriptblock]::Create($(get-content $scriptFile | out-string))} 
            Default {Write-Error ("Must provide ScriptBlock or ScriptFile"); Return} 
        } 
         
        #Define the initial sessionstate, create the runspacepool 
        Write-Verbose "Creating runspace pool with $Throttle threads" 
        $sessionState = [system.management.automation.runspaces.initialsessionstate]::CreateDefault() 
        $pool = [Runspacefactory]::CreateRunspacePool(1, $Throttle, $sessionState, $host) 
        $pool.open() 
         
        #array to hold details on each thread 
        $threads = @() 
 
        #If inputObject is bound get a total count and set bound to true 
        $bound = $false 
        if( $PSBoundParameters.ContainsKey("inputObject") ){ 
            $bound = $true 
            $totalCount = $inputObject.count 
        } 
         
    } 
 
    PROCESS { 
         
$run = @' 
        #For each pipeline object, create a new powershell instance, add to runspacepool 
        $powershell = [powershell]::Create().addscript($scriptblock).addargument($InputObject) 
        $powershell.runspacepool=$pool 
        $startTime = get-date 
 
        #add references to inputobject, instance, handle and startTime to threads array 
        $threads += New-Object psobject -Property @{ 
            Object = $inputObject; 
            instance = $powershell; 
            handle = $powershell.begininvoke(); 
            startTime = $startTime 
        } 
 
        Write-Verbose "Added $inputobject to the runspacepool at $startTime" 
'@ 
 
        #Run the here string.  Put it in a foreach loop if it didn't come from the pipeline 
        if($bound){    
            $run = $run -replace 'inputObject', 'object' 
            foreach($object in $inputObject){  
                Invoke-Expression -command $run 
            } 
        } 
 
        else{ 
         
            Invoke-Expression -command $run 
        } 
 
    } 
    END { 
        $notdone = $true 
         
        #Loop through threads. 
        while ($notdone) { 
 
            $notdone = $false 
            for ($i=0; $i -lt $threads.count; $i++) { 
                $thread = $threads[$i] 
                if ($thread) { 
 
                    #If thread is complete, dispose of it. 
                    if ($thread.handle.iscompleted) { 
                        Write-verbose "Closing thread for $($thread.Object)" 
                        $thread.instance.endinvoke($thread.handle) 
                        $thread.instance.dispose() 
                        $threads[$i] = $null 
                    } 
 
                    #Thread exceeded maxruntime timeout threshold 
                    elseif( $Timeout -ne 0 -and ( (get-date) - $thread.startTime ).totalminutes -gt $Timeout ){ 
                        Write-Error "Closing thread for $($thread.Object): Thread exceeded $Timeout minute limit" -TargetObject $thread.inputObject 
                        $thread.instance.dispose() 
                        $threads[$i] = $null 
                    } 
 
                    #Thread is running, loop again! 
                    else { 
                        $notdone = $true 
                    } 
                }            
            } 
 
            #Sleep for specified time before looping again 
            Start-Sleep -Milliseconds $sleepTimer 
        } 
        $pool.close() 
    } 
}

It basically has all the things we would like it to have. We are able to specify number of threads, we also can easily pass script block and write to console. We execute this code in the following way:

1..10 | foreach-parallel -throttle 4 { cat "input$_.txt" | .\Solver.exe | out-file -encoding ascii "output$_.txt" }

When the task is finished its output will be print out to the console so we can easily verify our commands by echoing them. This is especially useful when we don’t have much time to debug scripts.

Summary

We now know how to execute multiple instances of our program. In the next parts we are going to see how to introduce parallelism to our algorithms by solving every test case on different thread using thread pools.

Basic case

We can load any type using Add-Type commandlet. Basic invocation requires type definition and language:
Add-Type -TypeDefinition 'your code here' -Language CSharp
For instance, to load and execute hello world use this script:

$code = @"
using System;
namespace HelloWorld
{
	public class Program
	{
		public static void Main(){
			Console.WriteLine("Hello world!");
		}
	}
}
"@

Add-Type -TypeDefinition $code -Language CSharp	
iex "[HelloWorld.Program]::Main()"

Save this code to file “Hello.ps1”, start PowerShell, change directory, and run command .\Hello.ps1. You should see “Hello world!” in the console. Congratulations! You just compiled and executed your first C# program in PowerShell.

Multiple executions

If you modify the code and try to run this script again, you will get the following error:

Add-Type : Cannot add type. The type name 'HelloWorld.Program' already exists.
At Z:\Hello.ps1:14 char:1
+ Add-Type -TypeDefinition $code -Language CSharp
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (HelloWorld.Program:String) [Add-Type], Exception
    + FullyQualifiedErrorId : TYPE_ALREADY_EXISTS,Microsoft.PowerShell.Commands.AddTypeCommand

This happens because you cannot load type into the same App Domain again, unfortunately, you cannot unload type. If you try to google solution, you will probably land on this SO question. All these solutions are cumbersome and hard to remember. Also starting nested PowerShell might be very slow if you have complicated logic in your user script and forget to add -noprofile parameter.

There is a simple workaround for this problem. Just add random suffix to class name and you are good to go. See this script:

$id = get-random
$code = @"
using System;
namespace HelloWorld
{
	public class Program$id
	{
		public static void Main(){
			Console.WriteLine("Hello world!");
		}
	}
}
"@

Add-Type -TypeDefinition $code -Language CSharp	
iex "[HelloWorld.Program$id]::Main()"

Observe that we obtain random number in line 1, append it to the class name in line 6, and finally we modify the line 16 to use correct class. Now you can modify C# code and execute it without restarting PowerShell or doing other tricks.

Referencing assemblies

At the beginning of each C# code we import namespaces with the using keyword. In order to do that PowerShell needs to be able to find these namespaces, so it needs to have correct DLL-s loaded. In VS we can add library by adding reference to a project. We can do the same in PowerShell by adding referencedAssemblies parameter to our script:

$assemblies = ("System.Core","System.Xml.Linq","System.Data","System.Xml", "System.Data.DataSetExtensions", "Microsoft.CSharp")
Add-Type -ReferencedAssemblies $assemblies -TypeDefinition $code -Language CSharp

You can see that we first define an array of assemblies to load and then we pass this array as an argument to Add-Type commandlet. Now PowerShell will take care of loading libraries and necessary namespaces will be available to you so you will be able to import them with using.

What about custom libraries? In order to load them PowerShell needs to be able to locate them in the system. By default PowerShell will look for them in GAC. If you have custom assemblies to load, you can add them manually before adding your C# code. See the example:

Add-Type -Path "CustomLibrary.dll"
$assemblies = ("Custom.Namespace.From.Library")
Add-Type -ReferencedAssemblies $assemblies -TypeDefinition $code -Language CSharp

In the first line we manually load required dll using Add-Type commandlet. We can specify path to the library so PowerShell will find it easily and load with no problems. After that we will be able to import namespaces from custom dll.

However, this method will work only for managed dlls. If you want to load native library, you need to load it directly into the PowerShell process using native functions:

$loadLibrary = @'
[DllImport("kernel32", SetLastError=true, CharSet = CharSet.Ansi)]
public static extern IntPtr LoadLibrary([MarshalAs(UnmanagedType.LPStr)]string lpFileName);
'@
$Kernel32 = Add-Type -MemberDefinition $loadLibrary -Name 'Kernel32' -Namespace 'Win32' -PassThru
$Kernel32::LoadLibrary("NativeLibrary.dll") | out-null

We first define header of LoadLibrary WinAPI function using P/Invoke. Next, we load it (line 5), finally we load native library using LoadLibrary function. Now we should be able to use native dlls in C# code.

Compilation errors

Compiling code in the presented way has one important drawback: it does not show compilation errors. You probably will not have the IDE nearby, so every typo will be difficult to track without information from the compiler what exactly went wrong. However, if you compile your code using -Path parameter you will get description of problems with your code.

Running WPF from a PowerShell

Nothing stops you from running not only hello worlds but also more fancy application. PowerShell is integrated with .NET so you can create objects of C# classes and run almost arbitrary code. For instance, script below (created by The Scripting Guys) shows window using WPF:

Add-Type -AssemblyName PresentationFramework
[xml]$xaml =
@"
< Window
  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
  Title="Window1" Height="300" Width="408">
    < Grid>
      < Button x:Name="button1"
                Width="75"
                Height="23"
                Canvas.Left="118"
                Canvas.Top="10"
                Content="Click Here" />
    < /Grid>
< /Window>
"@
Clear-Host
$reader=(New-Object System.Xml.XmlNodeReader $xaml)
$target=[Windows.Markup.XamlReader]::Load($reader)
$control=$target.FindName("button1")
$eventMethod=$control.add_click
$eventMethod.Invoke({$target.Title="Hello $((Get-Date).ToString('G'))"})
$target.ShowDialog() | out-null

You might need to run this script from STA PowerShell. Just start you PowerShell with -STA parameter and you are good to go. For more details see this blog post.

Summary

Running arbitrary C# code can be very useful. You can always try to rewrite your application using PowerShell language, but it might be much easier just to load and execute existing code.