Read file line by line in PowerShell

I want to read a file line by line in PowerShell. Specifically, I want to loop through the file, store each line in a variable in the loop, and do some processing on the line.

I know the Bash equivalent:

while read line do if [[ $line =~ $regex ]]; then # work here fi done < file.txt 

Not much documentation on PowerShell loops.

4 Answers

Not much documentation on PowerShell loops.

Documentation on loops in PowerShell is plentiful, and you might want to check out the following help topics: about_For, about_ForEach, about_Do, about_While.

foreach($line in Get-Content .\file.txt) { if($line -match $regex){ # Work here } } 

Another idiomatic PowerShell solution to your problem is to pipe the lines of the text file to the ForEach-Object cmdlet:

Get-Content .\file.txt | ForEach-Object { if($_ -match $regex){ # Work here } } 

Instead of regex matching inside the loop, you could pipe the lines through Where-Object to filter just those you're interested in:

Get-Content .\file.txt | Where-Object {$_ -match $regex} | ForEach-Object { # Work here } 

Get-Content has bad performance; it tries to read the file into memory all at once.

C# (.NET) file reader reads each line one by one

Best Performace

foreach($line in [System.IO.File]::ReadLines("C:\path\to\file.txt")) { $line } 

Or slightly less performant

[System.IO.File]::ReadLines("C:\path\to\file.txt") | ForEach-Object { $_ } 

The foreach statement will likely be slightly faster than ForEach-Object (see comments below for more information).

Reading Large Files Line by Line

Original Comment (1/2021) I was able to read a 4GB log file in about 50 seconds with the following. You may be able to make it faster by loading it as a C# assembly dynamically using PowerShell.

[System.IO.StreamReader]$sr = [System.IO.File]::Open($file, [System.IO.FileMode]::Open) while (-not $sr.EndOfStream){ $line = $sr.ReadLine() } $sr.Close() 

Addendum (3/2022) Processing the large file using C# embedded in PowerShell is even faster and has less "gotchas".

$code = @" using System; using System.IO; namespace ProcessLargeFile { public class Program { static void ProcessLine(string line) { return; } public static void ProcessLogFile(string path) { var start_time = DateTime.Now; StreamReader sr = new StreamReader(File.Open(path, FileMode.Open)); try { while (!sr.EndOfStream){ string line = sr.ReadLine(); ProcessLine(line); } } finally { sr.Close(); } var end_time = DateTime.Now; var run_time = end_time - start_time; string msg = "Completed in " + run_time.Minutes + ":" + run_time.Seconds + "." + run_time.Milliseconds; Console.WriteLine(msg); } static void Main(string[] args) { ProcessLogFile("c:\\users\\tasaif\\fake.log"); Console.ReadLine(); } } } "@ Add-Type -TypeDefinition $code -Language CSharp PS C:\Users\tasaif> [ProcessLargeFile.Program]::ProcessLogFile("c:\\users\\tasaif\\fake.log") Completed in 0:17.109 

The almighty switch works well here:

'one two three' > file $regex = '^t' switch -regex -file file { $regex { "line is $_" } } 

Output:

line is two line is three