Reading a file line by line in Go

I'm unable to find file.ReadLine function in Go.

How does one read a file line by line?

12 Answers

Answer recommended by Go Language

In Go 1.1 and newer the most simple way to do this is with a bufio.Scanner. Here is a simple example that reads lines from a file:

package main import ( "bufio" "fmt" "log" "os" ) func main() { file, err := os.Open("/path/to/file.txt") if err != nil { log.Fatal(err) } defer file.Close() scanner := bufio.NewScanner(file) // optionally, resize scanner's capacity for lines over 64K, see next example for scanner.Scan() { fmt.Println(scanner.Text()) } if err := scanner.Err(); err != nil { log.Fatal(err) } }

This is the cleanest way to read from a Reader line by line.

There is one caveat: Scanner will error with lines longer than 65536 characters. If you know your line length is greater than 64K, use the Buffer() method to increase the scanner's capacity:

... scanner := bufio.NewScanner(file) const maxCapacity int = longLineLen // your required line length buf := make([]byte, maxCapacity) scanner.Buffer(buf, maxCapacity) for scanner.Scan() { ...

NOTE: The accepted answer was correct in early versions of Go. See the highest voted answer contains the more recent idiomatic way to achieve this.

There is function ReadLine in package bufio.

Please note that if the line does not fit into the read buffer, the function will return an incomplete line. If you want to always read a whole line in your program by a single call to a function, you will need to encapsulate the ReadLine function into your own function which calls ReadLine in a for-loop.

bufio.ReadString('\n') isn't fully equivalent to ReadLine because ReadString is unable to handle the case when the last line of a file does not end with the newline character.

EDIT: As of go1.1, the idiomatic solution is to use bufio.Scanner

I wrote up a way to easily read each line from a file. The Readln(*bufio.Reader) function returns a line (sans \n) from the underlying bufio.Reader struct.

// Readln returns a single line (without the ending \n) // from the input buffered reader. // An error is returned iff there is an error with the // buffered reader. func Readln(r *bufio.Reader) (string, error) { var (isPrefix bool = true err error = nil line, ln []byte ) for isPrefix && err == nil { line, isPrefix, err = r.ReadLine() ln = append(ln, line...) } return string(ln),err }

You can use Readln to read every line from a file. The following code reads every line in a file and outputs each line to stdout.

f, err := os.Open(fi) if err != nil { fmt.Printf("error opening file: %v\n",err) os.Exit(1) } r := bufio.NewReader(f) s, e := Readln(r) for e == nil { fmt.Println(s) s,e = Readln(r) }

Cheers!

There two common way to read file line by line.

Use bufio.Scanner
Use ReadString/ReadBytes/... in bufio.Reader

In my testcase, ~250MB, ~2,500,000 lines, bufio.Scanner(time used: 0.395491384s) is faster than bufio.Reader.ReadString(time_used: 0.446867622s).

Source code:

Read file use bufio.Scanner,

func scanFile() { f, err := os.OpenFile(logfile, os.O_RDONLY, os.ModePerm) if err != nil { log.Fatalf("open file error: %v", err) return } defer f.Close() sc := bufio.NewScanner(f) for sc.Scan() { _ = sc.Text() // GET the line string } if err := sc.Err(); err != nil { log.Fatalf("scan file error: %v", err) return } }

Read file use bufio.Reader,

func readFileLines() { f, err := os.OpenFile(logfile, os.O_RDONLY, os.ModePerm) if err != nil { log.Fatalf("open file error: %v", err) return } defer f.Close() rd := bufio.NewReader(f) for { line, err := rd.ReadString('\n') if err != nil { if err == io.EOF { break } log.Fatalf("read file line error: %v", err) return } _ = line // GET the line string } }

Example from this gist

func readLine(path string) { inFile, err := os.Open(path) if err != nil { fmt.Println(err.Error() + `: ` + path) return } defer inFile.Close() scanner := bufio.NewScanner(inFile) for scanner.Scan() { fmt.Println(scanner.Text()) // the line } }

but this gives an error when there is a line that larger than Scanner's buffer.

When that happened, what I do is use reader := bufio.NewReader(inFile) create and concat my own buffer either using ch, err := reader.ReadByte() or len, err := reader.Read(myBuffer)

Another way that I use (replace os.Stdin with file like above), this one concats when lines are long (isPrefix) and ignores empty lines:

 func readLines() []string { r := bufio.NewReader(os.Stdin) bytes := []byte{} lines := []string{} for { line, isPrefix, err := r.ReadLine() if err != nil { break } bytes = append(bytes, line...) if !isPrefix { str := strings.TrimSpace(string(bytes)) if len(str) > 0 { lines = append(lines, str) bytes = []byte{} } } } if len(bytes) > 0 { lines = append(lines, string(bytes)) } return lines }

You can also use ReadString with \n as a separator:

 f, err := os.Open(filename) if err != nil { fmt.Println("error opening file ", err) os.Exit(1) } defer f.Close() r := bufio.NewReader(f) for { path, err := r.ReadString(10) // 0x0A separator = newline if err == io.EOF { // do something here break } else if err != nil { return err // if you return error } }

bufio.Reader.ReadLine() works well. But if you want to read each line by a string, try to use ReadString('\n'). It doesn't need to reinvent the wheel.

// strip '\n' or read until EOF, return error if read error func readline(reader io.Reader) (line []byte, err error) { line = make([]byte, 0, 100) for { b := make([]byte, 1) n, er := reader.Read(b) if n > 0 { c := b[0] if c == '\n' { // end of line break } line = append(line, c) } if er != nil { err = er return } } return }

Another method is to use the io/ioutil and strings libraries to read the entire file's bytes, convert them into a string and split them using a "\n" (newline) character as the delimiter, for example:

import ( "io/ioutil" "strings" ) func main() { bytesRead, _ := ioutil.ReadFile("something.txt") file_content := string(bytesRead) lines := strings.Split(file_content, "\n") }

Technically you're not reading the file line-by-line, however you are able to parse each line using this technique. This method is applicable to smaller files. If you're attempting to parse a massive file use one of the techniques that reads line-by-line.

In the code bellow, I read the interests from the CLI until the user hits enter and I'm using Readline:

interests := make([]string, 1) r := bufio.NewReader(os.Stdin) for true { fmt.Print("Give me an interest:") t, _, _ := r.ReadLine() interests = append(interests, string(t)) if len(t) == 0 { break; } } fmt.Println(interests)

import ( "bufio" "os" ) var ( reader = bufio.NewReader(os.Stdin) ) func ReadFromStdin() string{ result, _ := reader.ReadString('\n') witl := result[:len(result)-1] return witl }

Here is an example with function ReadFromStdin() it's like fmt.Scan(&name) but its takes all strings with blank spaces like: "Hello My Name Is ..."

var name string = ReadFromStdin() println(name)

In the new version of Go 1.16 we can use package embed to read the file contents as shown below.

package main import _"embed" func main() { //go:embed "hello.txt" var s string print(s) //go:embed "hello.txt" var b []byte print(string(b)) //go:embed hello.txt var f embed.FS data, _ := f.ReadFile("hello.txt") print(string(data)) }

For more details go through And