What is fast way to check 'isDir' in python?

In my project, my program check very many dir and files.

I use python 3.8.

first, i try this. lower is my code snippet.

if '.' not in filename: # dir 

I know, it's not right. directory name can include '.'!

I try this.

if os.path.isdir(os.path.join(path_dir, filename)): # dir 

It's right. but, upper code is terribly slow!

first code use 0.48 sec. but second use 17 sec...

I change [os.path.join] -> [path_dir + '\' + filename]. It takes 15 sec. it's too slow, too.

I know os.path.isdir is problem. lower is test.

@logging_time def test_func1(dir_name, file_name): full_path = os.path.join(dir_name, file_name) for i in range(100000): if os.path.isdir(full_path): res = 'a' @logging_time def test_func2(dir_name, file_name): for i in range(100000): if '.' not in file_name: res = 'a' 

test_func2 use just 0.003 sec, but test_func1 use 3 sec!

What is the fast way to check isDir?

Target OS is windows 10. I don't need cross-flatform. just window.

2 Answers

os.path.isdir seems to be the fastest implementation:

from pathlib import Path import os import win32api import glob import timeit def os_isDir(files): return [os.path.isdir(fullPath) for fullPath in files] def os_NOT_isFile(files): return [not os.path.isfile(fullPath) for fullPath in files] def pathlib_isDir(files): return [Path(fullPath).is_dir() for fullPath in files] def winapi_GetFileAttributesA(files): return [win32api.GetFileAttributes(fullPath) == 16 for fullPath in files] files = glob.glob('C:\\Windows\\System32\\*') for func in ("os_isDir", "os_NOT_isFile", "pathlib_isDir", "winapi_GetFileAttributesA"): t1 = timeit.Timer(f"{func}({files})", f"from __main__ import {func}") print(f"Evaluation {len(files)} paths using {func} took: {t1.timeit(number=5)} seconds\n") 

Out (on a Win7 VM):

Evaluation 3294 paths using os_isDir took: 2.1250754 seconds Evaluation 3294 paths using os_NOT_isFile took: 2.7074766999999995 seconds Evaluation 3294 paths using pathlib_isDir took: 3.4624453000000006 seconds Evaluation 3294 paths using winapi_GetFileAttributesA took: 2.2789655999999994 seconds 

Perhaps the fastest way to check if something is a directory is to not have to check at all. If you wish to iterate all files in a directory tree (you haven't actually specified whether this is the case), you should consider using os.walk:

os.walk(top, topdown=True, onerror=None, followlinks=False)

Generate the file names in a directory tree by walking the tree either top-down or bottom-up. For each directory in the tree rooted at directory top (including top itself), it yields a 3-tuple (dirpath, dirnames, filenames).

The 3rd element of the tuple (filenames) is guaranteed to not be a directory. This is how you would print all file names in a directory tree:

import os def walk_directory(directory_name): for dirpath, dirnames, filenames in os.walk(directory_name): for filename in filenames: fullpathname = os.path.join(dirpath, filename) print(fullpathname) 

Otherwise, you should be using an isDir method call. For example, if you just wanted to iterate all the files in a single directory, then one way is as follows:

from pathlib import Path def list_directory(directory_name): # non-recursive for p in Path(directory_name).glob('*'): if not p.is_dir(): #filename = str(p) # to convert Path to a string print(p)