Getting all files under a folder and subfolders in a recursive manner

  • Python
  • Thread starter Arman777
  • Start date
  • Tags
    files
In summary, this conversation is about finding a function that can produce a list of all file paths under a specific folder and its subfolders recursively. The code provided by the other person uses the os.walk function and the import os module to achieve this. The person also mentions a previous code they wrote in Pascal that had a similar function.
  • #1
Arman777
Insights Author
Gold Member
2,168
193
I want to list all the full path of the files under a folder and all its subfolders recursively. Is there a way to do it? It seems that if the files go into 2 levels, the code can be written like is,


Code:
    import os

    folderPATH = r'C:\Users\Arman\Desktop\Cosmology\Articles'
    filePATHS = [x[2] for x in os.walk(folderPATH)]

    for i in filePATHS:
        for j in i:
            print(j)
prints


Code:
    Astrophysical Constants And Parameters.pdf
    desktop.ini
    Physics Latex Manuel.pdf
    Spactimes.pdf
    A parametric reconstruction of the cosmological jerk from diverse observational data.pdf
    A Thousand Problems in Cosmology Horizons.pdf
    An Almost Isotropic CM Temperature Does Not Imply An Almost Isotropic Universe.pdf
    Big Bang Cosmology - Review.pdf
    desktop.ini
    Expanding Confusion common misconceptions of cosmological horizons and the superluminal expansion of the universe.pdf
    Hubble Radius.pdf
    Is the Universe homogeneous.pdf
    LCDM and Mond.pdf
    Near galaxy clusters.pdf
    The Cosmological Constant and Dark Energy.pdf
    The mass of the Milky Way from satellite dynamic.pdf
    The Status of Cosmic Topology after Planck Data.pdf
    An upper limit to the central density of dark matter haloes from consistency with the presence of massive central black holes.pdf
    Dark Matter - Review.pdf
    Dark Matter Accretion into Supermassive Black Holes.pdf
    desktop.ini
    Andrew H. Jaffe - Cosmology - Imperial College Lecture Notes - Thermodynamics and Particle Physics.pdf
    Big Bang Nucleosynthesis.pdf
    Claus Grupen - Astroparticle Physics - The Early Universe.pdf
    Daniel Baumann - Cosmology - Mathematical Tripos III - Thermal History.pdf
    desktop.ini
    James Rich - Fundamentals of Cosmology - The Thermal History of the Universe.pdf
    Lars Bergström, Ariel Goobar - Cosmology and Particle Astrophysics -  Thermodynamics in the Early Universe.pdf
    Steven Weinberg - Cosmology - The Early Universe.pdf
    Andrei Linde - On the problem of initial conditions for inflation.pdf
    ...

I want a function that produces the same results but with recursive logic and with the full paths. So that for `n` nested folders, I can find the paths. I need something like this,


Code:
    import os

    def get_all_filePATHs(folderPATH):
        ...
        return get_all_filePATHs()

    folderPATH = r'C:\Users\Arman\Desktop\Cosmology\Articles'
    print(get_all_filePATHs(folderPATH))
Note that I am only interested in the full path of the files and not the path of the subfolders, as you can see from the example above.
 
Technology news on Phys.org
  • #2
Okay this works,

Code:
import os

def get_all_filePaths(folderPATH):
    result = []
    for dirpath, dirnames, filenames in os.walk(folderPATH):
        result.extend([os.path.join(dirpath, filename) for filename in filenames])
    return result
 
  • #3
I seem to remember writing something like that years ago - ah, here it is (in Pascal):
Code:
procedure TForm1.DoDirectory;
var
  sRec: TSearchRec;
  sRslt: Integer;
  specDir, First: Boolean;
  currFN, altFN: String;
Begin
  First := True;
  sRslt := FindFirst ('*.*',faReadOnly    Or faHidden    Or faSysFile Or faDirectory, sRec);
  While (sRslt = 0) Do Begin
    altFN := sRec.FindData.cAlternateFileName;
    currFN := sRec.FindData.cFileName;
    If (altFN<>'') and (altFN<>currFN) Then Begin
      If First Then Begin
        Writeln(resF);
        Writeln(resF,'[Directory]');
        Writeln(resF,GetCurrentDir);
        First := False;
        End;
      WriteLn(resF, altFN, ' "', currFN,'"');
      End;
    sRslt := FindNext(sRec);
    End;
  sRslt := FindFirst ('*.*', faDirectory, sRec);
  While  (sRslt = 0) Do Begin
    currFN := sRec.FindData.cFileName;
    specDir := (currFN = '.') Or (currFN = '..') Or ((sRec.Attr And faDirectory)=0);
    If (not specDir) and (testCnt<32) Then Begin
      Inc(testCnt);
      ChDir(currFN);
      DoDirectory;
      Chdir('..');
      End;
    sRslt := FindNext(sRec);
    End;
End;
 

FAQ: Getting all files under a folder and subfolders in a recursive manner

How can I retrieve all files under a folder and its subfolders in a recursive manner?

The most common approach to this is by using a recursive function. This function will loop through all the files and folders within the specified folder and its subfolders, and retrieve the necessary data for each file.

What is the benefit of using recursion for this task?

Recursion allows for a more efficient and concise code, as it eliminates the need for multiple nested loops. It also simplifies the process of retrieving files from subfolders, as the function will automatically traverse through all the subfolders.

Are there any potential limitations or drawbacks of using recursion?

One potential limitation is that recursion can consume a large amount of memory if the folder structure is too deep. It is important to optimize the recursive function to prevent this issue. Additionally, recursive functions can be more complex and difficult to debug compared to traditional loops.

How can I ensure that the recursive function does not retrieve any system files or hidden files?

You can include a condition within the recursive function to exclude certain file types or hidden files. This can be done by checking the file extension or using a specific file attribute such as "hidden".

Is there a way to limit the depth of recursion for this task?

Yes, you can set a maximum depth for the recursive function to prevent it from going too deep into the folder structure. This can be done by including a counter within the function and terminating the recursion when it reaches the specified depth.

Similar threads

Replies
4
Views
2K
Back
Top