Python | Write multiple files data to master file
Given a number of input files in a source directory, write a Python program to read data from all the files and write it to a single master file.
Source directory contains n number of files, and structure is same for all files. The objective of this code is to read all the files one by one and then append the output into a single master file having structure same as source files.
Taking three input files as example, named emp_1.txt, emp_2.txt, emp_3.txt, output will contain data from all the input files.
Input:
Output:
Method #1: Using os module
import os # list the files in directory lis = os.listdir('D:\\python' '\\data_files\\data_files') print(lis) tgt = os.listdir('D:\\python' '\\data_files\\target_file') file_dir ='D:\\python\\data_files\\data_files'out_file = r'D:\\python\\data_files\\target_file\\master.txt'ct = 0 print('target file :', tgt) try: # check for if file exists # if yes delete the file # otherwise data will be appended to existing file if len(tgt)>0: os.remove('D:\\python' '\\data_files\\target_file\\master.txt') open(tgt, 'a').close() else: # create an empty file open(tgt, 'a').close() except: head = open('D:\\python' '\\data_files\\target_file\\master.txt', 'a+') line ='empno, ename, sal' # write header to output print(head, line) head.close() # below loop to write data to output file for line1 in lis: f_dir = file_dir+'\\'+line1 # open files in read mode in_file = open(f_dir, 'r+') # open output in append mode w = open(out_file, 'a+') d = in_file.readline() d = in_file.readlines() w.write("\n") for line2 in d: print(line2) w.write(line2) ct = ct + 1 w.close() |
chevron_right
filter_none
Output:
Method #2: Using pandas
import pandas as pd # pd.read_csv creates dataframes df1 = pd.read_csv('D:\python\data_files\data_files\emp_1.txt') df2 = pd.read_csv('D:\python\data_files\data_files\emp_2.txt') df3 = pd.read_csv('D:\python\data_files\data_files\emp_3.txt') frames = [df1, df2, df3] # concat function concatenates the frames result = pd.concat(frames) # to_csv function writes output to file result.to_csv('D:\\python\\data_files' '\\target_file\\master.txt', encoding ='utf-8', index = False) |
chevron_right
filter_none
Output:
Recommended Posts:
- Python | How to use Multiple kv files in kivy
- Rename multiple files using Python
- Python Program to merge two files into a third file
- Python | Create and write on excel file using xlsxwriter module
- reStructuredText | .rst file to HTML file using Python for Documentations
- Downloading files from web using Python
- Working with PDF files in Python
- Working with zip files in Python
- Working with csv files in Python
- Listing out directories and files in Python
- Writing files in background in Python
- Python | Merge two text files
- How to move Files and Directories in Python
- Python | Reading .ini Configuration Files
- Python | Create Archives and Find Files by Name
If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.
