Writer Thread - Python


A common problem when we use threads on python is the concurrency of instances on I/O objects or print screen cases, in this case, I show a thread in parallel, to write outputs from other threads instances to a file, sharing a variable called "work_queue", in this code is a list data type, but you can use others like queues from Queue library, using a flag to break the writer thread and give it a last chance to get into While sentence and check for last time the "work_queue" list.

In this way, we able to write a results from threads while they are not finished yet and savig data from scripts or programs avoiding lost data when the proccess is interrupted before finish.

import time
import threading

class ThreadWriter(threading.Thread):
global last_round, work_queue
def __init__(self, file_name, path="", delimiter=""):
threading.Thread.__init__(self, args=())
self.file_name = file_name
self.path = path
self.name = "Writer"
self.delimiter = delimiter
self.daemon = True
self.exit_flag = False # flag to break?

def run(self):
while True:
file_handler = open(self.path + self.file_name, 'a')
if work_queue.__len__() > 0:
# write on file here
for _ in range(0, work_queue.__len__()):
file_handler.write(str(work_queue.pop()) + self.delimiter)

if self.exit_flag:
print(f"{self.name}: All Done!")

if last_round:
self.exit_flag = True # why?: after order to end task, we check that work queue is empty

work_queue = []

last_round = False
writer = ThreadWriter("testwrite.txt", delimiter="\n") # create instance
writer.start() # run writer thread

fr = open("data.txt", "rb") # fill with initial work
for line in fr.readlines():

for x in range(200, 3999): # append data after init

last_round = True # flag to kill thread after last work ajust stuff
writer.join() # wait thread

Current rating: 5


There are currently no comments

New Comment


required (not published)