Writer Thread - Python

(0 comments)

A common problem when we use threads on python is the concurrency of instances on I/O objects or print screen cases, in this case, I show a thread in parallel, to write outputs from other threads instances to a file, sharing a variable called "work_queue", in this code is a list data type, but you can use others like queues from Queue library, using a flag to break the writer thread and give it a last chance to get into While sentence and check for last time the "work_queue" list.


In this way, we able to write a results from threads while they are not finished yet and savig data from scripts or programs avoiding lost data when the proccess is interrupted before finish.

import time
import threading

class ThreadWriter(threading.Thread):
global last_round, work_queue
def __init__(self, file_name, path="", delimiter=""):
threading.Thread.__init__(self, args=())
self.file_name = file_name
self.path = path
self.name = "Writer"
self.delimiter = delimiter
self.daemon = True
self.exit_flag = False # flag to break?

def run(self):
while True:
file_handler = open(self.path + self.file_name, 'a')
if work_queue.__len__() > 0:
# write on file here
for _ in range(0, work_queue.__len__()):
file_handler.write(str(work_queue.pop()) + self.delimiter)
file_handler.close()

if self.exit_flag:
print(f"{self.name}: All Done!")
break

if last_round:
self.exit_flag = True # why?: after order to end task, we check that work queue is empty


work_queue = []

last_round = False
writer = ThreadWriter("testwrite.txt", delimiter="\n") # create instance
writer.start() # run writer thread

fr = open("data.txt", "rb") # fill with initial work
for line in fr.readlines():
work_queue.append(line.strip())


time.sleep(5)
for x in range(200, 3999): # append data after init
work_queue.append(x)


last_round = True # flag to kill thread after last work ajust stuff
writer.join() # wait thread

Current rating: 5

Comments

There are currently no comments

New Comment

required

required (not published)

optional

required

captcha

required