A common problem when we use threads on python is the concurrency of instances on I/O objects or print screen cases, in this case, I show a thread in parallel, to write outputs from other threads instances to a file, sharing a variable called "work_queue", in this code is a list data type, but you can use others like queues from Queue library, using a flag to break the writer thread and give it a last chance to get into While sentence and check for last time the "work_queue" list.
In this way, we able to write a results from threads while they are not finished yet and savig data from scripts or programs avoiding lost data when the proccess is interrupted before finish.
import timeShare on Twitter Share on Facebook
import threading
class ThreadWriter(threading.Thread):
global last_round, work_queue
def __init__(self, file_name, path="", delimiter=""):
threading.Thread.__init__(self, args=())
self.file_name = file_name
self.path = path
self.name = "Writer"
self.delimiter = delimiter
self.daemon = True
self.exit_flag = False # flag to break?
def run(self):
while True:
file_handler = open(self.path + self.file_name, 'a')
if work_queue.__len__() > 0:
# write on file here
for _ in range(0, work_queue.__len__()):
file_handler.write(str(work_queue.pop()) + self.delimiter)
file_handler.close()
if self.exit_flag:
print(f"{self.name}: All Done!")
break
if last_round:
self.exit_flag = True # why?: after order to end task, we check that work queue is empty
work_queue = []
last_round = False
writer = ThreadWriter("testwrite.txt", delimiter="\n") # create instance
writer.start() # run writer thread
fr = open("data.txt", "rb") # fill with initial work
for line in fr.readlines():
work_queue.append(line.strip())
time.sleep(5)
for x in range(200, 3999): # append data after init
work_queue.append(x)
last_round = True # flag to kill thread after last work ajust stuff
writer.join() # wait thread
Comments
There are currently no comments
New Comment