victorNilsson
New Contributor II

Hi, thanks for taking your time to check this. 

I also tried serverless and then I was unable to recreate the problem. 

I've narrowed down the problem to a fuse mount problem. Polars seems to use mmap in shared mode, which seems to be unreliable at best. A minimal way to test this is 

import mmap
csv_file = "test.csv"

with open(csv_file, "w", encoding="utf8") as f:
    f.writelines(["a,c\n", "hello,world\n"])

with open(csv_file, 'r+') as f:
    mm = mmap.mmap(f.fileno(), 0, mmap.MAP_SHARED, prot=mmap.PROT_READ)


Which gives the exact same problem. Running in private mode works just fine though

import mmap
csv_file = "test.csv"

with open(csv_file, "w", encoding="utf8") as f:
    f.writelines(["a,c\n", "hello,world\n"])

with open(csv_file, 'r+') as f:
    mm = mmap.mmap(f.fileno(), 0, mmap.MAP_PRIVATE, prot=mmap.PROT_READ)


I tried your two suggestions but it still didnt work unfortunatelly:/

 

import mmap
import os
import time
csv_file = "test.csv"

with open(csv_file, "w", encoding="utf8") as f:
    f.writelines(["a,c\n", "hello,world\n"])
    f.flush()
    os.fsync(f.fileno())

time.sleep(10)

with open(csv_file, 'r+') as f:
    mm = mmap.mmap(f.fileno(), 0, mmap.MAP_SHARED, prot=mmap.PROT_READ)