How to efficiently write large files to disk on background thread (Swift)

Performance depends wether or not the data fits in RAM. If it does, then you should use NSData writeToURL with the atomically feature turned on, which is what you’re doing. Apple’s notes about this being dangerous when “writing to a public directory” are completely irrelevant on iOS because there are no public directories. That section …

Read more

Parallel.ForEach can cause a “Out Of Memory” exception if working with a enumerable with a large object

The default options for Parallel.ForEach only work well when the task is CPU-bound and scales linearly. When the task is CPU-bound, everything works perfectly. If you have a quad-core and no other processes running, then Parallel.ForEach uses all four processors. If you have a quad-core and some other process on your computer is using one …

Read more

Shared memory in multiprocessing

Because this is still a very high result on google and no one else has mentioned it yet, I thought I would mention the new possibility of ‘true’ shared memory which was introduced in python version 3.8.0: https://docs.python.org/3/library/multiprocessing.shared_memory.html I have here included a small contrived example (tested on linux) where numpy arrays are used, which …

Read more