Hi Yutaka, I read your message only now.
I made some debug version to test multi-thread. They are public at "Debug version for test" topic on this forum. Please test them, and compare which is fast for when.
OK, I'll try to do some tests
"read/write all at once" style IO is used for FFT. For example, FFT requires a set of data [ A ][ B ][ C ]. RSC32 may rearrange file data by using temporary file. The data of source file is like; [ A ][ A ][ A ][ B ][ B ][ B ][ C ][ C ][ C ]
I read a little about Galois Fields. You use p = 2^16 + 1 and you are bounded to an FFT of dimension 2^16, on blocks of 16 bits. So you have to compute many times the FFT.
Why not use instead GF(2^N), there is a library to work on fields of characteristic 2. You can then use FFT of greater order, requiring less (slow) HD seeking.
Otherwise, as GF(p) there is p = 2^64 - 2^32 + 1 which allows an FFT up to 2^32, without divisions, as described here:http://www.craig-wood.com/nick/armprime/math.html
I think the following. The relative FFT speed/efficiency is (Data_Buffer_Size)/(Data_Buffer_Size+Recovery_Buffer_Size). Normally it is about 80-90%.
But you want to use say 256M RAM for Data_Buffer_Size and say 8G RAM for Recovery_Buffer_Size,
so FFT usability will be 1/32. So this slowdown recovery creation in 32 times as compared with the temp file case.
sorry I don't understand why use 256 MB for data and 8GB for recovery, instead for example of 4 GB each.
And why should FFT slowdown recovery instead of speeding it up?
Why you are so hate a swapping?
The problem is not swapping itself - it is that I'm bound on low free space, I fullfill whatever HD I buy.