Fixing Git "out of memory" errors on Windows

Googling around, I found a lot of people had the same problem with Git on Windows that I did: large files (where “large” means above 350MB or so) cause “out of memory” errors.

A quick look into the file causing the error, (ugh, perl), shows the reason right away: they’re doing nice, small 1KB reads of the file blob in a loop… but collecting the whole thing in memory before writing it all out. Not only does that use up a ton of needless memory for large files, it reallocates the buffer to fit everything it’s read on every 1KB read. Arg. Simple fix - write it out as you read it in, and don’t keep it around in memory. Viola.

I haven’t submitted this patch because, well, I don’t use Git all that often, and I don’t really care to jump through the hoops of writing tests to support it and all that. But if you’re hitting this error, maybe this will help.


©2011-2020 Justin C. Miller.   What a horrible night to have a curse.