Scott Gress
556b79e2d2
for #24829 See https://github.com/axios/axios/issues/1045 -- by default Axios buffers uploaded files into memory fully, to support redirects. For large file uploads this means we get OOM errors, especially when sending to multiple teams. There's a few other optimizations we can put in place here but in the short term we can fix the buffering issue by setting `maxRedirects: 0` on the requests. I tested this by adding an `onUploadProgress` handler to the Axios request that dumps memory usage, and uploading a 209mb software file to 3 teams. Before the update, the readout ticket up continuously (the first number is the # of bytes uploaded): ``` 1540129 {rss: 161652736, heapTotal: 65880064, heapUsed: 55625552, external: 28411157, arrayBuffers: 24338844} edit-software.js:177 1554313 {rss: 149254144, heapTotal: 65880064, heapUsed: 52445200, external: 25193635, arrayBuffers: 21121327} edit-software.js:177 2339833 {rss: 151703552, heapTotal: 66404352, heapUsed: 52269280, external: 12664377, arrayBuffers: 8592064} ...a minute later... 192708641 {rss: 619323392, heapTotal: 95240192, heapUsed: 55320960, external: 618952429, arrayBuffers: 614879965} edit-software.js:177 201523233 {rss: 634613760, heapTotal: 95240192, heapUsed: 58514992, external: 636581613, arrayBuffers: 632509154} edit-software.js:177 209326677 {rss: 637399040, heapTotal: 95240192, heapUsed: 56800016, external: 639441633, arrayBuffers: 635369173} ``` so we start at ~161mb, and by the time we're done, we're using 637mb of RAM. Render's free tier has a 250mb limit on apps. With `maxRedirects: 0`, we see: ``` 2669337 {rss: 151846912, heapTotal: 66404352, heapUsed: 53297400, external: 26446868, arrayBuffers: 22374419} edit-software.js:177 2279929 {rss: 152641536, heapTotal: 66404352, heapUsed: 53453664, external: 27233300, arrayBuffers: 23160851} edit-software.js:177 2228585 {rss: 153038848, heapTotal: 66404352, heapUsed: 53537096, external: 27626516, arrayBuffers: 23554067} ...a minute later... 209326677 {rss: 146989056, heapTotal: 92094464, heapUsed: 53802856, external: 14617518, arrayBuffers: 10545071} edit-software.js:177 209326677 {rss: 153051136, heapTotal: 92094464, heapUsed: 55376336, external: 22447478, arrayBuffers: 18375026} edit-software.js:177 209326677 {rss: 152129536, heapTotal: 92094464, heapUsed: 51857632, external: 22447478, arrayBuffers: 16540013} ``` showing that we start and finish with around the same amount of RAM used.
Welcome to the "source available" section of the Fleet codebase. Please note files and functionality under this directory are covered by the Fleet EE License, and require a valid Fleet subscription for production use. See the full license for details.