Memory leak reading HDF5-files using listvarinfile() and load() without variable names
Reported by Tilman Blumhagen
Originally assigned to Tilman Blumhagen
BUG DESCRIPTION:
----------------
When reading HDF5 based scilab data there is a memory leak in listvarinfile().
The same leak affects load() when it is called without variable names.
This problem is serious for appliations that have to load many files with an unknown list of variables.
ERROR LOG:
----------
Watch any process monitoring tool and see how the scilab process uses more and more memory.
HOW TO REPRODUCE THE BUG:
-------------------------
// Run this snippet and watch the memory usage. If you wait long enough the
// process will die because memory allocations can no longer be served.
fn = tempname();
myVariable = [1 2 3; 4 5 6];
save(fn, 'myVariable');
while %t
[names, types] = listvarinfile(fn);
mprintf('names: %s\ntypes: %s\n', sci2exp(names), sci2exp(types));
end
OTHER INFORMATION:
------------------
Stacktrace of the leaking code path:
MSVCR100D!malloc+1B
hdf5!H5MM_malloc+1E (f:\downloads\hdf5-1.8.16\hdf5-1.8.16\src\h5mm.c, 66)
hdf5!H5FL_malloc+24 (f:\downloads\hdf5-1.8.16\hdf5-1.8.16\src\h5fl.c, 199)
hdf5!H5FL_reg_malloc+13C (f:\downloads\hdf5-1.8.16\hdf5-1.8.16\src\h5fl.c, 399)
hdf5!H5T_copy+10F (f:\downloads\hdf5-1.8.16\hdf5-1.8.16\src\h5t.c, 3131)
hdf5!H5A_read+1B7 (f:\downloads\hdf5-1.8.16\hdf5-1.8.16\src\h5aint.c, 646)
hdf5!H5Aread+22A (f:\downloads\hdf5-1.8.16\hdf5-1.8.16\src\h5a.c, 641)
hdf5!readAttribute+145 (c:\src\scilab\scilab\modules\hdf5\src\c\h5_readdatafromfile.c, 168)
hdf5!getScilabTypeFromDataSet+1B (c:\src\scilab\scilab\modules\hdf5\src\c\h5_readdatafromfile.c, 1037)
hdf5!read_data+13 (c:\src\scilab\scilab\modules\hdf5\sci_gateway\cpp\sci_listvar_in_hdf5.cpp, 262)
hdf5!sci_listvar_in_hdf5+3AD (c:\src\scilab\scilab\modules\hdf5\sci_gateway\cpp\sci_listvar_in_hdf5.cpp, 154)
core!callFunctionFromGateway+7E (c:\src\scilab\scilab\modules\core\src\c\callfunctionfromgateway.c, 46)
The leaking memory block is allocated in H5A_read() where it says "Check for type conversion required" and copies a type information objects. For me it is not obvious who is responsible for freeing this copy. I am not an expert in the HDF5 API. Probably the scilab code forgets to close one of the many handles.