Re: Memory still not being released

On 30 Aug, Michael Drake wrote in message
<54010C15.8040209@netsurf-browser.org>:

> So that's how NetSurf deals with memory. On RISC OS you have a further
> issue. NetSurf uses UnixLib, which implements the memory allocation stuff
> using DAs. If a DA contains contents 1, 2 and 3, and they sit in memory
> like this
>
> 111111222222222222222333333
>
> and NetSurf says it wants to free the allocation for 2, the area now looks
> like this
>
> 111111 333333
>
> The space between 1 and 3 is not used by NetSurf any more but on RISC OS,
> it isn't available for use by other apps. Next time NetSurf allocates
> some memory it might not be given the space between 1 and 3, but something
> beyond 3, causing the DA to gow.
>
> I think UnixLib can reduce the size of the DA, when the last 64 Kb (or
> something) is unused, but in practice that never happens.
>
> If UnixLib used sparse DAs, then the unused memory between what NetSurf's
> using would be available for other apps.

I think it's worth adding that the "many other apps [which] manage to
release their DAs when documents are closed", referred to by Chris, are
almost certainly using some kind of sliding block heap (such as the
ubiquitous flexlib for C) to manage their memory heaps.

Given the amount of different bits of data that NetSurf has to keep track
of, using flex would be near impossible due to the restrictions that it
imposes on an application. No copies can be taken of flex block pointers,
because they're not static: there was an interesting discussion on
csa.programmer recently about an obscure bug in Pluto which was apparently
caused by the simple act of taking a local copy of a flex pointer for use
within a single loop. Bugs resulting from this kind of thing can be near
impossible to reproduce and track down, due to the fact that the flex heap
can shift and invalidate copies of pointers at *any* time (even when memory
isn't being claimed from or freed back to the heap, in some set-ups).

>From what I've seen, NetSurf uses pointers to allocated blocks inside its DA
as 'handles' to refer to the stored objects: that's pretty standard practice
everywhere but on RISC OS. This isn't just a NetSurf issue: two other apps
that I have detailed knowledge of (CashBook and Locate) use flex, and their
memory handling is orders of magnitude more complex (and hence harder to
keep bug free, due to the potential for issues like the one found in Pluto)
than it would have been if I'd written them for a more sane operating
system. I wouldn't fancy converting NetSurf's memory handling to use flex
(if it were even an option: NetSurf runs on other platforms, so flex can't
be used).

The bottom line is that RISC OS needs to handle malloc()-controlled heaps
properly: both UnixLib (and the Shared C Library, which has similar issues)
need to use sparse DAs to release their free memory back to the system.

> Finally, I must say, please create bug reports for issues like this, with
> log files. If for example, we're leaking DOM trees right and left, it
> will be clear for the log data and we can fix it. Without hard
> information, we're only guessing.

Definitely. There could still be memory leaks in NetSurf (NetSurf itself can
re-use freed memory from within its DA, as long as there are big enough
blocks of contiguous free memory available there), so problems do need
investigating.

Proper bug reports, with full log files, are essential, however.

--
Steve Fryatt - Leeds, England

http://www.stevefryatt.org.uk/