Re: [Rpcemu] Large files in HostFS was Re. RPCEmu Mac OS X test build of 0.8.14

Hi,

In article <20160219203733.GA16763@spod.org>,
Peter Howkins <rpcemu.howkins@marutan.net> wrote:
> On Tue, Feb 16, 2016 at 05:52:36PM +0000, Sprow wrote:
> > In article <20160216173419.GA20045@spod.org>,
> > > For the long explanation of why this HostFS patch is not included in
> > > RPCEmu, please see this post from 2011.
> > >
> > > http://www.riscos.info/pipermail/rpcemu/2011-October/001383.html
> >
> > I recall that discussion at the time, and the thread continued
> > http://www.riscos.info/pipermail/rpcemu/2011-October/001384.html
> > then fell silent.
>
> Unfortuanately it is not harmless on OSes < 5.20 nor harmless on >= 5.20.
>
> At this time it seems to be important to state the requirements that
> HostFS should have. These may only have been implicit before.
>
> 1) HostFS needs to work on all versions of RISC OS that RPCEmu can run (at
> least 3.5, possibly even 3.1 for arcem support).
> 2) RPCEmu users can place any size file in their HostFS directory on the
> Host Side, and HostFS must handle this gracefully.
> 3) HostFS should protect RPCEmu user's data from loss or corruption.
>
> One specific case discussed here of 3) is that allowing files to be opened
> that are larger than the maximum size that RO supports risks data loss, as
> a program can only work on only some of the data in a file, which can cause
> corruption.

I think we've come full circle now, because it was when I spotted that RPCEmu
was causing loss and corruption that I traced the problem to HostFS not
implementing 4GB files properly.

> Here follows a table describing the current situation in RPCEmu.
> I have used the phrase 'Data Safe' to represent that they don't allow
> files to be opened that are larger than OS supports.
>
> 32bit builds 64bit builds
>
> RO 3,4,6 Data Safe (2GB filesize limit) Data Unsafe (no filesize limit)
> RO 5 Data Safe (2GB filesize limit) Data Unsafe (no filesize limit)
>
> This is the effect your patch has:
>
> 32bit builds 64bit builds
>
> RO 3,4,6 Data Unsafe (no filesize limit) Data Unsafe (no filesize limit)
> RO 5 Data Unsafe (no filesize limit) Data Unsafe (no filesize limit)

I used the following simple test program

*Create test 7FFFFFF0
F=OPENUP"test"
IFF=0 THENPRINT"Where's it gone?":END
PRINT~EXT#F
PTR#F=EXT#F
BPUT#F,"The quick brown fox jumps over the lazy dog"
PRINT~EXT#F
CLOSE#F

I only checked with RISC OS 3.71 (since you'd grouped it with 4 & 6, I'd tend
to agree they'll behave the same) and RISC OS 5.22.

With a vanilla 0.8.14 download I get
RO 3.71 Written data is lost, file remains at &7FFFFFF0.
RO 5.22 Written data is lost, but extent is reported at &8000001C,
file on host disc is now &80000400. Attempting to *DUMP the data
carries on beyond &80000400 because EOF is never reported.

With 0.8.14 with my HostFS patch applied
RO 3.71 Written data is lost, file remains at &7FFFFFF0.
RO 5.22 Written data is saved, file is now &8000001C.

No doubt other more complex edge cases can be dreamed up, I just picked an
easy one to do in BASIC here.

Casual inspection of hostfs.c suggests that there's no extra code in there to
guard against case (3) it's just a happy coincidence that the original file
I/O functions were limited to 2GB, and RISC OS had problems above 2GB.

You're quite right about (2) in that a file parachuted in from outside the
emulator could cause confusion (larger than 2GB for 3,4,6; larger than 4GB
for 5) but I've managed not to do that in 4 years of using the patch,
Sprow.


_______________________________________________
Rpcemu mailing list
Rpcemu@riscos.info
http://www.riscos.info/cgi-bin/mailman/listinfo/rpcemu