ZFS in VMs

[Database Administrator]

On Nov 12, 1:22 pm, Casper H.S. Dik wrote:

> If it works on the real hardware then IT MUST BE a problem
> in the software implementation of the hardware device.

I don't think that's correct (although I am not implying this is a
ZFS / driver bug). It could be the case that there is something that
the real hardware *could* do, but actually never does do, but emulated
hardware does. To be specific, it might be the case that there is some
behaviour of the hardware/interface which is within its specification,
but that actual hardware never does in practice. Timing issues might
be a good example of this: it might be legal (say) for a SCSI device
to exhibit timing behaviours which no real SCSI device does, although
the emulated one might. So there could be "latent" bugs in drivers
which are never exposed for real hardware (or for hardware that
currently exists, anyway).

I would think it's fairly hard for VMs not to have fairly different
timing characteristics than real machines, because they're constrained
by, on the one hand, the requirement not to stretch or compress real
time, and on the other hand uncontrollable influences from the host
OS.

However, I suspect strongly that what I am seeing here is not a client
OS issue, but either with the VM or the host OS / hardware.

--tim

[Database Administrator]

Chris Morgan writes:

>Casper H.S. Dik writes:

>> >Ever heard of driver bugs? Solaris has 'em too.
>>
>> If it works on the real hardware then IT MUST BE a problem
>> in the software implementation of the hardware device.

>Sorry that seems much too strong a claim. Consider the hypothetical
>case where solaris works perfectly on all known hardware. Then a new
>design is encountered and Solaris fails to work on it. Either the new
>hardware has a problem, or Solaris has a problem that was never
>uncovered before. I'm sure the latter happens from time to time.

I could follow this until you write ".. uncovered before".
It depends a bit on what type of device it is; if it's device
which uses a specific protocol, say SCSI, then, if the device
properly implements the protocol but in a new way, then I would
say that the device driver is fault.

But if it's a specific device driver where the implementation
of the driver is coupled to a specific silicon implementation,
then a problem in the virtualization is its fault.

>Well this is the same situation, only the hardware is emulated. The
>emulation does not change the fact that either the behaviour of the
>"hardware" is wrong, or Solaris has a previously-undiagnosed bug in
>dealing with that hardware.

Not really as that problem never occurs on real hardware.

>For example, let's say the emulated disks break the ATA
>protocol. That's an emulation bug.

>However, let's say they do one particular operation "too slow" (or too
>fast!) and encounters a hitherto unnoticed race condition in Solaris
>and/or the device driver. That's a Solaris and/or device driver bug,
>no?

Depends on what the specsheet says.

The virtual machines often cut some corners; e.g., making sure that
data has been written or that partial order on writes is kept
might be too expensive.

Specifically the I/O stack becomes quickly convoluted:


we have ZFS
we have some virtual disks
there are files which are the virtual disks
these files might live on some remote filesystem
the filesystem sends the files to the disk

and ZFS will ONLY work if a "commit this group" at the top
will result in data to be written at every level.
Synchronously!

Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

[Database Administrator]

> So that leaves me with two awkward possibilities:
>
> 1. Both of the VM systems (presumably using independent code) are
> broken and cause corruption in the virtual disk.
> 2. ZFS has bugs.
>
> Has anyone seen anything like this?


Just 2 hints:

1. install smartctl and verify disk

http://sixtyfive.xmghosting.com/products/smartctl/
/Applications/smartctl.app/Contents/Resources/smartctl -t long /dev/
disk0
wait 2 hours
/Applications/smartctl.app/Contents/Resources/smartctl -a /dev/disk0
look at SMART Self-test log

2. be sure you not running some not standard OSX kernel driver

For example, by using coolbook (which replaces OSX power saving
driver) I went into big problems (almost reliable Solaris10 panic (as
vmware guest) during high IO activity, random vmware crash)

After deleting coolbook, everything now works as expected

Regards,
Daniel

[Database Administrator]

On Tue, 11 Nov 2008 07:29:06 -0800 (PST) in comp.unix.solaris, Tim Bradshaw
in glistered weave wrote large for all to see:

>I have had various Solaris VMs, based previously on Parallels, and now
>on VMware Fusion, both (obviously) hosted on a mac.
>
>The Parallels instances used to quite regularly indicate checksum
>errors from ZFS, to the point that I scrapped it and bought Fusion.
>
>I now get checksum errors in Fusion.
>
>Of course, the obvious conclusion is that the underlying disk is just
>bust, but, although the Mac FS probably will not detect errors, I
>assume that at the rate I am seeing them I would be getting weird
>behaviour from the machine &/or unexplained file corruption. I have
>never seen this, and I use this machine for ~8 hrs a day 5 days a
>week.
>
>So that leaves me with two awkward possibilities:
>
>1. Both of the VM systems (presumably using independent code) are
>broken and cause corruption in the virtual disk.
>2. ZFS has bugs.
>
>Has anyone seen anything like this?

Yes. In a different venue, but it looks like it may be a similar situation
(not the same, similar)

(My background is not Unix, but IBM 360 architecture, and a smattering of
picked up and transfer knowledge from there to the descendants of QDOS, so I
am a late-comer and most of Unix is Greek to me.)

I am not familiar with Parallels or Fusion, but some problems span
platforms. As info, I am using VirtualBox for Linux and Solaris platforms
and MS-PC 2007 for Windows and DOS platforms.......

Question: Are you using dynamic (expanding) disks?

FACE