+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 10 of 12

testing files for read errors by copying to /dev/null

  1. testing files for read errors by copying to /dev/null

    I would like to test the integrity of a CD (after burning from .iso) by
    reading all its files and looking for error messages. (I'm using Cygwin,
    so I cannot access the CD drive as a dev/, only as a mounted filesystem.)

    If I try
    cp -rv /cygdrive/d/* /dev/null
    then I get a message that /dev/null is not a file or directory

    If I try
    tar -cvf /dev/null /cygdrive/d
    then the command runs so fast it cannot possibly be reading the whole CD.

    If I try
    find /cygdrive/d -type f -name '*' -print -exec cat {} >/dev/null \;
    then the disk drive seems active for a reasonable length of time, but
    despite the -print flag, I get no output to confirm what is actually
    happening. I expected the filename to be printed before each file was
    read.

    What is the best way to do what I am attempting?

  2. Re: testing files for read errors by copying to /dev/null

    * Charles Russell :
    > I would like to test the integrity of a CD (after burning from .iso) by
    > reading all its files and looking for error messages. (I'm using Cygwin,
    > so I cannot access the CD drive as a dev/, only as a mounted filesystem.)
    >
    > If I try
    > cp -rv /cygdrive/d/* /dev/null
    > then I get a message that /dev/null is not a file or directory
    >
    > If I try
    > tar -cvf /dev/null /cygdrive/d
    > then the command runs so fast it cannot possibly be reading the whole CD.
    >
    > If I try
    > find /cygdrive/d -type f -name '*' -print -exec cat {} >/dev/null \;
    > then the disk drive seems active for a reasonable length of time, but
    > despite the -print flag, I get no output to confirm what is actually
    > happening. I expected the filename to be printed before each file was
    > read.


    The shell is responsible for redirection so...

    $ find /cygdrive/d -type f -print -exec sh -c 'cat "$1" >/dev/null' {} {} \;

    > What is the best way to do what I am attempting?


    Not sure whether it is the *best* way, but the following should work:

    $ find /cygdrive/d -type f -exec cp -v {} /dev/null \;

    --
    James Michael Fultz
    Remove this part when replying ^^^^^^^^

  3. Re: testing files for read errors by copying to /dev/null

    Charles Russell wrote:

    > I would like to test the integrity of a CD (after burning from .iso) by
    > reading all its files and looking for error messages. (I'm using Cygwin,
    > so I cannot access the CD drive as a dev/, only as a mounted filesystem.)
    >
    > If I try
    > cp -rv /cygdrive/d/* /dev/null
    > then I get a message that /dev/null is not a file or directory
    >
    > If I try
    > tar -cvf /dev/null /cygdrive/d
    > then the command runs so fast it cannot possibly be reading the whole CD.
    >
    > If I try
    > find /cygdrive/d -type f -name '*' -print -exec cat {} >/dev/null \;
    > then the disk drive seems active for a reasonable length of time, but
    > despite the -print flag, I get no output to confirm what is actually
    > happening. I expected the filename to be printed before each file was
    > read.
    >
    > What is the best way to do what I am attempting?


    The best way to check burned CDs is to calculate a hash on the medium and
    compare it with the hash of the original .iso file. If the medium is
    mounted, this is not possible (you can do mkisofs | md5sum on the directory
    where you mounted the medium, but the results will almost certainly be
    different even if the CD has no errors).

    If you are content with just trying to read all files on the cd and see
    whether any error is reported (a method far from reliable), then something
    like (on a single line)

    find /path/to/mnt -type f -print -exec sh -c 'cat "$1"
    > /dev/null' '{}' '{}' \;


    should kind of work.

    --
    All the commands are tested with bash and GNU tools, so they may use
    nonstandard features. I try to mention when something is nonstandard (if
    I'm aware of that), but I may miss something. Corrections are welcome.

  4. Re: testing files for read errors by copying to /dev/null

    Charles Russell wrote:

    >> The best way to check burned CDs is to calculate a hash on the medium and
    >> compare it with the hash of the original .iso file.

    >
    > My understanding is that the .iso file lacks the error correcting code
    > included when a CD is burned, so I would not expect hashes to match
    > unless the operating system handles this transparently. Just out of
    > curiosity (or not purely from curiosity; I could boot Puppy Linux from
    > CD and still have access to the CD drive) what would be the approach if
    > I did have direct access to the /dev ?


    Well, this has always worked for me:

    $ md5sum install-x86-minimal-2007.0-r1.iso
    81f244b5978b682cf9f2a313410cdbdf install-x86-minimal-2007.0-r1.iso

    Burn the .iso to the CD, and then:

    $ md5sum /dev/cdrom
    81f244b5978b682cf9f2a313410cdbdf /dev/cdrom

    (of course, you can use the hash tool of your choice, like sha1sum or
    others, Also, the CD device might not be /dev/cdrom in your system, but
    instead /dev/hd, or even something else).

    In any case, if the burned CD has errors in it (not just physical read
    errors - even single-bit errors), the hashes will be different.

    --
    All the commands are tested with bash and GNU tools, so they may use
    nonstandard features. I try to mention when something is nonstandard (if
    I'm aware of that), but I may miss something. Corrections are welcome.

  5. Re: testing files for read errors by copying to /dev/null

    pk wrote:
    > Charles Russell wrote:
    >
    >> I would like to test the integrity of a CD (after burning from .iso) by
    >> reading all its files and looking for error messages. (I'm using Cygwin,
    >> so I cannot access the CD drive as a dev/, only as a mounted filesystem.)
    >>
    >> If I try
    >> cp -rv /cygdrive/d/* /dev/null
    >> then I get a message that /dev/null is not a file or directory
    >>
    >> If I try
    >> tar -cvf /dev/null /cygdrive/d
    >> then the command runs so fast it cannot possibly be reading the whole CD.
    >>
    >> If I try
    >> find /cygdrive/d -type f -name '*' -print -exec cat {} >/dev/null \;
    >> then the disk drive seems active for a reasonable length of time, but
    >> despite the -print flag, I get no output to confirm what is actually
    >> happening. I expected the filename to be printed before each file was
    >> read.
    >>
    >> What is the best way to do what I am attempting?

    >
    > The best way to check burned CDs is to calculate a hash on the medium and
    > compare it with the hash of the original .iso file.


    My understanding is that the .iso file lacks the error correcting code
    included when a CD is burned, so I would not expect hashes to match
    unless the operating system handles this transparently. Just out of
    curiosity (or not purely from curiosity; I could boot Puppy Linux from
    CD and still have access to the CD drive) what would be the approach if
    I did have direct access to the /dev ?

    If the medium is
    > mounted, this is not possible (you can do mkisofs | md5sum on the directory
    > where you mounted the medium, but the results will almost certainly be
    > different even if the CD has no errors).
    >
    > If you are content with just trying to read all files on the cd and see
    > whether any error is reported (a method far from reliable), then something
    > like (on a single line)
    >
    > find /path/to/mnt -type f -print -exec sh -c 'cat "$1"
    >> /dev/null' '{}' '{}' \;

    >
    > should kind of work.
    >


    Yes, that works.

  6. Re: testing files for read errors by copying to /dev/null

    James Michael Fultz wrote:
    > * Charles Russell :
    >> I would like to test the integrity of a CD (after burning from .iso) by
    >> reading all its files and looking for error messages. (I'm using Cygwin,
    >> so I cannot access the CD drive as a dev/, only as a mounted filesystem.)
    >>
    >> If I try
    >> cp -rv /cygdrive/d/* /dev/null
    >> then I get a message that /dev/null is not a file or directory
    >>
    >> If I try
    >> tar -cvf /dev/null /cygdrive/d
    >> then the command runs so fast it cannot possibly be reading the whole CD.
    >>
    >> If I try
    >> find /cygdrive/d -type f -name '*' -print -exec cat {} >/dev/null \;
    >> then the disk drive seems active for a reasonable length of time, but
    >> despite the -print flag, I get no output to confirm what is actually
    >> happening. I expected the filename to be printed before each file was
    >> read.

    >
    > The shell is responsible for redirection so...
    >
    > $ find /cygdrive/d -type f -print -exec sh -c 'cat "$1" >/dev/null' {} {} \;



    Yes, that works

    >> What is the best way to do what I am attempting?

    >
    > Not sure whether it is the *best* way, but the following should work:
    >
    > $ find /cygdrive/d -type f -exec cp -v {} /dev/null \;
    >


    That yields the error message:
    cp: cannot create regular file `/dev/null': Invalid request code

    I don't know whether this is typical unix behavior or something peculiar
    to Cygwin, which is usually a pretty close emulation. As noted in my
    original posting, cp does not like /dev/null.

  7. Re: testing files for read errors by copying to /dev/null

    * Charles Russell :
    > James Michael Fultz wrote:
    >> $ find /cygdrive/d -type f -exec cp -v {} /dev/null \;
    >>

    >
    > That yields the error message:
    > cp: cannot create regular file `/dev/null': Invalid request code
    >
    > I don't know whether this is typical unix behavior or something peculiar
    > to Cygwin, which is usually a pretty close emulation. As noted in my
    > original posting, cp does not like /dev/null.


    My thought is that it is a pecularity of Cygwin since it worked when I
    tested on Linux.

    Your cp command lines seemed to fail for different reasons which are
    also failures on Linux. It is possible to copy a single file onto
    '/dev/null' which is itself a file, but it is not possible to copy
    directories onto '/dev/null' as the '-r' would require cp to do.

    --
    James Michael Fultz
    Remove this part when replying ^^^^^^^^

  8. Re: testing files for read errors by copying to /dev/null

    Charles Russell wrote:
    > James Michael Fultz wrote:
    >> * Charles Russell :




    >> $ find /cygdrive/d -type f -exec cp -v {} /dev/null \;
    >>

    >
    > That yields the error message:
    > cp: cannot create regular file `/dev/null': Invalid request code
    >
    > I don't know whether this is typical unix behavior or something peculiar
    > to Cygwin, which is usually a pretty close emulation. As noted in my
    > original posting, cp does not like /dev/null.


    It's cygwin. Remember that cygwin is an emulation running on top of
    windows. In linux /dev/null is a real file, whereas in cygwin it's
    almost an illusion. There are no real devices under cygwin, it simply
    uses the underlying windows resources with a lot of filtering. The real
    /dev/null file can actually be opened in linux. Under cygwin if you look
    at strace output you can see where cygwin passes /dev/null down to
    windows which tries to open a nul file which fails. It works with
    redirection because of some clever hacking that hides the fact there is
    no real /dev/null.

    Still it seems this used to work better in an earlier version of cygwin
    so one day I might go look at the code again and see if I'm rememberng
    badly or if it changed.

    I'm back :) It seems the codes changed on February 23 2007 for reasons I
    didn't grok completely with a first glance. The /dev/null open stuff was
    removed according to the changelog. It seems like they attempted to let
    windows try and open the windows nul device rather than keep emulating a
    cygwin fake null device.

    Anyway, I don't have any time to really run this down right now. I'm
    guessing that /dev/null won't work in any context where you are trying
    to open /dev/null (such as cp file /dev/null) but will work in
    redirection (such as ls /cygdrive/c/windows > /dev/null) I may be wrong
    but the code looks like that upon a quick glance.

    FWIW I tried copy file nul in a dos command line and it appears to work
    normally. So maybe somethng is broken with they way cygwin passes
    windows the open(nul) call. If that was fixed then the whole thing might
    work properly. I add this to my list of things to either find the
    problem and send a patch or file a real bug report unless someone else
    would like to step up.

  9. Re: testing files for read errors by copying to /dev/null

    Thanks everyone.

  10. Re: testing files for read errors by copying to /dev/null

    On Mon, 24 Mar 2008 20:33:43 +0100, pk wrote:

    > Charles Russell wrote:
    >
    >>> The best way to check burned CDs is to calculate a hash on the medium
    >>> and compare it with the hash of the original .iso file.

    >>
    >> My understanding is that the .iso file lacks the error correcting code
    >> included when a CD is burned, so I would not expect hashes to match
    >> unless the operating system handles this transparently. Just out of
    >> curiosity (or not purely from curiosity; I could boot Puppy Linux from
    >> CD and still have access to the CD drive) what would be the approach if
    >> I did have direct access to the /dev ?

    >
    > Well, this has always worked for me:
    >
    > $ md5sum install-x86-minimal-2007.0-r1.iso
    > 81f244b5978b682cf9f2a313410cdbdf install-x86-minimal-2007.0-r1.iso
    >
    > Burn the .iso to the CD, and then:
    >
    > $ md5sum /dev/cdrom
    > 81f244b5978b682cf9f2a313410cdbdf /dev/cdrom
    >
    > (of course, you can use the hash tool of your choice, like sha1sum or
    > others, Also, the CD device might not be /dev/cdrom in your system, but
    > instead /dev/hd, or even something else).
    >
    > In any case, if the burned CD has errors in it (not just physical read
    > errors - even single-bit errors), the hashes will be different.


    Hashes are great when you can use them to optimize performance, but in
    this case, it seems like a byte for byte comparison would be just as
    fast, and actually a little easier - using something like cmp.

    Also, be wary of your buffer cache, assuming that cdrom's aren't written
    to using O_DIRECT or similar. You could end up with the right stuff in
    cache but the wrong stuff on the physical media. Popping the cd and
    shoving it right back in should be sufficient to be sure you don't have
    the cd cached.


+ Reply to Thread
Page 1 of 2 1 2 LastLast