How to improve 'find' by skipping .svn directories?

[Database Administrator]

Hi All,

I am developing with SVN and have to find some text in all *.c/*.h/*.S
files.

find ! -name '.svn' -a -name '*.c' | xargs grep ...
But the performance is not very good.
And I try
find ! -name '.svn'
the speed is not lower than the case where there are no any .svn
directories under my working directory.
Is there any option for `find' to improve performance?

Best Regards,
PRC
Nov 14, 2008

[Database Administrator]

On 2008-11-14, PRC wrote:
> Hi All,
>
> I am developing with SVN and have to find some text in all *.c/*.h/*.S
> files.
>
> find ! -name '.svn' -a -name '*.c' | xargs grep ...
> But the performance is not very good.
> And I try
> find ! -name '.svn'
> the speed is not lower than the case where there are no any .svn
> directories under my working directory.
> Is there any option for `find' to improve performance?
>
> Best Regards,
> PRC
> Nov 14, 2008

I don't know the exact syntax but you have to tell find not to check
subdirectories of .svn
Your instruction isn't ignoring the .svn directories, it's just not
displaying files named '.svn'

[Database Administrator]

PRC wrote:
>
> I am developing with SVN and have to find some text in all *.c/*.h/*.S
> files.

I also do this regularly. I strongly recommend to use zsh as the
interactive shell (it has _so_ many useful interactive features).
For your task:
grep whatever **/*.{c,h,S}
(OK, this excludes even all ".*" subdirectories/files,
but this is probably what you want, anyway).

> find ! -name '.svn' -a -name '*.c' | xargs grep ...

find . -name '.svn' -prune -o -name '*.c' -exec grep whatever '{}' '+'

[Database Administrator]

On Nov 14, 5:12*am, wimpunk wrote:
> On 2008-11-14, PRC wrote:
>
>
>
>
>
> > Hi All,
>
> > I am developing with SVN and have to find some text in all *.c/*.h/*.S
> > files.
>
> > find ! -name '.svn' -a -name '*.c' | xargs grep ...
> > But the performance is not very good.
> > And I try
> > find ! -name '.svn'
> > the speed is not lower than the case where there are no any .svn
> > directories under my working directory.
> > Is there any option for `find' to improve performance?
>
> > Best Regards,
> > PRC
> > Nov 14, 2008
>
> I don't know the exact syntax but you have to tell find not to check
> subdirectories of .svn
> Your instruction isn't ignoring the .svn directories, it's just not
> displaying files named '.svn'- Hide quoted text -
>
> - Show quoted text -

Maybe something like this is more like what you really want:

find . -type d -a ! -name '.svn' | xargs grep whatever '{}/*.[xhS]'

i.e. use "find" to find all directories except the ".svn" ones, then
use grep on all *.c/*.h/*.S files in each directory. Check the
syntax...

Ed.

[Database Administrator]

2008-11-14, 06:15(-08), Ed Morton:
[[[.]
> Maybe something like this is more like what you really want:
>
> find . -type d -a ! -name '.svn' | xargs grep whatever '{}/*.[xhS]'


I don't know of any xargs implementation that would expand
wildcards.

> i.e. use "find" to find all directories except the ".svn" ones, then
> use grep on all *.c/*.h/*.S files in each directory. Check the
> syntax...
[...]

More like:

find . -name .svn -prune -o -type f -name '*.[xhS]' -exec \
grep whatever {} +

Or if you really want to use wildcards, for instance so that
files are sorted by names in every directory:

find . -type d ! -name .svn |
sed 's/./\\&/g;s|.*|grep whatever &/*.[xhS]|' | sh

(note that the obove skips dot files, but not dot directories).

With zsh:

setopt extendedglob
grep whatever (^.svn/)#*.[xhS](.)

(skips dot files and dot directories)

--
Stéphane

[Database Administrator]

I don't understand why not use -exec or -execdir action...

man find is written:
> -exec command {} +
> The command line is built in much the same way that
> xargs builds its command lines

....
?

[Database Administrator]

2008-11-14, 12:35(+00), Martin Vaeth:
> PRC wrote:
>>
>> I am developing with SVN and have to find some text in all *.c/*.h/*.S
>> files.
>
> I also do this regularly. I strongly recommend to use zsh as the
> interactive shell (it has _so_ many useful interactive features).
> For your task:
> grep whatever **/*.{c,h,S}

Note that brace expansion occurs before filename generation and
with zsh, if a pattern doesn't match, the command is aborted. So
the above is like writing:

grep whatever **/*.c **/*.h **/*.S

So we scan the directories 3 times, and if there are no .S
files, the command will be aborted even if there are .h or .c
files (unless you do a setopt CSH_NULL_GLOB or setopt
NO_NOMATCH).

So you may prefer:

grep whatever **/*.[chS]
or (with EXTENDED_GLOB):
grep whatever **/*.(c|h|S)

> (OK, this excludes even all ".*" subdirectories/files,
> but this is probably what you want, anyway).
[...]

And if not:

grep whatever (^.svn/)#*.[chS](D)

See also the grep -r --include=... --exclude=... of GNU grep.

--
Stéphane

[Database Administrator]

Stephane CHAZELAS wrote:
>
> grep whatever **/*.c **/*.h **/*.S
>
> So we scan the directories 3 times, and if there are no .S
> files, the command will be aborted even if there are .h or .c
> files (unless you do a setopt CSH_NULL_GLOB or setopt
> NO_NOMATCH).

You are right. I just never realized this, because I have nonomatch
turned on - I have worked with bash for years, and essential different
behavior would give me lots of bad surprises (like this one...).

> So you may prefer:
>
> grep whatever **/*.[chS]
> or (with EXTENDED_GLOB):
> grep whatever **/*.(c|h|S)

The former has the disadvantage that it cannot be generalized to the
important case **/*.(cc|h) but fortunately the latter can.
Thanks. So again I learnt something new.

> grep whatever (^.svn/)#*.[chS](D)

But admittedly this is not much shorter than the find command...

[Database Administrator]

On 14 Nov, 10:50, PRC wrote:

> I am developing with SVN and have to find some text in all *.c/*.h/*.S
> files.

> Is there any option for `find' to improve performance?

$ git svn clone $SVN_URL git-copy
$ cd git-copy
$ git grep -e 'some text' *.[chS]

This will improve many aspects of performance.

[Database Administrator]

Stephane CHAZELAS wrote:
> 2008-11-14, 06:15(-08), Ed Morton:
> [[[.]
>> Maybe something like this is more like what you really want:
>>
>> find . -type d -a ! -name '.svn' | xargs grep whatever '{}/*.[xhS]'
>
>
> I don't know of any xargs implementation that would expand
> wildcards.
> [...]
>
> Or if you really want to use wildcards, for instance so that
> files are sorted by names in every directory:
> [...]

Wouldn't something like this work?
find . -type d -a ! -name '.svn' | xargs sh -c 'grep whatever "{}/*.[xhS]"'

-Wayne