awk help needed

[Jamon Camisso]

Chris F.A. Johnson wrote:
> On Sun, 19 Jul 2009, Jamon Camisso wrote:
> 
>> Chris F.A. Johnson wrote:
>>> On Sun, 19 Jul 2009, Jamon Camisso wrote:
>>>
>>>> Giles Orr wrote:
>>>>> 2009/7/18 Alex Beamish <talexb-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org>:
>>>>>> On Sat, Jul 18, 2009 at 11:46 AM, Giles Orr<gilesorr-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org> 
>>>>>> wrote:
>>>>>>> By way of introduction: I'm finally, finally trying to get a new
>>>>>>> edition of the Bashprompt HOWTO out there.  This will probably 
>>>>>>> result
>>>>>>> in me posting a lot of detailed and mildly weird questions to this
>>>>>>> list.  This is the first.
>>>>>>>
>>>>>>> The intention of this script is to figure out how much space the 
>>>>>>> files
>>>>>>> in the current directory take up.  There are about a million ways to
>>>>>>> do this - and yes, I know that "ls -l" spits out a "total" line: I
>>>>>>> don't know what it's totaling, but my math has never agreed with it
>>>>>>> ... feel free to explain though.  I decided that I'd like to do this
>>>>>>> as much in awk as possible since it does decimal math (unlike bash)
>>>>>>> and it's certainly the easiest way to do the text parsing.  I've 
>>>>>>> tried
>>>>>>> bc as well, but you have to use other utility programs to parse and
>>>>>>> split the input for it.
>>>>>> Sorry, but I usually use the du command for this.
>>>>>>
>>>>>>  root at music:/etc/init.d# du -h .
>>>>>>  504K    .
>>>>>>
>>>>>> Sometimes I want to know how heavy an entire tree is, so I use
>>>>>>
>>>>>>  $ du -sh foo
>>>>>>
>>>>>> The 'h' argument does intelligent size management, so shows K, M 
>>>>>> and G.
>>>>>
>>>>> I would love to use "du" because initially it seems like precisely the
>>>>> right tool ... but I want only the sum of the sizes of the files in
>>>>> the current directory, and "du" is by default recursive (which also
>>>>> makes it painfully slow to return, not a good thing for something
>>>>> incorporated into a Bash prompt).  If it's possible to stop "du" from
>>>>> recursing, I'll use it immediately - but that looks difficult to
>>>>> impossible.  Any thoughts?
>>>>>
>>>>> Thanks to everyone else who answered too: it all helped.  It certainly
>>>>> sounds like piping into "awk" is the way to go rather than trying to
>>>>> write a self-contained "awk" script.
>>>>>
>>>>
>>>> Try this:
>>>
>>>   DON'T.
>>>
>>>> #!/bin/bash
>>>>
>>>> i=0 j=0 k=0
>>>> echo -n "Enter directory: "
>>>> read dir
>>>> for i in `ls $dir`
>>>
>>>     Not only is ls unnecessary, but it will cause the script to fail
>>>     if any filenames contain spaces or other pathological characters.
>>>     Use:
>>>
>>> for i in "$dir"/*
>>>
>>>>  do
>>>>    if [ -f $i ]; then
>>>>      j=`du -s $i |awk '{print $1}'`
> 
>    Always quote variable references unless there is a good reason not
>    to: "$i". Without them, it fails if the value contains spaces.

Right, like your last point, if there are spaces. Guess it's a good 
habit to get into so I'll definitely make a point of it.

>>>>      k=`expr $k + $j`
> 
>     Why are you using an external command for integer arithmetic?

Dunno how to do it in bash, grimace. Than fancy $(( )) stuff? Will have 
a go at that instead.

>>>>    fi
>>>>  done
>>>> echo $k
> 
> du -s "$dir"/* | awk '{ total += $1 } END { print total }'

Or I'll just use awk since that's what started this thread :)

>> Interesting, I started with for i in * but that barfs on stuff too.
> 
>    There are other problems, too. I just reacted to the horrible
>    "for i in `ls dir`". There is no point to using ls without any
>    options; shell expansion can do it much more efficiently.
> 
>> Suppose since I don't have an files I can see with spaces in their
>> names (pathological hatred for such names), it appears to work
>> correctly for me :)
> 
>    I feel the same way of filenames with spaces, but they are hard to
>    avoid. I never create any, but I expect that every distro contains
>    many, and downloads often do. Unless you get into the habit of
>    handling them, your scripts will fail sooner or later.

Got it, I suppose I should approach writing scripts as more than one off 
single use tools and see about making them more robust, your pointers 
are very much appreciated.

Jamon
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists