Tag Archives: FILENAME

Linux

Count and print the number of occurences

Leave a comment

By arch

I have some files as shown below

GLL ALK 654-656 654 656
SEM LYG 655-657 655 657
SEM LYG 655-657 655 657
ALM ARN 656-658 656 658
ALM ARN 656-658 656 658
ALM ARN 656-658 656 658
LEG LEG 658-660 658 660
LEG LEG 658-660 658 660

The value of GLL is 654. Th value of ALK is 656. In the same way, 4th column represents the values of first column. 5th column represents the values of second column.

I tried the following program to count the occurrences of each number in the fourth and fifth column.

for i in folder1/*.pdb;
do
awk ‘
BEGIN {
path=sprintf(“%s”, “/home/arch/Desktop/folder2/”)
}
!s[1″:”$4″:”$5]++{sU[$4]++;tot++}
!s[2″:”$4″:”$5]++{sU[$5]++;tot++}
END {
sub(/.*//,””,FILENAME)
for (x in sU)
print x, sU[x], sU[$1] > path FILENAME;
}’ $i;
done

The above program prints as follows

660 1
654 1
655 1
656 2
657 1
658 2

But I would like to get the output as follows

660 LEG 1
654 GLL 1
655 SEM 1
656 ALM 2
657 LYG 1
658 LEG 2

your suggestions would be appreciated!!

…read more

Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

Conditional awk

Leave a comment

By EAGL€

Hello All,

I have a file like this:

Code:
bash-3.00$ cat 1.txt
201112091147|0|1359331220|1025
201112091147|0|1359331088|1024
201112091144|0|1359331172|1025
201112091147|0|1359331220|1021
201112091149|0|1359331088|1027
201112091144|0|1359331172|1029

and a list of MSISDNs in another file where there is only one column which is MSISDN:

Code:
bash-3.00$ cat 2.txt
201112091149
201112091140

I need to print out the records from first file 1.txt where MSISDN from second file 2.txt equals to the first column of 1.txt. so output should like this (at the end there must be filename too):

Code:
201112091149|0|1359331088|1027,1.txt

I have tried something like this but could not finalise and make it work, appreciate any help to write a report with the msisdns and the file names which they are in.

Code:
nawk 'FNR=NR { a[$1] = $1; next }{ printf ("%st%sn", $0, FILENAME)}' 1.txt 2.txt

KR.

…read more

Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

How to get result from both of these modules?

Leave a comment

By bluemind2005

Hallo All unix Guru’s

I have written a small (static) pl/sql function which is inserting data into database. It is as follows:

Code:
sqlplus $CONNECT <<-EOF
DECLARE
Counter integer :=1;
BEGIN
WHILE Counter <= ${no_of_files} LOOP
INSERT INTO stuck_files(COL_DATE,SER_NAME,TYPE_FILE,FILENAME,FILE_STATUS) VALUES (sysdate,'pqrs','pqrs','pqrs','pqrs');
Counter := Counter + 1;
END LOOP;
END;
/
commit;
exit;
EOF

However the above “insert” needs to be more of dynamic in nature, as such you can see that we want to insert data depending upon the variable value ${no_of_files}.

Now let me show you another module which is giving me the file(s)names which I want to insert.

Code:
N=0
for i in $(find . -path */waiting/* -type f -not -name "SS*" -mmin +120 -print) ; do

testarray[$N]="$i"
echo "$i"

let "N= $N + 1"
done

Sample Output from above command is as follows:

Code:
/d1/d2/d3/d4/waiting/abcd.txt
/d1/d2/d31/d42/waiting/pqrs.txt
/d1/d2/d32/d43/waiting/xyz.txt

Now the data insert which is given in the above pl/sql block has directly to do with this output. It going to be dynamic in nature.

Let us come back to mapping of how we need to insert data based on this output. I will go through the first insert.2 insert and 3rd insert so you would get an idea of design.
But these are very dynamic, meaning the insert depends on the output produced but yes the directory structure would remain same.

Code:
COL_DATE,SER_NAME,TYPE_FILE,FILEFILENAME,FILE_STATUS
sysdate,d3,d4,abcd.txt,waiting
sysdate,d31,d42,pqrs.txt,waiting
sysdate,d32,d3,xyz.txt,waiting

I believe this can definitely be implemented with help of magic like awk or anything but not able to implement it.

Any ideas/help would be greatly appreciated.

Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

File type error (not a regular file)

Leave a comment

By Jewel

Hi friend,

i have written script as below to check the file existance. but i got error

Code:
path="/k/p1100/users/jewel/Output"

FILENAME=`ls -lrt $path/*HT|tail -1|cut -d "/" -f 8`
if [ -s $path/$FILENAME ] ; then
echo "$FILENAME is available "
chmod 755 $path/$FILENAME
/usr/bin/scp $path/$1 xxx@333:xxx/inbox/
else
echo "FILE IS NOT AVAILABLE"
fi

error:
/k/p1100/users/jewel/Output: not a regular file

From: http://www.unix.com/shell-programming-scripting/221721-file-type-error-not-regular-file.html

Linux

Call a awk script with variable and input filename

Leave a comment

By Onkar Banerjee

HI,

MY question is a very simple one:

if i want to call an awk script with the input file name and also pass a variable value , then how to do it.

#>awk -f my_script.awk -v variable=value my_inputfile.txt

I can’t do it like this.
throws error:
awk: my_script.awk:18: (FILENAME=y_inputfile.txt FNR=1) fatal: attempt to access field -2147483648

Any idea how to do it then?

From: http://www.unix.com/shell-programming-scripting/221567-call-awk-script-variable-input-filename.html

Linux

Urgent:rename file to kanji — Solaris

Leave a comment

By tariq_m

Hi,

My shell script calls a perl script to create an excel and the shell
script emails the excel. This excel file needs to be renamed
to some Kanji name.

I have a flat file that has the required file name in kanji and i extract it
within the shell script and try to rename the file, but the filename shows junk.

the flat file itself has proper data coz the excel gets some kanji from the same flat file and is displayed properly.

fname=`grep FILENAME ${abcd}/etc/flat.cfg`
fname=`echo “${fname}” | awk ‘{ print substr($0,13,100) }’`
echo ${fname} –> itself displays junk 🙁

please help.

…read more
Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

Why result is wrong here ? whether break statement is wrong ?

Leave a comment

By Akshay Hegde

Hi ! all I am just trying to check range in my datafile

pls tell me why its resulting wrong

Code:
admin@IEEE:~/Desktop$ cat test.txt
0 28.4
5 28.4
10 28.4
15 28.5
20 28.5
25 28.6
30 28.6
35 28.7
40 28.7
45 28.7
50 28.8
55 28.8
60 28.8
65 28.1
70 26.5

This I tried

Code:
awk 'FNR==1{i++}{LC[i]=NR}
{for(k=1; k<=NF; k++)A[i,FNR,k]=$k} #ARRAY
END{
j=1;
while(j<=LC[1]) # IF J<=Line Count do following
{
x=A[1,j,2]-A[1,1,2] # Column2 records - Column2 1st Record
if(x>0.1) # IF above statement result is greater than 0.1
break; # Break loop, save index in variable,save column2 record
p1=j;
h1=A[1,j,1]; # IF not increment j
j++;
}

l=1;
while(l<=LC[1]) # IF l<=Line Count do following
{
y=A[1,l,2]-A[1,1,2] # Column2 records - Column2 1st Record
if(y<-0.1) # IF above statement result is less than 0.1
break; # Break loop, save index in variable,save column2 record
p2=l;
h2=A[1,l,1]; # IF not increment l
l++;
}
{
for(m=p1;m<=p2;m++) # Use Index stored in p1 and p2 of 1st and 2nd while loop print records in that range
{print A[1,m,1],A[1,m,2],h1,h2,p1,p2,FILENAME,x,y,A[1,1,2],A[1,1,1]}
}
}' OFS="t" test.txt

Resulting

Code:
admin@IEEE:~/Desktop$ sh awk.sh
10 28.4 10 60 3 13 test.txt 0.1 -0.3 28.4 0 --This is wrong
15 28.5 10 ...read more
Source: FULL ARTICLE at The UNIX and Linux Forums
Linux

How to print and append output of nawk script in commandline and as well into a file?

Leave a comment

By Optimus81

Hi All,
I am working on nawk script, has the small function which prints the output on the screen.Am trying to print/append the same output in a file.

Basically nawk script should print the output on the console/screen and as well it should write/append the same result to a file.

script :

Code:
nawk 'BEGIN {CmpAr["VerifyInterface"]=2 # some definitions
CmpAr["CoreFileGenerated"]=3
CmpAr["VerifyRegister"]=4
CmpAr["VerifyComponent"]=5
CmpAr["VerifyServiceAvailability"]=6
split ("%20s %30s %22s %29s %26s %35s", FMT, " ")
red="33[1;31m"
grn="33[1;32m"
rst="33[0m"
clr["Failed"]=red
clr["Passed"]=grn
}
#function prt(){printf"|";for (i=0; i<=6; i++) printf FMT[i], clr[OutAr[i]] OutAr[i] rst; printf " |n" }
function prt(){
printf"|";for (i=0; i> $AUTOMAT_DEST/${date_file}_Daily_Check_result"
}
#NR == FNR {print; next} # read & print header
#($2 in CmpAr) {gsub (" ", "", $1);
($2 in CmpAr) {gsub (/[ t]/,"",$1);
OutAr[CmpAr[$2]]=$1; # put $1 into the right column
OutAr[1]=$3} # save server name
FILENAME != oldfn { # print if old file ends
...read more
Source: FULL ARTICLE at The UNIX and Linux Forums
Linux

How to combine awk and bash commands in script ?

Leave a comment

By Akshay Hegde

Dear friends,

I am just trying write one script using 2 files
one file will contain details like below

Code:
#X SERVER X LOCATION
URL="http://www.abcd.com"
FILENAME="abc.txt"
ID_NAME="myabc_xyz"
SERVER_PATH="/usr/local/dummy/html/....."

#Y SERVER Y LOCATION
URL="http://www.xyz.com"
FILENAME="xyz.txt"
ID_NAME="my_dummy_xyz"
SERVER_PATH="/usr/local/server/htm/dummy"

#DUMMY SERVER
URL="http://www.dummy.com"
FILENAME="dummy.txt"
ID_NAME="LMNOP"
SERVER_PATH="/home/user/abc"

another file will be shell script to process above details
which looks like this

Code:
cd /mypath
sh javascript.sh -x $URL

mv myjava_file $FILENAME

sed -e "s/abcdic...... name="/$ID_NAME name="/g" $FILENAME

find $SERVER_PATH -iname "*$FILENAME" -delete

mv $FILENAME $SERVER PATH

Here my problem is how do I use above server detail one after another in processing shell script ?

processing shell script should use X server and X location details such as URL, SERVER_PATH, FILENAME, ID_NAME first, then Y server, Z server till end of file..

please help…

…read more
Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

Cp: cannot stat No such file or directory

Leave a comment

By Rami Reddy

Hi

Please review this script and let me know what i need to do.
This is my script

Code:
#!/bin/bash
#SCRIPT: forms.sh
#PURPOSE: Process a file line by line with redirected while-read loop.
#PURPOSE: and copy the forms to the follder

foldername=sample_dir
mkdir -p $foldername

while read file;
do
FILENAME=$(basename $file)

LINE=$FILENAME
count=0

cp $LINE $foldername/
done < forms.txt

i have a file called forms.txt which contains unix directory path.
i want to extract the filename alone from the unix directory path. now i am having copy issue with this script.
This is the error, i am getting

Code:
cp: cannot stat `AKDAPREG.fmb': No such file or directory
cp: cannot stat `AKDATTRS.fmb': No such file or directory
cp: cannot stat `AKDFLOWB.fmb': No such file or directory
cp: cannot stat `AKDFLOWS.fmb': No such file or directory
cp: cannot stat `AKDOBFKS.fmb': No such file or directory
cp: cannot stat `AKDOBJWB.fmb': No such file or directory
cp: cannot stat `AKDOBPKS.fmb': No such file or directory
cp: cannot stat `AKDPICKF.fmb': No such file or directory
cp: cannot stat `AKDQUERY.fmb': No such file or directory

Please let me know your suggestions.

Thanks,
Rami Reddy.

…read more
Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

Copying a file using variable

Leave a comment

By Rami Reddy

Hi this is the code
i am using copying a file by using a variable
i have a file test.txt which contains 5 lines

Code:
#!/bin/bash
#SCRIPT: method2.sh
#PURPOSE: Process a file line by line with redirected while-read loop.

FILENAME=test.txt
count=0
foldername=OUTPUT
mkdir $foldername

chmod 777 $foldername

while read LINE
do
(( count++ ))
cp $LINE $foldername/

done < $FILENAME
echo -e "nTotal $count Lines read"

now, i am getting this error

Code:
cp: cannot stat `$AP_TOP/reports/US/APXINSWP.rdf': No such file or directory
cp: cannot stat `$AP_TOP/reports/US/APXINROH.rdf': No such file or directory
cp: cannot stat `$AP_TOP/reports/US/APXWTGNR.rdf': No such file or directory
cp: cannot stat `$AP_TOP/reports/US/APXVDREV.rdf': No such file or directory
cp: cannot stat `$AP_TOP/reports/US/APXVDLBL.rdf': No such file or directory

Please help me.

Thanks

…read more
Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

Process ssh command in while loop

Leave a comment

By westmoreland

I have a script that reads a file containing a list of server names. It’s suppose to loop through the list of names and execute a command on the remote server using ssh. It processes the ssh command for the first server in the list and then exits. Here’s the code:

#!/bin/bash

FILENAME=server_list.txt

cat $FILENAME | while read LINE
do
echo "$LINE"

# Run the Users.sh script on the remote server
ssh root@$LINE '/usr/local/bin/Users.sh > /root/UserAccounts.txt'

done

When I comment out the ssh command the script processes as expected. Does ssh do something when it exits the first time that is maybe stopping the rest of the script from processing? I’d appreciate any suggestions.

…read more
Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

Bash script puts r at the end of variable

Leave a comment

By westmoreland

I’m pretty new to bash scripts and I’m trying to work through some issues. Would appreciate any suggestions.

I have a list of servers in a text file (I used the FQDN‘s), I assign the file name to a variable, I then use cat to read the list of file names and echo them to the screen. But when it assigns the variable it seems to append a r to the end of the variables value. Although the r doesn’t show up unless I use the set command to debug. As you can see from code below, I’m trying to use a server list to perform a secure copy of my password files so that I don’t have to log on to each server. Here is the code snippet in question:

Code:
#!/bin/bash
#
set -x

FILENAME=server_list.txt

cat $FILENAME | while read LINE
do
echo "$LINE"
# mkdir /sysadmin/bin/passwd_audit/$LINE
# scp root@$LINE:/etc/passwd /sysadmin/passwd/$LINE-passwd.txt
# scp root@$LINE:/etc/group /sysadmin/passwd/$LINE-group.txt
# scp root@$LINE:/etc/sudoers /sysadmin/passwd/$LINE-sudoers.txt

done

set +x

Here are the first few lines in the file:

Quote:

api01r5v.lamar.edu
appworxdbdev.lamar.edu
appworxdbprod.lamar.edu
appwxdbdevr5p.lamar.edu

Here’s what the output from the script looks like:

Quote:

[root@netbkpmaster passwd_audit]# ./getinfo_script
+ FILENAME=server_list.txt
+ cat server_list.txt
+ read LINE
+ echo $’api01r5v.lamar.edur’
api01r5v.lamar.edu
+ read LINE
+ echo $’appworxdbdev.lamar.edur’
appworxdbdev.lamar.edu
+ read LINE
+ echo $’appworxdbprod.lamar.edur’
appworxdbprod.lamar.edu
+ read LINE
+ echo $’appwxdbdevr5p.lamar.edur’
appwxdbdevr5p.lamar.edu

Is this normal behavior? I’ve used this same script in the past and it’s worked fine for me. Not sure why it changed suddenly. Any suggestions would be greatly appreciated.

…read more
Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

Find and split the list of files with suffiz of seg**

Leave a comment

By Satish Shettar

Hi,.

I am writing a script to get the new files and split them.

Requirement
Find the new files under the path “/wload/scmp/app/data/OAS” (There are 5 sub folders).
Gunzip the files which are having .gz suffix.
Put the list of files in the filename in the format “fiels_to_split_2013012911300.txt where as the numeric part is date and time.
Perform the split command on the files to split each file containing 50000 lines.
The output of the split filename should have the same file with suffix seg** where ** can be either numbers or alphabets.

I got the code from Pikk45 and tried to add/modify as per the need.

find /wload/scmp/app/data/OAS -name “*.gz” -size +100000000 -type f -mtime -1 -exec gunzip {} ;
FILENAME=new_fiels_to_split$(date +”%Y%m%d%H%M”).txt
find /wload/scmp/app/data/OAS -type f -mtime -1 > /wload/scmp/app/data/OAS/$FILENAME
FILES=/wload/scmp/app/data/OAS/$FILENAME
FIRST=1
FILE=1
SPL=50000
while read file; do
if [ -e “${FILE}” ]; then
while [ $SPL -le `wc -l $FILES`]
do
sed -n ‘$FIRST,$SPLp’ $FILES > $FILES.seg{$FILE}
fi
done < ${FILES}
FIRST=`expr $FIRST + 50000`
SPL=`expr $SPL + 50000`
FILE=`expr $FILE + 1`
done

for example
the source files


-rw-rw-r– 1 scmpadm scmpuser 755861762 Jan 26 20:04 activity.log.20130125.gz
-rw-rw-r– 1 scmpadm scmpuser 0 Jan 26 20:05 trigger.activity.log.20130125
-rw-rw-r– 1 scmpadm scmpuser 913911244 Jan 27 20:24 activity.log.20130126.gz
-rw-rw-r– 1 scmpadm scmpuser 0 Jan 27 20:25 trigger.activity.log.20130126
-rw-rw-r– 1 scmpadm scmpuser 329351168 Jan 28 19:10 activity.log.20130127.gz

After gunzip

-rw-rw-r– 1 scmpadm scmpuser 3012939863 Jan 28 19:10 activity.log.20130127

Files after split

-rw-rw—- 1 scmpadm scmpuser 49877675 Jan 29 07:04 activity.log.20130127.segaa
-rw-rw—- 1 scmpadm scmpuser 47552800 Jan 29 07:07 activity.log.20130127.segab
-rw-rw—- 1 scmpadm scmpuser 45867421 Jan 29 07:08 activity.log.20130127.segac
-rw-rw—- 1 scmpadm scmpuser 48985303 Jan 29 07:09 activity.log.20130127.segad
-rw-rw—- 1 scmpadm scmpuser 50846638 Jan 29 07:10 activity.log.20130127.segae
-rw-rw—- 1 scmpadm scmpuser 50946829 Jan 29 07:13 activity.log.20130127.segaf
-rw-rw—- 1 scmpadm scmpuser 50968144 Jan 29 07:14 activity.log.20130127.segag
-rw-rw—- 1 scmpadm scmpuser 51758156 Jan 29 07:18 activity.log.20130127.segah
-rw-rw—- 1 scmpadm scmpuser 52290589 Jan 29 07:23 activity.log.20130127.segai
-rw-rw—- 1 scmpadm scmpuser 19329024 Jan 29 07:26 activity.log.20130127.segaj

Source: FULL ARTICLE at The UNIX and Linux Forums