Tag Archives: NR

Linux

Filter file by length, looking only at lines that don't begin with ">"

Leave a comment

By pathunkathunk

I have a file that stores data in pairs of lines, following this format:
line 1: header (preceded by “>”)
line 2: sequence

Example.txt:

Quote:

>seq1 name
GATTGATGTTTGAGTTTTGGTTTTT
>seq2 name
TTTTCTTC

I want to filter out the sequences and corresponding headers for all sequences that are less than 11 characters. Desired output:

Quote:

>seq2 name
TTTTCTTC

I can search each line for lines less than 11 characters, and print that line along with the header. The problem I’m having is ignoring the headers (i.e. lines beginning with “>”) when I do the length search.

For example

Code:
awk '{lines[NR] = $0} length($0) < 11 {print lines [NR-1]; print lines [NR]} ' example.txt

Gives me

Quote:

>seq1 name
GATTGATGTTTGAGTTTTGGTTTTT
>seq2 name
>seq2 name
TTTTCTTC

How do I tell awk not to ignore lines beginning with “>”?

…read more

Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

Extracting lines from a file with sed and awk

Leave a comment

By bash_in_my_head

My source file is structured with two words on each line

Code:
word1 word2
word1 word2

I am using sed and awk to grab groups of specific lines

Code:
line=`awk 'NR>=4 && NR<=7' file1`; echo $line
line=` sed -n '1,5'p file1`; echo $line

The resulting output is

Code:
word1 word2 word1 word2 word1 word2

These are all on one line. I need the output to be in the same format as the source file.

Code:
word1 word2
word1 word2

…read more

Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

Retaining file versions based on input parameter

Leave a comment

By pchang

Hello Forum.

I have the following files in one directory:

Code:
abc_july01_2013.txt
abc_july02_2013.txt
abc_july03_2013.txt
abc_july04_2013.txt
abc_july05_2013.txt
abc_july06_2013.txt
abc_july07_2013.txt
abc_july08_2013.txt

If I want to be able to keep the last 5 versions of the file and delete the rest so that all that remains is:

Code:
abc_july04_2013.txt
abc_july05_2013.txt
abc_july06_2013.txt
abc_july07_2013.txt
abc_july08_2013.txt

Then the following code will accomplish this:

Code:
rm -f `ls -tp abc* | grep -v "/$" | awk '{ if (NR > 5) print; }'`

But I don’t want to hard-code the # of versions (which is 5 in this case). I want to be able to pass in a parameter value to the command so that it’s more flexible. Is there an easy way that it can be done?

Thanks

…read more

Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

Pass awk array variable to shell

Leave a comment

By littlewenwen

Hi, all

suppose I have following myfile (delimited by tab)

Code:
aa bb
cc dd
ee ff

and I have following awk command:

Code:
awk 'BEGIN{FS="t"}{AwkArrayVar_1[x++]=$1;AwkArrayVar_2[y++]=$2};END{for(i=0; i<NR; i++) print i, AwkArrayVar_1[i], AwkArrayVar_2[i],}' myfile

My question is: how can I assign the awk array variable to shell array variable, such as ShellArrayVar_1 = AwkArrayVar_1, and ShellArrayVar_2 = AwkArrayVar_2?

From: http://www.unix.com/shell-programming-scripting/221659-pass-awk-array-variable-shell.html

Linux

How to append values to a string?

Leave a comment

By vkumbhakarna

Hi,

Requesting some help with a problem I am facing with string function in UNIX. I wish to create 2 string variables: 1st header string containing output_1, output_2, .. , output_ and 2nd data string containing the filename separated by colon (“:”) and corresponding filesize separated by comma as shown below
E.g.
output_1,output_2,output_3,
file1:303,file2:304:file3:443

I have written the following piece of code to perform the same, but I am not getting the values of the variables appended to my string variables header_str and output_str? Can you please advise how can i get the two string variables as shown below:
header_str=output_1,output_2,output_3,
output_str=file1:303,file2:304:file3:443,

Code:
set -x;
typeset -L header_str; typeset -L output_str;
typeset -i op_cnt=1;
ls -lrt |awk '{FS = ",|[ t]+"} NR>1 { print $9, $5 }'|tr " " ":"|while read line
do
header_str=$(echo $header_str,output_file$op_cnt)
output_str=$(echo $output_str,$line)
op_cnt=`expr $op_cnt + 1`
done

echo header_str is $header_str;
echo output_str is $output_str;

From: http://www.unix.com/unix-dummies-questions-answers/221631-how-append-values-string.html

Linux

Create csv with output filenames and file size

Leave a comment

By vkumbhakarna

Hello All,

Here is seeking a bit of help in trying to solve a problem.

I am required to create a csv file as shown below:
output.csv ->
output_1,output_2,output_3,…,output_
filename1:20,filename2:30,filename3:30,…,filename:30

by listing output_1, output_2,… , output as header separated by a comma in line 1 for all file names in a particular directory and their corresponding file names and file size(s) separated by a colon in line 2

I’ve tried to use the code shown below but I’m having trouble trying to resolve the value of the parameter in the log.

#capture the file size of output files
typeset -i op_cnt=1;
ls -lrt |awk ‘{FS = “,|[ t]+”} NR>1 { print $9, $5 }’|tr ” ” “:”|while read line
do
op_file_$op_cnt=`echo $line`
op_val_$op_cnt=”echo $$op_file_$op_cnt”
op_val_$op_cnt=`eval $op_val_$op_cnt`
echo “$op_file_$op_cnt=$op_val_$op_cnt” >>
done

Requesting your help and expert guidance.

From: http://www.unix.com/unix-dummies-questions-answers/221439-create-csv-output-filenames-file-size.html

Linux

Getting a command not found when comparing in tsch

Leave a comment

By tastybrownies

Hey everyone,

I am almost done with this assignment I have to do but there is one last thing nagging me that I can’t seem to find. In my script I am feeding line by line to a ./test file and I am getting the return answer correctly. The goal is to test the current script (check.sh) and if it’s output is the same as the testcase.txt file then everything is good. So I have most of the script done but am still getting the “command not found” issue. I’m thinking it’s something with my if statement and possibly the comparison. If anyone could point me in the right direct or let me know how to get rid of this error, I would sincerely appreciate it!!! Thank you! Below is my program and then the output it gives.

Code:
set dir="$PWD/./test"
set dir2="$PWD/testcase.txt"
set ODD=""
set EVEN=""
set oddlines=`sed -n 1~2p $PWD/testcase.txt > ODD`
set evenlines=`awk NR%2==0 $PWD/testcase.txt > EVEN`
set fodd=$PWD/ODD
set odd="`cat $PWD/ODD`"
set even="`cat $PWD/EVEN`"
set FILERETURN=""
set i=1
set j=1
set counttot=0
set countpass=0

while ($i <= $#odd)
while ($j <= $#even)

set FILERETURN=(`echo $odd[$i] | $dir`)

#echo $FILERETURN
#echo $even[$j]
if $FILERETURN == $even[$j]
@ countpass = $countpass + 1
endif
#echo $odd[$i]

#echo $even[$j]
@ counttot = $counttot + 1
@ i = $i + 1
@ j = $j + 1
end
end

echo "passed/total tests" $countpass "/" $counttot
rm ODD EVEN

Code:
% check.sh
-1: Command not found.
0: Command not found.
-1: Command not found.
-1: Command not found.
-1: Command not found.
-1: Command not found.
-1: Command not found.
0: Command not found.
passed/total tests 10 / 10

From: http://www.unix.com/homework-coursework-questions/221105-getting-command-not-found-when-comparing-tsch.html

Linux

Inserting text into a file with awk or sed

Leave a comment

By Schubi

Hello,

I’ve been trying to get a script working that fetches weather-data and converts it into an .ics file. The script works so far put I’m stuck at the point where I need to add specific static data. A thorough search through the forum did not point me into the right direction.

Code:
#!/bin/bash
##### Begin
DATE=$(date +'%Y%d%m')
TIME=$(date +'%H:%M:%S')
TEMPFILE="$(mktemp temp.XXX)"

if [ $? -ne 0 ]; then
echo "$0: Can't create temp file, exiting..."
exit 1
fi

##### Fetching weather data
curl http://ical.wunderground.com/auto/ical/global/stations/48698.ics?units=metric | awk 'NR==1 || NR==5 || NR==6 || NR==10 || NR==17 || NR==18 || NR==22 || NR==24 || NR==135' > $TEMPFILE

##### Adding specific calendar data

##### Cleaning up
cat $TEMPFILE | tr -d "r" > $DATE.ics
rm $TEMPFILE
exit 0

The output is as expected a .ics file that can be imported into a calendar:

Code:
BEGIN:VCALENDAR
VERSION:2.0
CALSCALE:GREGORIAN
BEGIN:VEVENT
DTSTAMP:20130412T000000Z
DTSTART;VALUE=DATE:20130412
SUMMARY:Thunderstorm 32C / 26C
END:VEVENT
END:VCALENDAR

In addition to the above output, the following data set needs to be imported between lines 3 and 4:

Code:
BEGIN:VTIMEZONE
TZID:Asia/Singapore
BEGIN:DAYLIGHT
TZOFFSETFROM:+0700
DTSTART:19330101T000000
TZNAME:GMT+08:00
TZOFFSETTO:+0720
RDATE:19330101T000000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0730
DTSTART:19820101T000000
TZNAME:GMT+08:00
TZOFFSETTO:+0800
RDATE:19820101T000000
END:STANDARD
END:VTIMEZONE

I’ve been playing around with the following string but did not manage to produce a result (example):
awk ‘NR==4{$0=”BEGIN:VTIMEZONE”}1′ $TEMPFILE

Any help is appreciated!

From: http://www.unix.com/unix-dummies-questions-answers/221013-inserting-text-into-file-awk-sed.html

Business, Finance, Occupation & Industry

AECOM to announce fiscal year 2013 second-quarter results on May 7

Leave a comment

By Business Wirevia The Motley Fool

Filed under:

AECOM to announce fiscal year 2013 second-quarter results on May 7

Company to hold conference call and webcast

LOS ANGELES–(BUSINESS WIRE)– AECOM Technology Corporation (NYS: ACM) , a leading provider of professional technical and management support services for public and private clients in more than 140 countries around the world, will hold a conference call with analysts and investors on May 7 to discuss second-quarter 2013 financial results.

The company will conduct a live, audio-only webcast of its conference call at 11 a.m. Eastern Daylight Time, during which management will make a brief presentation focusing on the company’s results, strategies and operating trends.

The live webcast and a replay will be available online at http://investors.aecom.com. The site also will host the press release announcing financial results and slides containing additional financial and operating information for the presentation on the day of the call.

About AECOM

AECOM is a global provider of professional technical and management support services to a broad range of markets, including transportation, facilities, environmental, energy, water and government. With approximately 45,000 employees around the world, AECOM is a leader in all of the key markets that it serves. AECOM provides a blend of global reach, local knowledge, innovation and technical excellence in delivering solutions that create, enhance and sustain the world’s built, natural, and social environments. A Fortune 500 company, AECOM serves clients in more than 140 countries and had revenue of $8.2 billion during the 12 months ended Dec. 31, 2012. More information on AECOM and its services can be found at www.aecom.com.

NR 13-0401

AECOM Technology Corporation
Media:
Paul Gennaro
SVP & Chief Communications Officer
212-973-3167
Paul.Gennaro@aecom.com
or
Investor:
Lynn Antipas Tyson
SVP, Investor Relations
646-432-8428
Lynn.Tyson@aecom.com

KEYWORDS:   United States  North America  California

INDUSTRY KEYWORDS:

The article AECOM to announce fiscal year 2013 second-quarter results on May 7 originally appeared on Fool.com.

Try any of our Foolish newsletter services free for 30 days. We Fools may not all hold the same opinions, but we all believe that considering a diverse range of insights makes us better investors. The Motley Fool has a disclosure

From: http://www.dailyfinance.com/2013/04/11/aecom-to-announce-fiscal-year-2013-second-quarter-/

Linux

How to pass bash variable inside EOF?

Leave a comment

By Akshay Hegde

Hi all I am trying to generate some program using following command

Code:
n=1
for file in `ls *.txt | sort -t"-" -k2n,2`;do

nc=`awk 'END{print NR}' $file`

cat >program_$n.xyz<<EOF

let linecount = $nc

some more commands

EOF

n=`expr $n+1`
done

here my problem is

Code:
$nc

inside

Code:
EOF

is not working, those who have solution kindly help me.

Source: FULL ARTICLE at The UNIX and Linux Forums

Business, Finance, Occupation & Industry

Liberty Star's Hay Mountain Project Update: ZTEM

Leave a comment

By Business Wirevia The Motley Fool

Filed under:

Liberty Star’s Hay Mountain Project Update: ZTEM

TUCSON, Ariz.–(BUSINESS WIRE)– Liberty Star Uranium & Metals Corp. (“Liberty Star” or the “Company”) (OTCQB: LBSR) is pleased to announce geophysical contractor Geotech Ltd.(“Geotech”) has proposed a money saving program of fixed wing aircraft (vs helicopter) borne ZTEM geophysical surveys over the Hay Mountain target and the Tombstone Super Project in Cochise County, Arizona.

Geotech geophysicists are currently adjusting flight lines and preparing a new bid for the ZTEM survey which the Company hopes to fly as soon as possible, possibly by the end of the month.

Other Hay Mountain news:

  1. The 2nd archeological study (NR 149) has been submitted to the Arizona State Land Department and along with the previously submitted archaeological study (Feb 21, 2013) awaits comment and award of permit from State officials. No significant artifacts were located in either area.
  2. Geochemical field work will proceed on the State MEP portions Hay Mountain claims as soon as those permits have been received from the State. When these rock chip, soil, and vegetation samples are assayed all 17 rare earth elements (REEs) will be included in the analysis array, instead of only four REEs previously included in the first assay package of the area (NR 127).
  3. Analytic maps of the area are, and have been in preparation. A few of these will be made available through the Liberty Star website (www.libertystaruranium.com) in due course.

Comments CEO/Chief Geologist Jim Briscoe: “While we would like to move even faster on our Hay Mountain project it is clear we are progressing toward drilling. ZTEM electromagnetic (EM) studies are essential to a modern, efficient drilling program in the porphyry copper-caldera setting and I believe the surveys will identify strong porphyry copper and associated metals targets. Should we obtain adequate funding, the process can continue quite soon.”

About Hay Mountain, Southeast Arizona

The Hay Mountain block of the Tombstone Super Project (Cochise County, Arizona) consists of 57 unpatented federal lode mining claims in addition to 15 Arizona State Mineral Exploration Permits (MEPs). MEPs are the first step in completing an Arizona State Land Mining Lease. The Company has completed the required assessment work to hold the MEPs. This land position …read more

Source: FULL ARTICLE at DailyFinance

Linux

Display Additional Variable string in awk print command

Leave a comment

By ckwan

Hi all,
I have script to monitor and sum up the total memory use up for each individual process.

proc=$1
svmon -P -O summary=basic,unit=MB|awk ‘NR>4’|grep -w “${proc}” |awk ‘{sum+=$3} END {printf “t” sum “””n”;}’

But I would like the script to be able to display as following

Execute
——
./script java

Output
—–
java -> 1233

Can anyone has bright idea on this ?

Thanks.

…read more

Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

How to preserve gawk in ssh?

Leave a comment

By newbie2010

I am trying to extract some information out of a policy listing command. I have the following:

Code:
for server in test1 test2; do ssh $server '/usr/listinfo policy -L |gawk 'NR == 2'';done

This runs without the awk and sshs to remote server, but with the gawk then I get gawk command not found. How do I make sure the gawk is picked up? Is there a way to avoid the pipe?

…read more
Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

System function in awk

Leave a comment

By EAGL€

Hello Friends,

I have written a script like below,

I aimed to move some CDR files (call data record) whose the last field is “1” (NF=1 ) from a spesific directory to a new directory
Field Seperator is pipe.
If the directory does not exitst i should create it.
I will give the script two arguments so i check if they are given or not

Here I have written this; Im suspicious if the below system function in awk will work or not, appreciate your opinions.

Code:
if [ "$#" -ne 2 ]
then
echo
echo "--- There are two arguments which is expected after script! ---"
echo "--- USAGE: "$0" | FULL PATH of NEPS DIRECTORY | NEW FULL PATH for INITIALLY CHARGED CDRs ---"
echo
elif [ "$#" -eq 2 ]
then
if [ ! -d $2 ];
then
mkdir $2
fi

for file in `find $1 -type f -name "*cdr*.txt" 2>/dev/null`
do
nawk -F| -v PATH=$2 'NR==1{ if $NF==1 system( "mv FILENAME PATH/FILENAME ") }'
done;
fi

Thanks in advance
BR

…read more
Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

Finding maximum occurrence value using awk

Leave a comment

By meet77

Hi everyone, I’m a new member at the forum
I have a file like this: field 2 values are either 0 or negative. file test4:

100815 -20
118125 0
143616 0
154488 0
154488 0
154488 -6
196492 -5
196492 -9
196492 -7
27332 0
29397 0

I would like to print a line containing the maximum value in field 2 of all occurrences for each value in field 1. So the desired output should be:

100815 -20
118125 0
143616 0
154488 0
196492 -5
27332 0
29397 0

I am using awk to do this. But, there are two problems: first awk prints nothing when I try to get the maximum for the values with negative numbers but it has no problem when I try the same with positive numbers. This made me take the absolute values which I can later turn back to the original values. The second problem, I get the following output with my code:

100815 20
118125 0
143616 0
154488 6
196492 7
27332 0
29397 0

Although with absolute values I should get:

100815 20
118125 0
143616 0
154488 0
196492 5
27332 0
29397 0

My code is:

awk ‘{$2>0?$2=$2:$2=-$2} $2==0 {$2=0} {print}’ test4 | awk ‘NR==1 {a[$1]=$2} {a[$1]=$2 ; if ($2<a[$1]) a[$1]=$2; else a[$1]=a[$1];} END {for(i in a) print i"t"a[i];}'

I am sure I’m missing something basic and this could probably be done in a much simpler way. Any help is appreciated
Best reagrds to all

…read more
Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

Help using awk with a text file

Leave a comment

By ziggy6

Basically I have a text file that is made up of customer statements. I want to be able to separate the customer statements into their own file. The file is consistant in that the account # (part of the key) is always on line 10 then on every 55th line from there on out so my logic would be like: Read file put 10th line for charcters 1-5 in variable (account #), compare that to line 65 and if the same, read the next key (65 + 55), if they are different then take 10 lines less and write output using key plus date so key would be 90313.txt for Act 9 on March 2013 statement. I know how to use awk to get it to show x line like awk NR==10 example.txt but I need to know the script to keep this ongoing and to output each statement to its own file.

…read more
Source: FULL ARTICLE at The UNIX and Linux Forums

Business, Finance, Occupation & Industry

Copper, Gold, REE Explorer Liberty Star Operations Update, Southeast Arizona

Leave a comment

By Business Wirevia The Motley Fool

Filed under:

Copper, Gold, REE Explorer Liberty Star Operations Update, Southeast Arizona

TUCSON, Ariz.–(BUSINESS WIRE)– Liberty Star Uranium & Metals Corp. (“Liberty Star” or the “Company”)(LBSR: OTCQB) is pleased to announce a tentative schedule beginning in mid- to late April for a ZTEM airborne electromagnetic survey over the Company’s Hay Mountain claims in Cochise County, Arizona. Liberty Star has set up a separate operating account for the Hay Mountain portion of the Tombstone Super Project and has deposited $50 thousand in that account. Additional funds will be directed into this account to pay for ZTEM, geochem and diamond drilling. Geotech believes that electromagnetic conditions will be good for the airborne survey as early as mid- to late April. The helicopter borne flight will take about a week, assuming good weather. Modeling and analysis of the data will take approximately 2 weeks.

Other highlights for the 1st quarter 2013:

  1. Operational overhead has been substantially reduced. The Company no longer maintains offices separate from the Kolb Rd. warehouse, and the small headquarters office at Sutler Lane, central Tucson. A new accounting firm has been engaged based on its competitive pricing and specialization in the small cap and mining industries. These and other measures will save approximately $150,000 per year in operating overhead.
  2. The second archeological study announced in NR 148 (Feb 22 “Liberty Star Completes Archeological Study on Hay Mountain MEPs“) has been completed and has been sent to the Arizona State Land Department in order that we can obtain permits for geochem sample collection and future drill siting if warranted in our northern most MEPs (Mineral Exploration Permits). No archaeological remains were identified in this survey.

Comments CEO and Chief Geologist Jim Briscoe: “By cutting costs on an already lean operation and judiciously drawing down on the Deer Valley funding we are able to move ahead at Hay Mountain. ZTEM will allow us to manage our targets efficiently and at a substantially lower cost than without it. I was an early adopter of ZTEM technology, which is now almost a standard procedure for porphyry copper targets throughout the world.”

About Hay Mountain, Southeast Arizona

The Hay Mountain block of the Tombstone Super Project (Cochise County, Arizona) consists of 57 unpatented federal lode mining claims in addition to 15 Arizona State Mineral Exploration Permits (MEPs). MEPs …read more
Source: FULL ARTICLE at DailyFinance

Linux

Filter (by max length) only lines not matching regex

Leave a comment

By pathunkathunk

I have a large file of many pairs of sequences and their headers, which always begin with ‘>’

Quote:

>gi|abc| Species one
ATCAGGTATAATT
>gi|bcd| Species two
ATTTGATC
>gi|cdf| Species three
ATTTGATCT

I’m looking for help on how to retain only sequences (and their headers) below a certain length. So if min length was 10, output would be

Quote:

>gi|bcd| Species two
ATTTGATC
>gi|cdf| Species three
ATTTGATCT

I can filter by length, but I’m not sure how to exclude the header lines.

Code:
awk '{lines[NR] = $0} length($0) < 10 {print lines [NR-1]; print lines [NR]} ' file.name

…read more
Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

How to take awk result out (piping to other program)?

Leave a comment

By Akshay Hegde

Hi! all here is my code
which is working fine no errors but I want to know how to take result and input to other program

Code:
awk 'FNR==1{i++}{LC[i]=NR}
{for(k=1; k<=NF; k++) A[i,FNR,k]=$k}
END{for (i=1;i<=LC[1];i++)
{
for(j=1;j<=LC[2];j++)
if(A[2,j,8]=='$UID' && A[2,j,4]>='$MX'+A[1,i,3] && A[2,j,4]<='$MIN'+A[1,i,4] &&
A[2,j,4]>='$MN'+A[1,i,1] && A[2,j,5]<='$MX'+A[1,i,2])
print A[2,j,6],A[2,j,7]
}}' OFS="t" STD_FIILE FILE2 | program(print) command -XA[2,j,5] -Y[2,j,5] -------------> if statement is true

if if statement is awk program in false

then

Code:
OFS="t" STD_FIILE FILE2 | program(mask) command -XA[2,j,5] -Y[2,j,5] ------------->

please help me

…read more
Source: FULL ARTICLE at The UNIX and Linux Forums

Linux

Help with executing awk and While loop

Leave a comment

By machomaddy

Hi All,
I have a file say, sample.txt

Code:
Source Name: xxx
Department|Revenue
1001|3252
1002|3345

I am using the above file in one of my script. I need to read from Line 3 of the above the file and do some process.

My script has a code:

Code:
awk 'NR > 2' sample.txt | while read Dep; do awk -F '|' '{print $1}' ; done

The Output is:

Code:
1002 #2nd Department Name in Line 4

Expected is:

Code:
1001 #1st Department Name in Line 3
1002 #2nd Department Name in Line 4

The above command ignores line three and starts from line 5 even after giving as awk ‘NR > 2′. Not sure why this happens.

1 more things: When I changed the above code awk ‘NR > 2′ to awk ‘NR >= 2′ I get my expected output.

Just want to understand why awk skips Line 3 even if it mentioned as ‘NR > 2′ 😮

…read more
Source: FULL ARTICLE at The UNIX and Linux Forums