Paste vs Cat

There are two commands that are handy in combining files in Unix. If you want to combine files vertically (think cake layers) - use cat and to combine horizontally (think columns) - use paste.

Use awk if you want to switch the order of the column after the files have been combined.

cat file1
a1 b1 c1
a2 b2 c2
 

cat file2
d1 e1
d2 e2 

cat file1 file2
a1 b1 c1
a2 b2 c2
d1 e1
d2 e2 

paste file1 file2 
a1 b1 c1 d1 e1
a2 b2 c2 d2 e2 

paste file1 file2 > file3
awk -F' ' '{print $5,$2,$3,$4,$1}' file3
e1 b1 c1 d1 a1
e2 b2 c2 d2 a2 

If you liked this or found it useful, leave me a comment. Thanks!

Use searched pattern in substitution

Sometimes we want to search for a pattern and use that pattern in replace as well. Like the substitution is an extension of what we already have in the file.

E.g. append all employee ages with 'years'

emp_name:Jane
emp_age:25
emp_name:John
emp_age:22
emp_name:Mary
emp_age:30
emp_name:Natalie
emp_age:29
other data here

Command

:%s/\(emp_age.*\)/\1 years/g

Breaking it up -

emp_name:Jane

%s - apply thru out the file
\( and \) - escape the parentheses 
(emp_age.*) - consider it a group
\1 - escape 1; 1 means the first pattern, in our case it's the only one
\1 years -  use the first pattern and append years to it
g - replace multiple occurrences on the same line

Output

emp_name:Jane
emp_age:25 years
emp_name:John
emp_age:22 years
emp_name:Mary
emp_age:30 years
emp_name:Natalie
emp_age:29 years
other data here

While I was typing this up, I thought let me try using a second pattern as well.

:%s/\(emp_age.*\)\(years\)/\1- \2/g 

If we run this in the above output we get

emp_name:Jane
emp_age:25 - years
emp_name:John
emp_age:22 - years
emp_name:Mary
emp_age:30 - years
emp_name:Natalie
emp_age:29 - years
other data here

Hope this helps.

NVL equivalent in Unix

Sometimes, we want to assign an alternate value to a variable if it were null. I recently found that it has a one liner in Unix.

var1=${new_var1:-$default_var1}

 Which means if $new_var1 is not null, use it else use the $default_var1.

Add newline in vim

I am often analyzing SQL queries from Informatica session log and they are wrapped around and difficult to read. That's when I needed to look for a way to insert a newline after every comma.
Great documentation and examples here.

http://vim.wikia.com/wiki/Add_a_newline_after_given_patterns

select wall_of_text,block_of_text,cube_of_text from textbook;

Add a newline after comma: s/,/\r&/g

select wall_of_text
,block_of_text
,cube_of_text from textbook

 Though brackets let you specify multiple values that you can replace (the vim wiki has an example), I was not able to figure out how to get a newline after every comma and "from". Drop in a comment if you know how to.

What is /dev/null?

I've used /dev/null for two purposes -
1. to discard output of a command
2. to suppress any potential error messages from a command

You may already know that you can redirect the output to a
1. file using greater than symbol ">" and 
2. command using pipe symbol "|"

Redirection to /dev/null works just as if the redirection was to a file. The only exception that the output cannot be retrieved.

Before we dive in, it's important to know that standard output is represented as 1 and standard error as 2.

ls -1 c* 2> /dev/null  #redirects the standard error (2)
ls -1 c* &> /dev/null  #redirects both the stderr (2) and stdout(1)
                              #variations include 2>&1, >>
ls -1 c* 1> /dev/null  #redirects the std output (1)
                          #variation is >

Here's an example to show an application of /dev/null. Run the code below to see how differently they behave.

if test ! -z "`ls -1 y* 2> /dev/null`"  ; then
   echo "Files starting with y exist."
else
   echo "Files starting with y DO NOT exist."
fi

if test ! -z "`ls -1 x* `"  ; then
   echo "Files starting with x exist."
else
   echo "Files starting with x DO NOT exist."
fi

To think about -
1. What happens when you do > filename?
2. What happens with you do > instead of 1> in the first example?
3. What does  this do? ls -1 c* 2> logfile > file.lst

How to count words in a delimited file?

I was asked how would one do word count in a delimited file and it took me by a surprise since my goto Unix command "wc" works only with tab or space delimited text.

From wc manual page - "A word is defined as a string of characters delimited by white space characters."

This is where sed is comes to rescue.

ruch:coding ruchi$ cat poem.csv 
twinkle,twinkle,little
star,how,I, wonder
what,you, are.
ruch:coding ruchi$ cat poem.csv | wc -w
       5
ruch:coding ruchi$ sed 's/,/ /g' poem.csv | wc -w
      10
ruch:coding ruchi$ 

Can you guess why wc returns 5 instead of 3 from poem.csv?

Substring : Right to Left

Informatica does not have this documented but you could use SUBSTR function to go from Right to Left as well. Just use a negative start.

IIF(SUBSTR(FILENM,-4)='.PDF', 'PORTABLE')