Thursday, October 30, 2014

Delete 10 newest files in Linux

Delete 10 newest files in Linux

The code you'd want to include in your script is
 rm -f $(ls -1t /path/to/your/logs/ | tail -n +11)
The -1 (numeric one) option prints each file on a single line, to be safe. The -f option to rm tells it to ignore non-existent files for when ls returns nothing.
Or use the below command
ls -d -1tr /path/to/folder/* | head -n -10 | xargs -d '\n' rm -f

Delete latest 3 files:
rm -f $(ls -1t ./ | head -3)   

Find a folder
Just find directories and skip file names pass the -type d option as follows:
find  / -type d -name "apt" -ls

locate httpdocs


Gives the time stamp

date +%Y%m%d -s "20081128"
date +%T -s "11:18:00"

Stat file.txt

newest=$(find subdir -newer timestamp -printf "%A%p\n" |
                sort -n |
                tail -1 |
                cut -d: -f2- )
     touch -r "${newest:-timestamp}" timestamp
The command above works by generating a list of the timestamps and names of all the files which are newer than the timestamp. The sort, tail and cut commands simply pull out the name of the file with the largest timestamp value (that is, the latest file). The touch command is then used to update the timestamp,
The "${newest:-timestamp}" expression simply expands to the value of $newest if that variable is set, but to timestamp otherwise. This ensures that an argument is always given to the ‘-r’ option of the touch command.

SED COMMAND TO DELETE LINES IN FILE

Sed Command to Delete Lines: Sed command can be used to delete or remove specific lines which matches a given pattern or in a particular position in a file. Here we will see how to delete lines using sed command with various examples.

The following file contains a sample data which is used as input file in all the examples:

> cat file
linux
unix
fedora
debian
ubuntu


SED COMMAND TO DELETE LINES - BASED ON POSITION IN FILE

In the following examples, the sed command removes the lines in file that are in a particular position in a file.

1. 
Delete first line or header line

The d option in sed command is used to delete a line. The syntax for deleting a line is:

> sed 'Nd' file

Here N indicates Nth line in a file. In the following example, the sed command removes the first line in a file.

> sed '1d' file
unix
fedora
debian
ubuntu

2. 
Delete last line or footer line or trailer line

The following sed command is used to remove the footer line in a file. The $ indicates the last line of a file.

> sed '$d' file
linux
unix
fedora
debian

3. 
Delete particular line

This is similar to the first example. The below sed command removes the second line in a file.

> sed '2d' file
linux
fedora
debian
ubuntu

4. 
Delete range of lines

The sed command can be used to delete a range of lines. The syntax is shown below:

> sed 'm,nd' file

Here m and n are min and max line numbers. The sed command removes the lines from m to n in the file. The following sed command deletes the lines ranging from 2 to 4:

> sed '2,4d' file
linux
ubuntu

5. 
Delete lines other than the first line or header line

Use the negation (!) operator with d option in sed command. The following sed command removes all the lines except the header line.

> sed '1!d' file
linux

6. 
Delete lines other than last line or footer line

> sed '$!d' file
ubuntu

7. 
Delete lines other than the specified range

> sed '2,4!d' file
unix
fedora
debian

Here the sed command removes lines other than 2nd, 3rd and 4th.

8. 
Delete first and last line

You can specify the list of lines you want to remove in sed command with semicolon as a delimiter.

> sed '1d;$d' file
unix
fedora
debian

9. 
Delete empty lines or blank lines

> sed '/^$/d' file

The ^$ indicates sed command to delete empty lines. However, this sed do not remove the lines that contain spaces.


SED COMMAND TO DELETE LINES - BASED ON PATTERN MATCH

In the following examples, the sed command deletes the lines in file which match the given pattern.

10. 
Delete lines that begin with specified character

> sed '/^u/d' file
linux
fedora
debian

^ is to specify the starting of the line. Above sed command removes all the lines that start with character 'u'.

11. 
Delete lines that end with specified character

> sed '/x$/d' file
fedora
debian
ubuntu

$ is to indicate the end of the line. The above command deletes all the lines that end with character 'x'.

12. 
Delete lines which are in upper case or capital letters

> sed '/^[A-Z]*$/d' file

13. 
Delete lines that contain a pattern

> sed '/debian/d' file
linux
unix
fedora
ubuntu

14. 
Delete lines starting from a pattern till the last line

> sed '/fedora/,$d' file
linux
unix

Here the sed command removes the line that matches the pattern fedora and also deletes all the lines to the end of the file which appear next to this matching line.

15. 
Delete last line only if it contains the pattern

> sed '${/ubuntu/d;}' file
linux
unix
fedora
debian

Here $ indicates the last line. If you want to delete Nth line only if it contains a pattern, then in place of $ place the line number.

Tuesday, November 5, 2013

Command used in day to day activity

Command to find the used ports in server

netstat -ln | grep '80 ' | grep 'LISTEN'
netstat -atnp|grep LISTEN 

netstat -ln | grep '99 ' | grep 'LISTEN'
 netstat -ln | grep '16'

telnet <machine_IP> <port>

Command  for copying files to server

 scp -pr RMVServices-0.0.2-SNAPSHOT.war shib@100.1.127.11:/home/shib

Command for multiple search 

tail -f /tmp/Liferay-JBoss.out |grep 'PAGENAME\|tabName'

grep 'PAGENAME\|tabName'  for multiple search

Command to set line no in VI Editor

:set nu   for line no

:/ERROR   for ERROR search in VI


grep -r devapp214 ./|grep tst2

Command to shutdown jboss 

./shutdown.sh -s 172.23.2.5:7109

./shutdown.sh -s 172.23.2.25:1399

Command to search files

find ./ -name jboss-admin.sh

Command to see difference in  files

sdiff  ./pdf.css  /opt/dadm/liferay-portal-6.0-ee-sp2/jboss-.1.0/server/default/deploy/PPortlet.war/css/pdf.css

 Schedule a Background Cron Job For Every 10 Minutes.

Use the following, if you want to check the disk space every 10 minutes.
*/10 * * * * /home/ramesh/check-disk-space

Monday, January 28, 2013

Hiding a file or directory under Linux


Hiding a file or directory under Linux
To hide a file or directory in Linux, just give it a name preceded by a full stop (.).


For example:
. bash_history
. bash_profile
. ssh 

 
Hide UNIX file structure
 
1. Open a terminal and execut "ls -1 / | sudo tee /.hidden"

2. Do "sudo chmod a+r .hidden"

3. If you want all standard files hidden, skip to 5. Otherwise. Open the file with "sudo gedit /.hidden".

4. Remove every directory you don't want to hide from the file. Here's how mine looks like, as an example:

bin
boot
opt
usr
etc
sys
tmp
srv
proc
root
sbin
lib
lost+found
mnt
initrd
dev
debootstrap
var
cdrom
initrd.img
initrd.img.old
vmlinuz
vmlinuz.old
home
media


5. Now, let's create some entries that are easier for new users: select "home" and create a link to link to it (for example, by dragging it and holding <ctrl>+<shift>. Your mouse cursor should show two linked circles when dragging while holding them, if they do you're creating a link). Rename that link to "User Data" or something similar. Do the same with media; rename it to "Drives" or "Media" or what you want. (If the files are already hidden. show them with <ctrl>-<h>.)

6. If you want to have a discoverable way to some or all of the now hidden files, create a folder called "System Data". Create links to all the other files and directories there.

7. You're done! :) Update an open Nautilus with <ctrl>-<r>, or restart it with "killall nautilus".

I attached a screenshot of how it looks like.

Now, some anticipated FAQs:

Q: Won't this royally screw my system?
A: I see no reson why it should. Everything is still in it's original place, only no longer visible by default. It definitely works fine here and on several other Ubuntu installations.

Q: Are there any negative side effects?
A: Not that I know of. From the command line, everything stays the same. If you encounter any problem, please tell me!

Q: I want to access one of the hidden folders. How do I do that?
A: View -> Show Hidden Files. Or press <ctrl>-h

Q: I don't like this! How can I undo it?
A: Simple, just remove the .hidden file. Delete the links you created in step 5 and 6 if you want, or leave them there. Just make sure you don't accidentally delete the original folders. :)

Q: Can I have all the files shown automatically if I have a sudo'd nautilus?
A: Certainly. Make sure that the .hidden file is owned by root (should be automatically), then, in the permissions tab, disallow read access for the owner.

Q: Can I do that .hidden stuff in other directories too?
A: Yes. It's a pretty cool feature.

Q: Are there any limitations to this?
A: Yes, the file selector (ie open dialog) will still show all the files. The .hidden file currently only applies to Nautilus.

Wednesday, January 9, 2013

Touch & More


Touch

touch [-acm] [-f agefile] [-r agefile] [[-t] time] file ...

touch [-acm] time file ...
DESCRIPTION

The touch command changes certain dates for each file argument. By default, touch sets both the date of last file modification and the date of last file access to the current time. This is useful for maintaining correct release times for software and is particularly useful in conjunction with the MKS Make software development facility.
Options

-a    sets only the access time.
-c    does not create any files that do not already exist. Normally, touch creates such files.
-m    sets only the modification time.
If you do not specify -a or -m, touch behaves as though you specified both.

To tell touch to use a time other than the current, use one of the following options:

-f agefile    is an obsolete version of the POSIX-compliant -r option.
-r agefile    sets the access and modification times (as indicated by the other options) to     those kept for agefile.
-t time     specifies a particular time using this format:

[[[[cc]yy]MM]dd]hhmm[.ss]

where cc is the optional first 2 digits of the year, yy is the optional last 2 digits of the year, MM is the optional number of the month (01-12), dd is the optional day of the month, hh is the hour in 24 hour format (required), mm is the minutes (required), ss is the optional seconds.

EXAMPLES

touch newfile

sets the modification time of newfile to the present.

touch -t 8001031305 oldfile

sets the modification time of oldfile to 13:05 on January 3, 1980.

touch -r oldfile newfile

sets the modification time of newfile to that of oldfile.

More

More is a command used to read text files. For example, we could do this:

% more poems

The effect of this to let you read the file "poems ". It probably will not fit in one screen, so you need to know how to "turn pages". Here are the basic commands:

q --- quit more
spacebar --- read next page
return key --- read next line
b --- go back one page

For still more information, use the command man more.
To display the file named letter.txt on the screen, the user can type either of the following two commands:

more < letter.txt
type letter.txt | more

The command displays the first screen of information from letter.txt, and then the following prompt appears:

-- More --

When the spacebar is pressed, the next screen of information will be displayed. It is also possible to clear the screen and remove all extra blank lines before displaying the file:

more /c /s < letter.txt
type letter.txt | more /c /s


How to concatenate two files line by line 

I have two text files; each of them contains information by line such like that
file1.txt            file2.txt
----------           ---------
linef11              linef21
linef12              linef22
linef13              linef23

I would like to concatenate theses files lines by lines using a bash script in order to obtain:
fileresult.txt
--------------
linef11     linef21
linef12     linef22
linef13     linef23

1.      paste file1.txt file2.txt > fileresults.txt

2.      awk '
  # store the first file, indexed by col2
  NR==FNR {f1[$2] = $0; next}
  # output only if file1 contains file2's col2
  ($2 in f1) {print f1[$2], $0}
' file1 file2

3.      You can do that with a combination of the sort and join commands. The straightforward approach is
join -j2 <(sort -k2 file1) <(sort -k2 file2)
But that displays slightly differently than you're looking for. It just shows the common join field and then the remaining fields from each file

Beginners UNIX Interview Questions Answers




1. Write command to list all the links from a directory?
In this UNIX command interview questions interviewer is generally checking whether user knows basic use of "ls" "grep" and regular expression etc
You can write command like:
ls -lrt | grep "^l"

2. Create a read-only file in your home directory?
This is a simple UNIX command interview questions where you need to create a file and change its parameter to read-only by using chmod command you can also change your umask to create read only file.
touch file
chmod 400 file

3. How will you find which operating system your system is running on in UNIX?
By using command "uname -a" in UNIX

4. How will you run a process in background? How will you bring that into foreground and how will you kill that process?
For running a process in background use "&" in command line. For bringing it back in foreground use command "fg jobid" and for getting job id you use command "jobs", for killing that process find PID and use kill -9 PID command. This is indeed a good Unix Command interview questions because many of programmer not familiar with background process in UNIX.

5. How do you know if a remote host is alive or not?
You can check these by using either ping or telnet command in UNIX. This question is most asked in various Unix command Interview because its most basic networking test anybody wants to do it.

6. How do you see command line history in UNIX?
Very useful indeed, use history command along with grep command in unix to find any relevant command you have already executed. Purpose of this Unix Command Interview Questions is probably to check how familiar candidate is from available tools in UNIX operation system.

7. How do you copy file from one host to other?
Many options but you can say by using "scp" command. You can also use rsync command to answer this UNIX interview question or even sftp would be ok.

8. How do you find which process is taking how much CPU?
By using "top" command in UNIX, there could be multiple follow-up UNIX command interview questions based upon response of this because “TOP” command has various interactive options to sort result based upon various parameter.

9. How do you check how much space left in current drive?
By using "df" command in UNIX. For example "df -h ." will list how full your current drive is. This is part of anyone day to day activity so I think this Unix Interview question will be to check anyone who claims to working in UNIX but not really working on it.

10. What is the difference between Swapping and Paging?
Swapping:
Whole process is moved from the swap device to the main memory for execution. Process size must be less than or equal to the available main memory. It is easier to implementation and overhead to the system. Swapping systems does not handle the memory more flexibly as compared to the paging systems.
Paging:
Only the required memory pages are moved to main memory from the swap device for execution. Process size does not matter. Gives the concept of the virtual memory. It provides greater flexibility in mapping the virtual address space into the physical memory of the machine. Allows more number of processes to fit in the main memory simultaneously. Allows the greater process size than the available physical memory. Demand paging systems handle the memory more flexibly.

Intermediate UNIX Interview Questions Answers
1. What is difference between ps -ef and ps -auxwww?
This is indeed a good Unix Interview Command Question and I have faced this issue while ago where one culprit process was not visible by execute ps –ef command and we are wondering which process is holding the file.
ps -ef will omit process with very long command line while ps -auxwww will list those process as well.

2. How do you find how many cpu are in your system and there details?
By looking into file /etc/cpuinfo for example you can use below command:
cat /proc/cpuinfo

4. What is Zombie process in UNIX? How do you find Zombie process in UNIX?
When a program forks and the child finishes before the parent, the kernel still keeps some of its information about the child in case the parent might need it - for example, the parent may need to check the child's exit status. To be able to get this information, the parent calls 'wait()'; In the interval between the child terminating and the parent calling 'wait()', the child is said to be a 'zombie' (If you do 'ps', the child will have a 'Z' in its status field to indicate this.)
Zombie : The process is dead but have not been removed from the process table.

5. What is "chmod" command? What do you understand by this line “r-- -w- --x?

6. There is a file some where in your system which contains word "UnixCommandInterviewQuestions” How will find that file in Unix?

7. In a file word UNIX is appearing many times? How will you count number?
grep -c "Unix" filename

8. How do you set environment variable which will be accessible form sub shell?
By using export   for example export count=1 will be available on all sub shell.

9. How do you check if a particular process is listening on a particular port on remote host?
By using telnet command for example “telnet hostname port”, if it able to successfully connect then some process is listening on that port. To read more about telnet read networking command in UNIX

10. How do you find whether your system is 32 bit or 64 bit ?
Either by using "uname -a" command or by using "arch" command.

Advanced UNIX Interview Questions and Answers
1. How do you find which processes are using a particular file?
By using lsof command in UNIX. It wills list down PID of all the process which is using a particular file.

2. How do you find which remote hosts are connecting to your host on a particular port say 10123?
By using netstat command execute netstat -a | grep "port" and it will list the entire host which is connected to this host on port 10123.

3. What is ephemeral port in UNIX?
Ephemeral ports are port used by Operating system for client sockets. There is a specific range on which OS can open any port specified by ephemeral port range.

4. If one process is inserting data into your MySQL database? How will you check how many rows inserted into every second?
Purpose of this Unix Command Interview is asking about "watch" command in UNIX which is repeatedly execute command provided with specified delay.

5. There is a file Unix_Test.txt which contains words Unix, how will you replace all Unix to UNIX?
You can answer this Unix Command Interview question by using SED command in UNIX for example you can execute sed s/Unix/UNIX/g fileName.

6. You have a tab separated file which contains Name, Address and Phone Number, list down all Phone Number without there name and Addresses?
To answer this Unix Command Interview question you can either you AWK or CUT command here. CUT use tab as default separator so you can use
cut -f3 filename.

7. Your application home directory is full? How will you find which directory is taking how much space?
By using disk usage (DU) command in Unix for example du –sh . | grep G  will list down all the directory which has GIGS in Size.

8. How do you find for how many days your Server is up?
By using uptime command in UNIX

9. You have an IP address in your network how will you find hostname and vice versa?
This is a standard UNIX command interview question asked by everybody and I guess everybody knows its answer as well. By using nslookup command in UNIX



Wednesday, June 27, 2012

Basic shell scripting part 3


1. What is Shell Scripting ?
        Shell scripting, in Linux or Unix,  is programming with the shell using which you can automate your tasks. A shell is the command interpreter which is the interface between the User and the kernel. A shell script allows you to submit a set of commands to the kernel in a batch. In addition, the shell itself is very powerful with many properties on its own, be it for string manipulation or some basic programming stuff. 

2. The command "cat file" gives error message "--bash: cat: Command not found". Why?
    It is because the PATH variable is corrupt or not set appropriately. And hence the error because the cat command is not available in the directories present PATH variable.

3. How to find the length of a string  in Linux?

$ x="welcome"
$ echo ${#x}
7
4. What are the different timestamps associated with a file?
  • Modification time:- Refers to the time when the file is last modified.
  • Access time :- The time when the file is last accessed.
  • Changed time :- The time when the attributes of the file are last changed.
5. How to get the list of files alone in a directory in Linux?
$ ls -lrt | grep ^-
6. How to find the last modified file or the newest file in a directory?
$ ls -lrt | grep ^- | awk 'END{print $NF}'
7. How to access the 10th command line argument in a shell script in Linux?
     $1 for 1st argument, $2 for 2nd, etc... For 10th argument, ${10}, for 11th, ${11} and so on.

8. How to find the sum of all numbers in a file in Linux?
$ awk '{x+=$0}END{print x}' file
9. How to delete a file which has some hidden characters in the file name?
       Since the rm command may not be able to delete it, the easiest way to delete a file with some hidden characters in its name is to delete it with the find command using the inode number of the file.

$ ls -li
total 32
9962571 -rw-r--r-- 1 guru users 0 Apr 23 11:35
$ find . -inum 9962571 -exec rm '{}' \;
10. Using the grep command, how can you display or print the entire file contents?
$ grep '.*' file
11. What is the difference between a local variable and environment variable in Linux?
         A local variable is the one in which the scope of the variable is only in the shell in which it is defined. An environment variable has scope in all the shells invoked by the shell in which it is defined.

12. What does the 'execute' permission in a directory stand for?
       Without the execute permission on a directory, the user will not be able to traverse or in other words, do a "cd" to the directory.

13. How to find the total number of arguments in a shell script in Linux?
      The shell special variable, $# ,contains the total number of arguments passed to a shell script.

14.  How to remove the Control-M character from a file in Linux?

$ dos2unix file
15. In which file should a variable be set  in order to make the setting permanent?
      The variable should be set in the profile file to make the setting permanent. The appropriate profile depends on the default shell being set for the user.

16. What is a she-bang line in a shell script?
          She-bang line in a shell script is the first line, if present. It starts with '#!' and followed up with a full path of a shell. The shell specified indicates the shell in which this script will be run. The entry of she-bang is not mandatory, however, if present, should be the first line of the script. With a number of shells available and syntax being specific for a given shell, it is always good to specify the she-bang line in a shell script.

17. A file contains many lines, and each line containing multiple words. How to find out the unique words and the word count of each of the words?
$ cat file
apple orange
banana apple orange
papaya
$ awk '{for(i=1;i<=NF;i++)a[$i]++;}END{for(i in a){print i, a[i];}}' file
banana 1
apple 2
orange 2
papaya 1
18. What is an internal command in Linux?
        Internal commands are also called shell built-in commands. Example: cd,fg. Since these are shell built-in, no process is created while executing these commands, and hence are considered to be much faster.

19. x and y are two variables containing numbers? How to add these 2 numbers?

$ expr $x + $y
20.  How to add a header record to a file in Linux?

$ sed -i '1i HEADER' file
21. How to find the list of files modified in the last 30 mins in Linux?

$ find . -mmin -30
22. How to find the list of files modified in the last 20 days?

$ find . -mtime -20
23. How to find the files modified exactly before 30minutes?

$ find . -mmin 30

24. A string contains a absolute path of a file. How to extract the filename alone from the absolute path in Linux?

$ x="/home/guru/temp/f1.txt"
$ echo $x | sed 's^.*/^^'
25. How to find all the files created after a pre-defined date time, say after 10th April 10AM?
       This can be achieved in 2 steps:
       1. Create a dummy file with the time stamp, 10th April 10AM.
       2. Find all the files created after this dummy file.

$ touch -t 1004101000 file
$ find . -newer file
26. How to print the contents of a file line by line in Linux?

$ while read line
> do
>   echo $line
> done < file
27. The word "Unix" is present in many .txt files which is present across many files and also files present in sub directories. How to get the total count of the word "Unix" from all the .txt files?
$ find  . -name *.txt -exec grep -c Unix '{}' \; | awk '{x+=$0;}END{print x}'
28.  How to get tomorrow's date in Linux?
$ date -d "1 day"
29. How to join all lines in a file using comma?
$ paste -s -d, file
30. How to join all lines in a file without any delimiter?
$ paste -s --delimiter="" file
31. How to join every 2 lines in a file in Linux?
$ sed 'N;s/\n//' file
32. A shell script will ask for 3 inputs. The user will not be physically present to be able to give it manually. How can the script be run without having to manually give the input?
    Put those 3 input values in a file, and make the script to read this file as input. For example:
     Assume the 3 values to be : 3, 10 and 20:
$ cat file
3
10
20
  and assuming the script is hello.sh, run it like:
./hello.sh < file
33.  How to find the total number of a lines in a file in Linux?
$ wc -l file | awk '{print $1}'


34. How to print the first line or the header record in a file?
$ head -1 file

35. How to  replace all occurrences of "Unix" to "Linux" in a file?

$ sed 's/Unix/Linux/g' file
36. How to make the above changes permanent in the file?

$ sed -i 's/Unix/Linux/g' file
37. How to  replace only the first occurrence of "Unix" to "Linux" in a string in Linux?

$ sed 's/Unix/Linux/' file
38. How to  replace only the second occurrence of "Unix" to "Linux" in a string in Linux?

$ sed 's/Unix/Linux/2' file
    In fact, to replace nth occurrence of a string in a file, it is:

$ sed 's/Unix/Linux/n' file  #where n is the nth occurrence

39. How to add leading zeros to every line in a file in Linux?

$ sed 's/^/0000/' file
40. How to add trailing zeros to  every line in a file in Linux?

$ sed 's/$/00/' file

41. How to get yesterday's date in Linux?

$ date -d "1 day ago"
42. I have a file with SQL commands. How can I open a sqlplus session in Linux and run this SQL file?

$ sqlplus guru/unix11@XE @file.txt
    where file.txt is the ASCII file containing the sql instructions.

43. The ps command will disclose the sqlplus connect string if any sqlplus session is ON. How to prevent the sqlplus connect string from appearing in the ps command in Linux?
     While connecting to sqlplus, instead of connecting in the normal way, connect as below:

$ sqlplus /nolog
> connect guru/unix11@XE
44. How to rename a group of files from .txt to .exe in Linux?

for i in *.txt
do
  x=`basename $i .txt`
  mv $i $x.exe
done
45. After logging in to your account in Linux, you did "cd log".  There was no "log" directory under the current directory, still the "cd" command traversed to a log directory under a different location? How it happened?
     It is because the CDPATH variable is set.

46. How to zero pad a number in Linux?
      Say, to zero pad a number to 4 places:

$ x=20
$ printf "%04d\n" $x
46. How to find all the .c and .h files in Linux?

$ find . -name "*.[ch]"
47. How to find the list of all the .c files and display only the file name, instead of the default find output which includes the relative path?

$ find . -name *.c | sed 's^.*/^^' 
48. How to copy a file with the same time stamp as the source file in Linux?

$ cp --preserve=timestamp file1 file2
49. How to copy a file "file1" to "file2" by passing only one argument to cp command?

$ cp file{1,2}
50. What is the difference between the source command and dot(.) command in Linux?
       No difference. Both are used for sourcing a file in Linux. bash, csh and some more shells using source command, whereas ksh uses dot(.) command to source a file.