Use ‘dd’ to create identified size file:
# dd if=/dev/urandom of=targerfile bs=1M count=100
Here, the created file size will be the result of ‘bs’ x ‘count’, in other word:
size = bs x count
# vmstat -n
if the ‘vs’ and ‘sy’ values under CPU are high, and the idle time value under CPU is closed to zero. Your server’s CPU is heavily loaded, Try using a faster machine as your NFS server. Do not use a gateway or a terminal server as an NFS or NTS server.
I wrote the following scripts to copy and compare bigfiles:
#!/bin/bash
# test script
# /mnt/bigfile is 150G,and /NFS_test is mounted from remote host
# mount 192.168.123.78:/Test /NFS_test
# set counter
COUNTER=0
# loop
while :
do
# copy local file “a” to remote host “b”,and then cmp
echo “—————————-” >> /root/NFS_Rep/time
echo “cp a–>b from: `date`” >> /root/NFS_Rep/time
cp -v /mnt/bigfile /NFS_test/bigfile1 >> /root/NFS_Rep/cp_ab 2>&1
echo “cp a–>b end: `date`” >> /root/NFS_Rep/time
echo ” ” >> /root/NFS_Rep/time
# sleep time
sleep 60
#starrt cmp
echo “cmp a–>b from: `date`” >> /root/NFS_Rep/time
cmp -l /mnt/bigfile /NFS_test/bigfile1 >> /root/NFS_Rep/cmp_ab 2>&1
echo “cmp a–>b end: `date`”>> /root/NFS_Rep/time
echo ” ” >> /root/NFS_Rep/time
# sleep time
sleep 60
# copy local file “a” to remote host “c”,and then cmp
echo “cp a–>c from: `date`” >> /root/NFS_Rep/time
cp -v /mnt/bigfile /NFS_test/bigfile2 >> /root/NFS_Rep/cp_ac 2>&1
echo “cp a–>c end: `date`” >> /root/NFS_Rep/time
echo ” ” >> /root/NFS_Rep/time
#sleep time
sleep 60
#start cmp
echo “cmp a–>c from: `date`” >> /root/NFS_Rep/time
cmp -l /mnt/bigfile /NFS_test/bigfile2 >> /root/NFS_Rep/cmp_ac 2>&1
echo “cmp a–>c end: `date`” >> /root/NFS_Rep/time
echo ” ” >> /root/NFS_Rep/time
#sleep time
sleep 60
# compare “b” and “c”
echo “cmp b–>c from: `date`” >> /root/NFS_Rep/time
cmp -l /NFS_test/bigfile1 /NFS_test/bigfile2 >> /root/NFS_Rep/cmp_bc 2>&1
echo “cmp b–>c end: `date`” >> /root/NFS_Rep/time
echo “—————————-” >> /root/NFS_Rep/time
# delete “b” and “c”
rm -rf /NFS_test/bigfile1
rm -rf /NFS_test/bigfile2
# reset the counter
COUNTER=`expr $COUNTER + 1`
echo “The loop counter is $COUNTER” > /root/NFS_Rep/counter.rep
done
This script would create 7 report files in /root/NFS_Rep directory: cp_ab,cmp_ab,cp_ac.cmp_ac,cmp_bc,counter.rep,time.
Note,I’ve made some mistakes when wrote $ as &, and the cmp showed error when it had taken a very long time to complete copy bigfile(150G), how sadly I am! This issue gave me a good lesson, it was better to use small files to test the scripts beforn running it in order to checking the code and saving time. Late, I modified the script again with adding “date” marks.
Due to the network configuration of “192.168.123.78” changed automatically in the day before yesterday, I was very caring the remote host’s net state, so writing the following script:
# !/bin/bash
# test 192.168.123.78
while :
do
if ping -c 1 192.168.123.78 > /dev/null 2>&1
then
echo “At `date` : The status:192.168.123.78 is ok” > /root/net_report
sleep 300
else
echo “At `date` : Warning, the remote host maybe lost!!!” >> /root/net_report
wall “Warning, the remote host 192.168.123.78 maybe lost!!!”
fi
done
This script will monite the remote host’s network, and create report.
In the morning, I rebooted my computer, and found soft raid 0 made yesterday had been unavailable.Even I tried the steps like yesterday doing, but the RAID still could not be rebuild.
Later,More tell me using the following command to remount the RAID:
# mdadm -A /dev/md0 /dev/hdb /dev/hdc
and then check the /proc/mdstat, yes, not only the RAID was found but also, all the date on the RAID are still here!
RAID has two forms, striping and partity.In fact,RAID 0 was not really RAID because it isn’t redundant. RAID has good write and read performance. There are two important parameters: Strip, and chunk size. Strip means the whole array’s transfer rate per second, which is always used by hardware controllers. Chunk size,belong to Linux kernel, is the single disk’s rate.
RAID1 is mirroing, good read performance nearly twice of the single one, but not good at writing performance for writing data to two disks at time.RAID 2,RAID3 are useless, and RAID 4 just like RAID 5,which has one dedicated disk to save paritition information.
For we want to test big file transation in NFS, Spring gave me two IDE harddisk and ask me to install them in my computer which os is centos4.3. There were some trouble during the recognizing harddisk, only two IDE harddisks could be found, the other one disappeared even I changed the jumps. I feel a litter disappointed but not gave up. I put the “disappear” harddisk in the primary IDE connectors, it was recognize rightly! What’s wrong?
I looked at the mainboard carefully, then found that this board has four IDE connectors:Primary IDE,Slave IDE, Primary RAID IDE and Slave RAID IDE,I just put the third harddisk in the Slave RAID IDE slot! After changed the cable to slave IDE slot, everything is ok. The system found three harddisk as I wanted.
How to make soft RAID on linux?
First I downloaded RAID tools(raidtools), and edited the /etc/raidtab file as the following:
[root@localhost ~]# vi /etc/raidtab
raiddev /dev/md0
raid-level 0
nr-raid-disks 2
chunk-size 32
persistent-superblock 1
device /dev/hdb
raid-disk 0
device /dev/hdc
raid-disk 1
and then issued the “mdraid /dev/md0”, note, “/dev/md0” had been created when installed the “raidtools” package.It failed with some error messages which said /dev/hdb had been part of RAID? Why? My second IDE harddisk was just taken in the computer! Later, More explained that perhaps the system created /dev/hdb automaticlly when I installed the Centos 4.3. I’m still confused with what he said.
Other way to make soft RAID was the “mdadm” program, downloaded from Internet,and installed it.Then I used the following steps to create soft RAID 0 successfully:
# cat /proc/partitions Checking HD partitions
# cat /proc/mdstat Checking RAID stat
# ls /dev/mapper
# dmsetup lower level logical volume management
# dmsetup table
# dmsetup remove pdc_jbaiadif remove /dev/hdb
# mdadm -C /dev/md0 –lever 0 -n /dev/hdb /dev/hdc (After issuing this step, there was still error saying /dev/hdb had been used as RAID, so I planed to stop RAID first)
# mdadm /dev/md0 –stop
Then,create RAID again:
# mdadm -C /dev/md0 –lever 0 -n /dev/hdb /dev/hdc
OK, the RAID 0 was created successfully.
How to create very large file by “dd” command?
Since I made RAID 0 to merge two harddisk’s capacies together(150G), I wanted to create a big file as large as 150G.
First, format the RAID partition and mount it to /mnt:
# mkfs /dev/md0
# mount /dev/md0 /mnt
Second, using “dd” command to create file:
# dd if=/dev/urandom of=targetfile_name
A large file would be created when full of the /mnt capacity.
I’ve resighed from Powerleader at Tuesday, July 27th 2006 and will start my new seaon on Linux development. In my PL days when put all my heart on Brandon,Sapello,and Star Lake, I am very appreciated to all my friends, both including Powerleader and Intel TMEs. I believe that I could not forget all your kindness and great support.
Former Powerleader friends: James, Chris, Yan, Gary(the best striker who kept the highest goal recorder of PL history, key player), Dave(Du Du)
Current Powerleader friends: Alan, Wang Yu, Chris(Xie Wenyu), Juliet, Wu Kang, Tiger, Zhu Lin,Zhang Kai, Yin Zhenxian, Zhang Yeping, Huang Xiaoyong, Luis (Xu Zhencheng),Qiao XiangYun(a very kind brother), Chen Junlian, Wang Ling, Nancy(Liang Xiaolei), Gao Hongfei, Ouyang Meiwang, Zephuros(Feng Qi)
Intel TMEs: Lily(Luo Xiaojing), Sun Peng, Yue Pengfei, Zhang Jianguo
Intel Support team: Cai Qing, Jiang Lei, Shen Yong, Zhao Qing
especial thanks to my best friends: Cheng Feng and Zhao Yu, you brothers always encourage me and help me to find myself when I was lost. It’s very lucky I have you two good brothers in Shenzhen.
Wish all of you well.
Now,it’s a new begining, new season starts. Go forward with no hesitance. I plan to use two years devoting on Linux. Forza!!!
stty,change and print terminal line settings.
stty -echo
It always use “-echo” parameter when input some characters could not seen by others, for example, password. We should echo the text for security reasons.
If the “stty -echo” command is executed, every typed characters were hidden, but the causing result displays as normal. How to dismiss this echo setting? Use a variable to keep the stty setting before echo typed character action.
# STTYSAVED=`stty -g`
-g:print all current settings in a stty-readable form
later, we could back to the former configuration for stty, # stty $STTYSAVED
every setting was recovered. We could see the typed characters again.
exit is very useful to debug the program, its syntax is : # exit n, Here n is a number between 0-255, if it is bigger than 255, for example, 256, the value retuned by issuing”echo $? ” will be evaluted in the following equation: n-255-1
That says, when n is 266, the returned result is 256-255-1 = 0 which is also the value of “$?”
eg:
# !/bin/bash
# ifexit
echo “Please input a number: ”
# input the testing number
read NUMB
# if statement
if [ “$NUMB” -eq “1” ]
then
echo “$NUMB = 1”
elif [ “$NUMB” -eq “2” ]
then
exit 222
else
exit 278
fi
Input | Output
—————————–
2 222
3 278-256=22
test: is used for testing string, file and number.
expr: command testing and output the result
test
1.file testing: # test condition
or # [ condition ] (the are two space characters in each side of “condition” strings)
file testing have the following condition expressions:
-d: directory, to check if the directory exists or not
-f: general file
-L:this “L” is upcase,means link.
-r: readable
-w: is able to write
-x: is able to execute
-s: check the files length is bigger than 0, and NOT Null
-u: if there is suid been set.
logical expression:
-a: AND
-o: OR
!: NOT
for example: # [ -w test1.txt -a -w test2.txt ]
Pay attention here, # [ -w test1.txt ] -a [ -w test2.txt ] is wrong because only one pair [] could be used.
2.string testing
syntax: # test “string”
# test string_operator “string”
# test “string” string operator “string”
# [ string_operator string ]
# [ “string” string_operator “string”]
these string operators would be: ” = ” ,” != ” , “-z”, “-n”
-z: NULL
-n: NOT NULL
for example, to see if the EDITOR was NULL or not:
# [-z $EDITOR]
# echo $?
Is the EDITOR “vi”?
#[ $EDITOR = “vi” ]
# echo $?
# [ $TEST1 = $TEST2 ]
# echo $?
# [ $TEST1 != $TEST2 ]
# echo $?
3.numeric testing
syntax: # test “number” numeric_operator “number”
or # [ “number” numeric_operator “number” ]
The following operator are used here:
-eq: ==
-ne: !=
-gt:>
-lt:<
-ge:>=
Expr: evaluation expressions
syntax: expr argument operator argument
Note,there must be two space characters in each side of “operator”, and when using “*”, the back slash should be added as “*”