Study Notes

In the last two weeks, I fixed two P1 bugs. One is saving setting when update ftp share, the other is displaying patch install process. Almostly all my energy were put into programming. It’s so long that I did not touch program since 2002 writing ASP in Wuhan. I read many Perl manuals to learn the perl syntax and try to apply it in a short time, though in fact, these two fixing took me nearly two weeks. It was low working-rate. I’m still pleased with my working, a famous German writer said “It maybe not the best, but is good as being finished.”

I learn more from two cases not only Perl experience, but also, the most important thing, I am really programmer. The programing seems to be annoyed and charming job, some lines of code could work as what you want. Still do I have litter difficult in improve the rate, and I believe I could be more familar with more practises.

More patience, will get more. Read the whole program first, do more thinking, then start modify. Bruce said, the preparing work is the key during fixing problems. Sometimes, perhaps I’m too anxious to have enough preparing thinking, it wasted my external spirit to try and try. It’s not benifit for me. I’ll pay more attention on the ready process.

nscd and RAID6 will be the next points researched.

I’ve been in Plasmon for four months. Before Dec,25th, we will fly to Sanya for our holiday, and I brought a DV last week. In the December of last year, I did not go to Hainan even emergency AF2 issue happened in Hainan.

Spin out/in SATA module from kernel when physically remove the drivers, especial Hot-Swap case. I did a whole testing on SNAZ-S4-400 platform(RAID0, 1,5), plug the disk out when there’s (or not) I/O, when the “miss” hard disk inserted back, the RAID were all recoveried. Bruce said, Irish had produced many disk failure issues when hotswap. I would follow up this testing and plan to use larger I/O pression.

“smbpasswd” add user to LDAP
1.In the “/etc/samba/smb.conf”, modify the second line as the following:
Passdb backend=ldapsam: ldap://192.168.123.33:389
This means the new users will be added here.
2.In localhost, # adduser phillip
# smbpasswd -a phillip
3. Then issue “ldapsearch -x”, the new user “phillip” is in the returned entries.

IE 7.0 running as IE6.0:
Microsoft supplied UASU tool for opening an IE7.0 instruce that emluates IE6 by sending the IE6 use agent string. I downloaded UASU.msc and installed, it works well on IE7.0 platform.

Perl, for about two months, I restarted to learn it now, and I must pay more attention on enhancing my program ability, such as read, write and debug. Bruce gave me a good advice, learn this program when using it, read the manul when needed. I think it is right and will be on this learning road. My schedule is spending one week, to master this program as almost our general programs are all written in Perl.

Before do anything, think about it careful, try to make it more clearly. Then, just do it.

Check the RAID setting by running the following command:
# cat /proc/mdstat
Personalites: [md5]
md0: active raid5 sdd1[2] sde1[3] sdc1[1] sdb1[0]
75469842 blocks level 5, 128k chunk, algorithm[3/3] [UUU]
Here, the first “3” in “[3/3]” indicates the how many devices in the array; the second “3” means active device numbers. If one device in array has been destoried, the second number would decrease “1”, for example, one disk failed, the second number is “2” as the result of “3-1”.
“UUU” marks the currently using device status. Assume /dev/sdb1 failed, it should be [_UU], at this point, the array runs in degrade mode without redundant.
“sdd1[2]”, assume “n” is the number of devices in array: if the number in “[]” is less than “n”, it means the device is active,else if the device is for backup. When one device down, the coresponding “[]” will be marked as “[F]”.

Mark the /dev/sdb1 as failed:
# mdadm /dev/md0 -f /dev/sdb1
Note, in this case, “/dev/sdb1” was component of “/dev/md0”.

Remove failed device:
# mdadm /dev/md0 -r /dev/sdb1

Add device to array:
# mdadm /dev/md0 -a /dev/sdb1

Start all arrays in “mdadm.conf”
# mdadm -As

Start idenfied array:
# mdadm -As /dev/md0

Stop array:
# mdadm -S /dev/md0

List detail information:
# mdadm -D /dev/md0

Chkconfig: maintain /etc/rc.d directory structure, add, remove change serivce, list time stamp information. Check the state of particular service.
# chkconfig –list sendmail
# chkconfig –level 4 sendmail off

The “ntsysv” command takes the functionality of chkconfig and wraps it into an easy-to-use screen interface.

kdbconfig: set the type of keyboard, refer to “/etc/sysconfig/keyboard”
timeconfig: set time zone, refer to “/etc/sysconfig/clock”
mouseconfi: set mouse, refer to “/etc/sysconfig/mouse”
sndconfig: screen-oriented interface to make install sound card.

NT boot, NT’s loader installs to the master boot record(MBR) and relies on three major files: ntdetect.com, ntldr, boot.ini.

Once you have a working LILO boot disk, you can simply copy the boot sector from the disk instead of the harddriver,eg:
# dd if=/dev/fd0 of=/bootsect1n1 bs=512 count=1
I think the first 512 byte in /dev/fd0 may record the boot information.

RAID: Only three are implemented in Linux, linear,0,1,5
You can assign additional disks from failover, called spare disks within the RAID set, when one disk fails it is marked as bad and the data is reconstructed on the first space disk, which is used as replacement, resulting in litter or no downtime. When using fdisk to format RAID component disks, the Partition ID must be set as “FD”.
# mk2fs -b 4096 -R stride=8 /dev/md0

To check what interrupts you are using:
# cat /proc/interrrupts
i386 PC have up to 15 usable interrupts out of 16: 0-15

To get a list of IO address and DMA channels used:
# cat /proc/ioports

To check DMA resources:
# cat /proc/dma

To load modules into kernel:
# modprobe sb irq=5 io=0x220 dma=0
comment, here “sb” means sound device.

Laptops have a special peripheral interface bus called PCMCIA, that requires a special set of drivers and service to handle it. Whenever a card is loaded, you can check the “/var/run/stab”. To see waht setting your card is set to, you can issue the “cardctl” command.

Advanced Power Management(APM)
When the BIOS tells the daemon about a pending suspend or a standbycall, it immediately calls sync(and writes all cached file system information to disk immediatelly). It will then sleep for two seconds and tell BIOS to continute.

# runlevel
5 3
means, you were in level 5 last and are now in level 3.

LCD: Liquid Crystal Display

To have the kernel dynamically load and unload modules as needed, a special kernel thread, “kmod” is called upon to control the loading and unloading of modules. Refer to “/etc/modules.conf”.

To load modules on command line:
# depmod -a
# modproble eth0
Place modules in “modules.conf” to have them automatic loaded.

Two limits manage disk consumption:
1. Limit the number fo inodes a user may have;
2. Limit the number of disk blocks a user’s file may consume

Quota are enabled on a file system basis only, they CAN NOT be set on directories.

NetVault is used to backup data. The SNAZ system has been updated from NetVault 7.4 to latest NetVault 7.4.3. thus supports backup and restore ACL and Extended Attributes.I am verifing this new function these days.

First, in SNAZ operating system, start up backup service. Create a large volume as virtual harddisk for backup and restore. and then create other shares and files with assigning ACL and EAs. Second step, install NetVault GUI management software in Windows platform(or Linux), configure the SNAZ as NetVault server: Addministrator-Domain, select SNAZ; Device-Management-Server-Selection, select SNAZ, Add-Add Standalone Driver, and create virtual disk(right click) on precious large volume(Driver location-/exports/large_volume). Now, could run Backup and Restore.

Extended file attributes:
Permission, records of creation and modification times, author of a document, character encoding of a plain-text document, or a checksum. There are four name spaces: user,trusted,security and system.

Attr:(Add, remove and print)
set or add extended attribute to file/folder:
# attr -s attrname -V attr_value pathname(file/folder name)
print/show extended attribute of file/folder:
# attr -g attrname pathname(file/folder name)
remove identified attribute:
# attr -r attrname pathname(file/folder name)

getfattr -d: print file/folder’s EA entries.

According to NetVault supporting max 64KB EAs, how to create a pure 64K test file?
#!/bin/bash
# create 64KB EA file
for ((i=0;i<=524288;i++)) do printf "i"; done This script will create a 64KB test file, with 524288 "i" characters. But when I tried to used this output file in the following commands: # attr -s Club -V `cat 64KB_file_name` /exports/barcelona There will be an error messages: argument list too long. I had to assign EA one by one: # !/bin/bash # assign EA one by one for ((i=0;i<=524388;i++)) do attr -s Team_$i -V i /exports/barcelona done It would be added successful, but issue "getfattr -d /export/barcelona" still meet "Argument list too long" errors. I'll fixed this issue tomorrow.

Keep on reading Study Guide.

I have been working in Plasmon for three months. During these days, I tried to know the SNAZ system and the architecture of Plasmon.com. The important jobs are building LAT, install Iplanet, install OpenLDAP+SSL and did some performance testing on these platforms. Though sometimes it was hardly time, keep on researching and looking up in Internet, query, or join email group. Everything would be resolved.

I feel I’ve learned much in this new team. The next target, for me, is RHCE. I plan to accomplish this certificated test in about January or Febrary,2007.

Information about the local and remotely mounted file systems is stored in/etc/fstab. Each file system is described on a separate line. Each line is composed of multiple fields, each separated by spaces or tabs. When your system boots, it processes each file system in the order listed.

A sample /etc/fstab might look like the following:

/dev/hda1 / ext2 defaults 1 1
/dev/hda2 swap swap defaults 0 0
/dev/hda5 /usr ext2 defaults 1 2
/dev/hda7 /tmp ext2 defaults 1 2
/dev/hda8 /var ext2 defaults 1 2
/dev/hda9 /home ext2 defaults 1 2
/dev/hdc /cdrom iso9660 ro,noauto,user 0 0
none /proc proc defaults 0 0

Field Name Description
——————————-
fs_spec: Describes the block device or remote file system to be mounted.
fs_file: Describes the mount point (the directory the file system will be mounted as). For swap partitions, this should be “none.”
fs_vfstype: Describes the file system type. Valid file system types are minix, ext, ext2, xiafs, msdos, hpfs, iso9660, nfs, and swap. This field can also be set to “ignore,” which will cause the system to ignore the entry. Ignoring an entry is useful for marking unused partitions.
fs_mntops: Mount options. Specifies mounting options. Mount options are covered later in this section.
fs_freq: Used by the dump(8) command to determine which file systems need to be dumped. A value of 0 indicates that the file system does not need to bedumped.
fs_passno: Used by the fsck(8) program to determine the order in which filesystems are checked upon boot. The root file system should have an fs_passno setting of 1, and other local file systems should have an fs_passno of 2. Remote file systems should have an fs_passno of 0, which indicates they should not be checked on boot.

Mount Options

Although most file systems in /etc/fstab are given the mount option of Default, there are other options you can use. Options are listed in /etc/fstab, and are separated by commas (no spaces or tabs).

Mount Option Description
—————————-
async: I/O is done asynchronously to this file system.
atime: Timestamps for each inode are updated when accessed.
auto: Can be mounted with the -a option of the mount(8) command.
defaults: Use the default mount options: rw, suid, dev, exec, auto, nouser,and async.
dev: Interpret character or block special devices on the file system.
exec: Allow binaries (programs) to be executed on this file system.
noatime: Timestamps for each inode are not updated when accessed.
noauto: Cannot be mounted with the -a option of the mount(8) command (it must be mounted explicitly).
nodev: Do not interpret character or block special devices on the file system.
noexec: Do not allow binaries (programs) to be executed on this file system.
nosuid: Do not allow setuid or setgid permissions on programs to take effect.
nouser: Do not allow nonroot users to mount this file system.
remount: Attempt to remount a file system that has already been mounted. This is typically used to change mount options, and is only used by the mount(8) command (never used in /etc/mnttab).
ro: Mount the file system as read-only.
rw: Mount the file system as read-write.
suid: Allow setuid or setgid permissions on programs to take effect.
sync: I/O is done synchronously to this file system.
user: Allow nonroot users to mount this file system. This option also sets the noexec, nosuid, and nodev options, unless overridden.

It took me about two days on fixing a P1 bug in AA system. I did not debug programs before, especally the AA is a very large program. In the fixing process, I stared on the code, and asked for help from other engineers, try to add debug information into code. Though I failed again and again, but I feel it was more nearly to the success, because, at least I known what would happen and who could make effect through all the failure.

First, know the actual factor the debugs describes, you’d better do it by yourself, it would be very benifit for your understanding the trouble. Second, try to locate which program would be the target debuged, in this case, Bruce told me to focus on “/bin/SYS_INSTALLSTARTOS”, and thanks for Sam’s advice, I watch on the “/webui/software/update.cgi” as the target which is proved to be right choice. In the “update.cgi”. I put many “print” and “echo” lines to collect the returned values created by the program running.

The issue is, the GUI hang or has no information when the patch is installed. Using the original “update.cgi”, the patch would be installed successfully without error, but the last screen has not inforamtion said the operating status, as normaly, there would be some lines messages said the patch was installed successfully and reboot the system. Other way, if I comment some lines, the successfully information displayed. but in fact, the patch is not installed which is proved by issueing “rpm -qa | grep UDO”.

In the first way, I found the XML parser seemed hang, so I commented these XML lines and added the “reboot” and “pre_err_message” returned from the second way, yes, the program would run well, install the patch and display successfully information. But it could not be considered as the final resolution.

I watched the code again, very carefully, and found other XML function which executes without hang, How do I use this XML formation to instead the trouble one? Just modify the trouble ones as the precious, remove the strings “>>INSTALL_LOG”. Yeah! I got it! The update.cgi runs well as what I want.

Summary, partience is the most important thing, and more carefully debug tracing. Nothing is difficult if put all heart on it, or nothing is easy if always thinks it’s too hard to do it.

The following is some modified part of the “update.cgi”source code:
Precious XML:
my $xml_out = `$config{‘update_system_cmd’} $install_file 0 pre_install 2>/dev/null`;
# Get parameter from XML input
my $parser = new XML::DOM::Parser;
my $doc = $parser->parse($xml_out);
my $pre_msg = &xml_in($doc, “pre_msg”, 0);
my $force_install_flag = 0;

Second XML(also my target):
my $cmd_out = `$config{‘update_system_cmd’} $install_file $force_install install 2>/dev/null`;
# ori: my $cmd_out = `$config{‘update_system_cmd’} $install_file $forceinstall install >> INSTALL_LOG 2>/dev/null`
$errnum = $?;
# Get parameter from XML input
my $parser = new XML::DOM::Parser;
my $doc = $parser->parse($cmd_out); # even confuse here
$patch_extra_msg = &xml_in($doc, “err_msg”, 0);
$reboot = &xml_in($doc, “need_reboot”, 0);
Summary, the term “INSTALL_LOG” makes the code confused and XML parser not work.

When use vi to open the text files created in Windows, I found there’s “^M” in the end of each line. Such file is not able to recognized by Linux, it is also why I could not import the Iplanet LDIF files created in Windows to OpenLDAP. There are two methods to resolve this issue.

First use vi edit this file to instead of “^M” string combination. Or second way maybe the more easier, use “dos2unix” command to convert.
# man dos2unix
Dos/MAC to Unix text file format converter.
“iconv” is a tools to convert big5 to UTF-8 character set.