SMART Hard Drive monitoring and email on Ubuntu 14.xx Server

kalazzerx/ October 15, 2016/ Uncategorized/ 0 comments

I run a server at home that runs a multitude of services for me including running my own cloud services.  Because of this I have a lot of redundancy storage to ensure I do not loose anything in a catastrophic hardware failure.  Anyway, even running SATA hardware raids I have still run into problems with drives failing causing data corruption.  Because of this issue I decided to configure some nice tools that already exist to allow me to monitor my drives and report any issues as soon as they are detected.  Do do this I am using smartmontools which is a set of tools used to monitor and test SMART capable hard drives (Most modern hard drives are now SMART capable).  The second tool I use it ssmtp which allows me to send emails out of my server using a gmail account.  The ssmtp ervice also allows me to send messages out with other scripts which is handy too.  These steps are suited for Ubuntu but should be available on other platforms with some small variations.

Configuring SMART monitoring

First thing needed is to identify drives on the system as described by Ubuntu using the commands blkid and df commands.  We are looking for /dev/sd type lines which are your sata drives as seen by your system.   The df command will show you standard drives mounted on you system but will not show you the actual drives of any lvm’s.  The df command will also not show you any hardware raid drives.  Because of this will use the blkid to find the drives that in my case are will show me the hardware raid drives attached to my machine.  Highlighted are the ones I am interested in.  Now when looking at the devices we are only interested in the actual drive which is the first three letters of the device name without any of the numbers – numbers like sdb1 and sdb2 are the partitions – sdb is the drive/device we care about.

 # df  
Filesystem 1K-blocks Used Available Use% Mounted on  
udev 7857576 8 7857568 1% /dev  tmpfs 1573680 1448 1572232 1% 
/run /dev/sda1 480486104 12762772 462824148 3% 
/  none 4 0 4 0% /sys/fs/cgroup none 5120 0 5120 0% 
/run/lock  none 7868388 4 7868384 1% 
/run/shm none 102400 0 102400 0% 
/run/user /dev/mapper/pdc_cecgbijbe1 480486104 214127972 261458948 46% 
/data /dev/sdd1 961302560 903070916 58215260 94% 
/other /dev/sdf1 1922729860 1040514340 882199136 55% 
/mnt/backup-external-drive 
/dev/sde1 961301000 16970036 895476580 2% 
/external 
# blkid 
/dev/sda1: LABEL="ka_root" UUID="9b98e551-3af7-4a1a-b18a-912d1f46f9ee" TYPE="ext4" 
/dev/sda2: UUID="23793eaa-b8c8-4e3b-9347-90125961cc17" TYPE="swap" 
/dev/sdb: TYPE="promise_fasttrack_raid_member" /dev/sdc: TYPE="promise_fasttrack_raid_member" 
/dev/sdd1: LABEL="drive_int" UUID="56a0c3a4-7c3e-4588-ad87-bd5ad11d1098" TYPE="ext4" 
/dev/mapper/pdc_cecgbijbe1: LABEL="ka_data" UUID="9e4f4fa7-49c1-4132-8347-dddd779a2bfa" TYPE="ext4" 
/dev/sde1: LABEL="external" UUID="5eac9e6f-e9fa-49f9-a879-e892068bd984" TYPE="ext4" 
/dev/sdf1: LABEL="BackupUSBdrive" UUID="8a05efe2-a397-4699-90c8-e7bdeed11523" TYPE="ext4" 

From looking at these two commands I see a bunch of devices but they boil down to the following list that I am interested in –

• /dev/sda
• /dev/sdb
• /dev/sdc
• /dev/sdd
• /dev/sde
• /dev/sdf

Now to run some commands to ensure each drive supports the SMART protocol and enable monitoring of them if they do.  This is not 100% required as is can be implied later on but is how I prefer to do this.

smartctl -i /dev/sda 
 smartctl -i /dev/sdb 
 smartctl -i /dev/sdd 
 smartctl -i /dev/sdc 
 smartctl -i /dev/sde 
smartctl -s on /dev/sda 
 smartctl -s on /dev/sdb 
 smartctl -s on /dev/sdc 
 smartctl -s on /dev/sdd 
 smartctl -s on /dev/sdf

Some manual test/commands

The following are some example test we can run on the command line to let us know how our drives are doing – this is not required to setup automated monitoring but is helpful to know if you want to check on your drives manually. More info can be had about these commands using the man pages and resource list at the bottom of this post.

smartctl -c /dev/sda 
smartctl -t long /dev/sda 
smartctl -t short /dev/sda

Configure automated monitoring of our SMART drives

A couple files need to be modified to enable the smartmontools service and configure it to send out an alert if needed.   Again for detailed information on configuring this service see the resource pages at the bottom of this post as I will not go thru all the variations, just the what I used to get this up and running.

First need to enable the service by using modifying the following file – /etc/default/smartmontools

vim /etc/default/smartmontools
vim /etc/smartd.conf 
service smartmontools restart 
ps aux | grep smart

Configuring Email notifications with ssmtp

apt-get install ssmtp 
 cd /etc/ssmtp/ 
 vim revaliases ssmtp.conf 
 echo test | mail -v -s "Test ssmtp setup" chris@example.com 
echo test | mail -v -s "Another TEST of ssmtp" chris@example.com 
ls /usr/bin/ | grep mail 
 mail

References

• http://linuxtecsun.blogspot.com/2013/02/ubuntu-how-to-configure-smartmontools.html
• https://help.ubuntu.com/community/Smartmontools
• http://techedemic.com/2011/06/28/s-m-a-r-t-monitoring-and-email-notification-ubuntu-10-04/
• https://wiki.archlinux.org/index.php/SSMTP
• https://guylabs.ch/2013/07/18/monitor-s-m-a-r-t-with-smartmontools-in-ubuntu-13-04/