CentOS 6 IScsi Tgtd Target setup and performance adjustments

Home » Computer Articles » Linux » CentOS 6 IScsi Tgtd Target setup and performance adjustments
January 9, 2012 Linux No Comments

IScsi installation on CentOS 6

This server has a Intel i5 processor with 8G memory. 3ware data raid adapter with 14 - 1TB sata hard drives. The Iscsi server is running VMware Virtual Windows 2008 r2 and it is performing well.
Do a minimal Centos 6 server installation.
Do all updates on Server
# yum update

Other needed applications

# yum install sysstat sg3_utils

3ware Storage Adapter Software

unzip 3DM2_CLI-Linux_10.2.1_9.5.4.zip
sudo sh ./install.sh choose default options.
When you are finished with your installation you should see /usr/sbin/3dm2 running 3 times.
To connect go to https://domainname:888

Installation of Iscsi Target software

# yum install scsi-target-utils
Disk partition
Make sure you have GPT Kernel Support. (CentOS and Redhat Enterprise has this option enabled by default)

File Systems > Partition Types > [*] Advanced partition selection and [*] EFI GUID Partition support

Use parted and add LVM Partitions of the size you would like.
Example:
# parted /dev/sdb
mklabel gpt (Creates a new GPT disklabel, only need to do once on new disk)
mkpart primary 0 3001G (Create 3TB partition size)
set 1 lvm on (Change number as for the partition number you are setting up)
quit
# mkfs.ext4 -j /dev/sda1 (Use mkfs to format file system for a local partition)

Partition Problems

Server boots up into disk repair mode and you want to remount / rw
mount -n -o remount,rw /

Find UUID of a partition

# blkid

Network Setup

# vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE="eth0"
IPADDR="10.0.0.2"
HWADDR="mac address"
NETWORK="10.0.0.0"
NETMASK="255.255.255.0"
USERCTL="no"
BOOTPROTO="none"
GATEWAY="10.0.0.1"
ONBOOT="yes"
IPV6INIT="no"

Create a Bond0 Configuration File

# vi /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE="bond0"
IPADDR="192.168.0.4"
NETWORK="192.168.0.0"
NETMASK="255.255.255.0"
USERCTL="no"
BOOTPROTO="none"
ONBOOT="yes"
IPV6INIT="no"

MTU="9000"

# ifconfig bond0 mtu 9000 (This worked)

To confirm the MTU used between two specific devices.

# ip route get {IP-address}

# vi /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE="eth1"
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes

BOOTPROTO=none

# vi /etc/sysconfig/network-scripts/ifcfg-eth2
DEVICE="eth2"
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes

BOOTPROTO=none

# vi /etc/modprobe.d/bonding.conf
alias bond0 bonding

options bond0 mode=balance-alb miimon=100

# modprobe bonding
# service network restart
# cat /proc/net/bonding/bond0 Check output

Configuration of Iscsi Target software

# vi /etc/tgt/targets.conf add similar to below, Lots of examples is file.
# This is a sample config file for tgt-admin.
# By default, tgt-admin looks for its config file in /etc/tgt/targets.conf
#
# The "#" symbol disables the processing of a line.

# This one includes other config files:

#include /etc/tgt/temp/*.conf

# Set the driver. If not specified, defaults to "iscsi".

default-driver iscsi

## For Server1
<target iqn.2008-05.domainname.3ware.vlm2>
    direct-store /dev/sda2
    initiator-address 192.168.0.8
</target>

## For VMware Host Servers
<target iqn.2008-05.domainname.3ware.vmware1>
    direct-store /dev/sda3
    initiator-address 192.168.0.10
    initiator-address 192.168.0.11
    initiator-address 192.168.0.12
    initiator-address 192.168.0.13
    initiator-address 192.168.0.14
    initiator-address 192.168.0.15
</target>

# Parameters below are only global. They can't be configured per LUN.
# Tuning parameters (global, per target)
MaxRecvDataSegmentLength 262144
MaxXmitDataSegmentLength 262144
#HeaderDigest None
#DataDigest None
InitialR2T No
MaxOutstandingR2T 8
ImmediateData Yes
#FirstBurstLength 65536
#MaxBurstLength 262144
#DataPDUInOrder Yes
#DataSequenceInOrder Yes
#ErrorRecoveryLevel 0
#IFMarker No
#OFMarker No
#DefaultTime2Wait 2
#DefaultTime2Retain 20
#OFMarkInt Reject
#IFMarkInt Reject
#MaxConnections 1

Start and Stop Iscsi Target Services

# /etc/init.d/tgtd start
# /etc/init.d/tgtd stop
# /etc/init.d/tgtd reload (Use this after changes to targets.conf)
# /etc/init.d/tgtd force-restart (Sometimes I had to run this to force changes from targets.conf)
# /etc/init.d/tgtd status

Reload may not reload ACL information for connecting more servers.

tgtadm --lld iscsi --op bind --mode target --tid 3 --initiator-address 192.168.0.10
--tid point to Target 3 and you can change bind to unbind so you can remove a address.

Show Target Information

# tgtadm --lld iscsi --mode target --op show

Setup IscsiTarget server to start with server

# chkconfig tgtd on

Performance Adjustments

Edit # vi sysctl.conf and add theses changes
# 3Ware Performance Adjustments
vm.dirty_background_ratio = 20

vm.dirty_ratio = 60

# Network Performance Adjustments
net.core.rmem_max = 1048576
net.core.rmem_default = 1048576
net.core.wmem_max = 1048576
net.core.wmem_default = 1048576
net.ipv4.tcp_mem = 1048576 1048576 1048576
net.ipv4.tcp_wmem = 1048576 1048576 1048576
net.ipv4.tcp_rmem = 1048576 1048576 1048576

For Server with more memory

# Network Performance Adjustments
net.core.rmem_max = 8388608
net.core.wmem_max = 8388608
net.core.rmem_default = 65536
net.core.wmem_default = 65536
net.ipv4.tcp_rmem = 4096 87380 8388608
net.ipv4.tcp_wmem = 4096 65536 8388608
net.ipv4.tcp_mem = 8388608 8388608 8388608
net.ipv4.route.flush = 1

Special Tweaks

# vi /etc/rc.local
echo "deadline" > /sys/block/sda/queue/scheduler
echo "deadline" > /sys/block/sdb/queue/scheduler
echo "64" > /sys/block/sda/queue/max_sectors_kb
echo "512" > /sys/block/sda/queue/nr_requests
Share This:

Leave a Reply

Your email address will not be published. Required fields are marked *