Contents
Deploying RSYNC and Configuring SSH
This section describes how to install RSYNC on the hosts that will be part of your Log File Management Tool (LFMT) deployment.
RSYNC
RSYNC is a tool that provides quick and incremental file transfer. It is a highly efficient file transfer mechanism that maximizes throughput while minimizing bandwidth utilization. It is used by LFMT to upload log files from an application server to the LFMT Server.
Deploying RSYNC Procedures
To use LFMT, RSYNC must be deployed on the host that is running the LFMT Server, and on each host that is running applications that are generating log files. For hosts running Windows, the Cygwin utility must be installed to facilitate the RSYNC transfer. OpenSSH is a daemon process that is used to ensure file transfer.
The required procedures are described in the following Operating Systems:
On Linux
OpenSSH must be installed on all Linux hosts running applications that are
generating log files (application server hosts). Use this procedure to install it
on those hosts where it is not already installed.
1. Download OpenSSH from the following URL: http://www.openssh.org/
2. From the Linux command line, enter the following command and press Enter:
yum -y install openssh-server openssh-clients
3. To start and enable the sshd server, enter the following commands, pressing Enter after each:
chkconfig sshd on service sshd start
4. To open port 22 over which all IP addresses will communicate with RSYNC and the LFMT, enter the following commands, pressing Enter after each:
/sbin/iptable -A input -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT service iptables save
5. Repeat this procedure for each application server host on which OpenSSH is not already installed.
Use this procedure to install RSYNC on all Linux hosts, including the LFMT
Server host and the application server hosts.
Prerequisites
- OpenSSH is installed on the application server host.
- On Ubuntu systems—yum install rsync
- On Red Hat Enterprise Linux and CentOS systems—yum -y install rsync
Purpose: Use this procedure to automatically create public and private
encryption keys to ensure secure transfer files to CLS data repository. This
procedure also automatically distributes the public keys between the
LFMT/CLS and all application server hosts to enable the secure transfer of log
files between the LFMT/CLS and the applications.
1. On the command line, enter the following commands, and press Enter after each:
yum install tcl yum install expect
2. Open (or create, if necessary) the keys_conf.txt file in the text editor. (A
sample copy is located under the “<LFMT Server Install
Directory>/utilities” directory.)
3. Enter all the information about each application server host (host name, user name, password), one server per
line with information fields separated by a tab character, as follows:
<host><tab><username><tab><password>
For example, the following is a keys_conf.txt file for three application
server hosts:
ca-to-alum genesysWIN password ca-to-bond genesysWIN password ca-to-port genesysLNX password
$ mkpasswd -d -u <username>
The above command will produce an output similar to that below:
The case used for defining username should reflect the first instance of its occurrence in the above output. The same case for username should be used when defining unix_user_account or windows_user_account in the LFMT configuration options.
4. Save the file.
5. Ensure that the <LFMT Server Install Directory>/lfm_keys.sh file is in
the same directory as the keys_conf.txt file.
6. Open a Linux terminal and navigate to the directory where the lfm_keys.sh
file is stored.
7. Execute the script:
./lfm_keys.sh. The time taken for this script to execute depends on the number of application server hosts defined in the
keys_conf.txt file.
On Solaris
- Download the following software packages from the website www.sunfreeware.com:
- libgcc-3.4.6-sol10-sparc-local.gz
- popt-1.16-sol10-sparc-local.gz
- libiconv-1.14-sol10-sparc-local.gz
- libintl-3.4.0-sol10-sparc-local.gz
- rsync-3.0.9-sol10-sparc-local.gz
- Copy each file to the /usr/sfw/bin directory on the Solaris host.
- Unzip and install the file libgcc-3.4.6-sol10-sparc-local.gz from the
/usr/sfw/bin directory, as follows:
- Enter the following commands to unzip the file and start the installation:
gunzip libgcc-3.4.6-sol10-sparc-local.gz pkgadd -G -d libgcc-3.4.6-sol10-sparc-local
- When prompted for the packages to be processed, enter all or press Enter.
- When prompted if you want to create the base directory, enter y.
When installation is complete, the following message appears:
Installation of <SMClgcc346> was successful. - Enter the following commands to unzip the file and start the installation:
- Unzip and install the file popt-1.16-sol10-sparc-local.gz from the
/usr/sfw/bin directory, as follows:
- Enter the following commands to unzip the file and start the
installation:
gunzip popt-1.16-sol10-sparc-local.gz pkgadd -G -d popt-1.16-sol10-sparc-local
- When prompted for the packages to be processed, enter all or press Enter.
When installation is complete, the following message appears:
Installation of <SMCpopt> was successful. - Enter the following commands to unzip the file and start the
installation:
- Unzip and install the file libiconv-1.14-sol10-sparc-local.gz from the
/usr/sfw/bin directory, as follows:
- Enter the following commands to unzip the file and start the
installation:
gunzip libiconv-1.14-sol10-sparc-local.gz pkgadd -G -d libiconv-1.14-sol10-sparc-local
- When prompted for the packages to be processed, enter all or press Enter.
When installation is complete, the following message appears:
Installation of <SMCliconv> was successful. - Enter the following commands to unzip the file and start the
installation:
- Unzip and install the file libintl-3.4.0-sol10-sparc-local.gz from the
/usr/sfw/bin directory, as follows:
- Enter the following commands to unzip the file and start the
installation:
gunzip libintl-3.4.0-sol10-sparc-local.gz pkgadd -G -d libintl-3.4.0-sol10-sparc-local
- When prompted for the packages to be processed, enter all or press Enter.
When installation is complete, the following message appears:
Installation of <SMClintl> was successful. - Enter the following commands to unzip the file and start the
installation:
- Unzip and install the file rsync-3.0.9-sol10-sparc-local.gz from the
/usr/sfw/bin directory, as follows:
- Enter the following commands to unzip the file and start the
installation:
gunzip rsync-3.0.9-sol10-sparc-local.gz pkgadd -G -d rsync-3.0.9-sol10-sparc-local
- When prompted for the packages to be processed, enter all or press Enter.
When installation is complete, the following message appears:
Installation of <SMCrsync> was successful. - Enter the following commands to unzip the file and start the
installation:
- Make sure that RSYNC is in the /usr/bin directory. To check, enter the
command which rsync. You will get one of two responses:
- /usr/bin/rsync—RSYNC is in the correct directory.
- <install directory/rsync>—RSYNC is not in the correct directory; enter cp <install directory/rsync> /usr/bin to copy it to the correct directory.
- Repeat this procedure, starting at step 2 for each Solaris host.
On Windows
You must install Cygwin on all windows before you can install RSYNC. This procedure allows you to install Cygwin in one of two ways:
- Install manually, on one host, by completing this procedure. Then, automatically deploy Cygwin to all of your other Windows hosts using this procedure.
- Install manually on all hosts.
- Locate and extract the cygwin.zip file that is found in the <LFMT Server Install Directory>\utilities directory.
- From the location where the cygwin.zip file was extracted, run the setup.exe file from the Cygwin Setup File folder.
- Select Install from Local Directory.
- Set the following parameters:
- Set the root directory to the path where Cygwin is to be installed.
- Select Install for All Users.
- In the Local Package Directory field, specify the path to the Cygwin local directory folder found in the location where the cygwin.zip file was extracted.
- Click Next.
- Select the packages to install. For each package, click Skip where
appropriate in the new column, and the version number appears. This
indicates the package will be installed with that, the latest, version.
Select the following packages:- From Admin, select all of the packages.
- From Archive, select the unzip and zip packages.
- From Editors, select the vim package.
- From Net, select the openssh, rsync, and tcp_wrappers packages.
- From Tcl, select expect package.
- Click Next to install the packages.
- Add values to the Environment PATH variable, as follows:
- Right-click on the My Computer icon on your desktop and select Properties from the drop-down menu.
- Open the Advanced tab and click Environment Variables.
- Create a new system variable, CYGWIN_HOME, and set the value to the directory where the cygwin folder is stored. For example, name=CYGWIN_HOME, value=C:\Cygwin. Click OK.
- Scroll through the System variables list, and select PATH, then click Edit.
- Add %CYGWIN_HOME%\bin to the existing parameters. Make sure that the %CYGWIN_HOME%\bin is set AFTER the path to PERL lib folder.
- Click OK twice more to close the System Properties dialog box.
- On the Installation Complete screen, click Finish.
You must install SSH manually on the LFMT host to ensure secure file transfer. Installing and configuring SSH on the LFMT host is also required for the automatic deployment of Cygwin.
1. Right-click on the Cygwin terminal icon and select “Run as administrator”.
2. Navigate to the directory containing the cygwin_setup.sh file.
3. Convert the cygwin_setup.sh file to the Unix text format. Enter:
dos2unix cygwin_setup.sh
4. Configure the SSH service. Enter:
bash cygwin_setup.sh <username> "<password>"
5. Copy cygwin_setup.sh and cygrun.bat files under <LFMT Server Install
Directory>\utilities to "<cygwin install directory>" (for example,
C:\cygwin) on the LFMT host and on the application server hosts that will
be transferring files.
The screenshot below shows the messages received during a successful deployment of the SSH service.
6. Repeat step 1 through 5 for all Windows hosts if Cygwin is installed manually.
Purpose: Use this procedure to automatically deploy Cygwin to those Windows hosts specified in the cygwin_conf.txt file.
- WMIC is installed on the application server host, and the Windows firewall is configured to allow a WMIC connection. To test that WMIC is working properly on the application server host, go to http://wiki.serverscheck.com/index.php/Testing_wmi.
- The host file is updated with the correct DNS IP addresses and host names of the remote hosts. To find your hosts file, go to http://en.wikipedia.org/wiki/Hosts_%28file%29.
2. Create a new network shared folder from the extracted cygwin.zip file.
3. Open (or create, if necessary) the cygwin_conf.txt file (a sample copy is located under found under “<LFMT Server Install directory>\utilities”) in the text editor.
4. Enter in this file all information about each application server host (computer name, domain, username, password), one server per line, with information fields separated by a tab character. The format of each line is as follows:
<host><tab><domain><tab><primary drive><tab><username><tab><password>
For example, the following is a cygwin_conf.txt file for three application server hosts:
ca-to-alum Genesys C: genesysWIN password ca-to-bond Genesys L: genesysWIN password ca-to-west Genesys I: genesysWIN password
$ mkpasswd -d -u <username>
The above command will produce an output similar to that below:
<username>:unused:77777:11111:<User Name>,U-<DOMAIN>\<username>,S-1-8-21-2222222222-965789356-1112345605-55555:/home/<username>:/bin/bash
When you are finished entering the information, save the file.
5. Right-click on the command prompt and select “Run as administrator”.
6. Navigate to the directory containing the deploy_cygwin.pl file.
7. Enter perl deploy_cygwin.pl. Follow the prompts. Wait for the
script to run to completion; the time it takes will depend on how many
application server hosts are listed in cygwin_conf.txt.
1. Open (or create, if necessary) the keys_conf.txt file (a sample copy is
located under the “<LFMT Server Install Directory>\utilities”
directory) in the text editor.
2. Enter all the information about each application server host (host name, username, password), one server per
line with information fields separated by a tab character as follows:
<host><tab><username><tab><password>
For example, the following is a keys_conf.txt file for three application
server hosts:
ca-to-alum genesysWIN password ca-to-bond genesysWIN password ca-to-port genesysLNX password
$ mkpasswd -d -u <username>
The above command will produce an output similar to that below:
<username>:unused:77777:11111:<User Name>,U-<DOMAIN>\<username>,S-1-8-21-2222222222-965789356-1112345605-55555:/home/<username>:/bin/bash
Save the file.
3. Verify that the lfm_keys.sh file is in the same directory as the
keys_conf.txt file.
4. Right-click on the Cygwin terminal icon and select “Run as administrator”.
5. Navigate to lfm_keys.sh directory.
6. Enter dos2unix lfm_keys.sh.
7. Enter bash lfm_keys.sh. The time taken for this script to execute depends on the number of application server
hosts defined in the keys_conf.txt file.
Important Notes
After you have installed and configured RSYNC, you must also do the following:
- Check Firewalls—On Linux hosts, ensure that the SSH port is not blocked. The default SSH port should be port 22. On Windows hosts, ensure that the Windows firewall does not block SSH.
- Check Hosts File—The hosts file on the CLS/LFMT server host must contain the names and IP addresses of each application server host. Consult your system administrator for the location of the hosts file.
- Re-start the server before Deploying the Server.
Feedback
Comment on this article: