How to install Ambari or HDP on Linux

Today we will learn how to install Ambari or HDB on Linux(CentOS):

Table of Contents

Before installation below are some prerequisite:

Check the Maximum Open File Descriptors

The recommended maximum number of open file descriptors is 10000, or more. To check the current value set for the maximum number of open file descriptors, execute the following shell commands on each host:

ulimit –Sn

ulimit -Hn

If the output is not greater than 10000, run the following command to set it to a suitable default:

ulimit -n 10000

root@node1#ulimit –Sn

root@node1#ulimit –Hn

root@node1#ulimit –n 1000

root@node1#ulimit –Sn

1000

2. Adding hosts to “/etc/hosts” file on every host.

root@node1#vi /etc/hosts

(you have to add all the hosts in /etc/hosts file in order of)

IP       FullyDNS         hostname

Eg. 172.29.44.54  hostname.com   hostname

3. Set Up Password-less SSH

[root@node1#~]# ssh-keygen

Generating public/private rsa key pair.

Enter file in which to save the key (/root/.ssh/id_rsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /root/.ssh/id_rsa.

Your public key has been saved in /root/.ssh/id_rsa.pub.

The key fingerprint is:

e4:4e:e6:9d:02:c6:a7:ab:8a:79:74:70:a8:a1:33:9a root@INPUNPCLX1537E

The key's randomart image is:

+--[ RSA 2048]----+

|                 |

|                 |

|   .    .        |

|. o .. o         |

|.o o  + S        |

|= . .. O . .     |

|.= .  . + o      |

|Eo.    . .       |

|o......          |
 

Authorization for ssh to any data node from access node, follow below steps.

[root@node1#~]# ssh-copy-id -i /root/.ssh/id_rsa.pub INPUNPCLX1825E.xyz.com

Warning: Permanently added 'inpunpclx1825e.xyz.com,172.29.79.124' (RSA) to the list of known hosts.

root@node1# password:

Now try logging into the machine, with "ssh root@node2#", and check in

.ssh/authorized_keys

to make sure we haven't added extra keys that you weren't expecting.

Stoping the IPtables service.

root@node1#chkconfig iptables off

root@node1#/etc/init.d/iptables stop

sudo chkconfig iptables –list

Disabled selinux

setenforce 0

sudo vi /etc/selinux/config

( change below parameter )

SELINUX=enforcing to SELINUX=disabled

6. How to disabled THP

root@node1echo never > /sys/kernel/mm/transparent_hugepage/enabled

root@node1# echo never > /sys/kernel/mm/transparent_hugepage/defrag

7. Download the repository

root@node1#sudo wget -nv http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.1.2/ambari.repo -O /etc/yum.repos.d/ambari.repo

8. Install Ambari server

root@node1#sudo yum install ambari-server

9. Run the below command for setting up Ambari Server and select the right options as given in the screenshots.

root@node1#ambari-server

setup let complete the setupit will ask for SELinux is set to ‘permissive’ mode and temporarily disabled.OK to continue [y/n] (y)? y

Enter choice (1): 1 (select according to requirement of jdk)Do you accept oracle binary code license Agreement [y/n] (y)? Y

Enter advanced database configuration [y/n] (y)? n  (select according to database base requirement // but by default it will install PostgreSQL) After that it will show “Ambari server ‘setup’ completed successfully”.

10. To start the Ambari server run the below command :-

root@node1#Ambari-server startAfter showing ambari-server is running move to the next steps.

11. Once the Ambari server is started on the CLI. You have to start the Ambari GUI on browser.

You need to specify the host name and port number 8080. Check the screenshot for more details.

Eg. IP:8080

Use the username-“admin” & password-“admin” for login.

12. Click on “Launch install Wizard” to start with the HDP deployment.

13. You need to provide a name to the cluster.

14. Select thee right HDP stack. In this case we are going to deploy HDP 2.3.

15. Install options :

Here, You need to provide the FQDN’s of all the Hosts you want to include in your cluster. Please refer to the screenshot example to complete this step.

In our case the FQDNs provided are :-

Eg. Hostname1.com

Hostname2.com

Hostname3.com

16. You would also need to provide the private ssh key to automatically install the ambari agents on all the Hosts. Command is below-

Node1#cat  .ssh/id_rsa ( whatever o/p print, all copy and paste in “provide your private key” place)

16.1. To install Ambari agent manually on all the hosts in the cluster.

16.2. Download the Ambari repository file to a directory on your installation host.

root@node1#wget -nv http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.1.0/ambari.repo -O /etc/yum.repos.d/ambari.repo

16.3. Confirm that the repository is configured by checking the repo list.

root@node1#yum repolist

16.4. Install the Ambari Agent on every host in your cluster.

root@node1#yum install ambari-agent

16.5Using a text editor, configure the Ambari Agent by editing the ambari-agent.ini file as shown in the following example

root@node1#vi /etc/ambari-agent/conf/ambari-agent.ini

[server] hostname=<your.ambari.server.hostname>

url_port=8440

secured_url_port=8441

16.6. Start the agent on every host in your cluster.

root@node1#ambari-agent start

17. Confirm Hosts prompts you to confirm that Ambari has located the correct hosts for your cluster and to check those hosts to make sure they have the correct directories, packages, and processes required to continue the install.

Choose services :-

In this step we need to choose all the services which we need to install in the cluster.

18. Assign Masters and Slaves :-

In this step we need to specify the Hosts on which we want to install the Master and Slave services.

Review :-

We need to finally review all the selection we have made and click on deploy.

19. After that customize service window will appear.

If you have hive service install we need to create data base. Create data base name , id and password. Follow below steps to create data base for hive

[root@node1~]# sudo -u postgres -i

-bash-4.1$ psql

psql (8.4.20)

postgres=# create database hivemeta;

CREATE DATABASE

postgres=# create user hive with password 'hive';

CREATE ROLE

postgres=# grant all privileges on database hivemeta to hive;

GRANT

postgres=#

vi /var/lib/pgsql/data/pg_hba.conf

host     all         all  0.0.0.0/0     trust

(Note- on customize service window click on hive tab and enter all the privileges which you will create. If you have selected oozie service follow same process to create data base.)

20. Create the JDBC connection by following below command. Note- path for this directory postgresql-9.3-1101-jdbc4.jar will be different for your server.

root@node1#ambari-server setup --jdbc-db=postgres  --jdbc-driver=/usr/lib/ambari-server/postgresql-9.3-1101-jdbc4.jar

21. Add service winzard window will appears. Just click on deploy and continue.