Today we will learn how to install Ambari or HDB on Linux(CentOS):
Before installation below are some prerequisite:
Check the Maximum Open File Descriptors
The recommended maximum number of open file descriptors is 10000, or more. To check the current value set for the maximum number of open file descriptors, execute the following shell commands on each host:
ulimit –Sn ulimit -Hn
If the output is not greater than 10000, run the following command to set it to a suitable default:
ulimit -n 10000
root@node1#ulimit –Sn root@node1#ulimit –Hn root@node1#ulimit –n 1000 root@node1#ulimit –Sn 1000
2. Adding hosts to “/etc/hosts” file on every host.
root@node1#vi /etc/hosts (you have to add all the hosts in /etc/hosts file in order of) IP FullyDNS hostname Eg. 172.29.44.54 hostname.com hostname
3. Set Up Password-less SSH
[root@node1#~]# ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: e4:4e:e6:9d:02:c6:a7:ab:8a:79:74:70:a8:a1:33:9a root@INPUNPCLX1537E The key's randomart image is: +--[ RSA 2048]----+ | | | | | . . | |. o .. o | |.o o + S | |= . .. O . . | |.= . . + o | |Eo. . . | |o...... | Authorization for ssh to any data node from access node, follow below steps. [root@node1#~]# ssh-copy-id -i /root/.ssh/id_rsa.pub INPUNPCLX1825E.xyz.com Warning: Permanently added 'inpunpclx1825e.xyz.com,172.29.79.124' (RSA) to the list of known hosts. root@node1# password: Now try logging into the machine, with "ssh root@node2#", and check in .ssh/authorized_keys to make sure we haven't added extra keys that you weren't expecting.
- Stoping the IPtables service.
root@node1#chkconfig iptables off root@node1#/etc/init.d/iptables stop sudo chkconfig iptables –list
- Disabled selinux
setenforce 0 sudo vi /etc/selinux/config ( change below parameter ) SELINUX=enforcing to SELINUX=disabled
6. How to disabled THP
root@node1echo never > /sys/kernel/mm/transparent_hugepage/enabled root@node1# echo never > /sys/kernel/mm/transparent_hugepage/defrag
7. Download the repository
root@node1#sudo wget -nv http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.1.2/ambari.repo -O /etc/yum.repos.d/ambari.repo
8. Install Ambari server
root@node1#sudo yum install ambari-server
9. Run the below command for setting up Ambari Server and select the right options as given in the screenshots.
root@node1#ambari-server setup let complete the setupit will ask for SELinux is set to ‘permissive’ mode and temporarily disabled.OK to continue [y/n] (y)? y Enter choice (1): 1 (select according to requirement of jdk)Do you accept oracle binary code license Agreement [y/n] (y)? Y Enter advanced database configuration [y/n] (y)? n (select according to database base requirement // but by default it will install PostgreSQL) After that it will show “Ambari server ‘setup’ completed successfully”.
10. To start the Ambari server run the below command :-
root@node1#Ambari-server startAfter showing ambari-server is running move to the next steps.
11. Once the Ambari server is started on the CLI. You have to start the Ambari GUI on browser.
You need to specify the host name and port number 8080. Check the screenshot for more details.
Use the username-“admin” & password-“admin” for login.
12. Click on “Launch install Wizard” to start with the HDP deployment.
13. You need to provide a name to the cluster.
14. Select thee right HDP stack. In this case we are going to deploy HDP 2.3.
15. Install options :
Here, You need to provide the FQDN’s of all the Hosts you want to include in your cluster. Please refer to the screenshot example to complete this step.
In our case the FQDNs provided are :-
16. You would also need to provide the private ssh key to automatically install the ambari agents on all the Hosts. Command is below-
Node1#cat .ssh/id_rsa ( whatever o/p print, all copy and paste in “provide your private key” place)
16.1. To install Ambari agent manually on all the hosts in the cluster.
16.2. Download the Ambari repository file to a directory on your installation host.
root@node1#wget -nv http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.1.0/ambari.repo -O /etc/yum.repos.d/ambari.repo
16.3. Confirm that the repository is configured by checking the repo list.
16.4. Install the Ambari Agent on every host in your cluster.
root@node1#yum install ambari-agent
16.5Using a text editor, configure the Ambari Agent by editing the ambari-agent.ini file as shown in the following example
16.6. Start the agent on every host in your cluster.
17. Confirm Hosts prompts you to confirm that Ambari has located the correct hosts for your cluster and to check those hosts to make sure they have the correct directories, packages, and processes required to continue the install.
- Choose services :-
In this step we need to choose all the services which we need to install in the cluster.
18. Assign Masters and Slaves :-
In this step we need to specify the Hosts on which we want to install the Master and Slave services.
We need to finally review all the selection we have made and click on deploy.
19. After that customize service window will appear.
If you have hive service install we need to create data base. Create data base name , id and password. Follow below steps to create data base for hive
[root@node1~]# sudo -u postgres -i -bash-4.1$ psql psql (8.4.20) postgres=# create database hivemeta; CREATE DATABASE postgres=# create user hive with password 'hive'; CREATE ROLE postgres=# grant all privileges on database hivemeta to hive; GRANT postgres=# vi /var/lib/pgsql/data/pg_hba.conf host all all 0.0.0.0/0 trust
(Note- on customize service window click on hive tab and enter all the privileges which you will create. If you have selected oozie service follow same process to create data base.)
20. Create the JDBC connection by following below command. Note- path for this directory postgresql-9.3-1101-jdbc4.jar will be different for your server.
root@node1#ambari-server setup --jdbc-db=postgres --jdbc-driver=/usr/lib/ambari-server/postgresql-9.3-1101-jdbc4.jar
21. Add service winzard window will appears. Just click on deploy and continue.
22. Install, Start and Test :-
Click on next once all the installation on the all the Hosts is complete.
23. Complete :-
If you get any type of error post your comment here and if you have ambari-blog
other requirement related to linux or hadoop comment below