DhansWorld

connect to impala from VM Ubuntu using Kerberos key tab                        On client machine:(Ubuntu-VM) 1)   Download unix odbc driver: http://www.unixodbc.org/unixODBC-2.3.4.tar.gz ./configure  make  make install 2)  install Cloudera impala odbc driver   clouderaimpalaodbc_2.5.32.1002-2_amd64.deb http://www.cloudera.com/downloads/connectors/impala/odbc/2-5-32.html  dpkg -i clouderaimpalaodbc_2.5.32.1002-2_amd64.deb 3) cp /etc/odbc.ini and /etc/ odbcinst.ini  to home directory. 4)  cat /home/user_impala/odbc.ini [ODBC DATA Sources] Impala_DSN=Cloudera Impala ODBC Driver 64-bit [Impala_DSN] Driver=/opt/cloudera/impalaodbc/lib/64/libclouderaimpalaodbc64.so HOST=Impala.dev.Server.com Port=21050 Database=iris AuthMech=1 KrbRealm=DEV.SERVER.COM KrbFQDN=Impala.dev.Server.com KrbServiceName=impala UID=user_impala UseKeytab=1; #UPNKeytabMappingFile=/home/user_i...

Friends, This is a streaming MapReduce job (shell script) that reads any text input and computes the average length of all words that start with each character . --------------------------------------------------------------------------------------------------------------------------------------------------------------- $ cat avg_ln_mpr.sh #! /bin/bash while  read  line do  for word in `echo $line`  do     c=`expr substr $word 1 1`     l=`expr length $word`     echo $c $l  done     done --------------------------------------------------------------------------------------------------------------------------------------------------------------- $ cat avg_ln_rdr.sh #! /bin/bash old='' new='' val='' cnt=1 sum=0 avg=0 start=0 while  read  line do new=`echo $line|cut -d' ' -f1` val=`echo $line|cut -d' ' -f2` if [ "$old" != "$new" ]; then [ $start -ne 0 ] &...

Hello friends, Let's check how to run one simple map reduce program in Linux environment. It's a word count program. 1. create file words.txt with few words like shown below. words.txt -------------------------------- cow india japan america japan hindu muslim christian india cow america america america china india china pakistan 2. cp words.txt to hdfs (give appropriate path) hadoop fs -copyFromLocal words.txt /user/cloudera/words.txt 3. create mapper.sh wc_mapper.sh -------------------------- #! /bin/bash while  read line do  for  word in $line  do     echo  $word 1  done done 4.create reducer.sh wc_reducer.sh ------------------------ #! /bin/bash cnt=0 old='' new='' start=0 while read line do new=`echo $line|cut  -d' ' -f1` if  [ "$new" != "$old" ]; then [ $start -ne 0 ] && echo -e "$old\t$cnt" old=$new cnt=1 start=1 else cnt=$(( $cnt + 1 )) fi; done echo -e ...

Twitter sentiment analysis using r-studio (r console) in ubuntu 1)  login as user su 2) Download  all files from following link: https://drive.google.com/folderview?id=0B1WeP8XHW0OzcEY2TEtwMlZDTmc&usp=sharing#list keep these files in /home/hduser/sentiment ( sample path) positive-words.txt negative-words.txt sentiment.r 3) pre-requisites open  /etc/apt/sources.list and add deb http://<my.favorite.cran.mirror>/bin/linux/ubuntu raring/ sudo apt-get install r-base -- now we have R Console in ubuntu. $ cd /home/hduser/sentiment/ call "R" from command prompt $ R -- Sett working directory(wd) and libraries required for analysis. setwd("/home/hduser/sentiment") install.packages('twitteR') install.packages("ROAuth") install.packages("RCurl") install.packages("plyr") install.packages("stringr") --  it's not required since we have this file. if u want u can download it  b...

Changes Done in UBUNTU 12.04 64-bit : Post Installation 1. Access To Root:     eagroup@BI-Lab:~$ sudo su     [sudo] password for eagroup:     root@BI-Lab:/home/eagroup# sudo passwd     Enter new UNIX password: password     Retype new UNIX password: password     passwd: password updated successfully     root@BI-Lab:/home/eagroup# 2.  Add and manage users and groups:     a. Add user from GUI - hduser,     Username     Password    Privilage     root        password    root     eagroup        password    admin     hduser        hduser        hadoop user 3. Making hduser sudoer:     a. login as root   ...