Wednesday, December 31, 2014

Apache Sqoop

  1. Introduction
    1. Sqoop is a tool designed to transfer data between Hadoop and relational databases. You can use Sqoop to import data from a relational database management system (RDBMS) such as MySQL or Oracle into the Hadoop Distributed File System (HDFS), transform the data in Hadoop MapReduce, and then export the data back into an RDBMS.
  2. Creating password file
    1. echo -n password > .password
    2. hdfs dfs -put .password /user/$USER/
  3. Installing the MySQL JDBC driver in CDH
    1. mkdir -p /var/lib/sqoop 
    2. chown sqoop:sqoop /var/lib/sqoop 
    3. chmod 755 /var/lib/sqoop
    4. donwload JDBC dirver from http://dev.mysql.com/downloads/connector/j/5.1.html
    5. sudo cp mysql-connector-java-version/mysql-connector-java-version-bin.jar /var/lib/sqoop/
  4. Reference
    1. http://sqoop.apache.org/
    2. http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_ig_jdbc_driver_install.html

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.