Thursday, June 15, 2017

Hive 2 Hive

  • Use import/export for small table
  • For big table
    • Get the schema of the remote hive table
      • E.g. beeline -u jdbc:hive2://x.x.x.x:10000 --silent=true --showheader=false --outputformat=tsv2 -e "show create table table_name" > $DDL
    • Change the name node or name services information
      • E.g. 
      • sed -e 's/remote.hadoop.namenode:8020/my.hadoop.namenode:8020/' -i $DDL
      • sed -e 's/remote.hadoop.nameservices/my.hadoop.nameservices/' -i $DDL
    • Create the hive table
      • E.g. hive -f $DDL
    • Get the data of the remote hive table
      • E.g. using distcp

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.