Saturday, 9 July 2016

02. Sqoop Import Exception SqlException : Failed to start database.

Sqoop Import Exception SqlException : Failed to start database.




cmd:

''sqoop import --connect jdbc:mysql://localhost/Gopal_Lab --table emp -m 1 --hive import"




Fix:
remove .lck in dir and re-run.

sudo rm -rf /var/lib/hive/metastore/metastore_db/*.lck

Permanent fix:
Default hive metastore embedded derby,  not possible to start multiple instance of hive at the same time. 
changing hive metastore to mysql or postgres server this issue can be solved the problem.

Monday, 20 June 2016

01. Setup MYSQL as derby for Apache HIVE in ubuntu


Setup MYSQL as derby for Apache HIVE  in ubuntu:

1. Open /usr/local/hadoop/hive/conf$ nano hive-site.xml

2. change following settings and restart hive prompt:

<configuration>
  <property>
    <name>hive.metastore.local</name>
    <value>true</value>
  </property>
  <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
  </property>
  <property>
    <name>hive.metastore.uris</name>
    <value></value>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost:3306/hive_db?createDatabaseIfNotExist=true</value>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
  </property>
  <property>
    <name>hive.querylog.location</name>
    <value></value>
  </property>
  <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>root</value>
  </property>
  <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>root</value>
  </property>
</configuration>


check MYSQL show databases result with DB name:

Arun@ubuntu:/usr/local/hadoop/hive/conf$ mysql -u root -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 36
Server version: 5.5.43-0ubuntu0.12.04.1 (Ubuntu)

Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| classicmodels      |
| hive_db            |
| mysql              |
| performance_schema |
| retail_db          |
+--------------------+
6 rows in set (0.04 sec)