Apache Hive
Interview Questions
HiveQL, partitioning vs bucketing, ORC vs Parquet, SerDe, metastore architecture, and Tez vs MapReduce.
Difference between internal and external table in Hive?
EasyInternal Table: Internal Table is also known as the managed table. When the user creates a table in Hive without specifying it as external, then by default, an internal table gets created in a specific location in HDFS. By default, an internal table will be created in a folder path /user/hive/warehouse If we drop the managed table or partition, the table data and the metadata associated with that table will be deleted from the HDFS.
It support truncate command
External Table: External tables are stored outside the warehouse directory. They can access data stored in sources such as remote HDFS locations or Azure Storage Volumes. Whenever we drop the external table, then only the metadata associated with the table will get deleted, the table data remains untouched by Hive.
It does not support truncate command
What is differnece between static and dynamic partition in hive?:
Easy| Static Partition | Dynamic Partition |
|---|---|
| Usually when loading files (big files) into Hive tables static partitions are preferred. | When you have large data stored in a table then Dynamic partition is suitable. |
| Static Partition saves your time in loading data compared to dynamic partition. You “statically” add a partition in table and move the file into the partition of the table. | Single insert to partition table is known as dynamic partition. Usually dynamic partition load the data from non partitioned table. Dynamic Partition takes more time in loading data compared to static partition. |
| You can get the partition column value form the filename, day of date etc without reading the whole big file. | When you have large data stored in a table then Dynamic partition is suitable. |
| We can alter the partition in static partition. | If you want to partition number of column but you don’t know how many columns then also dynamic partition is suitable. we can’t perform alter on Dynamic partition. |
If you want to use Static partition in hive you should set property:set hive.mapred.mode = strict | If you want to partition number of column but you don’t know how many columns then also dynamic partition is suitable.Dynamic partition in hive then mode is in nonstrict mode. Here is hive dynamic partition properties you should allow:SET hive.exec.dynamic.partition = true;SET hive.exec.dynamic.partition.mode = nonstrict; |