hue(2):Hue与Hadoop集成

2024-05-24 11:18

文章标签 集成 hadoop hue

本文主要是介绍hue(2):Hue与Hadoop集成，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

一、实现功能

hue界面可以浏览、更改、添加hdfs上的文件，从而，便于更多人员操作大数据平台。

二、实现步骤

1.hadoop配置

（1）配置etc/Hadoop/hdfs-site.xml

<property><name>dfs.webhdfs.enabled</name><value>true</value>
</property>
<!--下面这个自己本来就有！-->
<property><name>dfs.permissions.enabled</name><value>false</value>
</property>

（2）配置etc/Hadoop/core-site.xml:

<property><name>hadoop.proxyuser.hue.hosts</name><value>*</value>
</property>
<property><name>hadoop.proxyuser.hue.groups</name><value>*</value>
</property>

（3）重启Hadoop服务

2.修改desktop/conf/hue.ini

 [[hdfs_clusters]]# HA support by using HttpFs[[[default]]]# Enter the filesystem urifs_defaultfs=hdfs://hadoop:8020# NameNode logical name.## logical_name=# Use WebHdfs/HttpFs as the communication mechanism.# Domain should be the NameNode or HttpFs host.# Default port is 14000 for HttpFs.webhdfs_url=http://hadoop:50070/webhdfs/v1# Change this if your HDFS cluster is Kerberos-secured## security_enabled=false# Default umask for file and directory creation, specified in an octal value.## umask=022# Directory of the Hadoop configurationhadoop_conf_dir=/opt/modules/hadoop-2.6.0-cdh5.7.0/etc/hadoophadoop_hdfs_home=/opt/modules/hadoop-2.6.0-cdh5.7.0/hadoop_bin =/opt/modules/hadoop-2.6.0-cdh5.7.0/bin/[[yarn_clusters]][[[default]]]# Enter the host on which you are running the ResourceManagerresourcemanager_host=hadoop# The port where the ResourceManager IPC listens onresourcemanager_port=8032# Whether to submit jobs to this clustersubmit_to=True# Resource Manager logical name (required for HA)## logical_name=# Change this if your YARN cluster is Kerberos-secured## security_enabled=false# URL of the ResourceManager APIresourcemanager_api_url=http://hadoop:9090 #这个自己由于yarn收到攻击所以更改8088到9090# URL of the ProxyServer APIproxy_api_url=http://hadoop:9090	#这个自己由于yarn收到攻击所以更改8088到9090# URL of the HistoryServer APIhistory_server_api_url=http://hadoop:19888	[filebrowser]# Location on local filesystem where the uploaded archives are temporary stored.archive_upload_tempdir=/tmp