格式化HDFS出问题

Question

我是在阿里云上配置的，云上的配置只看了13-6.7.8
之后就从前面开始一点点往后做了
使用的包都是前面的课程提到的
hadoop-2.6.0-cdh5.15.1.tar.gz
jdk-8u91-linux-x64.tar.gz

执行 hdfs namenode -format 时，提示

21/02/17 21:16:50 INFO namenode.FSImage: Allocated new BlockPoolId: BP-332358726-172.17.239.134-1613567810743
21/02/17 21:16:50 WARN namenode.NameNode: Encountered exception during format:
java.io.IOException: Cannot create directory /home/hadoop/app/tmp/dfs/name/current
at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:358)atorg.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:573)atorg.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:594)atorg.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:174)atorg.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1047)atorg.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488)atorg.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1615)21/02/1721:16:50ERRORnamenode.NameNode:Failedtostartnamenode.java.io.IOException:Cannotcreatedirectory/home/hadoop/app/tmp/dfs/name/currentatorg.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:358)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:573)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:594)
at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:174)
at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1047)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1615)
21/02/17 21:16:50 INFO util.ExitUtil: Exiting with status 1
21/02/17 21:16:50 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at hadoop000/172.17.239.134
************************************************************/

暂时不知道错误在哪

第二次编辑

对比了一下 3-11HDFS格式化 和后面的阿里云配置 13-10 HDFS部署，找到了不同的地方

尝试在 vi hdfs-site.xml
中修改成下图

依然报错

21/02/17 22:35:41 WARN namenode.NameNode: Encountered exception during format:
java.io.IOException: Cannot create directory /home/hadoop/app/tmp/dfs/name/current
at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:358)atorg.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:573)atorg.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:594)atorg.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:174)atorg.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1047)atorg.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488)atorg.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1615)21/02/1722:35:41ERRORnamenode.NameNode:Failedtostartnamenode.java.io.IOException:Cannotcreatedirectory/home/hadoop/app/tmp/dfs/name/currentatorg.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:358)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:573)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:594)
at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:174)
at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1047)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1615)
21/02/17 22:35:41 INFO util.ExitUtil: Exiting with status 1
21/02/17 22:35:41 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at hadoop000/172.17.239.134
************************************************************/

Answer 1

Michael_PK 2021-02-17 22:55:59

还有个问题，2c4g的太小了，很大可能是带不动Hadoop的

• 提问者 weixin_慕神5384537 #1

  这样嘛。但是老师你的视频里面推荐购买的云服务器好像就是2c4g的，我跟着买的。如果不够要多大阿

Answer 2

Michael_PK 2021-02-17 22:55:07

ERROR namenode.NameNode: Failed to start namenode. java.io.IOException: Cannot create directory /home/hadoop/app/tmp/dfs/name/current

这个报错了，你的操作用户是否有这个创建权限

• 提问者 weixin_慕神5384537 #1

  刚考完试，才回来
  
  hadoop用户给他添加过权限了
  hadoop	ALL=(ALL) 	ALL
  
  刚才用别的方法给hadoop用户root权限了
  
  新错误：
  21/03/30 22:00:17 ERROR namenode.FSNamesystem: FSNamesystem initialization failed.
  org.apache.hadoop.security.KerberosAuthException: failure to login: javax.security.auth.login.LoginException: java.lang.NullPointerException: invalid null input: name
  	at com.sun.security.auth.UnixPrincipal.<init>(UnixPrincipal.java:71)
  	at com.sun.security.auth.module.UnixLoginModule.login(UnixLoginModule.java:133)
  	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  	at java.lang.reflect.Method.invoke(Method.java:498)
  	at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
  	at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)

• Michael_PK 回复 提问者 weixin_慕神5384537 #2

  你把那个app下tmp的目录全删了，重新格式化，注意用Hadoop用户，别用其他用户

• 提问者 weixin_慕神5384537 回复 Michael_PK #3

  删除目录有效，谢谢老师

点开查看后面3条评论