请稍等 ...
×

采纳答案成功!

向帮助你的同学说点啥吧!感谢那些助人为乐的人

启动 ./spark-shell 报错

./spark-shell
19/12/06 14:23:03 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable
Setting default log level to “WARN”.
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
19/12/06 14:23:10 WARN util.Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
19/12/06 14:23:10 WARN util.Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
19/12/06 14:23:10 WARN util.Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
19/12/06 14:23:10 WARN util.Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
19/12/06 14:23:10 WARN util.Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
19/12/06 14:23:10 WARN util.Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
19/12/06 14:23:11 WARN util.Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
19/12/06 14:23:11 WARN util.Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
19/12/06 14:23:11 WARN util.Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
19/12/06 14:23:11 WARN util.Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
19/12/06 14:23:11 WARN util.Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
19/12/06 14:23:11 WARN util.Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
19/12/06 14:23:11 WARN util.Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
19/12/06 14:23:11 WARN util.Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
19/12/06 14:23:11 WARN util.Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
19/12/06 14:23:11 WARN util.Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
19/12/06 14:23:11 ERROR spark.SparkContext: Error initializing SparkContext.
java.net.BindException: Cannot assign requested address: Service ‘sparkDriver’ failed after 16 retries (on a random free port)! Consider explicitly setting the appropriate binding address for the service ‘sparkDriver’ (for example spark.driver.bindAddress for SparkDriver) to the correct binding address.
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:128)
at io.netty.channel.AbstractChannelAbstractUnsafe.bind(AbstractChannel.java:558)atio.netty.channel.DefaultChannelPipelineAbstractUnsafe.bind(AbstractChannel.java:558) at io.netty.channel.DefaultChannelPipelineAbstractUnsafe.bind(AbstractChannel.java:558)atio.netty.channel.DefaultChannelPipelineHeadContext.bind(DefaultChannelPipeline.java:1283)
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:501)
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:486)
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:989)
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:254)
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:364)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
at io.netty.util.concurrent.SingleThreadEventExecutor5.run(SingleThreadEventExecutor.java:858)atio.netty.util.concurrent.DefaultThreadFactory5.run(SingleThreadEventExecutor.java:858) at io.netty.util.concurrent.DefaultThreadFactory5.run(SingleThreadEventExecutor.java:858)atio.netty.util.concurrent.DefaultThreadFactoryDefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
at java.lang.Thread.run(Thread.java:745)
java.net.BindException: Cannot assign requested address: Service ‘sparkDriver’ failed after 16 retries (on a random free port)! Consider explicitly setting the appropriate binding address for the service ‘sparkDriver’ (for example spark.driver.bindAddress for SparkDriver) to the correct binding address.
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:128)
at io.netty.channel.AbstractChannelAbstractUnsafe.bind(AbstractChannel.java:558)atio.netty.channel.DefaultChannelPipelineAbstractUnsafe.bind(AbstractChannel.java:558) at io.netty.channel.DefaultChannelPipelineAbstractUnsafe.bind(AbstractChannel.java:558)atio.netty.channel.DefaultChannelPipelineHeadContext.bind(DefaultChannelPipeline.java:1283)
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:501)
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:486)
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:989)
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:254)
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:364)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
at io.netty.util.concurrent.SingleThreadEventExecutor5.run(SingleThreadEventExecutor.java:858)atio.netty.util.concurrent.DefaultThreadFactory5.run(SingleThreadEventExecutor.java:858) at io.netty.util.concurrent.DefaultThreadFactory5.run(SingleThreadEventExecutor.java:858)atio.netty.util.concurrent.DefaultThreadFactoryDefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
at java.lang.Thread.run(Thread.java:745)
:14: error: not found: value spark
import spark.implicits._
^
:14: error: not found: value spark
import spark.sql
^
Welcome to
____ __

正在回答 回答被采纳积分+3

1回答

提问者 qq_L21_0 2019-12-06 14:33:10

已经按下面方法操作,还是报错

https://github.com/mattshma/bigdata/issues/107

0 回复 有任何疑惑可以回复我~
  • 你jps看看,肯定是不是有很多进程存在了
    回复 有任何疑惑可以回复我~ 2019-12-06 16:56:58
  • 提问者 qq_L21_0 回复 Michael_PK #2
    jps后显示如下
    [hadoop@hadoop000 bin]$ jps
    2653 SparkSubmit
    3838 SparkSubmit
    3919 Jps
    
    重启电脑后,再运行./spark-shell还是上面的错误
    
    我/etc/hosts 文件内容如下
    192.168.199.102 hadoop000
    192.168.199.247 hadoop001
    192.168.199.138 hadoop002
    127.0.0.1 hadoop000
    回复 有任何疑惑可以回复我~ 2019-12-06 17:14:39
  • Michael_PK 回复 提问者 qq_L21_0 #3
    起的太多了,超过参数的值的,你可以调整那个参数的值
    回复 有任何疑惑可以回复我~ 2019-12-06 18:52:24
问题已解决,确定采纳
还有疑问,暂不采纳
意见反馈 帮助中心 APP下载
官方微信