请稍等 ...
×

采纳答案成功！

向帮助你的同学说点啥吧！感谢那些助人为乐的人

使用spark-submit 提交flume作业报错

将编译好的文件jar提交到服务器，
先启动flume，
flume-ng agent
–name simple-agent
–conf $FLUME_HOME/conf
–conf-file $FLUME_HOME/conf/flume_pull_streaming.conf
-Dflume.root.logger=INFO,console
没有报错，

在提交jar运行：
spark-submit
–class com.derek.spark.FlumePullWordCount
–master local[2]
–packages org.apache.spark:spark-streaming-flume_2.11:2.2.0
/home/hadoop/lib/sparktrain-1.0.jar
hadoop000 41414

我的问题是，我代码在idea里运行，将两个参数在启动时传入，没有问题。
在服务器上按照流程运行就不行。下面是log

error log：

19/12/09 22:21:43 ERROR TaskSetManager: Task 0 in stage 402.0 failed 1 times; aborting job
19/12/09 22:21:43 ERROR ReceiverTracker: Receiver has been stopped. Try to restart it.
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 402.0 failed 1 times, most recent failure: Lost task 0.0 in stage 402.0 (TID 388, localhost, executor driver): java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.initializeLogIfNecessary(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.log(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.logInfo(FlumePollingInputDStream.scala:61)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:88)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:87)
        at scala.collection.immutable.Range.foreach(Range.scala:160)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:87)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:601)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:591)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
        at org.apache.spark.scheduler.Task.run(Task.scala:123)
        at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

Driver stacktrace:
        at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1889)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1877)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1876)
        at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
        at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1876)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
        at scala.Option.foreach(Option.scala:257)
        at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:926)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2110)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2059)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2048)
        at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
Caused by: java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.initializeLogIfNecessary(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.log(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.logInfo(FlumePollingInputDStream.scala:61)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:88)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:87)
        at scala.collection.immutable.Range.foreach(Range.scala:160)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:87)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:601)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:591)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
        at org.apache.spark.scheduler.Task.run(Task.scala:123)
        at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
19/12/09 22:21:43 ERROR Executor: Exception in task 0.0 in stage 403.0 (TID 389)
java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.initializeLogIfNecessary(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.log(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.logInfo(FlumePollingInputDStream.scala:61)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:88)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:87)
        at scala.collection.immutable.Range.foreach(Range.scala:160)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:87)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:601)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:591)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
        at org.apache.spark.scheduler.Task.run(Task.scala:123)
        at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
19/12/09 22:21:43 ERROR TaskSetManager: Task 0 in stage 403.0 failed 1 times; aborting job
19/12/09 22:21:43 ERROR ReceiverTracker: Receiver has been stopped. Try to restart it.
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 403.0 failed 1 times, most recent failure: Lost task 0.0 in stage 403.0 (TID 389, localhost, executor driver): java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.initializeLogIfNecessary(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.log(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.logInfo(FlumePollingInputDStream.scala:61)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:88)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:87)
        at scala.collection.immutable.Range.foreach(Range.scala:160)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:87)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:601)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:591)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
        at org.apache.spark.scheduler.Task.run(Task.scala:123)
        at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

Driver stacktrace:
        at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1889)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1877)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1876)
        at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
        at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1876)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
        at scala.Option.foreach(Option.scala:257)
        at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:926)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2110)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2059)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2048)
        at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
Caused by: java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.initializeLogIfNecessary(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.log(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.logInfo(FlumePollingInputDStream.scala:61)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:88)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:87)
        at scala.collection.immutable.Range.foreach(Range.scala:160)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:87)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:601)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:591)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
        at org.apache.spark.scheduler.Task.run(Task.scala:123)
        at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
19/12/09 22:21:43 ERROR Executor: Exception in task 0.0 in stage 404.0 (TID 390)
java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.initializeLogIfNecessary(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.log(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.logInfo(FlumePollingInputDStream.scala:61)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:88)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:87)
        at scala.collection.immutable.Range.foreach(Range.scala:160)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:87)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:601)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:591)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
        at org.apache.spark.scheduler.Task.run(Task.scala:123)
        at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
19/12/09 22:21:43 ERROR TaskSetManager: Task 0 in stage 404.0 failed 1 times; aborting job
19/12/09 22:21:43 ERROR ReceiverTracker: Receiver has been stopped. Try to restart it.
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 404.0 failed 1 times, most recent failure: Lost task 0.0 in stage 404.0 (TID 390, localhost, executor driver): java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.initializeLogIfNecessary(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.log(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.logInfo(FlumePollingInputDStream.scala:61)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:88)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:87)
        at scala.collection.immutable.Range.foreach(Range.scala:160)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:87)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:601)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:591)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
        at org.apache.spark.scheduler.Task.run(Task.scala:123)
        at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

Driver stacktrace:
        at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1889)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1877)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1876)
        at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
        at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1876)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
        at scala.Option.foreach(Option.scala:257)
        at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:926)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2110)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2059)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2048)
        at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
Caused by: java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.initializeLogIfNecessary(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.log(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.logInfo(FlumePollingInputDStream.scala:61)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:88)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:87)
        at scala.collection.immutable.Range.foreach(Range.scala:160)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:87)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:601)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:591)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
        at org.apache.spark.scheduler.Task.run(Task.scala:123)
        at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
19/12/09 22:21:43 ERROR Executor: Exception in task 0.0 in stage 405.0 (TID 391)
java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.initializeLogIfNecessary(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.log(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.logInfo(FlumePollingInputDStream.scala:61)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:88)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:87)
        at scala.collection.immutable.Range.foreach(Range.scala:160)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:87)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:601)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:591)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
        at org.apache.spark.scheduler.Task.run(Task.scala:123)
        at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
19/12/09 22:21:43 ERROR TaskSetManager: Task 0 in stage 405.0 failed 1 times; aborting job
19/12/09 22:21:43 ERROR ReceiverTracker: Receiver has been stopped. Try to restart it.
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 405.0 failed 1 times, most recent failure: Lost task 0.0 in stage 405.0 (TID 391, localhost, executor driver): java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.initializeLogIfNecessary(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.log(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.logInfo(FlumePollingInputDStream.scala:61)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:88)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:87)
        at scala.collection.immutable.Range.foreach(Range.scala:160)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:87)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:601)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:591)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
        at org.apache.spark.scheduler.Task.run(Task.scala:123)
        at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

Driver stacktrace:
        at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1889)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1877)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1876)
        at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
        at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1876)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
        at scala.Option.foreach(Option.scala:257)
        at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:926)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2110)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2059)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2048)
        at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
Caused by: java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.initializeLogIfNecessary(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.log(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.logInfo(FlumePollingInputDStream.scala:61)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:88)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:87)
        at scala.collection.immutable.Range.foreach(Range.scala:160)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:87)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:601)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:591)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
        at org.apache.spark.scheduler.Task.run(Task.scala:123)
        at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
19/12/09 22:21:43 ERROR Executor: Exception in task 0.0 in stage 406.0 (TID 392)
java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.initializeLogIfNecessary(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.log(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.logInfo(FlumePollingInputDStream.scala:61)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:88)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:87)
        at scala.collection.immutable.Range.foreach(Range.scala:160)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:87)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:601)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:591)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
        at org.apache.spark.scheduler.Task.run(Task.scala:123)
        at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
19/12/09 22:21:43 ERROR TaskSetManager: Task 0 in stage 406.0 failed 1 times; aborting job
19/12/09 22:21:43 ERROR ReceiverTracker: Receiver has been stopped. Try to restart it.
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 406.0 failed 1 times, most recent failure: Lost task 0.0 in stage 406.0 (TID 392, localhost, executor driver): java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.initializeLogIfNecessary(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.log(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.logInfo(FlumePollingInputDStream.scala:61)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:88)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:87)
        at scala.collection.immutable.Range.foreach(Range.scala:160)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:87)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:601)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:591)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
        at org.apache.spark.scheduler.Task.run(Task.scala:123)
        at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

Driver stacktrace:
        at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1889)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1877)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1876)
        at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
        at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1876)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
        at scala.Option.foreach(Option.scala:257)
        at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:926)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2110)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2059)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2048)
        at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
Caused by: java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.initializeLogIfNecessary(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.log(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.logInfo(FlumePollingInputDStream.scala:61)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:88)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:87)
        at scala.collection.immutable.Range.foreach(Range.scala:160)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:87)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:601)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:591)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
        at org.apache.spark.scheduler.Task.run(Task.scala:123)
        at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
19/12/09 22:21:43 ERROR Executor: Exception in task 0.0 in stage 407.0 (TID 393)
java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.initializeLogIfNecessary(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.log(FlumePollingInputDStream.scala:61)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.logInfo(FlumePollingInputDStream.scala:61)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:88)
        at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$2.apply(FlumePollingInputDStream.scala:87)
        at scala.collection.immutable.Range.foreach(Range.scala:160)
        at org.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:87)
zhangyulei 2019-12-09 22:31:10
源自：10-2  -Spark Streaming整合Kafka的版本选择详解
695
收起

正在回答

1回答

Michael_PK 回答被采纳获得+3积分 2019-12-09 23:23:16

先了解下你服务器的spark版本和你packages里面指定的spark的版本号是否一致，根据这异常信息，可能是依赖的包版本不太一致

0 回复 有任何疑惑可以回复我~
收起回答
  • 提问者 zhangyulei #1 
    非常感谢！
    回复 有任何疑惑可以回复我~ 2019-12-09 23:49:10
  • 提问者 zhangyulei #2 
    我的spark 包 不是cdh的那个，  换成 cdh那个就好了
    回复 有任何疑惑可以回复我~ 2019-12-09 23:49:54
取消 回复

相似问题

spark spark
1031 0 3
老师，有多个streaming架包作业能同时运行吗？
613 0 3
请问，为什么spark-sql里show tables有表，spark-submit提交job里没有表？
1400 0 4
server模式Cache表的生命周期
418 0 3
使用spark-submit 提交作业报错
601 0 3

登录后可查看更多问答，登录/注册

问题已解决，确定采纳
还有疑问，暂不采纳
Spark Streaming实时流处理项目实战
  • 参与学习       1404    人
  • 解答问题       571    个

Flume+Kafka+Spark Streaming 构建通用实时流处理平台

了解课程

本课精华内容

问答

读英语的时候能不能不要propertie撕，topic撕

1.7k 89

cannot assign requested address

1.1k 14

kafka生产者消息发不出去

2.3k 13

kafka.serializer.StringDecoder死活导不进来

1.7k 12

spark streaming运行较长时间后报错kafka.common.OffsetOutOfRangeException

2.4k 12

查看更多本课问答
意见反馈 帮助中心 APP下载
官方微信