spark-2.1.0的编译问题咨询，非常感谢

Question

[hadoop@hadoop001 spark-2.1.0]$ ./build/mvn -Pyarn -Phadoop-2.6 -Phive -Phive-thriftserver -Dhadoop.version=2.6.0-cdh5.7.0 -DskipTests clean package

exec: curl --progress-bar -L https://downloads.typesafe.com/zinc/0.3.9/zinc-0.3.9.tgz

######################################################################## 100.0%

Using `mvn` from path: /home/hadoop/source/spark-2.1.0/build/apache-maven-3.3.9/bin/mvn

Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0

[INFO] ------------------------------------------------------------------------

[INFO] Building Spark Project Launcher 2.1.0

[INFO] ------------------------------------------------------------------------

[WARNING] The POM for org.apache.hadoop:hadoop-client:jar:2.6.0-cdh5.7.0 is missing, no dependency information available

[INFO] ------------------------------------------------------------------------

[INFO] Reactor Summary:

[INFO] 

[INFO] Spark Project Parent POM ........................... SUCCESS [  5.356 s]

[INFO] Spark Project Tags ................................. SUCCESS [ 19.086 s]

[INFO] Spark Project Sketch ............................... SUCCESS [  6.654 s]

[INFO] Spark Project Networking ........................... SUCCESS [ 13.416 s]

[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [  9.109 s]

[INFO] Spark Project Unsafe ............................... SUCCESS [ 11.544 s]

[INFO] Spark Project Launcher ............................. FAILURE [  0.034 s]

[INFO] Spark Project Core ................................. SKIPPED

[INFO] Spark Project ML Local Library ..................... SKIPPED

[INFO] Spark Project GraphX ............................... SKIPPED

[INFO] Spark Project Streaming ............................ SKIPPED

[INFO] Spark Project Catalyst ............................. SKIPPED

[INFO] Spark Project SQL .................................. SKIPPED

[INFO] Spark Project ML Library ........................... SKIPPED

[INFO] Spark Project Tools ................................ SKIPPED

[INFO] Spark Project Hive ................................. SKIPPED

[INFO] Spark Project REPL ................................. SKIPPED

[INFO] Spark Project YARN Shuffle Service ................. SKIPPED

[INFO] Spark Project YARN ................................. SKIPPED

[INFO] Spark Project Hive Thrift Server ................... SKIPPED

[INFO] Spark Project Assembly ............................. SKIPPED

[INFO] Spark Project External Flume Sink .................. SKIPPED

[INFO] Spark Project External Flume ....................... SKIPPED

[INFO] Spark Project External Flume Assembly .............. SKIPPED

[INFO] Spark Integration for Kafka 0.8 .................... SKIPPED

[INFO] Spark Project Examples ............................. SKIPPED

[INFO] Spark Project External Kafka Assembly .............. SKIPPED

[INFO] Spark Integration for Kafka 0.10 ................... SKIPPED

[INFO] Spark Integration for Kafka 0.10 Assembly .......... SKIPPED

[INFO] Kafka 0.10 Source for Structured Streaming ......... SKIPPED

[INFO] Spark Project Java 8 Tests ......................... SKIPPED

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 01:06 min

[INFO] Finished at: 2018-06-03T17:45:23+08:00

[INFO] Final Memory: 46M/365M

[INFO] ------------------------------------------------------------------------

[ERROR] Failed to execute goal on project spark-launcher_2.11: Could not resolve dependencies for project org.apache.spark:spark-launcher_2.11:jar:2.1.0: Failure to find org.apache.hadoop:hadoop-client:jar:2.6.0-cdh5.7.0 in https://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]

[ERROR] 

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR] 

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

[ERROR] 

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR]   mvn <goals> -rf :spark-launcher_2.11

以上spark编译没有成功，报错了，是什么原因呢，谢谢

Answer 1

Michael_PK 2018-06-03 23:11:41

按照视频里面来，你这需要在pom中添加cdh的仓库

• 提问者 慕神816625 #1

  老师，您好，非常感谢，后来找到了，谢谢老师

Answer 2

提问者 慕神816625 2018-06-03 18:03:25

执行mvn的时候，是用自带的mvn命令，还是用apache-maven-3.5.3或者apache-maven-3.3.9再配置好环境变量后，即，$MAVEN_HOME后再执行mvn呢，非常感谢老师