SSSApp提交spark-submit在yarn运行时Resetting offset问题

Question

老师，我提交spark-submit运行后，看到日志这样显示

22/01/14 11:09:41 INFO internals.SubscriptionState: [Consumer clientId=consumer-spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0-1, groupId=spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0] Seeking to LATEST offset of partition access-topic-prod-0
22/01/14 11:09:41 INFO internals.SubscriptionState: [Consumer clientId=consumer-spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0-1, groupId=spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0] Resetting offset for partition access-topic-prod-0 to offset 133079.
22/01/14 11:09:41 INFO internals.SubscriptionState: [Consumer clientId=consumer-spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0-1, groupId=spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0] Seeking to LATEST offset of partition access-topic-prod-0
22/01/14 11:09:41 INFO internals.SubscriptionState: [Consumer clientId=consumer-spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0-1, groupId=spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0] Resetting offset for partition access-topic-prod-0 to offset 133079.
22/01/14 11:09:41 INFO internals.SubscriptionState: [Consumer clientId=consumer-spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0-1, groupId=spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0] Seeking to LATEST offset of partition access-topic-prod-0
22/01/14 11:09:41 INFO internals.SubscriptionState: [Consumer clientId=consumer-spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0-1, groupId=spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0] Resetting offset for partition access-topic-prod-0 to offset 133079.
22/01/14 11:09:41 INFO internals.SubscriptionState: [Consumer clientId=consumer-spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0-1, groupId=spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0] Seeking to LATEST offset of partition access-topic-prod-0
22/01/14 11:09:41 INFO internals.SubscriptionState: [Consumer clientId=consumer-spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0-1, groupId=spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0] Resetting offset for partition access-topic-prod-0 to offset 133079.
22/01/14 11:09:41 INFO internals.SubscriptionState: [Consumer clientId=consumer-spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0-1, groupId=spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0] Seeking to LATEST offset of partition access-topic-prod-0
22/01/14 11:09:41 INFO internals.SubscriptionState: [Consumer clientId=consumer-spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0-1, groupId=spark-kafka-source-f1e3a0f1-c7cb-498a-aac0-0e173529602e-1691655236-driver-0] Resetting offset for partition access-topic-prod-0 to offset 133079.

Answer 1

提问者 夜愿小夜 2022-01-14 12:52:57

后来发现是正常日志输出，程序运行正确！

• Michael_PK #1

  这正常的，要学会看日志，报错的一般都是error级别的，这些都是info的

• 提问者 夜愿小夜 回复 Michael_PK #2

  谢谢老师！