Apache Sparkに関する非互換を以下に示します。
内容 | 詳細 | JIRA | Summary |
|---|---|---|---|
コマンド仕様の変更 | 実行結果 | SPARK-25250 | Race condition with tasks running when new attempt for same stage is created leads to other task in the next attempt running on the same partition id retry multiple times |
SPARK-27419 | When setting spark.executor.heartbeatInterval to a value less than 1 seconds, it will always fail | ||
SPARK-27494 | Null keys/values don't work in Kafka source v2 | ||
SPARK-28152 | Add a legacy conf for old MsSqlServerDialect numeric mapping | ||
SPARK-28699 | Cache an indeterminate RDD could lead to incorrect result while stage rerun | ||
SPARK-29651 | Fix parsing of interval seconds fraction | ||
SPARK-29890 | Fix DataFrameNaFunctions.fill to handle duplicate columns | ||
SPARK-29918 | RecordBinaryComparator should check endianness when compared by long | ||
SPARK-30065 | Fix DataFrameNaFunctions.drop to handle duplicate columns | ||
例外処理 | 例外追加 | SPARK-28015 | Check stringToDate() consumes entire input for the yyyy and yyyy-[m]m formats |
ファイルのパーミッションの変更 | 権限変更 | SPARK-30312 | Preserve path permission and acl when truncate table |
参照
詳細は、下記のサイトを参照してください。
