Apache Kafkaに関する非互換を以下に示します。
内容 | 詳細 | JIRA | Summary |
|---|---|---|---|
コマンド仕様の変更 | 実行結果 | HADOOP-15922 | DelegationTokenAuthenticationFilter get wrong doAsUser since it does not decode URL |
HADOOP-16255 | ChecksumFS.Make FileSystem.rename(path, path, options) doesn’t rename checksum | ||
HDFS-13386 | RBF: Wrong date information in list file(-ls) result | ||
HDFS-13854 | RBF: The ProcessingAvgTime and ProxyAvgTime should display by JMX with ms unit. | ||
HDFS-14166 | Ls with -e option not giving the result in proper format | ||
HDFS-14389 | getAclStatus returns incorrect permissions and owner when an iNodeAttributeProvider is configured | ||
YARN-8145 | yarn rmadmin -getGroups doesn’t return updated groups for user | ||
YARN-8791 | When STOPSIGNAL is not present then docker inspect returns an extra line feed | ||
YARN-8987 | Usability improvements node-attributes CLI | ||
YARN-9099 | GpuResourceAllocator#getReleasingGpus calculates number of GPUs in a wrong way | ||
例外処理修正 | HDFS-12981 | renameSnapshot a Non-Existent snapshot to itself should throw error | |
HDFS-13292 | Crypto command should give proper exception when trying to set key on existing EZ directory | ||
例外処理追加 | HADOOP-14841 | Kms client should disconnect if unable to get output stream from connection. | |
HADOOP-15390 | Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens | ||
HADOOP-15625 | S3A input stream to use etags/version number to detect changed source files | ||
HADOOP-15809 | ABFS: better exception handling when making getAccessToken call | ||
HADOOP-15846 | ABFS: fix mask related bugs in setAcl, modifyAclEntries and removeAclEntries. | ||
HDFS-13758 | DatanodeManager should throw exception if it has BlockRecoveryCommand but the block is not under construction | ||
HDFS-13806 | EC: No error message for unsetting EC policy of the directory inherits the erasure coding policy from an ancestor directory | ||
HDFS-14459 | ClosedChannelException silently ignored in FsVolumeList.addBlockPool() | ||
YARN-8107 | Give an informative message when incorrect format is used in ATSv2 filter attributes | ||
YARN-8316 | Diagnostic message should improve when yarn service fails to launch due to ATS unavailability | ||
YARN-9217 | Nodemanager will fail to start if GPU is misconfigured on the node or GPU drivers missing | ||
チェック | チェック強化 | HADOOP-13649 | s3guard: implement time-based (TTL) expiry for LocalMetadataStore |
HADOOP-13738 | DiskChecker should perform some disk IO | ||
HADOOP-15667 | FileSystemMultipartUploader should verify that UploadHandle has non-0 length | ||
HADOOP-15850 | CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0 | ||
HDFS-11239 | [SPS]: Check Mover file ID lease also to determine whether Mover is running | ||
HDFS-11264 | [SPS]: Double checks to ensure that SPS/Mover are not running together | ||
HDFS-13602 | Add checkOperation(WRITE) checks in FSNamesystem | ||
HDFS-13663 | Should throw exception when incorrect block size is set | ||
HDFS-13815 | RBF: Add check to order command | ||
HDFS-13858 | RBF: Add check to have single valid argument to safemode command | ||
HDFS-13977 | NameNode can kill itself if it tries to send too many txns to a QJM simultaneously | ||
HDFS-14706 | Checksums are not checked if block meta file is less than 7 bytes | ||
YARN-7221 | Add security check for privileged docker container | ||
YARN-8202 | DefaultAMSProcessor should properly check units of requested custom resource types against minimum/maximum allocation | ||
YARN-8249 | Few REST api’s in RMWebServices are missing static user check | ||
YARN-8455 | Add basic ACL check for all ATS v2 REST APIs | ||
YARN-8917 | Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource | ||
YARN-9335 | [atsv2] Restrict the number of elements held in timeline collector when backend is unreachable for async calls | ||
YARN-9718 | Yarn REST API, services endpoint remote command ejection | ||
ファイルのパーミッションの変更 | 権限変更 | YARN-9442 | container working directory has group read permissions |
オプション | オプション | YARN-8798 | [Submarine] Job should not be submitted if “-input_path” option is missing |
公開しているファイルの内容／形式 | ログ修正 | HADOOP-14987 | Improve KMSClientProvider log around delegation token checking |
HADOOP-15377 | Improve debug messages in MetricsConfig.java | ||
HADOOP-15382 | Log kinit output in credential renewal thread | ||
HADOOP-15476 | fix logging for split-dns multihome | ||
HADOOP-15586 | Fix wrong log statement in AbstractService | ||
HADOOP-15733 | Correct the log when Invalid emptier Interval configured | ||
HADOOP-15772 | Remove the ‘Path … should be specified as a URI’ warnings on startup | ||
HADOOP-15925 | The config and log of gpg-agent are removed in create-release script | ||
HADOOP-16169 | ABFS: Bug fix for getPathProperties | ||
HDFS-11966 | [SPS] Correct the log in BlockStorageMovementAttemptedItems#blockStorageMovementResultCheck | ||
HDFS-13217 | Audit log all EC policy names during addErasureCodingPolicies | ||
HDFS-13435 | RBF: Improve the error loggings for printing the stack trace | ||
HDFS-13440 | Log HDFS file name when client fails to connect | ||
HDFS-13544 | Improve logging for JournalNode in federated cluster | ||
HDFS-13772 | Erasure coding: Unnecessary NameNode Logs displaying for Enabling/Disabling Erasure coding policies which are already enabled/disabled | ||
HDFS-14451 | Incorrect header or version mismatch log message | ||
YARN-8082 | Include LocalizedResource size information in the NM download log for localization | ||
YARN-8140 | Improve log message when launch cmd is ran for stopped yarn service | ||
YARN-8155 | Improve ATSv2 client logging in RM and NM publisher | ||
YARN-8165 | Incorrect queue name logging in AbstractContainerAllocator | ||
YARN-8185 | Improve log in DirectoryCollection constructor | ||
YARN-8459 | Improve Capacity Scheduler logs to debug invalid states | ||
YARN-8566 | Add diagnostic message for unschedulable containers | ||
YARN-8588 | Logging improvements for better debuggability | ||
YARN-8610 | Yarn Service Upgrade: Typo in Error message | ||
YARN-9087 | Improve logging for initialization of Resource plugins | ||
YARN-9682 | Wrong log message when finalizing the upgrade | ||
YARN-9729 | [UI2] Fix error message for logs when ATSv2 is offline | ||
ログ削除 | HADOOP-15295 | Remove redundant logging related to tags from Configuration | |
YARN-7818 | Remove privileged operation warnings during container launch for the ContainerRuntimes | ||
YARN-8045 | Reduce log output from container status calls | ||
YARN-8210 | AMRMClient logging on every heartbeat to track updation of AM RM token causes too many log lines to be generated in AM logs | ||
YARN-8351 | Node attribute manager logs are flooding RM logs | ||
ログ削除 例外メッセージ追加 | HADOOP-16208 | Do Not Log InterruptedException in Client | |
ログレベル | HADOOP-15441 | Log kms url and token service at debug level. | |
HADOOP-15581 | Set default jetty log level to INFO in KMS | ||
HADOOP-15591 | KMSClientProvider should log KMS DT acquisition at INFO level | ||
HADOOP-16191 | AliyunOSS: improvements for copyFile/copyDirectory and logging | ||
HDFS-13692 | StorageInfoDefragmenter floods log when compacting StorageInfo TreeSet | ||
HDFS-13695 | Move logging to slf4j in HDFS package | ||
HDFS-14231 | DataXceiver#run() should not log exceptions caused by InvalidToken exception as an error | ||
HDFS-14340 | Lower the log level when can’t get postOpAttr | ||
MAPREDUCE-7063 | Fix log level inconsistency in CombineFileInputFormat.java | ||
YARN-8208 | Add log statement for Docker client configuration file at INFO level | ||
YARN-8322 | Change log level when there is an IOException when the allocation file is loaded | ||
YARN-8407 | Container launch exception in AM log should be printed in ERROR level | ||
YARN-8988 | Reduce the verbose log on RM heartbeat path when distributed node-attributes is enabled | ||
ログ頻度変更 | HDFS-13712 | BlockReaderRemote.read() logging improvement | |
ログ追加 | HADOOP-15611 | Log more details for FairCallQueue | |
HADOOP-15662 | ABFS: Better exception handling of DNS errors | ||
HADOOP-15742 | Log if ipc backoff is enabled in CallQueueManager | ||
HADOOP-16269 | ABFS: add listFileStatus with StartFrom | ||
HDFS-13727 | Log full stack trace if DiskBalancer exits with an unhandled exception | ||
HDFS-13862 | RBF: Router logs are not capturing few of the dfsrouteradmin commands | ||
HDFS-13863 | FsDatasetImpl should log DiskOutOfSpaceException | ||
HDFS-13885 | Add debug logs in dfsclient around decrypting EDEK | ||
HDFS-13973 | getErasureCodingPolicy should log path in audit event | ||
HDFS-14462 | WebHDFS throws “Error writing request body to server” instead of DSQuotaExceededException | ||
MAPREDUCE-7191 | JobHistoryServer should log exception when loading/parsing history file failed | ||
YARN-7225 | Add queue and partition info to RM audit log | ||
YARN-8194 | Exception when reinitializing a container using LinuxContainerExecutor | ||
YARN-8199 | Logging fileSize of log files under NM Local Dir | ||
YARN-8273 | Log aggregation does not warn if HDFS quota in target directory is exceeded | ||
YARN-8995 | Log events info in AsyncDispatcher when event queue size cumulatively reaches a certain number every time. | ||
YARN-9030 | Log aggregation changes to handle filesystems which do not support setting permissions | ||
ログ修正 例外追加 | YARN-8415 | TimelineWebServices.getEntity should throw ForbiddenException instead of 404 when ACL checks fail | |
ログ変更 | YARN-8601 | Print ExecutionType in Container report CLI | |
YARN-8910 | Misleading log statement in NM when max retries is -1 | ||
出力追加 | HADOOP-15596 | Stack trace should not be printed out when running hadoop key commands | |
メッセージの修正 | メッセージ修正 | HADOOP-15349 | S3Guard DDB retryBackoff to be more informative on limits exceeded |
HADOOP-15612 | Improve exception when tfile fails to load LzoCodec | ||
HADOOP-15869 | BlockDecompressorStream#decompress should not return -1 in case of IOException. | ||
HADOOP-16019 | ZKDelegationTokenSecretManager won’t log exception message occured in function setJaasConfiguration | ||
HDFS-13045 | RBF: Improve error message returned from subcluster | ||
HDFS-13357 | Improve AclException message “Invalid ACL: only directories may have a default ACL.” | ||
HDFS-13622 | mkdir should print the parent directory in the error message when parent directories do not exist | ||
HDFS-13635 | Incorrect message when block is not found | ||
HDFS-13690 | Improve error message when creating encryption zone while KMS is unreachable | ||
HDFS-14056 | Fix error messages in HDFS-12716 | ||
YARN-7961 | Improve status response when yarn application is destroyed | ||
YARN-8429 | Improve diagnostic message when artifact is not set properly | ||
YARN-8723 | Fix a typo in CS init error message when resource calculator is not correctly set | ||
YARN-8782 | Fix exception message in Resource.throwExceptionWhenArrayOutOfBound | ||
YARN-8785 | Improve the error message when a bind mount is not whitelisted | ||
YARN-8970 | Improve the debug message in CS#allocateContainerOnSingleNode | ||
YARN-9463 | Add queueName info when failing with queue capacity sanity check | ||
YARN-9529 | Log correct cpu controller path on error while initializing CGroups. | ||
メッセージ追加 | HDFS-13363 | Record file path when FSDirAclOp throws AclException | |
YARN-8043 | Add the exception message for failed launches running under LCE | ||
YARN-8129 | Improve error message for invalid value in fields attribute | ||
非推奨機能 | 非対応 削除 | HADOOP-15330 | Remove jdk1.7 profile from hadoop-annotations module |
非推奨 削除 | HADOOP-14833 | Remove s3a user:secret authentication | |
非使用 削除 | HADOOP-15146 | Remove DataOutputByteBuffer | |
YARN-8162 | Remove Method DirectoryCollection#verifyDirUsingMkdir | ||
使用するポートの変更 | デフォルト値の変更 | HDFS-9427 | HDFS should not default to ephemeral ports |
参照
詳細は、下記のサイトを参照してください。
