Move QJM-related backports into 2.0.3 release section in CHANGES.txt after backport to branch-2
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1417601 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Todd Lipcon
parent
6cc49f1a8b
commit
d6c50b4a67
@ -92,17 +92,12 @@ Trunk (Unreleased)
|
|||||||
HDFS-3040. TestMulitipleNNDataBlockScanner is misspelled. (Madhukara Phatak
|
HDFS-3040. TestMulitipleNNDataBlockScanner is misspelled. (Madhukara Phatak
|
||||||
via atm)
|
via atm)
|
||||||
|
|
||||||
HDFS-3049. During the normal NN startup process, fall back on a different
|
|
||||||
edit log if we see one that is corrupt (Colin Patrick McCabe via todd)
|
|
||||||
|
|
||||||
HDFS-3478. Test quotas with Long.Max_Value. (Sujay Rau via eli)
|
HDFS-3478. Test quotas with Long.Max_Value. (Sujay Rau via eli)
|
||||||
|
|
||||||
HDFS-3498. Support replica removal in BlockPlacementPolicy and make
|
HDFS-3498. Support replica removal in BlockPlacementPolicy and make
|
||||||
BlockPlacementPolicyDefault extensible for reusing code in subclasses.
|
BlockPlacementPolicyDefault extensible for reusing code in subclasses.
|
||||||
(Junping Du via szetszwo)
|
(Junping Du via szetszwo)
|
||||||
|
|
||||||
HDFS-3571. Allow EditLogFileInputStream to read from a remote URL (todd)
|
|
||||||
|
|
||||||
HDFS-3510. Editlog pre-allocation is performed prior to writing edits
|
HDFS-3510. Editlog pre-allocation is performed prior to writing edits
|
||||||
to avoid partial edits case disk out of space.(Colin McCabe via suresh)
|
to avoid partial edits case disk out of space.(Colin McCabe via suresh)
|
||||||
|
|
||||||
@ -146,8 +141,6 @@ Trunk (Unreleased)
|
|||||||
HDFS-4052. BlockManager#invalidateWork should print log outside the lock.
|
HDFS-4052. BlockManager#invalidateWork should print log outside the lock.
|
||||||
(Jing Zhao via suresh)
|
(Jing Zhao via suresh)
|
||||||
|
|
||||||
HDFS-4110. Refine a log printed in JNStorage. (Liang Xie via suresh)
|
|
||||||
|
|
||||||
HDFS-4124. Refactor INodeDirectory#getExistingPathINodes() to enable
|
HDFS-4124. Refactor INodeDirectory#getExistingPathINodes() to enable
|
||||||
returning more than INode array. (Jing Zhao via suresh)
|
returning more than INode array. (Jing Zhao via suresh)
|
||||||
|
|
||||||
@ -160,10 +153,6 @@ Trunk (Unreleased)
|
|||||||
HDFS-4152. Add a new class BlocksMapUpdateInfo for the parameter in
|
HDFS-4152. Add a new class BlocksMapUpdateInfo for the parameter in
|
||||||
INode.collectSubtreeBlocksAndClear(..). (Jing Zhao via szetszwo)
|
INode.collectSubtreeBlocksAndClear(..). (Jing Zhao via szetszwo)
|
||||||
|
|
||||||
HDFS-4153. Add START_MSG/SHUTDOWN_MSG for JournalNode. (liang xie via atm)
|
|
||||||
|
|
||||||
HDFS-3935. Add JournalNode to the start/stop scripts (Andy Isaacson via todd)
|
|
||||||
|
|
||||||
HDFS-4206. Change the fields in INode and its subclasses to private.
|
HDFS-4206. Change the fields in INode and its subclasses to private.
|
||||||
(szetszwo)
|
(szetszwo)
|
||||||
|
|
||||||
@ -277,108 +266,6 @@ Trunk (Unreleased)
|
|||||||
the nodes in the same nodegroup should also be excluded. (Junping Du
|
the nodes in the same nodegroup should also be excluded. (Junping Du
|
||||||
via szetszwo)
|
via szetszwo)
|
||||||
|
|
||||||
BREAKDOWN OF HDFS-3077 SUBTASKS
|
|
||||||
|
|
||||||
HDFS-3077. Quorum-based protocol for reading and writing edit logs.
|
|
||||||
(todd, Brandon Li, and Hari Mankude via todd)
|
|
||||||
|
|
||||||
HDFS-3694. Fix getEditLogManifest to fetch httpPort if necessary (todd)
|
|
||||||
|
|
||||||
HDFS-3692. Support purgeEditLogs() call to remotely purge logs on JNs
|
|
||||||
(todd)
|
|
||||||
|
|
||||||
HDFS-3693. JNStorage should read its storage info even before a writer
|
|
||||||
becomes active (todd)
|
|
||||||
|
|
||||||
HDFS-3725. Fix QJM startup when individual JNs have gaps (todd)
|
|
||||||
|
|
||||||
HDFS-3741. Exhaustive failure injection test for skipped RPCs (todd)
|
|
||||||
|
|
||||||
HDFS-3773. TestNNWithQJM fails after HDFS-3741. (atm)
|
|
||||||
|
|
||||||
HDFS-3793. Implement genericized format() in QJM (todd)
|
|
||||||
|
|
||||||
HDFS-3795. QJM: validate journal dir at startup (todd)
|
|
||||||
|
|
||||||
HDFS-3798. Avoid throwing NPE when finalizeSegment() is called on invalid
|
|
||||||
segment (todd)
|
|
||||||
|
|
||||||
HDFS-3799. QJM: handle empty log segments during recovery (todd)
|
|
||||||
|
|
||||||
HDFS-3797. QJM: add segment txid as a parameter to journal() RPC (todd)
|
|
||||||
|
|
||||||
HDFS-3800. improvements to QJM fault testing (todd)
|
|
||||||
|
|
||||||
HDFS-3823. QJM: TestQJMWithFaults fails occasionally because of missed
|
|
||||||
setting of HTTP port. (todd and atm)
|
|
||||||
|
|
||||||
HDFS-3826. QJM: Some trivial logging / exception text improvements. (todd
|
|
||||||
and atm)
|
|
||||||
|
|
||||||
HDFS-3839. QJM: hadoop-daemon.sh should be updated to accept "journalnode"
|
|
||||||
(eli)
|
|
||||||
|
|
||||||
HDFS-3845. Fixes for edge cases in QJM recovery protocol (todd)
|
|
||||||
|
|
||||||
HDFS-3877. QJM: Provide defaults for dfs.journalnode.*address (eli)
|
|
||||||
|
|
||||||
HDFS-3863. Track last "committed" txid in QJM (todd)
|
|
||||||
|
|
||||||
HDFS-3869. Expose non-file journal manager details in web UI (todd)
|
|
||||||
|
|
||||||
HDFS-3884. Journal format() should reset cached values (todd)
|
|
||||||
|
|
||||||
HDFS-3870. Add metrics to JournalNode (todd)
|
|
||||||
|
|
||||||
HDFS-3891. Make selectInputStreams throw IOE instead of RTE (todd)
|
|
||||||
|
|
||||||
HDFS-3726. If a logger misses an RPC, don't retry that logger until next
|
|
||||||
segment (todd)
|
|
||||||
|
|
||||||
HDFS-3893. QJM: Make QJM work with security enabled. (atm)
|
|
||||||
|
|
||||||
HDFS-3897. QJM: TestBlockToken fails after HDFS-3893. (atm)
|
|
||||||
|
|
||||||
HDFS-3898. QJM: enable TCP_NODELAY for IPC (todd)
|
|
||||||
|
|
||||||
HDFS-3885. QJM: optimize log sync when JN is lagging behind (todd)
|
|
||||||
|
|
||||||
HDFS-3900. QJM: avoid validating log segments on log rolls (todd)
|
|
||||||
|
|
||||||
HDFS-3901. QJM: send 'heartbeat' messages to JNs even when they are
|
|
||||||
out-of-sync (todd)
|
|
||||||
|
|
||||||
HDFS-3899. QJM: Add client-side metrics (todd)
|
|
||||||
|
|
||||||
HDFS-3914. QJM: acceptRecovery should abort current segment (todd)
|
|
||||||
|
|
||||||
HDFS-3915. QJM: Failover fails with auth error in secure cluster (todd)
|
|
||||||
|
|
||||||
HDFS-3906. QJM: quorum timeout on failover with large log segment (todd)
|
|
||||||
|
|
||||||
HDFS-3840. JournalNodes log JournalNotFormattedException backtrace error
|
|
||||||
before being formatted (todd)
|
|
||||||
|
|
||||||
HDFS-3894. QJM: testRecoverAfterDoubleFailures can be flaky due to IPC
|
|
||||||
client caching (todd)
|
|
||||||
|
|
||||||
HDFS-3926. QJM: Add user documentation for QJM. (atm)
|
|
||||||
|
|
||||||
HDFS-3943. QJM: remove currently-unused md5sum field (todd)
|
|
||||||
|
|
||||||
HDFS-3950. QJM: misc TODO cleanup, improved log messages, etc. (todd)
|
|
||||||
|
|
||||||
HDFS-3955. QJM: Make acceptRecovery() atomic. (todd)
|
|
||||||
|
|
||||||
HDFS-3956. QJM: purge temporary files when no longer within retention
|
|
||||||
period (todd)
|
|
||||||
|
|
||||||
HDFS-4004. TestJournalNode#testJournal fails because of test case execution
|
|
||||||
order (Chao Shi via todd)
|
|
||||||
|
|
||||||
HDFS-4017. Unclosed FileInputStream in GetJournalEditServlet
|
|
||||||
(Chao Shi via todd)
|
|
||||||
|
|
||||||
Release 2.0.3-alpha - Unreleased
|
Release 2.0.3-alpha - Unreleased
|
||||||
|
|
||||||
INCOMPATIBLE CHANGES
|
INCOMPATIBLE CHANGES
|
||||||
@ -504,6 +391,17 @@ Release 2.0.3-alpha - Unreleased
|
|||||||
|
|
||||||
HDFS-4199. Provide test for HdfsVolumeId. (Ivan A. Veselovsky via atm)
|
HDFS-4199. Provide test for HdfsVolumeId. (Ivan A. Veselovsky via atm)
|
||||||
|
|
||||||
|
HDFS-3049. During the normal NN startup process, fall back on a different
|
||||||
|
edit log if we see one that is corrupt (Colin Patrick McCabe via todd)
|
||||||
|
|
||||||
|
HDFS-3571. Allow EditLogFileInputStream to read from a remote URL (todd)
|
||||||
|
|
||||||
|
HDFS-4110. Refine a log printed in JNStorage. (Liang Xie via suresh)
|
||||||
|
|
||||||
|
HDFS-4153. Add START_MSG/SHUTDOWN_MSG for JournalNode. (liang xie via atm)
|
||||||
|
|
||||||
|
HDFS-3935. Add JournalNode to the start/stop scripts (Andy Isaacson via todd)
|
||||||
|
|
||||||
OPTIMIZATIONS
|
OPTIMIZATIONS
|
||||||
|
|
||||||
BUG FIXES
|
BUG FIXES
|
||||||
@ -663,6 +561,109 @@ Release 2.0.3-alpha - Unreleased
|
|||||||
children of the child have to be updated to the new child. (Jing Zhao
|
children of the child have to be updated to the new child. (Jing Zhao
|
||||||
via szetszwo)
|
via szetszwo)
|
||||||
|
|
||||||
|
BREAKDOWN OF HDFS-3077 SUBTASKS
|
||||||
|
|
||||||
|
HDFS-3077. Quorum-based protocol for reading and writing edit logs.
|
||||||
|
(todd, Brandon Li, and Hari Mankude via todd)
|
||||||
|
|
||||||
|
HDFS-3694. Fix getEditLogManifest to fetch httpPort if necessary (todd)
|
||||||
|
|
||||||
|
HDFS-3692. Support purgeEditLogs() call to remotely purge logs on JNs
|
||||||
|
(todd)
|
||||||
|
|
||||||
|
HDFS-3693. JNStorage should read its storage info even before a writer
|
||||||
|
becomes active (todd)
|
||||||
|
|
||||||
|
HDFS-3725. Fix QJM startup when individual JNs have gaps (todd)
|
||||||
|
|
||||||
|
HDFS-3741. Exhaustive failure injection test for skipped RPCs (todd)
|
||||||
|
|
||||||
|
HDFS-3773. TestNNWithQJM fails after HDFS-3741. (atm)
|
||||||
|
|
||||||
|
HDFS-3793. Implement genericized format() in QJM (todd)
|
||||||
|
|
||||||
|
HDFS-3795. QJM: validate journal dir at startup (todd)
|
||||||
|
|
||||||
|
HDFS-3798. Avoid throwing NPE when finalizeSegment() is called on invalid
|
||||||
|
segment (todd)
|
||||||
|
|
||||||
|
HDFS-3799. QJM: handle empty log segments during recovery (todd)
|
||||||
|
|
||||||
|
HDFS-3797. QJM: add segment txid as a parameter to journal() RPC (todd)
|
||||||
|
|
||||||
|
HDFS-3800. improvements to QJM fault testing (todd)
|
||||||
|
|
||||||
|
HDFS-3823. QJM: TestQJMWithFaults fails occasionally because of missed
|
||||||
|
setting of HTTP port. (todd and atm)
|
||||||
|
|
||||||
|
HDFS-3826. QJM: Some trivial logging / exception text improvements. (todd
|
||||||
|
and atm)
|
||||||
|
|
||||||
|
HDFS-3839. QJM: hadoop-daemon.sh should be updated to accept "journalnode"
|
||||||
|
(eli)
|
||||||
|
|
||||||
|
HDFS-3845. Fixes for edge cases in QJM recovery protocol (todd)
|
||||||
|
|
||||||
|
HDFS-3877. QJM: Provide defaults for dfs.journalnode.*address (eli)
|
||||||
|
|
||||||
|
HDFS-3863. Track last "committed" txid in QJM (todd)
|
||||||
|
|
||||||
|
HDFS-3869. Expose non-file journal manager details in web UI (todd)
|
||||||
|
|
||||||
|
HDFS-3884. Journal format() should reset cached values (todd)
|
||||||
|
|
||||||
|
HDFS-3870. Add metrics to JournalNode (todd)
|
||||||
|
|
||||||
|
HDFS-3891. Make selectInputStreams throw IOE instead of RTE (todd)
|
||||||
|
|
||||||
|
HDFS-3726. If a logger misses an RPC, don't retry that logger until next
|
||||||
|
segment (todd)
|
||||||
|
|
||||||
|
HDFS-3893. QJM: Make QJM work with security enabled. (atm)
|
||||||
|
|
||||||
|
HDFS-3897. QJM: TestBlockToken fails after HDFS-3893. (atm)
|
||||||
|
|
||||||
|
HDFS-3898. QJM: enable TCP_NODELAY for IPC (todd)
|
||||||
|
|
||||||
|
HDFS-3885. QJM: optimize log sync when JN is lagging behind (todd)
|
||||||
|
|
||||||
|
HDFS-3900. QJM: avoid validating log segments on log rolls (todd)
|
||||||
|
|
||||||
|
HDFS-3901. QJM: send 'heartbeat' messages to JNs even when they are
|
||||||
|
out-of-sync (todd)
|
||||||
|
|
||||||
|
HDFS-3899. QJM: Add client-side metrics (todd)
|
||||||
|
|
||||||
|
HDFS-3914. QJM: acceptRecovery should abort current segment (todd)
|
||||||
|
|
||||||
|
HDFS-3915. QJM: Failover fails with auth error in secure cluster (todd)
|
||||||
|
|
||||||
|
HDFS-3906. QJM: quorum timeout on failover with large log segment (todd)
|
||||||
|
|
||||||
|
HDFS-3840. JournalNodes log JournalNotFormattedException backtrace error
|
||||||
|
before being formatted (todd)
|
||||||
|
|
||||||
|
HDFS-3894. QJM: testRecoverAfterDoubleFailures can be flaky due to IPC
|
||||||
|
client caching (todd)
|
||||||
|
|
||||||
|
HDFS-3926. QJM: Add user documentation for QJM. (atm)
|
||||||
|
|
||||||
|
HDFS-3943. QJM: remove currently-unused md5sum field (todd)
|
||||||
|
|
||||||
|
HDFS-3950. QJM: misc TODO cleanup, improved log messages, etc. (todd)
|
||||||
|
|
||||||
|
HDFS-3955. QJM: Make acceptRecovery() atomic. (todd)
|
||||||
|
|
||||||
|
HDFS-3956. QJM: purge temporary files when no longer within retention
|
||||||
|
period (todd)
|
||||||
|
|
||||||
|
HDFS-4004. TestJournalNode#testJournal fails because of test case execution
|
||||||
|
order (Chao Shi via todd)
|
||||||
|
|
||||||
|
HDFS-4017. Unclosed FileInputStream in GetJournalEditServlet
|
||||||
|
(Chao Shi via todd)
|
||||||
|
|
||||||
|
|
||||||
Release 2.0.2-alpha - 2012-09-07
|
Release 2.0.2-alpha - 2012-09-07
|
||||||
|
|
||||||
INCOMPATIBLE CHANGES
|
INCOMPATIBLE CHANGES
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user