big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Move QJM-related backports into 2.0.3 release section in CHANGES.txt after backport to branch-2

Browse Source 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1417601 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Todd Lipcon 2012-12-05 19:33:05 +00:00
parent 6cc49f1a8b
commit d6c50b4a67
1 changed files with 114 additions and 113 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

227
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
View File

@ -92,17 +92,12 @@ Trunk (Unreleased)
HDFS-3040. TestMulitipleNNDataBlockScanner is misspelled. (Madhukara Phatak HDFS-3040. TestMulitipleNNDataBlockScanner is misspelled. (Madhukara Phatak
via atm) via atm)
HDFS-3049. During the normal NN startup process, fall back on a different
edit log if we see one that is corrupt (Colin Patrick McCabe via todd)
HDFS-3478. Test quotas with Long.Max_Value. (Sujay Rau via eli) HDFS-3478. Test quotas with Long.Max_Value. (Sujay Rau via eli)
HDFS-3498. Support replica removal in BlockPlacementPolicy and make HDFS-3498. Support replica removal in BlockPlacementPolicy and make
BlockPlacementPolicyDefault extensible for reusing code in subclasses. BlockPlacementPolicyDefault extensible for reusing code in subclasses.
(Junping Du via szetszwo) (Junping Du via szetszwo)
HDFS-3571. Allow EditLogFileInputStream to read from a remote URL (todd)
HDFS-3510. Editlog pre-allocation is performed prior to writing edits HDFS-3510. Editlog pre-allocation is performed prior to writing edits
to avoid partial edits case disk out of space.(Colin McCabe via suresh) to avoid partial edits case disk out of space.(Colin McCabe via suresh)
@ -146,8 +141,6 @@ Trunk (Unreleased)
HDFS-4052. BlockManager#invalidateWork should print log outside the lock. HDFS-4052. BlockManager#invalidateWork should print log outside the lock.
(Jing Zhao via suresh) (Jing Zhao via suresh)
HDFS-4110. Refine a log printed in JNStorage. (Liang Xie via suresh)
HDFS-4124. Refactor INodeDirectory#getExistingPathINodes() to enable HDFS-4124. Refactor INodeDirectory#getExistingPathINodes() to enable
returning more than INode array. (Jing Zhao via suresh) returning more than INode array. (Jing Zhao via suresh)
@ -160,10 +153,6 @@ Trunk (Unreleased)
HDFS-4152. Add a new class BlocksMapUpdateInfo for the parameter in HDFS-4152. Add a new class BlocksMapUpdateInfo for the parameter in
INode.collectSubtreeBlocksAndClear(..). (Jing Zhao via szetszwo) INode.collectSubtreeBlocksAndClear(..). (Jing Zhao via szetszwo)
HDFS-4153. Add START_MSG/SHUTDOWN_MSG for JournalNode. (liang xie via atm)
HDFS-3935. Add JournalNode to the start/stop scripts (Andy Isaacson via todd)
HDFS-4206. Change the fields in INode and its subclasses to private. HDFS-4206. Change the fields in INode and its subclasses to private.
(szetszwo) (szetszwo)
@ -277,108 +266,6 @@ Trunk (Unreleased)
the nodes in the same nodegroup should also be excluded. (Junping Du the nodes in the same nodegroup should also be excluded. (Junping Du
via szetszwo) via szetszwo)
BREAKDOWN OF HDFS-3077 SUBTASKS
HDFS-3077. Quorum-based protocol for reading and writing edit logs.
(todd, Brandon Li, and Hari Mankude via todd)
HDFS-3694. Fix getEditLogManifest to fetch httpPort if necessary (todd)
HDFS-3692. Support purgeEditLogs() call to remotely purge logs on JNs
(todd)
HDFS-3693. JNStorage should read its storage info even before a writer
becomes active (todd)
HDFS-3725. Fix QJM startup when individual JNs have gaps (todd)
HDFS-3741. Exhaustive failure injection test for skipped RPCs (todd)
HDFS-3773. TestNNWithQJM fails after HDFS-3741. (atm)
HDFS-3793. Implement genericized format() in QJM (todd)
HDFS-3795. QJM: validate journal dir at startup (todd)
HDFS-3798. Avoid throwing NPE when finalizeSegment() is called on invalid
segment (todd)
HDFS-3799. QJM: handle empty log segments during recovery (todd)
HDFS-3797. QJM: add segment txid as a parameter to journal() RPC (todd)
HDFS-3800. improvements to QJM fault testing (todd)
HDFS-3823. QJM: TestQJMWithFaults fails occasionally because of missed
setting of HTTP port. (todd and atm)
HDFS-3826. QJM: Some trivial logging / exception text improvements. (todd
and atm)
HDFS-3839. QJM: hadoop-daemon.sh should be updated to accept "journalnode"
(eli)
HDFS-3845. Fixes for edge cases in QJM recovery protocol (todd)
HDFS-3877. QJM: Provide defaults for dfs.journalnode.*address (eli)
HDFS-3863. Track last "committed" txid in QJM (todd)
HDFS-3869. Expose non-file journal manager details in web UI (todd)
HDFS-3884. Journal format() should reset cached values (todd)
HDFS-3870. Add metrics to JournalNode (todd)
HDFS-3891. Make selectInputStreams throw IOE instead of RTE (todd)
HDFS-3726. If a logger misses an RPC, don't retry that logger until next
segment (todd)
HDFS-3893. QJM: Make QJM work with security enabled. (atm)
HDFS-3897. QJM: TestBlockToken fails after HDFS-3893. (atm)
HDFS-3898. QJM: enable TCP_NODELAY for IPC (todd)
HDFS-3885. QJM: optimize log sync when JN is lagging behind (todd)
HDFS-3900. QJM: avoid validating log segments on log rolls (todd)
HDFS-3901. QJM: send 'heartbeat' messages to JNs even when they are
out-of-sync (todd)
HDFS-3899. QJM: Add client-side metrics (todd)
HDFS-3914. QJM: acceptRecovery should abort current segment (todd)
HDFS-3915. QJM: Failover fails with auth error in secure cluster (todd)
HDFS-3906. QJM: quorum timeout on failover with large log segment (todd)
HDFS-3840. JournalNodes log JournalNotFormattedException backtrace error
before being formatted (todd)
HDFS-3894. QJM: testRecoverAfterDoubleFailures can be flaky due to IPC
client caching (todd)
HDFS-3926. QJM: Add user documentation for QJM. (atm)
HDFS-3943. QJM: remove currently-unused md5sum field (todd)
HDFS-3950. QJM: misc TODO cleanup, improved log messages, etc. (todd)
HDFS-3955. QJM: Make acceptRecovery() atomic. (todd)
HDFS-3956. QJM: purge temporary files when no longer within retention
period (todd)
HDFS-4004. TestJournalNode#testJournal fails because of test case execution
order (Chao Shi via todd)
HDFS-4017. Unclosed FileInputStream in GetJournalEditServlet
(Chao Shi via todd)
Release 2.0.3-alpha - Unreleased Release 2.0.3-alpha - Unreleased
INCOMPATIBLE CHANGES INCOMPATIBLE CHANGES
@ -504,6 +391,17 @@ Release 2.0.3-alpha - Unreleased
HDFS-4199. Provide test for HdfsVolumeId. (Ivan A. Veselovsky via atm) HDFS-4199. Provide test for HdfsVolumeId. (Ivan A. Veselovsky via atm)
HDFS-3049. During the normal NN startup process, fall back on a different
edit log if we see one that is corrupt (Colin Patrick McCabe via todd)
HDFS-3571. Allow EditLogFileInputStream to read from a remote URL (todd)
HDFS-4110. Refine a log printed in JNStorage. (Liang Xie via suresh)
HDFS-4153. Add START_MSG/SHUTDOWN_MSG for JournalNode. (liang xie via atm)
HDFS-3935. Add JournalNode to the start/stop scripts (Andy Isaacson via todd)
OPTIMIZATIONS OPTIMIZATIONS
BUG FIXES BUG FIXES
@ -663,6 +561,109 @@ Release 2.0.3-alpha - Unreleased
children of the child have to be updated to the new child. (Jing Zhao children of the child have to be updated to the new child. (Jing Zhao
via szetszwo) via szetszwo)
BREAKDOWN OF HDFS-3077 SUBTASKS
HDFS-3077. Quorum-based protocol for reading and writing edit logs.
(todd, Brandon Li, and Hari Mankude via todd)
HDFS-3694. Fix getEditLogManifest to fetch httpPort if necessary (todd)
HDFS-3692. Support purgeEditLogs() call to remotely purge logs on JNs
(todd)
HDFS-3693. JNStorage should read its storage info even before a writer
becomes active (todd)
HDFS-3725. Fix QJM startup when individual JNs have gaps (todd)
HDFS-3741. Exhaustive failure injection test for skipped RPCs (todd)
HDFS-3773. TestNNWithQJM fails after HDFS-3741. (atm)
HDFS-3793. Implement genericized format() in QJM (todd)
HDFS-3795. QJM: validate journal dir at startup (todd)
HDFS-3798. Avoid throwing NPE when finalizeSegment() is called on invalid
segment (todd)
HDFS-3799. QJM: handle empty log segments during recovery (todd)
HDFS-3797. QJM: add segment txid as a parameter to journal() RPC (todd)
HDFS-3800. improvements to QJM fault testing (todd)
HDFS-3823. QJM: TestQJMWithFaults fails occasionally because of missed
setting of HTTP port. (todd and atm)
HDFS-3826. QJM: Some trivial logging / exception text improvements. (todd
and atm)
HDFS-3839. QJM: hadoop-daemon.sh should be updated to accept "journalnode"
(eli)
HDFS-3845. Fixes for edge cases in QJM recovery protocol (todd)
HDFS-3877. QJM: Provide defaults for dfs.journalnode.*address (eli)
HDFS-3863. Track last "committed" txid in QJM (todd)
HDFS-3869. Expose non-file journal manager details in web UI (todd)
HDFS-3884. Journal format() should reset cached values (todd)
HDFS-3870. Add metrics to JournalNode (todd)
HDFS-3891. Make selectInputStreams throw IOE instead of RTE (todd)
HDFS-3726. If a logger misses an RPC, don't retry that logger until next
segment (todd)
HDFS-3893. QJM: Make QJM work with security enabled. (atm)
HDFS-3897. QJM: TestBlockToken fails after HDFS-3893. (atm)
HDFS-3898. QJM: enable TCP_NODELAY for IPC (todd)
HDFS-3885. QJM: optimize log sync when JN is lagging behind (todd)
HDFS-3900. QJM: avoid validating log segments on log rolls (todd)
HDFS-3901. QJM: send 'heartbeat' messages to JNs even when they are
out-of-sync (todd)
HDFS-3899. QJM: Add client-side metrics (todd)
HDFS-3914. QJM: acceptRecovery should abort current segment (todd)
HDFS-3915. QJM: Failover fails with auth error in secure cluster (todd)
HDFS-3906. QJM: quorum timeout on failover with large log segment (todd)
HDFS-3840. JournalNodes log JournalNotFormattedException backtrace error
before being formatted (todd)
HDFS-3894. QJM: testRecoverAfterDoubleFailures can be flaky due to IPC
client caching (todd)
HDFS-3926. QJM: Add user documentation for QJM. (atm)
HDFS-3943. QJM: remove currently-unused md5sum field (todd)
HDFS-3950. QJM: misc TODO cleanup, improved log messages, etc. (todd)
HDFS-3955. QJM: Make acceptRecovery() atomic. (todd)
HDFS-3956. QJM: purge temporary files when no longer within retention
period (todd)
HDFS-4004. TestJournalNode#testJournal fails because of test case execution
order (Chao Shi via todd)
HDFS-4017. Unclosed FileInputStream in GetJournalEditServlet
(Chao Shi via todd)
Release 2.0.2-alpha - 2012-09-07 Release 2.0.2-alpha - 2012-09-07
INCOMPATIBLE CHANGES INCOMPATIBLE CHANGES