big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
18,650 Commits 2 Branches 2 Tags 264 MiB
acfae2716a
Commit Graph

5003 Commits

Author SHA1 Message Date
Hanisha Koneru
4eeff62f69 HDFS-13244. Add stack, conf, metrics links to utilities dropdown in NN webUI. Contributed by Bharat Viswanadham. 2018-03-09 15:27:17 -08:00
Hanisha Koneru
9a082fbe6e HDFS-11394. Support for getting erasure coding policy through WebHDFS#FileStatus. 2018-03-09 13:20:32 -08:00
Yiqun Lin
122805b43a HDFS-13233. RBF: MountTableResolver doesn't return the correct mount point of the given path. Contributed by wangzhiyuan. 2018-03-09 15:42:57 +08:00
Inigo Goiri
0c2b969e01 HDFS-13232. RBF: ConnectionManager's cleanup task will compare each pool's own active conns with its total conns. Contributed by Chao Sun. 2018-03-08 09:32:05 -08:00
Sean Mackrory
46d29e3d7e HDFS-13176. WebHdfs file path gets truncated when having semicolon (;) inside. Contributed by Zsolt Venczel. 2018-03-07 13:33:41 -07:00
Tsz-Wo Nicholas Sze
88fba00caa HDFS-13222. Update getBlocks method to take minBlockSize in RPC calls. Contributed by Bharat Viswanadham 2018-03-07 11:27:53 -08:00
Yiqun Lin
58ea2d7a65 HDFS-13214. RBF: Complete document of Router configuration. Contributed by Yiqun Lin. 2018-03-07 15:26:38 +08:00
Tsz-Wo Nicholas Sze
346caa2095 HDFS-13227. Add a method to calculate cumulative diff over multiple snapshots in DirectoryDiffList. Contributed by Shashikant Banerjee 2018-03-06 13:35:03 -08:00
Tsz-Wo Nicholas Sze
871d0d39fa HDFS-13223. Reduce DiffListBySkipList memory usage. Contributed by Shashikant Banerjee 2018-03-06 12:23:03 -08:00
Inigo Goiri
7060725662 HDFS-13188. Disk Balancer: Support multiple block pools during block move. Contributed by Bharat Viswanadham. 2018-03-06 09:09:32 -08:00
Arpit Agarwal
a3c304c5dd HDFS-13178. Disk Balancer: Add a force option to DiskBalancer Execute command. Contributed by Bharat Viswanadham. 2018-03-03 15:42:16 -08:00
Tsz-Wo Nicholas Sze
ba82e5c488 HDFS-13173. Replace ArrayList with DirectoryDiffList(SnapshotSkipList) to store DirectoryDiffs. Contributed by Shashikant Banerjee 2018-03-02 17:47:48 -08:00
Xiao Chen
c75105f07b HDFS-13040. Kerberized inotify client fails despite kinit properly. Contributed by Istvan Fajth, Wei-Chiu Chuang, Xiao Chen. 2018-03-02 16:46:04 -08:00
Tsz-Wo Nicholas Sze
60080fbfcf HDFS-13171. Handle Deletion of nodes in SnasphotSkipList. Contributed by Shashikant Banerjee 2018-03-02 10:51:05 -08:00
Tsz-Wo Nicholas Sze
96e8f260ab HDFS-13211. Fix a bug in DirectoryDiffList.getMinListForRange. Contributed by Shashikant Banerjee 2018-03-01 14:12:15 -08:00
Yiqun Lin
ececeea8f2 HDFS-13202. Fix the outdated javadocs in HAUtil. Contributed by Chao Sun. 2018-03-01 17:51:43 +08:00
Tsz-Wo Nicholas Sze
81d9446a92 HDFS-13102. Implement SnapshotSkipList class to store Multi level DirectoryDiffs. Contributed by Shashikant Banerjee 2018-02-28 13:42:58 -08:00
Xiaoyu Yao
f20e10b2dd HDFS-13081. Datanode#checkSecureConfig should allow SASL and privileged HTTP. Contributed by Ajay Kumar. 2018-02-28 09:44:39 -08:00
Xiaoyu Yao
2574375bf5 HDFS-13114. CryptoAdmin#ReencryptZoneCommand should resolve Namespace info from path. Contributed by Hanisha Koneru. 2018-02-28 08:37:01 -08:00
Yiqun Lin
a9c14b1119 HDFS-13194. CachePool permissions incorrectly checked. Contributed by Jianfei Jiang. 2018-02-28 10:17:43 +08:00
Inigo Goiri
d86f301d46 HDFS-13199. RBF: Fix the hdfs router page missing label icon issue. Contributed by maobaolong. 2018-02-27 17:39:02 -08:00
Arpit Agarwal
1cc9a58dda HDFS-13181. DiskBalancer: Add an configuration for valid plan hours . Contributed by Bharat Viswanadham. 2018-02-27 10:18:07 -08:00
Inigo Goiri
cd4a97bf80 HDFS-13192. Change the code order in getFileEncryptionInfo to avoid unnecessary call of assignment. Contributed by LiXin Ge. 2018-02-27 08:53:00 -08:00
Inigo Goiri
225d877092 HDFS-13193. Various Improvements for BlockTokenSecretManager. Contributed by BELUGA BEHR. 2018-02-27 08:48:03 -08:00
Chao Sun
ae290a4bb4 HDFS-13145. SBN crash when transition to ANN with in-progress edit tailing enabled. Contributed by Chao Sun. 2018-02-26 16:15:00 -08:00
Wei Yan
6ce9f79cc9 HDFS-13187. RBF: Fix Routers information shown in the web UI. Contributed by Inigo Goiri and Wei Yan. 2018-02-26 15:13:41 -08:00
Arpit Agarwal
46c93453e9 HDFS-12781. After Datanode down, In Namenode UI Datanode tab is throwing warning message.. Contributed by Brahma Reddy Battula. 2018-02-26 13:56:34 -08:00
Kihwal Lee
451265a83d HDFS-12070. Failed block recovery leaves files open indefinitely and at risk for data loss. Contributed by Kihwal Lee. 2018-02-26 10:28:04 -06:00
Arpit Agarwal
2fa7963c3d HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi. 2018-02-24 14:41:55 -08:00
Xiaoyu Yao
1e84e46f16 HDFS-13052. WebHDFS: Add support for snasphot diff. Contributed by Lokesh Jain. 2018-02-23 19:35:12 -08:00
Xiaoyu Yao
84a1321f6a HDFS-13136. Avoid taking FSN lock while doing group member lookup for FSD permission check. Contributed by Xiaoyu Yao. 2018-02-22 11:32:32 -08:00
Anu Engineer
121e1e1280 HDFS-13175. Add more information for checking argument in DiskBalancerVolume. 
Contributed by  Lei (Eddy) Xu.
 2018-02-20 19:16:30 -08:00
Inigo Goiri
6f81cc0bee HDFS-13167. DatanodeAdminManager Improvements. Contributed by BELUGA BEHR. 2018-02-20 15:18:27 -08:00
Inigo Goiri
17c592e6cf HDFS-13168. XmlImageVisitor - Prefer Array over LinkedList. Contributed by BELUGA BEHR. 2018-02-20 15:16:01 -08:00
Yiqun Lin
8896d20b91 HDFS-13119. RBF: Manage unavailable clusters. Contributed by Yiqun Lin. 2018-02-20 09:37:08 +08:00
Chen Liang
c008d10a11 HDFS-13161. Update comment in start-dfs.sh to mention correct variable for secure datanode user. Contributed by Ajay Kumar. 2018-02-16 16:04:50 -08:00
Arpit Agarwal
8d5ea7470a HDFS-13153. Enable HDFS diskbalancer by default. Contributed by Ajay Kumar. 2018-02-16 09:38:34 -08:00
Bharat Viswanadham
a1e05e0292
HDFS-13151. Fix the javadoc error in ReplicaInfo 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-02-16 16:22:24 +09:00
Kihwal Lee
47473952e5 HDFS-13112. Token expiration edits may cause log corruption or deadlock. Contributed by Daryn Sharp. 2018-02-15 15:32:42 -06:00
Kihwal Lee
da59acd8ca xattr api cleanup 2018-02-15 11:11:55 -06:00
Tsz-Wo Nicholas Sze
6ea7d78ccb HDFS-13142. Define and Implement a DiifList Interface to store and manage SnapshotDiffs. Contributed by Shashikant Banerjee 2018-02-15 19:33:44 +08:00
Steve Loughran
f20dc0d577 HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions. 
Contributed by Andras Bokor.
 2018-02-14 16:20:14 +00:00
Brahma Reddy Battula
35c17351ca HDFS-8693. Addendum patch to execute the command using UGI. Contributed by Brahma Reddy Battula. 2018-02-12 22:14:34 +05:30
Arpit Agarwal
96bb6a51ec HDFS-10453. ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.. Contributed by He Xiaoqiao. 2018-02-12 07:17:40 -08:00
Yiqun Lin
25fbec67d1 HDFS-13130. Log object instance get incorrectly in SlowDiskTracker. Contributed by Jianfei Jiang. 2018-02-11 12:02:10 +08:00
Yiqun Lin
543f3abbee HDFS-13099. RBF: Use the ZooKeeper as the default State Store. Contributed by Yiqun Lin. 2018-02-09 13:57:42 +08:00
Steve Loughran
996796f104 HADOOP-15214. Make Hadoop compatible with Guava 21.0. 
Contributed by Igor Dvorzhak
 2018-02-08 10:55:54 -08:00
Xiaoyu Yao
8faf0b50d4 HDFS-13120. Snapshot diff could be corrupted after concat. Contributed by Xiaoyu Yao. 2018-02-08 08:59:48 -08:00
Yongjun Zhang
f491f717e9 HDFS-13115. In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted. Contributed by Yongjun Zhang. 2018-02-07 12:58:09 -08:00
Brahma Reddy Battula
01bd6ab18f HDFS-12935. Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up. Contributed by Jianfei Jiang. 2018-02-07 23:10:33 +05:30
Tsz-Wo Nicholas Sze
4d1bf33d0f HDFS-12947. Limit the number of Snapshots allowed to be created for a Snapshottable Directory. Contributed by Shashikant Banerjee 2018-02-06 14:46:44 -08:00
Anu Engineer
4304fcd5bd HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 2018-02-06 13:43:45 -08:00
Wei-Chiu Chuang
2021f4bdce HDFS-11187. Optimize disk access for last partial chunk checksum of Finalized replica. Contributed by Wei-Chiu Chuang. 2018-02-02 17:18:42 -08:00
Hanisha Koneru
0ef639235b HDFS-13073. Cleanup code in InterQJournalProtocol.proto. Contributed by Bharat Viswanadham. 2018-02-02 16:28:03 -08:00
Akira Ajisaka
4aef8bd2ef
HDFS-13048. LowRedundancyReplicatedBlocks metric can be negative 2018-02-02 14:34:07 +09:00
Yiqun Lin
b0627c891b HDFS-13068. RBF: Add router admin option to manage safe mode. Contributed by Yiqun Lin. 2018-02-02 11:25:41 +08:00
Anu Engineer
aa45faf0b2 HDFS-12942. Synchronization issue in FSDataSetImpl#moveBlock. Contributed by Ajay Kumar. 2018-02-01 18:03:01 -08:00
Hanisha Koneru
dd50f53997 HDFS-13062. Provide support for JN to use separate journal disk per namespace. Contributed by Bharat Viswanadham. 2018-02-01 12:28:17 -08:00
Xiaoyu Yao
b3ae11d597 HDFS-12997. Move logging to slf4j in BlockPoolSliceStorage and Storage. Contributed by Ajay Kumar. 2018-02-01 10:45:34 -08:00
Inigo Goiri
6ca7204ceb HDFS-13043. RBF: Expose the state of the Routers in the federation. Contributed by Inigo Goiri. 2018-02-01 10:37:14 -08:00
Xiao Chen
ae2177d296 HDFS-12897. getErasureCodingPolicy should handle .snapshot dir better. Contributed by LiXin Ge. 2018-01-31 22:50:25 -08:00
Xiaoyu Yao
af015c0b23 HDFS-13060. Adding a BlacklistBasedTrustedChannelResolver for TrustedChannelResolver. Contributed by Ajay Kumar. 2018-01-31 22:39:51 -08:00
Hanisha Koneru
3ce2190b58 HDFS-13092. Reduce verbosity for ThrottledAsyncChecker#schedule. Contributed by Mukul Kumar Singh. 2018-01-31 11:05:17 -08:00
Weiwei Yang
2e7331ca26 HDFS-12528. Add an option to not disable short-circuit reads on failures. Contributed by Xiao Chen. 2018-01-31 07:25:03 +08:00
Yiqun Lin
dbb9dded33 HDFS-13044. RBF: Add a safe mode for the Router. Contributed by Inigo Goiri. 2018-01-30 12:12:08 +08:00
Kihwal Lee
fde95d463c HDFS-12574. Add CryptoInputStream to WebHdfsFileSystem read call. Contributed by Rushabh S Shah 2018-01-29 17:23:29 -06:00
Xiao Chen
b63dcd583f HDFS-12974. Exception message is not printed when creating an encryption zone fails with AuthorizationException. Contributed by fang zhenyi. 2018-01-28 22:19:49 -08:00
Yiqun Lin
7721fff744 HDFS-13042. RBF: Heartbeat Router State. Contributed by Inigo Goiri. 2018-01-25 15:51:26 +08:00
Inigo Goiri
95743c672e HDFS-12772. RBF: Federation Router State State Store internal API. Contributed by Inigo Goiri. 2018-01-23 19:15:44 -08:00
Yiqun Lin
d95c13774e HDFS-12963. Error log level in ShortCircuitRegistry#removeShm. Contributed by hu xiaodong. 2018-01-24 10:43:36 +08:00
Hanisha Koneru
6347b2253d HDFS-13023. Journal Sync does not work on a secure cluster. Contributed by Bharat Viswanadham. 2018-01-22 16:02:32 -08:00
Chen Liang
62c9e7fa99 HDFS-13036. Reusing the volume storage ID obtained by replicaInfo. Contributed by liaoyuxiangqin. 2018-01-19 17:51:53 -08:00
Lei Xu
2ed9d61aad HDFS-13039. StripedBlockReader.createBlockReader leaks socket on IOException. (Lei (Eddy) Xu) 2018-01-19 16:40:58 -08:00
Yiqun Lin
9e4f52d323 HDFS-12973. RBF: Document global quota supporting in federation. Contributed by Yiqun Lin. 2018-01-19 14:18:19 +08:00
Vinayakumar B
09efdfe9e1 HDFS-9049. Make Datanode Netty reverse proxy port to be configurable. Contributed by Vinayakumar B. 2018-01-17 14:16:48 +05:30
Brahma Reddy Battula
880b9d24ff HDFS-8693. refreshNamenodes does not support adding a new standby to a running DN. Contributed by Ajith S. 2018-01-16 16:21:02 +05:30
Yiqun Lin
9afb8025d6 HDFS-12972. RBF: Display mount table quota info in Web UI and admin command. Contributed by Yiqun Lin. 2018-01-13 13:16:43 +08:00
Inigo Goiri
d5d6a0353b HDFS-12919. RBF: Support erasure coding methods in RouterRpcServer. Contributed by Inigo Goiri. 2018-01-12 16:25:39 -08:00
Arpit Agarwal
b278f7b293 HDFS-12984. BlockPoolSlice can leak in a mini dfs cluster. Contributed by Ajay Kumar. 2018-01-12 08:04:00 -08:00
Yiqun Lin
d98a2e6e23 HDFS-12934. RBF: Federation supports global quota. Contributed by Yiqun Lin. 2018-01-10 13:59:11 +08:00
Inigo Goiri
d9006d8a4e HDFS-12802. RBF: Control MountTableResolver cache size. Contrubuted by Inigo Goiri. 2018-01-09 18:53:25 -08:00
Manoj Govindassamy
73ff09b79a HDFS-12985. NameNode crashes during restart after an OpenForWrite file present in the Snapshot got deleted. 2018-01-08 15:34:00 -08:00
Wei Yan
2ee0d64ace HDFS-12945. Switch to ClientProtocol instead of NamenodeProtocols in NamenodeWebHdfsMethods. Contributed by Wei Yan. 2018-01-08 14:25:39 -08:00
Yiqun Lin
bf5c948995 HDFS-11848. Enhance dfsadmin listOpenFiles command to list files under a given path. Contributed by Yiqun Lin. 2018-01-06 14:31:08 +08:00
Inigo Goiri
83b513ac6d HDFS-12988. RBF: Mount table entries not properly updated in the local cache. Contributed by Inigo Goiri. 2018-01-05 09:11:38 -08:00
Arpit Agarwal
739d3c394d HDFS-12987. Document - Disabling the Lazy persist file scrubber.. Contributed by Karthik Palanisamy. 2018-01-04 10:32:00 -08:00
Yiqun Lin
2a48b3594c HDFS-12948. DiskBalancer report command top option should only take positive numeric values. Contributed by Shashikant Banerjee. 2018-01-04 10:48:44 +08:00
Manoj Govindassamy
5c28804b95 HDFS-12629. NameNode UI should report total blocks count by type - replicated and erasure coded. 2018-01-02 15:05:27 -08:00
Manoj Govindassamy
42a1c98597 HDFS-11847. Enhance dfsadmin listOpenFiles command to list files blocking datanode decommissioning. 2018-01-02 14:59:36 -08:00
Lei Xu
6e3e1b8cde HDFS-12915. Fix findbugs warning in INodeFile$HeaderFormat.getBlockLayoutRedundancy. (Contributed by Chris Douglas) 2017-12-29 12:21:57 -08:00
Xiao Chen
5bf7e594d7 HDFS-9023. When NN is not able to identify DN for replication, reason behind it can be logged. 2017-12-28 11:54:08 -08:00
Chen Liang
826507c41b HDFS-12951. Incorrect javadoc in SaslDataTransferServer.java#receive. Contributed by Mukul Kumar Singh. 2017-12-21 11:20:30 -08:00
Wei Yan
a78db99190 HDFS-12932. Fix confusing LOG message for block replication. Contributed by Chao Sun. 2017-12-20 08:55:46 -08:00
Yiqun Lin
e040c97b77 HDFS-12937. RBF: Add more unit tests for router admin commands. Contributed by Yiqun Lin. 2017-12-19 15:31:34 +08:00
Virajith Jalaparti
8239e3afb3 HDFS-12712. [9806] Code style cleanup 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
80c3fec3a1 HDFS-12912. [READ] Fix configuration and implementation of LevelDB-based alias maps 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
c89b29bd42 HDFS-12893. [READ] Support replication of Provided blocks with non-default topologies. 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
0f6aa9564c HDFS-12905. [READ] Handle decommissioning and under-maintenance Datanodes with Provided storage. 2017-12-15 17:51:41 -08:00
Chris Douglas
2298f2d76b HDFS-12874. Documentation for provided storage. Contributed by Virajith Jalaparti 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
71ec170107 HDFS-12887. [READ] Allow Datanodes with Provided volumes to start when blocks with the same id exist locally 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
4531588a94 HDFS-11640. [READ] Datanodes should use a unique identifier when reading from external stores 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
fb996a32a9 HDFS-12894. [READ] Skip setting block count of ProvidedDatanodeStorageInfo on DN registration update 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
9c35be86e1 HDFS-12713. [READ] Refactor FileRegion and BlockAliasMap to separate out HDFS metadata and PROVIDED storage metadata. Contributed by Ewan Higgs 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
a027055dd2 HDFS-12885. Add visibility/stability annotations. Contributed by Chris Douglas 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
b634053c4d HDFS-12591. [READ] Implement LevelDBFileRegionFormat. Contributed by Ewan Higgs. 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
352f994b64 HDFS-12665. [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb). Contributed by Ewan Higgs. 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
cc933cba77 HDFS-12685. [READ] FsVolumeImpl exception when scanning Provided storage volume 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
4d59dabb7f HDFS-12809. [READ] Fix the randomized selection of locations in {{ProvidedBlocksBuilder}}. 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
3d3be87e30 HDFS-12778. [READ] Report multiple locations for PROVIDED blocks 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
3b1d30301b HDFS-12775. [READ] Fix reporting of Provided volumes 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
e1a28f95b8 HDFS-12777. [READ] Reduce memory and CPU footprint for PROVIDED volumes. 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
6cd80b2521 HDFS-12779. [READ] Allow cluster id to be specified to the Image generation tool 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
90d1b47a2a HDFS-12776. [READ] Increasing replication for PROVIDED files should create local replicas 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
71d0a82571 HDFS-12607. [READ] Even one dead datanode with PROVIDED storage results in ProvidedStorageInfo being marked as FAILED 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
98f5ed5aa3 HDFS-11902. [READ] Merge BlockFormatProvider and FileRegionProvider. 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
d6a9a89973 HDFS-12605. [READ] TestNameNodeProvidedImplementation#testProvidedDatanodeFailures fails after rebase 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
2407c9b93a HDFS-12093. [READ] Share remoteFS between ProvidedReplica instances. 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
663b3c08b1 HDFS-12091. [READ] Check that the replicas served from a ProvidedVolumeImpl belong to the correct external storage 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
546b95f484 HDFS-11673. [READ] Handle failures of Datanode with PROVIDED storage 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
55ade54b8e HDFS-11792. [READ] Test cases for ProvidedVolumeDF and ProviderBlockIteratorImpl 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
89b9faf529 HDFS-11703. [READ] Tests for ProvidedStorageMap 2017-12-15 17:51:37 -08:00
Virajith Jalaparti
aa5ec85f7f HDFS-11663. [READ] Fix NullPointerException in ProvidedBlocksBuilder 2017-12-15 17:51:37 -08:00
Virajith Jalaparti
1108cb7691 HDFS-11653. [READ] ProvidedReplica should return an InputStream that is bounded by its length 2017-12-15 17:51:37 -08:00
Virajith Jalaparti
d65df0f273 HDFS-11190. [READ] Namenode support for data stored in external stores. 2017-12-15 17:51:37 -08:00
Virajith Jalaparti
b668eb9155 HDFS-10675. Datanode support to read from external stores. 2017-12-15 17:51:37 -08:00
Lei Xu
1c15b1751c HDFS-12819. Setting/Unsetting EC policy shows warning if the directory is not empty. (Contributed by Lei (Eddy) Xu) 2017-12-15 10:04:43 -08:00
Yiqun Lin
ee028bfdf1 HDFS-12895. RBF: Add ACL support for mount table. Contributed by Yiqun Lin. 2017-12-15 14:09:24 +08:00
Xiao Chen
e1cb278cd0 HDFS-12910. Secure Datanode Starter should log the port when it fails to bind. Contributed by Stephen O'Donnell and Nanda kumar. 2017-12-14 19:22:51 -08:00
Kihwal Lee
f5a72424c0 HDFS-12907. Allow read-only access to reserved raw for non-superusers. Contributed by Rushabh S Shah. 2017-12-14 14:58:47 -06:00
Jason Lowe
80db744ee5 HDFS-12881. Output streams closed with IOUtils suppressing write errors. Contributed by Ajay Kumar 2017-12-14 09:45:44 -06:00
Yiqun Lin
91c96bdf8e HDFS-12883. RBF: Document Router and State Store metrics. Contributed by Yiqun Lin. 2017-12-14 16:00:53 +08:00
Chen Liang
46e18c8da7 HADOOP-14914. Change to a safely casting long to int. Contributed by Ajay Kumar. 2017-12-13 14:56:14 -08:00
Chris Douglas
693169ef34 HDFS-12882. Support full open(PathHandle) contract in HDFS 2017-12-11 20:14:15 -08:00
Wei-Chiu Chuang
55fc2d6485 HDFS-12891. Do not invalidate blocks if toInvalidate is empty. Contributed by Zsolt Venczel. 2017-12-11 16:43:03 -08:00
Inigo Goiri
5cd1056ad7 HDFS-12875. RBF: Complete logic for -readonly option of dfsrouteradmin add command. Contributed by Inigo Goiri. 2017-12-11 15:14:57 -08:00
Manoj Govindassamy
ef7d334d36 HDFS-12825. Fsck report shows config key name for min replication issues (Contributed by Gabor Bota). 2017-12-08 11:46:30 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
f19638333b HADOOP-15059. Undoing the switch of Credentials to PB format as default - done via HADOOP-12563 for supporting 2.x to 3.x upgrades. 2017-12-08 08:00:21 -08:00
Wei-Chiu Chuang
d6c31a3e6b HDFS-11915. Sync rbw dir on the first hsync() to avoid file lost on power failure. Contributed by Vinayakumar B. 2017-12-07 14:50:30 -08:00
Lei Xu
67662e2ac9 HDFS-12840. Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog. Contributed by Lei (Eddy) Xu. 2017-12-07 11:15:40 -08:00
Akira Ajisaka
6cca5b3bcb
HDFS-12889. Addendum patch to add missing file. 2017-12-07 13:23:37 +09:00
Xiao Chen
56b1ff80dd HDFS-12872. EC Checksum broken when BlockAccessToken is enabled. 2017-12-05 20:48:16 -08:00
Brahma Reddy Battula
6d16a99ae8 HDFS-11751. DFSZKFailoverController daemon exits with wrong status code. Contributed by Bharat Viswanadham 2017-12-05 19:53:46 +05:30
Arpit Agarwal
e00c7f78c1 HADOOP-14976. Set HADOOP_SHELL_EXECNAME explicitly in scripts. 2017-12-04 21:02:04 -08:00
Xiao Chen
404eab4dc0 HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 2017-12-04 10:40:11 -08:00
Chris Douglas
42307e3c3a HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-01 22:34:30 -08:00
Wei-Chiu Chuang
0faf506245 HDFS-12836. startTxId could be greater than endTxId when tailing in-progress edit log. Contributed by Chao Sun. 2017-12-01 12:01:21 -08:00
Chris Douglas
53bbef3802 Revert "HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak" 
This reverts commit 5304698dc8.
 2017-12-01 11:19:38 -08:00
Chris Douglas
5304698dc8 HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-01 10:29:30 -08:00
Konstantin V Shvachko
60fd0d7fd7 HDFS-12638. Delete copy-on-truncate block along with the original block, when deleting a file being truncated. Contributed by Konstantin Shvachko. 2017-11-30 18:18:28 -08:00
Tsz-Wo Nicholas Sze
b1c7654ee4 HDFS-12594. snapshotDiff fails if the report exceeds the RPC response limit. Contributed by Shashikant Banerjee 2017-11-30 12:18:29 -08:00
Chris Douglas
0e560f3b8d HDFS-12681. Make HdfsLocatedFileStatus a subtype of LocatedFileStatus 2017-11-29 20:28:06 -08:00
Kihwal Lee
53509f295b HDFS-11754. Make FsServerDefaults cache configurable. Contributed by Mikhail Erofeev. 2017-11-29 15:11:14 -06:00
Wei Yan
301641811d HDFS-12835. Fix the javadoc errors in Router-based federation. 2017-11-29 09:43:03 -08:00
Konstantin V Shvachko
d331762f24 HDFS-12832. INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit. Contribuited by Konstantin Shvachko. 2017-11-28 17:14:23 -08:00
Surendra Singh Lilhore
30941d99c9 HDFS-12857. StoragePolicyAdmin should support schema based path. Contributed by Surendra Singh Lilhore. 2017-11-28 23:57:03 +05:30
Chen Liang
60fc2a1388 HDFS-12804. Use slf4j instead of log4j in FSEditLog. Contributed by Mukul Kumar Singh. 2017-11-20 12:49:53 -08:00
Manoj Govindassamy
9fb4effd2c HDFS-12730. Verify open files captured in the snapshots across config disable and enable. 2017-11-20 10:28:12 -08:00
Inigo Goiri
e182e77794 HDFS-12801. RBF: Set MountTableResolver as default file resolver. Contributed by Inigo Goiri. 2017-11-16 16:58:47 -08:00
Weiwei Yang
462e25a3b2 HDFS-12814. Add blockId when warning slow mirror/disk in BlockReceiver. Contributed by Jiandan Yang. 2017-11-16 16:19:53 +08:00
Chris Douglas
675e9a8f57 Revert "HDFS-12681. Fold HdfsLocatedFileStatus into HdfsFileStatus." 
This reverts commit b85603e3f8.
 2017-11-15 19:20:37 -08:00
Arpit Agarwal
6d201f77c7 HDFS-12498. Journal Syncer is not started in Federated + HA cluster. Contributed by Bharat Viswanadham. 2017-11-10 16:30:38 -08:00
Arpit Agarwal
1d6f8bebe9 HADOOP-14929. Cleanup usage of decodecomponent and use QueryStringDecoder from netty. Contributed by Bharat Viswanadham. 2017-11-10 16:28:12 -08:00
Chen Liang
10a1f557e7 HDFS-12791. NameNode Fsck http Connection can timeout for directories with multiple levels. Contributed by Mukul Kumar Singh. 2017-11-09 18:47:34 -08:00
Akira Ajisaka
3a3566e1d1
HDFS-12732. Correct spellings of ramdomly to randomly in log. Contributed by hu xiaodong. 2017-11-09 15:14:46 +09:00
Brahma Reddy Battula
410d0319cf HDFS-12788. Reset the upload button when file upload fails. Contributed by Brahma Reddy Battula 2017-11-08 14:41:16 +05:30
Weiwei Yang
bb8a6eea52 HDFS-7060. Avoid taking locks when sending heartbeats from the DataNode. Contributed by Jiandan Yang. 2017-11-08 10:22:13 +08:00
Xiaoyu Yao
169cdaa38e HDFS-10528. Add logging to successful standby checkpointing. Contributed by Xiaoyu Yao. 2017-11-03 20:34:40 -07:00
Chris Douglas
b85603e3f8 HDFS-12681. Fold HdfsLocatedFileStatus into HdfsFileStatus. 2017-11-03 14:30:57 -07:00
Kihwal Lee
4d2dce40bb HDFS-12771. Add genstamp and block size to metasave Corrupt blocks list. Contributed by Kuhu Shukla. 2017-11-03 15:16:46 -05:00
Xiao Chen
299d38295d HDFS-11467. Support ErasureCoding section in OIV XML/ReverseXML. Contributed by Huafeng Wang. 2017-11-03 12:05:45 -07:00
Xiao Chen
b00f828d84 HDFS-12725. BlockPlacementPolicyRackFaultTolerant fails with very uneven racks. 2017-11-02 21:53:13 -07:00
Xiao Chen
e565b5277d HDFS-12682. ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED. 2017-11-02 21:27:35 -07:00
Weiwei Yang
56b88b0670 HDFS-12744. More logs when short-circuit read is failed and disabled. Contributed by Weiwei Yang. 2017-11-01 16:41:45 +08:00
Yiqun Lin
20304b91cc HDFS-12219. Javadoc for FSNamesystem#getMaxObjects is incorrect. Contributed by Erik Krogen. 2017-11-01 14:37:08 +08:00
Lei Xu
9367c25dbd HDFS-12482. Provide a configuration to adjust the weight of EC recovery tasks to adjust the speed of recovery. (lei) 2017-10-31 21:58:14 -07:00
Andrew Wang
5f681fa821 Revert "HDFS-12499. dfs.namenode.shared.edits.dir property is currently namenode specific key. Contributed by Bharat Viswanadham." 
This reverts commit b922ba7393.
 2017-10-31 10:46:10 -07:00
Arpit Agarwal
b922ba7393 HDFS-12499. dfs.namenode.shared.edits.dir property is currently namenode specific key. Contributed by Bharat Viswanadham. 2017-10-31 10:23:00 -07:00
Chris Douglas
d015e0bbd5 HDFS-7878. API - expose a unique file identifier. 2017-10-31 09:44:01 -07:00
Chris Douglas
d55a84951a HDFS-12582. Replace HdfsFileStatus constructor with a builder pattern. Contributed by Bharat Viswanadham 2017-10-27 15:36:03 -07:00
Arpit Agarwal
7da15eac7a HDFS-12579. JournalNodeSyncer should use fromUrl field of EditLogManifestResponse to construct servlet Url. Contributed by Hanisha Koneru. 2017-10-25 14:24:22 -07:00
Xiao Chen
f66ad1fab9 HADOOP-14957. ReconfigurationTaskStatus is exposing guava Optional in its public api. Contributed by Haibo Chen, Xiao Chen. 2017-10-25 14:11:30 -07:00
Manoj Govindassamy
075dd45a24 HDFS-12544. SnapshotDiff - support diff generation on any snapshot root descendant directory. 2017-10-25 10:54:40 -07:00
Zhe Zhang
17cd8d0c17 Revert "HDFS-12502. nntop should support a category based on FilesInGetListingOps." 
This reverts commit 60bfee270e.
 2017-10-24 15:06:50 -07:00
Wei-Chiu Chuang
0f2a69127a HDFS-12249. dfsadmin -metaSave to output maintenance mode blocks. Contributed by Wellington Chevreuil. 2017-10-23 15:24:34 -07:00
Arpit Agarwal
96be795656 HDFS-12650. Use slf4j instead of log4j in LeaseManager. Contributed by Ajay Kumar. 2017-10-23 13:20:46 -07:00
Arpit Agarwal
4b00c9a47f HDFS-12683. DFSZKFailOverController re-order logic for logging Exception. Contributed by Bharat Viswanadham. 2017-10-23 10:47:16 -07:00
Xiao Chen
248d9b6fff HDFS-12518. Re-encryption should handle task cancellation and progress better. 2017-10-20 16:25:04 -07:00
Kai Zheng
ce7cf66e5e HDFS-12448. Make sure user defined erasure coding policy ID will not overflow. Contributed by Huafeng Wang 2017-10-20 09:42:04 +08:00
Wei-Chiu Chuang
4ab0c8f96a HDFS-12619. Do not catch and throw unchecked exceptions if IBRs fail to process. Contributed by Wei-Chiu Chuang. 2017-10-19 06:17:59 -07:00
Zhe Zhang
60bfee270e HDFS-12502. nntop should support a category based on FilesInGetListingOps. 2017-10-18 23:51:24 -07:00
Lei Xu
31ebccc962 HDFS-12613. Native EC coder should implement release() as idempotent function. (Lei (Eddy) Xu) 2017-10-16 19:44:30 -07:00
Manoj Govindassamy
b406d8e375 HDFS-12614. FSPermissionChecker#getINodeAttrs() throws NPE when INodeAttributesProvider configured. 2017-10-16 17:42:41 -07:00
Xiao Chen
21bc855587 HDFS-12642. Log block and datanode details in BlockRecoveryWorker. 2017-10-16 10:34:06 -07:00
Xiao Chen
035c6ee587 HDFS-12603. Enable async edit logging by default. Contributed by Andrew Wang. 2017-10-16 09:51:10 -07:00
Arpit Agarwal
8dd1eeb94f HDFS-12553. Add nameServiceId to QJournalProtocol. Contributed by Bharat Viswanadham 2017-10-13 14:22:21 -07:00
Arpit Agarwal
bb0a742aac HDFS-12627. Fix typo in DFSAdmin command output. Contributed by Ajay Kumar. 2017-10-11 12:26:14 -07:00
Manoj Govindassamy
78af6cdc53 HDFS-12573. Divide the total blocks metrics into replicated and erasure coded. Contributed by Takanobu Asanuma. 2017-10-10 14:23:29 -07:00
Jitendra Pandey
f29e55bf5e HADOOP-14910. Upgrade netty-all jar to latest 4.0.x.Final. Contributed by Vinayakumar B. 2017-10-10 11:00:07 -07:00
Andrew Wang
82cd85d868 Revert "HDFS-12603. Enable async edit logging by default." 
This reverts commit afb42aeabf.
 2017-10-09 17:53:56 -07:00
Andrew Wang
afb42aeabf HDFS-12603. Enable async edit logging by default. 2017-10-09 11:20:00 -07:00
Inigo Goiri
5d63a388d1 HDFS-12577. Rename Router tooling. Contributed by Inigo Goiri. 
(cherry picked from commit 53e8d0d030525e4c7f3875e23807c6dbe778890f)
 2017-10-06 18:50:52 -07:00
Inigo Goiri
81601dac8e HDFS-12273. Federation UI. Contributed by Inigo Goiri. 
(cherry picked from commit adbb2e00c7b85524fd43bd68895d49814c16680a)
 2017-10-06 18:50:52 -07:00
Inigo Goiri
6c69e23dcd HDFS-12580. Rebasing HDFS-10467 after HDFS-12447. Contributed by Inigo Goiri. 
(cherry picked from commit b12440d3479f19138bc66ea59baf41eb89061906)
 2017-10-06 18:50:51 -07:00
Inigo Goiri
ad41c81559 HDFS-12381. [Documentation] Adding configuration keys for the Router. Contributed by Inigo Goiri. 
(cherry picked from commit c2d6aa79055ef72406fa598e1c743b0c994b5da8)
 2017-10-06 18:50:51 -07:00
Inigo Goiri
67785fe006 HDFS-12450. Fixing TestNamenodeHeartbeat and support non-HA. Contributed by Inigo Goiri. 
(cherry picked from commit 928d1e87f9dbe64f89b858ccc1780723f3af58e7)
 2017-10-06 18:50:50 -07:00
Inigo Goiri
1f06b81ecb HDFS-12430. Rebasing HDFS-10467 After HDFS-12269 and HDFS-12218. Contributed by Inigo Goiri. 
(cherry picked from commit 3302e792d469b7e8f3bfa10151e4e1c546589734)
 2017-10-06 18:50:50 -07:00
Inigo Goiri
bc9e588a19 HDFS-12335. Federation Metrics. Contributed by Inigo Goiri. 
(cherry picked from commit 3b19e77752afce87936f5c0d1e6d272fba798d7b)
 2017-10-06 18:50:50 -07:00
Inigo Goiri
cc58e7a983 HDFS-12384. Fixing compilation issue with BanDuplicateClasses. Contributed by Inigo Goiri. 
(cherry picked from commit fabe02c8fafa807198054da0c02b2ebaafda76aa)
 2017-10-06 18:50:49 -07:00
Inigo Goiri
7cb6bdf09e HDFS-10631. Federation State Store ZooKeeper implementation. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 23c4ddee11ab1300325a6361124ee8ad6f68d7a4)
 2017-10-06 18:50:49 -07:00
Inigo Goiri
346c9fce43 HDFS-12312. Rebasing HDFS-10467 (2). Contributed by Inigo Goiri. 
(cherry picked from commit 90ba6843fb3ac5dc7576535e66a75a5e3433247b)
 2017-10-06 18:50:48 -07:00
Inigo Goiri
b3e6bd22e3 HDFS-10646. Federation admin tool. Contributed by Inigo Goiri. 
(cherry picked from commit ae27e31fbcf546481db0b0345772db2e9132372e)
 2017-10-06 18:50:48 -07:00
Inigo Goiri
6f0de27318 HDFS-10880. Federation Mount Table State Store internal API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 58b97df661441150d35abd44b3a8606206b46441)
 2017-10-06 18:50:47 -07:00
Inigo Goiri
d8c8107332 HDFS-11826. Federation Namenode Heartbeat. Contributed by Inigo Goiri. 
(cherry picked from commit 928f8dab52191e733984d37f47b69719ccf11313)
 2017-10-06 18:50:47 -07:00
Inigo Goiri
55da7fd7eb HDFS-10687. Federation Membership State Store internal API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 95cae08849d23cf12c5d280c29f90908e6a90d40)
 2017-10-06 18:50:47 -07:00
Inigo Goiri
0ec82b8cdf HDFS-12223. Rebasing HDFS-10467. Contributed by Inigo Goiri. 
(cherry picked from commit 47db6e9d8e2c264671c89fdd6cb11a7c762d2cce)
 2017-10-06 18:50:46 -07:00
Inigo Goiri
ca4f209b49 HDFS-11546. Federation Router RPC server. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 8a9cdebebf26841a0f1e99fb08135f4597f2eba2)
 2017-10-06 18:50:46 -07:00
Inigo Goiri
4bf877b03f HDFS-10630. Federation State Store FS Implementation. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit c6e0bd640cdaf83a660fa050809cad6f1d4c6f4d)
 2017-10-06 18:50:45 -07:00
Inigo
2c740a684a HDFS-10882. Federation State Store Interface API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 6d94c90ece1c1d23d4c97e72c54e9991f5dbc481)
 2017-10-06 18:50:45 -07:00
Inigo
533b986633 HDFS-10881. Federation State Store Driver API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 0f88e049156dce173afc0dbda864e29190dd2210)
 2017-10-06 18:50:45 -07:00
Inigo
2761bbc91a HDFS-10629. Federation Roter. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 6821e801724ac38e9737538b2164c9ae88792282)
 2017-10-06 18:50:44 -07:00
Andrew Wang
644c2f6924 HDFS-12567. BlockPlacementPolicyRackFaultTolerant fails with racks with very few nodes. 2017-10-05 16:58:43 -07:00
Arpit Agarwal
b6942cbe9b HDFS-12420. Add an option to disallow 'namenode format -force'. Contributed by Ajay Kumar. 2017-10-05 15:26:52 -07:00
Xiaoyu Yao
107c177782 HDFS-12455. WebHDFS - Adding "snapshot enabled" status to ListStatus query result. Contributed by Ajay Kumar. 2017-10-03 13:02:12 -07:00
Arpit Agarwal
b91305119b HDFS-11968. ViewFS: StoragePolicies commands fail with HDFS federation. Contributed by Mukul Kumar Singh. 2017-10-03 11:23:40 -07:00
Anu Engineer
015abcd8ce HDFS-12576. JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured. Contributed by Bharat Viswanadham. 2017-10-02 15:40:00 -07:00
Lei Xu
27ffd43b64 HDFS-12569. Unset EC policy logs empty payload in edit log. (Lei (Eddy) Xu) 2017-10-02 15:35:49 -07:00
Chen Liang
5a68c2cef3 HDFS-12552. Use slf4j instead of log4j in FSNamesystem. Contributed by Ajay Kumar. 2017-10-02 11:26:53 -07:00
Daryn Sharp
0da29cbeea [HDFS-12386] Add fsserver defaults call to WebhdfsFileSystem. (Rushabh Shah via daryn) 2017-09-26 15:15:24 -05:00
Lei Xu
1267ff22ce HDFS-12523. Thread pools in ErasureCodingWorker do not shutdown. (Huafeng Wang via Lei) 2017-09-26 10:08:16 -07:00
Brahma Reddy Battula
9d3e4cccf9 HDFS-5040.Audit log for admin commands/ logging output of all DFS admin commands. Contributed by Kuhu Shukla. 2017-09-26 21:59:22 +05:30
Akira Ajisaka
02e2a9b115
HDFS-12304. Remove unused parameter from FsDatasetImpl#addVolume. Contributed by Chen Liang. 2017-09-26 01:25:07 +09:00
Anu Engineer
d0b2c5850b HDFS-12516. Suppress the fsnamesystem lock warning on nn startup. Contributed by Ajay Kumar. 2017-09-24 21:03:06 -07:00
Anu Engineer
415e5a157a HDFS-12375. Fail to start/stop journalnodes using start-dfs.sh/stop-dfs.sh. Contributed by Bharat Viswanadham. 2017-09-23 15:05:25 -07:00
Anu Engineer
40ef9fa171 HADOOP-14904. Fix javadocs issues in Hadoop HDFS. Contributed by Mukul Kumar Singh. 2017-09-23 12:16:48 -07:00
Anu Engineer
423ebcaf51 HDFS-12320. Add quantiles for transactions batched in Journal sync. Contributed by Hanisha Koneru. 2017-09-23 10:34:13 -07:00
Anu Engineer
36fb90c939 HDFS-12064. Reuse object mapper in HDFS. Contributed by Hanisha Koneru. 2017-09-23 10:11:00 -07:00
Jitendra Pandey
cda3378659 HDFS-12486. GetConf to get journalnodeslist. Contributed by Bharat Viswanadham. 2017-09-22 16:52:47 -07:00
Arpit Agarwal
08fca508e6 HDFS-12530. Processor argument in Offline Image Viewer should be case insensitive. Contributed by Hanisha Koneru. 2017-09-22 09:47:57 -07:00
Arpit Agarwal
b9e423fa8d HDFS-12496. Make QuorumJournalManager timeout properties configurable. Contributed by Ajay Kumar. 2017-09-21 08:44:43 -07:00
Kihwal Lee
6bf921a5c3 HDFS-12371. BlockVerificationFailures and BlocksVerified show up as 0 in Datanode JMX. Contributed by Hanisha Koneru. 2017-09-21 08:42:50 -05:00
Akira Ajisaka
50849ec9eb
HDFS-12526. FSDirectory should use Time.monotonicNow for durations. Contributed by Bharat Viswanadham. 2017-09-21 19:27:48 +09:00
Andrew Wang
a12f09ba3c HDFS-12447. Rename AddECPolicyResponse to AddErasureCodingPolicyResponse. Contributed by SammiChen. 2017-09-20 11:51:17 -07:00
Ming Ma
230b85d586 HDFS-12473. Change hosts JSON file format. 2017-09-20 09:03:59 -07:00
Andrew Wang
51edaacd09 HDFS-12445. Correct spellings of choosen to chosen. Contributed by hu xiaodong. 2017-09-19 13:48:23 -07:00
Brahma Reddy Battula
fda1221c55 HDFS-11799. Introduce a config to allow setting up write pipeline with fewer nodes than replication factor. Contributed by Brahma Reddy Battula 2017-09-19 11:25:45 +05:30
Kai Zheng
0f9af246e8 HDFS-12460. Make addErasureCodingPolicy an idempotent operation. Contributed by Sammi Chen 2017-09-18 18:07:12 +08:00
Erik Krogen
90894c7262 HDFS-12323. NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout. Contributed by Erik Krogen. 2017-09-15 13:56:27 -07:00
Lei Xu
fbe06b5880 HDFS-12349. Improve log message when it could not alloc enough blocks for EC. (Lei (Eddy) Xu) 2017-09-15 12:12:42 -07:00
Nathan Roberts
de197fc156 HDFS-12441. Supress UnresolvedPathException in namenode log. Contributed by Kihwal Lee. 2017-09-15 13:08:41 -05:00
Kihwal Lee
78bdf10ae4 HDFS-12424. Datatable sorting on the Datanode Information page in the Namenode UI is broken. Contributed by Shawna Martell. 2017-09-15 09:28:31 -05:00
Kai Zheng
08d996d3e9 HDFS-12395. Support erasure coding policy operations in namenode edit log. Contributed by Sammi Chen 2017-09-15 09:43:39 +08:00
Kai Zheng
ae8f55b932 HDFS-7859. Erasure Coding: Persist erasure coding policies in NameNode. Contributed by Sammi Chen 2017-09-15 09:08:18 +08:00
Lei Xu
61cee3a0b9 HDFS-12378. TestClientProtocolForPipelineRecovery#testZeroByteBlockRecovery fails on trunk. (Lei (Eddy) Xu) 2017-09-14 17:03:25 -07:00
Andrew Wang
390c2b5df0 Revert "HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang." 
This reverts commit 7fcc73fc0d.

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
 2017-09-14 15:12:26 -07:00
Arpit Agarwal
65a941008d Revert "HDFS-10391. Always enable NameNode service RPC port. Contributed by Gergely Novak." 
This reverts commit aa4b6fbe75.
 2017-09-14 11:17:08 -07:00
Kai Zheng
e12f3e85bd HDFS-12438. Rename dfs.datanode.ec.reconstruction.stripedblock.threads.size to dfs.datanode.ec.reconstruction.threads. Contributed by Andrew Wang 2017-09-14 15:39:10 +08:00
ChenSammi
e0b3c644e1 HDFS-12414. Ensure to use CLI command to enable/disable erasure coding policy. Contributed by Sammi Chen 2017-09-14 09:15:29 +08:00
Lei Xu
73aed34dff HDFS-12409. Add metrics of execution time of different stages in EC recovery task. (Lei (Eddy) Xu) 2017-09-13 17:14:13 -07:00
Lei Xu
123342cd07 HDFS-12412. Change ErasureCodingWorker.stripedReadPool to cached thread pool. (Lei (Eddy) Xu) 2017-09-12 18:12:07 -07:00
Arpit Agarwal
68282c8eac HDFS-12407. Journal node fails to shutdown cleanly if JournalNodeHttpServer or JournalNodeRpcServer fails to start. Contributed by Ajay Kumar. 2017-09-12 16:18:41 -07:00
Chen Liang
738c2a9682 HDFS-12406. dfsadmin command prints "Exception encountered" even if there is no exception, when debug is enabled. Contributed by Nandakumar. 2017-09-11 14:48:07 -07:00
Lei Xu
de9994bd89 Revert "HDFS-12349. Improve log message when it could not alloc enough blocks for EC. (lei)" 
This reverts commit 3e6d0ca2b2.
 2017-09-11 10:06:05 -07:00
Arpit Agarwal
aa4b6fbe75 HDFS-10391. Always enable NameNode service RPC port. Contributed by Gergely Novak. 2017-09-09 08:40:53 -07:00
Xiao Chen
b3a4d7d2a0 HDFS-12400. Provide a way for NN to drain the local key cache before re-encryption. 2017-09-07 20:51:02 -07:00
Lei Xu
3e6d0ca2b2 HDFS-12349. Improve log message when it could not alloc enough blocks for EC. (lei) 2017-09-07 18:01:37 -07:00
Manoj Govindassamy
3b3be355b3 HDFS-12404. Rename hdfs config authorization.provider.bypass.users to attributes.provider.bypass.users. 2017-09-07 17:20:42 -07:00
Andrew Wang
4e50dc976a HDFS-12218. Addendum. Rename split EC / replicated block metrics in BlockManager. 2017-09-07 16:57:19 -07:00
Andrew Wang
40c2f31f8d HDFS-12218. Rename split EC / replicated block metrics in BlockManager. 2017-09-07 16:56:35 -07:00
Xiao Chen
52b894db33 HDFS-12369. Edit log corruption due to hard lease recovery of not-closed file which has snapshots. 2017-09-07 16:30:12 -07:00
Yongjun Zhang
d77ed238a9 HDFS-12357. Let NameNode to bypass external attribute provider for configured users. Contributed by Yongjun Zhang, Arun Suresh. 2017-09-07 09:50:36 -07:00
Arpit Agarwal
5ff74167dd HDFS-12376. Enable JournalNode Sync by default. Contributed by Hanisha Koneru. 2017-09-07 08:18:28 -07:00
Kai Zheng
2adf8bed71 HDFS-12402. Refactor ErasureCodingPolicyManager and related codes. Contributed by Sammi Chen 2017-09-07 20:38:23 +08:00
Xiao Chen
0ba8ff4b77 HDFS-12359. Re-encryption should operate with minimum KMS ACL requirements. 2017-09-05 10:08:27 -07:00
Xiao Chen
633c1ea455 HDFS-12383. Re-encryption updater should handle canceled tasks better. 2017-09-03 19:50:41 -07:00
Xiao Chen
1b3b9938cf HDFS-12300. Audit-log delegation token related operations. 2017-08-31 23:20:01 -07:00
Mingliang Liu
1fbb662c70 HDFS-12363. Possible NPE in BlockManager$StorageInfoDefragmenter#scanAndCompactStorages. Contributed by Xiao Chen 2017-08-31 22:36:56 -07:00
Rakesh Radhakrishnan
200b11368d HDFS-12258. ec -listPolicies should list all policies in system, no matter it's enabled or disabled. Contributed by Wei Zhou. 2017-08-30 12:58:56 +05:30
Xiao Chen
f3661fd08e HDFS-12336. Listing encryption zones still fails when deleted EZ is not a direct child of snapshottable directory. Contributed by Wellington Chevreuil. 2017-08-29 21:30:02 -07:00
Yongjun Zhang
cf93d60d3f HDFS-12191. Provide option to not capture the accessTime change of a file to snapshot if no other modification has been done to this file. Contributed by Yongjun Zhang. 2017-08-29 16:57:03 -07:00
Arpit Agarwal
a1e3f84afe HDFS-12293. DataNode should log file name on disk error. Contributed by Ajay Kumar. 2017-08-28 15:49:59 -07:00
Arpit Agarwal
077a5eed9f HDFS-12358. Handle IOException when transferring edit log to Journal current dir through JN sync. Contributed by Hanisha Koneru. 2017-08-26 22:47:55 -07:00
Brahma Reddy Battula
bb6a3c8330 HDFS-12248. SNN will not upload fsimage on IOE and Interrupted exceptions. (Brahma Reddy Battula) 2017-08-26 13:47:12 +08:00
Manoj Govindassamy
b89ffcff36 HDFS-11986. Dfsadmin should report erasure coding related information separately. 2017-08-25 17:21:56 -07:00
Lei Xu
36bada3032 HDFS-12215. DataNode.transferBlock does not create its daemon in the xceiver thread group. (Lei Xu) 2017-08-25 16:01:14 -07:00
Andrew Wang
f29a0fc288 HDFS-12303. Change default EC cell size to 1MB for better performance. Contributed by Wei Zhou. 2017-08-25 14:14:23 -07:00
Arpit Agarwal
4b2c442d4e HDFS-12319. DirectoryScanner will throw IllegalStateException when Multiple BP's are present. Contributed by Brahma Reddy Battula. 2017-08-25 10:41:52 -07:00
Xiao Chen
1000a2af04 HDFS-10899. Add functionality to re-encrypt EDEKs. 2017-08-23 17:06:16 -07:00
Andrew Wang
b298948897 HDFS-12072. Provide fairness between EC and non-EC recovery tasks. Contributed by Eddy Xu. 2017-08-17 15:26:11 -07:00
Andrew Wang
96b3a6b972 HDFS-11082. Provide replicated EC policy to replicate files. Contributed by SammiChen. 2017-08-16 22:17:06 -07:00
Akira Ajisaka
08aaa4b36f
HDFS-12269. Better to return a Map rather than HashMap in getErasureCodingCodecs. Contributed by Huafeng Wang. 2017-08-17 13:20:27 +09:00
Ravi Prakash
f34646d652 HDFS-12301. NN File Browser UI: Navigate to a path when enter is pressed 2017-08-15 15:44:59 -07:00
Wei-Chiu Chuang
e3ae3e2644 HDFS-12066. When Namenode is in safemode,may not allowed to remove an user's erasure coding policy. Contributed by lufei. 2017-08-15 07:41:43 -07:00
Wei-Chiu Chuang
1040bae6fc HDFS-12054. FSNamesystem#addErasureCodingPolicies should call checkNameNodeSafeMode() to ensure Namenode is not in safemode. Contributed by lufei. 2017-08-15 07:38:43 -07:00
Yiqun Lin
2e43c28e01 HDFS-11696. Fix warnings from Spotbugs in hadoop-hdfs. Contributed by Yiqun Lin. 2017-08-15 16:48:49 +08:00
Chris Douglas
645a8f2a4d HADOOP-14726. Mark FileStatus::isDir as final 2017-08-14 21:57:20 -07:00
Lei Xu
ce797a1706 HDFS-12221. Replace xcerces in XmlEditsVisitor. (Ajay Kumar via lei) 2017-08-14 10:27:47 -07:00
Yiqun Lin
f13ca94954 HDFS-12287. Remove a no-longer applicable TODO comment in DatanodeManager. Contributed by Chen Liang. 2017-08-11 14:13:45 +08:00
John Zhuge
312e57b954 HDFS-11957. Enable POSIX ACL inheritance by default. Contributed by John Zhuge. 2017-08-10 10:30:47 -07:00
Kihwal Lee
b5c02f95b5 HDFS-12278. LeaseManager operations are inefficient in 2.8. Contributed by Rushabh S Shah. 2017-08-09 16:46:05 -05:00
Kihwal Lee
69afa26f19 HDFS-12157. Do fsyncDirectory(..) outside of FSDataset lock. Contributed by inayakumar B. 2017-08-09 09:03:51 -05:00