big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
23,546 Commits 2 Branches 2 Tags 264 MiB
2338d25dc7
Commit Graph

8684 Commits

Author SHA1 Message Date
Surendra Singh Lilhore
29b411d5f0 HDFS-14263. Remove unnecessary block file exists check from FsDatasetImpl#getBlockInputStream(). Contributed by Surendra Singh Lilhore 2019-02-13 23:41:27 +05:30
Surendra Singh Lilhore
35d4f32b32 HDFS-14274. EC: NPE While Listing EC Policy For A Directory Following Replication Policy. Contributed by Ayush Saxena. 2019-02-13 23:06:46 +05:30
Vinayakumar B
7806403842 HDFS-14266. EC : Fsck -blockId shows null for EC Blocks if One Block Is Not Available. Contributed by Ayush Saxena. 2019-02-12 21:57:57 +05:30
Siyao Meng
5c10630ad8 HDFS-14261. Kerberize JournalNodeSyncer unit test. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-02-11 11:02:32 -08:00
Inigo Goiri
0ceb1b70f3 HDFS-14260. Replace synchronized method in BlockReceiver with atomic value. Contributed by BELUGA BEHR. 2019-02-11 10:09:44 -08:00
Yiqun Lin
177131793a HDFS-14172. Avoid NPE when SectionName#fromString returns null. Contributed by Xiang Li. 2019-02-08 20:51:30 +08:00
Surendra Singh Lilhore
4be87353e3 HDFS-14140. JournalNodeSyncer authentication is failing in secure cluster. Contributed by Surendra Singh Lilhore. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-02-07 16:51:59 -08:00
Steve Loughran
668817a6ce
Revert "HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth." 
(accidentally mixed in two patches)

This reverts commit fa8cd1bf28.
 2019-02-07 21:57:22 +00:00
Siyao Meng
6aa63452b3 HDFS-14242. OIV WebImageViewer: NPE when param op is not specified. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-02-06 12:19:21 -08:00
Erik Krogen
d3de8e162b HDFS-14250. [SBN read]. msync should always direct to active NameNode to get latest stateID. Contributed by Chao Sun. 2019-02-06 09:54:47 -08:00
Kitti Nanasi
49ddd8a6ed HDFS-14231. DataXceiver#run() should not log exceptions caused by InvalidToken exception as an error. Contributed by Kitti Nanasi. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-02-05 15:14:19 -08:00
Wangda Tan
308f3168fa Make upstream aware of 3.1.2 release 
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
 2019-02-05 14:03:18 -08:00
Da Zhou
fa8cd1bf28
HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth. 
Contributed by Da Zhou and Junhua Gu.
 2019-02-05 19:23:15 +00:00
Steve Loughran
f365957c63
HADOOP-15229. Add FileSystem builder-based openFile() API to match createFile(); 
S3A to implement S3 Select through this API.

The new openFile() API is asynchronous, and implemented across FileSystem and FileContext.

The MapReduce V2 inputs are moved to this API, and you can actually set must/may
options to pass in.

This is more useful for setting things like s3a seek policy than for S3 select,
as the existing input format/record readers can't handle S3 select output where
the stream is shorter than the file length, and splitting plain text is suboptimal.
Future work is needed there.

In the meantime, any/all filesystem connectors are now free to add their own filesystem-specific
configuration parameters which can be set in jobs and used to set filesystem input stream
options (seek policy, retry, encryption secrets, etc).

Contributed by Steve Loughran
 2019-02-05 11:51:02 +00:00
Kitti Nanasi
5f15a60e32 HDFS-14125. Use parameterized log format in ECTopologyVerifier. Contributed by Kitti Nanasi. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-02-04 14:52:07 -08:00
Inigo Goiri
0e79a86582 HDFS-14202. dfs.disk.balancer.max.disk.throughputInMBperSec property is not working as per set value. Contributed by Ranith Sardar. 2019-02-04 11:59:48 -08:00
tiwalter
9aa3dc872c
HDFS-14158. Checkpointer ignores configured time period > 5 minutes 
This closes #449

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2019-02-04 13:51:07 +09:00
Takanobu Asanuma
ec77e95bbd HDFS-14232. libhdfs is not included in binary tarball. Contributed by Akira Ajisaka. 2019-02-04 10:12:09 +09:00
Kitti Nanasi
0ab7fc9200 HDFS-14187. Make warning message more clear when there are not enough data nodes for EC write. Contributed by Kitti Nanasi. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-01-31 16:07:24 -08:00
Akira Ajisaka
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki. 2019-01-29 18:29:56 -08:00
Wei-Chiu Chuang
d1714c20e9 Revert "HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh." 
This reverts commit 1d523279da.
 2019-01-29 15:43:09 -08:00
Masatake Iwasaki
dc5af4c6b7 HDFS-12729. Document special paths in HDFS. Contributed by Masatake Iwasaki. 2019-01-26 05:34:32 +09:00
Pranay Singh
1d523279da HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-01-25 09:02:57 -08:00
Dinesh Chitlangia
45caeee6cf HDFS-14228. Incorrect getSnapshottableDirListing() javadoc. Contributed by Dinesh Chitlangia. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-01-25 07:37:01 -08:00
shwetayakkali@cloudera.com
f3e642d92b HDFS-14185. Cleanup method calls to static Assert methods in TestAddStripedBlocks 
(Contributed by Shweta Yakkali via Daniel Templeton)

Change-Id: I0d533f575a405ed3affd05994a4208bde7d9cbe9
 2019-01-23 15:57:39 -08:00
Kitti Nanasi
951cdd7e4c HDFS-14061. Check if the cluster topology supports the EC policy before setting, enabling or adding it. Contributed by Kitti Nanasi. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-01-23 14:40:57 -08:00
Surendra Singh Lilhore
221e308cb5 HDFS-14153. [SPS] : Add Support for Storage Policy Satisfier in WEBHDFS. Contributed by Ayush Saxena. 2019-01-23 17:00:37 +05:30
Takanobu Asanuma
7d6792e5d2 HDFS-14218. EC: Ls -e throw NPE when directory ec policy is disabled. Contributed by Ayush Saxena. 2019-01-23 10:03:06 +09:00
Arpit Agarwal
6f0756fc0e HDFS-14222. Make ThrottledAsyncChecker constructor public. Contributed by Arpit Agarwal. 2019-01-21 20:45:09 -08:00
Arpit Agarwal
1ff658b2ef HDFS-14221. Replace Guava Optional with Java Optional. Contributed by Arpit Agarwal. 2019-01-21 20:44:37 -08:00
Weiwei Yang
de34fc148c HDFS-14207. ZKFC should catch exception when ha configuration missing. Contributed by Fei Hui. 2019-01-22 12:01:08 +08:00
Sunil G
e9962240f0 Make 3.2.0 aware to other branches - jdiff 2019-01-21 22:41:26 +05:30
Inigo Goiri
f048512bb8 HDFS-14192. Track missing DFS operations in Statistics and StorageStatistics. Contributed by Ayush Saxena. 2019-01-16 10:14:22 -08:00
Giovanni Matteo Fumarola
fb8932a727 HADOOP-16029. Consecutive StringBuilder.append can be reused. Contributed by Ayush Saxena. 2019-01-11 10:54:49 -08:00
Surendra Singh Lilhore
9aeaaa0479 HDFS-14198. Upload and Create button doesn't get enabled after getting reset. Contributed by Ayush Saxena. 2019-01-11 14:36:55 +05:30
shwetayakkali@cloudera.com
4ab5260b7e HDFS-14132. Add BlockLocation.isStriped() to determine if block is replicated or Striped 
(Contributed by Shweta Yakkali via Daniel Templeton)

Change-Id: I0ed8996a0bae2ad2c7d3513143195533f7191af8
 2019-01-08 17:04:57 -08:00
Giovanni Matteo Fumarola
1a08302897 HDFS-14189. Fix intermittent failure of TestNameNodeMetrics. Contributed by Ayush Saxena. 2019-01-08 10:38:06 -08:00
Surendra Singh Lilhore
f660e5eaa3 HDFS-14163. Debug Admin Command Should Support Generic Options. Contributed by Ayush Saxena. 2019-01-03 18:58:03 +05:30
Surendra Singh Lilhore
21fe77e3ab HDFS-14184. [SPS] Add support for URI based path in satisfystoragepolicy command. Contributed by Ayush Saxena. 2019-01-03 18:36:53 +05:30
Akira Ajisaka
f30473f2ef
HDFS-14183. [SPS] Remove the -w parameter from the -satisfystoragepolicy usage. Contributed by Ayush Saxena. 2018-12-31 03:16:26 +09:00
Wei-Chiu Chuang
e9a005dfcf HDFS-14171. Performance improvement in Tailing EditLog. Contributed by Kenneth Yang. 2018-12-28 10:34:18 -08:00
Konstantin V Shvachko
bd2a59e508 HDFS-14170. [SBN read] Fix checkstyle warnings related to SBN reads. Contributed by Konstantin V Shvachko. 2018-12-24 09:39:20 -08:00
Chao Sun
c546b12e45 HDFS-14154. [SBN read] Document dfs.ha.tail-edits.period in user guide. Contributed by Chao Sun. 2018-12-24 09:34:01 -08:00
Konstantin V Shvachko
c9d7737431 HDFS-14160. [SBN read] ObserverReadInvocationHandler should implement RpcInvocationHandler. Contributed by Konstantin V Shvachko. 2018-12-24 09:34:01 -08:00
Chao Sun
fa8550337d HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun. 2018-12-24 09:34:01 -08:00
Chao Sun
c3efc299fe HDFS-14116. [SBN read] Fix class cast error in NNThroughputBenchmark with ObserverReadProxyProvider. Contributed by Chao Sun. 2018-12-24 09:34:01 -08:00
Erik Krogen
ef3e1929dd HDFS-14146. [SBN read] Handle exceptions from and prevent handler threads from blocking within internalQueueCall. Contributed by Chao Sun. 2018-12-24 09:34:01 -08:00
Erik Krogen
b73fb70f97 HDFS-13873. [SBN read] ObserverNode should reject read requests when it is too far behind. Contributed by Konstantin Shvachko. 2018-12-24 09:34:01 -08:00
Chen Liang
b8ad6c85a5 HDFS-14142. Move ipfailover config key out of HdfsClientConfigKeys. Contributed by Chen Liang. 2018-12-24 09:34:01 -08:00
Erik Krogen
2904aa3808 HDFS-14131. [SBN read] Create user guide for Consistent Reads from Observer feature. Contributed by Chao Sun. 2018-12-24 09:34:01 -08:00
Chen Liang
71cf66e29b HDFS-14120. [SBN read] ORFPP should also clone DT for the virtual IP. Contributed by Chen Liang. 2018-12-24 09:34:01 -08:00
Konstantin V Shvachko
3bb92a1d9a HDFS-14094. [SBN read] Fix the order of logging arguments in ObserverReadProxyProvider. Contributed by Ayush Saxena. 2018-12-24 09:34:01 -08:00
Chao Sun
cbc2f932eb HDFS-14067. [SBN read] Allow manual failover between standby and observer. Contributed by Chao Sun. 2018-12-24 09:34:01 -08:00
Chen Liang
a3aab48df0 HDFS-14017. [SBN read] ObserverReadProxyProviderWithIPFailover should work with HA configuration. Contributed by Chen Liang. 2018-12-24 09:34:01 -08:00
Chen Liang
652b257478 HDFS-14035. NN status discovery does not leverage delegation token. Contributed by Chen Liang. 2018-12-24 09:34:01 -08:00
Chao Sun
1a9ba9616a HDFS-14016. [SBN read] ObserverReadProxyProvider should enable observer read by default. Contributed by Chen Liang. 2018-12-24 09:34:01 -08:00
Chao Sun
8c49135078 HDFS-13924. [SBN read] Handle BlockMissingException when reading from observer. Contributed by Chao Sun. 2018-12-24 09:34:01 -08:00
Konstantin V Shvachko
b6f20c36c2 HDFS-13925. Unit Test for transitioning between different states. Contributed by Sherwood Zheng. 2018-12-24 09:34:00 -08:00
Konstantin V Shvachko
1c4265d7bc HDFS-13523. Support observer nodes in MiniDFSCluster. Contributed by Konstantin Shvachko. 2018-12-24 09:34:00 -08:00
Konstantin V Shvachko
b5b9b77707 HDFS-13961. [SBN read] TestObserverNode refactoring. Contributed by Konstantin Shvachko. 2018-12-24 09:34:00 -08:00
Chen Liang
a65bb97f5d HDFS-13791. Limit logging frequency of edit tail related statements. Contributed by Erik Krogen. 2018-12-24 09:34:00 -08:00
Erik Krogen
b74a7dbf88 HDFS-13898. [SBN read] Throw retriable exception for getBlockLocations when ObserverNameNode is in safemode. Contributed by Chao Sun. 2018-12-24 09:34:00 -08:00
Erik Krogen
25b63e8da8 HDFS-13749. [SBN read] Use getServiceStatus to discover observer namenodes. Contributed by Chao Sun. 2018-12-24 09:34:00 -08:00
Konstantin V Shvachko
28820fb6b1 HDFS-13778. [SBN read] TestStateAlignmentContextWithHA should use real ObserverReadProxyProvider instead of AlignmentContextProxyProvider. Contributed by Konstantin Shvachko and Plamen Jeliazkov. 2018-12-24 09:34:00 -08:00
Chen Liang
d2e85b0b6d HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang. 2018-12-24 09:34:00 -08:00
Erik Krogen
aa42fb0db7 HDFS-13779. [SBN read] Implement proper failover and observer failure handling logic for for ObserverReadProxyProvider. Contributed by Erik Krogen. 2018-12-24 09:34:00 -08:00
Konstantin V Shvachko
f9fc01cd7f HDFS-13782. ObserverReadProxyProvider should work with IPFailoverProxyProvider. Contributed by Konstantin Shvachko. 2018-12-24 09:34:00 -08:00
Konstantin V Shvachko
25d8e39b5c HDFS-13851. Remove AlignmentContext from AbstractNNFailoverProxyProvider. Contributed by Konstantin Shvachko. 2018-12-24 09:34:00 -08:00
Chen Liang
66ec3a67e8 HDFS-13767. Add msync server implementation. Contributed by Chen Liang. 2018-12-24 09:34:00 -08:00
Chao Sun
4c9baba066 HDFS-13789. Reduce logging frequency of QuorumJournalManager#selectInputStreams. Contributed by Erik Krogen. 2018-12-24 09:34:00 -08:00
Erik Krogen
eae0a5d54a HDFS-13688. [SBN read] Introduce msync API call. Contributed by Chen Liang. 2018-12-24 09:34:00 -08:00
Erik Krogen
1e22f2bfbb HDFS-13610. [SBN read] Edit Tail Fast Path Part 4: Cleanup. Integration test, documentation, remove unnecessary dummy sync, minors fixups. Contributed by Erik Krogen. 2018-12-24 09:34:00 -08:00
Chao Sun
64b7cf59bd HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun. 2018-12-24 09:33:59 -08:00
Konstantin V Shvachko
0105d7f597 HDFS-13706. [SBN read] Rename client context to ClientGSIContext. Contributed by Konstantin Shvachko. 2018-12-24 09:33:59 -08:00
Erik Krogen
00e99c6594 HDFS-13609. [SBN read] Edit Tail Fast Path Part 3: NameNode-side changes to support tailing edits via RPC. Contributed by Erik Krogen. 2018-12-24 09:33:59 -08:00
Erik Krogen
151c8ddbe4 HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen. 2018-12-24 09:33:59 -08:00
Erik Krogen
c81ac2ff02 HDFS-13607. [SBN read] Edit Tail Fast Path Part 1: Enhance JournalNode with an in-memory cache of recent edit transactions. Contributed by Erik Krogen. 2018-12-24 09:33:59 -08:00
Plamen Jeliazkov
e880660a20 HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov. 2018-12-24 09:33:59 -08:00
Erik Krogen
a109f2b32f HDFS-13578. [SBN read] Add ReadOnly annotation to methods in ClientProtocol. Contributed by Chao Sun. 2018-12-24 09:33:59 -08:00
Erik Krogen
091ad974cd HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2018-12-24 09:33:59 -08:00
Erik Krogen
9bf0696c73 HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov. 2018-12-24 09:33:59 -08:00
Plamen Jeliazkov
c9d73437e8 HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov. 2018-12-24 09:33:59 -08:00
Chao Sun
3a78e5ffd8 HDFS-12975. [SBN read] Changes to the NameNode to support reads from standby. Contributed by Chao Sun. 2018-12-24 09:33:59 -08:00
Surendra Singh Lilhore
686fcd4db3 HDFS-14046. In-Maintenance ICON is missing in datanode info page. Contributed by Ranith Sardar. 2018-12-24 11:02:36 +05:30
Surendra Singh Lilhore
d944d5ec46 HDFS-14165. In NameNode UI under DataNode tab ,the Capacity column is Non-Aligned. Contributed by Shubham Dewan. 2018-12-23 15:06:12 +05:30
Yiqun Lin
feb2664ac4 HDFS-13946. Log longest FSN write/read lock held stack trace. 2018-12-22 23:09:59 +08:00
Surendra Singh Lilhore
788e7473a4 HDFS-14096. [SPS] : Add Support for Storage Policy Satisfier in ViewFs. Contributed by Ayush Saxena. 2018-12-17 11:24:57 +05:30
Giovanni Matteo Fumarola
00d5e631b5 HDFS-14006. Refactor name node to allow different token verification implementations. Contributed by CR Hota. 2018-12-14 11:10:54 -08:00
Giovanni Matteo Fumarola
f858f18554 HDFS-14088. RequestHedgingProxyProvider can throw NullPointerException when failover due to no lock on currentUsedProxy. Contributed by Yuxuan Wang. 2018-12-14 11:04:36 -08:00
Giovanni Matteo Fumarola
cb3382f667 HDFS-14145. TestBlockStorageMovementAttemptedItems#testNoBlockMovementAttemptFinishedReportAdded fails sporadically in Trunk. Contributed by Ayush Saxena. 2018-12-14 10:59:02 -08:00
Zsolt Venczel
aa12859890 HDFS-14121. Log message about the old hosts file format is misleading 
(Contributed by Zsolt Venczel via Daniel Templeton)

Change-Id: I7ff548f6c82e0aeb08a7a50ca7c2c827db8726bb
 2018-12-14 13:54:57 +01:00
Akira Ajisaka
ca379e1c43
HDFS-13970. Use MultiMap for CacheManager Directives to simplify the code. Contributed by BELUGA BEHR. 2018-12-14 14:36:26 +09:00
Daniel Templeton
4aa0609fb0 HDFS-13985. Clearer error message for ReplicaNotFoundException 
(Contributed by Adam Antal via Daniel Templeton)

Change-Id: I68ae7a5bedecbc204c161fc3b3ce8878fe5d493d
 2018-12-13 16:21:45 +01:00
Giovanni Matteo Fumarola
561b61cfb1 HDFS-14144. TestPred fails in Trunk. Contributed by Ayush Saxena. 2018-12-12 11:16:16 -08:00
Giovanni Matteo Fumarola
60af851e59 HDFS-14137. TestMaintenanceState fails with ArrayIndexOutOfBound Exception. Contributed by Ayush Saxena. 2018-12-11 11:22:32 -08:00
Vinayakumar B
39dc7345b8 HDFS-14124. EC : Support EC Commands (set/get/unset EcPolicy) via WebHdfs. Contributed by Ayush Saxena. 2018-12-11 17:59:04 +05:30
Sean Mackrory
80e59e7876 HDFS-14101. Fixing underflow error in test. Contributed by Zsolt Venczel. 2018-12-10 14:03:08 -07:00
Inigo Goiri
8fc0d04517 HDFS-14001. [PROVIDED Storage] bootstrapStandby should manage the InMemoryAliasMap. Contributed by Virajith Jalaparti. 2018-12-07 18:30:36 -08:00
Vinayakumar B
c03024a530 HDFS-14113. EC : Add Configuration to restrict UserDefined Policies. Contributed by Ayush Saxena. 2018-12-06 18:20:28 +05:30
Giovanni Matteo Fumarola
69489ff2d1 HDFS-14119. Improve GreedyPlanner Parameter Logging. Contributed by Beluga Behr. 2018-12-03 12:55:52 -08:00
Adam Antal
fb10803dfa HDFS-13818. Extend OIV to detect FSImage corruption. Contributed by Adam Antal. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-12-03 10:34:39 -08:00
Kitti Nanasi
dd5e7c6b72 HDFS-12946. Add a tool to check rack configuration against EC policies. Contributed by Kitti Nanasi. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-12-03 10:01:09 -08:00
Surendra Singh Lilhore
042c8ef593 HDFS-14075. Terminate the namenode when failed to start log segment. Contributed by Ayush Saxena. 2018-12-02 12:31:08 +05:30
Giovanni Matteo Fumarola
99e201dfe2 HDFS-14105. Replace TreeSet in NamenodeFsck with HashSet. Contributed by Beluga Behr. 2018-11-30 11:07:11 -08:00
Giovanni Matteo Fumarola
b09cfad432 HDFS-14106. Refactor NamenodeFsck#copyBlock. Contributed by Beluga Behr. 2018-11-30 10:47:59 -08:00
Yiqun Lin
0e36e935d9 HDFS-13870. WebHDFS: Document ALLOWSNAPSHOT and DISALLOWSNAPSHOT API doc. Contributed by Siyao Meng. 2018-11-30 11:31:34 +08:00
Tsz Wo Nicholas Sze
0081b02e35 HDFS-14112. Avoid recursive call to external authorizer for getContentSummary. 2018-11-29 13:55:21 -08:00
Brahma Reddy Battula
f534736867 HDFS-14095. EC: Track Erasure Coding commands in DFS statistics. Contributed by Ayush Saxena. 2018-11-30 00:18:27 +05:30
Ewan Higgs
c1d24f8483
HDFS-13713. Add specification of Multipart Upload API to FS specification, with contract tests. 
Contributed by Ewan Higgs and Steve Loughran.
 2018-11-29 15:12:17 +00:00
Giovanni Matteo Fumarola
300f560fcc HDFS-14102. Performance improvement in BlockPlacementPolicyDefault. Contributed by Beluga Behr. 2018-11-28 11:33:22 -08:00
Giovanni Matteo Fumarola
4ca3a6b21a HDFS-14108. Performance improvement in BlockManager Data Structures. Contributed by Beluga Behr. 2018-11-28 11:25:47 -08:00
Vinayakumar B
b098281454 HDFS-13816. dfs.getQuotaUsage() throws NPE on non-existent dir instead of FileNotFoundException. Contributed by Vinayakumar B. 2018-11-26 18:22:32 +05:30
Brahma Reddy Battula
892b33e054 HDFS-14064. WEBHDFS: Support Enable/Disable EC Policy. Contributed by Ayush Saxena. 2018-11-22 00:13:56 +05:30
Akira Ajisaka
1a00b4e325
Add 2.9.2 release notes and changes documents. 2018-11-20 14:25:18 +09:00
Daniel Templeton
e56d9f2618 HDFS-14015. Improve error handling in hdfsThreadDestructor in native thread local storage 
Change-Id: Ida1e888c9231b9e46081338e3a206d8f6faabd36
 2018-11-16 16:24:40 -08:00
Weiwei Yang
ad5256e44d HDFS-14063. Support noredirect param for CREATE/APPEND/OPEN/GETFILECHECKSUM in HttpFS. Contributed by Íñigo Goiri. 2018-11-17 01:12:22 +08:00
Vinayakumar B
68d5dfdc78 HDFS-14056. Fix error messages in HDFS-12716. Contributed by Ayush Saxena. 2018-11-16 18:05:10 +05:30
Vinayakumar B
8571507efa HDFS-13963. NN UI is broken with IE11. Contributed by Ayush Saxena. 2018-11-16 17:26:36 +05:30
Arpit Agarwal
1e15c7e855 HADOOP-15936. [JDK 11] MiniDFSClusterManager & MiniHadoopClusterManager compilation fails due to the usage of '_' as identifier. Contributed by Zsolt Venczel. 2018-11-15 12:42:31 -08:00
Inigo Goiri
460a94a10f HDFS-14045. Use different metrics in DataNode to better measure latency of heartbeat/blockReports/incrementalBlockReports of Active/Standby NN. Contributed by Jiandan Yang. 2018-11-15 10:58:57 -08:00
Inigo Goiri
993c2140cc HDFS-14054. TestLeaseRecovery2: testHardLeaseRecoveryAfterNameNodeRestart2 and testHardLeaseRecoveryWithRenameAfterNameNodeRestart are flaky. Contributed by Zsolt Venczel. 2018-11-15 09:29:14 -08:00
Xiao Chen
9da6054ca4 Revert "HDFS-13732. ECAdmin should print the policy name when an EC policy is set. Contributed by Zsolt Venczel." 
This reverts commit 7dc79a8b5b.
 2018-11-13 12:44:25 -08:00
Brahma Reddy Battula
e7b63baca1 HDFS-14070. Refactor NameNodeWebHdfsMethods to allow better extensibility. Contributed by CR Hota 2018-11-13 12:45:44 +05:30
Arpit Agarwal
b6d4e19f34 HDFS-14065. Failed Storage Locations shows nothing in the Datanode Volume Failures. Contributed by Ayush Saxena. 2018-11-12 15:31:42 -08:00
Kitti Nanasi
8d99648c20 HDFS-14039. ec -listPolicies doesn't show correct state for the default policy when the default is not RS(6,3). Contributed by Kitti Nanasi. 
Signed-off-by: Xiao Chen <xiao@apache.org>
 2018-11-08 10:01:19 -08:00
Jason Lowe
0b6625a973 MAPREDUCE-7148. Fast fail jobs when exceeds dfs quota limitation. Contributed by Wang Yan 2018-11-07 08:20:49 -06:00
Inigo Goiri
887244de4a HDFS-14048. DFSOutputStream close() throws exception on subsequent call after DataNode restart. Contributed by Erik Krogen. 2018-11-06 11:18:15 -08:00
Inigo Goiri
f0ce072934 HDFS-14051. Refactor NameNodeHttpServer#initWebHdfs to specify local keytab. Contributed by CR Hota. 2018-11-06 10:05:58 -08:00
Daniel Templeton
349168c4b3 HDFS-14047. [libhdfs++] Fix hdfsGetLastExceptionRootCause bug in test_libhdfs_threaded.c 
(Contributed by Antoli Shein via Daniel Templeton)

Change-Id: Ia5546fd5f8270f285c373b9023a9bd9835059845
 2018-11-06 09:22:27 -08:00
Akira Ajisaka
6430c9890a
HADOOP-15904. [JDK 11] Javadoc build failed due to bad use of '>' in hadoop-hdfs. 2018-11-06 18:42:38 +09:00
Xiao Chen
ffc9c50e07 HDFS-14053. Provide ability for NN to re-replicate based on topology changes. Contributed by Hrishikesh Gadre. 2018-11-05 21:38:39 -08:00
Inigo Goiri
f3296501e0 HDFS-14043. Tolerate corrupted seen_txid file. Contributed by Lukas Majercak. 2018-11-05 16:48:37 -08:00
Giovanni Matteo Fumarola
f3f5e7ad00 HDFS-14042. Fix NPE when PROVIDED storage is missing. Contributed by Virajith Jalaparti. 2018-11-05 11:02:31 -08:00
Yiqun Lin
4e3df75eb7 HDFS-14049. TestHttpFSServerWebServer fails on Windows because of missing winutils.exe. Contributed by Inigo Goiri. 2018-11-04 09:15:53 +08:00
Siyao Meng
8fe85af63b HDFS-13996. Make HttpFS' ACLs RegEx configurable. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-11-01 17:36:50 -07:00
Pranay Singh
d98b881e9a HDFS-14008. NN should log snapshotdiff report. Contributed by Pranay Singh. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-11-01 17:26:00 -07:00
Sunil G
9c438abe52 HDFS-14033. [libhdfs++] Disable libhdfs++ build on systems that do not support thread_local. Contributed by Anatoli Shein. 2018-10-31 12:32:49 +05:30
Akira Ajisaka
fac9f91b29
HDFS-13942. [JDK10] Fix javadoc errors in hadoop-hdfs module. Contributed by Dinesh Chitlangia. 2018-10-31 14:43:58 +09:00
Xiao Chen
db7e636824 HDFS-14027. DFSStripedOutputStream should implement both hsync methods. 2018-10-29 19:06:15 -07:00
Wei-Chiu Chuang
63e7134df6 Revert "HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao." 
This reverts commit fb2b72e6fc.
 2018-10-28 17:04:59 -07:00
Anu Engineer
4f10d7e23f HDFS-14028. HDFS OIV temporary dir deletes folder. 
Contributed by Adam Antal.
 2018-10-26 16:41:23 -07:00
Inigo Goiri
1851d06eb3 HDFS-13959. TestUpgradeDomainBlockPlacementPolicy is flaky. Contributed by Ayush Saxena. 2018-10-26 09:29:12 -07:00
Arpit Agarwal
9b899f1ebd HDFS-14029. Sleep in TestLazyPersistFiles should be put into a loop. Contributed by Adam Antal. 2018-10-25 15:14:13 -07:00
Wei-Chiu Chuang
fb2b72e6fc HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao. 2018-10-25 09:33:31 -07:00
Ajay Kumar
97bd49fc36 HDFS-14026. Overload BlockPoolTokenSecretManager.checkAccess to make storageId and storageType optional. Contributed by Arpit Agarwal. 2018-10-24 21:55:42 -07:00
Inigo Goiri
936fc3f3c2 HDFS-14025. TestPendingReconstruction.testPendingAndInvalidate fails. Contributed by Ayush Saxena. 2018-10-24 16:18:43 -07:00
Inigo Goiri
c1874046e2 HDFS-14021. TestReconstructStripedBlocksWithRackAwareness#testReconstructForNotEnoughRacks fails intermittently. Contributed by Xiao Chen. 2018-10-24 09:20:38 -07:00
Chen Liang
635786a511 HDFS-13566. Add configurable additional RPC listener to NameNode. Contributed by Chen Liang. 2018-10-23 14:53:45 -07:00
Inigo Goiri
efdfe679d6 HDFS-14004. TestLeaseRecovery2#testCloseWhileRecoverLease fails intermittently in trunk. Contributed by Ayush Saxena. 2018-10-23 14:28:37 -07:00
Arpit Agarwal
c58811c77d HDFS-13941. make storageId in BlockPoolTokenSecretManager.checkAccess optional. Contributed by Ajay Kumar. 2018-10-22 14:44:28 -07:00
Inigo Goiri
f069d38c8d HDFS-13983. TestOfflineImageViewer crashes in windows. Contributed by Vinayakumar B. 2018-10-19 17:16:04 -07:00
Inigo Goiri
88cce32551 HDFS-9872. HDFS bytes-default configurations should accept multiple size units. Contributed by Yiqun Lin. 2018-10-19 10:38:04 -07:00
Inigo Goiri
8b64fbab1a HDFS-13994. Improve DataNode BlockSender waitForMinLength. Contributed by BELUGA BEHR. 2018-10-19 10:36:03 -07:00
Inigo Goiri
be1cffb085 HDFS-14002. TestLayoutVersion#testNameNodeFeatureMinimumCompatibleLayoutVersions fails. Contributed by Takanobu Asanuma. 2018-10-18 15:25:53 -07:00
Giovanni Matteo Fumarola
098d0b86fe HDFS-14005. RBF: Web UI update to bootstrap-3.3.7. Contributed by Inigo Goiri. 2018-10-18 14:25:17 -07:00
Inigo Goiri
0e56c883cd HDFS-14003. Fix findbugs warning in trunk for FSImageFormatPBINode. Contributed by Yiqun Lin. 2018-10-18 10:53:30 -07:00
Inigo Goiri
1e78dfca40 HDFS-13990. Synchronization Issue With HashResolver. Contributed by BELUGA BEHR. 2018-10-18 10:05:27 -07:00
Inigo Goiri
7347fa2df3 HDFS-14000. RBF: Documentation should reflect right scripts for v3.0 and above. Contributed by CR Hota. 2018-10-17 16:40:25 -07:00
Xiao Chen
533138718c HDFS-13662. TestBlockReaderLocal#testStatisticsForErasureCodingRead is flaky 2018-10-16 19:33:55 -07:00
Inigo Goiri
f90c64e624 HDFS-13993. TestDataNodeVolumeFailure#testTolerateVolumeFailuresAfterAddingMoreVolumes is flaky. Contributed by Ayush Saxena. 2018-10-16 11:22:57 -07:00
Xiao Chen
b6fc72a025 HADOOP-14445. Addendum: Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 2018-10-15 10:51:55 -07:00
Inigo Goiri
ee1c80ea32 HDFS-13987. RBF: Review of RandomResolver Class. Contributed by BELUGA BEHR. 2018-10-15 09:51:26 -07:00
Vinayakumar B
b60ca37914 Fix potential FSImage corruption. Contributed by Daryn Sharp. 2018-10-15 15:48:26 +05:30
Inigo Goiri
f63ee083db HDFS-13802. RBF: Remove FSCK from Router Web UI. Contributed by Fei Hui. 2018-10-12 10:10:12 -07:00
Xiao Chen
5ec86b445c HADOOP-14445. Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
Contributed by Daryn Sharp, Xiao Chen, Rushabh S Shah.
 2018-10-12 09:35:52 -07:00
Vinayakumar B
6e0e6daaf3 HDFS-13945. TestDataNodeVolumeFailure is Flaky. Contributed by Ayush Saxena. 2018-10-12 17:34:10 +05:30
Vinayakumar B
de44e1064f HDFS-13156. HDFS Block Placement Policy - Client Local Rack. Contributed by Ayush Saxena. 2018-10-12 17:27:23 +05:30
Shweta
c0118105b7 HDFS-13973. getErasureCodingPolicy should log path in audit event. Contributed by Shweta. 
Signed-off-by: Xiao Chen <xiao@apache.org>
 2018-10-11 20:39:41 -07:00
Siyao Meng
6dcfef79af HDFS-13878. HttpFS: Implement GETSNAPSHOTTABLEDIRECTORYLIST. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-10-11 15:01:50 -07:00
Inigo Goiri
3532aa3886 HDFS-13968. BlockReceiver Array-Based Queue. Contributed by BELUGA BEHR. 2018-10-11 13:49:39 -07:00
Steve Loughran
2addebb94f
HDFS-13951. HDFS DelegationTokenFetcher can't print non-HDFS tokens in a tokenfile. 
Contributed by Steve Loughran.
 2018-10-11 20:03:56 +01:00
Hrishikesh Gadre
bf3d591f0c HDFS-13956. iNotify should include information to identify a file as either replicated or erasure coded. Contributed by Hrishikesh Gadre. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-10-10 10:23:07 -07:00
Weiwei Yang
3ead525c71 HDFS-12459. Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-10-10 10:11:10 -07:00
Kitti Nanasi
10185d9a77 HDFS-13882. Set a maximum delay for retrying locateFollowingBlock. Contributed by Kitti Nanasi. 
Signed-off-by: Xiao Chen <xiao@apache.org>
 2018-10-10 08:55:32 -07:00
Nanda kumar
b39b802422 HDFS-13949. Correct the description of dfs.datanode.disk.check.timeout in hdfs-default.xml. Contributed by Toshihiro Suzuki. 2018-10-10 12:29:07 +05:30
Akira Ajisaka
6fa3feb577
HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter. 2018-10-10 10:16:57 +09:00
Inigo Goiri
605622c87b HDFS-11396. TestNameNodeMetadataConsistency#testGenerationStampInFuture timed out. Contributed by Ayush Saxena. 2018-10-09 17:03:21 -07:00
Inigo Goiri
a05bd1288c HDFS-13979. Review StateStoreFileSystemImpl Class. Contributed by BELUGA BEHR. 2018-10-09 12:00:21 -07:00
Yiqun Lin
d4626b4d18 HDFS-13967. HDFS Router Quota Class Review. Contributed by BELUGA BEHR. 2018-10-09 16:11:07 +08:00
Hrishikesh Gadre
08bb6c49a5 HDFS-13926. ThreadLocal aggregations for FileSystem.Statistics are incorrect with striped reads. 
Contributed by Xiao Chen, Hrishikesh Gadre.

Signed-off-by: Xiao Chen <xiao@apache.org>
 2018-10-08 20:31:57 -07:00
Yiqun Lin
1043795f7f HDFS-13962. Add null check for add-replica pool to avoid lock acquiring. Contributed by Surendra Singh Lilhore. 2018-10-09 10:33:13 +08:00
Inigo Goiri
3bfff0f79e HDFS-13964. RBF: TestRouterWebHDFSContractAppend fails with No Active Namenode under nameservice. Contributed by Ayush Saxena. 2018-10-06 10:40:49 -07:00
Inigo Goiri
73c660b43f HDFS-13958. Miscellaneous Improvements for FsVolumeSpi. Contributed by BELUGA BEHR. 2018-10-05 09:32:19 -07:00
Yiqun Lin
619e490333 HDFS-13957. Fix incorrect option used in description of InMemoryAliasMap. 2018-10-05 09:55:08 +08:00
Adam Antal
cc2babc1f7 HDFS-13950. ACL documentation update to indicate that ACL entries are capped by 32. Contributed by Adam Antal. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-10-04 15:22:44 -07:00
Siyao Meng
396ce0d9f4 HDFS-13877. HttpFS: Implement GETSNAPSHOTDIFF. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-10-04 15:17:47 -07:00
Inigo Goiri
1dc0adfac0 HDFS-13947. Review of DirectoryScanner Class. Contributed by BELUGA BEHR. 2018-10-03 11:19:57 -07:00
Akira Ajisaka
fa7f7078a7
HDFS-13944. [JDK10] Fix javadoc errors in hadoop-hdfs-rbf module. Contributed by Inigo Goiri. 2018-10-03 12:48:45 +09:00
Sunil G
58fa96b697 Changed version in trunk to 3.3.0-SNAPSHOT. 2018-10-02 22:41:41 +05:30
Yiqun Lin
5689355783 HDFS-13768. Adding replicas to volume map makes DataNode start slowly. Contributed by Surendra Singh Lilhore. 2018-10-02 09:43:14 +08:00
Takanobu Asanuma
f6c5ef9903 HDFS-13943. [JDK10] Fix javadoc errors in hadoop-hdfs-client module. Contributed by Akira Ajisaka. 2018-10-02 09:49:48 +09:00
Takanobu Asanuma
e2113500df HDFS-8196. Post enabled Erasure Coding Policies on NameNode UI. Contributed by Kitti Nanasi and Kai Sasaki. 2018-09-29 00:53:35 +09:00
Akira Ajisaka
284aca8b63
HDFS-13938. Add a missing break in BaseTestHttpFSWith. Contributed by Siyao Meng. 2018-09-28 15:44:46 +09:00
Akira Ajisaka
f9c0221623
HDFS-13932. [JDK 11] Casts to BlockStoragePolicy[] in unit tests raise ClassCastExceptions. Contributed by Andrew Purtell. 2018-09-27 11:44:42 +09:00
Brahma Reddy Battula
e0ff8e2c10 HDFS-13840. RBW Blocks which are having less GS should be added to Corrupt. Contributed by Brahma Reddy Battula 2018-09-26 23:44:16 +05:30
Inigo Goiri
6275b42870 HDFS-13927. Addendum: Improve TestDataNodeMultipleRegistrations#testDNWithInvalidStorageWithHA wait. Contributed by Ayush Saxena. 2018-09-26 09:50:09 -07:00
Siyao Meng
8de5c923b4 HDFS-13876. HttpFS: Implement ALLOWSNAPSHOT, DISALLOWSNAPSHOT. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-09-24 10:40:30 -07:00
Inigo Goiri
4758b4b6da HDFS-13927. Improve TestDataNodeMultipleRegistrations#testDNWithInvalidStorageWithHA wait. Contributed by Ayush Saxena. 2018-09-21 15:32:28 -07:00
Anu Engineer
6fc293fece HDFS-13892. Disk Balancer: Make execute command documentation better. 
Contributed by  Ranith Sardar.
 2018-09-19 20:48:41 -07:00
Arpit Agarwal
2eb597b151 HADOOP-15772. Remove the 'Path ... should be specified as a URI' warnings on startup. Contributed by Ayush Saxena. 2018-09-19 13:48:27 -07:00
Chen Liang
a30b4f9e71 HADOOP-15726. Create utility to limit frequency of log statements. Contributed by Erik Krogen. 2018-09-19 13:22:37 -07:00
Inigo Goiri
090272d7de HADOOP-15684. triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens. Contributed by Rong Tang. 2018-09-19 13:00:30 -07:00
Inigo Goiri
56e0d635e0 HDFS-13908. TestDataNodeMultipleRegistrations is flaky. Contributed by Ayush Saxena. 2018-09-19 09:30:25 -07:00
Wei-Chiu Chuang
28ceb34a72 HDFS-13868. WebHDFS: GETSNAPSHOTDIFF API NPE when param "snapshotname" is given but "oldsnapshotname" is not. Contributed by Pranay Singh. 2018-09-19 03:12:20 -07:00
Shweta
27978bcb66 HDFS-13833. Improve BlockPlacementPolicyDefault's consider load logic. Contributed by Shweta. 
Signed-off-by: Xiao Chen <xiao@apache.org>
 2018-09-18 20:23:50 -07:00
Wei-Chiu Chuang
44857476fa HDFS-13886. HttpFSFileSystem.getFileStatus() doesn't return "snapshot enabled" bit. Contributed by Siyao Meng. 2018-09-18 15:33:02 -07:00
Mingliang Liu
6ff509c32a HDFS-11719. Arrays.fill() wrong index in BlockSender.readChecksum() exception handling. Contributed by Tao Zhang 2018-09-18 10:25:28 -07:00
Inigo Goiri
d1541932db HDFS-13844. Fix the fmt_bytes function in the dfs-dust.js. Contributed by yanghuafeng. 2018-09-17 14:42:03 -07:00
Anu Engineer
fdf5a3fd63 HDFS-13919. Documentation: Improper formatting in Disk Balancer for Settings. 
Contributed by Ayush Saxena.
 2018-09-17 10:08:23 -07:00
Wei-Chiu Chuang
a1de8cbac5 HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng. 2018-09-14 05:22:56 +08:00
Inigo Goiri
e1b242a984 HDFS-13914. Fix DN UI logs link broken when https is enabled after HDFS-13902. Contributed by Jianfei Jiang. 2018-09-13 09:16:58 -07:00
Daniel Templeton
78bd3b1db9 HDFS-13846. Safe blocks counter is not decremented correctly if the block is striped 
(Contributed by Kitti Nanasi via Daniel Templeton)

Change-Id: Id41747a67dc946fdf0dbde90643bb1ea7e9e0f70
 2018-09-12 11:18:55 -07:00
Brahma Reddy Battula
b2432d254c HDFS-13902. Add JMX, conf and stacks menus to the datanode page. Contributed by fengchuang. 2018-09-12 06:30:57 +05:30
Xiao Chen
8e54da1511 HDFS-13051. Fix dead lock during async editlog rolling if edit queue is full. Contributed by Daryn Sharp. 2018-09-10 22:34:23 -07:00
Brahma Reddy Battula
96892c469b HDFS-13237. [Documentation] RBF: Mount points across multiple subclusters. Contributed Íñigo Goiri 2018-09-11 10:12:34 +05:30
Yiqun Lin
0da49642fc HDFS-13884. Improve the description of the setting dfs.image.compress. Contributed by Ryan Wu. 2018-09-10 13:57:36 +08:00
Vinayakumar B
eef3bafae8 HDFS-13895. EC: Fix Intermittent Failure in TestDFSStripedOutputStreamWithFailureWithRandomECPolicy. Contributed by Ayush Saxena. 2018-09-10 09:22:59 +05:30
Vinayakumar B
30eceec342 HDFS-13806. EC: No error message for unsetting EC policy of the directory inherits the erasure coding policy from an ancestor directory. Contributed by Ayush Saxena. 2018-09-10 09:10:51 +05:30
Brahma Reddy Battula
8944407d93 HDFS-13862. RBF: Router logs are not capturing few of the dfsrouteradmin commands. Contributed by Ayush Saxena 2018-09-09 23:20:26 +05:30
Hrishikesh Gadre
335a8139f5 HDFS-13820. Add an ability to disable CacheReplicationMonitor. Contributed by Hrishikesh Gadre. 
Signed-off-by: Xiao Chen <xiao@apache.org>
 2018-09-07 14:59:06 -07:00
Sean Mackrory
410dd3faa5 HDFS-13744. OIV tool should better handle control characters present in file or directory names. Contributed by Zsolt Venczel. 2018-09-07 12:39:33 -06:00
Inigo Goiri
527288ef89 HDFS-13836. RBF: Handle mount table znode with null value. Contributed by yanghuafeng. 2018-09-06 16:47:54 -07:00
Giovanni Matteo Fumarola
eca1a4bfe9 HDFS-13695. Move logging to slf4j in HDFS package. Contributed by Ian Pickering. 2018-09-06 14:48:00 -07:00
Giovanni Matteo Fumarola
9af96d4ed4 HADOOP-15707. Add IsActiveServlet to be used for Load Balancers. Contributed by Lukas Majercak. 2018-09-05 10:50:25 -07:00
Wei-Chiu Chuang
e780556ae9 HADOOP-15696. KMS performance regression due to too many open file descriptors after Jetty migration. Contributed by Wei-Chiu Chuang. 2018-09-05 09:52:35 -07:00
Yiqun Lin
9315db5f5d HDFS-13815. RBF: Add check to order command. Contributed by Ranith Sardar. 2018-09-05 23:33:27 +08:00
Hrishikesh Gadre
6ccb809c2d HDFS-13812. Fix the inconsistent default refresh interval on Caching documentation. Contributed by Hrishikesh Gadre. 
Signed-off-by: Xiao Chen <xiao@apache.org>
 2018-09-04 21:53:42 -07:00
Inigo Goiri
54f2044595 HDFS-13857. RBF: Choose to enable the default nameservice to read/write files. Contributed by yanghuafeng. 2018-09-04 12:17:17 -07:00
Kitti Nanasi
6e5ffb74dd HDFS-13885. Add debug logs in dfsclient around decrypting EDEK. Contributed by Kitti Nanasi. 
Signed-off-by: Xiao Chen <xiao@apache.org>
 2018-09-03 22:37:37 -07:00
Vinayakumar B
3801436e49 HDFS-13774. EC: 'hdfs ec -getPolicy' is not retrieving policy details when the special REPLICATION policy set on the directory. Contributed by Ayush Saxena. 2018-09-03 14:37:57 +05:30
Vinayakumar B
780df9034f HDFS-13867. RBF: Add validation for max arguments for Router admin ls, clrQuota, setQuota, rm and nameservice commands. Contributed by Ayush Saxena. 2018-09-03 14:28:31 +05:30
Wei-Chiu Chuang
8aa6c4f079 Revert "HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng." 
This reverts commit 26c2a97c56.
 2018-08-30 11:44:20 -07:00
Vinayakumar B
c36d69a7b3 HDFS-13027. Handle possible NPEs due to deleted blocks in race condition. Contributed by Vinayakumar B. 
(cherry picked from commit 65977e5d8124be2bc208af25beed934933f170b3)
 2018-08-30 22:15:51 +05:30
Yiqun Lin
582cb10ec7 HDFS-13863. FsDatasetImpl should log DiskOutOfSpaceException. Contributed by Fei Hui. 2018-08-30 11:21:13 +08:00
Yiqun Lin
a0ebb6b39f HDFS-13634. RBF: Configurable value in xml for async connection request queue size. Contributed by CR Hota. 2018-08-29 16:15:22 +08:00
Brahma Reddy Battula
64ad0298d4 HDFS-13854. RBF: The ProcessingAvgTime and ProxyAvgTime should display by JMX with ms unit. Contributed by yanghuafeng. 2018-08-29 08:29:50 +05:30
Zsolt Venczel
3e18b957eb HDFS-13731. ReencryptionUpdater fails with ConcurrentModificationException during processCheckpoints. Contributed by Zsolt Venczel. 2018-08-28 15:13:43 -07:00
Shweta
33f42efc94 HDFS-13837. Enable debug log for LeaseRenewer in TestDistributedFileSystem. Contributed by Shweta. 2018-08-28 13:56:32 -07:00
Brahma Reddy Battula
cb9d371ae2 HDFS-13861. RBF: Illegal Router Admin command leads to printing usage for all commands. Contributed by Ayush Saxena. 2018-08-29 00:29:05 +05:30
Vinayakumar B
75691ad600 HDFS-13858. RBF: Add check to have single valid argument to safemode command. Contributed by Ayush Saxena. 2018-08-28 09:21:07 +05:30
Wei-Chiu Chuang
26c2a97c56 HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng. 2018-08-27 16:02:35 -07:00
Giovanni Matteo Fumarola
7b1fa5693e HDFS-13849. Migrate logging to slf4j in hadoop-hdfs-httpfs, hadoop-hdfs-nfs, hadoop-hdfs-rbf, hadoop-hdfs-native-client. Contributed by Ian Pickering. 2018-08-27 10:18:05 -07:00
Yiqun Lin
b9b964d253 HDFS-13831. Make block increment deletion number configurable. Contributed by Ryan Wu. 2018-08-27 14:55:46 +08:00
Konstantin V Shvachko
a4121c71c2 HDFS-13848. Refactor NameNode failover proxy providers. Contributed by Konstantin Shvachko. 2018-08-24 18:27:30 -07:00
Surendra Singh Lilhore
96c4575d73 HDFS-13805. Journal Nodes should allow to format non-empty directories with -force option. Contributed by Surendra Singh Lilhore. 2018-08-24 08:14:57 +05:30
Yiqun Lin
81847392ba HDFS-13821. RBF: Add dfs.federation.router.mount-table.cache.enable so that users can disable cache. Contributed by Fei Hui. 2018-08-22 11:43:40 +08:00
Vinayakumar B
770d9d9bb6 HDFS-13772. Erasure coding: Unnecessary NameNode Logs displaying for Enabling/Disabling Erasure coding policies which are already enabled/disabled. Contributed by Ayush Saxena 2018-08-21 09:33:19 +05:30
Yiqun Lin
01ff817814 HDFS-13750. RBF: Router ID in RouterRpcClient is always null. Contributed by Takanobu Asanuma. 2018-08-20 23:01:59 +08:00
Brahma Reddy Battula
6425ed27ea HDFS-13790. RBF: Move ClientProtocol APIs to its own module. Contributed by Chao Sun. 2018-08-20 12:20:29 +05:30
Giovanni Matteo Fumarola
79c97f6a0b HADOOP-14624. Add GenericTestUtils.DelayAnswer that accept slf4j logger API. Contributed by Ian Pickering and Wenxin He. 2018-08-17 14:40:00 -07:00
Jason Lowe
a17eed1b87 HDFS-13822. speedup libhdfs++ build (enable parallel build). Contributed by Allen Wittenauer and Pradeep Ambati 2018-08-17 12:25:36 -05:00
Xiaoyu Yao
fb5b3dce61 Revert "HDFS-13790. RBF: Move ClientProtocol APIs to its own module. Contributed by Chao Sun." 
This reverts commit fa121eb66b.
 2018-08-17 08:01:44 -07:00
Brahma Reddy Battula
fa121eb66b HDFS-13790. RBF: Move ClientProtocol APIs to its own module. Contributed by Chao Sun. 2018-08-17 15:22:55 +05:30
Márton Elek
77b015000a HADOOP-8807. Update README and website to reflect HADOOP-8662. Contributed by Andras Bokor. 2018-08-17 11:10:29 +02:00
Xiao Chen
c67b0650ea HDFS-13747. Statistic for list_located_status is incremented incorrectly by listStatusIterator. Contributed by Antal Mihalyi. 2018-08-16 23:14:21 -07:00
Wei-Chiu Chuang
1290e3c647 HDFS-10240. Race between close/recoverLease leads to missing block. Contributed by Jinglun, zhouyingchao and Wei-Chiu Chuang. 2018-08-16 16:29:38 -07:00
Daniel Templeton
8512e1a91b HDFS-13746. Still occasional "Should be different group" failure in TestRefreshUserMappings#testGroupMappingRefresh 
(Contributed by Siyao Meng via Daniel Templeton)

Change-Id: I9fad1537ace38367a463d9fe67aaa28d3178fc69
 2018-08-16 15:00:45 -07:00
Yiqun Lin
6df606f1b4 HDFS-13829. Remove redundant condition judgement in DirectoryScanner#scan. Contributed by liaoyuxiangqin. 2018-08-16 18:44:18 +08:00
Xiao Chen
7dc79a8b5b HDFS-13732. ECAdmin should print the policy name when an EC policy is set. Contributed by Zsolt Venczel. 2018-08-15 13:53:47 -07:00
Xiao Chen
ef02f9b233 HDFS-13217. Audit log all EC policy names during addErasureCodingPolicies. Contributed by liaoyuxiangqin. 2018-08-15 09:23:05 -07:00
Akira Ajisaka
3e3963b035
HADOOP-15552. Move logging APIs over to slf4j in hadoop-tools - Part2. Contributed by Ian Pickering. 2018-08-16 00:31:59 +09:00
Daniel Templeton
4a5006b1d0 HDFS-13819. TestDirectoryScanner#testDirectoryScannerInFederatedCluster is flaky 
Change-Id: I1cea6e67fcec72702ad202775dee3373261ac5cd
 2018-08-14 17:22:15 -07:00
Wei-Chiu Chuang
363bd16e31 HADOOP-14212. Expose SecurityEnabled boolean field in JMX for other services besides NameNode. Contributed by Adam Antal. 2018-08-14 17:19:00 -07:00
Xiao Chen
cede33997f HDFS-13788. Update EC documentation about rack fault tolerance. Contributed by Kitti Nanasi. 2018-08-14 11:57:22 -07:00
Wei-Chiu Chuang
61a9b4f58b HDFS-13758. DatanodeManager should throw exception if it has BlockRecoveryCommand but the block is not under construction. Contributed by chencan. 2018-08-14 11:51:27 -07:00
Wei-Chiu Chuang
4023eeba05 HDFS-13738. fsck -list-corruptfileblocks has infinite loop if user is not privileged. Contributed by Yuen-Kuei Hsueh. 2018-08-13 17:36:13 -07:00
Wei-Chiu Chuang
23854443ef HDFS-13813. Exit NameNode if dangling child inode is detected when saving FsImage. Contributed by Siyao Meng. 2018-08-13 16:12:37 -07:00
Arpit Agarwal
f760a544a7 HDFS-13823. NameNode UI : "Utilities -> Browse the file system -> open a file -> Head the file" is not working. Contributed by Nanda kumar. 2018-08-13 11:35:19 -07:00
drankye
475bff6e8e HDFS-13668. FSPermissionChecker may throws AIOOE when check inode permission. Contributed by He Xiaoqiao. 2018-08-13 17:32:56 +08:00
Uma Maheswara Rao Gangumalla
3ac07b720b HDFS-13808: [SPS]: Remove unwanted FSNamesystem #isFileOpenedForWrite() and #getFileInfo() function. Contributed by Rakesh R. 2018-08-12 03:06:07 -07:00
Uma Maheswara Rao G
39ed3a66db HDFS-13076: [SPS]: Cleanup work for HDFS-10285 merge. Contributed by Rakesh R. 2018-08-12 03:06:06 -07:00
Rakesh Radhakrishnan
dfcb331ba3 HDFS-13076: [SPS]: Addendum. Resolve conflicts after rebasing branch to trunk. Contributed by Rakesh R. 2018-08-12 03:06:06 -07:00
Rakesh Radhakrishnan
db3f227d8a HDFS-13076: [SPS]: Resolve conflicts after rebasing HDFS-10285 branch to trunk. Contributed by Rakesh R. 2018-08-12 03:06:06 -07:00
Uma Maheswara Rao G
66e8f9b315 HDFS-13381 : [SPS]: Use DFSUtilClient#makePathFromFileId() to prepare satisfier file path. Contributed by Rakesh R. 2018-08-12 03:06:06 -07:00
Rakesh Radhakrishnan
2acc50b826 HDFS-13165: [SPS]: Collects successfully moved block details via IBR. Contributed by Rakesh R. 2018-08-12 03:06:05 -07:00
Surendra Singh Lilhore
75ccc1396b HDFS-13166: [SPS]: Implement caching mechanism to keep LIVE datanodes to minimize costly getLiveDatanodeStorageReport() calls. Contributed by Rakesh R. 2018-08-12 03:06:05 -07:00
Rakesh Radhakrishnan
8467ec24fb HDFS-13110: [SPS]: Reduce the number of APIs in NamenodeProtocol used by external satisfier. Contributed by Rakesh R. 2018-08-12 03:06:05 -07:00
Uma Maheswara Rao G
4402f3f855 HDFS-13097: [SPS]: Fix the branch review comments(Part1). Contributed by Surendra Singh. 2018-08-12 03:06:04 -07:00
Surendra Singh Lilhore
d3de4fb2a0 HDFS-13077. [SPS]: Fix review comments of external storage policy satisfier. Contributed by Rakesh R. 2018-08-12 03:06:04 -07:00
Rakesh Radhakrishnan
5845c36c16 HDFS-13050: [SPS]: Create start/stop script to start external SPS process. Contributed by Surendra Singh Lilhore. 2018-08-12 03:06:04 -07:00
Surendra Singh Lilhore
99594b48b8 HDFS-13075. [SPS]: Provide External Context implementation. Contributed by Uma Maheswara Rao G. 2018-08-12 03:06:04 -07:00
Uma Maheswara Rao G
3b83110d5e HDFS-13057: [SPS]: Revisit configurations to make SPS service modes internal/external/none. Contributed by Rakesh R. 2018-08-12 03:06:04 -07:00
Uma Maheswara Rao G
b0cb8d9bb4 HDFS-13033: [SPS]: Implement a mechanism to do file block movements for external SPS. Contributed by Rakesh R. 2018-08-12 03:06:03 -07:00
Rakesh Radhakrishnan
3159b39cf8 HDFS-13025. [SPS]: Implement a mechanism to scan the files for external SPS. Contributed by Uma Maheswara Rao G. 2018-08-12 03:06:03 -07:00
Rakesh Radhakrishnan
8d4f74e733 HDFS-12911. [SPS]: Modularize the SPS code and expose necessary interfaces for external/internal implementations. Contributed by Uma Maheswara Rao G 2018-08-12 03:06:03 -07:00
Surendra Singh Lilhore
05d4daf6ba HDFS-12982 : [SPS]: Reduce the locking and cleanup the Namesystem access. Contributed by Rakesh R. 2018-08-12 03:06:02 -07:00
Uma Maheswara Rao G
78420719eb HDFS-12955: [SPS]: Move SPS classes to a separate package. Contributed by Rakesh R. 2018-08-12 03:06:02 -07:00
Surendra Singh Lilhore
c561cb316e HDFS-12106: [SPS]: Improve storage policy satisfier configurations. Contributed by Surendra Singh Lilhore. 2018-08-12 03:06:02 -07:00
Rakesh Radhakrishnan
9b83f94f35 HDFS-12790: [SPS]: Rebasing HDFS-10285 branch after HDFS-10467, HDFS-12599 and HDFS-11968 commits. Contributed by Rakesh R. 2018-08-12 03:06:02 -07:00
Rakesh Radhakrishnan
68017e3349 HDFS-12310: [SPS]: Provide an option to track the status of in progress requests. Contributed by Surendra Singh Lilhore. 2018-08-12 03:06:02 -07:00
Surendra Singh Lilhore
5780f0624d HDFS-12556: [SPS] : Block movement analysis should be done in read lock. 2018-08-12 03:06:01 -07:00
Uma Maheswara Rao G
00eceed233 HDFS-12570: [SPS]: Refactor Co-ordinator datanode logic to track the block storage movements. Contributed by Rakesh R. 2018-08-12 03:06:01 -07:00
Uma Maheswara Rao G
bfd3f8bd8a HDFS-12291: [SPS]: Provide a mechanism to recursively iterate and satisfy storage policy of all the files under the given dir. Contributed by Surendra Singh Lilhore. 2018-08-12 03:06:01 -07:00
Uma Maheswara Rao G
7ea24fc06c HDFS-12225: [SPS]: Optimize extended attributes for tracking SPS movements. Contributed by Surendra Singh Lilhore. 2018-08-12 03:06:01 -07:00
Uma Maheswara Rao G
0e820f16af HDFS-12214: [SPS]: Fix review comments of StoragePolicySatisfier feature. Contributed by Rakesh R. 2018-08-12 03:06:01 -07:00
Uma Maheswara Rao G
3b601f2c0e HDFS-12152: [SPS]: Re-arrange StoragePolicySatisfyWorker stopping sequence to improve thread cleanup time. Contributed by Rakesh R. 2018-08-12 03:06:00 -07:00
Uma Maheswara Rao G
4bcf61c696 HDFS-12141: [SPS]: Fix checkstyle warnings. Contributed by Rakesh R. 2018-08-12 03:06:00 -07:00
Rakesh Radhakrishnan
9e82e5a86e HDFS-12146. [SPS]: Fix TestStoragePolicySatisfierWithStripedFile#testSPSWhenFileHasLowRedundancyBlocks. Contributed by Surendra Singh Lilhore. 2018-08-12 03:06:00 -07:00
Rakesh Radhakrishnan
68af4e199a HDFS-11874. [SPS]: Document the SPS feature. Contributed by Uma Maheswara Rao G 2018-08-12 03:06:00 -07:00
Uma Maheswara Rao G
5eb24ef7e7 HDFS-11264: [SPS]: Double checks to ensure that SPS/Mover are not running together. Contributed by Rakesh R. 2018-08-12 03:06:00 -07:00
Uma Maheswara Rao G
0b360b16ab HDFS-11965: [SPS]: Should give chance to satisfy the low redundant blocks before removing the xattr. Contributed by Surendra Singh Lilhore. 2018-08-12 03:06:00 -07:00
Uma Maheswara Rao G
00cf207192 HDFS-11670: [SPS]: Add CLI command for satisfy storage policy operations. Contributed by Surendra Singh Lilhore. 2018-08-12 03:05:59 -07:00
Rakesh Radhakrishnan
46f7523eb8 HDFS-11966. [SPS] Correct the log in BlockStorageMovementAttemptedItems#blockStorageMovementResultCheck. Contributed by Surendra Singh Lilhore. 2018-08-12 03:05:59 -07:00
Rakesh Radhakrishnan
20f9c62336 HDFS-11726. [SPS]: StoragePolicySatisfier should not select same storage type as source and destination in same datanode. Surendra Singh Lilhore. 2018-08-12 03:05:59 -07:00
Rakesh Radhakrishnan
d638a7dc03 HDFS-11762. [SPS]: Empty files should be ignored in StoragePolicySatisfier. Surendra Singh Lilhore. 2018-08-12 03:05:59 -07:00
Uma Maheswara Rao G
e53f89ccc3 HDFS-11883: [SPS] : Handle NPE in BlockStorageMovementTracker when dropSPSWork() called. Contributed by Surendra Singh Lilhore. 2018-08-12 03:05:59 -07:00
Uma Maheswara Rao G
5ce332dc9a HDFS-11695: [SPS]: Namenode failed to start while loading SPS xAttrs from the edits log. Contributed by Surendra Singh Lilhore. 2018-08-12 03:05:59 -07:00
Rakesh Radhakrishnan
6fe6c549e8 HDFS-11572. [SPS]: SPS should clean Xattrs when no blocks required to satisfy for a file. Contributed by Uma Maheswara Rao G 2018-08-12 03:05:58 -07:00
Uma Maheswara Rao G
695a402fca HDFS-11334: [SPS]: NN switch and rescheduling movements can lead to have more than one coordinator for same file blocks. Contributed by Rakesh R. 2018-08-12 03:05:58 -07:00
Uma Maheswara Rao G
11a08a7e8f HDFS-11338: [SPS]: Fix timeout issue in unit tests caused by longger NN down time. Contributed by Wei Zhou and Rakesh R 2018-08-12 03:05:58 -07:00
Uma Maheswara Rao G
c00be44463 HDFS-11336: [SPS]: Remove xAttrs when movements done or SPS disabled. Contributed by Yuanbo Liu. 2018-08-12 03:05:58 -07:00
Rakesh Radhakrishnan
9b15f5418d HDFS-11239: [SPS]: Check Mover file ID lease also to determine whether Mover is running. Contributed by Wei Zhou 2018-08-12 03:05:57 -07:00
Rakesh Radhakrishnan
e34331c31d HDFS-11243. [SPS]: Add a protocol command from NN to DN for dropping the SPS work and queues. Contributed by Uma Maheswara Rao G 2018-08-12 03:05:57 -07:00
Rakesh Radhakrishnan
f8fc96a66e HDFS-11309. [SPS]: chooseTargetTypeInSameNode should pass accurate block size to chooseStorage4Block while choosing target. Contributed by Uma Maheswara Rao G 2018-08-12 03:05:57 -07:00
Rakesh Radhakrishnan
681d2804c9 HDFS-11186. [SPS]: Daemon thread of SPS should start only in Active NN. Contributed by Wei Zhou 2018-08-12 03:05:57 -07:00
Uma Maheswara Rao G
6215e35bb6 HDFS-11150: [SPS]: Provide persistence when satisfying storage policy. Contributed by Yuanbo Liu 2018-08-12 03:05:57 -07:00
Uma Maheswara Rao G
df2b551e79 HDFS-11293: [SPS]: Local DN should be given preference as source node, when target available in same node. Contributed by Yuanbo Liu and Uma Maheswara Rao G 2018-08-12 03:05:56 -07:00
Rakesh Radhakrishnan
422f870607 HDFS-11289. [SPS]: Make SPS movement monitor timeouts configurable. Contributed by Uma Maheswara Rao G 2018-08-12 03:05:56 -07:00
Uma Maheswara Rao G
81ed364b63 HDFS-11193 : [SPS]: Erasure coded files should be considered for satisfying storage policy. Contributed by Rakesh R 2018-08-12 03:05:56 -07:00
Uma Maheswara Rao G
b7bed9f00a HDFS-11248: [SPS]: Handle partial block location movements. Contributed by Rakesh R 2018-08-12 03:05:55 -07:00
Uma Maheswara Rao G
d81611fe55 HDFS-11032: [SPS]: Handling of block movement failure at the coordinator datanode. Contributed by Rakesh R 2018-08-12 03:05:55 -07:00
Rakesh Radhakrishnan
5179d99b7e HDFS-11123. [SPS] Make storage policy satisfier daemon work on/off dynamically. Contributed by Uma Maheswara Rao G 2018-08-12 03:05:55 -07:00
Rakesh Radhakrishnan
cd5262aba0 HDFS-10885. [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on. Contributed by Wei Zhou 2018-08-12 03:05:55 -07:00
Rakesh Radhakrishnan
b07291e176 HDFS-11151. [SPS]: StoragePolicySatisfier should gracefully handle when there is no target node with the required storage type. Contributed by Rakesh R 2018-08-12 03:05:54 -07:00
Rakesh Radhakrishnan
b67ae6d9d7 HDFS-10802. [SPS]: Add satisfyStoragePolicy API in HdfsAdmin. Contributed by Yuanbo Liu 2018-08-12 03:05:54 -07:00
Uma Maheswara Rao G
19b5aee3e4 HDFS-11068: [SPS]: Provide unique trackID to track the block movement sends to coordinator. Contributed by Rakesh R 2018-08-12 03:05:54 -07:00
Rakesh Radhakrishnan
047526b4c2 HDFS-11029. [SPS]:Provide retry mechanism for the blocks which were failed while moving its storage at DNs. Contributed by Uma Maheswara Rao G 2018-08-12 03:05:53 -07:00
Rakesh Radhakrishnan
0f2d1ddc2c HDFS-10954. [SPS]: Provide mechanism to send blocks movement result back to NN from coordinator DN. Contributed by Rakesh R 2018-08-12 03:05:53 -07:00
Uma Maheswara Rao G
24add8c2f8 HDFS-10884: [SPS]: Add block movement tracker to track the completion of block movement future tasks at DN. Contributed by Rakesh R 2018-08-12 03:05:53 -07:00
Rakesh Radhakrishnan
e2a15d18bb HDFS-10801. [SPS]: Protocol buffer changes for sending storage movement commands from NN to DN. Contributed by Rakesh R 2018-08-12 03:05:53 -07:00
Uma Maheswara Rao G
1438da4944 HDFS-10800: [SPS]: Daemon thread in Namenode to find blocks placed in other storage than what the policy specifies. Contributed by Uma Maheswara Rao G 2018-08-12 03:05:53 -07:00
Kai Zheng
5692887395 HDFS-10794. [SPS]: Provide storage policy satisfy worker at DN for co-ordinating the block storage movement work. Contributed by Rakesh R 2018-08-12 03:05:52 -07:00
Chao Sun
a2a8c48699 Remove super user privilege requirement for NameNode.getServiceStatus. Contributed by Chao Sun. 2018-08-10 15:59:39 -07:00
Virajith Jalaparti
15241c6349 HDFS-13795. Fix potential NPE in InMemoryLevelDBAliasMapServer. 2018-08-10 09:38:40 -07:00
Chen Liang
5326a7906d HDFS-13735. Make QJM HTTP URL connection timeout configurable. Contributed by Chao Sun. 2018-08-09 10:11:47 -07:00
Wangda Tan
49c687608b Make 3.1.1 awared by other branches 2018-08-08 13:02:12 -07:00
Xiao Chen
9499df7b81 HDFS-13658. Expose HighestPriorityLowRedundancy blocks statistics. Contributed by Kitti Nanasi. 2018-08-08 10:40:20 -07:00
Márton Elek
36c0d742d4 HDFS-13447. Fix Typos - Node Not Chosen. Contributed by Beluga Behr. 2018-08-08 17:31:55 +02:00
Ewan Higgs
2ec97abb2e HADOOP-15576. S3A Multipart Uploader to work with S3Guard and encryption Originally contributed by Ewan Higgs with refinements by Steve Loughran. 2018-08-08 13:50:23 +02:00
Vinayakumar B
4203bc738c HDFS-13785. EC: 'removePolicy' is not working for built-in/system Erasure Code policies. Contributed by Ayush Saxena 2018-08-08 12:42:20 +05:30
Xiao Chen
6677717c68 HDFS-13728. Disk Balancer should not fail if volume usage is greater than capacity. Contributed by Stephen O'Donnell. 2018-08-07 22:05:17 -07:00
Vinayakumar B
2b0f977241 HDFS-13786. EC: Display erasure coding policy for sub-directories is not working. Contributed by Ayush Saxena. 2018-08-08 07:53:17 +05:30
Xiao Chen
0f8cb127cd HDFS-13799. TestEditLogTailer#testTriggersLogRollsForAllStandbyNN fails due to missing synchronization 
between rollEditsRpcExecutor and tailerThread shutdown. Contributed by Hrishikesh Gadre.
 2018-08-07 16:13:41 -07:00
Virajith Jalaparti
b1a59b1644 HDFS-13796. Allow verbosity of InMemoryLevelDBAliasMapServer to be configurable. 2018-08-07 10:15:28 -07:00
Takanobu Asanuma
2b18bb4f37 HDFS-11610. sun.net.spi.nameservice.NameService has moved to a new location. Contributed by Akira Ajisaka. 2018-08-03 23:38:05 +09:00
Aaron Fabbri
40f9b0c5c1
HDFS-13322 fuse dfs - uid persists when switching between ticket caches. Contributed by Istvan Fajth. 2018-07-31 18:44:49 -07:00
Brahma Reddy Battula
3108d27edd HDFS-12716. 'dfs.datanode.failed.volumes.tolerated' to support minimum number of volumes to be available. Contributed by Ranith Sardar and usharani 2018-07-30 15:50:04 +05:30
Arpit Agarwal
1c40bc2836 HDFS-13765. Fix javadoc for FSDirMkdirOp#createParentDirectories. Contributed by Lokesh Jain. 2018-07-27 10:14:01 -07:00
Anu Engineer
64e739e344 HDFS-13727. Log full stack trace if DiskBalancer exits with an unhandled exception. 
Contributed by Gabor Bota.
 2018-07-27 06:11:56 -07:00
Xiao Chen
be150a17b1 HDFS-13622. mkdir should print the parent directory in the error message when parent directories do not exist. Contributed by Shweta. 2018-07-26 10:24:32 -07:00
Wei-Chiu Chuang
e95c5e9f62 HDFS-11060. make DEFAULT_MAX_CORRUPT_FILEBLOCKS_RETURNED configurable. Contributed by Lantao Jin. 2018-07-25 11:04:18 -07:00
Xiao Chen
26864471c2 HDFS-13761. Add toString Method to AclFeature Class. Contributed by Shweta. 2018-07-24 21:42:47 -07:00
Daniel Templeton
849c45db18 HDFS-13448. HDFS Block Placement - Ignore Locality for First Block Replica 
(Contributed by BELUGA BEHR via Daniel Templeton)

Change-Id: I965d1cfa642ad24296038b83e3d5c9983545267d
 2018-07-24 16:05:27 -07:00
Yiqun Lin
17a87977f2 HDFS-13583. RBF: Router admin clrQuota is not synchronized with nameservice. Contributed by Dibyendu Karmakar. 2018-07-24 11:15:47 +08:00
Yiqun Lin
7b25fb949b HDFS-13743. RBF: Router throws NullPointerException due to the invalid initialization of MountTableResolver. Contributed by Takanobu Asanuma. 2018-07-20 17:28:57 +08:00
Yiqun Lin
1af87df242 HDFS-13733. RBF: Add Web UI configurations and descriptions to RBF document. Contributed by Takanobu Asanuma. 2018-07-17 10:45:08 +08:00
Konstantin V Shvachko
0a1e922f3d Fix potential FSImage corruption. Contributed by Ekanth Sethuramalingam & Arpit Agarwal. 2018-07-16 18:24:18 -07:00
Wei-Chiu Chuang
88b2794244 HDFS-13524. Occasional "All datanodes are bad" error in TestLargeBlock#testLargeBlockSize. Contributed by Siyao Meng. 2018-07-16 10:51:23 -07:00
Inigo Goiri
359ea4e181 HDFS-13475. RBF: Admin cannot enforce Router enter SafeMode. Contributed by Chao Sun. 2018-07-16 09:46:21 -07:00
Akira Ajisaka
88625f5cd9
HADOOP-15531. Use commons-text instead of commons-lang in some classes to fix deprecation warnings. Contributed by Takanobu Asanuma. 2018-07-13 11:42:12 -04:00
Xiao Chen
87eeb26e72 HDFS-13663. Should throw exception when incorrect block size is set. Contributed by Shweta. 2018-07-12 20:24:11 -07:00
Xiao Chen
b37074be5a HDFS-12837. Intermittent failure in TestReencryptionWithKMS. 2018-07-11 21:03:19 -07:00
Akira Ajisaka
418cc7f3ae
HDFS-13729. Fix broken links to RBF documentation. Contributed by Gabor Bota. 2018-07-11 14:46:43 -04:00
Wei-Chiu Chuang
162228e8db HDFS-13723. Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh. Contributed by Siyao Meng. 2018-07-11 10:02:08 -07:00
Yiqun Lin
2ae13d41dc HDFS-13726. RBF: Fix RBF configuration links. Contributed by Takanobu Asanuma. 2018-07-11 22:11:59 +08:00
Allen Wittenauer
5d0f01e1fe
HDFS-13722. HDFS Native Client Fails Compilation on Ubuntu 18.04 (contributed by Jack Bearden) 2018-07-10 12:17:44 -07:00
Bharat Viswanadham
c275a9a6a0 Merge trunk into HDDS-48 2018-07-09 12:13:03 -07:00
Andrew Wang
eecb5baaaa HDFS-13719. Docs around dfs.image.transfer.timeout are misleading. Contributed by Kitti Nansi. 2018-07-09 15:17:21 +02:00
Yiqun Lin
43f7fe8aae HDFS-13710. RBF: setQuota and getQuotaUsage should check the dfs.federation.router.quota.enable. Contributed by yanghuafeng. 2018-07-09 15:06:07 +08:00
Xiao Chen
936e0df0d3 HDFS-13721. NPE in DataNode due to uninitialized DiskBalancer. 2018-07-06 21:00:32 -07:00
Wei-Chiu Chuang
0247cb6318 HDFS-13121. NPE when request file descriptors when SC read. Contributed by Zsolt Venczel. 2018-07-06 14:59:49 -07:00
Yiqun Lin
3b637155a4 HDFS-13528. RBF: If a directory exceeds quota limit then quota usage is not refreshed for other mount entries. Contributed by Dibyendu Karmakar. 2018-07-04 15:03:24 +08:00
Andrew Wang
344f324710 HDFS-13712. BlockReaderRemote.read() logging improvement. Contributed by Gergo Repas. 2018-07-03 11:07:45 +02:00
Bharat Viswanadham
59a3038bc3 HDFS-13715:diskbalancer does not work if one of the blockpools are empty on a Federated cluster. Contributed by Bharat Viswanadham 2018-07-02 21:43:18 -07:00
Inigo Goiri
7296b644f7 HDFS-13714. Fix TestNameNodePrunesMissingStorages test failures on Windows. Contributed by Lukas Majercak. 2018-07-02 16:36:25 -07:00
Inigo Goiri
1804a31515 HDFS-13536. [PROVIDED Storage] HA for InMemoryAliasMap. Contributed by Virajith Jalaparti. 2018-07-02 10:48:20 -07:00
Wei-Chiu Chuang
fef20a446f HDFS-13635. Incorrect message when block is not found. Contributed by Gabor Bota. 2018-07-02 09:41:15 -07:00
Andrew Wang
5d748bd056 HDFS-13702. Remove HTrace hooks from DFSClient to reduce CPU usage. Contributed by Todd Lipcon. 2018-07-02 12:11:06 +02:00
Andrew Wang
6ba9974108 HDFS-13703. Avoid allocation of CorruptedBlocks hashmap when no corrupted blocks are hit. Contributed by Todd Lipcon. 2018-07-02 12:02:19 +02:00
Inigo Goiri
73746c5da7 HDFS-13707. [PROVIDED Storage] Fix failing integration tests in ITestProvidedImplementation. Contributed by Virajith Jalaparti. 2018-06-29 09:56:13 -07:00
Bharat Viswanadham
c104525168 Merge remote-tracking branch 'apache-commit/trunk' into HDDS-48 2018-06-28 14:28:23 -07:00
Akira Ajisaka
2b2399d623
HADOOP-15495. Upgrade commons-lang version to 3.7 in hadoop-common-project and hadoop-tools. Contributed by Takanobu Asanuma. 2018-06-28 14:37:22 +09:00
Todd Lipcon
7a3c6e9c3c HADOOP-15550. Avoid static initialization of ObjectMappers 2018-06-25 15:36:45 -07:00
Yiqun Lin
30728aced4 HDFS-13692. StorageInfoDefragmenter floods log when compacting StorageInfo TreeSet. Contributed by Bharat Viswanadham. 2018-06-22 10:50:54 +08:00
Xiao Chen
32f867a6a9 HDFS-13682. Cannot create encryption zone after KMS auth token expires. 2018-06-20 15:58:26 -07:00
Akira Ajisaka
fba9d7cd74
HDFS-13621. Upgrade commons-lang version to 3.7 in hadoop-hdfs-project. Contributed by Takanobu Asanuma. 2018-06-18 10:17:07 -07:00
Chris Douglas
980031bb04 HADOOP-13186. Multipart Uploader API. Contributed by Ewan Higgs 2018-06-17 11:54:26 -07:00
Inigo Goiri
8762e9cf10 HDFS-13681. Fix TestStartup.testNNFailToStartOnReadOnlyNNDir test failure on Windows. Contributed by Xiao Liang. 2018-06-15 16:49:06 -07:00
Inigo Goiri
d31a3ce767 HDFS-13686. Add overall metrics for FSNamesystemLock. Contributed by Lukas Majercak. 2018-06-15 15:59:19 -07:00
Wei-Chiu Chuang
c966a3837a HDFS-13174. hdfs mover -p /path times out after 20 min. Contributed by Istvan Fajth. 2018-06-15 13:36:46 -07:00
Inigo Goiri
eebeb6033f HDFS-13676. TestEditLogRace fails on Windows. Contributed by Zuoming Zhang. 2018-06-15 13:07:54 -07:00
Inigo Goiri
43d994e4a6 HDFS-13673. TestNameNodeMetrics fails on Windows. Contributed by Zuoming Zhang. 2018-06-15 10:33:28 -07:00
Mukul Kumar Singh
3e37a9a70b HDFS-13679. Fix Typo in javadoc for ScanInfoPerBlockPool#addAll. Contributed by Shashikant Banerjee. 2018-06-15 14:58:20 +05:30
Inigo Goiri
5d7449d2b8 HDFS-13675. Speed up TestDFSAdminWithHA. Contributed by Lukas Majercak. 2018-06-14 13:43:14 -07:00
Inigo Goiri
8d4926f38b HDFS-13563. TestDFSAdminWithHA times out on Windows. Contributed by Lukas Majercak. 2018-06-14 09:58:50 -07:00
Arpit Agarwal
418cff4820 Merge remote-tracking branch 'apache-commit/trunk' into HDDS-48 2018-06-14 09:38:20 -07:00
Yiqun Lin
8e7548d33b HDFS-13641. Add metrics for edit log tailing. Contributed by Chao Sun. 2018-06-13 20:05:55 +08:00
Jitendra Pandey
f4c7c91123 HADOOP-15483. Upgrade jquery to version 3.3.1. Contributed by Lokesh Jain, Mukul Kumar Singh and Sunil Govindan. 2018-06-13 00:36:02 -07:00
Rohith Sharma K S
2df73dace0 YARN-8405. RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773. Contributed by Íñigo Goiri. 2018-06-12 17:13:09 +05:30
Yongjun Zhang
7c3dc39083 Update CHANGES, RELEASENOTES, and jdiff for 3.0.3 release. 
(cherry picked from commit 5a82f10e3254bc0745c2dda6bcef888d3ff0d1c3)
 2018-06-11 15:13:18 -07:00
Inigo Goiri
2e5cfe6df3 HDFS-13653. Make dfs.client.failover.random.order a per nameservice configuration. Contributed by Ekanth Sethuramalingam. 2018-06-11 11:20:29 -07:00
Bharat Viswanadham
ccfb816d39 HDFS-13667:Typo: Marking all datandoes as stale. Contributed by Nanda Kumar 2018-06-09 16:43:03 -07:00
Arpit Agarwal
000a678396 HDFS-12670. can't renew HDFS tokens with only the hdfs client jar. Contributed by Arpit Agarwal. 2018-06-08 17:57:12 -07:00
Chao Sun
fba1c42adc HDFS-13664. Refactor ConfiguredFailoverProxyProvider to make inheritance easier. Contributed by Chao Sun. 2018-06-08 16:36:42 -07:00
Xiao Chen
cf4108313d HDFS-13642. Creating a file with block size smaller than EC policy's cell size should fail. 2018-06-08 15:14:11 -07:00
James Clampffer
7969cc4667 HDFS-13615. libhdfs++ SaslProtocol hanging while accessing invalid lock. Contributed by Mitchell Tracy 2018-06-07 16:02:57 -04:00
James Clampffer
b79ae5d93d HDFS-13534. libhdfs++: Fix GCC7 build. Contributed by James Clampffer. 2018-06-07 14:16:17 -04:00
Wei-Chiu Chuang
e39b113db0 HDFS-13659. Add more test coverage for contentSummary for snapshottable path. Contributed by Wei-Chiu Chuang. 2018-06-07 08:30:06 -07:00
Brahma Reddy Battula
9654dd1f47 HDFS-12950. [oiv] ls will fail in secure cluster. Contributed by Wei-Chiu Chuang. 2018-06-07 10:16:02 +05:30
Hanisha Koneru
772c95395b HDDS-140. Add DU usage to VolumeInfo. 2018-06-06 16:21:21 -07:00
Xiao Chen
774c1f199e HDFS-13511. Provide specialized exception when block length cannot be obtained. Contributed by Gabor Bota. 2018-06-05 21:25:08 -07:00
Xiao Chen
ba4011d64f HADOOP-15217. FsUrlConnection does not handle paths with spaces. Contributed by Joseph Fourny and Zsolt Venczel. 2018-06-05 21:18:31 -07:00
Akira Ajisaka
0e3c31579d
HDFS-13545. "guarded" is misspelled as "gaurded" in FSPermissionChecker.java. Contributed by Jianchao Jia. 2018-06-05 17:53:24 +09:00
Xiao Chen
6d5e87aec2 HADOOP-15507. Add MapReduce counters about EC bytes read. 2018-06-04 21:13:47 -07:00
Inigo Goiri
8d31ddcfeb HDFS-13650. Randomize baseDir for MiniDFSCluster in TestDFSStripedInputStream and TestDFSStripedInputStreamWithRandomECPolicy. Contributed by Anbang Hu. 2018-06-04 18:28:11 -07:00
Inigo Goiri
16316b6011 HDFS-13649. Randomize baseDir for MiniDFSCluster in TestReconstructStripedFile and TestReconstructStripedFileWithRandomECPolicy. Contributed by Anbang Hu. 2018-06-04 18:21:38 -07:00
Inigo Goiri
a28848104f HDFS-13652. Randomize baseDir for MiniDFSCluster in TestBlockScanner. Contributed by Anbang Hu. 2018-06-04 18:12:43 -07:00
Rushabh Shah
e2289c8d14 HDFS-13281 Namenode#createFile should be /.reserved/raw/ aware.. Contributed by Rushabh S Shah 2018-06-04 09:19:03 -05:00
Wei-Chiu Chuang
bccdfeee0a HDFS-13155. BlockPlacementPolicyDefault.chooseTargetInOrder Not Checking Return Value for NULL. Contributed by Zsolt Venczel. 2018-06-04 07:03:05 -07:00
Xiao Chen
9efb4b7db0 HDFS-13339. Volume reference can't be released and may lead to deadlock when DataXceiver does a check volume. Contributed by liaoyuxiangqin and Zsolt Venczel. 2018-06-03 22:33:34 -07:00
Inigo Goiri
8f83b9abf9 HDFS-13648. Fix TestGetConf#testGetJournalNodes on Windows due to a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 2018-06-02 20:19:59 -07:00
Inigo Goiri
e308ac88d3 HDFS-13651. TestReencryptionHandler fails on Windows. Contributed by Anbang Hu. 2018-06-02 20:12:35 -07:00
Inigo Goiri
e11d674049 HDFS-13637. RBF: Router fails when threadIndex (in ConnectionPool) wraps around Integer.MIN_VALUE. Contributed by CR Hota. 2018-06-01 16:59:04 -07:00
Chao Sun
ff013d2c95 HDFS-13602. Add checkOperation(WRITE) checks in FSNamesystem. Contributed by Chao Sun. 2018-05-31 17:37:44 -07:00
Wei-Chiu Chuang
3f4a29813b HDFS-13646. DFSAdmin doesn't display specialized help for triggerBlockReport. Contributed by Takanobu Asanuma. 2018-05-31 15:26:08 -07:00
Wei-Chiu Chuang
c95b9b5c07 HDFS-13440. Log HDFS file name when client fails to connect. Contributed by Gabor Bota. 2018-05-31 15:20:59 -07:00
Konstantin V Shvachko
ebe5853a45 HDFS-12978. Fine-grained locking while consuming journal stream. Contributed by Konstantin Shvachko. 2018-05-31 14:56:32 -07:00
Inigo Goiri
1f324e9661 HDFS-13631. TestDFSAdmin#testCheckNumOfBlocksInReportCommand should use a separate MiniDFSCluster path. Contributed by Anbang Hu. 2018-05-31 10:12:38 -07:00
Sean Mackrory
1361030e59 HDFS-13582. Improve backward compatibility for HDFS-13176 (WebHdfs file path gets truncated when having semicolon (;) inside). Contributed by Zsolt Venczel. 2018-05-31 07:59:21 -06:00
Inigo Goiri
47c31ff16b HDFS-13629. Some tests in TestDiskBalancerCommand fail on Windows due to MiniDFSCluster path conflict and improper path usage. Contributed by Anbang Hu. 2018-05-30 10:22:04 -07:00
Inigo Goiri
8197b9b560 HDFS-13632. Randomize baseDir for MiniJournalCluster in MiniQJMHACluster for TestDFSAdminWithHA. Contributed by Anbang Hu. 2018-05-30 10:13:52 -07:00
Yiqun Lin
b24098bc8f HDFS-13626. Fix incorrect username when deny the setOwner operation. Contributed by Zsolt Venczel. 2018-05-30 16:52:21 +08:00
Inigo Goiri
9dbf4f0166 HDFS-13591. TestDFSShell#testSetrepLow fails on Windows. Contributed by Anbang Hu. 2018-05-28 16:34:02 -07:00
Inigo Goiri
91d7c74e6a HDFS-13627. TestErasureCodingExerciseAPIs fails on Windows. Contributed by Anbang Hu. 2018-05-28 10:26:47 -07:00
Takanobu Asanuma
04757e5864
HDFS-13628. Update Archival Storage doc for Provided Storage 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-05-28 19:06:34 +09:00
Inigo Goiri
8605a38514 HDFS-13620. Randomize the test directory path for TestHDFSFileSystemContract. Contributed by Anbang Hu. 2018-05-25 19:43:33 -07:00
Inigo Goiri
13d2528907 HDFS-13619. TestAuditLoggerWithCommands fails on Windows. Contributed by Anbang Hu. 2018-05-25 13:32:34 -07:00
Inigo Goiri
1e0d4b1c28 HDFS-13618. Fix TestDataNodeFaultInjector test failures on Windows. Contributed by Xiao Liang. 2018-05-25 09:14:28 -07:00
Andrew Wang
c9b63deb53 HDFS-13611. Unsafe use of Text as a ConcurrentHashMap key in PBHelperClient. 2018-05-24 09:56:23 -07:00
Yiqun Lin
7a87add4ea HDFS-13598. Reduce unnecessary byte-to-string transform operation in INodesInPath#toString. Contributed by Gabor Bota. 2018-05-24 10:57:35 +08:00
Inigo Goiri
cddbbe5f69 HDFS-13493. Reduce the HttpServer2 thread count on DataNodes. Contributed by Erik Krogen. 2018-05-23 12:12:08 -07:00
Inigo Goiri
c13dea87d9 HDFS-13587. TestQuorumJournalManager fails on Windows. Contributed by Anbang Hu. 2018-05-23 11:36:03 -07:00
Bharat Viswanadham
bc6d9d4c79 HDFS-13589: Add dfsAdmin command to query if upgrade is finalized. Contributed by Hanisha Koneru 2018-05-23 10:15:40 -07:00
Inigo Goiri
c0c9b7a8ef HDFS-13588. Fix TestFsDatasetImpl test failures on Windows. Contributed by Xiao Liang. 2018-05-23 09:46:35 -07:00
Sammi Chen
34e8b9f9a8 HDFS-13540. DFSStripedInputStream should only allocate new buffers when reading. Contributed by Xiao Chen. 2018-05-23 19:10:09 +08:00
Andrew Wang
1d2640b613 HDFS-13601. Optimize ByteString conversions in PBHelper. 2018-05-22 23:55:20 -07:00
Inigo Goiri
9775ecb235 HDFS-13593. TestBlockReaderLocalLegacy#testBlockReaderLocalLegacyWithAppend fails on Windows. Contributed by Anbang Hu. 2018-05-18 09:46:02 -07:00
Inigo Goiri
57b893de3d HDFS-13592. TestNameNodePrunesMissingStorages#testNameNodePrunesUnreportedStorages does not shut down cluster properly. Contributed by Anbang Hu. 2018-05-18 09:36:58 -07:00
Yiqun Lin
f749517cc7 HDFS-13573. Javadoc for BlockPlacementPolicyDefault is inaccurate. Contributed by Zsolt Venczel. 2018-05-18 10:43:53 +08:00
Inigo Goiri
53b807a6a8 HDFS-13560. Insufficient system resources exist to complete the requested service for some tests on Windows. Contributed by Anbang Hu. 2018-05-17 17:03:23 -07:00
Inigo Goiri
a97a2042f2 HDFS-13556. TestNestedEncryptionZones does not shut down cluster. Contributed by Anbang Hu. 2018-05-17 16:53:23 -07:00
Inigo Goiri
65476458fa HDFS-13554. TestDatanodeRegistration#testForcedRegistration does not shut down cluster. Contributed by Anbang Hu. 2018-05-17 14:48:04 -07:00
Inigo Goiri
26f1e22fc9 HDFS-13558. TestDatanodeHttpXFrame does not shut down cluster. Contributed by Anbang Hu. 2018-05-17 13:35:09 -07:00
Inigo Goiri
c0ec061e28 HDFS-13570. TestQuotaByStorageType,TestQuota,TestDFSOutputStream fail on Windows. Contributed by Anbang Hu. 2018-05-17 09:21:57 -07:00
Kihwal Lee
0ce6290de6 HDFS-13581. DN UI logs link is broken when https is enabled. Contributed by Shashikant Banerjee. 2018-05-17 09:23:24 -05:00
Inigo Goiri
454de3b543 HDFS-13559. TestBlockScanner does not close TestContext properly. Contributed by Anbang Hu. 2018-05-16 19:23:02 -07:00
Arpit Agarwal
0fc988e6a3 HDFS-13512. WebHdfs getFileStatus doesn't return ecPolicy. Contributed by Ajay Kumar. 2018-05-16 11:28:39 -07:00
Inigo Goiri
55d5549087 HDFS-13550.TestDebugAdmin#testComputeMetaCommand fails on Windows. Contributed by Anbang Hu. 2018-05-16 10:08:49 -07:00
Inigo Goiri
6b67161ca3 HDFS-13557. TestDFSAdmin#testListOpenFiles fails on Windows. Contributed by Anbang Hu. 2018-05-16 09:31:46 -07:00
Inigo Goiri
63480976a0 HDFS-13567. TestNameNodeMetrics#testGenerateEDEKTime,TestNameNodeMetrics#testResourceCheck should use a different cluster basedir. Contributed by Anbang Hu. 2018-05-15 15:34:54 -07:00
Inigo Goiri
2d6195c0af HDFS-13548. TestResolveHdfsSymlink#testFcResolveAfs fails on Windows. Contributed by Anbang Hu. 2018-05-15 11:20:32 -07:00
Inigo Goiri
eaca7980ad HDFS-11700. TestHDFSServerPorts#testBackupNodePorts doesn't pass on Windows. Contributed by Anbang Hu. 2018-05-15 10:27:36 -07:00
Inigo Goiri
92ebd466c7 HDFS-13551. TestMiniDFSCluster#testClusterSetStorageCapacity does not shut down cluster. Contributed by Anbang Hu. 2018-05-15 10:21:42 -07:00
Hanisha Koneru
6beb25ab7e HDFS-13544. Improve logging for JournalNode in federated cluster. 2018-05-14 10:12:08 -07:00
Xiao Chen
960940e0e0 HDFS-13539. DFSStripedInputStream NPE when reportCheckSumFailure. 2018-05-14 09:28:39 -07:00
Sammi Chen
f3f544b004 Add 2.9.1 release notes and changes documents 2018-05-14 15:14:02 +08:00
sammichen
66c9905b57 Revert "Add 2.9.1 release notes and changes documents" 
This reverts commit e4dc346d65.
 2018-05-14 14:38:40 +08:00
littlezhou
e4dc346d65 Add 2.9.1 release notes and changes documents 2018-05-14 14:24:01 +08:00
Anu Engineer
4b4f24ad5f HDDS-40. Separating packaging of Ozone/HDDS from the main Hadoop. 
Contributed by Elek, Marton.
 2018-05-11 14:24:36 -07:00
Inigo Goiri
d50c4d71dc HDFS-13542. TestBlockManager#testNeededReplicationWhileAppending fails due to improper cluster shutdown in TestBlockManager#testBlockManagerMachinesArray on Windows. Contributed by Anbang Hu. 2018-05-11 09:47:57 -07:00
Yiqun Lin
a922b9c82c HDFS-13346. RBF: Fix synchronization of router quota and nameservice quota. 2018-05-11 14:51:30 +08:00
Inigo Goiri
914b98a713 HDFS-13537. TestHdfsHelper does not generate jceks path properly for relative path in Windows. Contributed by Xiao Liang. 2018-05-09 16:52:09 -07:00
Xiao Chen
696a4be0da HDFS-12981. renameSnapshot a Non-Existent snapshot to itself should throw error. Contributed by Kitti Nanasi. 2018-05-07 15:36:29 -07:00
Xiao Chen
118bd75805 Revert "HDFS-13430. Fix TestEncryptionZonesWithKMS failure due to HADOOP-14445." 
This reverts commit 6503593711.
 2018-05-07 13:32:27 -07:00
Aaron Fabbri
a732acd873
HDFS-5926 Documentation should clarify dfs.datanode.du.reserved impact from reserved disk capacity (Gabor Bota) 2018-05-04 16:38:30 -07:00
Inigo Goiri
a3b416f69d HDFS-13525. RBF: Add unit test TestStateStoreDisabledNameservice. Contributed by Yiqun Lin. 2018-05-03 11:24:57 -07:00
James Clampffer
19ae588fde HDFS-11807. libhdfs++: Get minidfscluster tests running under valgrind. Contributed by Anatoli Shein. 2018-05-02 11:49:12 -04:00
Yiqun Lin
37269261d1 HDFS-13488. RBF: Reject requests when a Router is overloaded. Contributed by Inigo Goiri. 2018-05-02 14:49:39 +08:00
Inigo Goiri
9e2cfb2d3f HDFS-13503. Fix TestFsck test failures on Windows. Contributed by Xiao Liang. 2018-05-01 08:12:46 -07:00
Inigo Goiri
fc074a359c HDFS-13283. Percentage based Reserved Space Calculation for DataNode. Contributed by Lukas Majercak. 2018-04-30 13:28:33 -07:00
Inigo Goiri
eb7fe1d588 HDFS-13509. Bug fix for breakHardlinks() of ReplicaInfo/LocalReplica, and fix TestFileAppend failures on Windows. Contributed by Xiao Liang. 2018-04-28 09:05:30 -07:00
Inigo Goiri
484440602c HDFS-13508. RBF: Normalize paths (automatically) when adding, updating, removing or listing mount table entries. Contributed by Ekanth S. 2018-04-27 16:28:17 -07:00
Inigo Goiri
f469628bba HDFS-13434. RBF: Fix dead links in RBF document. Contributed by Chetna Chaudhari. 2018-04-27 15:13:47 -07:00
Ewan Higgs
84ecfe3ceb
HADOOP-14188. Remove the usage of org.mockito.internal.util.reflection.Whitebox 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-27 19:21:37 +09:00
Yiqun Lin
0a293de00d HDFS-13499. RBF: Show disabled name services in the UI. Contributed by Inigo Goiri. 2018-04-27 10:38:15 +08:00
Inigo Goiri
09c9cf1d43 HDFS-13326. RBF: Improve the interfaces to modify and view mount tables. Contributed by Gang Li. 2018-04-26 13:54:40 -07:00
Lei Xu
a8e428b2dc HDFS-13468. Add erasure coding metrics into ReadStatistics. (Contributed by Lei (Eddy) Xu) 2018-04-26 13:54:40 -07:00
Takanobu Asanuma
6f655aed3b HADOOP-15404. Remove multibyte characters in DataNodeUsageReportUtil 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-26 13:54:40 -07:00
Yiqun Lin
48269c370c HDFS-13484. RBF: Disable Nameservices from the federation. Contributed by Inigo Goiri. 2018-04-26 13:54:40 -07:00
Owen O'Malley
2adda92de1
HDFS-7240 git commit fixup 
Signed-off-by: Owen O'Malley <omalley@apache.org>
 2018-04-26 13:52:17 -07:00
Anu Engineer
3d18ca4926 HDFS-13459. Ozone: Clean-up of ozone related change from MiniDFSCluste. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
979bbb4019 HDFS-13423. Ozone: Clean-up of ozone related change from hadoop-hdfs-project. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
25f2398bbd HDFS-13395. Ozone: Plugins support in HDSL Datanode Service. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
0ec88ea42b HDFS-13301. Ozone: Remove containerPort, ratisPort and ozoneRestPort from DatanodeID and DatanodeIDProto. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Anu Engineer
4e61bc431e HDFS-13258. Ozone: restructure Hdsl/Ozone code to separated maven subprojects. 
Contributed by Elek Marton, Mukul Kumar Singh, Xiaoyu Yao, Ajay Kumar, Anu Engineer, Lokesh Jain, Nanda Kumar.
 2018-04-26 05:36:04 -07:00
Anu Engineer
5e7164c614 HDFS-13137. Ozone: Ozonefs read fails because ChunkGroupInputStream#read does not iterate through all the blocks in the key. 
Contributed by Mukul Kumar Singh.
 2018-04-26 05:36:04 -07:00
Anu Engineer
61651dcf5c HDFS-13108. Ozone: OzoneFileSystem: Simplified url schema for Ozone File System. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
9a914126a7 HDFS-13008. Ozone: Add DN container open/close state to container report. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Nanda kumar
f4466402ae HDFS-12965. Ozone: Documentation : Add ksm -createObjectStore command documentation. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Nanda kumar
1094af072c HDFS-11699. Ozone:SCM: Add support for close containers in SCM. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Nanda kumar
b4a3cf1476 HDFS-13070. Ozone: SCM: Support for container replica reconciliation - 1. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
4c10a849e8 HDFS-13078. Ozone: Update Ratis on Ozone to 0.1.1-alpha-8fd74ed-SNAPSHOT. 
To fix large chunk reads (>4M) from Datanodes. Contributed by Mukul Kumar Singh.
 2018-04-26 05:36:04 -07:00
Anu Engineer
136b419594 HDFS-13149. Ozone: Rename Corona to Freon. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
9272e1021d HDFS-12636. Ozone: OzoneFileSystem: Implement seek functionality for rpc client. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
377b31ffa1 HDFS-13022. Block Storage: Kubernetes dynamic persistent volume provisioner. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
ee5495456e HDFS-13116. Ozone: Refactor Pipeline to have transport and container specific information. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
3cf07b43bc HDFS-13127. Fix TestContainerStateManager and TestOzoneConfigurationFields. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
443425a5d9 HDFS-12522. Ozone: Remove the Priority Queues used in the Container State Manager. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Nanda kumar
32245c78e2 HDFS-13080. Ozone: Make finalhash in ContainerInfo of StorageContainerDatanodeProtocol.proto optional. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
150c4e1d0b HDFS-12879. Ozone : add scm init command to document. Contributed by Rahul Pathak. 2018-04-26 05:36:04 -07:00
Nanda kumar
7b3179f551 HDFS-13072. Ozone: DatanodeStateMachine: Handling Uncaught Exception in command handler thread. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
6b03fca37d HDFS-12868. Ozone: Service Discovery API. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
fb09d75379 HDFS-13018. Block Storage: make the iscsi target addres configurable for discovery. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
4bb9ad8e8f HDFS-13017. Block Storage: implement simple iscsi discovery in jscsi server. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
ba4d5a52a8 HDFS-12986. Ozone: Update ozone to latest ratis snapshot build (0.1.1-alpha-0f7169d-SNAPSHOT). Contributed by Lokesh Jain 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
94c0346f35 HDFS-13024. Ozone: ContainerStateMachine should synchronize operations between createContainer and writeChunk. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Nanda kumar
4a051ba494 HDFS-12940. Ozone: KSM: TestKeySpaceManager#testExpiredOpenKey fails occasionally. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
1fb1ce107f HDFS-13012. TestOzoneConfigurationFields fails due to missing configs in ozone-default.xml. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
916f31f7d1 HDFS-13026. Ozone: TestContainerPersistence is failing becaue of container data mismatch. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
28b87af51b HDFS-13013. Fix closeContainer API with the right container state change. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
c4b88454a7 HDFS-12843. Ozone: Client: TestOzoneRpcClient#testPutKeyRatisThreeNodes is failing. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
98d62e55c4 Revert "HDFS-12794. Ozone: Parallelize ChunkOutputSream Writes to container. Contributed by Shashikant Banerjee." 
This reverts commit 6ce5ec676164b84a9e2f8dc65b5f2199a141506d.
 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
98eeabbdb3 HDFS-12794. Ozone: Parallelize ChunkOutputSream Writes to container. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
0c48cb2cf2 HFDS-13007. Ozone: Fix TestKeySpaceManager#testKSMInitializationFailure. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
98973cd0c4 HDFS-12966. Ozone: owner name should be set properly when the container allocation happens. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
5e31b920f0 HDFS-12871. Ozone: Service Discovery: Adding REST server details in ServiceList. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
3965f1ec99 HDFS-12853. Ozone: Optimize chunk writes for Ratis by avoiding double writes. Contributed by Mukul Kumar Singh 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
5cc5149bd8 HDFS-12961. Ozone: Use relative resource urls in KSM/SCM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
5766792cad HDFS-12870. Ozone: Service Discovery: REST endpoint in KSM for getServiceList. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
5f10c2d8ce HDFS-12980. Ozone: SCM: Restructuring container state transition and event. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
87b7774488 HDFS-12958. Ozone: remove setAllocatedBytes method in ContainerInfo. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Nanda kumar
ce19f09bd0 HDFS-12908. Ozone: write chunk call fails because of Metrics registry exception. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Nanda kumar
4b9f66a19e HDFS-12869. Ozone: Service Discovery: RPC endpoint in KSM for getServiceList. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
79432ee6cf HDFS-12863. Ozone: Avoid configuration.get() call for each container report. Contributed by Akira Ajisaka. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
0380dbb928 HDFS-12941. Ozone: ConfServlet does not trim values during the description parsing. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
4dae68eebf HDFS-12741. Ozone: KSM: ADD support for KSM --createObjectStore command. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Chen Liang
caeeb78ca4 HDFS-12799. Ozone: SCM: Close containers: extend SCMCommandResponseProto with SCMCloseContainerCmdResponseProto. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
966853894f HDFS-12795. Ozone: SCM: Support for Container LifeCycleState PENDING_CLOSE and LifeCycleEvent FULL_CONTAINER. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
26e270b908 HDFS-12751. Ozone: SCM: update container allocated size to container db for all the open containers in ContainerStateManager#close. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fd09c2ce5b HDFS-12890 . Ozone: XceiverClient should have upper bound on async requests. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Anu Engineer
2f3068bc72 HDFS-12698. Ozone: Use time units in the Ozone configuration values. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
8ff98e2af3 HDFS-12000. Ozone: Container : Add key versioning support-1. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
c0c87dea9b HDFS-12626. Ozone : delete open key entries that will no longer be closed. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
6c630d0b09 HDFS-12899. Ozone: SCM: BlockManagerImpl close is called twice during StorageContainerManager#stop. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
74362cb503 HDFS-12901. Ozone: SCM: Expose StorageContainerManager#getScmId through container protocol. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
11473b8204 HDFS-12898. Ozone: TestSCMCli#testHelp and TestSCMCli#testNonExistCommand fail consistently. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Anu Engineer
207523915d HDFS-12588. Use GenericOptionsParser for scm and ksm daemon. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
7bba3943f0 HDFS-12876. Ozone: moving NodeType from OzoneConsts to Ozone.proto. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
85ec48a646 HDFS-12867. Ozone: TestOzoneConfigurationFields fails consistently. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
538e17824d HDFS-12850. Ozone: TestContainerPersistence#testListContainer fails frequently due to timed out. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
e32e187448 HDFS-12864. Ozone: Turn off full synced write in RocksDB MetadataStore. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
8023ba50b5 HDFS-12807. Ozone: Expose RockDB stats via JMX for Ozone metadata stores. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
86802a461c HDFS-12851. Ozone: Upgrade to latest ratis build. Contributed by Mukul Kumar Singh 2018-04-26 05:36:04 -07:00
Nanda kumar
ea86945754 HDFS-12842. Ozone: SCM: During startup SCM should check if the cluster is INITIALIZED. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
73ca8608da HDFS-12786. Ozone: add port/service names to the ksm/scm web ui. Contributed by Elek Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
f9c11d952c HDFS-12740. SCM should support a RPC to share the cluster Id with KSM and DataNodes. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
c8d8270f72 HDFS-12787. Ozone: SCM: Aggregate the metrics from all the container reports. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
cbe4f314b3 HDFS-12731. Ozone: Fix inconsistent default value in ozone settings. Contributed by Bharat Viswanadham. 2018-04-26 05:36:04 -07:00
Anu Engineer
4deb845d7c HDFS-12759. Ozone: web: integrate configuration reader page to the SCM/KSM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
8e8ff85b16 HDFS-12830. Ozone: TestOzoneRpcClient#testPutKeyRatisThreeNodes fails. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
214b52dfce HDFS-12500. Ozone: add logger for oz shell commands and move error stack traces to DEBUG level. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
419e86d580 HDFS-12805. Ozone: Redundant characters printed in exception log. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
93f1b04949 HDFS-12798. Ozone: scm web: fix the node status table. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
4460ac9c57 HDFS-12756. Ozone: Add datanodeID to heartbeat responses and container protocol. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Nanda kumar
446e84357d HDFS-12796. SCM should not start if Cluster Version file does not exist. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Anu Engineer
740a06cdd7 HDFS-12719. Ozone: Fix checkstyle, javac, whitespace issues in HDFS-7240 branch. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Yiqun Lin
761926db2c HDFS-12793. Ozone : TestSCMCli is failing consistently. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Chen Liang
e395244e95 HDFS-12758. Ozone: Correcting assertEquals argument order in test cases. Contributed by Bharat Viswanadham. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
cec96b296f HDFS-12549. Ozone: OzoneClient: Support for REST protocol. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
9734f505ea HDFS-12739. Add Support for SCM --init command. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
709d56fdc9 HDFS-12474. Ozone: SCM: Handling container report with key count and container usage. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
2ec24a7542 HDFS-12774. Ozone: OzoneClient: Moving OzoneException from hadoop-hdfs to hadoop-hdfs-client. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Yiqun Lin
132f30c8ea HDFS-12443. Ozone: Improve SCM block deletion throttling algorithm. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
7ebe79e879 HDFS-12720. Ozone: Ratis options are not passed from KSM Client protobuf helper correctly. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
0559265822 HDFS-12750. Ozone: Fix TestStorageContainerManager#testBlockDeletionTransactions. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
0760418bec HDFS-12697. Ozone services must stay disabled in secure setup for alpha. Contributed by Bharat Viswanadham. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
b687902590 HDFS-12718. Ozone: fix thread number calculation in CBlockManager. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
76d34bca62 HDFS-12521. Ozone: SCM should read all Container info into memory when booting up. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Anu Engineer
6291ca1e32 HDFS-12663. Ozone: OzoneClient: Remove protobuf classes exposed to clients through OzoneBucket. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
80357c0eee HDFS-12678. Ozone: Corona: Add statistical information to json output. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Anu Engineer
1b56a73a4f HDFS-12680. Ozone: SCM: Lease support for container creation. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
338c909ac6 HDFS-12558. Ozone: Clarify the meaning of rpc.metrics.percentiles.intervals on KSM/SCM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
333dae888c HDFS-12690. Ozone: generate swagger descriptor for the Ozone REST Api. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
0582d8b1b1 Revert "HDFS-12558. Ozone: Clarify the meaning of rpc.metrics.percentiles.intervals on KSM/SCM web ui. Contributed by Elek, Marton." 
This reverts commit 18c0cdcec9ff5676b10ba711039143d07dd4a510.
 2018-04-26 05:36:04 -07:00
Anu Engineer
402a1c1ba3 HDFS-12558. Ozone: Clarify the meaning of rpc.metrics.percentiles.intervals on KSM/SCM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
731a69faa1 HDFS-12693. Ozone: Enable XFrame options for KSM/SCM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Yiqun Lin
6783dad766 HDFS-11468. Ozone: SCM: Add Node Metrics for SCM. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
393a02d8e1 HDFS-12692. Ozone: fix javadoc/unit test for calling MetadataStore.getRangeKVs with non existing key. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Weiwei Yang
6d9f914b55 HDFS-12684. Ozone: SCMMXBean NodeCount is overlapping with NodeManagerMXBean. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Nanda kumar
cc86719451 HDFS-12675. Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
0f0417a12c HDFS-12673. Ozone: Log elapsed time for block deletion tasks. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
06c6582694 HDFS-12585. Ozone: KSM UI: Add description for configs in UI. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
763484e596 HDFS-12661. Ozone: Support optional documentation link in KSM/SCM webui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
a467017068 HDFS-12411. Ozone: Add container usage information to DN container report. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
fb545e4291 HDFS-12415. Ozone: TestXceiverClientManager and TestAllocateContainer occasionally fails. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
895a4cc60a HDFS-12632. Ozone: OzoneFileSystem: Add contract tests to OzoneFileSystem. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Nanda kumar
9bb516118d HDFS-12490. Ozone: OzoneClient: Add creation/modification time information in OzoneVolume/OzoneBucket/OzoneKey. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Nandakumar
8881309002 HDFS-12519. Ozone: Lease Manager framework. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
ebc4d4ebc3 HDFS-12475. Ozone: add documentation about Datanode http address. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
c85d3b2d25 HDFS-12593. Ozone: update Ratis to the latest snapshot. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Nandakumar
5ed663b32f HDFS-12631. Ozone: ContainerStorageLocation#scmUsage should count only SCM usage. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Yiqun Lin
d19b4c8763 HDFS-12583. Ozone: Fix swallow exceptions which makes hard to debug failures. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
9445a9267f HDFS-12572. Ozone: OzoneFileSystem: delete/list status/rename/mkdir APIs. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
ae2b77a0e5 HDFS-12546. Ozone: DB listing operation performance improvement. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Chen Liang
b05ad0b315 HDFS-12611. Ozone: SCM Cli: Use the hostname of SCM server instead of the bind address. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
yuanbo
e76e10fd7a HDFS-12609. Ozone: SCM: Refactoring of chill mode logic in NodeManager. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
e3b51d9074 HDFS-12598. Ozone: Fix 3 node ratis replication in Ozone. Contributed by Mukul Kumar Singh 2018-04-26 05:36:04 -07:00
Nandakumar
3504af9411 HDFS-12537. Ozone: Reduce key creation overhead in Corona. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Nandakumar
9630621be7 HDFS-12610. Ozone: OzoneClient: RpcClient list calls throw NPE when iterating over empty list. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Nandakumar
c3ef381011 HDFS-12543. Ozone : allow create key without specifying size. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
c97ce55d06 HDFS-12602. Add Ozone config UI to KSM and SCM index page. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
303ecae5d5 HDFS-12557. Ozone: Improve the formatting of the RPC stats on web UI. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
c17521b1bd HDFS-12387. Ozone: Support Ratis as a first class replication mechanism. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Anu Engineer
0b3270a8de HDFS-12513. Ozone: Create UI page to show Ozone configs by tag. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
c52019c291 HDFS-12539. Ozone: refactor some functions in KSMMetadataManagerImpl to be more readable and reusable. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer
ceec14bff5 HDFS-12467. Ozone: SCM: NodeManager should log when it comes out of chill mode. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
d7a94a21ae HDFS-12471. Ozone: Reduce some KSM/SCM deletion log messages from INFO to DEBUG. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
6a754caca0 HDFS-12568. Ozone: Cleanup the ozone-default.xml. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
05246e2b32 HDFS-12060. Ozone: OzoneClient: Add list calls in RpcClient. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
7846636a5d HDFS-12551. Ozone: Documentation: Add Ozone overview documentation. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Yiqun Lin
cb1fb3436e HDFS-12563. Ozone: Creation time wrongly showed in OzoneRest.md. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
aaec2689f4 HDFS-12511. Ozone: Add tags to config. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
7151a9b604 HDFS-12488. Ozone: OzoneRestClient timeout is not configurable. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
7363c8fc09 HDFS-12540. Ozone: node status text reported by SCM is a bit confusing. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
13fdb58490 HDFS-12554. Ozone: Fix TestDatanodeStateMachine#testDatanodeStateMachineWithInvalidConfiguration. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Chen Liang
9530153f33 HDFS-12268. Ozone: Add metrics for pending storage container requests. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
94090acf3a HDFS-12469. Ozone: Create docker-compose definition to easily test real clusters. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
a44f3a777d HDFS-12501. Ozone: Cleanup javac issues. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
6b10723ec3 HDFS-12525. Ozone: OzoneClient: Verify bucket/volume name in create calls. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
5e4a6b686c HDFS-12425. Ozone: OzoneFileSystem: read/write/create/open/getFileInfo APIs. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
a1a3ba6529 HDFS-12454. Ozone : the sample ozone-site.xml in OzoneGettingStarted does not work. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
2804435386 HDFS-12017. Ozone: Container: Move IPC port to 98xx range. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
b2447f8dd9 HDFS-11614. Ozone: Cleanup javadoc issues. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
9ff136bb02 HDFS-12210. Block Storage: volume creation times out while creating 3TB volume because of too many containers. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
dddded0016 Revert "Block Storage: volume creation times out while creating 3TB volume because of too many containers. Contributed by Mukul Kumar Singh." to fix commit message. 
This reverts commit 087c69ba2434813044089f733581607d172efbd1.
 2018-04-26 05:36:04 -07:00
Weiwei Yang
d303b7f40f HDFS-12524. Ozone: Record number of keys scanned and hinted for getRangeKVs call. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Yiqun Lin
712bd70ac7 HDFS-12205. Ozone: List Key on an empty ozone bucket fails with command failed error. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Chen Liang
8052374e79 Block Storage: volume creation times out while creating 3TB volume because of too many containers. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
fd1564b87e HDFS-12506. Ozone: ListBucket is too slow. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
bf6f0cd831 HDFS-11563. Ozone: enforce DependencyConvergence uniqueVersions. 2018-04-26 05:36:04 -07:00
Weiwei Yang
46b65ec4fb HDFS-12503. Ozone: some UX improvements to oz_debug. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
1c354e51ac HDFS-12477. Ozone: Some minor text improvement in SCM web UI. Contributed by Elek Marton. 2018-04-26 05:36:04 -07:00
Nandakumar
0313bdf886 HDFS-11897. Ozone: KSM: Changing log level for client calls in KSM. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Anu Engineer
27ec22e20d HDFS-12517. Ozone: mvn package is failing with out skipshade. Contributed by Bharat Viswanadham. 2018-04-26 05:36:04 -07:00
Weiwei Yang
3b290fe3bf HDFS-12466. Ozone: KSM: Make ozone.ksm.address as mandatory property for client. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
8ddf75da17 HDFS-12489. Ozone: OzoneRestClientException swallows exceptions which makes client hard to debug failures. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
c329d3b4b4 HDFS-12329. Ozone: Ratis: Readonly calls in XceiverClientRatis should use sendReadOnly. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
607d1fcd31 HDFS-12256. Ozone : handle inactive containers on DataNode. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
77d4b18700 HDFS-12385. Ozone: OzoneClient: Refactoring OzoneClient API. Contributed by Nadakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
b984e903f3 HDFS-12481. Ozone: Corona: Support for variable key length in offline mode. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00